IoT analytics guide: What to expect from Internet of Things data

The growth of the Internet of Things (IoT) is having a big impact on lots of areas within enterprise IT, and data analytics is one of them.

Companies are gathering huge volumes of information from all kinds of connected of objects, such as data about how consumers are using certain products, the performance of corporate assets, and the environmental conditions in which systems operate. By applying advanced analytics to these incoming streams of data, organizations can gain new insights that can help them make more informed decisions about which actions to take. And with companies placing IoT sensors on more and more objects, the volumes of incoming data will continue to grow.

"Sensor-based computing is a core trend in digital transformation," says Maureen Fleming, an analyst at research firm IDC. "Operational intelligence using condition-based monitoring assures organizations about the health of sensor-attached devices, machines and systems. Depending on the use case, applying machine learning [ML] to sensor data is aimed at predicting probability of outages, propensity to buy, health problems, etc."

Applying ML to sensor data in combination with data from enterprise applications can also fundamentally change how an organization works, by predicting problems with meeting service-level agreements on services for customers or logistics problems within a supply chain, Fleming says.

IoT "is driving the blending of the digital and physical worlds," says Brian Hopkins, vice president and principal analyst at Forrester Research. "Almost all businesses want real-time data from the physical world to take the next step in their quest for insights that deliver competitive advantage."

Forrester sees three primary scenarios for gaining insight through analytics. One is insight about the smart connected products themselves. Another is insight about how connected things work efficiently together, which can help companies improve processes that involve physical assets. And the third is insight about things and people that come from the IoT data of business partners such as suppliers.

IoT necessitates new infrastructure

For many enterprises, the existing data analytics infrastructure will not adequately handle the expected increases in volume generated by the IoT, however. They will need to alter their IT environments to make them more “IoT-ready.”

“IoT is creating an unprecedented amount of data in the enterprise in terms of both volume and velocity,” says Mark Hung, research vice president at research firm Gartner. “In order to extract value out of this data, the enterprise’s data analytics architecture needs to be revamped.”

For enterprises to act on IoT data in a timely manner, streaming or real-time analytics is often required, Hung says. The need to incorporate new analytics methods such as streaming analytics and new infrastructure such as edge gateways places new architectural requirements on the existing IT infrastructure, he says.

Analytics for IoT has some unique requirements compared with analytics for other kinds of data. This includes data format, data richness, time sensitivity, where the data is stored and how long it is stored.

“The key analytics need is to close the gap between data generation in the physical world and the need for action either in the physical or digital world,” Hopkins says. “This inevitably means pushing some analytics logic to the edge—out of the cloud or data center. The problem is that servers and devices have far less compute power.” Some have battery or power limitations, and far less storage, than the analytics require, Hopkins says. Therefore, the analytics needs to be distributed. “Some things happen on devices, others on edge servers and gateways, others in central processing environments,” he says.

Building out real-time data capture, data governance, and availability of services are among the biggest challenges IT will face in creating an IoT analytics environment, Hopkins says.

“Since not all data is held neatly in a database, each device that produces data has to be cataloged, the data it produces put under governance, etc.,” Hopkins says. “There are hosts of security and privacy issues that have traditionally fallen into IT’s lap. The problem is that a lot of IoT investment is happening outside of IT [within the operations area], but IT still feels pressure to secure the system and protect the data.”

IoT analytics will also place new pressures on network infrastructure. “As data volumes grow, networks must become a lot more flexible and achieve higher throughput, all while being secure,” Hopkins says. “A tall order.”

Depending on the application and industry, IoT’s requirements will create more demand for additional bandwidth and less tolerance of latency within the network infrastructure, Hung says.

Determining whether an organization should deploy outside services or in-house analytics is a complex topic that is multi-faceted, Hung says. “Some of the factors include the enterprise’s data privacy requirements and in-house analytics capabilities,” he says.

The availability of skills is a fundamental consideration, Fleming says. “Another is whether there are out-of-the-box libraries that speed up development, versus a need to build proprietary algorithms,” she says. “Also, IoT analytics is often focused on times series, which may require new capabilities.”

Among the industries embracing IoT analytics are energy exploration (for example oil and gas), which traditionally have been at the forefront of adopting IoT analytics, Hung says. “However, other key industries, such as manufacturing and transportation, are becoming increasingly active in evaluating IoT analytics as well,” he says.

Choosing an IoT analytics platform

A number of vendors are offering IoT analytics systems. For example, IBM offers the Watson IoT platform, a managed, cloud-hosted service that provides capabilities such as device registration, connectivity, rapid visualization and storage of IoT data. IBM Watson provides natural language processing, machine learning, and image and text analytics for IoT applications.

Customers use the platform to procure and store data embedded in devices in order to make decisions in near real-time using Watson analytics and artificial intelligence (AI), says Stephan Biller, vice president for offering management at IBM Watson IoT. Sensors send data over a Z-Wave radio network to gateways that are connected through a cable LAN to the Internet. Data is captured and stored on the IBM Cloud.

"IoT analytics rules can be set according to specific conditions that trigger specific actions,” Biller says. For example, a customer might create a rule to ensure that an alert is sent to a data dashboard, and that an email is concurrently sent to an administrator, when a device is dropped or when the temperature of the device spikes, he says.

IBM continues to see a steady increase in demand for the IoT platform, Biller says. “We see enterprise IoT adoption growing as projects move from proof-of-concept to production, he says. “Platform investments are critical, as clients recognize the fundamental need to connect sensors and devices and manage, store, and secure the data.”

Platforms such as Watson IoT are designed to help clients perform basic analytics, such as generating alerts and spotting anomalies from the data streams. “But most of the growth we are seeing is coming from clients who recognize that [the] real value is beyond basic ‘connect and collect,’” Biller says. “These clients are interested in the advanced analytics, machine learning, and other AI technologies that can be deployed to help them understand their data and drive benefits like improved operational efficiencies and asset uptime.”

IoT analytics catered to specific industries are critical for customers, Biller says. “Often this comes in the form of industry model templates we jointly build with IBM research and our clients,” he says. “While certain cross-industry techniques can be used for basic data [preparation] and initial insights, we find that each client's business conditions and often unique data sources require a higher degree of customization.”

Amazon offers AWS IoT Analytics, a managed service designed to make it easier to run and operationalize sophisticated analytics on massive volumes of IoT data without having to worry about the cost and complexity typically required to build an IoT analytics platform.

AWS IoT Analytics automates each of the steps needed to analyze data from IoT devices, says Marco Argenti, AWS vice president of technology. It filters and enriches IoT data before storing it in a time-series data store for analysis. Organizations can set up the service to collect only the data they need from their devices, apply mathematical transforms to process the data, and enrich the data with device-specific metadata such as device type and location before storing the processed data.

Then, they can analyze their data by running ad hoc or scheduled queries using the built-in SQL query engine, or perform more complex analytics and machine learning inference. AWS IoT Analytics includes pre-built models for common IoT use cases, Argenti says.

Organizations can also use their own custom analysis, packaged in a container, to execute on AWS IoT Analytics. The platform automates the execution of custom analyses created in Jupyter Notebook or the organization’s own tools.

In addition, AWS also has AWS Greengrass in its IoT portfolio. AWS Greengrass is software that lets companies run local compute, messaging, data caching, sync, and ML inference capabilities for connected devices, Argenti says.

With AWS Greengrass, connected devices can keep data in sync, and communicate with other devices securely, even when not connected to the Internet. Using AWS Lambda, Greengrass ensures that IoT devices can respond quickly to local events, use Lambda functions running on Greengrass Core to interact with local resources, and operate with intermittent connections.

“Due to the unique challenges of IoT data, there has been pent-up demand [for analytics] as connected device manufacturers and enterprises had to build custom software and hardware applications dedicated to managing specific devices and their data,” Argenti says. “These applications were expensive to build, did not scale well to large fleets of different device types, and were typically inflexible,” he says.

Finding success with IoT analytics

Companies that have deployed IoT analytics platforms are seeing benefits.

Georgia Pacific, one of the world’s leading makers of tissue, pulp, paper, packaging, building products, and related chemicals, has deployed AWS IoT Analytics.

The company’s dispensers allow it to deliver products to customers, and Georgia Pacific is focused on making these dispensers “smart” by adding sensors and connectivity, says Erik Cordsen, IoT program architect and product leader.

That allows the company to improve customer experience by providing real-time information about product levels and other statistics, Cordsen says. With thousands of endpoints continuously feeding in data, Georgia Pacific is using AWS IoT Analytics to enrich messages with location and product metadata, in order to provide better customer services.

KONE Americas, which provides elevators, escalators, and automatic building doors, is using the IBM platform to analyze IoT data. “We always look for ways new technologies and innovation can allow us to better serve our customers,” says Danilo Elez, senior vice president of services for KONE.

“Elevators and escalators generate lots of data, and we wanted to leverage the data to bring value to our customers and personalize the customer’s experience of flowing from one floor to another—or one space to another—within a building,” Elez says.

After deploying the IBM platform in 2016 to build intelligence and analytics, KONE was able to launch new offerings such as KONE 24/7 Connected Services. The services enable the company to better predict malfunctions before they happen and boost equipment performance and reliability. “It means improved safety, full transparency, and ease of mind, because if something would happen, we’d already know,” Elez says.

KONE serves 450,000 customers and has 1.2 million elevators and escalators in its service base. The IBM Watson IoT platform and IBM Cloud can analyze, in real-time, vast amounts of data from embedded elevator and escalator sensors. When analysis of the IoT data detects an impending malfunction, technicians show up at the scene with the right parts and at the right location, to make the needed fixes.

“This helps to accurately predict equipment needs and help our technicians perform the right maintenance at the right time,” Elez says. The result is KONE can better predict and respond to technical issues in real-time, keeping equipment up and running, and also saving time and money.

“IoT empowers our [more than 20,000 worldwide] technicians to deliver better service, greater equipment availability, and more personalized experiences for consumers,” Elez says.