IoT and Streaming Data increases Efficiencies
Operational efficiencies are improving by leveraging “Internet of Things” (IoT) devices, allowing companies to collect data from systems, devices, etc. that was previously difficult or even impossible to obtain. Nowadays, with the evolution of data storage, networks, devices and technologies, data streaming is becoming a key factor to consider while designing architectures and applications due to the increased possibility of ingesting new sources of data to combine it with social media and other streaming data sources, managing them all together alongside traditional batch-based.
Listen to Pablo Peranzola talking to Karl Woolfenden on Link’d AM
However, there are some aspects of the streaming of data by IoT devices that are different from other sources. It tends to be more time sensitive, the volume is typically quite large, it may not have value in the long term, and, generally speaking, you get the most value when it’s combined with other enterprise data to complement the business view. Some other challenges, examples, and possible solutions are listed below.
Question 1.
How are companies using streaming data from IoT devices to improve operational efficiencies? Can you provide 2-3 examples?
Some companies are using IoT sensors to collect information from mailboxes or cities’ trash cans. These devices sense how full the containers are and, based on the information being streamed to their platforms, they can input the data and leverage optimization algorithms to analyze the best route for their trucks. These algorithms also include other data sources like traffic status so the route can be further optimized, suggesting drivers a route as well as the best time schedule. In this example, companies can save money on both fuel and transportation maintenance. Other companies create efficiency indexes based on historical data. Data from vessels such as speed, wind speed, route, and temperature is collected, processed and then compared with historical data, allowing the creation of real time energy efficiency tracking. Additionally, it is possible to do prescriptive recommendations on how to reduce costs.
A new breed of energy generation companies are expecting an increasing amount of data to be gathered so they have a better understanding of how their investments are performing. Data is streamed by sensors on connected turbines measuring speed and the volume of fuel consumed. Leveraging this data results in smarter business decisions being made.
In this particular line of business, contracts include clauses for energy producers where an agreement is made on the amount of downtime for their assets. There could be penalties if they don’t comply with these agreements due to other obligations for energy distribution. Having the connected turbines, for example, allows producers to be able to predict maintenance based on historical data to avoid an unexpected inconvenience, and also have precise information about the amount of energy that was generated by a specific turbine in a given period. This data allows them to know exactly how much they will be earning and how much fuel was consumed. The later also opens the possibility of identifying malfunctions based on unusual fuel consumption.
The livestock industry has an example of improving operational efficiencies through IoT as well. These devices make it possible to determine if a cow is eating or not by collecting data from the cow when it is near the eating area with a connected scale that communicates weight data. There are also collars that allow producers to identify the position, acceleration and direction of a cow when it is moving. With this data it’s possible to clearly see the livestock’s behaviour during the night and anticipate illness just by seeing a decrease in the animal’s average walking distance.
Question 2.
What are the challenges that companies need to overcome to use streaming data from IoT devices effectively? How do you suggest they overcome these obstacles?
There are a variety of challenges that companies need to overcome to use streaming data from IoT devices effectively. Some of the common challenges that companies have to face are:
- Interoperability – our ecosystem will contain different types of devices, networks, and platforms as subsystems of a bigger system, all working in a collaborative way. For example, there are connected cars streaming more data as they evolve and not all the components of the cars are being generated by the same company.
- Performance/Scalability – the edge of the system is extended with devices that have different levels of processing power and storage. Despite this variety of devices, the IoT solution should perform at an acceptable level and also handle the increased workload that could happen suddenly.
- Device management – the volume of devices being incorporated is generating challenges in the way that they are being managed. Basic activities such as the need to upgrade the firmware can be a hard quest if you don’t have a system that allows you to roll out these changes in an efficient and proper way. You don’t want your whole solution impacted in the event of an unexpected error. The more devices you add the more complex this management gets. A good deployment strategy and the correct tools must be selected to mitigate this challenge.
- Availability – our ecosystem must aim to be running 100% of the time, especially in critical IoT solutions such as the ones developed for the healthcare industry. IoT devices must anticipate a variety of possible scenarios, for example, the moments in which the strength of network connectivity is unstable.
- Evolution of the devices – IoT devices and other systems evolve over time, and in this fast growing business, evolution is presented in many different ways. It is important that any given solution being built must be thought through in a way that allows for changes without causing damage or downtime of the whole ecosystem. This is tightly related with the Interoperability concept mentioned above, since any change/evolution could impact how the system’s parts collaborate and communicate with each other.
- Security – it might be required for these devices to store sensitive data, which adds a risk of attackers accessing that information. Depending on the specific situation, the devices themselves can be stolen by the attackers. Because of the current scale of IoT, there are more communication channels between different systems and these devices might have limited computation power and battery limitations, meaning that they cannot run complicated cryptography.
- Data processing time – timing can also be a challenge. Data can be processed in real-time, near real-time or batch. A smartwatch streams data in real time of our running performance when we are exercising. A historical dashboard showing the telemetry of sensors of a vessel to keep track of the fuel consumption is an example of near real-time. And finally, a batch processing example could be a company processing historical data of a group of turbines to define when the best time is to go through the maintenance process.
Overcoming these challenges, although complex, is not impossible. As a first step, having a technology partner that can help you deal with these challenges and be a two-way bridge between the hardware world and the digital world is beneficial. This partner should not only understand how the digital world works, but also understands how IoT devices impact business, and can help companies stay fit and updated with technology trends. The technology partner helps a company collect data from all IoT devices and assures that data-driven decisions are being made to add value to the company’s business. They can also assist with selecting the best technologies, building and maintaining platforms, setting up teams of data specialists, understanding how your industry works, and even providing specialists that know about hardware, firmware and how these products should operate collectively.
Designing a well thought out architecture is another key aspect to overcoming the obstacles to leveraging streaming data from IoT devices effectively This includes the way in which data flows between IoT devices and platforms. It is very important to devote the necessary time to design processes that maximize data flow. Some things to consider are the communication between devices and the cloud, and whether it is directly or indirectly using a hub or gateway. Additionally, at the design stage, it is key to contemplate data security. If IoT devices exchange data, it is crucial to guarantee the security of all devices.
On the edge, fog computing could be a solution to performance and latency problems. Even though cloud computing has been beneficial for hardware management and service configuration for the IoT space, it generates a latency that is a disadvantage. Fog computing enables devices at the edge of the network that need external computing power or storage to access these on local nodes instead of the Cloud, increasing response time by moving application logic and data storage to the edge, and decentralizing the system. However, as performance increases there is a cost to pay. Having smarter nodes near the edge increases the security risks of attackers being able to more easily connect.
If security fails, the system should still be able to recover to an acceptable state. Backup systems play a crucial role in resiliency. If a node fails, a backup node can seamlessly take over its transactions so the system is always stable and the end user is not impacted. Resiliency should always be considered as a critical part of availability.
Additionally, it’s ideal to plan for potential sensor outages. There are several things to consider when planning for back-up solutions, but redundancy in sensors can help increase availability during outages. Adding more sensors for a given solution can be an option rather than only having one sensor to rely on for data streaming. Another common technique is to have a regular monitor of the device components.
There are techniques commonly used to improve the security and privacy of the data stored in the devices. These techniques can include keeping the firmware up-to-date, changing some logic to make it harder to breech, and changing the way that the device use its memory.
Finally, there are IoT platforms specifically designed to overcome the challenge of device management. There are solutions that are simple and some with increasing degrees of complexity, some open source and some licensed. When making the choice between an existing platform or building one from scratch, it is important for a company to consider the amount of devices to connect, the communication protocols that detail how they are planning to connect with those devices, their means to update firmware (as an example, to batch or not), and how flexible the platform is to adapt to the fast-changing technology.
