The Climate Change of Data

Estimated reading time: 3 minutes

I have started to hear people referring to their cloud systems as "the fog." I sense that the witticism here is hiding a very real experience. In our industrial past, we do have a history of simply venting gases into the atmosphere, so is the same happening in this digital age to our data? Can we just dump data into a cloud and hope that some very clever analytic software will sort it out for us when we need it?

The real answer lies with understanding what the cloud can really offer, what it cannot, and what is needed to build the cloud into an effective strategy for data storage and accessibility.

The Search Engine Experience

Search engines such as Google have really spoiled us over the years, setting our expectation to be able to find something specific from the seemingly unlimited amount of information out there, by typing just a few words or saying a simple sentence. Trying to imagine the sheer volume of data that these search engines access, which we expect is in some kind of cloud, through which the search engine algorithm needs to go through to deliver the result in less than a second, is mind-boggling. The search engine is a great example of a cloud-based system. Other than the simple user interface, none of the software is on your device, and other than the result of the search, none of the data.

Search engines are designed to be extremely well optimized to cope with, let's call it "human" data, the majority of which was not created with searching in mind. It then takes a human mind to select the results that make sense, which may not be the one at the top of the list. As the internet has matured, search engine optimization (SEO) techniques have been created which embed key items of data within web pages designed to help search engines understand the content and intent of the web data. Even for humans, just random data in the cloud is not good enough.

Considerations of Cloud Storage

Web pages on the Internet do not of course reflect the true nature of cloud data storage. From a typical manufacturing standpoint, cloud storage is simply another way or place in which to put data. From a usage perspective, many see it as being no different from an on-site server (now marketed as a "local cloud") or even the external hard disk on your laptop (now marketed as your "personal cloud"). Off-site cloud storage is vast and can easily be cost effective, requiring no maintenance or fixed overhead other than paying the service bills. There are however a couple of very important things to consider with cloud storage.

Firstly, you must trust that there is appropriate security in place, and in some cases, for example with ITAR restricted data, make sure that the physical storage distribution is within friendly places. Cloud data is generally physically spread across data-centers all over the world and will naturally be driven towards areas that offer lower costs. Adequate security for data access must be provided.

The second thing to consider with cloud data storage is the requirement for how the data will be accessed. Links from a site to the cloud have a finite speed, which is basically the Internet connection. Though theoretical upload and download speeds may seem quite fast these days, these same connections are supporting the email system, as well as everyone in the company on a browser, and even the coffee machine these days. Mission critical systems must share bandwidth with an increasing number of devices and services unless equipped with dedicated lines.

Limited connection speed is all very well for the continuous trickle of upload data, but once up there, it is rather impractical to ever consider downloading all the data again. As with the search engine example, if your data and software are both in the cloud, system performance and capability will not be limited by connection issues. In many use-cases, this is fine, for example the use of enterprise-grade Business Intelligence tools to look at longer term statistical trends.

To read the full article, which appeared in the March 2018 issue of SMT007 Magazine, click here.