I’ve been writing recently about the dimensions of Big Data in the IoT: urgency, importance, frequency, consequences, remedy, cost. You can find my stories on IoT Journal or at my personal website.
The big item missing from this list is, of course, size. How big are individual messages or files? A couple of kilobytes? Many megabytes? And are there still a lot of people who think of Big Data as massive, petabyte-size repositories of epidemiological, meteorological, or particle acceleration data?
I didn’t originally include it, because I saw it as twinned with frequency. In the IoT, most message & file sizes are going to be small. Big Data in the IoT has more to do with sensors and less to do with massive scientific apps.
Frequency might mean a real-time sampling rate of 10 times per second, 100 times per second, or more. Or it may be tethered to a passive device that reports in a timeframe we can call “every once in awhile.” So the frequency dimension might vary by several million times from one application to another. Even if expressed logarithmically, that’s a lot of variance.
Nevertheless, I think I’ll combine the two, and change one of the six categories to size/frequency.
Pay to Play
I go through all this because, in the end, someone has to pay for the dataflow and all tha supports it. Most analyses seem to focus on the amount of data being consumed and stores, ie the size/frequency dimension.
But urgency and importance are mission-critical in setting up proper IoT monitoring, analysis, and the speed of that analysis. Furthermore, the consequences of something gone bad can vary from “there’s a lightbulb out on the bridge” to “a section of the bridge just collapsed.”
The dimensions of consequence and remedy are therefore potentially much more important to cost than the dataflow.
Designing and deploying an IoT project, whether something fun like installing a smart public bench or something grave like monitoring traffic flow, is thus far more than an IT project.
As we get involved in analyzing the cloud computing infrastructures that will no doubt underpin and serve most of the IoT, it’s time to focus not only on cost-per-whateverbyte—the size/frequency dimension–but also the other dimensions, which tell us what this IoT thing is supposed to do and what happens when things go wrong.