I've observed the same in manufacturing … and fitness trackers a la FitBit.

There's initial value from training yourself on what something looks/feels like … but diminishing returns after that. Whether there is more value to be found doesn't seem to matter.

Factories would sensor up, go nuts with data, find one or two major insight, tire of data, and then just continue operating how they were before … but with a few new operational tools in their quiver.

Same is true of fitness trackers: you excitedly get one, learn how much you really are sitting(!), adjust your patterns, time passes … then one day you realize you haven't put it on for a week. It stays in the drawer.

Not unless they're threatened with ruin will people make changes to the standard way of doing things. This is actually … not bad! Continuity is important, and this is kind of a subconscious gating function to prevent deviation from a proven way of working. So, the change has to be so compelling or so pressing that they're forced to. Not a bad thing.

While we think things change overnight in this world, they generally take awhile … stay patient … it's worth it.

Classic paper on soil moisture sensors (from 2010!) -- the title says it all:

"Mate, we don't need a chip to tell us the soil's dry"

https://doi.org/10.1145/1858171.1858211

I tend to think the problem is the "random digging for correlations" part.

Having tons of data is a Good Thing, so long as you can afford the marginal cost of gathering and managing all that data so that it's ready at hand when you need it later.

It's how you use the data that makes all the difference. If you're facing an issue you don't understand at all, don't go digging for random correlations in your mountain of data to find an explanation.

Think like a scientist: you need a valid hypothesis first! Once you have a hypothesis about what your issue might plausibly be, then you make a prediction: "If I'm right, I suspect our Foobar data will show very low values of Xyzzy around 3AM every weekday night". Only then do you go look at that specific data to confirm or refute the hypothesis. If you don't get a confirmation, you need to go back to hypothesizing and predicting before you look again. You can't prove causation by merely correlating data.

Fine-grained measurement is useful when you have options for fine-grained action.

You don't need a chip to tell you that the soil is dry, but if you can use that chip to regulate drip irrigation that can apply substantially different flow to different plants, then you can get a not-too-much, not-too-little watering even if you have a big variation in conditions.

You don't need a big analysis to acknowledge that everybody knows that a particular competitor has lower or higher prices and adjust your pricing; but doing that continuously on a per-product basis does require data and analysis.

Most of the time this story is true, but think this way, the person that was using the system was an expert on the subject. If you can replace the expert with a person just looking at a graph from time to time to know if you have to irrigate the soils it's a different thing. Most of the data or ML tools show us something that the client as an expert already knows, but the true power of this tools is to give them to a non expert user and have roughly the same level of proficiency

I've been telling folks, storing everything all the time is wasteful, a better alternative is:

1. Keep the raw full data for short period of time, at most 1 month.

2. Downsample what you need for longer period of time (5-10% of the full data).

3. Aggregate your metrics on a yearly basis to save money and compute costs.

This analysis reminds me of the big interest in the use of hyperspectral imaging for agriculture. The idea was the greater spectral resolution (greater than Landsat) would result in more interesting information. Agriculture was one of the applications. But, once you did find the interesting stuff, you no longer needed a hyperspectral sensor. You could just look at one spot with a much lower cost sensor.

So hyperspectral, like big data, is useful up front. But in the end, much simpler tools and algorithms will solve the problem on a continuing basis.

Oh, those soil moisture sensors, they are so fascinating.

I spent a number of exciting year developing a high frequency soil impedance scanner and finally understood why I was doing it. To confirm the obvious :)

Interesting. Sounds as if what's really needed isn't so much collecting and analysing lots of data, but an alarm that's triggered when observations deviate from a set of assumptions. Observations that confirm some definition of "normalcy" -- as most observations would -- can be discarded.

I think there's a problem at the heart of the matter, specifically the idea that the act of measurement is in itself powerful when in point of fact that this isn't universally the case. As the old adage goes: "garbage in, garbage out." Even more troubling, there is a physical limit to our ability to model what we measure. Take the retina, it has around a million light receptors and even if you assumed they only have two valid states then you're left with around 10^300,000 bits of information to process, so good luck with that. Same thing applies to whatever firms are measuring and what they think is conveying relevant information as they'll have similarly exponential increases if they don't filter out the vast majority of irrelevant data points and states.

Very interesting. Left AgTech last year but had similar experiences, even worse where often the single most prominent use-case was to follow some painful necessary documentation of ag inputs (chemical, seeds, fertiliser) to get subsidies. Real inputs from data? Nah!

But isn't this the essence of industrialization and automation? Messure, adjust process; repeat until feedback loop is stable - document and keep doing the thing that works, over and over?

If you want Toyota style continuous improvement you would need to improve in new areas of the process / new metrics, most of the time?

On the flip side, there's some great action coming from data insights. Look at Strella Biotech - they're putting sensors in sealed warehouses to detect spoilage for certain vegetables and fruits. That's something that can have great returns with just a few IoT devices and a novel sensor.

Or they could put on their work boots and go walk around the field and kick a few dirt clods, or science forbid! put a hand in the soil to check the moisture content.

> goes geek-crazy for the first year or so

The problem is that they don't stay geek-crazy?