At some point past your tenth sensor, somebody will tell you it's time to get serious about your data. The advice always arrives as a choice between two doors: keep limping along in spreadsheets, or step up to "a real data platform" with a real invoice and, usually, a real hiring plan.
I think that choice is false. Not slightly misframed. False. After years of building environmental monitoring teams and years of building and buying solutions for them, I've become convinced the market doesn't actually split into "toy" and "serious." It splits into two camps that fail the same team in opposite ways, and the map has a hole in the middle where most monitoring programs actually live.
The commute test
Here's the analogy I reach for when I explain this. Say your job is to move a field crew across town every single day. One vendor offers you a bicycle: cheap, familiar, and technically capable of the trip — if you do all the pedaling, forever. The other offers you a freight train: genuinely powerful, built for a scale you may never reach, and it comes with a crew you have to hire, schedule, and pay whether or not the train moves.
Nobody in this story is lying to you. The bicycle really is cheap. The train really is powerful. The false choice isn't that either option is bad, it's the suggestion that these are the only two options, when what you needed all along was a van.
Camp one: cheap, because you're the data engineer now
The first camp is the do-it-yourself stack: Excel plus some Python, files in a shared drive, maybe a BI tool on top. It also includes the more sophisticated versions, like an open-source IoT platform you host yourself, or a custom pipeline built from a workflow orchestrator and cloud database services. The sticker prices range from almost nothing to modest.
The catch is that the sticker isn't the price, you have to pedal, you just didn't notice you signed up for it. Every one of these paths works by quietly converting your scientists into part-time data engineers. Somebody reconciles the vendor export whose headers changed after a firmware update. Somebody maintains the resampling script. Somebody is the only person who knows why the March numbers look like that. In our cost model, a ~$25/month spreadsheet stack lands around $1,500/month once you price the hours at what an environmental scientist actually costs, which is roughly sixty times the invoice.
And there's a quieter failure than the labor bill - data correctness. The default moves in this camp: a resample().mean(), a forward-fill, a pivot that fills blanks, these all silently fabricate readings where a sensor was actually offline. A gap is information. The cheap camp's default tooling erases it without asking.
Camp two: powerful, heavy, and built for someone else
The second camp is real infrastructure: the cloud lakehouses (Snowflake, Databricks) and the industrial data historians (AVEVA PI, Canary). I'll say plainly what some competitive posts won't: these are excellent systems. The lakehouses can process almost anything at almost any scale (we know, because Datasparks is a lakehouse). A historian will capture high-frequency plant data with a reliability we respect.
But they were built for a different premise. The train is magnificent, and it only leaves the station once you've laid the track to it yourself. The lakehouse assumes a data team: it ships enormous capability and zero environmental domain model, so your first project is hiring someone to build the domain model. The historian assumes a plant floor: OT protocols (the control-network plumbing a plant runs on), control-room operations, an integrator on contract. Neither knows what a monitoring well is, what a gap means, or why your quality flags have to survive an audit. You can build all of that on top of them. That's the point: you build it, at enterprise cost, with engineering you're funding indefinitely rather than a line item you can cancel.
The heavy camp hides its real cost differently than the cheap camp: not your evenings, but a standing team, metered compute billed whether the job is efficient or not, and, for anyone whose product is sharing data out, cloud egress fees that grow in step with your program's success.
The hole in the map
Put the two camps on one chart and explore what each path really costs per month once labor is included, against how much of the environmental job it actually covers. The shape of the market is hard to unsee.
The method behind that chart is public and deliberately simple: tooling at list price, labor at loaded rates (an environmental scientist around $59/hour, a ≈$42 base rate plus roughly 40% for benefits and overhead, and a data engineer around $200k/year fully loaded), counted at the fraction of a month each path realistically consumes for a network of about a dozen sensors. The domain-fit score is our own 12-dimension rubric. The dimensions, and what to ask any vendor about each, are laid out in the companion rubric post. You may weight the dimensions differently than we do. I think that's healthy. The point of publishing a method is that you can argue with it.
What the chart says is simple. The cheap camp clusters at low-to-middling fit and a much higher true cost than its sticker. The heavy camp clusters at high cost and a fit that never gets domain-complete, because domain-completeness was never its job. And the upper-left, high fit, low true cost, is nearly empty.
That's the gap Datasparks was built into. Domain-complete for environmental time-series. Self-serve, so a scientist runs it without borrowing an engineer. Fast enough to poke at interactively. And pay-as-you-go with a free allowance. Not a small freight train. Not a motorized bicycle. A van.
Where I'll happily concede
An honest map shows your own edges too, so here are ours. At the largest industrial scale, thousands of high-frequency tags, plant-wide OT integration, the heavy platforms' throughput and sharing maturity are real, and our lead narrows. If you already run a historian for operations, we'd rather sit on top of it as the analysis and sharing layer than pretend you should rip it out. And if what you need this week is one clean analysis of one dataset, well, we have a free tier for that, we can be your electric bicycle.
The world has mostly outrun the premise behind both camps, though. "Serious data work needs big-data infrastructure" made sense when the alternative couldn't handle the volume. For the data sizes a sensor network actually produces, that constraint quietly expired. A single modern engine handles it on hardware you don't need a team to run. The compromises got forgotten and became status quo. This series is me questioning the status quo, in public, with numbers.
What's coming in this series
Over the next few weeks we'll take the map one region at a time: the DIY stack, ThingsBoard, the data historians, Snowflake and Databricks, and building your own pipeline from scratch. Each one starts by crediting what that option is genuinely great at, and each one states where its side of the argument wins, because a comparison you can't lose isn't a comparison, it's an ad.
If you'd rather skip ahead: the single most useful thing you can do before talking to any vendor, including us, is to walk in with a rubric and make everyone score against it. Some of the questions on ours, we don't ace either. That's what makes it worth asking.
