All of my career I have worked on the side of the vendors - working with folks who build stuff and sell it to customers. That complicated space between the creation of product and the needs of the user has always fascinated me. It always seems like such a complicated space. It is fraught with miscommunication, attitude, ego and downright technical obfuscation.
The "customer" thinks they know what they need but in my experience, mostly do not. Their goals and appetites are biased by their understanding of what is possible. And we in the spatial world have done a very poor job of helping customers understand what is actually possible, what could be done and what the benefits might be. There are so many postings out there espousing the dire need for Location Intelligence but if, as a customer, you would like to know what benefits there may be for your organization, its gets vague very quickly. Or you end up paying very large amounts of money to a consultant for him to sell you something you probably don't need or want.
I am, for the first time for me, now working with a group who surely could use spatial analysis on a grand scale and realize what for me are clearly dramatically useful outcomes. Why doesn't an obviously sophisticated and singularly powerful organisation using advanced analytics on a truly massive data lake include spatial in their analytics ?
After a few weeks here, I think I know the answer. What is the absolute first and basic step before any useful spatial analytics can take place ?
Geocoding.
Pretty much all spatial analytics has to start with geometry information embedded alongside the other attributes. But here's the issue. If you try to justify the expense of geocoding in an IT budget, it literally has no useful direct outcome. Sure, you might correct some addresses and you may find redundant records but that's a feeble justification for an expensive and disruptive process. A process that actually loads more data and poses such ugly problems as keeping versions of addresses, understanding when a record needs to be geocoded again, retiring addresses and spatial indices.
Once the data is geocoded, it seems to me that the path to the inclusion of spatial analytics is still complicated but every addition of software, data or combination of both has a viable business justification.
I suggest then that geocoding should be available to all as a fundamental part of whatever IT mechanisms are incumbent. And it should either be free or bundled such that everybody who could use it, can. Using Pig ? There should be a free geocoding UDF for Pig. Hive ? Same thing. Microsoft SQL Server ? The Data Quality Services extension for geocoding should be included. And so forth.
With a free or at least native geocoder present in standard IT deployments we can remove a barrier that stops spatial analysis creeping (or vaulting ?) into enterprise analytics.
