Searching all possibilties

Doooh’s Picture Of the Day

About the only thing around here that is green, just yet…

One of the things we do here at work, or rather what the application that we write here, does, is it allows the user to search structured and unstructured data. Structured data is information that is in a particular, familiar form and is, more or less, always in that form. Unstructured data is anything and everything else, including information like news feeds from the internet, or conversations over heard at the water cooler. So we allow our users to be able to search all of those different types of information easily. I remember an old TV commercial from a long time ago, where the final statement it would say was “…An educated consumer is our best customer…” and that is how you can describe being able to search structured data. Knowing the data you want to search for is very important. On the other hand, huge internet search engines like Google, I believe are spoiling our abilities to do some of the indepth search/researching that sometimes we all have to or want to do.

When you use Google, the easiest and most common thing to do with it is to type in a word, or any combination of words and hit the search button and let Google figure out what you might want to see as results. The more you use it, the more familiar you become with the subtle nuances of the ways you can perform queries with Google. How you can group words together, or put in certain keywords to tell Google how to generate a specific enough query so that the results you’re most interested in actually present themselves quickly. The result is that you perform a search and after seeing the initial results, further refine your searching until you’re given a set of results that you might be able to locate the information you’re actually looking for, in. Because of the vastness that is Google you will hardly ever be returned to you, a succinct enough set of results that will satisfy your search. Once you have done your best to refine and further refine your searches, you still have to wade through the results searching for what it actually is you are looking for. Thats searching unstructured data.

With structured data you know exactly what information is stored, and in what form its in, so you can search for specific criteria. The results are more precise and in alot of cases, based on the specific criteria, more accurate. Instead of just typing in some random groupings of words, you create specific criteria like “I want all companies, who have active projects in South America, who mine copper and have a production greater than 4000 metric tonnes per day and whose owners’ head offices are in Texas that have stock offerings greater than 4% over last years offerings”. From this query you may only get one or two results, but the only way you could have come up with this criteria in the first place is by knowing the data that you are searching. Thats searching structured data.

With the popularity of Google and the extent to which it is used for “research”, it is no wonder that everyone’s idea of doing a search has been changed to expect some similar process as you would use when searching with Google. Instead of having some prior knowledge of whatever it is you are searching for all you need is to know some keywords related and boom you get some results. For companies researching specific industries, or activities within such industries, performing searches with Google just isn’t specific enough. You need to know that some companies operate a mine, whereas others might be owners, or that certain ores’ production is tracked in Millions of tonnes and others is based on volume. Our challenge is to provide the user with an easy-enough interface for searching to allow them to do the simple “Google-related” process of entering in a few keywords and allow them to sift through the results themselves as well as allow them to pick and choose the specific criteria to come up with complex queries that are going to return specific contextual results. In most cases not only do we have to allow them to do both, separately, but also allow them to integrate the results from one to the other as well as have one query affect the results of the other. My concern with all of this is that when it comes to searching, if it isn’t “Google-like” then its too complex for anyone to want to take the time to learn how to do. The ability then, to be able to come up with and perform complex searches gets lost as a general ability. No longer would it be necessary for someone to have or need specific industry knowledge because all a person would need are some keywords and access to Google to drop them into, to get some sort of results. Our abilities to discern the specific knowledge that we may require for our jobs gets lost in the shuffle of slapping all searches into Google and letting it do all of our work for us. Its like the teenager asking his math teacher “Why do I need to know how to add, when some sort of device can and likely will, do it for me?”. We seem to be at a stage of our evolution where we can and perhaps want to rely on other “things” to do some of what we commonly did without thinking before, for us. Thats ok, for some I guess, as long as we realize that those things will then be lost to us. I think we may be going down this route with searching, and I think Google is allowing us to do it, and I’m not convinced its the best for us.


Two strangers are sitting in an adjacent seats in airplane.

One guy says to the other, “Let’s talk. I hear that the flight will go faster if you strike up a conversation with your fellow passenger.”

The other guy, who had just opened a good book, closes it slowly, takes off his glasses and asks, “What would you like to discuss?”

The first guy says, “Oh, I don’t know; how about nuclear power?”

The other guy says, “OK, that could make for some pretty interesting conversation. But let me ask you a question first:

“A horse, a cow, and a deer all eat the same stuff, but the deer excretes pellets, the cow, big patties, and the horse, clumps of dried grass. Why is that?”

The first guy says, “I don’t know.”

The other guy says, “Oh? Well then, do you really think you’re qualified to discuss Nuclear Power when you don’t know shit?”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: