Tuesday, June 26, 2007

Solving the “last mile” problem with human-powered search

Yesterday’s New York Times article Human Touch May Loosen Google's Grip covers a topic near to my heart – the difference human-powered search can make in quality of results.

Professional search users – in our case portfolio managers and analysts – require very high relevancy in their research results. Relevancy means delivering articles that match only the companies and topics of interest. So, if an investor is not interested in Microsoft Office, but is interested in the adoption rates for Vista, the system needs to only return the latter. If the investor cares about CEO’s comments to the media, but not every time the CEO’s name is referenced the system needs to only return media interviews - or if you really want the less guarded insight - only media interviews given to international news or blogs.

There are countless examples where you'd get frustrated without a specialized system. Think about trying to find out when your small-cap investment wins a local contract - for example winning an outsource contract from a large customer that does not get picked up by the national press but does get written up by an employee blog or the small town newspaper - or wanting to be notifed when your international electronics component company wins a contract with Apple for iPhone production - but it's not picked up by the news wires because it's only covered in local press. Imagine the time and effort it would take you to make any standard search engine find these types of intelligence.

This is where human-powered search comes in. Yes, great search engines can get you close, but they can’t get you the “last mile”. Having a team of skilled search authors is essential to the final step of configuration. In the ideal case the users would be able to do this themselves, but the reality is that a) it requires one level of complexity beyond the user’s skill level and b) 90% of professionals using search for a business application simply cannot put the time in to configure a system to get them that last mile of quality relevance, even if they have the IT team who could get trained up enough to do the work.

By picking the target user base, whether it is consumers with the 10,000 most likely searches, or portfolio managers investing in global securities based on fundamental research, the system can get to genuine content that matters to that group of users. Algorithms are important to review the millions of documents that apparently match an investment topic and select the set most likely to be relevant. Then human editors review the content and both refine the search configuration to get better and better results over time for that topic, and to hand select particularly interesting insights for users who subscribe to the topic in question.

It’s all about that last step. It takes a man-machine combination to get to consistently relevant results for every user. And this combination of algorithms and editors needs to be designed to scale over thousands of users. That’s what makes it interesting.

No comments:

There was an error in this gadget