Natural language search – a new breakthrough?

While I am still waiting for an invitation from Twine  (probably you too?) I have received one from Powerset – natural language search. Powerset obviously is a promising company (and is promising a lot), so I was excited when I was starting to play around with this new tool which still isn´t available for the public.

The very first impression was good. The interface is well done and there are a couple of new ideas how wikipedia (and similar knowledge bases) can be navigated in the future. But unfortunately after a while it was clear, that search results must be improved. However Powerset might be implemented, the only benchmark which counts at the end is, which improvement the new application (semantic web or not) delivers compared to existing ones. Some examples:

– The question: “who is the president of the united states?” delivers some similar questions or related articles of wikipedia but NOT the right answer.

In comparison: ask.com delivers this perfect result.

– My next question “where was mozart born?” delivers “Getreidegasse” which is correct but actually too much detailed. Again, ask.com delivers the perfect answer.

A third try which was an even more difficult question: “how far is london from paris?” was again correctly answered by ask.com, powerlabs wasn´t even close…

When I was asking START, the world’s first Web-based question answering system, which has been on-line and continuously operating since December, 1993 (!) those three questions – all of them – were answered correctly.

So finally I was asking something really tricky: “What is the relationship between RDF and XML?”: Only START gave the right answer which was: “Unfortunately, I don’t have that information.”