Natural language search – a new breakthrough?

While I am still waiting for an invitation from Twine  (probably you too?) I have received one from Powerset – natural language search. Powerset obviously is a promising company (and is promising a lot), so I was excited when I was starting to play around with this new tool which still isn´t available for the public.

The very first impression was good. The interface is well done and there are a couple of new ideas how wikipedia (and similar knowledge bases) can be navigated in the future. But unfortunately after a while it was clear, that search results must be improved. However Powerset might be implemented, the only benchmark which counts at the end is, which improvement the new application (semantic web or not) delivers compared to existing ones. Some examples:

– The question: “who is the president of the united states?” delivers some similar questions or related articles of wikipedia but NOT the right answer.

In comparison: ask.com delivers this perfect result.

– My next question “where was mozart born?” delivers “Getreidegasse” which is correct but actually too much detailed. Again, ask.com delivers the perfect answer.

A third try which was an even more difficult question: “how far is london from paris?” was again correctly answered by ask.com, powerlabs wasn´t even close…

When I was asking START, the world’s first Web-based question answering system, which has been on-line and continuously operating since December, 1993 (!) those three questions – all of them – were answered correctly.

So finally I was asking something really tricky: “What is the relationship between RDF and XML?”: Only START gave the right answer which was: “Unfortunately, I don’t have that information.”

Advertisements

2 thoughts on “Natural language search – a new breakthrough?

  1. Hey Andreas, first off, thanks for the kind words on the interface! That’s actually the team that I work on 🙂

    I just wanted to point out that while our index is currently limited to Wikipedia, the queries you submitted to Ask (as well as START) were not. If you try them out on Ask with the limit in place, you’ll see a more direct comparison –

    who is the president of the united states:
    http://www.ask.com/web?qsrc=167&o=0&l=dir&q=who+is+the+president+of+the+united+states+site%3Aen.wikipedia.org&search=search

    where was mozart born:
    http://www.ask.com/web?qsrc=167&o=0&l=dir&q=where+was+mozart+born+site%3Aen.wikipedia.org&search=search

    how far is london from paris?
    http://www.ask.com/web?qsrc=2352&o=0&l=dir&q=how+far+is+london+from+paris%3F+site%3Aen.wikipedia.org&search=search

    Also, you’ll see on the Mozart query that our right sidebar returns Salzburg.

  2. Pingback: Macedonia in Natural Language Search Engines! (ερωτήσεις στο Semantic Web για τη Μακεδονία) « OMADEON

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s