You've indexed your data and people are searching it. But how do you know if they are happy with the results? How do you know if they are finding what they need? With search incresingly becoming the primary information access mechanism, knowing how your search is doing is not just a matter of mere curiosity, but often has direct business impact.
In this talk we'll talk about Search Analytics and how it can be used to answer questions like:
Are too many users getting the dreaded "no matches" results?
How deep into search results do people dig?
Which hits are they clicking on, or what percentage of them don't click on any hits?
How much do they use the Did You Mean or Auto-Complete suggestions?
We'll explore what specific Search Analytics reports tell us and what specific actions you should take based on those reports.
Interview: Otis GospodnetiÄ‡
by Sebastian Arnold, TU Berlin
Q: Hello Otis, you just gave a talk about Search Analytics. What is your main work behind that and why do you think this is important?
A: I am the founder of Sematext, we're focused on the development of scalable search and analytics. We are consulting companies to improve their search services by analyzing the user's behaviour on the site. This is related to web analytics, but the reports are based on much more data and include knowledge about the site and its content. It's interesting to see how few people actually use search analytics. It is important to know if people are really finding what they need and if they are happy with the search results. You can't tell that from web server log files only.
Q: So, I've seen you are collecting a large amount of logs about clicks and queries on the site, you basically try to monitor "everything" that happens there. You analyze that data and generate reports about the usage of different site functions and rates of search failure etc. But all of this happens on single transactions. Is there a possibility to combine longer click paths of a user's search intention and analyze the whole navigation sequence?
A: That's fuzzy. The problem is what I refered to as "search sessions" in my talk. You can try to group the log lines by user and then sort by time to get a sequential click log. But how can you tell if the user is still following the same search trail? Maybe he already gave up on one topic and now tries something completely different. You could try to cluster by similarity, for example to find different spellings of the same search. But to really find sessions you have to know more about the relations of his search inputs.
Q: I'm working on something like this at the moment. Our data base is very structured and so we can tell if two results are somehow related. Then, we're writing more detailed metadata about the origin of an event in the logs. We instantly see if the user stays on the same object, object type or general topic. This helps us to find the boundaries of a user's search intention in the clickstream.
A: That's interesting. But I think you wouldn't find many matches for the same actions of different users. And it's still hard on a site with a lot of traffic.
Q: You're right. What about doing this on a higher level, e.g. on the navigational structure of a site? We can then try to find typical paths like "search" -> "not found" -> "re-search".
A: Yeah, of course it somehow is possible. Especially if you have more information about the data itself. I just haven't done this yet.
Q: Alright. Now we're finally at the end of the lunch queue. Thank you for your time, Otis.
A: You're welcome - thanks for the nice talk. See you later!