In this case study we are discussing search ranking optimization through live A/B testing -- a well-known method which is used by many large-scale online search engines. The specific example here is about a user feature in Nokia Maps (maps.ovi.com) that let's you discover new places. This user feature is powered by a SOLR-based geographical search engine. For effective ranking optimization we implemented a rank interleaving scheme and collect and evaluate click feedback. The presentation will also highlight the specific caveats of this method.