For ambiguous queries, conventional retrieval systems are bound by two conflicting goals. On the one hand, they should diversify and strive to present results for as many query intents as possible. On the other hand, they should provide depth for each intent by displaying more than a single result. Since both diversity and depth cannot be achieved simultaneously in the conventional static retrieval model, we propose a new dynamic ranking approach. In particular, our proposed two-level dynamic ranking model allows users to adapt the ranking through interaction, thus overcoming the constraints of presenting a one-size-fits-all static ranking.

Surf Canyon is again referenced, along with our 2009 SIGIR research paper:

We argue that a key to solving the conflict between depth and diversity lies in the move to dynamic retrieval models [2] that can take advantage of user interactions. Instead of presenting a single one-size-fits-all ranking, dynamic retrieval models allow users to adapt the ranking dynamically through interaction, as is done by surfcanyon.com [5].

In February 2008, Surf Canyon launched its Dynamic Ranked Retrieval application to rave reviews. As the body of research relating to Dynamic Ranked Retrieval grows, we continue to be encouraged by the potential of this technology to vastly enhance the quality of information retrieval.

In the session “Reinventing the Search Experience,” between Microsoft, Yahoo! and Google, Mark will be talking about “Search as a Conversation – The End of the ‘Stateless’ Results Page.” Having dynamically re-ranked over 1.3 billion queries over the past few years, Surf Canyon is uniquely positioned to discuss the potential of one of the most significant relevance-enhancing innovations since PageRank.

[Update 5/10/11] The video of Mark’s presentation is now online:

The competition, however, is extremely stiff:

• AdBlock Plus – A perennial favorite, it has been download from Mozilla over 110 million times, has over 2000 user comments and has been the most popular add-on for as long as we can remember.
• DownThemAll! – With over 40 million downloads, this is another favorite for people who want to download videos.
• Greasemonkey – Surf Canyon started out as a Greasemonkey script. With over 40 million downloads, too, not only are we fans, but we owe this one a debt of gratitude.
• StumbleUpon – Funded by some of the most prominent angels in Silicon Valley, acquired by eBay for $75MM in 2007 and then sold back to its founders in 2009, this hugely popular add-on is also one of the most financially successful add-ons ever.

We are honored, as we were last year, to be in such esteemed company. But we, too, bring quite a few things to the table. As the developer of an innovative and powerful search technology, there are few companies (if any) that can lay claim to dramatically improving the relevancy of Google’s own search results. We have also received many plaudits from users and the media as well as significant recognition in the academic community. The future of search, arguably one of the most important endeavors on the internet, is not always easy to see, but we are delivering it for millions of people every day.

So vote for Surf Canyon! Thank you and, as always, we appreciate the support.

As of last Thursday, Surf Canyon now offers users the option of having these same content farm results automatically removed from their searches on Google, Bing and Yahoo!. Simply download the latest version of our browser extension and then either click the button here or go to the Domains tab at my.SurfCanyon.com, select the “Remove content farm sites from the search page” box and save your preference. It is that easy.

ProductivityPortfolio blogged about this new feature this morning. It is unclear how they discovered the capability since we are only announcing it right now, but good for them!

Enjoy!

Note: The title of this post is from the Monty Python sketch.

[Update 2/16/11 - David Harry, with whom we spoke previously, references Mark Cramer and offers his take on the content farm situation in the latest issue of Search Engine Land.]

In §4.6.1, entitled “Beyond Predicting Static Ranking,” Yisong writes (emphasis added):

“It is common for information retrieval research to focus either on relevance estimation or user interface design, but rarely both simultaneously. However, for many tasks, it can be useful to model both jointly… One major limitation of result diversification over static rankings is that it sacrifices recall in favor of some minimal amount of utility for all usage intents – such a limitation could be dealt with by moving towards more dynamic interfaces.

Consider the example interface shown in Figure 4.8, which is inspired by and adapted from the SurfCanyon.com search engine [48]… by clicking or mousing over a result that matches the user’s intent, additional indented results are inserted into the original ranking… This interaction is quite natural, since the process resembles navigating a dropdown menu and since users are already familiar with result indentation. And yet even this one additional degree of freedom in content display can offer tremendous benefits…”

After the University College Dublin paper “A Recommender System Approach to Enhance Web Search and Query Formulation” and the Universidade Estadual de Maringá paper “An Approach to the Customization of Web Search Results,” this is the third academic reference to Surf Canyon. We’re delighted by the attention we’re receiving from the academic community.

Lastly, the team at Cornell recently drafted a brilliant paper, entitled “Dynamic Ranked Retrieval,” which dives deep into the study of real-time implicit ranking and offers statistical support for the “tremendous benefits” described above. It has been accepted for publication at WSDM 2011. While not yet public (we’ll post here when it is), we’ve been given permission to offer a sneak preview from the introduction (emphasis added):

“… most queries are ambiguous at some level. For such queries, there is often no single ranking that satisfies all users and query intents. While result diversification aims to provide a “compromise ranking” that provides some utility for all intents, diversification necessarily sacrifices recall…

The key idea is to make the ranking “dynamic” – namely, allowing it to change in response to user interactions after the query was issued.

From the user’s perspective, this may look as illustrated in Figure 1. This interface is inspired by and adapted from the SurfCanyon.com search engine…”

[Update 11/28/2010] The final version of “Dynamic Ranked Retrieval” is now available.

[Update 2/9/2011] The WSDM 2011 conference selected “Dynamic Ranked Retrieval” as one of six Best Paper Candidates.

They begin by observing that “the traditional search interface has remained relatively static” before going on to describe “a recommender system approach to Web search which allows users to dynamically interact with the result-space that is of interest to them,” which they see as “an overlay interface as a complement to an existing search engine.”  This will sound familiar to anyone who has used Surf Canyon, which the researchers acknowledge (emphasis added):

“… Surf Canyon is an excellent example of a next-generation approach, and allows users to interact with results returned by existing engines; in short, users can select a result and receive recommendations drawn  from related results that appear further down a result list. Surf Canyon launched in the latter stages of our own research and presents an interface similar to what is proposed here.”

While there are significant differences in terms or implementation, their results are similar to what Surf Canyon has demonstrated:

“A key question is whether the resulting recommendations were found to be relevant. In fact… at least one of the 3 recommended results was selected approximately 25% of the time, a very significant indicator of relevance…”

The portion about Surf Canyon begins at the 6m55s mark. Mr. Pisani starts with “Pick of the Week, use it, it’s called Surf Canyon,” before continuing with a demonstration and then:

This is artificial intelligence and [it makes your searches] more efficient.

The French version of the video is also available with the segment about Surf Canyon starting at the 6m37s mark.

The most useful is Surf Canyon, which offers a “real-time personalized search” that uses an artificial intelligence formula to adapt to the specific demands of the user. The idea is simple: only a small percentage of responses offered by any search engine correspond to what really interests us. Surf Canyon interprets each click as an indication of what may concern them and then finds similar information in the results offered by Google on the following pages (2, 5, 10, etc.) which are then deployed immediately below the newly activated link. The selection improves with every click, saving the user time.

Gordon had some very flattering things to say. After referring to one of the experiments outlined in our research paper, which was recently published by SIGIR, he replied, “A 40% uptick in user satisfaction by any measure is pretty impressive.” Later, after describing a little about the algorithm behind the re-ranking he responded, “That was impressive. An algorithm that recognizes the subtlety of subcontext has some decent IP cred behind it.” He concluded with:

“… I installed Surf Canyon and was fairly impressed. It was more helpful than I expected (though there may be few more skeptical critics of search user experiences than me)… If you’re not a search marketer [who often needs to see the “vanilla” version of the SERP], I’d definitely recommend taking it for a spin.”

Gordon’s next post in the series will be Part II of the conversation with Mark Cramer in which they will “speculate on where search might go in the future.”

[Update 10/1/2010] – Search Engine Land has published the second part of the interview.

“Results offered by search engines mix websites that are truly relevant to the context of the user with websites that have a marginal similarity to the query, or that are totally irrelevant… results offered by search engines, which tend to return many irrelevant websites, require the user to perform a manual filtering of results to obtain those that truly reflect his or her interest. The effort employed in this filtering may be high according to the classification accuracy of the results offered by the search engine (HARDTKE, 2009; BRUSILOVSKY, 2009; [B] MICARELLI, 2007; PANT, 2003).”

“According KHOPKAR (2003), search engines that have features to enable user interaction in order to construct a model of user interests have a higher level of complexity of interaction. This interaction can occur in two ways: implicit feedback, in which user actions are captured in a non-intrusive manner, or explicit feedback, which requires the user’s direct intervention… Software such as Surf Canyon (HARDTKE, 2009) uses both modes of interaction, however, lay users have difficulty interacting with additional features such as icons displayed at the end of each link enabling the approximation of similar websites…”

“Surf Canyon is an interactive IR system that dynamically modifies the query results using Web-based personalization… This software is an extension of the user’s browser and operates on various search engines. The personalization of Web query results offered by this tool comes from the combination of implicit and explicit feedback. The classification is based on a user model that infers the immediate interest of the user. Surf Canyon adds an icon at the end of each result offered by the search engine. When you visit the result or click on this icon, Surf Canyon recommends links similar in context to the one clicked (HARDTKE, 2009).”

“User actions that are considered to identify a positive interest in a website are:

• Length of stay on the document between 2 and 30 minutes;
• Action to Save the document on the computer;
• Action to Add the document in the browser favorites.

These choices are given considering the work of Goecks and Shavlik (1999), LIU (2006), [B] MICARELLI (2007) and MANNING (2008), HARDTKE (2009) and also a survey of users described in Chapter II… HARDTKE (2009) comments in his work that a user click on a hyperlink returned by a query already represents an interest in that website…”

“The development of this work meets the discussions exposed by BRUSILOVSKY (2009), HARDTKE(2009), MANNING (2008), [B] MICARELLI (2007), among others, regarding the need to reduce the effort of locating relevant content on the Web by adapting the results obtained from the search engines according to the interests of user.”

We’re flattered and humbled by the continued recognition of our work.