For ambiguous queries, conventional retrieval systems are bound by two conflicting goals. On the one hand, they should diversify and strive to present results for as many query intents as possible. On the other hand, they should provide depth for each intent by displaying more than a single result. Since both diversity and depth cannot be achieved simultaneously in the conventional static retrieval model, we propose a new dynamic ranking approach. In particular, our proposed two-level dynamic ranking model allows users to adapt the ranking through interaction, thus overcoming the constraints of presenting a one-size-fits-all static ranking.
Surf Canyon is again referenced, along with our 2009 SIGIR research paper:
We argue that a key to solving the conflict between depth and diversity lies in the move to dynamic retrieval models  that can take advantage of user interactions. Instead of presenting a single one-size-fits-all ranking, dynamic retrieval models allow users to adapt the ranking dynamically through interaction, as is done by surfcanyon.com .
In February 2008, Surf Canyon launched its Dynamic Ranked Retrieval application to rave reviews. As the body of research relating to Dynamic Ranked Retrieval grows, we continue to be encouraged by the potential of this technology to vastly enhance the quality of information retrieval.
This week Brandon McMullin started work at Surf Canyon as a Software Engineer. He will focus primarily on the development of our new social search application, Chummo.
In addition to his passion for software engineering, Brandon studies Shorinji Kempo, an art that focuses on personal improvement and the balance between strength and compassion. He is also a father. His 13-month-old daughter just recently gained some proficiency in walking and is intensely interested in exploring the world around her. She’s also very interested in daddy’s toys (like the laptop and Xbox controller) and has begun building structures with Mega Bloks. Clearly an engineer already in the making!
Our first hire in a few years, we’re thrilled to welcome Brandon to the team!
Last month, the High Tech Practice at McKinsey & Co., one of the most prestigious global management consulting firms, authored “Impact of Internet Technologies: Search” which “examines the value of technologies used to navigate the Internet and is part of a series that focuses on different, Internet-related technologies.” Surf Canyon is honored to be one of the few companies, and the only private company besides Facebook and Twitter, mentioned in “The future of search” section:
Importantly, relevant search results are increasingly deemed to be personalized. Autonomous search agents that make suggestions based on personal data, including the user’s location, metadata, and more advanced algorithms, are in sight. For example, Surf Canyon, a US company, is developing real-time, personalized search capabilities that transform static lists of search results into dynamic pages that rerank results based on a user’s real-time online activity.
Surf Canyon is proud to be a sponsor of the Fifth Annual International Conference on Web Search and Data Mining (WSDM 2012). After having our research selected for oral presentation at SIGIR ’09 in Boston and then having attended other academic conferences, we have a strong appreciation for the effort and dedication required to produce high-quality research in the very challenging field of search. While we didn’t have a paper to submit this time, we will naturally be attending the conference and look forward to the presentations as well as connecting, and reconnecting, with talented researches from around the world.
Mark Cramer, CEO of Surf Canyon, has been selected to present at the exclusive Search Insider Summit to be held at the South Sea Island Resort on Captiva Island, Florida from May 4 to 7. The semiannual event is chaired by Gord Hotchkiss, CEO of Enquiro, and “brings the best minds in the search industry together to share cutting edge information and experience.”
In the session “Reinventing the Search Experience,” between Microsoft, Yahoo! and Google, Mark will be talking about “Search as a Conversation – The End of the ‘Stateless’ Results Page.” Having dynamically re-ranked over 1.3 billion queries over the past few years, Surf Canyon is uniquely positioned to discuss the potential of one of the most significant relevance-enhancing innovations since PageRank.
[Update 5/10/11] The video of Mark’s presentation is now online:
For the past 40 years since the inception of search, the way it’s worked is this: user enters a query, that query is matched to an index of documents, tremendous activity is deployed to try to determine the relevancies of those different documents, and a search result set is produced. But that search result set is static. There’s an order to those results. They go from 1 to 10 and then 11 to 50 million, and that order does not change. It’s stateless.
What we have been doing for the past few years is essentially applying state to the search page in order to make the results dynamic. And if you consider “dynamic” to be something resembling a conversation in the sense that the search result page is actually responding to every input from the user to alter the content on the fly, then I think that’s an interesting way of looking at searching.
As the fans of Surf Canyon are already aware, last year we beat out Google, Yahoo! and Bing to win the 2010 About.com Reader’s Choice Award for “Best Search Engine.” For 2011 we are proud to announce that we have been nominated in the category “Best Overall Add-on (Non-security).”
The competition, however, is extremely stiff:
AdBlock Plus – A perennial favorite, it has been download from Mozilla over 110 million times, has over 2000 user comments and has been the most popular add-on for as long as we can remember.
DownThemAll! – With over 40 million downloads, this is another favorite for people who want to download videos.
Greasemonkey – Surf Canyon started out as a Greasemonkey script. With over 40 million downloads, too, not only are we fans, but we owe this one a debt of gratitude.
StumbleUpon – Funded by some of the most prominent angels in Silicon Valley, acquired by eBay for $75MM in 2007 and then sold back to its founders in 2009, this hugely popular add-on is also one of the most financially successful add-ons ever.
There has been a bit of discussion recently about how content farms (websites that use low-cost labor to churn out pages of little or mediocre quality) are polluting the results on the major search engines. Blekko, a new search engine that enables users to slash the web, has been among those leading the charge by pledging to remove content farm sites from its results. Blekko reports that 1 million new spam pages, which may harm users, steal publisher traffic and defraud advertisers, are created every hour.
As of last Thursday, Surf Canyon now offers users the option of having these same content farm results automatically removed from their searches on Google, Bing and Yahoo!. Simply download the latest version of our browser extension and then either click the button here or go to the Domains tab at my.SurfCanyon.com, select the “Remove content farm sites from the search page” box and save your preference. It is that easy.
ProductivityPortfolio blogged about this new feature this morning. It is unclear how they discovered the capability since we are only announcing it right now, but good for them!
Note: The title of this post is from the Monty Python sketch.
[Update 2/16/11 - David Harry, with whom we spoke previously, references Mark Cramer and offers his take on the content farm situation in the latest issue of Search Engine Land.]
On December 31st we bid farewell to our spacious, wood-floored offices above a silk shop near downtown Oakland for a brand new set of digs at a co-working facility in San Francisco near 10th and Mission. The new space is excellent, the people are friendly and we’re excited about the next chapter in our journey.
Additionally, for the month of January, we welcome an intern from MIT: Connie Chan. She’ll be assisting with development before returning to Boston to continue her junior year of studies in Computer Science.
The Surf Canyon team, from left to right: Mike Wertheim, Mark Cramer, Connie Chan and Keith Flippin
“It is common for information retrieval research to focus either on relevance estimation or user interface design, but rarely both simultaneously. However, for many tasks, it can be useful to model both jointly… One major limitation of result diversification over static rankings is that it sacrifices recall in favor of some minimal amount of utility for all usage intents – such a limitation could be dealt with by moving towards more dynamic interfaces.
Consider the example interface shown in Figure 4.8, which is inspired by and adapted from the SurfCanyon.com search engine … by clicking or mousing over a result that matches the user’s intent, additional indented results are inserted into the original ranking… This interaction is quite natural, since the process resembles navigating a dropdown menu and since users are already familiar with result indentation. And yet even this one additional degree of freedom in content display can offer tremendous benefits…”
Lastly, the team at Cornell recently drafted a brilliant paper, entitled “Dynamic Ranked Retrieval,” which dives deep into the study of real-time implicit ranking and offers statistical support for the “tremendous benefits” described above. It has been accepted for publication at WSDM 2011. While not yet public (we’ll post here when it is), we’ve been given permission to offer a sneak preview from the introduction (emphasis added):
“… most queries are ambiguous at some level. For such queries, there is often no single ranking that satisfies all users and query intents. While result diversification aims to provide a “compromise ranking” that provides some utility for all intents, diversification necessarily sacrifices recall…
The key idea is to make the ranking “dynamic” – namely, allowing it to change in response to user interactions after the query was issued.
From the user’s perspective, this may look as illustrated in Figure 1. This interface is inspired by and adapted from the SurfCanyon.com search engine…”
Surf Canyon develops real-time personalized search. By transforming static lists of links into dynamic search pages that automatically re-rank results "on the fly," users are able to more quickly and easily find pertinent information buried among the irrelevant results, significantly accelerating the search process.