When doing something that has never been done before (as we do), it can be challenging to describe it using familiar terminology in a way that doesn’t create confusion while still conveying the newness of the idea. Large companies are sometimes capable of creating new terminology that is then adopted by others, but this can be particularly difficult for small entities. As such, for the sake of clarity, we describe how we have referred to our technology over the years and how we have now settled on “real-time contextualized search.”
When Surf Canyon launched its ground-breaking technology for dynamically re-ordering search results in response real time behavioral signals, we called our product a “Discovery Engine for Search“. The technology was referred to as “real-time implicit personalization” or simply “real-time personalization.” Unfortunately, this created a bit of confusion with some people in the search community:
- The term “real-time” is an often abused and misunderstood. Technically “real-time computing” refers to guaranteeing a response “within strict time constraints.” More generally it is used to refer to a system that responds very quickly. “Quickly,” however, is subject to interpretation which is why we then sometimes referred to our technology as “instant” or “immediate personalization.”
- Additionally, the term “personalization” may also lead to misunderstanding. In information retrieval “personalization” generally refers to collecting data about an individual over an extended period of time in order to generate models of that particular person’s long-term preferences in order to then use those models to modestly adjust relevance scores for future queries. Despite efforts to clarify the distinction with “real-time personalization” the term “personalization” can lead to premature interpretation.
In 2009 the team at Surf Canyon authored a paper entitled “Demonstration of Improved Search Result Relevancy Using Real-Time Implicit Relevance Feedback.” The paper was subsequently published by SIGIR after Professor Thorsten Joachims at Cornell University offered a glowing review. “Real-time implicit relevance feedback” is a mouthful but seems to alleviate misconceptions caused by the term “personalization.”
The next year, a team of researchers at Cornell, lead by Professor Joachims, published a paper called “Dynamic Ranked Retrieval” which built upon our SIGIR research by running tests using labeled results to compute relevance metrics. The results were not real-world, but they were very impressive and their paper was selected as one of the six best at WSDM 2011. We found “dynamic ranked retrieval” to be more punchy than “real-time implicit relevance feedback.”
Nevertheless, while “dynamic ranked retrieval” has its appeal, a recent post in Search Engine Watch regarding Yahoo!’s interests in search offered this:
Contextual search works by algorithmically trying to determine what you really mean to search for, such as picking up cues from the immediate preceding searches, and presenting results based on that. [Emphasis added]
Algorithmically determining user intent by observing user interactions (“cues”) is what Surf Canyon has been doing since the very beginning. Our contextual search, however, is taken one large and very important step further – rather than waiting for subsequent searches in order to exploit user behavior signals, our technology immediately re-orders the result set in response to every user action that imparts additional understanding of the at-the-moment intent. As such, we henceforth declare that we develop Real-time Contextual Search.
Tags: - Top Posts - · Contextualization · Discovery · Personalization · Recommendations
February 14th, 2014 · 1 Comment
Anyone familiar with Etsy knows that it is a fantastic website for finding handmade and vintage items, and a wonderful resource for gift-giving. Now, thanks to their search API and Surf Canyon’s dynamic ranking technology, there is an even better way to search through the millions of items for sale on Etsy. Click over to the Surf Canyon – Etsy Demo to try for yourself.
If you run a search for “gloves” on Etsy you’ll be presented with almost 70,000 results. There are facets on the left for drilling down, and of course users have the option of reformulating their queries to something more precise, but Surf Canyon has always been about alleviating the cognitive load by automatically and immediately assisting users with finding what they need.
As such, a query for “gloves” on the Surf Canyon – Etsy Demo will produce a page that looks like this:
As it so happens, tomorrow is Valentine’s Day, so perhaps the very first result fancies you, and so you select it. If your shopping is done, congratulations. It’s very rare indeed to find something to purchases with only a single click! More likely, however, you’ll return to the search page to check out more, which is when you’ll helpfully be provided with real-time recommendations from Surf Canyon’s dynamic ranking engine:
As you can see, these Valentine’s Day glove results are coming from pages 35, 37 and 41. It is hard to believe that anyone would ever dig that deep for a search result, but here they are helpful and automatically brought to page 1.
Every selection you make and every result you skip gives Surf Canyon’s dynamic search engine more information about your information need, enabling a superior ranking every step of the way. Clicking “More Results” on page 1 will produce more re-ranked results on page 1. Moving over to page 2 will the produce a second page of results tailored in real time to your needs.
In this particular example, you get more of what you want – gloves for Valentine’s Day:
The results are coming from pages 29, 23 and, somewhat amazingly, 88. Clicking a result on page 2 produces yet more re-ranked results, presented as “recommendations,” and the process continues.
Surf Canyon’s technology has been proven to deliver dramatic improvement in relevance, so give it a try!
Tags: Demonstration · Personalization · Recommendations
The College Humor video below is NSFW, contains some profanity and mild sexual situations, but we’re posting it here because not only is it amusing, but it is an interesting insight into the relationship between users and search engines.
People struggle to find the “magic” set of keywords to describe their information need. As we first mentioned back in 2007:
In their paper entitled “Beyond the Commons: Investigating the Value of Personalizing Web Search,” Teevan et al. state that, “Web queries are very short, and it is unlikely that a two- or three-word query can unambiguously describe a user’s informational goal.”
Simply put, it can be difficult for the user to accurately express what he or she is looking for with just a few words. Even with many words, depending on the type of query, it can be difficult to express what is ultimately desired from the results, regardless of whether the user is an expert in the domain or not. How many times in real life, speaking with real people, do people need to employ many words, over many sentences, to express a thought or need?
As a result, people will often ultimately end up fumbling and guessing with their search engine, which is humorously displayed in the video below.
Tags: Fun · Reformulation
To put together a demonstration of Surf Canyon’s search technology with Etsy’s products, we needed some Java code that would search Etsy and return a list of matching products. We assumed that this could probably be done in about 100 lines of Java code.
We also figured someone must have done this before, but after searching online the code that we found was much longer and a lot more complicated (mainly because it was trying to access parts of the Etsy API that required OAuth authentication) than what we had been expecting. However, just doing a product search doesn’t require any authentication, so we went ahead and wrote some much simpler code that did only what we needed.
The Java code, written by Mike Wertheim, is about 100 lines long and can be found at http://www.surfcanyon.com/EtsyListingFetcher.jsp.
The code makes use of Jackson (an open source Java library that handles JSON). To compile and run, download the two Jackson jar files (http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar and http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar) and put them in your Java CLASSPATH. Then copy the code from the web page (http://www.surfcanyon.com/EtsyListingFetcher.jsp) to the clipboard and paste it into a file called EtsyListingFetcher.java. (If you download the code directly, instead of doing copy and paste, you will probably run into problems with HTML entities.)
If you have any questions or feedback, please post them in the comments below or contact us.
Henry Feild in the Computer Science Department at Endicott College and James Allan in the Center for Intelligent Information Retrieval at the University of Massachusetts, Amherst, gave a quick mention to Surf Canyon in their paper entitled “Using CrowdLogger for In Situ Information Retrieval System Evaluation”:
… a popular tool called Surf Canyon modifies SERPs for major search engines by surfacing as-yet unseen search results from deeper in the rankings every time the user clicks on a result link (the goal is for the surfaced results to be similar to the clicked result).
Their paper discusses how CrowdLogger, an open-source browser extension for Firefox and Google Chrome, can be used as an in situ evaluation platform for “evaluating retrieval systems in the wild.” This is something Surf Canyon has been doing with its own retrieval system for many years.
In a previous post we mentioned the research on Dynamic Ranked Retrieval conducted by Professor Thorsten Joachims, Christina Brandt, Yisong Yue and Jacob Bank at Cornell University and that their paper was accepted for publication and then selected as one of the six Best Paper Candidates by WSDM 2011. While this is a bit belated, we are happy to have discovered the video of Professor Joachims presenting that paper.
From the 1m40s mark to the 3m20s mark a screen shot of Surf Canyon is used to demonstrate dynamic ranking in action. Professor Joachims then offers DCG analysis of a sample search senario and concludes, at the 5m50s, that, “by being dynamic, and adaptive, you can gain a lot of retrieval performance.” Surf Canyon is then mentioned at the 12m00s mark as an Interactive Information Retrieval Model. The Adaptivity Gain, defined previously as the increase in retrieval performance offered by dynamic ranking over traditional static ranking, calculated from empirical studies done on two collections of TREC queries labeled for multiple intents, is then presented at the 14m10s mark and described as “quite substantial” with NDCG going from 55% to 70%.
“When you think about how much effort search engines are spending to get a 1% improvement in NDCG, this is a lot and could potentially change the upper bound of how good you can get with a single ranking.” – Professor Thorsten Joachims
Tags: - Top Posts - · Presentations · Research
The 2013 ACM International Conference on Information and Knowledge Management (CIKM) is one of the major forums for research on database management, information retrieval, and knowledge management. As such, we are proud to announce that our latest research paper, entitled “Understanding How People Interact with Web Search Results that Change in Real-Time Using Implicit Feedback,” co-authored with Jin Young Kim from Bing, Jaime Teevan from Microsoft Research and Dmitry Lagun from Emory University, has been accepted by the conference. Only 106 papers out of 848 submissions were selected, so the conference was very competitive.
The data once again demonstrates the extent to which dynamically ranking search results in response to real-time user feedback dramatically improves relevance:
Compared to traditional web search, users presented with dynamically ranked results exhibit higher engagement and find information faster, particularly during exploratory tasks. These findings have implications for how search engines might best exploit implicit feedback in real-time in order to help users identify the most relevant results as quickly as possible.
Enjoy the paper and if you attend the CIKM 2013 conference in San Francisco we look forward to seeing you there.
[Update - 2013-8-15] The paper may also be found on Jaime Teevan’s website at Microsoft Research.
[Update - 2013-9-9] We will be presenting during the Short Paper Session 52 – IR Track with is from 4-6:15pm on Thursday, October 31st.
Tags: - Top Posts - · Announcements · Collaboration · Research
In celebration of the third patent issued to Surf Canyon by the USPTO, we designed some t-shirts! They arrived today. If you requested one, you will be receiving it shortly.
Team Surf Canyon models its latest designer wear
Tags: Announcements · Fun
April 30th, 2013 · 1 Comment
We are delighted to announce that on Friday the USPTO awarded Surf Canyon its third patent, U.S. Patent No. 8,442,973, to be issued on May 14th, for “Real Time Implicit User Modeling for Personalized Search.” This is an important piece of intellectual property, as evidenced by the fact it has been referenced over 25 times by other patents issued to companies such as Google, Microsoft, Yahoo!, IBM and Samsung. The Abstract of the invention is as follows:
A method and apparatus for utilizing user behavior to immediately modify sets of search results so that the most relevant documents are moved to the top. In one embodiment of the invention, behavior data, which can come from virtually any activity, is used to infer the user’s intent. The updated inferred implicit user model is then exploited immediately by re-ranking the set of matched documents to best reflect the information need of the user. The system updates the user model and immediately re-ranks documents at every opportunity in order to constantly provide the most optimal results. In another embodiment, the system determines, based on the similarity of results sets, if the current query belongs in the same information session as one or more previous queries. If so, the current query is expanded with additional keywords in order to improve the targeting of the results.
Along with U.S. Patent No. 8,095,582 for “Dynamic Search Engine Results Employing User Behavior” and U.S. Patent No. 8,117,197 for “Adaptive User Interface for Real-Time Search Relevance Feedback,” the new patent rounds out Surf Canyon’s portfolio concerning the real-time re-ranking of search results in response to implicit user feedback.
To commemorate this momentous occasion, we have designed and will be giving away a special, universally loved item of tchotchke: t-shirts! You may check out our design and, should you like one for yourself, submit a request on the sign-up sheet. If you leave your address, we’ll do our best to get you one.
Tags: - Top Posts - · Announcements · Research
This paper, entitled “Personalizing Web Search using Long Term Browsing History,” was submitted to WSDM in 2011, but only recently came to our attention. The authors, Nicolaas Matthijs from the University of Cambridge and Filip Radlinski from Microsoft, write about “a user interest
profile using users’ complete browsing behavior,” but we thank them for taking a moment to recognize Surf Canyon’s contribution to the field:
Recently, personalized search has also been made available in some mainstream web search engines including Google and Yahoo!. These appear to use a combination of explicitly and implicitly collected information about the user. Many more companies are engaging in personalization both for search (e.g. surfcanyon.com) and for advertising based on user behavior.
Tags: Personalization · Research