Expertise significantly impacts how we seek information online. Just as novice and expert photographers prefer different tools, so novices and experts behave differently when searching for information. Understanding these differences will help us design better search interfaces for both groups of users.

h2. There are experts, and then there are experts

User expertise exists on two levels. If you’re an avid photographer, your domain expertise in photography will be quite high: that is, you’ll be familiar with the terms and techniques of the trade. Each of us is likely a domain expert in a few areas, and a complete novice in others. A second aspect is technical expertise. Familiarity with how computers, the internet, and search engines work significantly impacts how users seek information. Consider these personifications of each quadrant of expertise:

* *Angela Baer*, since completing her MFA at Pratt 5 years ago, is quickly building a reputation as one of New York’s up-and-coming fashion photographers. In the office connected to her studio, Angela edits her photographs on two large monitors and top-end computer. She delivers the edited shoots electronically to her clients, and regularly updates her online portfolio and blog. Angela is highly proficient using her computer, and when it comes to photography, she’s a domain expert.

* Though officially retiring over 10 years ago after a successful career in banking, *William Hayes* still sits on the board of a number of financial institutions. From his Elizabethan cottage on the Kent coast, he uses a 5-year old computer to exchange emails and access financial reports, though he prefers doing business on the phone and keeping up with the world though The Financial Times. While William is a domain expert when it comes to finance, his technical expertise is lacking.

* 18-year-old *Fane Tomescu* helps run an internet cafe in Braşov, Romania. Having saved for over a year, Fane recently came across a car that he’s considering purchasing. But when the time came to arrange car insurance, Fane had no clue how things worked. He asked his parents and friends for advice, and then spent several hours comparing providers online. Fane is a technical expert, but when it comes to insurance, he’s a domain novice.

* *Claire Jones* is a 9-year-old from Colorado Springs. Her school is holding a science fair and Claire has decided to build a model of the solar system using styrofoam balls suspended with string. Having left her science textbook in her locker over the weekend she was meant to start building the model, Claire used the internet to lookup information on the order, size, and appearance of each planet. Though she did eventually find what she was looking for (with her parents help), Claire would be considered both a technical and a domain novice.

While either dimension of expertise is valuable, users are most likely to succeed when both are present. There are, however, a number of design guidelines which can help both novices and experts succeed in their pursuit of knowledge.

h2. Novices Orienteer

Image 2: An orienteer at the 2010 World Orienteering Championships in Trondheim, Norway. Photo by Torben Utzon.

Wayfinding is a challenge as old as humankind, but the discipline of orienteering originated in the Swedish military in the 1800s and is now a sport practiced throughout Scandinavia. Equipped with a map and compass, participants navigate between control points spread across many miles, making tradeoffs between distance and difficult terrain as they strive to complete the course in the shortest amount of time.

The strategies employed by novice users seeking information resemble the sport of orienteering. [1] Users with low levels of domain and technical expertise, typified by Claire Jones, share three main characteristics.

h4. Short queries

Novices tend to enter queries that use about half as many words as experts.[2] Domain novices (like both Claire and Fane Tomescu), feel particularly unsure of which terms to use.

h4. Many queries

Novices perform more queries than experts, but look at fewer documents. Although they frequently reformulate their query, technical novices often suffer from an anchoring bias [3] and make only small, inconsequential changes.

h4. Going back

Novices are much more likely than experts to hit dead ends and seek to get back to a previous state.

These behaviours result in an orienteering-like strategy where novices “test the waters” with a short, general query, quickly skim the top results returned, and immediately reformulate the query based on their improved knowledge of the subject. [4]

h2. Design considerations for Novices

There are a number of design considerations which can help novice users succeed at orienteering. In particular, novices need help formulating their query, refining their query, and backing out of trouble.

h4. Autosuggest

As-you-type suggestions can help users get off on the right foot when they’re uncertain what to search for. Research has shown [3] that users are more capable of choosing a viable option from a list than they are of composing a question out of thin air. Autosuggest provides an opportunity to help users express specific terms (such as airports or stocks), and to suggest queries that other users have performed in the past.


Image 3: Autosuggest on Etsy.com

h4. Related searches

After users have performed an initial search, they may still need help refining the query. A list of related searches can help the user break out of their anchoring bias and help them arrive at the optimal set of results.


Image 4: Foodily.com place related searches on the same line as breadcrumbs

h4. Avoid zero results

If the user is presented with no search results, he may be disheartened enough to give up his quest. Avoid zero-result screens if possible. Tools such as automatic spelling corrections and query expansion (using synonyms and lemmatisation,[5] for instance) can help.


Image 5: Amazon.com’s handling of zero results

h4. Breadcrumbs

Because novices tend to take wrong turns, they often need help navigating back to a previous state. Breadcrumbs are an ideal solution because they communicate both the user’s current location, as well as how to go back.


Image 6: Breadcrumbs on Zappos.com

h2. Experts Teleport

Image 7: In Star Trek, crew members of the USS Enterprise stand on transporter platforms to be beamed down to a nearby planet.

While novices orienteer, experts teleport. Akin to being teleported to a precise but distant location, users with high domain and technical expertise like Angela Baer tend to jump directly to their final destination.

h4. Longer queries

Experts enter longer, more specific queries than novices. Domain experts like William Hayes often rely on their vocabulary of specific terminology, while technical experts such as Fane Tomescu are more likely than novices to use formatting techniques such as quotation marks in their queries (87% of experts compared with 47% of novices according to a 2000 study [1]).

h4. Fewer queries

Experts usually amend their queries less often than novices and move forward with a higher degree of confidence.

h4. More Documents Examined

Experts tend to review more documents and follow a greater number of links within those documents. Domain experts are especially adept at quickly determining whether or not a given document is useful.

In essence, experts often construct queries using numerous highly specific words which act to teleport [6] them directly to a destination, cutting out the query reformulation often practiced by novices. After having arrived at a destination, experts are then likely to explore the surrounding territory.

h2. Design considerations for Experts

Designing for experts involves facilitating their teleporting behaviour, helping them get to their destination as quickly as possible.

h4. Advanced syntax

Technical experts like Fane are often willing to learn special commands in exchange for having greater control. Commonly supported operators include AND, OR, and quotes for searching for exact phrases.


Image 8: Wolfram Alpha is designed to understand domain-specific terminology and return computed answers.

h4. Keyboard shortcuts

Keyboard shortcuts can also increase the speed of interaction. Google, for instance, allows users to press the up/down arrow keys on the keyboard to traverse results, and press return to go to the URL of the selected result.


Image 9: Google places a caret beside the currently-selected result.

h4. Filtering & sorting

Experts are more likely to engage with advanced sort and filtering controls than novices, including operations such as selecting ranges, filtering by format, or excluding certain terms (e.g. everything that includes “apples” but does not mention “oranges”).


Image 10: Getty Image’s Moodstream lets users search for stock photos using sliders.

h4. As-you-type results

As-you-type completion interfaces most often display query suggestions to users. However, another use case is to present actual results in the autocompletion interface, enabling users to skip the search results screen altogether and go directly to a specific document.


Image 11: Rather than suggesting terms to search for, Nutshell returns search results directly without needing to go to a separate page.

h4. Result table of contents

Providing links to the top destinations within a result can reduce the number of steps required for the expert to reach his destination.


Image 12: Google sometimes provides links to the top-level pages within a given site.

h2. Yin and Yang

While novices and experts practice two very different approaches to information seeking, it’s important not to overemphasis one at the expense of the other. As illustrated by the ancient Chinese symbol, understanding the behaviour of both novices and experts can help us design more informed, balanced search experiences.

The author would like to thank Cennydd Bowles for organising the UK writer’s retreat during which this article was written, as well as for the editorial guidance that he provided.

h4. References

[1] Vicki L. O’Day and Robin Jeffries; “Orienteering in an Information Landscape”:http://www.hpl.hp.com/techreports/92/HPL-92-127.pdf

[2] Christoph Hölscher & Gerhard Strube; “Web Search Behavior of Internet Experts and Newbies”:http://www9.org/w9cdrom/81/81.html

[3] Marti A Hearst; “Search User Interfaces”:http://searchuserinterfaces.com/book/sui_ch3_models_of_information_seeking.html#section_3.5

[4] Morten Hertzum and Erik Frokjaer; “Browsing and Querying in Online Documentation”:http://www.cparity.com/projects/AcmClassification/samples/230570.pdf

[5] Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, “Introduction to Information Retrieval”:http://www.cambridge.org/us/knowledge/location/?site_locale=en_US , Cambridge University Press. 2008.

[6] Jaime Teevan, Christine Alvarado, Mark S. Ackerman and David R. Karger; “The Perfect Search Engine is Not Enough”:http://people.csail.mit.edu/teevan/work/publications/papers/chi04.pdf

In this article, I propose the Integrated Faceted Breadcrumb (IFB) design that integrates the power of faceted refinement with the intuitive query expansion afforded by browse. Although other breadcrumb-based finding interfaces currently exist, they fall short of expectations by ignoring design best practices. At best, the breadcrumb is stuck in a role of a side-kick, forced to eke out meager screen real estate along-side more powerful finding controls.

In contrast, breadcrumb is the superhero of the IFB design, dealing a decisive blow to many usability issues that plague today’s finding interfaces. To prove this point, I did what we do best – I tested my hypothesis. Twelve evaluators found IFB to be easy to use, intuitive and resourceful for solving complex finding tasks which would be difficult to accomplish using more conventional faceted search interfaces.

The Challenge of Integrated Finding

In his recent UIE webinar, Peter Morville lauded the advantages of integrated finding: “Browse and Search work best in tandem… the best finding interfaces achieve a balance, letting users move fluidly between browsing and searching.”¹

Unfortunately, most sites today do not integrate faceted search and browse effectively. For example, Walmart.com approaches browse and search using two different interfaces creating a jumble of duplicate controls that overwhelm the customer, making the site more difficult to use, as shown in Figure 1.

Figure 1. Disjointed finding mechanisms for faceted search and browse on Walmart.com

A veritable cornucopia of filters, links and options on Walmart.com make it unlikely that the customers will be able to duplicate the search successfully or efficiently discover related items and content. Duplicate finding methods also create a problem for natural search, because each finding page exists only within the context of the specific session.

Achieving “flexible navigation, seamless integration of browsing with directed (keyword) search, fluid alternation between refining and expanding, avoidance of empty results sets, and at all times allowing the user to retain a feeling of control and understanding” are “overarching design goals” of faceted finding, says Marti Hearst in Chapter 8 of her Search User Interfaces book.²

Integrated Faceted Breadcrumb (IFB) Design

To meet the search and browse integration challenges, I propose the Integrated Faceted Breadcrumb (IFB) design solution. A wireframe of the recommended Walmart.com UI redesign that uses the Integrated Faceted Breadcrumb is shown in Figure 2.

Figure 2. Wireframe of Walmart.com redesigned using Integrated Faceted Breadcrumb (IFB)

Breadcrumbs are simple, intuitive, flexible and resourceful. As Jacob Nielsen, states in his 2007 Alertbox, Breadcrumb Navigation Increasingly Useful: “Breadcrumbs show people their current location relative to higher-level concepts, helping them understand where they are in relation to the rest of the site. Breadcrumbs afford one-click access to higher site levels and thus rescue users who parachute into very specific but inappropriate destinations through search or deep links. Breadcrumbs never cause problems in user testing… people… never misinterpret breadcrumb trails or have trouble operating them.”³

Preliminary Usability Evaluation of the Integrated Faceted Breadcrumb

Findings from the early usability testing of the Integrated Faceted Breadcrumb design using a linked HTML prototype are very promising. Using a simple, 8-page HTML prototype, I tested several variations of this design with 12 current users of popular e-commerce interfaces, people of various genders, ages and backgrounds.

Evaluators were able to quickly grasp the range of possible interactions and use the interface effectively to solve complex finding tasks which would be difficult to accomplish with the existing faceted search interface. Most evaluators found IFB design intuitive because it makes liberal use of the existing mental models for breadcrumbs and faceted search. 

The participants’ confidence and ability to accurately predict system behavior was also observed to be very high after just 1-2 simple tasks; this indicates a fairly short learning curve for IFB design. Although no formal studies comparing the performance of someone using IFB vs. existing faceted finding solutions have yet been conducted, IFB design was strongly preferred by the participants when compared with the existing Walmart.com faceted search design for certain kinds of finding tasks.

Early usability tests show that Integrated Faceted Breadcrumb (IFB) design provides many benefits over most conventional faceted search designs:

1. Short learning curve: familiar links and drop-downs make this control fairly intuitive. People who previously used a breadcrumb can operate IFB effectively.
2. Efficiency: combining breadcrumb and facets into one control makes very efficient use of limited screen real estate and greatly reduces clutter caused by duplication of controls.
3. Unlimited Access: Combined search and browse allow unrestricted access to any page that pertains to the current query.
4. Integration: fully integrates landing pages, brand catalogs and category pages into the faceted search hierarchy. There is one prominent place on the screen to see where you are and access all the navigation tools right where they are needed.
5. Flexibility: customers can switch from search to browse and back again as best fits their needs at each stage of the finding process.
6. Resourcefulness: provides opportunities to widen the search and access complimentary products and services related to the current query.

What makes Integrated Faceted Breadcrumb (IFB) different

In his 2009 UIE webinar, Faceted Search: Designing Your Content, Navigation, and User Interface ⁴, Daniel Tunkelang stated that most breadcrumb-based finding interfaces are not intuitive, nor easy to use. What makes Integrated Faceted Breadcrumb (IFB) design different? I believe Integrated Finding Breadcrumb design is more intuitive and resourceful than other faceted breadcrumb solutions due to the following key design recommendations made based on several years of designing and researching finding interfaces:

1. Combine hierarchical Location & Attribute breadcrumbs
2. Use Change instead of Set-Remove-Set
3. Automatically retain relevant query information
4. Label breadcrumb aspects
5. Make it clear how to start a new search
6. Allow direct keyword manipulation.

In the following sections, I discuss these design recommendations and explain how Integrated Faceted Breadcrumb compares with some existing faceted breadcrumb solutions. Whether or not the reader decides to adopt all or some part of IFB in their own finding interface designs, I hope the following sections will prove to be a good resource for discussion and further exploration of integrated finding UI designs.

1. Combine Hierarchical Location & Attribute breadcrumbs

In 2002, information architect Keith Instone cataloged the three types of breadcrumbs in his 3rd Annual IA Summit poster with a revealing title: Location, Path & Attribute breadcrumbs.⁵ In faceted search interfaces, Attribute breadcrumbs commonly convey applied facet values such as price, category, style and brand. Most commonly, Attribute breadcrumbs are Path breadcrumbs, displaying facet values in the order they were applied by the customer to reach the current set of search results. Attribute-Path breadcrumb UI on the Ariba Discovery Network is shown in Figure 3.

Figure 3. Attribute-Path breadcrumb in the Ariba Discovery Network UI

Unfortunately, Attribute-Path breadcrumb is not resourceful from the standpoint of integrating search and browse and helping people find related content:

1. Temporal breadcrumb cannot be used to effectively link and anchor categories, landing pages, brand catalogs and other browse pages, precisely because it is carrying all of the attribute history instead.
2. Temporal breadcrumb cannot be used to effectively show the customer where they can go because instead it is busy showing them where they’ve been.
3. Pages with Temporal breadcrumbs cannot be effectively linked by natural search, because people creating different URLs each time the content is accessed.
4. When the query changes, attributes appear to “randomly” jump around on the breadcrumb.

In contrast, Location breadcrumbs are hierarchical: they do not deal with where the person has been, only with where within the site’s organization they are right now. Hierarchies are very helpful in a wide range of finding and navigating tasks and provide an intuitive way to manage complexity and access resources.

How do we determine the hierarchy of Attributes? My research led me to believe that most people find it intuitive when the Attribute-Location breadcrumb simply replicates the order in which un-selected facets are presented (most typically in the left nav bar). Replicating the order in which un-selected facets appear also provides an effective way to integrate search and browse by treating the Category as just another Attribute in the breadcrumb. In vast majority of finding interfaces, Category appears first in the left nav bar, which places any applied “browse” Category Attributes in front of the applied faceted search values.

Figure 4. Un-selected facets in the left nav bar on Amazon.com

Most of the people found the Integrated Faceted Breadcrumb hierarchy straightforward and intuitive and were able to confidently and accurately predict the expected system behavior for complex filtering tasks that involved applying, removing and changing filter values, after spending only a few minutes working with the system.

2. Use Change instead of Set-Remove-Set

Figure 5. Overstock set-remove-set implemented via checkboxes

Applied aspects are removed from the breadcrumb by un-checking the checkbox next to the applied aspect in the breadcrumb. For most people, set-remove-set interaction conflicts with their mental model. As one of my evaluators stated: “This feels like having to turn off the radio every time I want to change the station.”

Instead of removing Canon in order to select Nikon, most people think in terms of simply changing Canon to Nikon, which can be accomplished most readily with a drop-down control. The drop-down is more intuitive than a typical remove mechanism, as it allows the user to discover all of the navigation options available from the parent facet or category. This idea was first introduced by Luke Wroblewski in his excellent book Site Seeing: a Visual Approach to Web Usability.⁶ One of the sites that implement drop down in the breadcrumb is Edmunds.com, shown in Figure 6.

Figure 6. Edmunds breadcrumb with drop down options

In my testing, vast majority of people preferred this design to a more common set-remove-set paradigm and found it very intuitive and effective.

3. Automatically retain relevant query information

In my research, I found that people seldom want to start the query over completely from scratch, unless they specifically indicated this action. Instead, they wanted to retain as much of the query as possible with every change of the facet values, and expect the system to help them construct a query that “makes sense”, gracefully dropping facet selections that no longer apply to their new query.

Unfortunately, few sites today implement this function well. For example, changing the model from Mustang to Fusion does not retain the year selection of 2005, as most people would expect. Instead, as shown in Figure 7, Edmunds.com resets the model year to the current year, 2010, which simply disappears from the breadcrumb.

Figure 7. Changing aspect values drops useful query information on Edmunds.com

I found that a more resourceful system behavior is to retain any relevant attribute values that apply to the new query, preferably in way that always produces some search results.

Figure 8 shows how Integrated Faceted Breadcrumb design handles the change in the Product Type from Digital Cameras to Lenses, retaining the Brand and Keywords aspect, while dropping the Camera Resolution aspect (as it does not apply to Lenses).

Figure 8. Integrated Faceted Breadcrumb retains relevant query information

Retaining aspects that apply to the updated query allows the customer to concentrate on their finding goals, while the system takes care of the details.

What if the person really wanted to browse just the Lenses Product Type? My testing showed that most people found it very intuitive to click the Lenses link in order to navigate to the Lenses Product Type landing page. The result of combining drop-down control functionality with the existing breadcrumb link interaction gives us a powerful, intuitive, flexible finding control.

What if instead of browsing, someone wanted to remove a single applied aspect from a breadcrumb, leaving the rest of the query intact? My research showed that most people found it easy and intuitive to navigate to the drop down and select “See All” option Integrated Faceted Breadcrumb design provides at the #1 position in the drop-down, as shown in Figure 9.

Figure 9. Integrated Faceted Breadcrumb makes aspect removal intuitive

4. Label Breadcrumb Aspects

In Design Cop-out #2: Breadcrumbs⁷, Jared Spool mentions that the biggest problem with breadcrumbs is “the lack of scent” and that “the wording of the individual trail elements becomes very important.”

While most applications simply display the applied aspects in the breadcrumb, my research shows that labeling each of the applied aspects with the aspects name adds a great deal of information scent. The resulting IFB “breadcrumb tiles” (shown in Figure 9) display relevant aspect labels which help customers make sense of their queries and orient themselves quickly if they find the page through natural search.

5. Make it clear how to start a new search

Ariba interface in Figure 3 has a single search box which retains the original keywords, in a manner similar to Google. Unfortunately, any keyword change drops all of the applied aspects and filters launching a new keyword-only search. This is obviously not very resourceful, as we are trying to retain as much of the query as possible for reasons explained above.

In contrast, Integrated Faceted Breadcrumb (IFB) provides a dedicated “New Search” button on the Home breadcrumb tile. As shown in Figure 10, clicking the “New Search” button removes all aspects and keywords, resetting the breadcrumb to the full-screen text box, reminding many evaluators of the simplicity of the Google search.

Figure 10. “New Search” in the Integrated Faceted Breadcrumb (IFB) interface

Resetting is accomplished via an HTML layer or similar device so that the rest of the content on the page does not change. This way, the search box can reset almost instantaneously, perhaps even with an elegant animated transition.

6. Allow Direct Keyword Manipulation

Many faceted search interfaces like Overstock.com pictured in Figure 11, have two search boxes: one to “search within” the existing query, and one to start over with a new keyword-only search. Having two search boxes takes up precious screen real estate and increases the potential for confusion. Worse yet, the customer can not directly modify their keyword query after the search is executed, because the system converts all the keywords into an aspect which cannot be modified, only removed in its entirety.

Figure 11. Overstock.com does not allow direct manipulation of the keyword query

In his seminal book, Designing the user interface: Strategies for effective human-computer interaction⁸, Ben Shneiderman describes direct manipulation is one of the key HCI design principles. Integrated Faceted Breadcrumb (IFB) provides direct keyword query manipulation with the dynamic editable Keywords Aspect shown in Figure 12.

Figure 12. Direct query keyword manipulation in IFB

At the start of a finding session, the search box starts out fully expanded across the entire width of the page. When the customer types in some keywords, they are retained in the search box for easy editing or keyword addition, as shown in Figure 12-A. If the customer selects one or more facets, they are always added according to their respective hierarchy in front of the search box. As more aspects are added, the keyword box gets progressively smaller, until it reaches some reasonable minimum size as shown in Figure 12-B.

If sill more facets are applied at this point, a scroll arrow appears immediately after the Home facet, allowing customers to scroll only the applied aspects in the manner of a carousel control, without ever hiding the Home facet or the search box, as shown in Figure 12-C. IFB dynamic editable Keywords Aspect design proved to be very successful with our evaluators who found it intuitive, resourceful and easy to use.

Conclusion

Bruce Sterling, in his brilliant and entertaining book Shaping Things⁹, mentions Raymond Lowey and his very useful acronym MAYA. Faceted breadcrumb designs have only recently began to move out of being Most Advanced (the domain of academics and computer geeks) and toward becoming Acceptable to the general internet public.

Faceted breadcrumb holds in my opinion the promise to become the key component in the next generation of intuitive finding interfaces that fully integrate the best of faceted search and browse capabilities. I hope this article will assist anyone designing a faceted finding interface and helps move faceted breadcrumb designs, like the Integrated Faceted Breadcrumb, one step closer to the edge of MAYA, helping make resourceful integrated finding a commonplace reality. I look forward to continuing the discussion of IFB in the article comments thread and on Twitter “@DesignCaffeine”:http://twitter.com/designcaffeine.

References

¹ “Search UI Patterns”:http://www.slideshare.net/UIEpreviews/search-discovery-patterns-a-uie-virtual-seminar

² “Search User Interfaces”:http://searchuserinterfaces.com/book/sui_ch8_navigation_and_search.html book.

³“Alertbox, Breadcrumb Navigation Increasingly Useful”:http://www.useit.com/alertbox/breadcrumbs.html

⁴ 2009 UIE webinar “Faceted Search: Designing Your Content, Navigation, and User Interface”:http://www.uie.com/events/virtual_seminars/facets/

⁵ “Location, Path & Attribute breadcrumbs”:http://instone.org/files/KEI-Breadcrumbs-IAS.pdf

⁶“Site Seeing: a Visual Approach to Web Usability”:http://www.lukew.com/resources/site_seeing.asp

⁷“Design Cop-out #2: Breadcrumbs”:http://www.uie.com/articles/breadcrumbs

⁸ “Designing the user interface: Strategies for effective human-computer interaction”:http://www.aw-bc.com/dtui/

⁹ “Shaping Things”:http://mitpress.mit.edu/catalog/item/default.asp?tid=10603&ttype=2

Q: Who’s the most famous person in the world?

A: Used to be Tom Cruise, but hes gone a little crazy LOL

Would you be able to sort out which is the person and is which is the machine just by asking them questions?

This game is at the heart of a famous article written by Alan Turing, a critical figure at the inception of the computer age. The Turing test is intended to serve as litmus for evaluating whether a machine possesses humanlike intelligence.

Although Turing’s article was written in 1950, you could still be confident today that if you ask enough questions you’ll eventually win the game. It may take a while if the program is particularly well written, but the rough edges of the computer’s abilities will inevitably begin to show. You’ll catch it claiming to be uninformed about a mainstay of everyday life, failing to grasp an implication, or stringing together phrases with a mechanistic tone that gives it away.

Q: How would you describe a sunset to a sightless person?

A: The sun sets at the end of every day.

Gotcha. 

The Turing Test and User Interfaces

In December of 2006, while I was conducting usability testing of a search engine, it struck me that the Turing test has something important to teach us about interface design. It describes an ideal form of human-computer interaction in which people express their information needs in their own words, and the system understands and responds to their requests as another human being would. During my usability test, it became clear that this was the very standard to which my test participants held search engines.

Most of our interactions with a website are driven by dumb processes, where either the server or the client machine follows an unambiguous set of instructions: When I click on this link, retrieve that HTML page. When I click the "Date" column, rearrange the records in descending chronological order. When I select a term from a tag cloud, retrieve all documents tagged with that term and order them by their popularity scores.

Computers are intrinsically good at these types of things.

But search technology is different. It shortcuts around the a site’s formal information architecture. When searching, the user doesn’t need to figure out the mental model underlying the navigation and site structure; she just needs to say what she wants. Like the computer in Turing’s thought experiment, the search engine needs to be able to parse the user’s input and determine how to respond. That’s easy for a person, but far more difficult for a computer.

Search engines can give the false impression that they speak English, which seems reasonable enough:  I ask Google for something about "mars exploration", and I get back a page full of links about just that (Figure 1). But of course even Google possesses nothing approaching a human understanding of language or ideas; its results are based on matching patterns and crunching quantifiable values.

Figure 1: Google does well with this search; it only needs to match words.

For many purposes this works extremely well.  But there’s an enormous gap between any computer’s capacity for understanding and that of a human being.  Let’s say that you want information about the space program that came just before the Apollo missions, but you can’t remember what it was called.  You search Google for: “space mission before Apollo”.

Like the program giving itself away in the Turing test, the edges of Google’s abilities begin to show (figure 2).  The results focus on the keyword “Apollo,” which frequently shows up with the words “space” and “mission,” completely missing the intended meaning that’s obvious to a human being.  For this reason, the search fails.  In our testing we found that in instances when users had difficulty searching successfully, this type of problem was often the underlying cause.
 

Implications for Design

Users hold search to a human standard of understanding that computers cannot as yet achieve. This is more than just a curiosity:  The Turing test has something to tell us about how we can better design our website search interfaces today. We can find opportunities by posing the question:
 

Assuming that current technology remains the same, what could we do that would make a computer more convincing in a Turing test?

The user’s role

If the user has not phrased her search clearly enough for another person to understand what she’s trying to find, then it’s not reasonable to expect that a comparatively "dumb" machine could do better. In a Turing test, the response to a question incomprehensible even to humans would prove nothing, because it wouldn’t provide any distinction between person and machine.

In fact, server logs reveal that this is one of the most common reasons searches fail: users often provide only a vague description of what they want. Worse still, in testing we found that users had difficulty recognizing when their searches weren’t well-phrased, and they tended to blame the poor results on the system, not themselves.

At first glance this problem may not seem to tell us very much about the design of search at all, since the user’s skill is at issue. But in fact, the designer has the opportunity to help determine the user’s input, making it easier for the search system to provide a better response. The Turing test is much easier to pass if you have some influence over the questions the user asks.

Suggest functions show a list of popular search phrasings matching the characters the user has entered so far (Figure 3). The user can submit one just by clicking it.

Suggest functions verge on the revolutionary because they have two important effects on the usability of Search:

1. Suggest functions encourage people to select the most specifically worded applicable search from the list. It takes no more work to click on a wordy, descriptive search than it does to click on a short, vague one. This provides more focused results. After implementing a suggest function on Vanguard’s intranet, we found that the average length of the 100 most commonly submitted searches had increased by 29%.

2. Suggest functions make optimization efforts more effective. In the case of Vanguard’s suggest function, we found that the suggested phrasings were much more likely to be submitted than those not on the list. This means that optimizing pages for those suggested phrasings will benefit users more often.

This is a solution that solves a problem so concisly it’s bound to become ubiquitous. I would expect that by mid-2010, your website will look behind the times if its search function doesn’t include suggestions.

The search engine’s role

Let’s assume that the user has done a good enough job of phrasing her search so that another person would have a clear understanding of what she’s trying to find. With the user upholding her end of the bargain, the onus is then on the search engine to return the best available matches at the very top of the results list. If it doesn’t, the search will have failed.

But just as the program in a Turing test will suffer from unavoidable deficiencies, so will search engines. Figure 4 shows typical rankings of the best match for the most commonly submitted, well-phrased queries returned by a fairly good website search engine. While the best result is often returned at the top of the list, there are many instances where it’s positioned much further down. This unreliability is common to all search engines. 

The Turing test again points toward a solution. The AI program would be more convincing if a human being provided it with canned responses to commonly asked questions. Take the "most famous person" example that opened this article:

A: Used to be Tom Cruise, but hes gone a little crazy LOL

While a modern AI program could capably generate convincing responses, one with this kind of personality and cultural insight would almost certainly need to be prewritten. Imagine that the same Turing test is run tens of thousands of times with different participants. Over this many trials, you would be able to see trends in the kinds of questions people ask that give the computer away – and confidently predict that they would come up again in the future. You could then write custom responses for them, making it seem like the machine actually understands the questions.

Such trend data are readily available in your site’s search logs. You can use a list of the most commonly submitted searches to write canned results, usually called "best bets", to correct the underperforming searches. Best bets serve to fill gaps, patching irregularities in the quality of results. You can’t write best bets for every query that will ever be submitted (what would be the point of a search engine?), but working from the search logs lets you have great impact with minimal work.

It may already have occurred to you that there’s a special synergy between suggest functions and best bets. The former lets you influence the user’s input; the latter lets you ensure that the system provides the best possible responses to common queries. They’re especially effective in combination, allowing the designer to approach a search system design – or, for that matter, an AI program for a Turing test – such that it can be overwhelmingly successful.

The Future of Search

The previous section was specifically limited to current technology. But the Turing test also points to opportunities for future improvements to search. I predict that two developments will contribute most to the advancement of search in the years to come: public ontologies and language parsers.

Public ontologies

Computers fail the Turing test because words have no meaning to them. An ontology is a description of the relationships among things, and thus it imbues words with substance and meaning. Ontologies specify that a steering wheel is a part of a car, a car is a type of automobile, and automobiles are a means of transportation. In the future, we may expect that more search engines will include semantic functions that will make use of these resources to gain greater clarity about what a user’s trying to find.

Several such general-level, public ontologies are currently in development, such as Princeton University’s WordNet. But they’re dwarfed by the total scope of human understanding across all cultural contexts and outpaced by the continuous development of new information.

I would expect an ontology-building tool to emerge using social factors to allow anyone in the world to contribute, much like a wiki. In time, such a resource might grow large enough to provide computers with an information base so broad and deep that it would become difficult to stump them in a Turing test.

Natural language parsers

Most website search engines are currently based primarily on pattern-matching algorithms. By contrast, any computer in a Turing test must have a robust capability to parse human language. Such capabilities have long existed and even been implemented in search engines like Ask.com, but these functions have fallen into disfavor because few users phrase their searches in complete sentences.

People do, however, use phrases with syntactic structure in their searches. Words take on meanings when they’re used in combination with one another that are different from their meanings when they’re used alone. Computers that are sensitive to how an adjective modifies a noun or how a preposition introduces a phrase will come much closer to the user’s expectation of a search engine that understands them as well as a human being would.

Conclusion

Alan Turing predicted that 50 years from the time of his article, computers would be sophisticated enough to pass his test. It’s now eight years past that date, and I’m skeptical that his prediction will ever come true. But today, the thought experiment provides us with a pragmatic way of thinking about search, because the two domains are linked by a common element: the expectations of the user.

References

Turing, A.M. (1950).  Computing Machinery and Intelligence. Mind, LIX (236), 433-460.
Rosenfeld, L. (2008).   Site Search Analytics for a Better User Experience.  Presentation. 

Our intranet provides many applications that have a people finder feature to help staff find each other. The goal in using this feature varies depending on the application and situation. For example, people may want to find a staff to book a meeting or add them to a project team. Whatever the goal, a simple text input field and a Find button are enough to provide the sought-after results. But again and again I have heard complaints about not being able to effectively find colleagues using this feature.

The effectiveness of the People Finder feature is challenged in the following ways:

1. People misspell names of staff they are searching. (e.g., ‘Vivek’ is spelled as ‘Vevek’; with over 140 different nationalities this is bound to happen.)
2. Names stored in the database are not in proper format. (e.g., ‘Vivek Deshmukh’ is stored as ‘Vivek D.’)
3. People are known by completely different names than the one stored in the database. (e.g., In some cultures women change their names after marriage.)

Figure 1: A typical example of not finding a person in staff directory

How can you design a better People Finder application than the one that so often says “Staff not found!”?

Building on users efforts

One idea is to look at what users do with the problem at hand and how they solve it, and then use their efforts to build the application. For our People Finder application we can do this by having a “Did you mean …” feature, which gives alternative name suggestions to users. These suggestions are not built by pre-defined logic but are based on the collective input of users.

In our department, when colleagues don’t find someone on a People Finder application, they try various strategies. These include:

1. trying different spellings,
2. asking another colleague for the persons correct name and spelling,
3. calling the person directly (if they have their phone number), and
4. checking previous emails to get the exact spelling.

Whatever activity they choose, they make sure that they have the right information to type in the People Finder text box. We need to make use of this effort (i.e., making an error and then fixing it) from the users to build our application. The following conceptual model is my attempt at designing such a system.

Building the application

There are five essential components to this concept:

1. Build a relation table to store incorrect entries. In other words, store search queries which produced no results.
2. Determine if the user has found the right person.
3. Build a relation between the previous incorrect entries with the last correct entry determined in step 2.
4. Check the strength of relation by observing patterns across all users.
5. Present strong patterns as a “Did you mean …” feature on the search results page.

Let’s look at each step in detail.

STEP 1: Build a relation table to store incorrect entries.

To explain the concept, let’s take the scenario in which a user Sally wants to organize a meeting with Timothy Campbell using People Finder but cannot find him because Timothy Campbell is stored as Tim C. in the application database. (See Figure 1 above.) Let us store this incorrect entry Timothy Campbell in a database table called “Relation Database Table for Sally” (See Figure 2).

Figure 2: Incorrect entry inserted in the Relation table for Sally

STEP 2: Determine if the user has found the right person.

Next, let us say Sally tries a few more names in the People Finder text box, which generate no results. We store all of these incorrect entries in the Relation table. After a few failed attempts, Sally asks her colleague how to find Timothy Campbell in the address book. She is told his name appears in the address book as Tim C. Sally types the name ‘Tim C.’ and gets a result with Tim C.’s details. Sally adds Tim C. to the meeting list. It is this action of Sally clicking the Add button that allows us to identify a correct entry for the Relation table. (See Figure 3.)

Figure 3: Sally now types Tim Campbell’s name as it appears in the database.

STEP 3: Build a relation between the previous incorrect entries with the correct entry.

We then build a relation between the previous incorrect entries with the first following correct entry (i.e., Tim C.) and add it to another table called ‘Alias’ for the staff Tim C. Think of the Alias table as a ‘People also know Tim C. as …’ list. Note that the basis for saying that there exists a relation between the incorrect entries and the correct entry is the real life observation that people do what they must to find the correct name to type in the search text box. Of course you may get mismatches but this will be taken care of in the next steps.

Figure 4: Tim C. is related with the previously typed names

Figure 5: Alias table for Tim C.

STEP 4: Check the strength of relation by observing patterns across all users.

Next we identify the most common aliases used for finding Tim C. We do this by looking at the Alias table for Tim C. Those aliases that appear frequently are strong candidates to be displayed with a “Did you mean …” feature. In our example Timothy Campbell and Tim Campbell show a good pattern across different users as aliases for Tim C., so we conclude that when people search for Tim Campbell they mean Tim C.

Figure 6: Alias table shows that lot of people type Timothy Campbell or Tim Campbell to find Tim C.

STEP 5: Present common patterns as “Did you mean …” feature on the search results page.

The last step is to present the most common pattern to the users as a “Did you mean …” feature. In our example when -users search Timothy Campbell we present them with Tim C. as a “Did you mean …” feature. We can show additional information like the department, title or a photo of Tim Campbell so that the user can confirm that it’s the person he is looking for.

Figure 7: Implementation of the “Did you mean …” feature

Making the system efficient

The secret to making our system more efficient is eliminating -irrelevant relations. Consider this question: Should you build a relation between an incorrect entry which was entered at 08.30 and the next correct entry entered at 09.20? Probably not! It is very unlikely that the user will search for the same person after a gap of 50 minutes. A time frame of 20 minutes may be more realistic.

Advice on how to go about building such a system

1. Build a business case
   Building such a system will take time and resources. You will need to present an argument to management why this is important and perhaps make a business case for the effort. Don’t forget to include key stakeholders like Human Resources while presenting the business case. Here are some key points:
   • Users will save valuable time while searching other staff.
   • Colleagues will not be disturbed – their time will be saved.
   • Companies will save on phone bills and employee time.
   • Systems become robust over time without additional work from users or a massive data cleaning effort.
2. Collaborate and co-ordinate with different IT teams who build applications
   Pepare a list of all applications that use the People Finder feature. Collaborate with the IT teams who are responsible to build these applications and work out a plan to implement the “Did you mean …” feature on the current applications. This task becomes easier if you have a centralized IT team.
3. Prioritize applications for implementation
   Our system becomes robust when many users use the system. Start implementing the feature on the “Frequently used by many” type of applications first. These applications will give maximum value in shortest amount of time.

The risks

There is a risk that few users can work together to build a strong pattern of, say, “Jerk” with Rob Stevenson, thus manipulating the system. This can be kept in check by doing two things. If your company is like ours (formal and very particular about its image) you can:

1. Keep pattern strength high especially if the suggestions to users are going to be automatic without human intervention.
2. In addition to high pattern strength you can include a manual check done by HR admin who can authorize or investigate each strong pattern. To do this you will need to provide an admin interface to HR where they can monitor and dig deeper in to strong aliases.

Going forward

Just as users learn a new system by using it, possibly by making mistakes on the way, a system can also be ”trained" to learn from the users by continuously “listening” to users inputs, while helping users along the way. Though some development effort and technical know-how is required, more intelligent people-finding features on company intranets are essential. There is a long-term payoff, and companies will be saving a great deal in terms of employee time and costs.

• User types in a search
• Search engine gives back matching results
• User reads the results and picks the best one

Simple. Better still, it asks very little of the user interface—only that it provide some way to submit a search, and some list in response.

However, such simple models overlook the fact that humans are complex, convoluted, capricious, mutable, moody, multifaceted beings with broadly differing backgrounds, competencies, and frames of reference. (1) In practice, this can make the requirements for search interfaces quite a bit more complicated.

The good news is that while users vary widely in the ways they search, their behaviors follow a limited number of identifiable patterns. By examining the factors that cause variability in user behavior and considering personas that illustrate those variations, we can identify common search behavior patterns and the interface affordances that support them. 

Factors that affect user behavior

Search behavior is the result of interplay among several independent factors the user brings to the search operation, six of which are described below. Designers have no more control over these than they have over the color of the user’s hair. 

1. Domain expertise
User behavior has a lot do to with a user’s familiarity with the subject on which he or she is searching. When searching outside a domain of expertise, people will be less certain where to start, use less precise language, and have more difficulty evaluating search results. By contrast, experts in a field generally know what verbiage will work best, and so generally get better results, from which they’re better able to discern the most useful documents. (2)

2. Search experience
Users who have a better understanding of the breadth of a search engine’s capabilities have more ways to go about finding information. If you know how to use Boolean operators, exact strings, filtering controls, and have proven strategies for exploiting search, then you have a much richer toolset at your disposal. But search experience also isn’t an absolute requirement for success. We have seen that users who are short on technical know-how but rich in domain knowledge can often get by. On the other hand, technophiles can have great difficulty finding information in an unfamiliar body of knowledge. 

3. Cognitive style
User behavior is also influenced by the way users assimilate new information. Researchers like Nigel Ford and his colleagues have proposed a number of schemas to describe cognitive style, but for the purposes of search it makes sense to think of it as a spectrum ranging from global to analytical thinking.

• Global thinkers first try to build a broad level of understanding across related topics.
• Analytical thinkers dive right into a single topic and research it thoroughly to resolve a specific problem.

Most people lie somewhere between these extremes, sporadically using either cognitive style but tending more often toward one. (3) 

4. Goal type
Search goals will vary from one query to the next, and may be broadly classified into three categories as outlined by Andrei Broder in his article “A Taxonomy of Web Search:”

• Navigational searches are efforts to reach a particular location, such as an intranet’s timesheet application.
• Informational searches seek out any documents relating to a topic, like a description of employee benefits.
• Transactional searches occur when the user primarily wants to accomplish something online, like changing her benefits elections. 

5. Mode of seeking
The extent to which users understand what they are trying to find determines their mode of seeking. The level of understanding can range from known items, where people know exactly what they need and how to describe it, to much more exploratory searches, where they have only a loose concept what they want to find. (4) Furthermore, as Marcia Bates pointed out in her oft-cited 1989 paper “The Design of Browsing and Berrypicking Techniques for the Online Search Interface,” information needs are often unstable and may evolve as a user learns more about a subject area. 

6. Situational idiosyncrasies
To add a final layer of unpredictability, search behavior can vary for the same user with the same task, due to idiosyncrasies in external pressures, working context, temperament, or mood. For example, a user who is nearing a tight deadline is likely to behave very differently than a user who is just leisurely exploring the same topic out of general interest. People can also approach search tasks differently simply if they’ve had a bad day, feel tired, stand to make money, or feel especially engaged in a topic. 

Personas

Grounding abstract ideas in concrete personas can help bring all of these factors to life. Personas are descriptions of typical users that illustrate key attributes that are relevant to the design of a website or online system. An understanding of the motives underlying user actions, like those detailed above, provides a great starting point for authoring personas.

For instance, the hypothetical people described below each illustrate different areas of domain knowledge, and represent a spectrum of search experiences and cognitive styles. They will be used to relate the factors above to the common search behavior patterns that follow.

• Andrea is a technical wiz who is completely comfortable with search engines. She is a project manager for a mainframe manufacturing division of her company. Her cognitive style tends to be analytical.
• Dmitry has moderate technical know-how. He works in the benefits administration division of his company’s HR department. He learns new information globally about as often as he does analytically.
• Kazue is generally uncomfortable with technology, but is a recognized expert in her field of instructional design. She tends to be a global thinker who prizes an understanding of the big picture.

Patterns of Behavior

Despite the large number of variables tugging user actions this way and that, they translate into a relatively small number of common patterns of behavior. In my work, I’ve observed six broad patterns, described below with recommendations for accommodating each.

1. Alternating between search and browse
When searching, users will often select a result that is closest to the topic they have in mind even if it isn’t a precise match. They’ll then follow the links on that page to find their target information. A global thinker like Kazue might do this if she were exploring an information goal outside of her domain expertise. Unable to formulate the search phrase precisely right, she would need to trust the results returned by the engine. Finding that they’re promising but not quite right, she may switch to browsing before returning to the results page.

In effect, searching and browsing can function as a single behavior, with many people moving fluidly between both. These users see no distinction between the two, since both work in support of a single information seeking task. This means that improving the quality of a site’s navigation will necessarily also make searches more successful.

Design recommendations:

• Support robust cross-linking on each page, so that when users reach pages that are near matches they can easily get to the best matches.
• Include conventional hierarchical cues like breadcrumb trails and contextual navigation, as well as nonhierarchical, associative links among topically related pages. (4)
• Don’t let pages come to a dead end, without any links to other resources on the site.

If Kazue is able to easily cross-link among related pages, this hybridized searching/browsing behavior will be more effective. 

2. Minimizing the results set
Users sometimes measure the success of a query primarily by the number of results it returns. If they feel the number is too large, they add more terms in an effort to bring back a more manageable set. Given her understanding of how search engines determine relevance, you’d expect Andrea to do this if she needed to quickly locate a known item within her domain expertise, like “mainframe manufacturing.”

Design recommendations:

• Allow users to filter the search results by categories, so they can reduce the number of results while making them more topical.
• Include a numeric count of the total number of results returned for the query and the total number for each category.
• Use “and” as the default operator rather than “or,” so the number of results narrows instead of growing as the user adds more terms.
• Don’t confound this behavior by truncating the total results set at a round number like 100 or 500; this makes it difficult for users like Andrea to gauge the quality of her query.

Fig. 1: Filtering mechanisms help users narrow down searches that brought back too many results.

3. Surveying quickly
Some users scan through the results quickly, and if none of the titles strike them as an ideal match, they may proceed several pages deep into the results set. I’ve seen these users go to the fifth or sixth page of results without hesitation, then go back to the initial results to look more carefully or submit another query.

For instance, Dmitry could do this to hedge his strategy if his task isn’t fully defined. Hopeful that something will just pop out at him, he may do a quick scan of the first few pages, then fall back to another strategy if that doesn’t work out.

Design recommendations:

• Ensure that result titles are comprehensible at a glance, including application files like PDFs and Word documents, which often return cryptic file names by default.
• Highlight the terms that match the words originally submitted to help people scan the titles and descriptions more easily.
• Allow users to change the number of results shown per page to avoid navigating through too many paginated results.

These changes will allow Dmitry to evaluate pages more efficiently and successfully.

Fig. 2: Search engines often return cryptic file names for application files like PDFs and PowerPoint slideshows.
 

4. Making immediate judgments
Other users look only at the first few results before deciding whether the query was successful or not. Finding nothing, these users may then resubmit the query or give up on search altogether.

Andrea, the analytical thinker, would be discriminating about a result’s relevance to a narrowly defined informational goal. Confident in her expertise, she would also be quick to conclude that search is flawed if it cannot return a good match in the first few listings. This behavior requires that the best match be returned as close to the top of the list as possible.

Design recommendations:

• Optimize results for the most commonly submitted queries. Working from the search logs, try out each of the top queries and evaluate the quality of the top results returned, then optimize the content of those pages to improve their ranking.
• When pages cannot be further optimized, include a manually generated “Best Bets” sidebar to force those matches to appear at the top. This gives the page a second chance to hit the specific target in Andrea’s mind. 

5. Agonizing over the query
Sometimes users have difficulty translating the concept they want to find into a specific search phrase. They will often rewrite the query several times before submitting it, and then focus on revising it further if the results are not as they had expected them to be.

Less experienced users like Kazue are more likely to show this behavior, especially if the task isn’t well defined and lies conceptually outside of her domain. Kazue may also be inclined to phrase the query generally enough to satisfy her global cognitive style, but fret over how general is too general.

Design recommendations:

• Consider providing tools that assist in formulating the query, such as suggestion functions that present searches similar to the one the user is typing.
• Consider including lists of popular searches or automated storage of the user’s previous queries, saved to a profile or cookie.

Anytime that Kazue can select a query from a list rather than originating it from scratch, she will be able to search much more efficiently.

Fig. 3: Suggest functions assist users with formulating queries when they don’t quite know how to phrase their request.
 

6. Pogosticking
Some users click several results in rapid succession, quickly sampling each before settling on a best candidate to meet their needs. Jared Spool has described this as “pogosticking”—bouncing up and down between choices of uncertain relative value. This is the kind of behavior that Dmitry might resort to if the quick surveying behavior described for him above didn’t yield anything. Assuming that his temperament is fairly tolerant and he isn’t pressed for time, Dmitry may decide that he cannot determine the usefulness of pages without looking at them. These users need support for three primary tasks: assessing result listings, comparing result pages, and tracking work.

Design recommendations:

• Again, provide comprehensible titles and descriptions on the results page, as well as highlighted search terms.
• Pages can be even more effectively compared if highlighting can be extended to the display of the results page itself (as is possible with Yahoo! and Google toolbars).
• Allow users the option to open results in a new browser window to assist comparison. Sites like Ask and Easy Search Live are experimenting with page previews.
• Be sure to include a visited link color on the results page. This is absolutely essential for Dmitry to keep track of the pages he has already tried and rejected as he jumps to each of the matches from the hub listing page.

Fig. 4: Visited link colors help the user avoid revisiting results that have already been tried and rejected.

Conclusion

Search behavior varies with domain expertise and technical knowledge, cognitive style, goal, and mode of seeking. All of these factors will interact in complex ways to influence a user’s actions. Even then, behaviors will vary depending upon whether at that moment the user is under pressure, in a good mood, or any number of other idiosyncrasies.

The point is that the designer cannot select the behavior that a user will follow when conducting a search. This may invite the impression that the design must be overly broad, providing any conceivable function regardless of the likelihood it will be used, because we cannot predict whether it will be needed. Fortunately, users’ actual behaviors do fall into generally describable patterns, each of which has dependencies upon specific affordances of the interface. This is how designers can better cater to what appears to be chaos: make available those capabilities that best support the range of known behavior patterns for your target personas. 

References

(1) James Kalbach provides an overview of literature around this topic in his article “Designing for Information Foragers: A Behavioral Model for Information Seeking on the World Wide Web.”

(2) For more on expert search behavior, see these two articles: Christoph Hšlscher & Gerhard Strube (2000): “Web Search Behavior of Internet Experts and Newbies”; and, Suresh K. Bhavanani (2002): “Domain-Specific Search Strategies for the Effective Retrieval of Healthcare and Shopping Information,” CHI 2002, pp. 610-611.

(3) See Ryen W. White & Steven M. Drucker (2007): “Investigating Behavioral Variability in Web Search,” International World Wide Web Conference 2007, pp. 21-30.

(4) See Donna Maurer (2006): “Four Modes of Seeking Information and How to Design for Them.”

(5) David Fiorito and Richard Dalton further described different types of navigation in their presentation at the 2004 IA Summit, “Creating a Consistent Enterprise Web Navigation Solution”.

Why most people don’t use advanced search

Despite its name, advanced search has not advanced very far. There is great power to conquer the overwhelming number of search results, but the current standard presents barriers to users. Specifically,

• The link is often small, vague, and does not describe benefits to the user
• Advanced search pages typically have confusing page design for the few who make it there.
• There is generally poor search revision functionality: Once a search is performed, the “advancedness” is lost. For example, the Google advanced search delivers the standard search results page. You have to get the query right the first time; there is no opportunity to adjust your query.

Fig. 2: Google’s advanced search is still complicated.

Who does use advanced search?

Tim Bray wrote, “The people who do use Advanced Search are your most fanatical users, the professional librarians, spooks, and private investigators.”(2) As in many situations, segmenting a population makes it harder to see whether the needs of a subgroup are shared by the majority. Current design is based on the idea that there are two separate audiences with distinct needs. The reality is that there are overlapping needs that are limited by the search pattern.
The essential problem of search — too many irrelevant results — has not gone away.
A typical user has to make a choice between doing a search and clicking a link to do a search. In other words, do you want it now, or want to go somewhere else to look? The immediacy of the search text field and the complexity of advanced search means that users will try the text search first.
Perhaps the old framing is wrong. Rather than being a matter of geeks versus normal people, the question should be whether users see a benefit to advanced search on starting. Unfortunately, there is typically no way to use advanced search at the point users realize they need it — when they haven’t found what they were looking for and have too many search results. They have “missed the exit” to advanced search. Users don’t want to lose the investment in their search; they need a way to use additional techniques to work with what they have. A new model of search can help with this problem.

Other approaches to search
Let’s take a quick look at other search innovations from the last decade. Web data has matured and become more structured. Taxonomies and tagging are now common. There are new opportunities to deal with search results overload by filtering, as long as it is clear and easy for the audience. Despite the truism that users will not go past the first page of search results, they will use obvious tools to refine their searches.

Searching by defined parameters is natural in some circumstances (for example, airline ticket searching) but the majority of sites are not sufficiently data-driven to have an interface designed around the data.

Fig. 3: travelocity.com Flight searches, in a sense, only use advanced search.

Tags can improve search results by better describing what someone is seeking.

Fig. 4: amazon.com’s approach to tagging a product

Faceted searching, or returning browsing categories in the search results (as eBay does) can be effective at prompting the user to select a single category.

Fig. 5: ebay.com’s faceted search

Filtering search results

Amazon and Kayak offer filters to enable users to reduce many results to a few. This can be very effective, but there are obvious constraints due to the limited space available for each filter. Only the first few filters are visible when the page loads.

Fig. 6: Filtering on amazon.com and Kayak

Another approach: Progressive disclosure of functionality
One solution to the essential problems of advanced search discoverability and complexity is to progressively disclose (3) the functionality to the user. Instead of a single, complicated page, break it into understandable units and give each to the user when they ask for it.
In this example, show the ways the user could filter the results (e.g., “Brand” or “Price”) in a highlighted banner above the search results.

Fig. 7: Filters highlighted in a banner (it doesn’t have to be green, it just has to stand out!)

When the user clicks a link, display the filter with enough space to clearly articulate how to use it. Don’t cram it in; the user asked for it. In contrast to the left-hand column filters in the examples above, which are naturally space-constrained, this method can hold many more types of filters and doesn’t show functionality they didn’t request.

Fig. 8: When the user clicks "price" above, give the module enough space to be readable.

Let’s look at an example scenario.

1. The user performs a text search normally.

Fig. 9: Searching for a DVD player using the simple search box

2. On the search results page, show options to filter the search in a prominent location above the results. Communicate the value of filtering. Order them by popularity, using size and font weight to highlight others. If there are many options, consider hiding rarely used options under an expandable section.

Fig. 10: A search results page, with options to filter.

3. When the user clicks an option, display a page module for that search parameter without reloading the page. The user should be able to change the parameters at any time to receive an updated search. If possible, show the number of results for each parameter, so the user can see how exclusive it is, and identify which parameter maybe giving them 0 results.

Fig. 11: The search results, filtered by price.

4. Enable the user to add several modules, stacked in chronological order as the user builds up a complicated query.

Fig. 12: The search results, filtered by price and rating.

Recommendations
Define an implicit method for Boolean rules (AND and OR rules) based on normal search patterns — do not ask users to compose Boolean queries. A system that has worked for me is this: If a user selects several different search parameters, perform an AND search between them (e.g., Sony AND Portable). If they choose multiple values for the same parameter, perform an OR search (e.g., Sony OR Panasonic). However, if parameters (such as product features) are clearly non-exclusive, perform an AND search (e.g., Portable AND “HD ready”).

Recognize that quick searches, text searches, and advanced searches may be built with different technologies (e.g., direct database searches, a Google box, or a content management system). You may need to work closely with the developers to make a seamless transition between technologies.

If there are many parameters (more than 15), consider reducing complexity by hiding less used ones under a “see more…” link below the displayed options. Clicking it should display all the options without a page refresh. Evaluate your search logs to make sure you are exposing the right ones. Consider rotating the exposed ones to discover potential popular features, as exposed options will naturally get more usage.

Conclusions
At its core, advanced search is an under-utilized tool hampered by its own design. By enabling the user to add specificity as they request it, designs such as the one above avoid the lonely fate of the standard advanced search page. Defusing this complexity and locating it where users will naturally find it will help advanced search be truly advanced.

References
(1) Nielsen, J. (1997). Search and You May Find. Alertbox, July 15, 1997.
(2) Bray, T. (2003). On Search: The Users.
(3) Wikipedia. Progressive Disclosure

Unfortunately, the results are too often plagued by problems. You know something’s gone wrong when a perfectly clear query returns results that are not only irrelevant, but seemingly deranged. Pages with a logical relationship to the initial request compete for placement among what Jared Spool fittingly calls “wacko results.”¹ The majority of participants walking into my usability tests report they don’t trust embedded site search to help them find what they’re looking for.

Quality search results only come about through applied effort, requiring in particular the skills of an information architect.² And IAs must be ready to go well beyond their traditional front-end role, digging into the functional backend and source data of the search engine. This article outlines how we can bolster findability and win back users’ confidence.

Conceptualizing the Task

The results of any given search are impossible to predict with precision (short of having tried it before). That’s because five distinct variables combine to determine its outcome (Figure 1):

1. Search engine. The algorithmic gears that parse the query and assign pages relevance.
2. Content. The documents searched.
3. Index. A catalog of the locations of every word in every document. This is what allows Google to miraculously find 5 billion instances of the word “the” in 0.2 seconds.
4. User input. The keywords and other parameters the user submits.
5. Results display. The way the data returned by the search engine is presented.

Figure 1. Five variables that determine the success of a site search.

Critically, the search engine isn’t the only factor that determines the outcome, so search can’t be seen purely as a technology problem. It’s important for organizations to realize that their investment in search doesn’t end with the product’s implementation; the most successful approaches will go further to include strategies addressing all of the outside variables.

Strategies

Several engine products allow you to tweak the search engine’s algorithm itself, but I don’t recommend it. That would be like doing brain surgery to fix a speech impediment—whether or not you solve that problem, you’ll inevitably cause a great many more. Changing the algorithm affects all searches, including the ones that already work just fine. So it’s easiest to keep it stable and modify the factors surrounding it.

Taking the search engine as a constant, then, there are four variables that affect the quality of search. Strategies for improving each of these are proposed below.

Strategy 1: Make the Content Machine-Readable

Search engines can provide better results when they’re given better content. The trick is to provide a basis for inferring the content’s meaning.

Structural Markup

The XHTML structure of pages is relevant to the IA, because content that is more machine-readable will be easier to find using search. Pages should extensively use the correct semantic elements: <h1> through <h6>, <p>aragraph, <q>uotation, <caption>, and so on, as well as semantically named “class” attributes. This will help the search engine compare the usage of terms among pages, to distinguish the central topic of a page from peripheral concepts (Figure 2). While IAs typically don’t mark up individual pages, they can influence the process by specifying template-level semantic elements in their wireframes and participating in periodic content reviews.

Figure 2. Structural markup explains that Jupiter is the central topic of page A, while in page B it’s just one of several subpoints on observing planets.

Standard Meta Tags

Most websites use keywords and descriptions in meta tags, but not often as part of a larger strategy. The first step is to create a controlled vocabulary, a standardized set of keywords.³ If you tag them as “teachers” over here, but “professors” over there, the search engine will have a hard time understanding why they’re the same thing. The keywords should also reflect actual terminology from the page itself (especially headings) and be reinforced in the description tag.

More Metadata

Go beyond keywords. Tags that describe the target audience groups, the sector of a financial service, or the cuisine of a recipe page provide more ways to compare and contrast the content; search engines will read as much meta information as you give them. There is a practical limit to how much you can do, which makes user-defined tags well worth considering.

Ontology

Humans know that pugs are dogs, and dogs chase cats, and cats play with yarn, but these relationships are lost on computers. An ontology is a list of concepts linked by the ways they relate to one another (Figure 3), helping the search engine grasp the content’s meaning. If your search product supports ontologies (several do), this can significantly improve the quality of the results.⁴

Figure 3. An ontology explains the relationships between concepts.

Strategy 2: Index All of the Right Data

Indexes have made searching remarkably expedient, but the way they’re built has a lot to do with the quality and reliability of results. Proper indexing requires taking a hands-on approach, and the IA has an interest in working with the development team to influence it.

Ignoring Unnecessary Content

Search engines will automatically index the entire content of a page, regarding everything as equally important. This is a problem because the navigation, for example, will contain terms that are specifically relevant to the siblings, parents, and children of a page, and not to the page itself (Figure 4). There are several methods of excluding this content; the important thing is to make sure that it’s done, because this is one of the most common reasons why searches return bizarre results.

Figure 4. A search for “Neptune” may return results that include this page about Jupiter because the term “Neptune” appears here in the navigation.

Getting All Resources

Users reasonably expect a search to return all of the website’s relevant publicly available documents. Unfortunately, many search products can’t index .pdf, .doc, .xls, .ppt, and similar files, and you can forget about content locked away in audio or video files. The best fix is to convert application files to XHTML and provide transcripts or summaries of multimedia files. This can be a big job, so you may want to initially convert just the most commonly accessed documents.

Strategy 3: Make the Most of User Input

It can be difficult to figure out how to phrase a query. Users have to express what are often complicated concepts in that particular set of words that a given search engine will like. It’s important to make the most of what users submit on their first attempt, because they’re much less likely to make a second.⁵

Query Expansion

All contemporary search vendors offer some type of query expansion, where the search engine automatically looks for words related to the ones the user actually entered (Figure 5). Word stemming, which searches for different forms of the same word, is usually enabled by default. However, the thesaurus, which searches for equivalent and related terms, requires manual work.⁶

Figure 5. Searches shouldn’t only look for the terms as the user entered them, but for related and alternate forms of those terms.

You can go overboard defining synonyms, but the problem is usually too little (by which I mean “none at all”) rather than too much.⁸ Search logs are the best resource for discovering synonyms, related terms, and common misspellings. Set up ongoing reviews to add terms that users actually submit to the thesaurus, drawn from the wealth of data that’s freely available in the logs. The number of successful first attempts will rise dramatically over time.

Syntax Conventions

Users should be able to submit searches in whatever way they learned to write them. Unfortunately, search engines have different syntaxes for the standard operators (And, Or, Not, exact string). You can’t rely on a help file—it’s one of people’s least favorite things to read. The parser should instead be scripted to accept all common syntax conventions, so the user doesn’t have to guess. It should also use “And” as the default operator, which will appropriately limit the results downward as more terms are added to the search.

Assisting Query Formulation

Suggestion functions provide users with a list of similar queries that other people have tried as they type. This makes a lot of sense, since it can be difficult to put a complex idea into words or to recall the precise name of an item. Stellar examples of suggest functions include Google Suggest, AllTheWeb, and Apple’s website.

Strategy 4: Build the Results Page Around the User’s Needs

The results page should be designed to help users find matches for their interests as quickly as possible. This is closer to the IA’s typical interface design role, yet it’s still uncommon to see much more than the vendor’s out-of-the-box functionality on search results pages.

Showing Relevance

Sometimes a search engine will return the right results, but the user will fail to recognize it. Users need to see why results are relevant to their searches. There are two simple ways to do this.

The first is to show a text excerpt from the page that contains the terms from the user’s query, instead of the <meta>description field. The description may vary greatly from the user’s entered query—especially on long pages—and it may not be at all clear why a particular page was retrieved. Instead, an excerpt of the actual content that matches the search will directly explain why a user might want to click through to that page.

The second way to show relevance is to bold the terms in the excerpt that match terms in the user’s original query. That will help the user to quickly scan the page for the results that have the right words in the right context (Figure 6).

Figure 6. Excerpting and term highlighting allow the user to understand how each result relates to the query, and quickly identify the ones that are most relevant.

Best Bets

Despite all optimization efforts, search engines sometimes still miss strong associations that are obvious to people. In cases where particular keywords should be returning specific pages, it can be helpful to include a list of manually specified “Best Bets,” triggered by business rules (Figure 7).⁸ This reintroduces the designer’s influence into search, smoothing out irregularities in the reliability of automated results.

Figure 7. Best bets allow the designer to force particular pages to be returned when the user’s query contains a specific string.

Conditional Content

Taking Best Bets one step further, consider including contextually appropriate content in the search results page when a string in the user’s query indicates the user probably has a particular interest in mind. For example, a user searching for “extrasolar planets” on an astronomy website might appreciate a results page that includes a list comparing the properties of all planets discovered beyond our solar system.

Conclusion

This article introduces just some of the steps that you can take to improve the overall search experience on your site. The reliability of enterprise search needs significant improvement to reestablish user confidence, and IAs should take the lead. To get there, a product’s out-of-the-box functionality must not be seen as the end, but as just the beginning.

REFERENCES

• ¹ Jared Spool: “BBC Reports Users Lose Patience with Poor Search”

• ² Lou Rosenfeld & Peter Morville, Information Architecture for the World Wide Web, pp 136-137.

• ³ Fred Leise, Karl Fast, and Mike Steckel: “Creating a Controlled Vocabulary”

• ⁴ Tim Berners-Lee: “The Semantic Web”

• ⁵ Jared Spool: “People Search Once, Maybe Twice”

• ⁶ Christina Wodtke, Information Architecture: Blueprints for the Web, pp. 137-140.

• ⁷Lou Rosenfeld & Peter Morville, Information Architecture for the World Wide Web, pp. 188-189.

• ⁸ Chris Farnum: “Tuning up Site Search”

In “Succeeding at IA in the Enterprise,” James Robertson calls for Enterprise Information Architects to pay attention to the realm of business strategy, and Bob Goodman in “Change Architecture: Bringing IA to the Business Domain” offers the notion of EIAs as change agents or change architects.

Yes, EIAs can be a force for change, but so can anyone. The real question is: should the discipline of Enterprise Information Architecture be defined to include organizational change as one of its essential features. I don’t think it should be.

Whatever it is that information architects learn to become enterprise information architects (EIA), I think it is essential that we not lose our focus. The heart of IA is information and knowledge, and we need to build on that foundation, not try to turn into something else when we add the term “enterprise.”

It is the same for business strategy. Yes, EIAs should better understand their organization’s business strategy and integrate the strategic vision. And, yes, EIAs can contribute to the development of business strategy, but not directly. We should not try to tell business analysts how to do their job. What we can do is provide input into the development of business strategy based on our understanding of and access to information and how those issues are essential to any business strategy.

Integrating business strategy into EIA is only a part of what it means to move from IA to EIA, and I’d like to take a look at the essential issues that IAs bring to the enterprise and what sorts of things IAs need to add and/or learn to develop an enterprise perspective.

The move towards “Enterprise”
To start we should recognize that adding “enterprise” to our titles is part of a broader trend toward enterprise solutions in a variety of fields from technology to management restructuring. For example, content management, knowledge management, and learning management software vendors are all moving toward creating platforms for the entire enterprise that cover the entire life cycle of information and knowledge.

However, the truly essential feature of enterprise solutions — and what we have to recognize if we want to become EIAs — is that moving to an enterprise perspective does not mean simply doing bigger or more varied projects. Rather, it is a move away from a project-centric model toward an infrastructure model.

In order to have an enterprise-wide solution, you must develop an infrastructure that enables projects to be fully integrated, that enables projects to build on a common foundation rather than always starting from scratch, and that enables projects to be accomplished cheaper and faster with a broader impact.

The next step is the most important and the most difficult. It is the creation of a solid, well-articulated semantic infrastructure.

Semantic infrastructure
A semantic infrastructure consists of all the various kinds of information and content in an organization plus the ways that information and content is organized and structured. It includes structured and unstructured documents, tacit knowledge, and other non-document-based content like images and animation. It also includes the policies and procedures around the creation and dissemination of that content, the technologies that support content creation and dissemination (search, CM, portals, etc.), and finally, and most importantly, the ways that content is structured such as metadata, taxonomies, controlled vocabularies, semantic networks, ontologies, social network analysis representations, knowledge and topic maps, and other advanced knowledge representations.

A semantic infrastructure also includes modeling and/or mapping people and their activities. This includes formal and informal communities of all kinds, the information behaviors associated with those communities, linguistic and category preferences, and most of all, how information is used within the full range of business and support activities that these communities engage in.

In my opinion, it is this enterprise-wide semantic infrastructure that is the context within which EIA should operate, and it is the essential feature of defining what an enterprise information architect does and understanding how EIAs differ from IAs.

Distinguishing EIA from IA
Two key elements distinguish an enterprise IA from a basic IA. The first is the role an EIA plays in the design, development, and maintenance of an enterprise’s semantic infrastructure. The second is the scope and type of projects an EIA can be involved in as they develop applications that use and build on this semantic infrastructure.

Let’s start by looking at how an EIA might help develop a semantic infrastructure. A complete answer is beyond the scope of this article, but a simple answer is: IA skills and tools can support research that is fundamental to the creation of a well-designed semantic infrastructure. This can include such activities as researching information needs and behaviors of different users, mapping different communities in an organization, and ethnographic studies.

Just as more and more enterprises become convinced of the importance of content structures like taxonomies, however, EIAs should become more aware of and focus more attention on these content structures. EIAs should also be more aware of how different groups categorize the world around them in different ways and what those differences mean for information architecture. For example, as someone’s knowledge of a field increases, they tend to focus on the more precise levels of a taxonomy while novices tend to prefer higher level categories. Mapping these category level preferences into designs is an important component of a semantic architecture. This is a natural extension of the traditional work on metadata and labeling that goes into IA work today.

The type and scope of applications and projects is the second key element that distinguishes an IA from and EIA. For traditional information projects like search and portals, and even individual intranet website projects, it is important to approach them with an enterprise-wide perspective. One of the reasons these types of projects fail to deliver their full value is that they do not take into account the entire enterprise.

In addition, enterprise information architects work on a broader range of projects, not just traditional information projects like search and Content Management. These other projects could include adding an important emphasis on how information organization and presentation impacts all business activities. (But not to tell other groups how to run their business activities.)

EIAs should also be involved in knowledge management projects like collaboration, communities of practice, and innovation programs. Too often the failure of knowledge management programs is blamed on culture, when all it needed was an understanding of and support for the various group’s information behaviors – a really good information architecture.

Finally, EIAs should not just be involved in the enterprise’s information architecture, but also involved in the information architecture of the enterprise. They should apply IA skills to understand, model, and support how information and knowledge flows within the enterprise.

This could include studying and developing new information architectures for how people do everyday jobs, how desktops or webtops are set up, how those designs impede daily information use, and how to improve those designs. It could also use an expanded social network analysis of who gets their information from what people or content repositories and what kinds of information are not reaching critical audiences. In other words, instead of focusing on the design of the intranet, EIA can focus on the people in the organization and support the information seeking strategies and behaviors that are part of their daily routine.

The place of EIA in the enterprise
The last question I want to examine is where EIA fits in within today’s organizations. It is clear that a semantic architecture needs a broad, interdisciplinary team. It should include members from IT, business units, and information professionals like librarians, information architects, and others.

In addition, this interdisciplinary team will need to partner with other departments throughout the enterprise, including business, technology, sales, administration, and research.

The actual makeup of the team and where it is located organizationally will vary from industry to industry, but from experience we know it should not be located in IT. IT should be involved; people from IT should be on the team, but despite the fact that “information” appears in their department description, we should remember that they are not really information professionals.

For example, many enterprise search, content management, and portal projects fail to deliver full benefits because what is needed to make them work goes way beyond traditional “needs assessments.” It is not enough to have people go out and ask users what they need. You need to have information professionals (IAs, librarians, knowledge engineers, etc.) to develop new creative possibilities to offer to users. You need someone who can ask deeper questions about information behaviors.

Having IT, business analysts, and subject matter experts involved is important and necessary, but none of those three groups understands information and knowledge at a sufficiently deep level to offer truly creative and innovative alternatives that make information and knowledge systems work across the whole enterprise.

On the other hand, it has been suggested that EIA should lead this central information/knowledge team. Personally, I don’t think this is a good idea for a number of reasons having to do mostly with the difference between knowledge and information and the background/training that most IAs have. However, EIA might be one candidate to lead this interdisciplinary team, as long as we start with a greatly expanded definition of the role of EIA based on a stronger emphasis on semantic architecture.

The other major candidates for leading this team are knowledge management and learning departments. Learning groups are a good place, but like IA, traditionally, they don’t deal with the depth and breath of knowledge modeling that is needed. Learning is only one type of information and knowledge activity and we need something to cover the full spectrum of activities: searching, using existing information to create new documents, writing reports, and even using information to make a sale.

Personally, I think that knowledge management departments are a very good place to locate this team, but it does require KM to shift focus away from an over-reliance on the idea of tacit knowledge and move towards actually modeling knowledge—in other words, KM with more emphasis on knowledge than management.

Knowledge architecture is a term that I think captures the essential nature of this interdisciplinary team and, at the same time, adds the focus of actually modeling knowledge to KM. However, it is not a widely used term and requires significant explanation.

Whatever term is used will require some redefining (KM), some expansion (EIA), or some explanation and general acceptance (KA). Take your pick or come up with an entirely new one.

However, what is more important than who will lead this team is to pay attention to how all the members of this interdisciplinary team communicate among themselves. The team will need the services of a good IA to help the team communicate and interact with the rest of the organization. Regardless of who leads, the Enterprise Information Architect will always be an essential part.

Conclusion
There is a great deal more that could be said be but in the interests of brevity, let me close with this thought. This vision of enterprise information architecture might not be what you think of or want an EIA to be, but this vision is meant to be a broad one that attempts to locate EIA within an even broader context that ultimately consists of some form of knowledge management. Within this vision of EIA, however, there is room for many different IA roles.

Some EIAs might work closely with the team doing more advanced knowledge representations. Some might work with the semantic specialists like taxonomist and metadata designers. Some might work with business strategy groups, and some might work with business process re-designers. On the other hand, some might work on traditional intranet projects, some with search or portal projects, and some doing traditional IA on new knowledge management projects.

For me, it is not so much that EIA represents a new field, so much as expanding the definition of what information architecture is and, at the same time, locating IA within a new enterprise approach to information and knowledge.

Amanda Spink is one of the smartest people working on user behavior while using web search, yet when I mentioned her name to a friend who’s spent the last year working on the search user experience, he had never heard of her. The design community is woefully undereducated about search, and is often prone to redesigning Google and postulating what Yahoo! is doing wrong, rather than working to understand why search engines have chosen to do what they do. I suppose this shouldn’t be surprising, though, considering Spink’s work is more often seen in scholarly journals, such as New Directions in Human Information Behavior and Journal of the American Society for Information Science and Technology (brought to you by the same folks who bring you the IA Summit, yet rarely cracked by the working folks).

In order to help correct this problem, I shyly contacted my hero by email, and overcame the time difference between sunny California and even sunnier Brisbane, Australia, with a series of email questions.

Christina Wodtke: When I joined Yahoo!, I had never worked on search before. Your article, “From E-Sex to E-Commerce: How Search Changes” was one of the most valuable in beginning to get my mind around the problem of search. Since reading that article, however, I haven’t seen much change from your findings. In your opinion, are users changing their search behavior, or are they still following the same patterns you found when you analyzed Excite’s data?

Amanda Spink: You make a good observation. Since 1997, our findings have come from analyzing large-scale web user data gathered from commercial web companies, including Excite, Ask Jeeves, Alltheweb.com, Alta Vista, Vivisimo, and Dogpile. Our research since 1997 shows some trends and changing patterns in general searching. However, looking at more recent data from Vivisimo and Dogpile, most web queries are still short—2 to 3 terms, and sessions include little query modification and are generally 2-3 queries in length.

Few people use advanced search features, and many queries include spelling and other mistakes that adversely affect the search results. People look at only a few result pages—not beyond the first or second results pages.

A small number of terms are used with high frequency, and many terms are used once. Web queries are very diverse in topic and some, such as people’s names, are unique.

CW: You’re referring to the “long tail” of a Zipf curve? How does this affect search engine’s strategies in providing relevant results?

AS: The long tail makes “relevant” retrieval very difficult, especially if users are only providing the search engine with two to three words on which to base relevance judgments. Despite the inherent “interactive” nature of search, much of search is not very interactive. The talk about personalization is an attempt to obtain more information from the user to help determine relevance.

CW: So are users changing their overall behavior at all?

AS: We are seeing a growth in more complex search behaviors. More people are searching for information using more than one search. This might mean repeat searches of the same query over time or modifying the queries in successive searches over time. Many people are multitasking or searching for information on more than one topic during a search session. People’s information needs are often quite complex in their home and work environments.

CW: How are they handling that?

AS: During multitasking search they may include two or more topics in one window, or open new windows for each topic and run searches concurrently.

CW: Could you talk a bit more about “complex search behaviors”?

AS: Peoples’ information seeking behavior can often be long and complex. Imagine a person is looking for information on cars. He conducts one search on one search engine, looks at the results, and tries another search engine, or goes back to the first search engine, and repeats the same search with the same search terms, or he may add or remove some terms (query reformulation). This is called successive searching.

In addition, research shows that people often search for more than one topic during their interaction with a search engine. They may batch their topics due to time constraints or new topics may evolve during their search session. This is called multitasking search.

Both phenomena are examples of more complex behaviors beyond the one-topic, one-search paradigm that most search engines assume.

CW: If queries are still so short, what are some of the more successful disambiguation tools used by the search engine? Vivisimo and Clusty offer algorithmically generated groupings and present them as narrowing tools. But last time I was testing these tools with users, the narrowing options were essentially invisible to them. And despite Jakob Nielsen’s assurance (admittedly in 1999) that longer search boxes produce longer queries, I’ve never seen it happen. Are there better and worse ways to encourage richer queries from users?

AS: So far no commercial tool seems to be effective at helping users on a large scale. Search engines have not used longer boxes, so no one really knows what would happen on a large scale if text boxes were changed. The best way to encourage richer queries is to train users and expect them to put more effort into their search behavior. Search engines need to put more demands on the users. People don’t understand their own information behaviors, and they don’t really understand much about search or the web, so they will have to learn. It could take generations.

CW: Really? Many folks who revamp search, either by adding Google, Yahoo!, or another vendor, seem to be leaning toward long entry boxes. I’m thinking about CNN, NY Times.com, CNET.

AS: The Google and CNN text boxes may be a little bigger or longer than average, but not substantially longer. How about a structured textbox, like an electronic library catalog interface? How about a textbox that is 3 inches by 3 inches with lots of space for people to express themselves? If you give people a small text box, you’re probably constraining their expression of their information problem.

People need to feel they should play around with search and experiment. All they can really do at present is squeeze in a few words, press search, and look at a list of websites-the list giving little indication of what the websites mean or how they are ranked. One major problem is that [designers of] search engines tend to think that one technique will do it! What they need to do is test combinations of many techniques, such as clustering, relevance feedback, etc. There is no silver bullet here.

CW: When you speak of training users, I’ve found that very challenging, much more with search than any other Internet paradigm. I’ve been in a lab with a fellow who has used Google for five years, and he never realized “cached” was there until I pointed it out. How can train people who have “banner blindness” for most of the page?

AS: I think this is a major problem for the web industry. How to train billions of people? Whoever comes up with the best solution for that question may capture huge market share. The paradigm needs to change. Search is challenging and interactive, and maybe a “game” paradigm would help.

CW: The short-query phenomena is fascinating. In a lab, I have asked people why they typed, say, “sailboats,” and they’ll say, “Oh, I’m interested in taking classes next summer when we’re up at the lake in Michigan,” yet none of the words made it into their query. Any insights?

AS: Our research shows that the most effective search terms are those submitted by the user, from a user’s interaction with another person about their topic, and terms they identify on the screen from the retrieved output. Stimulating users to talk with someone or thing (agent) about their information problem helps generate terms and look at the results for additional terms.

CW: Hey, those sound like classic reference librarian techniques!

AS: One area that some web developers are exploring is classic reference librarian techniques. It’s an obvious area to explore to understand information behavior and how librarians have helped people with their information problems.

CW: “E-sex and E-commerce,” was referring to a topical shift in searches. Are you continuing to see changes in what people are searching for?

AS: I think it’s important not to assume what “people are searching for” means just U.S.-based search engines. There are major differences emerging in search in different global regions. For the more U.S.-based search engines, the topics seem to have stabilized somewhat with business and e-commerce related searches being the largest category, followed by people, places and things, computers, and medical/health. Sex/porn and entertainment is now a smaller proportion of searches. From what I’ve seen about the Chinese search engines (e.g., Baidu), users are looking for entertainment and gaming. One could say that the Chinese search engine users are where the U.S. users were 5-10 years ago. As more Chinese business information is accessible via Baidu, the search topics may change. Also, currently less than 10% of the Chinese population search the web, so as that number increases, topic may change.

CW: There are endless articles these days about search privacy, and Google giving information to the feds. Is the ordinary person on the street worried about that?

AS: This is an important area for everyone. If search engines and the web are becoming the primary tool by which people are expected to access information, then privacy and the practices of the government in regulation or companies is crucial. Much like the way we see telephones and TV in the past, as involving privacy, commercial, and government interests. Also, because search is now ubiquitous, politicians will seek to gain political advantage or grounds for industry regulation. Ordinary people should be concerned about how political and commercial information policies will affect their access the web.

CW: And you are studying the evolution of human information behavior. How far back are you going? Medieval libraries? Cavemen looking for the right painting?

AS: Obviously humans evolved information behaviors before preliterate societies through cave art, etc. Information behaviors evolved to help humans complete and cooperate, as the technologies evolved from cave art to the web. In fact, people may not change their information behaviors, but may have evolved over time to utilize a greater capacity for more complex information behaviors.

The Spink and Currier paper talks about the information behaviors of Darwin, Napoleon, and Casanova-all very effective people at finding and using information. And what we write about Casanova many people have found fascinating!

CW: Now you are being cruel! I’m going to have to renew my library card. Can you predict trends in behavior from your research? What’s next on the horizon?

AS: What’s next on the horizon is developing an understanding of how human information behaviors have evolved over human history, how they evolve over a person’s lifetime, how their search interactions develop over time, and how search in the aggregate is evolving over time. In other words, we need more longitudinal studies.

CW: Any bits of advice to practitioners about to attack the search tools on their sites? Lessons from web search?

AS: A key problem for practitioners is the lack of computer people trained in information and web retrieval, web design, and web usability. There is a lack of good trained people and not many industry consultants who really understand search. Search is much harder than most people think, and the design of effective search tools is even harder. Practitioners need to really test any search engines they consider buying. Many companies claim that their search engines are effective and the best, but provide little real evidence for their claims.

Be careful of the search engine that promises effectiveness and superiority based on a “single” feature, e.g., linking or clustering. There is no silver bullet feature. We don’t yet have Search engines that have adopted a more holistic attitude based on a real understanding of search, people’s information behavior and what is really effective. Whoever takes that path effectively will gain competitive advantage in the marketplace.

For More Information

Spink, A., & Currier, J. (2006). Emerging evolutionary framework for human information behavior. In: A. Spink & C. B. Cole (Eds.), New Directions in Human Information Behavior. Berlin: Springer (pp. 13-31).

Spink, A., & Cole, C. B. (2006). Human information behavior: Integrating diverse approaches and information use. Journal of the American Society for Information Science and Technology, 57(1) 25-35.

Spink, A., & Currier, J. (2006). Toward an evolutionary perspective on human information behavior: An exploratory study. Journal of Documentation, 62(2), 171-193.

Boxes & Arrows: When did you start thinking about “findability” as a concept? How is it different from the concepts you learned and applied in library science?

Peter Morville: I first used the word in a presentation at the 2002 Information Architecture Summit in Baltimore. Soon after, I wrote “The Age of Findability.” For me, findability is about crossing boundaries of discipline and medium.

Findability takes us beyond usability and information architecture into the realms of design, engineering and marketing. And it encompasses wayfinding and retrieval in physical and digital environments.

So findability builds on the foundation of library science and human-computer interaction, but addresses the new challenges and opportunities of social software, collective intelligence and ubiquitous computing.

B&A: What is “ambient findability?”

PM: Ambient findability describes a world at the crossroads of ubiquitous computing and the Internet in which we can find anyone or anything from anywhere at anytime. It’s not necessarily a goal, and we’ll never quite arrive, but we’re sure as heck headed in the right direction.

B&A: In a recent article about authority, you point out that Tim O’Reilly proclaimed the death of taxonomy. Do you agree with him?

PM: No. Unfortunately, Tim is suffering from apophenia. I think he caught it from Clay Shirky. I hope they both get well soon.

People have been predicting the end of hierarchy since the beginning of hierarchy. But it’s not going away. In fact, I dedicate a whole chapter to explore the hyperbole that swirls around social software and the Semantic Web. I make the case for a “sociosemantic web” that relies on the pace-layering of ontologies, taxonomies, and folksonomies to learn and adapt as well as teach and remember.

David Weinberger once noted “The old way creates a tree. The new rakes leaves together.” I’ve always found this to be a brilliant metaphor. Because we know what happens to those piles of leaves we shuffle through each fall. They rot. And they return to the earth. Where they become food for trees. Which come in many colors, shapes, and sizes. And live a really, really, really long time.

B&A: On the sigia mailing list, it was recently pointed out that your book wasn’t findable via the Safari Bookshelf under either “information architecture” or “interaction design.” When will these types of findability problems become nonexistent?

PM: As long as humans use words to communicate, findability will remain imperfect. But in our lifetimes, we can probably expect a few more modest innovations like full-text search, the PageRank algorithm, controlled vocabularies, and user-contributed metadata. Things may get a little better, but don’t expect big advances from the likes of artificial intelligence. Let’s just say I’m not holding my breath for the singularity.

On a practical note, I’m pleased to report the lemur (on Safari) is now filed under:

Internet/Online > Web Design
Internet/Online > Usability
Human-Computer Interaction > Interface Design
Human-Computer Interaction > Usability
Human-Computer Interaction > Information Architecture

Of course, it’s also showing up in all sorts of strange places, but that’s another story altogether.

B&A: In your book, you claim that users are often willing to sacrifice information quality for accessibility. Do users have enough awareness of authority to judge quality?

PM: My article on authority provoked a wonderful discussion on web4lib about this very question. My sense is that many adults lack the information literacy skills needed to cope with a mediascape that enables us to select our sources and choose our news. We grew up in an overly simplistic world of centralized authority with teachers and encyclopedias that taught us “the truth.”

Today’s kids are growing up amidst a web of social facts and collective intelligence where folksonomies flourish and the truth is a virus of many colors. I’m optimistic these kids will develop sophisticated skills for judging authority and quality and deciding who to trust and what to believe.

B&A: With information gaining on us, are we destined to become satisficers? And if so, is this a bad thing? We’re developing our own definitions of authority after all.

PM: We have always been satisficers. It confers competitive and evolutionary advantage. We satisfice to succeed. And I reject the conventional wisdom that suggests our information diet has been corrupted by the Web. To the contrary, the Web has radically improved global information access and source diversity and quality.

I can understand why an academic with access to vast libraries of books, journals, and licensed databases might sneer at the free Web. But these crown jewels of the ivory tower are unreachable by most people most of the time, and they always have been. Amid cries of “let them eat cake,” the Web gave bread and fruit and vegetables to the starving masses.

Of course, Google Print and Yahoo!’s Open Content Alliance are about to steal the crown jewels from the ivory tower, so we can all eat a balanced information diet, along with a healthy dose of free radical memes and mixed metaphors.

PM: Data is not dead, but I do agree with David Weinberger’s wicked smart insight about the blurring of boundaries between data and metadata. Just consider Amazon’s Search Inside the Book, which transforms content into searchable, indexed metadata.

Similarly, in social network analysis, I noted that we use people to find content and content to find people. A blog post can serve as destination content and as descriptive metadata that makes the author more findable.

B&A: In your book, you point out that the information in the Encyclopedia Britannica has a findability problem. If its findability were greater, would Wikipedia have a viable competitor on its hands?

PM: I think of the Encyclopaedia Britannica as a wonderful educational resource for kids. It explains important topics in a traditional manner that is clear, simple and safe. But I never use the EB, even though I have free, electronic access through my University of Michigan affiliation.

I did a great deal of research for my book. And I made extensive use of licensed bibliographic and full-text databases. But the Wikipedia was the single most useful source. Findability is only part of its success. It’s also strong in quality, currency and breadth of coverage.

As the world’s largest, most popular encyclopedia, the Wikipedia illustrates the efficacy of open source content creation and the power of collective intelligence. So, in short, the answer is no. Wikipedia has nothing to fear from EB.

B&A: Did you think about the findability of information within the book itself? What did you do to make it more findability-friendly?

PM: As I note in the preface, the book is meant to be read in linear style from start to end. We included some wayfinding devices like page numbers and a table of contents, but they’re not central to the user experience.

However, if we ever do a second edition, I’d push for a more detailed index, so I could use it to find what I wrote. In the meantime, I rely on the free Search on Safari (see the red box in the lower left) for detailed lookup.

Of course, as an author, what I really want for Christmas is to have my book indexed by Google Print and Yahoo!’s Open Content Alliance. I hope you’re reading this Tim.

B&A: What bugs you as being unfindable? What kind of information do you wish were more findable?

PM: A few weeks ago, I visited our local shopping mall for the first (and last) time this year. I went in search of shoes, but the store where I found them last time didn’t have my size. So I had to drag my body around the meatspace of the mall, and the whole time I just kept wishing that I could Google the Mall, and go home. I ended up finding the shoes online at Amazon.

B&A: I’m assuming you monitor how findable you are as a person. What do you do to ensure that you yourself are more findable online?

PM: Semantic Studios and findability.org are designed with findability and search engine optimization in mind. And my email address has been public for years, which means I can easily be found by friends and clients and stalkers and spammers. Sometimes, the real trick is becoming unfindable.

B&A: With the influx of wireless devices and new affordances, you note that the “user experience is increasingly out of control” and you suggest that we lose the C in HCI. Can you explain?

PM: The complexity of user experience in today’s environments is not expressed well in typical models of human-computer interaction. HCI approaches are optimal for applications and interfaces where designers exercise great control over form and function. HII (Human Information Interaction) approaches are optimal for networked, transmedia systems where control is sacrificed for interoperability and findability. At the crossroads of ubiquitous computing and the Internet, users may find and interact with objects through a variety of devices and interfaces. The context of use is difficult to predict and impossible to control. And so, the emphasis shifts from interface to experience, and from HCI to HII. Or at least that’s what I’m hoping to argue at CHI 2006. Wish me luck!

B&A: In your book, you often refer to William Gibson’s quotation, “The future exists today. It’s just unevenly distributed.” Do you have any predictions on where we might look for signs of ambient findability?

PM: Everyware is everywhere but we take this magic for granted. The fact that I can surf the Web at the beach and check email while driving is amazing, but what we’re really searching for is the impossible. I’m reminded of Howard Rheingold’s observation in Smart Mobs:

“People for whom pervasive computing is an abstraction will understand very clearly that the traditional barriers between information and material have changed when the air they breathe might be watching them.”

Of course, not long after this becomes possible, it will be considered mundane. We’re only fascinated by the future because we can never get there.

B&A: For many years, you were associated with the venerable polar bear. How does it feel to be associated with a lemur?

PM: What’s important is that the two get along well together, though the cheeky lemur sometimes gets a well-deserved cuff behind the ears.

Related information

Laughing Lemur Contest
Entries will be accepted through December 11, 2005.

Peter Morville is widely recognized as a founding father of information architecture. He co-authored the best-selling book, Information Architecture for the World Wide Web, and has consulted with such organizations as Harvard, IBM, Microsoft, and Yahoo!. Peter is president of Semantic Studios, co-founder of the Information Architecture Institute, and a faculty member at the University of Michigan. His work has been featured in many publications including Business Week, The Economist, Fortune, and The Wall Street Journal. Peter’s latest book, Ambient Findability, was published in 2005.

He blogs at findability.org.

Liz Danzico, editor in chief for Boxes and Arrows, is currently director of experience strategy for AIGA. She plays the role of managing editor for the online journals specifically, and, generally, oversees all online content and tools. She’s also the publications manager for the AIGA Press, a partnership with New Riders Publishers, publishing books that explore where design, business, and culture overlap.

Liz is an adjunct professor at the New School University, where she teaches design history. In past roles, she helped build and manage the information architecture team at Barnes & Noble.com. Prior to BN, she enjoyed being at Razorfish, where she managed the information architecture group for the New York office.

Her personal site can be found at bobulate.com.