Icon Analysis

by:   |  Posted on

“She gave up the search for the mouse settings icon in seconds and opted to just use the ridiculously over-sensitive mouse.”

An icon search task that lasts longer than anticipated can result in user annoyance or even premature abandonment. I once changed the mouse settings on my laptop to be overly sensitive, and had a colleague use it to show me a data analysis technique she had been working on. She immediately noticed and asked permission to change the settings. At my resolution of 1400×1050, the icons in the Windows control panel folder render at 16×16 pixels. In addition, I had the list pre-sorted by comment rather than application name. Not used to these settings or dealing with mouse preferences, she gave up the search for the mouse settings icon in seconds and opted to just use the ridiculously over-sensitive mouse while demonstrating her analysis technique.

You may think she was justified if only using my system for a short time. If so, you’d be surprised to know this was no small demo! It went on for almost a half an hour. She surfed the web to retrieve various files, used several applications, accessed her FTP space to download some of her own work, and showed the technique twice with different sets of user data. Scientist and user throughout, she sprinkled obscenities about the mouse amongst her thoughtful discussion of data analysis. I was astonished, and now far too afraid to tell her I had fooled with the mouse on purpose.

Two weeks later, I was discussing the analysis technique with another coworker and he said, “By the way, I heard your mouse is all messed up. I can fix that if you want.” Bad human computer interaction (HCI) experiences travel fast! The issue could have been avoided if only the mouse settings icon had been more identifiable.

Inability to discriminate one icon from another and/or find an icon in a set can be far more disastrous than my anecdote above. Systems used by first responders in hazardous materials incidents (see MARPLOT, for example) rely on icon design to signify entity classification (e.g. small icon of a schoolhouse) and level of critical danger to an entity (e.g. a school icon is painted red on a map). Immediately recognizing danger to a school amongst lumber yards, garbage dumps, and plant nurseries is imperative; any time-slip in the search and discrimination task could delay notification and evacuation of hundreds of children. How then can we diagnose problems with icons that fail in this regard?

Search and discrimination of icons

The human visual system is a complex mechanism that encodes information using many channels in two major pathways. The magnocellular pathway (M pathway, or “big neurons”) contains channels sensitive to gross shape, luminance, and motion. The parvocellular pathway (P pathway—“small neurons”) contains channels sensitive to color and detailed shape (Nicholls et al, 1992). In order to discriminate between two different visual signals—icons, in our case—the signals encoded in available channels must differ beyond some threshold. A common distinguishing technique is color. For example, try to find the red network settings icon on the right in figure 1.


Figure 1: Original icon list shown in the Windows control panel (left) and the icon list with the network icon highlighted red or feature-based search (right). Click to enlarge.

Searching by some distinguishing feature like color is called (not surprisingly) a feature-based search. Feature-based searches are limited in a few ways: their effectiveness drops if we apply a unique color to all icons in the image set and distinguishing by color only employs purposeful differences in only one of the two visual pathways (the P pathway). Additionally, icons tend to be small in a UI, thereby restricting differences in shape to “detailed shape” information—also encoded in the P pathway. Ideally, we would like to design icons that purposefully differ along channels in both M and P pathways.

Fig 2

Figure 2: Original Network Connections Icon with constituent M and P pathway representations.

An elegant technique to do this involves leveraging the core difference between the pathways. Large neurons are less densely packed in the retina of the eye than small ones. The spatial density leads to fundamentally different encodings of the visual image. Figure 2 shows an image of an icon that has been filtered to simulate the way it would be encoded in the M and P pathways.

Images filtered in this manner can be judged for distinctiveness along 2 pathway dimensions, assisting in economy of discrimination and search tasks. Distinctiveness in P pathway representations is easy enough to judge without the use of filtering techniques; designers weigh color and detailed shape decisions directly during the design process. The only tool a designer has to judge M pathway distinctiveness is the “squint test” (i.e. squint your eyes to obstruct sharp focus and rely mostly on dark and light values). However, the squint test is not very practical for HCI and Usability assessments; spatial frequency filtering is a better tool to simulate M pathway representations of icon images for evaluation purposes.

Spatial frequency filtering

The visual system maintains a set of scales that we associate with distance. If we see an object thought to have great size—say, a building—but that takes up little space on the retina (i.e. it looks very small), we immediately “perceive” it as being far away rather than perceiving it as a miniature building. The perception of scale is actually based on the encoding of visual spatial frequency (Schyns & Olivia, 1994).

This is interesting because you can encode images in specific spatial frequencies (Schyns & Olivia, 1999). View figure 3 from a foot away. Now stand back and view it from farther—say, 10 feet. Up close it is difficult to make out the image of Bill Frist in the center image. From farther away the image of Hillary Clinton disappears altogether in the center image. However, at both distances the outer images of Frist and Clinton are easily discernible. This phenomenon is based on our inability to perceive high frequency information from greater distances; if the image has no distinctive low frequency component, it simply disappears when viewed from a distance.

Fig 3

Figure 3: Hillary Clinton (left), frequency composite of Hillary and Bill Frist (center), and original Bill Frist image (right).

Not surprisingly, we hold specific spatial frequency registers for icons. Just as the color and shape choices for an icon design should be unique, so too should the frequency composition of the design. When a user searches through a UI in order to compare or find icons, his or her eyes jump all over the screen. Where eyes land are called fixation points, while the sharp eye movements are referred to as saccades. Users only see roughly 1.5 degrees of visual angle in sharp focus (roughly the size of your thumb nail held at arms distance); the rest of the image is processed in the M pathway and at lower spatial frequencies. At each fixation point, most of the icons in a UI fall outside of 1.5 degrees. The key is to filter the icon images to ensure that they differ in low spatial frequency so as to preserve their uniqueness during visual search. (Filtering methods discussed here are based on the work of Loftus and Harley, 2005, who used filtering to create representations of faces at a distance.)

The technique I show here requires the use of the R package (R Development Core Team, 2005) and the add-on package called “rimage” (Windows, Linux, and OSX versions are available here). Once you have downloaded and installed R, you can download and install the “rimage” addon from within the R program. (On Windows: Start R, then select packages » Install package(s). Choose a mirror from the dialog. Then select the “rimage” package.)

Filtering instructions

After the R program is set up and the rimage package has been installed, you are ready to start. Collect a set of icons you wish to analyze and put them all into a single image using your favorite image editing program, as shown in figure 4. Save the image as a jpeg.

Fig 4

Figure 4

Start the R program. Load the rimage library and the icon collection image into R using the following commands in the console window:

<br /> > library(rimage)<br /> > icons <- read.jpeg("address to your file”)<br />

Where “address to your file” is the full directory address where you saved the icon collection image. Make sure to enclose the address in quotes and use “/” instead of “” to signify subdirectories. Mine looks like this:

<br /> > icons <- read.jpeg("C:/Documents and Settings/Queen/My Documents/icon-collection.jpg”)<br />

Press Enter and then view the image in a display window by typing:

<br /> > plot(icons)<br />

Resize the window so that the images are full scale and not distorted. Now we’ll filter the images:

<br /> > plot(normalize(lowpass(icons,27.8)))<br />

Fig 5

Figure 5: Filtered icon set

Some explanation is necessary here. The number 27.8 defines the radius of the frequency filter in frequency space. I’ll spare you the math lesson and give you a short table of calculations that solve for radial lengths based on user distance from the screen (calculations based on size-distance-invariance equations; see Gilinsky, 1951).

 Icon Pixel Dimensions   Viewer Distance   Radius 
18 in.
24 in.
36 in.
18 in.
24 in.
36 in.
18 in.
24 in.
36 in.
18 in.
24 in.
36 in.

Using this table, you can see that I chose to assume the icons are 48×48 pixels in dimension and the viewer is 2 feet from the screen. As a general practice, filter the icons using all settings that might actually occur at use time and make sure that icons remain sufficiently unique (there are no studies that elaborate on what is sufficient—so be overly cautious).

I feel compelled to note that spatial frequency filtering is very different than just blurring the image; blurring removes detail that the M pathway relies on for recognition. Figure 6 shows the very different results of frequency filtering and blurring.


Figure 6: The filtered image (left) is far more representative of what a user actually sees than the blurred image (right). Extreme differences can be seen in icons with tight detailed patterns such as the second icon on the bottom row. Click to enlarge.

How effective are spatial frequency unique icons?

The following is a short study showing the benefits of using icons that have unique low spatial frequency compositions. 10 users were shown 20 icon images (called “trial icons”) of varied size. Simultaneously, they were presented with two additional icon images and asked to click on the icon that matched the trial icon, as shown in figure 7. Response times were recorded. The idea was to see if low frequency unique icons were easier to identify, and therefore result in faster response times.

Fig 7

Figure 7: Experiment screenshot

With each presentation of a trial icon, the match icon (fig. 7 – right) and distracter icon (fig. 7 – left) had either similar or different low frequency compositions. The response time data was then analyzed to determine if having all 3 icons contain similar low frequency compositions slowed responses. If responses were slower, the match icon was assumed to be more difficult to identify. A box plot of the resulting dataset is shown in figure 8.

Fig 8

Figure 8: Dataset box plot

As you can see, on average, users identified icons with unique low spatial frequency compositions faster than those with compositions similar to distracter icons. In fact, 75 percent of the time the fastest response times under normal conditions were just about average when frequency differences were present. The frequency-unique icons result in almost a half-second faster identification times. Summing that difference for every icon search task during use-time adds up to quite a bit of what could be critical decision-making time. Unique low-frequency compositions in icon designs make a noticeable difference.


  • MARPLOT: http://archive.orr.noaa.gov/cameo/marplot.html
  • Nicholls, J.G., Martin, A. R., and Wallace, B. G. (1992). From Neuron to Brain. Sinauer, 3rd edition.
  • Schyns, P.G., Oliva, A. (1994) From blobs to boundary edges: Evidence for time and spatial scale dependent scene recognition. Psychol Sci 5:195–200.
  • Schyns, P.G., Oliva, A. (1999) Dr. Angry and Mr. Smile: when categorization flexibly modifies the perception of faces in rapid visual presentations. Cognition 69:243–265.
  • Loftus, G. R., & Harley, E. M. (2005) Why is it easier to identify someone close than far away? Psychonomic Bulletin & Review, 12(1), 43-65.
  • Gilinsky, A.S. (1951) Perceived size and distance in visual space. Psychological Review, 58, 460-482.
  • R Development Core Team (2005) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, http://www.R-project.org.

Interaction Modeling

by:   |  Posted on

“The relationship between actions and cognitive processes is important because it explains user behavior and translates to supportive arguments for good design solutions.”

Interaction modeling is a good way to identify and locate usability issues with the use of a tool. Several methods exist (see Olson & Olson 1990 for a review of techniques). Modeling techniques are prescriptive in that they aim to capture what users will likely do, and not descriptive of what users actually did.

Most methods—descriptive or prescriptive—fail to incorporate the relationship between user actions and cognitive processes. Models of cognitive processing, for example, might attempt to explain how or why a particular task is mentally difficult, yet the difficulty does not directly relate to observable user actions. Conversely, descriptive techniques such as path metrics, click stream analysis, and bread crumb tracking take little or no account of cognitive processes that lead to those user actions. Continue reading Interaction Modeling