<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Metrics for Heuristics: Quantifying User Experience (Part 1 of 2)</title>
	<atom:link href="http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/feed/" rel="self" type="application/rss+xml" />
	<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/</link>
	<description>Boxes and Arrows is devoted to the practice, innovation, and discussion of design; including graphic design, interaction design, information architecture and the design of business.</description>
	<lastBuildDate>Wed, 15 May 2013 11:41:04 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	
	<item>
		<title>By: masoodnasser</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6130</link>
		<dc:creator>masoodnasser</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6130</guid>
		<description><![CDATA[Nice article Andrea. Providing a context is invaluable while defining Heuristics in the first place. Though you have mentioned that Rubinoffs chart is customizable, there are Couple of things I would like to add. Business is left out in a gazillion of heuristics that I have seen to date. While some may argue that every one of the 4 given broad categories affect business, in my opinion it is not so.

The entire purpose of a customer facing site is to realize the business objectives, and that should reflect in the design. When I say business, it means not only the Dollars that you are expecting. It can also include future strategies, and time to market of those strategies. Is your IA designed in a way to facilitate new additions, what would be the time to launch new campaigns? This involves not only looking at the site, but also the entire design development process including the CMS. This is just to name a few.

If you look at the site from a Customer experience perspective, then each and every “touch point” should be accounted for and the experience should be checked against metrics. How you define those metrics is an interesting challenge, more difficult than the actual rating and evaluating process. Waiting for the next part of this article..]]></description>
		<content:encoded><![CDATA[<p>Nice article Andrea. Providing a context is invaluable while defining Heuristics in the first place. Though you have mentioned that Rubinoffs chart is customizable, there are Couple of things I would like to add. Business is left out in a gazillion of heuristics that I have seen to date. While some may argue that every one of the 4 given broad categories affect business, in my opinion it is not so.</p>
<p>The entire purpose of a customer facing site is to realize the business objectives, and that should reflect in the design. When I say business, it means not only the Dollars that you are expecting. It can also include future strategies, and time to market of those strategies. Is your IA designed in a way to facilitate new additions, what would be the time to launch new campaigns? This involves not only looking at the site, but also the entire design development process including the CMS. This is just to name a few.</p>
<p>If you look at the site from a Customer experience perspective, then each and every “touch point” should be accounted for and the experience should be checked against metrics. How you define those metrics is an interesting challenge, more difficult than the actual rating and evaluating process. Waiting for the next part of this article..</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: andreawiggins</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6131</link>
		<dc:creator>andreawiggins</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6131</guid>
		<description><![CDATA[While I certainly agree that business goals are another key point of measurement and evaluation, I did not include them here because the focus of the user experience audit is the users&#039; goals.  I think everyone can agree that business and user goals are strongly related; in my personal philosophy of design, if a site does not meet user needs, it doesn&#039;t matter what the business goal behind it was because it cannot have succeeded at that goal.  Regardless of how user-centric or business-centric your perspective, my experience has been that the same web analytic measures often define success from both perspectives.

You make an excellent point of other ways in which web analytics can be used to improve site management strategies, and I agree wholeheartedly that evaluation of multiple aspects of a site should be an up-front consideration.  I look forward to more people adopting your perspective on the value of incorporating measurement in the entire iterative design process!]]></description>
		<content:encoded><![CDATA[<p>While I certainly agree that business goals are another key point of measurement and evaluation, I did not include them here because the focus of the user experience audit is the users&#8217; goals.  I think everyone can agree that business and user goals are strongly related; in my personal philosophy of design, if a site does not meet user needs, it doesn&#8217;t matter what the business goal behind it was because it cannot have succeeded at that goal.  Regardless of how user-centric or business-centric your perspective, my experience has been that the same web analytic measures often define success from both perspectives.</p>
<p>You make an excellent point of other ways in which web analytics can be used to improve site management strategies, and I agree wholeheartedly that evaluation of multiple aspects of a site should be an up-front consideration.  I look forward to more people adopting your perspective on the value of incorporating measurement in the entire iterative design process!</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: austingovella</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6132</link>
		<dc:creator>austingovella</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6132</guid>
		<description><![CDATA[Masood,

How would you formulate some of the things you mention as heuristics we could apply for sites?]]></description>
		<content:encoded><![CDATA[<p>Masood,</p>
<p>How would you formulate some of the things you mention as heuristics we could apply for sites?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: fred_beecher</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6133</link>
		<dc:creator>fred_beecher</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6133</guid>
		<description><![CDATA[Andrea,

At the &#039;06 IA Summit, Steve Mulder gave a presentation about bringing more science to persona creation, in which he advocated (and this is a gross oversimplification here) that personas be developed and then validated/updated through surveys &amp; research. Your insightful comments about using analytics data to validate a heuristic analysis got me to thinking that this same data could be used to either create or validate personas. I&#039;m not sure if this is something you&#039;ll be covering in Part 2, but it seems like another good use for this data.

- F.]]></description>
		<content:encoded><![CDATA[<p>Andrea,</p>
<p>At the &#8217;06 IA Summit, Steve Mulder gave a presentation about bringing more science to persona creation, in which he advocated (and this is a gross oversimplification here) that personas be developed and then validated/updated through surveys &amp; research. Your insightful comments about using analytics data to validate a heuristic analysis got me to thinking that this same data could be used to either create or validate personas. I&#8217;m not sure if this is something you&#8217;ll be covering in Part 2, but it seems like another good use for this data.</p>
<p>- F.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: michaelbeavers</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6134</link>
		<dc:creator>michaelbeavers</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6134</guid>
		<description><![CDATA[I agree with Fred--and by extension, Steve Mulder.  Personas can be a highly accurate starting point for fully assessing tasks and content needs through on behavioral research.  There really should be some effort to validate the data by comparing what research respondents are saying within the framework of issue categorization with Web analytics and analysis of behavior in a lab or other settings appropriate for contextual inquiry.

As an example, I&#039;m in the middle of evaluating the impact of AJAX and social/community architectures on a client&#039;s particular industry and marketing landscape.  We have performed a preliminary round of research in three cities and have established enough data to suggest 2 solid personas, 1 emerging persona type, and suggestions or mention of about 4 others.  Taking the first solid persona types, it was very easy to see them represented on YouTube and blogs as self-publishing persona types directly impacting my client&#039;s brand.  However, my client has been able to present little to no meaningful Web analytic data for their current site.

The good news is that we have some observable persona behavior for validation out there on the Web, but Web analytics data from my client&#039;s site would have helped us paint a more complete picture.  Without them, our heuristic analyses are based more on opinion than quantifiable activity--and I suspect would have taken us further in validating our persona types and helped to reduce our overall research timeframe.]]></description>
		<content:encoded><![CDATA[<p>I agree with Fred&#8211;and by extension, Steve Mulder.  Personas can be a highly accurate starting point for fully assessing tasks and content needs through on behavioral research.  There really should be some effort to validate the data by comparing what research respondents are saying within the framework of issue categorization with Web analytics and analysis of behavior in a lab or other settings appropriate for contextual inquiry.</p>
<p>As an example, I&#8217;m in the middle of evaluating the impact of AJAX and social/community architectures on a client&#8217;s particular industry and marketing landscape.  We have performed a preliminary round of research in three cities and have established enough data to suggest 2 solid personas, 1 emerging persona type, and suggestions or mention of about 4 others.  Taking the first solid persona types, it was very easy to see them represented on YouTube and blogs as self-publishing persona types directly impacting my client&#8217;s brand.  However, my client has been able to present little to no meaningful Web analytic data for their current site.</p>
<p>The good news is that we have some observable persona behavior for validation out there on the Web, but Web analytics data from my client&#8217;s site would have helped us paint a more complete picture.  Without them, our heuristic analyses are based more on opinion than quantifiable activity&#8211;and I suspect would have taken us further in validating our persona types and helped to reduce our overall research timeframe.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: andreawiggins</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6135</link>
		<dc:creator>andreawiggins</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6135</guid>
		<description><![CDATA[I can&#039;t agree more regarding the use of web analytic data to inform persona development.  While I didn&#039;t really get into that topic in this article, I did talk about personas and usability in a recent similarly-themed but differently-focused white paper, &quot;Data Driven Design: Leveraging Analytics to Improve your Website Overhaul,&quot; which is available at http://www.enlighten.com/pdfs/wp_analytics_08_06.pdf.]]></description>
		<content:encoded><![CDATA[<p>I can&#8217;t agree more regarding the use of web analytic data to inform persona development.  While I didn&#8217;t really get into that topic in this article, I did talk about personas and usability in a recent similarly-themed but differently-focused white paper, &#8220;Data Driven Design: Leveraging Analytics to Improve your Website Overhaul,&#8221; which is available at <a href="http://www.enlighten.com/pdfs/wp_analytics_08_06.pdf" rel="nofollow">http://www.enlighten.com/pdfs/wp_analytics_08_06.pdf</a>.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: andreawiggins</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6136</link>
		<dc:creator>andreawiggins</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6136</guid>
		<description><![CDATA[Someone mentioned trouble with the direct link to the PDF; you can also just go to http://www.enlighten.com and there&#039;s a link right from the home page.]]></description>
		<content:encoded><![CDATA[<p>Someone mentioned trouble with the direct link to the PDF; you can also just go to <a href="http://www.enlighten.com" rel="nofollow">http://www.enlighten.com</a> and there&#8217;s a link right from the home page.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: ccollingridge</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6137</link>
		<dc:creator>ccollingridge</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6137</guid>
		<description><![CDATA[Andrea,

I think all you&#039;d need to do to repair the link is remove the full stop (period) after .pdf

Chris.]]></description>
		<content:encoded><![CDATA[<p>Andrea,</p>
<p>I think all you&#8217;d need to do to repair the link is remove the full stop (period) after .pdf</p>
<p>Chris.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: andreawiggins</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6138</link>
		<dc:creator>andreawiggins</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6138</guid>
		<description><![CDATA[Chris, you&#039;re right on--the link is actually fine, but the period at the end of the sentence breaks it.  I can&#039;t edit the link to fix it, so I&#039;ll retype it:

http://www.enlighten.com/pdfs/wp_analytics_08_06.pdf]]></description>
		<content:encoded><![CDATA[<p>Chris, you&#8217;re right on&#8211;the link is actually fine, but the period at the end of the sentence breaks it.  I can&#8217;t edit the link to fix it, so I&#8217;ll retype it:</p>
<p><a href="http://www.enlighten.com/pdfs/wp_analytics_08_06.pdf" rel="nofollow">http://www.enlighten.com/pdfs/wp_analytics_08_06.pdf</a></p>
]]></content:encoded>
	</item>
	<item>
		<title>By: informationarchitect</title>
		<link>http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6139</link>
		<dc:creator>informationarchitect</dc:creator>
		<pubDate>Wed, 30 Nov -0001 00:00:00 +0000</pubDate>
		<guid isPermaLink="false">http://boxesandarrows.com/metrics-for-heuristics-quantifying-user-experience-part-1-of-2/#comment-6139</guid>
		<description><![CDATA[Andrea,

Very interesting article. I fully agree with part 2 and I agree with most points of part 1. However, I have strong doubts on the way you define branding and separate content, usability and branding from each other: 

&lt;blockquote&gt;&quot;&lt;b&gt;Branding:&lt;/b&gt; 
The site provides visitors with an engaging and memorable experience.
Graphics, collaterals and multimedia add value to the experience.&quot;&lt;/blockquote&gt;

I think a website provides an engaging and memorable experience if it is 
a) identifiable in its character purpose and value
b) pleasing 
c) memorable
It is easily identifiable if it has a clear purpose that it communicates, if it is consistent in its appearance, and if follows the guidelines of corporate design if it belongs to a bigger corporation.
It is memorable it solved my problem quicker and better than expected.
Graphics and multimedia should follow the corporate guidelines and be highly relevant if used within contents. If they&#039;re used just to please the user they create mistrust and annoyance.

&lt;blockquote&gt;&quot;&lt;b&gt;Usability&lt;/b&gt;
The site prevents errors and helps the user recover from them.
The site helps its visitors accomplish common goals and tasks.&quot;&lt;/blockquote&gt;

From my experience usability is a crucial element of the overall brand experience. If you go to a website that doesn&#039;t work, you have a bad brand experience. First of all it needs to work. People hardly pay attention to design. They feel it, but they don&#039;t really notice it. Design is important though, as it&#039;s critical in the first few seconds, where the user decides to stay or go. Good design doesn&#039;t mean the user stays. Again here usability is more important, but it helps building trust during the first judgment on the quality of the website.

Usability tests deliver good results within the test environments, they hardly assess one crucial moment of user experience: The first sight and the question: Should I stay or should I go. This moment is hard to test.

That&#039;s maybe why usability consultants started developing interest for branding and design in general (see Norman). Coming from a big brand company it&#039;s quite refreshing to see a usability expert get all excited about the matter:

http://video.google.com/videoplay?docid=5654878583447435228 

&lt;blockquote&gt;&quot;&lt;b&gt;Content&lt;/b&gt; 
Link density provides clarity and easy navigation. 
Content is structured in a way that facilitates the achievement of user goals.
Content is appropriate to customer needs and business goals.&quot;&lt;/blockquote&gt;

Again here the separation and usability and content is quite fuzzy. Structuring content is not content, it&#039;s IA, and as such rather a matter usability. Maybe I misread the separation, but I&#039;d understand the three categories more like this:

Brand &gt; Usability &gt; Content - And yet that&#039;s still not quite correct.

&lt;b&gt;What&#039;s my point? In as short as I can:&lt;/b&gt; 
1. Contemporary branding = interface (of a company or product - depending on weather you look at corporate branding or product branding), as nowadays brand is defined as an overall experience one makes in connection with a product. Best, i.e. most memorable and marketable (inciting word of mouth) experiences are interactive ones.
2. The interface is first of all a matter of usability. Products need to work. In the end the Interfaces get skinned adapting corporate guidelines. But only bad corporate guidelines intrude on the product.
3. An interactive product doesn&#039;t need too much skinning, its Interface shapes through the inert functionality of what it&#039;s built for (see craigslist, delicious, ebay, reddit).
4. The Internet has changed and will continue to change our understanding of quality, brand and communication. And – at least form the consumer’s point of view – in a good way.
5. Brand experience IS measurable. With web stats and usability tests on one side and by crosschecking with the corporate design manual on the other side.
6. Brand is still widely misunderstood as a superficial discipline that starts with the logo, leads into an expensive brand manual that costs millions defining a couple of grids, colors and fonts and ends with an emotional ad on TV.
7. Brand on the Internet does not equal to special graphic effects. The most special effect a website can make is that it&#039;s useful and easy to use. If in plus it looks good. That&#039;s branding. Branding and usability are not contradictory. They used to be quarreling twins. But they have grown up and now they can&#039;t be without each other anymore.

Well that was not so short... Anyway, I hope I&#039;ve not been too intrusive with my little speech here. But I am excited about that matter for over 4 years now, and as I am currently writing a book on usability and branding, I think this articles and the comment section provide a wonderful opportunity to get more insights in a matter that is widely ignored.]]></description>
		<content:encoded><![CDATA[<p>Andrea,</p>
<p>Very interesting article. I fully agree with part 2 and I agree with most points of part 1. However, I have strong doubts on the way you define branding and separate content, usability and branding from each other: </p>
<blockquote><p>&#8220;<b>Branding:</b><br />
The site provides visitors with an engaging and memorable experience.<br />
Graphics, collaterals and multimedia add value to the experience.&#8221;</p></blockquote>
<p>I think a website provides an engaging and memorable experience if it is<br />
a) identifiable in its character purpose and value<br />
b) pleasing<br />
c) memorable<br />
It is easily identifiable if it has a clear purpose that it communicates, if it is consistent in its appearance, and if follows the guidelines of corporate design if it belongs to a bigger corporation.<br />
It is memorable it solved my problem quicker and better than expected.<br />
Graphics and multimedia should follow the corporate guidelines and be highly relevant if used within contents. If they&#8217;re used just to please the user they create mistrust and annoyance.</p>
<blockquote><p>&#8220;<b>Usability</b><br />
The site prevents errors and helps the user recover from them.<br />
The site helps its visitors accomplish common goals and tasks.&#8221;</p></blockquote>
<p>From my experience usability is a crucial element of the overall brand experience. If you go to a website that doesn&#8217;t work, you have a bad brand experience. First of all it needs to work. People hardly pay attention to design. They feel it, but they don&#8217;t really notice it. Design is important though, as it&#8217;s critical in the first few seconds, where the user decides to stay or go. Good design doesn&#8217;t mean the user stays. Again here usability is more important, but it helps building trust during the first judgment on the quality of the website.</p>
<p>Usability tests deliver good results within the test environments, they hardly assess one crucial moment of user experience: The first sight and the question: Should I stay or should I go. This moment is hard to test.</p>
<p>That&#8217;s maybe why usability consultants started developing interest for branding and design in general (see Norman). Coming from a big brand company it&#8217;s quite refreshing to see a usability expert get all excited about the matter:</p>
<p><a href="http://video.google.com/videoplay?docid=5654878583447435228" rel="nofollow">http://video.google.com/videoplay?docid=5654878583447435228</a> </p>
<blockquote><p>&#8220;<b>Content</b><br />
Link density provides clarity and easy navigation.<br />
Content is structured in a way that facilitates the achievement of user goals.<br />
Content is appropriate to customer needs and business goals.&#8221;</p></blockquote>
<p>Again here the separation and usability and content is quite fuzzy. Structuring content is not content, it&#8217;s IA, and as such rather a matter usability. Maybe I misread the separation, but I&#8217;d understand the three categories more like this:</p>
<p>Brand &gt; Usability &gt; Content &#8211; And yet that&#8217;s still not quite correct.</p>
<p><b>What&#8217;s my point? In as short as I can:</b><br />
1. Contemporary branding = interface (of a company or product &#8211; depending on weather you look at corporate branding or product branding), as nowadays brand is defined as an overall experience one makes in connection with a product. Best, i.e. most memorable and marketable (inciting word of mouth) experiences are interactive ones.<br />
2. The interface is first of all a matter of usability. Products need to work. In the end the Interfaces get skinned adapting corporate guidelines. But only bad corporate guidelines intrude on the product.<br />
3. An interactive product doesn&#8217;t need too much skinning, its Interface shapes through the inert functionality of what it&#8217;s built for (see craigslist, delicious, ebay, reddit).<br />
4. The Internet has changed and will continue to change our understanding of quality, brand and communication. And – at least form the consumer’s point of view – in a good way.<br />
5. Brand experience IS measurable. With web stats and usability tests on one side and by crosschecking with the corporate design manual on the other side.<br />
6. Brand is still widely misunderstood as a superficial discipline that starts with the logo, leads into an expensive brand manual that costs millions defining a couple of grids, colors and fonts and ends with an emotional ad on TV.<br />
7. Brand on the Internet does not equal to special graphic effects. The most special effect a website can make is that it&#8217;s useful and easy to use. If in plus it looks good. That&#8217;s branding. Branding and usability are not contradictory. They used to be quarreling twins. But they have grown up and now they can&#8217;t be without each other anymore.</p>
<p>Well that was not so short&#8230; Anyway, I hope I&#8217;ve not been too intrusive with my little speech here. But I am excited about that matter for over 4 years now, and as I am currently writing a book on usability and branding, I think this articles and the comment section provide a wonderful opportunity to get more insights in a matter that is widely ignored.</p>
]]></content:encoded>
	</item>
</channel>
</rss>
