Description
This is an extended version of the April IAC talk that adds a case study about a B2B medical supply website and live answers to questions from the audience. Presented by Joseph Busch, Taxonomy Strategies’ principal analyst at a “Taxonomy Talk” Discord Community webinar on June 4, 2024.
Type: Webinar
Venue: Taxonomy Talk Discord Community
Date: June 4, 2024
Broad Topic: Information Architecture
Watch Presentation
Extended Edition
Concise Version of Presentation
Caption
In the same way we obsess over keyword rankings, do you wonder if your web content is organized effectively? Do you think about how to optimize the categories and taxonomic structure?
While we know we should proactively analyze overall taxonomy effectiveness—the distribution of content to categories, number of views per category, etc.—there is a tendency to focus on the content, and not the categories.
This short talk at the Information Architecture Conference about how to do taxonomy assessment as an integral part of website instrumentation. In the talk Joseph briefly describes how to flip analytics to get the category-level analysis, how to construct a set of qualitative practices that can generate taxonomy metrics that identify categories to merge or split, and some specific scorecards against which your taxonomy can be evaluated.
Video Script
Download Video Script Word (DOC)
Slide 1
In the same way we obsess over keyword rankings, do you wonder if your content is organized effectively and how to optimize the categories and taxonomic structure? While we know we should proactively analyze overall taxonomy effectiveness—the distribution of content to categories, number of views per category, etc.—there is a tendency to focus on the content, and not the categories. In this talk I will describe how to do taxonomy assessment periodically and ongoing as an integral part of website instrumentation. How to flip analytics to get the category level analysis, how to construct a set of qualitative practices that can generate taxonomy metrics that identify categories to merge or split, and some specific scorecards against which your taxonomy can be evaluated.
Slide 2: How to flip analytics to get the category level analysis
The goal of analytics is to count how many—how many users, how many uses and how many conversions. How many items are tagged with each category? We don’t expect all categories to be used at the same rate, but if some categories are never or infrequently used, then you should ask whether this category is useful or not. Generally, we don’t think it’s useful to have categories with zero uses, and to be sure to never ever expose categories with no content to users. It’s also helpful to review search logs (both site search and organic search) to see whether users are typing in the category labels or not. If categories exist in titles, headings, or first or last sentences or paragraphs in a content item, then there’s less value to adding these categories because they are already going to be exposed by the search engines.
Slide 3: How evenly does the taxonomy divide the content? What to merge or to split
You should not expect content to be distributed evenly across taxonomy categories. The natural distribution of content (or the words in text) is more likely to follow a Zipf distribution. It is usually found in the categorization of content collections that the most frequently used category has twice as much content as the next most frequent category, three times as much the third most frequent category, and so on. Examining the distribution of content by category and comparing this to the ideal distribution can be a clue to what categories could be candidates to merge or split—categories that exceed the expected frequency (above the curve) are candidates to split, and categories that are less frequent than expected (below the curve) are candidates to merge. The top chart shows the distribution of content types from the Library of Congress catalog compared to the ideal distribution. The bottom chart shows the distribution of content types from a corporate intranet compared to the ideal distribution.
Slide 4: How does taxonomy “shape” match that of content? What to merge or to split
Comparing the number of categories that cover a “broad topic” and the number of content items indexed by those categories across a collection is another way to identify candidate areas that could be streamlined or expanded. The table shows an analysis of 25,000 items sampled from ERIC a database of education research and information. For each broad topic, the relative number of terms is compared to the relative number of documents. Ideally, those percentages should be more or less in sync. The items shaded in red identify those where the percentages are not. Either there are too few or too many terms in the broad category, thus candidates for merging or splitting.
Slide 5: How intuitive (repeatable) are the categorizations?
It is important that the taxonomy is usable meaning that over time items are consistently categorized into the same categories. This is particularly important with product taxonomies whether for internal use in a bricks and mortar store or for merchandising products in an online store, especially when there is a large assortment of products. An effective way to test usability is card sorting or “find it” testing (usually called tree testing). The items to test should be the best-selling (high volume) or most profitable (high value) products. (By the way, the same criteria can be applied to content). The goal is to identify items that are frequently categorized differently in the test so that steps can be taken to improve the taxonomy. This may be to change the category label, change the category placement in the taxonomy, or to cross-market the category (that is, include the category in more than one place in the taxonomy).
Slide 6: Taxonomy evaluation scorecards
It’s helpful to have a standard set of criteria for evaluating taxonomies. This is sometimes called a checklist or scorecard. The fancy label is “heuristics”. We divide this into four areas—taxonomy structure, category labels, category attributes (that’s additional information associated with categories), and implementation.
Slide 7: Taxonomy evaluation scorecards
Classification structure heuristics are focused on taxonomy structure. Do the grouping principles make sense and are they fit for purpose? Are the categories mutually exclusive? Are the groups homogeneous? Are terms within each group consistent? Is the depth and breadth of the taxonomy appropriate for the use? Guidelines for these characteristics can be found in the relevant standards. These would be ISO 25964 Thesauri and interoperability with other vocabularies and NISO Z39.19 Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies. (References to these standards is on the Resources slide which is coming up in a minute).
Slide 8: Taxonomy evaluation scorecards
Category labeling heuristics are focused on naming conventions for taxonomy categories. Are the taxonomy labels in a consistent format in terms of capitalization, use of special characters, term order, punctuation, etc.? Are there editorial guidelines and have they been applied?
Slide 9: Taxonomy evaluation scorecards
Attributes heuristics are focused on providing additional information associated with categories to more clearly distinguish them such as definitions, and to support their use, especially in product catalogs. They may also support automated or semi-automated tagging.
Slide 10: Taxonomy evaluation scorecards
Implementation heuristics are focused on using the taxonomy for a specific purpose such as for tagging or to support search and retrieval. They focus on integration and usability.
Slide 11: Other methods
There are other ways to assess and test taxonomies such as user surveys and tagging some content. But these are topics for another day.
Slide 12: Resources
Here are some resources for more information.
Slide 13: Summary
In this short talk, I’ve described some quantitative and qualitative methods to assess taxonomy effectiveness and performance and shown some examples of these analyses and heuristics. Please let me know any questions you have now or write to me any time and I’ll do my best to answer them. Also, let me know methods that you’ve found effective in assessing and testing taxonomies in your practice. Thanks.