The “Tagging” book by Gene Smith is out. I am still awaiting a paper copy, but had a chance to look at the online version already. It looks really comprehensive, concise and covering all important tagging concepts. Which is not easy for such a moving target topic.

Besides, my elastic tag maps visualization is featured on p.102f. Nice!

PS: 12 days to go, wish me luck!

First one is the emerging topics histogram. I followed my own advice and vertically centered the stacked histogram. Additionally, I never liked those sharp edges, so now I do not only “fade out” tags visually, but also fade them in, resulting in a much more organic picture, and largely improved readability of the chart. New color scheme: old tags are cold, freshly introduced ones in warm color. This is all very much inspired by the fabulous last.fm charts by Lee Byron — thanks!

new interactive version here

and some pix:

On the left: the personal tags for the personal bookmarks — ordered by frequency — the containing box per is log-scaled so you get an impression of the long–tail position of a tag. Which means: Often used tags are large, bright and go to the top.

On the right: community tags for the tagged ressources.

In the middle: tags with the same name are connected. If a line is horizontal, the indvidual and the community essentially agree on the relevance of the tag for the ressources. The steeper it is — the larger the disagreement. If no line starts at a tag, it means it is not present in the other list

So in the picture above, you can see my tags. Same facts you can read from the picture: • “m.a.thesis” is a very often used, but private tag of mine. • The ressources I tag in general are mostly tagged with “design” by the community. I, however, use the tag “design” much less often. • “news”, “semantic web”, “webdev” are tags I use often, but not the community. etc.

And often course, you can click individual tags to see what the comparison is like for subsets of the bookmarks. That’s especially interesting for obscure tags like “guru” — you can see what the tagger “means” by looking at the distribution of the community bookmarks (in this case “design — art — programmer — artist”). Interesting!

Some more shots:

I wish I could say “click here for the interactive version” as usual — but unfortunately, del.icio.us offers a JSON API, but did not put a crossdomain.xml file on their server. Which means the visualization (which runs nicely on my harddisc) cannot load data when put in the web. Bummer. I hope I can figure something out.

So for now — I can only offer a download link. Click the index.html. You might have to adjust you Flash player security settings in order to load the community tags. Caveat: The application is still a bit buggy and pretty heavy concerning processor ressources.

For the impatient: » Check out the interactive version here

And here’s the explanation:

It has been shown before (for a plausibility argument, also check out the marvellous cloudalicious tool, where you can track tag proportions for any website on the web), that tag proportions for ressources stabilize over time. Which means that the tag cloud representing a tag profile for a resource does not change much, once a sufficient number of tags has been collected. In a folksonomy, this is generally considered a good sign, since this indicates a certain agreement on how to judge a certain ressource and what vocabulary to use.

For tagging individuals, and communities, this might — at first glance — hold true as well. Consider the following the visualization of a tagging community’s evolution, for example:

Each tag is assigned a band, with the thickness indicating the overall summed usage of a tag over time (time runs left to right). Thus, a vertical cut through the graph corresponds to taking a tag cloud snapshot at this time point. The vertical order is based on the overall frequency of the tags. The color is used to to give an impression of the long tail distortion — if all tags would appear equally often, you would see a linear transition from red to green instead of the skewed distribution. So — what do we see? Apparently, most of the bands seem to grow in parallel, indicating a stable growth proportion for all tags. Of course, we cannot see much for the smaller tags, and there are some edgy parts of the graph which might indicate different behavior at specific time points, but overall — pretty stable impression.

However, this does not make much sense. For individuals and communities, the topics of interest evolve over time, so there must be some hidden variability not captured by the visualization and the underlying linear model.

So I decided to provide an alternative visualization for the data based on a decay model, where tags “age” over time and finally get “forgotten” if they are not used anymore. This idea is loosely based on the Yules-Simon memory model for tag generation presented in this paper.

A radically different picture emerges. Not only does the overall shape now nicely display phases of community activity over time, but also the life cycle of single tags is much more transparent. You can rollover single layers highlight it and display the corresponding tag name. Great fun.

» Check out the interactive version here

What I am now curious about: – Is there a correlation between time-dependency and over-all frequency of tags? In other words, are frequent tags more evely distributed over time, whilst the low frequency tags tend to be more variable over time? – Is there a correlation between temporal synchronization and general co-occurrence? Which means — do related tags also appear and disappear together over time?

I think the answer is YES to both questions, but that would definitely need some statistical analysis (any bored neuroscientists around to help me? ;)

To-dos for the visualization: – Implement a slider, so you can see how a linear and decayed tag cloud would have looked like at a specific time point. – Stamen got it right: Maybe I should have plotted from the vertical center. Or at least provide an optional inversion of the sorting. Because right now, all the top (green) layers are really distorted, making visual analysis really hard. – Put some numbers on the axis – Show single tagging events on rollover. Or even “unfold” the layer to improve readability and avoid misconceptions.

A PHENOMENOLOGICAL FRAMEWORK FOR THE RELATIONSHIP BETWEEN THE SEMANTIC WEB AND USER-CENTERED TAGGING SYSTEMS

D. Grant Campbell Faculty of Information and Media Studies University of Western Ontario
London, Ontario N6A 5B7, Canada

Abstract This paper uses Husserl’s theory of phenomenology to provide a model for the relationship between user-centered tagging systems, such as del.icio.us, and the more highly structured systems of the Semantic Web. Using three aspects of phenomenological theory—the movement of the mind out towards an entity and then back in an act of reflection, multiplicities within unity, and the sharing of intentionalities within a community—the discussion suggests that both tagging systems and the Semantic Web foster an intersubjective domain for the sharing and use of information resources. The Semantic Web, however, resembles traditional library systems, in that it relies for this intersubjective domain on the conscious implementation of domain-centered standards which are then encoded for machine processing, while tagging systems work on implied principles of emergence.

For the impatient: HERE’S THE LINK

(Update again: The latest version can be found here)

And for the curious: Here’s the explanations:

What are tags? Tags are personally chosen, free-form keywords assigned to digital contents. So instead of putting a bookmark into a folder, you might assign it the tags “photography nature germany lomo”. If later you are looking for sites about photography, you will find it again under this term, but also if you are searching for “nature”. Another nice aspect is that your tagging creates an annotation to the existing content. If you share these with others (e.g. via a public bookmarking service such as delicious), everybody benefits by discovering new sites and getting better matches for their searches.

You can find more info about tagging e.g. at Wikipedia.

What are tags clouds? Tag clouds represent a whole bunch of tags as weighted lists. The more often a tag has been used, the larger it will be displayed in the list. This can be used to both characterize single users, webpages, as well as whole communities.

As an example, see my delicious tag cloud here:

Tag clouds can also be used for navigation: Click one of the tags and you will come to a web page displaying all of my bookmarks matching this tag.

What’s the problem with tags clouds?

Tag clouds are nice and really popular, but still there is quite some room for improvement: * Tag clouds are not suited for long tail navigation: By simply adding up over time, a certain pattern consistently emerges: there will be a some dominating tags (the “big head”) and a vast number of rarely used tags (the “long tail”). Whilst the “big head” tags remain pretty constant over time and broadly characterize your interests, the “long tail” contains all the variety of things you encounter. Tag clouds visually prioritize the “big head”. However, both for browsing and for searching, access to the long tail is vital, since this is where the real information is contained. * Summing up over time does not represent the dynamics of interests:Additionally, it can be questioned if merely summing up tags is the right approach in general. How about topics you were interested in, but now you aren’t anymore? Or conversely, very recent interests, which are pretty important to you but haven’t been tagged often enough to show up in the cloud? To solve this problem, Chirag Mehta had the nice idea of implementing tag clouds with a time slider. However, if you look at these, another problem becomes evident: * Tag clouds are not suitable for animation: This is due to their alphabetical list order and visual messiness. Since every tag’s position in a tag cloud is defined by its predecessor’s size and position, things start jumping around once you start scaling tags. So tag clouds are not really suited to display the dynamical nature of tagging structures — how tags appear and disappear. * Tag clouds are ordered the wrong way: Tags denote concepts. As such, they have meaningful relations to each other. Tag clouds are ordered alphabetically or by size — it would be much more effective, if tags that belong together could also be presented together. Some of these relations can be deduced automatically, by observing how tags are used: Some tags might always appear together, others sometimes and others never. If tags co-occur frequently or have many common “neighbors”, you can be sure the concepts denoted will be related in some manner.

So whatcha you gonna do about it?

All these issues lead me to developing a mapping algorithm to analyse and display tag structures based on how tags occur together. Technically, it is based on a vector-space model, where each tagging action is assigned a point in a high-dimensional vector space. By applying the dimensionality reduction algorithms PCA (Principal Components Analysis) and CCA (Curvilinear Component Analysis), I calculate a two-dimensional map, which places frequently co-coccurring tags close together. Additionally, covariance values for all tags are stored, so I know exactly how “related” each tag is to the others.

This information can be used to display both maps as well as lists, which are ordered by “relatedness”. You can play around with some of the maps in the interactive demo.

In its initial state, all tags are scaled according to their frequency. Clicking a tag will transform the map or list and will bring all related tags to the front according to their degree of “relatedness”. Tags are colored according to their “freshness”. Tags are considered fresh if their average usage has increased over the last 30 days.

To see how that freshness measure changes over time, there is an animated version as well described in an earlier post.

When clicking the 1D tab, you will find a list representation of the same information. The interaction principle is the same.

Additionally, I added two little diagrams depicting the “freshness” distribution and the number of occurrences for each tag. These will be filtered once you click a tag.

Insights gained and how to turn these experiments into usable interfaces

I would love to write a little bit about these issues, but this post is already long enough. I hope I will find the time in the next couple of days to write a follow-up!

A first glimpse into the dynamical nature of tag structures.

I will start with an overview of my subjective must-reads for now (all linked to citeulike):

Position Paper, Tagging, Taxonomy, Flickr, Article, ToRead Marlow C, Naaman M, Boyd D, Davis M Produced for the WWW06 workshop, this paper was an essential contribution to characterizing different tagging systems and the resulting tagging dynamics.

The Structure of Collaborative Tagging Systems Golder S, Huberman BA A seminal reference paper introducing tagging in the scientific community. The most notable contribution are the insights the authors provide into the emergence of stable patterns in tag proportions over time.

Folksonomies: Tidying up Tags? Guy M, Tonkin E A discussion of principle shortcomings of tagging and possible resolutions.

The vocabulary problem in human-system communication Furnas GW, Landauer TK, Gomez LM, Dumais ST Although not directly applicable to tagging, this paper from the late 80s is the baseline for experiments with free-form naming and annotation in the digital realm.

Magic Words: How Language Augments Human Computation Clark A A great discussion of words as cognitive tools from a cognitive science and philosophy perspective.

Collaborative Tagging and Semiotic Dynamics Cattuto C, Loreto V, Pietronero L An inspiring discussion of the temporal dynamics of tagging systems. The core contribution is a very simple probabilistic model based on Yule-Simon memory model. Although many questions remain open, this model has some direct consequences on dealing with tags in user interfaces, which I will elaborate on in a later post.