Something Entertaining, If You're Feeling Smart

Mark Bernstein, of Eastgate software and the powerful and intriguing Tinderbox program, was a guest-blogger here earlier this year. Last week, at the Web Science '11 conference in Koblenz, Germany, he made a presentation about the way the modern Web of internet-based knowledge could still fail.

By "fail" he means that we could see an end to the famous "long tail" pattern of net activity (and many other modern phenomena). According to long-tail principles, a few very popular sites will get a huge share of traffic or customers -- but then a very large number of minor entrants will each find the appropriate niche audience. Think of all the books on sale at Amazon or B&N, all the movies carried by Netflix, all the web sites around the world that don't get large audiences but find some people who are interested.

The point of Bernstein's presentation is that prevailing habits of creating links can make all the difference in the viability of the Web-based knowledge. If links are done the right way, he says, the long tail will indeed prevail and a wide variety of info will continue to find its proper audience. But if they are done the wrong way, the natural result could over time be a winner-take-all pattern, like what we've seen in other parts of life. A few main sites will receive even huger amounts of traffic; the vast majority of sites will get no traffic whatsoever and become invisible, as if they didn't exist at all.

The main part of his argument is that it wasn't inevitable that the Web would take today's long-tail shape, and it's not certain that this pattern will last. Eg:

>>Students of Web phenomenon are naturally tempted to regard the Web as part of Nature, that it unfolds as it always has and, more or less, as it ought or must. But the Web is still new, and quite recently it was entirely possible to envision that the World Wide Web would be quite different from the Web we know. As late as 1993, for example, expert consensus held that the cost of persuading a core of writers and publishers to contribute a small library's worth of information to a public docuverse might be a mere trillion dollars; not only was the estimate quantitatively off target, but the sign was wrong.<<

Don't try this if you're feeling logy. Or if the mention of a Poisson distribution makes you panic (he explains in context -- essentially, it's about how purely random changes in browsing and linking patterns affect the audience for a site). But if you're ready for an application of mathematical laws and models to modern habits of linking, browsing, Tweeting, and connecting, then by all means dig into what he says.

You can get a PDF that spells out his argument, or you can watch the animated slideshow he presented in Koblenz. He did it in verse: the lines of (sometimes doggerelish) poetry in the upper right hand corner of each page of the presentation. Then mathematical simulations of traffic patterns take over. Reportedly the presentation was greeted with a standing ovation. I guess you had to be there -- but I'll say that his analysis has made me think about the whole phenomenon of Web-based knowledge in a new way, and to consider changes in how I post and link items. I plan to say more about that later -- I "plan" to say more about so many things! But if you're feeling alert, check this out. For now, a screenshot from his presentation: