Wired has a short but interesting article about some researchers at Hewlett-Packard Labs who are studying how information flows through weblogs.
Using newly developed techniques for graphing the flow of information between blogs, the researchers have discovered that authors of popular blog sites regularly borrow topics from lesser-known bloggers — and they often do so without attribution.These findings are important to sociologists who are interested in learning how ideas grow from isolated topics into full-blown epidemics that "infect" large populations. Such an understanding is also important to marketers, who hope to be able to pitch products and ideas directly to the most influential people in a given group.
This "without attribution" aspect is being treated as a technical problem:
Indeed, the team at HP Labs found that when an idea infected at least 10 blogs, 70 percent of the blogs did not provide links back to another blog that had previously mentioned the idea.To get past this obstacle, the researchers developed techniques to infer where information might have come from, based on the similarities in text, links and infection rates.
A possibly-practical outcome of the work could be a search engine that's smarter about the origin of ideas, or at least the point at which the ideas were introduced to the web:
The researchers have incorporated their techniques into a search algorithm they call iRank. Unlike Google's PageRank algorithm, which ranks websites based on overall popularity, the iRank algorithm ranks sites based on how good they are at injecting ideas into the mainstream."A lot of sites that get listed by search engines as most relevant are not always the most relevant," said Adar. "For instance, Slashdot often gets listed at the top, but it's just an aggregator. I may want to go to the source."
I'm interested in this because I think a lot of the most frustratring and potentially damaging aspects of our society are tangled up with the oversupply of information that we're all trying to manage. Having convinced ourselves that information is valuable, we've built systems to go out and get lots of it, bring it home, lay it at our feet. Now what.
We have no time to sort through it all, is what, and I think this pops up everywhere in our culture. News aggragators. One hundred ballot initiatives. (I failed to vote on Super Tuesday, for the first time in a long time.) Loud-mouthed political pundits who skip past the analysis because they know that what we want is someone to give us the conclusions.
New technologies will erode this mountain and try to identify the kernels of information at the middle. Look at the success of Google and the iPod. Information managers. But they'll only go so far because at some point they need to read our minds, and our minds just say, "Wha?".
Maybe our great-grandkids will be able to assimilate and process information more quickly than we can, through practice. Maybe we'll learn to depend on filters that we trust. (How will we figure out who to trust? Today our techniques are crude: radio and TV hosts become popular by being vaguely entertaining, by acting dead-sure of their opinions, and by acting exasperated at supposed enemies, a triumvirate of forces that hold the universe together. Yeah, radio and TV hosts, but politicians, too. Anybody needing a populace.) Maybe processing information will only expose additional problems caused by each of us looking at drastically different slices of the world, a society fragmented.
Whatever the result, I think how we deal with the information glut will define this era. I dunno.