Wednesday, July 23, 2008

Knol: Google takes on Wikipedia

Just a few days ago, I was commenting on a New York Times article about Wikipedia's new approval system that the biggest problem with Wikipedia is anonymous authorship. By synchronous coincidence, Google unveiled Knol today, which is something of a cross between Wikipedia and Squidoo. It's most salient feature is that each entry will have a clearly identified author. They even allow authors to verify their identities using credit cards or phone directories.

It's a nice idea, since anonymous authorship is a a major factor in the adversarial nature of information retrieval on the web. Not only does the accountability of authorship inhibit vandalism and edit wars, but it also allows readers to decide for themselves whom to trust--at least to the extent that readers are able and willing to obtain reliable information about the authors. Without question, they are addressing Wikipedia's biggest weakness.

But it's too little, too late. Wikipedia is already there. And, despite complaints about its inaccuracy and bias, Wikipedia is a fantastic, highly utilized resource. The only way I see for Knol to supplant Wikipedia in reasonable time frame is through a massive cut-and-paste to make up for the huge difference in content.

Interestingly, Wikipedia does not seem to place any onerous restrictions on verbatim copying. However, unless a single author is 100% responsible for authoring a Wikipedia entry, it isn't clear that anyone can simply copy the entry into Knol.

I know that it's dangerous to bet against Google. But I'm really skeptical about this latest effort. It's a pity, because I think their emphasis is the right one. But for once I wish they'd been a bit more humble and accepted that they aren't going to build a better Wikipedia from scratch.

10 comments:

Anonymous said...

Hi,

Could you please elaborate on why you think that anonymity is a big weakness of Wikipedia?

Research has shown that vandalism is typically reverted within minutes and that content quality is comparable to the quality of traditional encyclopedias.

On the contrary, I think that anonymous contributions have strongly contributed to Wikipedia's success.

Thanks!

Daniel Tunkelang said...

In fairness, I can't measure the magnitude of the weakness. But it isn't just about vandalism.

Let me give a few examples:

* The edit war on the Fast Search & Transfer entry. I believe both sides would have been more civil if they'd had to sign posts with their full names. At the very least, readers would be in a position to consider the authors' reliability as sources, rather than counting on the Wikipedia editors to perform an impossible task of distilling their dispute to a "neutral" point of view.

* The Major Figures list for the Information Retrieval entry. Imagine that you weren't an expert in information retrieval. How would you decide whether to trust this list? Wouldn't you want to know if it were written by somebody on it? In fact, on May 26th I culled that list drastically. Who am I to make that determination? And who made up the previous list?

* The entry for Xylitol, a sugar substitute. Since I know that sugar substitutes attract a wide variety of viewpoints (e.g., the companies trying to sell them vs. "natural" foods activists who sometimes oppose them in principle), I'd like to know whom I'm reading before I decide to make a decision that could affect my health and well-being.

I could come up with more examples, but the general theme should be clear: clarity of sources is a major ingredient in building trust between writer and reader. It's the reason that newspapers like the New York Times have made an effort to reduce their dependence on anonymous sources.

I concede that there are times that anonymity is necessary because of fear of reprisals. But it comes at the cost of readers not knowing what agenda that anonymity may conceal. It also inhibits readers from building up trust in authors--though I supposed authors can still build up that trust through consistent use of a pseudonym. Still, I see no reason for anonymity to be the norm.

Anonymous said...

This is an anonymous comment, so you'll just have to evaluate my arguments on the merits :-)

If you're still reading: one problem with signed articles is that they don't lend themselves to collaboration. Sure, authors get feedback and suggestions and can incorporate them--but will they? They'll be in a position of constant maintenance for the life of the article, which is a genuine burden. The elegant thing about Wikipedia is there can be progress without any one person taking fulltime responsibility.

A second problem is the barrier to entry. People can learn to become Wikipedia editors starting with small, simple tasks. Beginning on Knol will be much harder.

That said, your examples are excellent. For the first, a technical article, I could imagine a Knol article being good--if anyone actually writes it. For the second, I have a vision of Knol containing several lists of major figures. (Plus there's the logical problem that anyone who could evaluate the author of a "major figures" list probably knows the major figures already :-)

In the third case, I think you have a very good point. In fact I'd say that no one should use Wikipedia for medical information, or really anything that's life or death.

Daniel Tunkelang said...

Touché. I suppose one of the best arguments in favor of anonymity is the same one used to justify blind review of scientific papers--it encourages us to evaluate arguments on their merits rather than on the reputation of their authors or their institutions.

To be clear, I don't think that articles need to be single-authored. I see a strong case for allowing revisions--but I'd like to see the editors identify themselves too. I suppose it can be disconcerting to an author who writes and signs the initial draft of an entry, only to see it revised by someone else. But it shouldn't be hard to show who wrote what--in fact, we do this all of the time with collaboratively authored documents.

So let me clarify my point about authorship and anonymity. I don't want to change the ownership model of Wikipedia. All I want to change is the norm that participants are anonymous.

Anonymous said...

How am I supposed to know whether to trust the authors on Knol? I can't exactly do background checks on all these random people.

It reminds me of the joke:

Q: What do you call the person who graduated at the bottom of their med school?
A: Doctor.

Daniel Tunkelang said...

Actually, the great thing about the web is that a "background check" is pretty trivial. I regularly research authors in order to obtain further context about their writing. I also expect many authors will make this research easy by offering links to their resumes or something equivalent. I really think authorship is a great idea--I just wish Google weren't so bull-headed about trying to replace rather than improve Wikipedia.

In response to the joke about the doctor, that's exactly why resources like HealthGrades are popping up.

Anonymous said...

I found it amusing that HealthGrades is itself a pay-to-view service (not that there's anything wrong with that -- I'm just not using it for a background check on a Knol author).

I find it easy to look up people who are well represented in academia or the mainstream media, but beyond that, it gets pretty sketchy.

For example, take this randomly selected Knol on palpitations. The byline says "Paul Nadler, MD" with a UCSF affiliation indicated. Does Google do background checks on this info?

If I run over to spock.com, I find 21 "Paul Nadler"s on the first three pages. Two of them are listed as MDs. There's Evan Paul Nadler, MD and a Paul Ira Nadler, MD, neither of whom seem to be at UCSF. Searching on Google's or Cuil's not much more helpful. Who knew there were this many Paul Nadlers?

If verifying this author's identity was really important, I could perhaps make more progress. Or I could pay a lawyer or someone else with Lexis-Nexis to do a better background check. But all I'm looking for here is an answer of whether or not to trust this article on palpitations.

Of course, for medicine, we can just go to the NIH page on palpitations.

Daniel Tunkelang said...

I'm not suggesting that readers should suddenly start investing significant effort or money in background checks on authors. To the contrary, I'd think that authors whose resumes add to their credibility would make it easy for readers to find those resumes. Given how easy it is for an author to link to this information, I'd hold the lack of such information against authors, much as I wouldn't hire someone who couldn't be bothered to provide me a resume.

Granted, in many cases I don't need to trust the author to benefit from an article. As the anonymous commenter suggested, sometimes it's sufficient to evaluate an argument on its own merits. But not always. Author reputation is no silver bullet for determining accuracy and bias, but it's helpful, especially when I'm researching new subjects.

By the way, there's nothing unique about authorship. An organization like the NIH serves a similar function when it stakes its reputation on the documents it publishes. But in this case even they identify an author and a reviewer, as well as their affiliations.

Anonymous said...

The proof will be in the pudding.

Daniel Tunkelang said...

I was catching up on Fred Wilson's A VC blog and discovered this NYT piece from a couple of years ago: Anonymous Source Is Not the Same as Open Source. I wholeheartedly agree with the author's sentiments.

Wednesday, July 23, 2008

Knol: Google takes on Wikipedia

Just a few days ago, I was commenting on a New York Times article about Wikipedia's new approval system that the biggest problem with Wikipedia is anonymous authorship. By synchronous coincidence, Google unveiled Knol today, which is something of a cross between Wikipedia and Squidoo. It's most salient feature is that each entry will have a clearly identified author. They even allow authors to verify their identities using credit cards or phone directories.

It's a nice idea, since anonymous authorship is a a major factor in the adversarial nature of information retrieval on the web. Not only does the accountability of authorship inhibit vandalism and edit wars, but it also allows readers to decide for themselves whom to trust--at least to the extent that readers are able and willing to obtain reliable information about the authors. Without question, they are addressing Wikipedia's biggest weakness.

But it's too little, too late. Wikipedia is already there. And, despite complaints about its inaccuracy and bias, Wikipedia is a fantastic, highly utilized resource. The only way I see for Knol to supplant Wikipedia in reasonable time frame is through a massive cut-and-paste to make up for the huge difference in content.

Interestingly, Wikipedia does not seem to place any onerous restrictions on verbatim copying. However, unless a single author is 100% responsible for authoring a Wikipedia entry, it isn't clear that anyone can simply copy the entry into Knol.

I know that it's dangerous to bet against Google. But I'm really skeptical about this latest effort. It's a pity, because I think their emphasis is the right one. But for once I wish they'd been a bit more humble and accepted that they aren't going to build a better Wikipedia from scratch.

10 comments:

Anonymous said...

Hi,

Could you please elaborate on why you think that anonymity is a big weakness of Wikipedia?

Research has shown that vandalism is typically reverted within minutes and that content quality is comparable to the quality of traditional encyclopedias.

On the contrary, I think that anonymous contributions have strongly contributed to Wikipedia's success.

Thanks!

Daniel Tunkelang said...

In fairness, I can't measure the magnitude of the weakness. But it isn't just about vandalism.

Let me give a few examples:

* The edit war on the Fast Search & Transfer entry. I believe both sides would have been more civil if they'd had to sign posts with their full names. At the very least, readers would be in a position to consider the authors' reliability as sources, rather than counting on the Wikipedia editors to perform an impossible task of distilling their dispute to a "neutral" point of view.

* The Major Figures list for the Information Retrieval entry. Imagine that you weren't an expert in information retrieval. How would you decide whether to trust this list? Wouldn't you want to know if it were written by somebody on it? In fact, on May 26th I culled that list drastically. Who am I to make that determination? And who made up the previous list?

* The entry for Xylitol, a sugar substitute. Since I know that sugar substitutes attract a wide variety of viewpoints (e.g., the companies trying to sell them vs. "natural" foods activists who sometimes oppose them in principle), I'd like to know whom I'm reading before I decide to make a decision that could affect my health and well-being.

I could come up with more examples, but the general theme should be clear: clarity of sources is a major ingredient in building trust between writer and reader. It's the reason that newspapers like the New York Times have made an effort to reduce their dependence on anonymous sources.

I concede that there are times that anonymity is necessary because of fear of reprisals. But it comes at the cost of readers not knowing what agenda that anonymity may conceal. It also inhibits readers from building up trust in authors--though I supposed authors can still build up that trust through consistent use of a pseudonym. Still, I see no reason for anonymity to be the norm.

Anonymous said...

This is an anonymous comment, so you'll just have to evaluate my arguments on the merits :-)

If you're still reading: one problem with signed articles is that they don't lend themselves to collaboration. Sure, authors get feedback and suggestions and can incorporate them--but will they? They'll be in a position of constant maintenance for the life of the article, which is a genuine burden. The elegant thing about Wikipedia is there can be progress without any one person taking fulltime responsibility.

A second problem is the barrier to entry. People can learn to become Wikipedia editors starting with small, simple tasks. Beginning on Knol will be much harder.

That said, your examples are excellent. For the first, a technical article, I could imagine a Knol article being good--if anyone actually writes it. For the second, I have a vision of Knol containing several lists of major figures. (Plus there's the logical problem that anyone who could evaluate the author of a "major figures" list probably knows the major figures already :-)

In the third case, I think you have a very good point. In fact I'd say that no one should use Wikipedia for medical information, or really anything that's life or death.

Daniel Tunkelang said...

Touché. I suppose one of the best arguments in favor of anonymity is the same one used to justify blind review of scientific papers--it encourages us to evaluate arguments on their merits rather than on the reputation of their authors or their institutions.

To be clear, I don't think that articles need to be single-authored. I see a strong case for allowing revisions--but I'd like to see the editors identify themselves too. I suppose it can be disconcerting to an author who writes and signs the initial draft of an entry, only to see it revised by someone else. But it shouldn't be hard to show who wrote what--in fact, we do this all of the time with collaboratively authored documents.

So let me clarify my point about authorship and anonymity. I don't want to change the ownership model of Wikipedia. All I want to change is the norm that participants are anonymous.

Anonymous said...

How am I supposed to know whether to trust the authors on Knol? I can't exactly do background checks on all these random people.

It reminds me of the joke:

Q: What do you call the person who graduated at the bottom of their med school?
A: Doctor.

Daniel Tunkelang said...

Actually, the great thing about the web is that a "background check" is pretty trivial. I regularly research authors in order to obtain further context about their writing. I also expect many authors will make this research easy by offering links to their resumes or something equivalent. I really think authorship is a great idea--I just wish Google weren't so bull-headed about trying to replace rather than improve Wikipedia.

In response to the joke about the doctor, that's exactly why resources like HealthGrades are popping up.

Anonymous said...

I found it amusing that HealthGrades is itself a pay-to-view service (not that there's anything wrong with that -- I'm just not using it for a background check on a Knol author).

I find it easy to look up people who are well represented in academia or the mainstream media, but beyond that, it gets pretty sketchy.

For example, take this randomly selected Knol on palpitations. The byline says "Paul Nadler, MD" with a UCSF affiliation indicated. Does Google do background checks on this info?

If I run over to spock.com, I find 21 "Paul Nadler"s on the first three pages. Two of them are listed as MDs. There's Evan Paul Nadler, MD and a Paul Ira Nadler, MD, neither of whom seem to be at UCSF. Searching on Google's or Cuil's not much more helpful. Who knew there were this many Paul Nadlers?

If verifying this author's identity was really important, I could perhaps make more progress. Or I could pay a lawyer or someone else with Lexis-Nexis to do a better background check. But all I'm looking for here is an answer of whether or not to trust this article on palpitations.

Of course, for medicine, we can just go to the NIH page on palpitations.

Daniel Tunkelang said...

I'm not suggesting that readers should suddenly start investing significant effort or money in background checks on authors. To the contrary, I'd think that authors whose resumes add to their credibility would make it easy for readers to find those resumes. Given how easy it is for an author to link to this information, I'd hold the lack of such information against authors, much as I wouldn't hire someone who couldn't be bothered to provide me a resume.

Granted, in many cases I don't need to trust the author to benefit from an article. As the anonymous commenter suggested, sometimes it's sufficient to evaluate an argument on its own merits. But not always. Author reputation is no silver bullet for determining accuracy and bias, but it's helpful, especially when I'm researching new subjects.

By the way, there's nothing unique about authorship. An organization like the NIH serves a similar function when it stakes its reputation on the documents it publishes. But in this case even they identify an author and a reviewer, as well as their affiliations.

Anonymous said...

The proof will be in the pudding.

Daniel Tunkelang said...

I was catching up on Fred Wilson's A VC blog and discovered this NYT piece from a couple of years ago: Anonymous Source Is Not the Same as Open Source. I wholeheartedly agree with the author's sentiments.

Wednesday, July 23, 2008

Knol: Google takes on Wikipedia

Just a few days ago, I was commenting on a New York Times article about Wikipedia's new approval system that the biggest problem with Wikipedia is anonymous authorship. By synchronous coincidence, Google unveiled Knol today, which is something of a cross between Wikipedia and Squidoo. It's most salient feature is that each entry will have a clearly identified author. They even allow authors to verify their identities using credit cards or phone directories.

It's a nice idea, since anonymous authorship is a a major factor in the adversarial nature of information retrieval on the web. Not only does the accountability of authorship inhibit vandalism and edit wars, but it also allows readers to decide for themselves whom to trust--at least to the extent that readers are able and willing to obtain reliable information about the authors. Without question, they are addressing Wikipedia's biggest weakness.

But it's too little, too late. Wikipedia is already there. And, despite complaints about its inaccuracy and bias, Wikipedia is a fantastic, highly utilized resource. The only way I see for Knol to supplant Wikipedia in reasonable time frame is through a massive cut-and-paste to make up for the huge difference in content.

Interestingly, Wikipedia does not seem to place any onerous restrictions on verbatim copying. However, unless a single author is 100% responsible for authoring a Wikipedia entry, it isn't clear that anyone can simply copy the entry into Knol.

I know that it's dangerous to bet against Google. But I'm really skeptical about this latest effort. It's a pity, because I think their emphasis is the right one. But for once I wish they'd been a bit more humble and accepted that they aren't going to build a better Wikipedia from scratch.

10 comments:

Anonymous said...

Hi,

Could you please elaborate on why you think that anonymity is a big weakness of Wikipedia?

Research has shown that vandalism is typically reverted within minutes and that content quality is comparable to the quality of traditional encyclopedias.

On the contrary, I think that anonymous contributions have strongly contributed to Wikipedia's success.

Thanks!

Daniel Tunkelang said...

In fairness, I can't measure the magnitude of the weakness. But it isn't just about vandalism.

Let me give a few examples:

* The edit war on the Fast Search & Transfer entry. I believe both sides would have been more civil if they'd had to sign posts with their full names. At the very least, readers would be in a position to consider the authors' reliability as sources, rather than counting on the Wikipedia editors to perform an impossible task of distilling their dispute to a "neutral" point of view.

* The Major Figures list for the Information Retrieval entry. Imagine that you weren't an expert in information retrieval. How would you decide whether to trust this list? Wouldn't you want to know if it were written by somebody on it? In fact, on May 26th I culled that list drastically. Who am I to make that determination? And who made up the previous list?

* The entry for Xylitol, a sugar substitute. Since I know that sugar substitutes attract a wide variety of viewpoints (e.g., the companies trying to sell them vs. "natural" foods activists who sometimes oppose them in principle), I'd like to know whom I'm reading before I decide to make a decision that could affect my health and well-being.

I could come up with more examples, but the general theme should be clear: clarity of sources is a major ingredient in building trust between writer and reader. It's the reason that newspapers like the New York Times have made an effort to reduce their dependence on anonymous sources.

I concede that there are times that anonymity is necessary because of fear of reprisals. But it comes at the cost of readers not knowing what agenda that anonymity may conceal. It also inhibits readers from building up trust in authors--though I supposed authors can still build up that trust through consistent use of a pseudonym. Still, I see no reason for anonymity to be the norm.

Anonymous said...

This is an anonymous comment, so you'll just have to evaluate my arguments on the merits :-)

If you're still reading: one problem with signed articles is that they don't lend themselves to collaboration. Sure, authors get feedback and suggestions and can incorporate them--but will they? They'll be in a position of constant maintenance for the life of the article, which is a genuine burden. The elegant thing about Wikipedia is there can be progress without any one person taking fulltime responsibility.

A second problem is the barrier to entry. People can learn to become Wikipedia editors starting with small, simple tasks. Beginning on Knol will be much harder.

That said, your examples are excellent. For the first, a technical article, I could imagine a Knol article being good--if anyone actually writes it. For the second, I have a vision of Knol containing several lists of major figures. (Plus there's the logical problem that anyone who could evaluate the author of a "major figures" list probably knows the major figures already :-)

In the third case, I think you have a very good point. In fact I'd say that no one should use Wikipedia for medical information, or really anything that's life or death.

Daniel Tunkelang said...

Touché. I suppose one of the best arguments in favor of anonymity is the same one used to justify blind review of scientific papers--it encourages us to evaluate arguments on their merits rather than on the reputation of their authors or their institutions.

To be clear, I don't think that articles need to be single-authored. I see a strong case for allowing revisions--but I'd like to see the editors identify themselves too. I suppose it can be disconcerting to an author who writes and signs the initial draft of an entry, only to see it revised by someone else. But it shouldn't be hard to show who wrote what--in fact, we do this all of the time with collaboratively authored documents.

So let me clarify my point about authorship and anonymity. I don't want to change the ownership model of Wikipedia. All I want to change is the norm that participants are anonymous.

Anonymous said...

How am I supposed to know whether to trust the authors on Knol? I can't exactly do background checks on all these random people.

It reminds me of the joke:

Q: What do you call the person who graduated at the bottom of their med school?
A: Doctor.

Daniel Tunkelang said...

Actually, the great thing about the web is that a "background check" is pretty trivial. I regularly research authors in order to obtain further context about their writing. I also expect many authors will make this research easy by offering links to their resumes or something equivalent. I really think authorship is a great idea--I just wish Google weren't so bull-headed about trying to replace rather than improve Wikipedia.

In response to the joke about the doctor, that's exactly why resources like HealthGrades are popping up.

Anonymous said...

I found it amusing that HealthGrades is itself a pay-to-view service (not that there's anything wrong with that -- I'm just not using it for a background check on a Knol author).

I find it easy to look up people who are well represented in academia or the mainstream media, but beyond that, it gets pretty sketchy.

For example, take this randomly selected Knol on palpitations. The byline says "Paul Nadler, MD" with a UCSF affiliation indicated. Does Google do background checks on this info?

If I run over to spock.com, I find 21 "Paul Nadler"s on the first three pages. Two of them are listed as MDs. There's Evan Paul Nadler, MD and a Paul Ira Nadler, MD, neither of whom seem to be at UCSF. Searching on Google's or Cuil's not much more helpful. Who knew there were this many Paul Nadlers?

If verifying this author's identity was really important, I could perhaps make more progress. Or I could pay a lawyer or someone else with Lexis-Nexis to do a better background check. But all I'm looking for here is an answer of whether or not to trust this article on palpitations.

Of course, for medicine, we can just go to the NIH page on palpitations.

Daniel Tunkelang said...

I'm not suggesting that readers should suddenly start investing significant effort or money in background checks on authors. To the contrary, I'd think that authors whose resumes add to their credibility would make it easy for readers to find those resumes. Given how easy it is for an author to link to this information, I'd hold the lack of such information against authors, much as I wouldn't hire someone who couldn't be bothered to provide me a resume.

Granted, in many cases I don't need to trust the author to benefit from an article. As the anonymous commenter suggested, sometimes it's sufficient to evaluate an argument on its own merits. But not always. Author reputation is no silver bullet for determining accuracy and bias, but it's helpful, especially when I'm researching new subjects.

By the way, there's nothing unique about authorship. An organization like the NIH serves a similar function when it stakes its reputation on the documents it publishes. But in this case even they identify an author and a reviewer, as well as their affiliations.

Anonymous said...

The proof will be in the pudding.

Daniel Tunkelang said...

I was catching up on Fred Wilson's A VC blog and discovered this NYT piece from a couple of years ago: Anonymous Source Is Not the Same as Open Source. I wholeheartedly agree with the author's sentiments.