Wednesday, July 23, 2008
Knol: Google takes on Wikipedia
It's a nice idea, since anonymous authorship is a a major factor in the adversarial nature of information retrieval on the web. Not only does the accountability of authorship inhibit vandalism and edit wars, but it also allows readers to decide for themselves whom to trust--at least to the extent that readers are able and willing to obtain reliable information about the authors. Without question, they are addressing Wikipedia's biggest weakness.
But it's too little, too late. Wikipedia is already there. And, despite complaints about its inaccuracy and bias, Wikipedia is a fantastic, highly utilized resource. The only way I see for Knol to supplant Wikipedia in reasonable time frame is through a massive cut-and-paste to make up for the huge difference in content.
Interestingly, Wikipedia does not seem to place any onerous restrictions on verbatim copying. However, unless a single author is 100% responsible for authoring a Wikipedia entry, it isn't clear that anyone can simply copy the entry into Knol.
I know that it's dangerous to bet against Google. But I'm really skeptical about this latest effort. It's a pity, because I think their emphasis is the right one. But for once I wish they'd been a bit more humble and accepted that they aren't going to build a better Wikipedia from scratch.
Wednesday, July 23, 2008
Knol: Google takes on Wikipedia
It's a nice idea, since anonymous authorship is a a major factor in the adversarial nature of information retrieval on the web. Not only does the accountability of authorship inhibit vandalism and edit wars, but it also allows readers to decide for themselves whom to trust--at least to the extent that readers are able and willing to obtain reliable information about the authors. Without question, they are addressing Wikipedia's biggest weakness.
But it's too little, too late. Wikipedia is already there. And, despite complaints about its inaccuracy and bias, Wikipedia is a fantastic, highly utilized resource. The only way I see for Knol to supplant Wikipedia in reasonable time frame is through a massive cut-and-paste to make up for the huge difference in content.
Interestingly, Wikipedia does not seem to place any onerous restrictions on verbatim copying. However, unless a single author is 100% responsible for authoring a Wikipedia entry, it isn't clear that anyone can simply copy the entry into Knol.
I know that it's dangerous to bet against Google. But I'm really skeptical about this latest effort. It's a pity, because I think their emphasis is the right one. But for once I wish they'd been a bit more humble and accepted that they aren't going to build a better Wikipedia from scratch.
10 comments:
-
-
Hi,
Could you please elaborate on why you think that anonymity is a big weakness of Wikipedia?
Research has shown that vandalism is typically reverted within minutes and that content quality is comparable to the quality of traditional encyclopedias.
On the contrary, I think that anonymous contributions have strongly contributed to Wikipedia's success.
Thanks! - July 24, 2008 at 6:38 AM
- Daniel Tunkelang said...
-
In fairness, I can't measure the magnitude of the weakness. But it isn't just about vandalism.
Let me give a few examples:
* The edit war on the Fast Search & Transfer entry. I believe both sides would have been more civil if they'd had to sign posts with their full names. At the very least, readers would be in a position to consider the authors' reliability as sources, rather than counting on the Wikipedia editors to perform an impossible task of distilling their dispute to a "neutral" point of view.
* The Major Figures list for the Information Retrieval entry. Imagine that you weren't an expert in information retrieval. How would you decide whether to trust this list? Wouldn't you want to know if it were written by somebody on it? In fact, on May 26th I culled that list drastically. Who am I to make that determination? And who made up the previous list?
* The entry for Xylitol, a sugar substitute. Since I know that sugar substitutes attract a wide variety of viewpoints (e.g., the companies trying to sell them vs. "natural" foods activists who sometimes oppose them in principle), I'd like to know whom I'm reading before I decide to make a decision that could affect my health and well-being.
I could come up with more examples, but the general theme should be clear: clarity of sources is a major ingredient in building trust between writer and reader. It's the reason that newspapers like the New York Times have made an effort to reduce their dependence on anonymous sources.
I concede that there are times that anonymity is necessary because of fear of reprisals. But it comes at the cost of readers not knowing what agenda that anonymity may conceal. It also inhibits readers from building up trust in authors--though I supposed authors can still build up that trust through consistent use of a pseudonym. Still, I see no reason for anonymity to be the norm. - July 24, 2008 at 9:21 AM
-
-
This is an anonymous comment, so you'll just have to evaluate my arguments on the merits :-)
If you're still reading: one problem with signed articles is that they don't lend themselves to collaboration. Sure, authors get feedback and suggestions and can incorporate them--but will they? They'll be in a position of constant maintenance for the life of the article, which is a genuine burden. The elegant thing about Wikipedia is there can be progress without any one person taking fulltime responsibility.
A second problem is the barrier to entry. People can learn to become Wikipedia editors starting with small, simple tasks. Beginning on Knol will be much harder.
That said, your examples are excellent. For the first, a technical article, I could imagine a Knol article being good--if anyone actually writes it. For the second, I have a vision of Knol containing several lists of major figures. (Plus there's the logical problem that anyone who could evaluate the author of a "major figures" list probably knows the major figures already :-)
In the third case, I think you have a very good point. In fact I'd say that no one should use Wikipedia for medical information, or really anything that's life or death. - July 24, 2008 at 11:44 AM
- Daniel Tunkelang said...
-
Touché. I suppose one of the best arguments in favor of anonymity is the same one used to justify blind review of scientific papers--it encourages us to evaluate arguments on their merits rather than on the reputation of their authors or their institutions.
To be clear, I don't think that articles need to be single-authored. I see a strong case for allowing revisions--but I'd like to see the editors identify themselves too. I suppose it can be disconcerting to an author who writes and signs the initial draft of an entry, only to see it revised by someone else. But it shouldn't be hard to show who wrote what--in fact, we do this all of the time with collaboratively authored documents.
So let me clarify my point about authorship and anonymity. I don't want to change the ownership model of Wikipedia. All I want to change is the norm that participants are anonymous. - July 24, 2008 at 12:05 PM
-
-
How am I supposed to know whether to trust the authors on Knol? I can't exactly do background checks on all these random people.
It reminds me of the joke:
Q: What do you call the person who graduated at the bottom of their med school?
A: Doctor. - July 28, 2008 at 6:05 PM
- Daniel Tunkelang said...
-
Actually, the great thing about the web is that a "background check" is pretty trivial. I regularly research authors in order to obtain further context about their writing. I also expect many authors will make this research easy by offering links to their resumes or something equivalent. I really think authorship is a great idea--I just wish Google weren't so bull-headed about trying to replace rather than improve Wikipedia.
In response to the joke about the doctor, that's exactly why resources like HealthGrades are popping up. - July 28, 2008 at 8:50 PM
-
-
I found it amusing that HealthGrades is itself a pay-to-view service (not that there's anything wrong with that -- I'm just not using it for a background check on a Knol author).
I find it easy to look up people who are well represented in academia or the mainstream media, but beyond that, it gets pretty sketchy.
For example, take this randomly selected Knol on palpitations. The byline says "Paul Nadler, MD" with a UCSF affiliation indicated. Does Google do background checks on this info?
If I run over to spock.com, I find 21 "Paul Nadler"s on the first three pages. Two of them are listed as MDs. There's Evan Paul Nadler, MD and a Paul Ira Nadler, MD, neither of whom seem to be at UCSF. Searching on Google's or Cuil's not much more helpful. Who knew there were this many Paul Nadlers?
If verifying this author's identity was really important, I could perhaps make more progress. Or I could pay a lawyer or someone else with Lexis-Nexis to do a better background check. But all I'm looking for here is an answer of whether or not to trust this article on palpitations.
Of course, for medicine, we can just go to the NIH page on palpitations. - July 29, 2008 at 5:47 PM
- Daniel Tunkelang said...
-
I'm not suggesting that readers should suddenly start investing significant effort or money in background checks on authors. To the contrary, I'd think that authors whose resumes add to their credibility would make it easy for readers to find those resumes. Given how easy it is for an author to link to this information, I'd hold the lack of such information against authors, much as I wouldn't hire someone who couldn't be bothered to provide me a resume.
Granted, in many cases I don't need to trust the author to benefit from an article. As the anonymous commenter suggested, sometimes it's sufficient to evaluate an argument on its own merits. But not always. Author reputation is no silver bullet for determining accuracy and bias, but it's helpful, especially when I'm researching new subjects.
By the way, there's nothing unique about authorship. An organization like the NIH serves a similar function when it stakes its reputation on the documents it publishes. But in this case even they identify an author and a reviewer, as well as their affiliations. - July 29, 2008 at 7:40 PM
-
-
The proof will be in the pudding.
- July 30, 2008 at 5:09 AM
- Daniel Tunkelang said...
-
I was catching up on Fred Wilson's A VC blog and discovered this NYT piece from a couple of years ago: Anonymous Source Is Not the Same as Open Source. I wholeheartedly agree with the author's sentiments.
- August 3, 2008 at 11:47 AM
Wednesday, July 23, 2008
Knol: Google takes on Wikipedia
It's a nice idea, since anonymous authorship is a a major factor in the adversarial nature of information retrieval on the web. Not only does the accountability of authorship inhibit vandalism and edit wars, but it also allows readers to decide for themselves whom to trust--at least to the extent that readers are able and willing to obtain reliable information about the authors. Without question, they are addressing Wikipedia's biggest weakness.
But it's too little, too late. Wikipedia is already there. And, despite complaints about its inaccuracy and bias, Wikipedia is a fantastic, highly utilized resource. The only way I see for Knol to supplant Wikipedia in reasonable time frame is through a massive cut-and-paste to make up for the huge difference in content.
Interestingly, Wikipedia does not seem to place any onerous restrictions on verbatim copying. However, unless a single author is 100% responsible for authoring a Wikipedia entry, it isn't clear that anyone can simply copy the entry into Knol.
I know that it's dangerous to bet against Google. But I'm really skeptical about this latest effort. It's a pity, because I think their emphasis is the right one. But for once I wish they'd been a bit more humble and accepted that they aren't going to build a better Wikipedia from scratch.
10 comments:
-
-
Hi,
Could you please elaborate on why you think that anonymity is a big weakness of Wikipedia?
Research has shown that vandalism is typically reverted within minutes and that content quality is comparable to the quality of traditional encyclopedias.
On the contrary, I think that anonymous contributions have strongly contributed to Wikipedia's success.
Thanks! - July 24, 2008 at 6:38 AM
- Daniel Tunkelang said...
-
In fairness, I can't measure the magnitude of the weakness. But it isn't just about vandalism.
Let me give a few examples:
* The edit war on the Fast Search & Transfer entry. I believe both sides would have been more civil if they'd had to sign posts with their full names. At the very least, readers would be in a position to consider the authors' reliability as sources, rather than counting on the Wikipedia editors to perform an impossible task of distilling their dispute to a "neutral" point of view.
* The Major Figures list for the Information Retrieval entry. Imagine that you weren't an expert in information retrieval. How would you decide whether to trust this list? Wouldn't you want to know if it were written by somebody on it? In fact, on May 26th I culled that list drastically. Who am I to make that determination? And who made up the previous list?
* The entry for Xylitol, a sugar substitute. Since I know that sugar substitutes attract a wide variety of viewpoints (e.g., the companies trying to sell them vs. "natural" foods activists who sometimes oppose them in principle), I'd like to know whom I'm reading before I decide to make a decision that could affect my health and well-being.
I could come up with more examples, but the general theme should be clear: clarity of sources is a major ingredient in building trust between writer and reader. It's the reason that newspapers like the New York Times have made an effort to reduce their dependence on anonymous sources.
I concede that there are times that anonymity is necessary because of fear of reprisals. But it comes at the cost of readers not knowing what agenda that anonymity may conceal. It also inhibits readers from building up trust in authors--though I supposed authors can still build up that trust through consistent use of a pseudonym. Still, I see no reason for anonymity to be the norm. - July 24, 2008 at 9:21 AM
-
-
This is an anonymous comment, so you'll just have to evaluate my arguments on the merits :-)
If you're still reading: one problem with signed articles is that they don't lend themselves to collaboration. Sure, authors get feedback and suggestions and can incorporate them--but will they? They'll be in a position of constant maintenance for the life of the article, which is a genuine burden. The elegant thing about Wikipedia is there can be progress without any one person taking fulltime responsibility.
A second problem is the barrier to entry. People can learn to become Wikipedia editors starting with small, simple tasks. Beginning on Knol will be much harder.
That said, your examples are excellent. For the first, a technical article, I could imagine a Knol article being good--if anyone actually writes it. For the second, I have a vision of Knol containing several lists of major figures. (Plus there's the logical problem that anyone who could evaluate the author of a "major figures" list probably knows the major figures already :-)
In the third case, I think you have a very good point. In fact I'd say that no one should use Wikipedia for medical information, or really anything that's life or death. - July 24, 2008 at 11:44 AM
- Daniel Tunkelang said...
-
Touché. I suppose one of the best arguments in favor of anonymity is the same one used to justify blind review of scientific papers--it encourages us to evaluate arguments on their merits rather than on the reputation of their authors or their institutions.
To be clear, I don't think that articles need to be single-authored. I see a strong case for allowing revisions--but I'd like to see the editors identify themselves too. I suppose it can be disconcerting to an author who writes and signs the initial draft of an entry, only to see it revised by someone else. But it shouldn't be hard to show who wrote what--in fact, we do this all of the time with collaboratively authored documents.
So let me clarify my point about authorship and anonymity. I don't want to change the ownership model of Wikipedia. All I want to change is the norm that participants are anonymous. - July 24, 2008 at 12:05 PM
-
-
How am I supposed to know whether to trust the authors on Knol? I can't exactly do background checks on all these random people.
It reminds me of the joke:
Q: What do you call the person who graduated at the bottom of their med school?
A: Doctor. - July 28, 2008 at 6:05 PM
- Daniel Tunkelang said...
-
Actually, the great thing about the web is that a "background check" is pretty trivial. I regularly research authors in order to obtain further context about their writing. I also expect many authors will make this research easy by offering links to their resumes or something equivalent. I really think authorship is a great idea--I just wish Google weren't so bull-headed about trying to replace rather than improve Wikipedia.
In response to the joke about the doctor, that's exactly why resources like HealthGrades are popping up. - July 28, 2008 at 8:50 PM
-
-
I found it amusing that HealthGrades is itself a pay-to-view service (not that there's anything wrong with that -- I'm just not using it for a background check on a Knol author).
I find it easy to look up people who are well represented in academia or the mainstream media, but beyond that, it gets pretty sketchy.
For example, take this randomly selected Knol on palpitations. The byline says "Paul Nadler, MD" with a UCSF affiliation indicated. Does Google do background checks on this info?
If I run over to spock.com, I find 21 "Paul Nadler"s on the first three pages. Two of them are listed as MDs. There's Evan Paul Nadler, MD and a Paul Ira Nadler, MD, neither of whom seem to be at UCSF. Searching on Google's or Cuil's not much more helpful. Who knew there were this many Paul Nadlers?
If verifying this author's identity was really important, I could perhaps make more progress. Or I could pay a lawyer or someone else with Lexis-Nexis to do a better background check. But all I'm looking for here is an answer of whether or not to trust this article on palpitations.
Of course, for medicine, we can just go to the NIH page on palpitations. - July 29, 2008 at 5:47 PM
- Daniel Tunkelang said...
-
I'm not suggesting that readers should suddenly start investing significant effort or money in background checks on authors. To the contrary, I'd think that authors whose resumes add to their credibility would make it easy for readers to find those resumes. Given how easy it is for an author to link to this information, I'd hold the lack of such information against authors, much as I wouldn't hire someone who couldn't be bothered to provide me a resume.
Granted, in many cases I don't need to trust the author to benefit from an article. As the anonymous commenter suggested, sometimes it's sufficient to evaluate an argument on its own merits. But not always. Author reputation is no silver bullet for determining accuracy and bias, but it's helpful, especially when I'm researching new subjects.
By the way, there's nothing unique about authorship. An organization like the NIH serves a similar function when it stakes its reputation on the documents it publishes. But in this case even they identify an author and a reviewer, as well as their affiliations. - July 29, 2008 at 7:40 PM
-
-
The proof will be in the pudding.
- July 30, 2008 at 5:09 AM
- Daniel Tunkelang said...
-
I was catching up on Fred Wilson's A VC blog and discovered this NYT piece from a couple of years ago: Anonymous Source Is Not the Same as Open Source. I wholeheartedly agree with the author's sentiments.
- August 3, 2008 at 11:47 AM
10 comments:
Hi,
Could you please elaborate on why you think that anonymity is a big weakness of Wikipedia?
Research has shown that vandalism is typically reverted within minutes and that content quality is comparable to the quality of traditional encyclopedias.
On the contrary, I think that anonymous contributions have strongly contributed to Wikipedia's success.
Thanks!
In fairness, I can't measure the magnitude of the weakness. But it isn't just about vandalism.
Let me give a few examples:
* The edit war on the Fast Search & Transfer entry. I believe both sides would have been more civil if they'd had to sign posts with their full names. At the very least, readers would be in a position to consider the authors' reliability as sources, rather than counting on the Wikipedia editors to perform an impossible task of distilling their dispute to a "neutral" point of view.
* The Major Figures list for the Information Retrieval entry. Imagine that you weren't an expert in information retrieval. How would you decide whether to trust this list? Wouldn't you want to know if it were written by somebody on it? In fact, on May 26th I culled that list drastically. Who am I to make that determination? And who made up the previous list?
* The entry for Xylitol, a sugar substitute. Since I know that sugar substitutes attract a wide variety of viewpoints (e.g., the companies trying to sell them vs. "natural" foods activists who sometimes oppose them in principle), I'd like to know whom I'm reading before I decide to make a decision that could affect my health and well-being.
I could come up with more examples, but the general theme should be clear: clarity of sources is a major ingredient in building trust between writer and reader. It's the reason that newspapers like the New York Times have made an effort to reduce their dependence on anonymous sources.
I concede that there are times that anonymity is necessary because of fear of reprisals. But it comes at the cost of readers not knowing what agenda that anonymity may conceal. It also inhibits readers from building up trust in authors--though I supposed authors can still build up that trust through consistent use of a pseudonym. Still, I see no reason for anonymity to be the norm.
This is an anonymous comment, so you'll just have to evaluate my arguments on the merits :-)
If you're still reading: one problem with signed articles is that they don't lend themselves to collaboration. Sure, authors get feedback and suggestions and can incorporate them--but will they? They'll be in a position of constant maintenance for the life of the article, which is a genuine burden. The elegant thing about Wikipedia is there can be progress without any one person taking fulltime responsibility.
A second problem is the barrier to entry. People can learn to become Wikipedia editors starting with small, simple tasks. Beginning on Knol will be much harder.
That said, your examples are excellent. For the first, a technical article, I could imagine a Knol article being good--if anyone actually writes it. For the second, I have a vision of Knol containing several lists of major figures. (Plus there's the logical problem that anyone who could evaluate the author of a "major figures" list probably knows the major figures already :-)
In the third case, I think you have a very good point. In fact I'd say that no one should use Wikipedia for medical information, or really anything that's life or death.
Touché. I suppose one of the best arguments in favor of anonymity is the same one used to justify blind review of scientific papers--it encourages us to evaluate arguments on their merits rather than on the reputation of their authors or their institutions.
To be clear, I don't think that articles need to be single-authored. I see a strong case for allowing revisions--but I'd like to see the editors identify themselves too. I suppose it can be disconcerting to an author who writes and signs the initial draft of an entry, only to see it revised by someone else. But it shouldn't be hard to show who wrote what--in fact, we do this all of the time with collaboratively authored documents.
So let me clarify my point about authorship and anonymity. I don't want to change the ownership model of Wikipedia. All I want to change is the norm that participants are anonymous.
How am I supposed to know whether to trust the authors on Knol? I can't exactly do background checks on all these random people.
It reminds me of the joke:
Q: What do you call the person who graduated at the bottom of their med school?
A: Doctor.
Actually, the great thing about the web is that a "background check" is pretty trivial. I regularly research authors in order to obtain further context about their writing. I also expect many authors will make this research easy by offering links to their resumes or something equivalent. I really think authorship is a great idea--I just wish Google weren't so bull-headed about trying to replace rather than improve Wikipedia.
In response to the joke about the doctor, that's exactly why resources like HealthGrades are popping up.
I found it amusing that HealthGrades is itself a pay-to-view service (not that there's anything wrong with that -- I'm just not using it for a background check on a Knol author).
I find it easy to look up people who are well represented in academia or the mainstream media, but beyond that, it gets pretty sketchy.
For example, take this randomly selected Knol on palpitations. The byline says "Paul Nadler, MD" with a UCSF affiliation indicated. Does Google do background checks on this info?
If I run over to spock.com, I find 21 "Paul Nadler"s on the first three pages. Two of them are listed as MDs. There's Evan Paul Nadler, MD and a Paul Ira Nadler, MD, neither of whom seem to be at UCSF. Searching on Google's or Cuil's not much more helpful. Who knew there were this many Paul Nadlers?
If verifying this author's identity was really important, I could perhaps make more progress. Or I could pay a lawyer or someone else with Lexis-Nexis to do a better background check. But all I'm looking for here is an answer of whether or not to trust this article on palpitations.
Of course, for medicine, we can just go to the NIH page on palpitations.
I'm not suggesting that readers should suddenly start investing significant effort or money in background checks on authors. To the contrary, I'd think that authors whose resumes add to their credibility would make it easy for readers to find those resumes. Given how easy it is for an author to link to this information, I'd hold the lack of such information against authors, much as I wouldn't hire someone who couldn't be bothered to provide me a resume.
Granted, in many cases I don't need to trust the author to benefit from an article. As the anonymous commenter suggested, sometimes it's sufficient to evaluate an argument on its own merits. But not always. Author reputation is no silver bullet for determining accuracy and bias, but it's helpful, especially when I'm researching new subjects.
By the way, there's nothing unique about authorship. An organization like the NIH serves a similar function when it stakes its reputation on the documents it publishes. But in this case even they identify an author and a reviewer, as well as their affiliations.
The proof will be in the pudding.
I was catching up on Fred Wilson's A VC blog and discovered this NYT piece from a couple of years ago: Anonymous Source Is Not the Same as Open Source. I wholeheartedly agree with the author's sentiments.
Post a Comment