tag:blogger.com,1999:blog-8016696494330504473.post6202771048223829525..comments2008-09-16T17:25:47.335-04:00Comments on Redirecting to http://thenoisychannel.com...: Beyond a Reasonable DoubtDaniel Tunkelanghttp://www.blogger.com/profile/10240432137428080022noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-8016696494330504473.post-20919093280206897782008-07-24T21:34:00.000-04:002008-07-24T21:34:00.000-04:00Actually, the interesting thing about relevance sc...Actually, the interesting thing about relevance scoring is that most approaches do not return a probability of relevance, but only promise to rank documents in an order corresponding to their probability of relevance. The difference is subtle but significant, since it means you can't use the scores to establish a meaningful cutoff between relevant and irrelevant documents.<BR/><BR/>In response to the broader question of relating our understanding of uncertainty to search, I was thinking more broadly about using information to make decisions. Search is simply a tool--what we're often doing, consciously or unconsciously, is testing hypotheses or generating conjectures. We're generally doing so with incomplete and noisy data, which means our analysis has to tolerate some degree of uncertainty. Hence, our ability to perform this analysis effectively relies on our ability to articulate and reason about uncertainty.Daniel Tunkelanghttps://www.blogger.com/profile/10240432137428080022noreply@blogger.comtag:blogger.com,1999:blog-8016696494330504473.post-24001806330533012632008-07-24T18:28:00.000-04:002008-07-24T18:28:00.000-04:00Daniel,I was happy to see you discuss this topic. ...Daniel,<BR/><BR/>I was happy to see you discuss this topic. I wonder if you would relate these principles of uncertainty to your work in the area of search? Isn't a relevance "score" really a probability that a given document is relevant in the context of a query? Are there benchmarks against which one might evaluate a relevance score generated from a predictive engine and some sort of actual or real relevance based on usage like your coin toss example?stefanoqhttps://www.blogger.com/profile/03957519878567929614noreply@blogger.com