Concepts
Similar pages
| Similarity |
Page |
Snapshot |
| 174 |
one unit of precision for an increase of one unit of recall,but will not sacrifice another unit of precision for a further unit increase in recall,i
...R 1,P 1 >R,P but R 1,P >R 2,P 1 We conclude that the interval between R 1 and R exceeds the interval between P and P 1 whereas the interval between R 1 and R 2 is smaller
...Finally,we incorporate into our measurement procedure the fact that users may attach different relative importance to precision and recall
...Definition 6
...Can we find a function satisfying all these conditions?If so,can we also interpret it in an intuitively simple way?The answer to both these questions is yes
...The scale functions are therefore,[[Phi]]1 P [[alpha]]1 P,and [[Phi]]2 R 1 [[alpha]]1 R
...We now have the effectiveness measure
... |
| 167 |
the possible ordering of this set is ignored
...Now,an intuitive way of measuring the adequacy of the retrieved set is to measure the size of the shaded area
...which is a simple composite measure
...The preceding argument in itself is not sufficient to justify the use of this particular composite measure
... |
| 10 |
In the past there has been much debate about the validity of evaluations based on relevance judgments provided by erring human beings
...Effectiveness and efficiency Much of the research and development in information retrieval is aimed at improving the effectiveness and efficiency of retrieval
... |
| 166 |
differ considerably from those which the user feels are pertinent Senko [21]...Fourthly,whereas Cooper has gone to some trouble to take account of the random element introduced by ties in the matching function,it is largely ignored in the derivation of Pnorm and Rnorm
...One further comment of interest is that Robertson 15 has shown that normalised recall has an interpretation as the area under the Recall Fallout curve used by Swets
...Finally mention should be made of two similar but simpler measures used by the SMART system
...and do not take into account the collection size N,n is here the number of relevant documents for the particular test query
...A normalised symmetric difference Let us now return to basics and consider how it is that users could simply measure retrieval effectiveness
... |
| 172 |
structures are decomposable
...A further simplification of the measurement function may be achieved by requiring a special kind of non interaction of the components which has become known as additive independence
...R,P >R,P <>[[Phi]]1 R [[Phi]]2 P >[[Phi]]1 R [[Phi]]2 P where F is simply the addition function
...R,P >R,P <>[[Phi]]1 R [[Phi]]2 P [[Phi]]1 R [[Phi]]2 P >[[Phi]]1 R [[Phi]]2 P [[Phi]]1 R [[Phi]]2 P It can be shown that starting at the other end given an additively independent representation the properties defined in 1 and 3,and the Archimedean property are necessary
...Here the term [[Phi]]1 [[Phi]]2 is referred to as the interaction term,its absence accounts for the non interaction in the previous condition
...We are now in a position to state the main representation theorem
...Theorem Suppose <R x P,>>is an additive conjoint structure,then there exist functions,[[Phi]]1 from R,and [[Phi]]2 from P into the real numbers such that,for all R,R [[propersubset]]R and P,P [[propersubset]]P:R,P >R,P <>[[Phi]]1 R [[Phi]]2 P >[[Phi]]1 R [[Phi]]2 P If [[Phi]]i []are two other functions with the same property,then there exist constants [[Theta]]>0,[[gamma]]1,and [[gamma]]2 such that [[Phi]]1 [][[Theta]][[Phi]]1 [[gamma]]1 [[Phi]]2 [][[Theta]][[Phi]]2 [[gamma]]2 The proof of this theorem may be found in Krantz et al
...Let us stop and take stock of this situation
... |
| 176 |
conjoint structure
...The analysis is not limited to the two factors precision and recall,it could equally well be carried out for say the pair fallout and recall
...Presentation of experimental results In my discussion of micro,macro evaluation,and expected search length,various ways of averaging the effectiveness measure of the set of queries arose in a natural way
...In this section the discussion will be restricted to single number measures such as a normalised symmetric difference,normalised recall,etc
...The measurements we have therefore are Za Q 1,Za Q 2,... |
| 146 |
There has been much debate in the past as to whether precision and recall are in fact the appropriate quantities to use as measures of effectiveness
...1 the most commonly used pair;2 fairly well understood quantities
...The final question How to evaluate?has a large technical answer
...Before proceeding to the technical details relating to the measurement of effectiveness it is as well to examine more closely the concept of relevance which underlies it
...Relevance Relevance is a subjective notion
... |
| 148 |
relevant to an information need if and only if it contains at least one sentence which is relevant to that need
...Earlier on I stated that this notion of relevance was only of limited use at the moment
...Saracevic [8]has summarised some of the more recent work on probabilistic interpretations of relevance
...Precision and recall,and others We now leave the speculations about relevance and return to the promised detailed discussion of the measurement of effectiveness
...It is helpful at this point to introduce the famous contingency table which is not really a contingency table at all
... |
| 108 |
If the summations instead of being over A and A are now made over A [[intersection]]Bi and A [[intersection]]Bi where Bi is the set of retrieved documents on the i th iteration,then we have a query formulation which is optimal for Bi a subset of the document collection
...where wi and w 2 are weighting coefficients
...Experiments have shown that relevance feedback can be very effective
...Finally,a few comments about the technique of relevance feedback in general
...Bibliographic remarks The book by Lancaster and Fayen [16]has written an interesting survey article about on line searching
... |
|
|
