Concepts
Similar pages
| Similarity |
Page |
Snapshot |
| 171 |
In other words we are ensuring that the equation R,P R,P is soluble for R provided that there exist R,R such that R,P >R,P >R,P
...The fifth condition is not limiting in any way but needs to be stated
...Definition 5
...Thus we require that variation in one while leaving the other constant gives a variation in effectiveness
...Finally we need a technical condition which will not be explained here,that is the Archimedean property for each component
...We now have six conditions on the relational structure <R x P,>>which in the theory of measurement are necessary and sufficient conditions for it to be an additive conjoint structure
...In our case we can therefore expect to find real valued functions [[Phi]]1 on R and [[Phi]]2 on P and a function F from Re x Re into Re,1:1 in each variable,such that,for all R,R [[propersubset]]R and P,P [[propersubset]]P we have:R,P >R,P <>F [[[Phi]]1 R,[[Phi]]2 P]>F [[[Phi]]1 R,[[Phi]]2 P]Note that although the same symbol >is used,the first is a binary relation on R x P,the second is the usual one on Re,the set of reals
...In other words there are numerical scales [[Phi]]i on the two components and a rule F for combining them such that the resultant measure preserves the qualitative ordering of effectiveness
... |
| 169 |
The model We start by examining the structure which it is reasonable to assume for the measurement of effectiveness
...If R is the set of possible recall values and P is the set of possible precision values then we are interested in the set R x P with a relation on it
...Definition 1
...1 Connectedness:either e 1 >e 2 or e 2 >e 1 2 Transitivity:if e 1 >e 2 and e 2 >e 3 then e 1 >e 3 We insist that if two pairs can be ordered both ways then R 1,P 1 R 2,P 2,i
...We now turn to a second condition which is commonly called independence
...Definition 2
...All we are saying here is,given that at a constant recall precision we find a difference in effectiveness for two values of precision recall then this difference cannot be removed or reversed by changing the constant value
...We now come to a condition which is not quite as obvious as the preceding ones
... |
| 176 |
conjoint structure
...The analysis is not limited to the two factors precision and recall,it could equally well be carried out for say the pair fallout and recall
...Presentation of experimental results In my discussion of micro,macro evaluation,and expected search length,various ways of averaging the effectiveness measure of the set of queries arose in a natural way
...In this section the discussion will be restricted to single number measures such as a normalised symmetric difference,normalised recall,etc
...The measurements we have therefore are Za Q 1,Za Q 2,... |
| 180 |
effectiveness can be calculated to infinite precision we may be insisting on a difference when in fact it only occurs in the tenth decimal place
...Finally,although I have just explained the use of the sign test in terms of single number measures,it is also used to detect a significant difference between precision recall graphs
...Bibliographic remarks Quite a number of references to the work on evaluation have already been given in the main body of the chapter
...Buried in the report by Keen Digger [32]Chapter 16 is an excellent discussion of the desirable properties of any measure of effectiveness
...A parameter which I have mentioned in passing but which deserves closer study in generality
...The trade off between precision and recall has for a long time been the subject of debate
...Guazzo [39]describe an approach to the measurement of retrieval effectiveness based on information theory
...The notion of relevance has at all times attracted much discussion
... |
|
|
