| Similarity |
Page |
Snapshot |
| 146 |
There has been much debate in the past as to whether precision and recall are in fact the appropriate quantities to use as measures of effectiveness
...1 the most commonly used pair;2 fairly well understood quantities
...The final question How to evaluate?has a large technical answer
...Before proceeding to the technical details relating to the measurement of effectiveness it is as well to examine more closely the concept of relevance which underlies it
...Relevance Relevance is a subjective notion
... |
| 10 |
In the past there has been much debate about the validity of evaluations based on relevance judgments provided by erring human beings
...Effectiveness and efficiency Much of the research and development in information retrieval is aimed at improving the effectiveness and efficiency of retrieval
... |
| 154 |
Composite measures Dissatisfaction in the past with methods of measuring effectiveness by a pair of numbers e
...S P R This is simply related to a measure suggested by Borko BK P R 1 More complicated ones are Vickery s measure V can be shown to be a special case of a general measure which will be derived below
...Some single number measures have derivations which can be justified in a rational manner
...The Swets model As early as 1963 Swets [12]showed that the suggested modifications were in fact simply related to an alternative measure already suggested by Swets
...It is interesting that although the Swets model is theoreticallyattractive |
| 168 |
Foundation Problems of measurement have arisen in physics,psychology,and more recently,the social sciences
...The problems of measurement in information retrieval differ from those encountered in the physical sciences in one important aspect
...The next three sections are substantially the same as those appearing in my paper:Foundations of evaluation,Journal of Documentation,30,365 373 1974
...no reason why we cannot postulate a particular ordering,or,to put it more mildly,why we can not show that a certain model for the measurement of effectiveness has acceptable properties
...1 all properties ascribed are consistent;2 they bring out into the open all the assumptions made in measuring effectiveness;3 each property has an acceptable interpretation;4 the model leads to a plausible measure of effectiveness
...It is as well to point out here that it does not lead to a uniquemeasure,but it does show that certain classes of measures can beregarded as being equivalent
... |
| 176 |
conjoint structure
...The analysis is not limited to the two factors precision and recall,it could equally well be carried out for say the pair fallout and recall
...Presentation of experimental results In my discussion of micro,macro evaluation,and expected search length,various ways of averaging the effectiveness measure of the set of queries arose in a natural way
...In this section the discussion will be restricted to single number measures such as a normalised symmetric difference,normalised recall,etc
...The measurements we have therefore are Za Q 1,Za Q 2,... |
| 9 |
Sparck Jones has carried on this work using measures of association between keywords based on their frequency of co occurrence that is,the frequency with which any two keywords occur together in the same document
...The term information structure for want of better words covers specifically a logical organisation of information,such as document representatives,for the purpose of information retrieval
...The organisation of these files is produced by an automatic classification method
...Evaluation of retrieval systems has proved extremely difficult
... |
| 181 |
A short paper by Good [44]which is in sympathy with the approach based on a theory of measurement given here,discusses the evaluation of retrieval systems in terms of expected utility
...One conspicuous omission from this chapter is any discussion of cost effectiveness
...References 1
...2
...3
...4
...5
...6
...7
...8
...26,321 343 1975
...9
...10
...11
...12
...13
...14
...15
...16
...17
...18
... |
| 149 |
A large number of measures of effectiveness can be derived from this table
......There is a functional relationship between all three involving a parameter called generality G which is a measure of the density of relevant documents in the collection
...For each request submitted to a retrieval system one of these tables can be constructed
... |
| 148 |
relevant to an information need if and only if it contains at least one sentence which is relevant to that need
...Earlier on I stated that this notion of relevance was only of limited use at the moment
...Saracevic [8]has summarised some of the more recent work on probabilistic interpretations of relevance
...Precision and recall,and others We now leave the speculations about relevance and return to the promised detailed discussion of the measurement of effectiveness
...It is helpful at this point to introduce the famous contingency table which is not really a contingency table at all
... |
