A short paper by Good[44] which is in sympathy with the approach based on a theory of measurement given here, discusses the evaluation of retrieval systems in terms of expected utility.

One conspicuous omission from this chapter is any discussion of cost-effectiveness. The main reason for this is that so far very little of importance can be said about it. A couple of attempts to work out mathematical cost models for IR are Cooper[45] and Marschak[46].

References

1. COOPER, W.S., 'On selecting a measure of retrieval effectiveness', Part 1: 'The "subjective" philosophy of evaluation', Part 2: 'Implementation of the philosophy', Journal of the American Society for Information Science, 24, 87-100 and 413-424 (1973).

2. JARDINE, N. and van RIJSBERGEN, C.J., 'The use of hierarchic clustering in information retrieval', Information Storage and Retrieval, 7, 217-240 (1971).

3. HEINE, M.H., 'Distance between sets as an objective measure of retrieval effectiveness', Information Storage and Retrieval, 9, 181-198 (1973).

4. COOPER, W.S., 'A definition of relevance for information retrieval', Information Storage and Retrieval, 7, 19-37 (1971).

5. BELNAP, N.D., An analysis of questions: Preliminary report, Scientific Report TM-1287, SDC, Santa Monica, California (1963).

6. BELNAP, N.D. and STEEL, T.B., The Logic of Questions and Answers, Yale University Press, New Haven and London (1976).

7. CHANG, C.L. and LEE, R.C.T., Symbolic Logic and Mechanical Theorem Proving, Academic Press, New York (1973).

8. SARACEVIC, T., 'Relevance: A review of and a framework for the thinking on the notion information science', Journal of the American Society for Information Science,

26, 321-343 (1975).

9. ROBERTSON, S.E., 'The probabilistic character of relevance', Information Processing and Management, 13, 247-251 (1977).

10. van RIJSBERGEN, C.J., 'Retrieval effectiveness', In: Progress in Communication Sciences, Vol. 1 (Edited by Melvin J. Voigt), 91-118 (1979).

11. KEEN, E.M., 'Evaluation parameters'. In Report ISR-13 to the National Science Foundation, Section IIK, Cornell University, Department of Computer Science (1967).

12. SWETS, J.A., 'Information retrieval systems', Science, 141, 245-250 (1963).

13. SWETS, J.A., Effectiveness of Information Retrieval Methods, Bolt, Beranek and Newman, Cambridge, Massachusetts (1967).

14. BROOKES, B.C., 'The measure of information retrieval effectiveness proposed by Swets', Journal of Documentation, 24, 41-54 (1968).

15. ROBERTSON, S.E., 'The parametric description of retrieval tests, Part 2: 'Overall measures', Journal of Documentation, 25, 93-107 (1969).

16. BOOKSTEIN, A., 'When the most "pertinent" document should not be retrieved - an analysis of the Swets Model', Information Processing and Management, 13, 377-383 (1977).

17. ROBERTSON, S.E. and TEATHER, D., 'A statistical analysis of retrieval tests: a Bayesian approach', Journal of Documentation, 30, 273-282 (1974).

18. EFRON, B. and MORRIS, C., 'Stein's parador in statistics', Scientific American, 236, 119-127 (1977).