Concepts
Similar pages
Similarity |
Page |
Snapshot |
| 42 |
nice property of being invariant under one to one transformations of the co ordinates
...A function very similar to the expected mutual information measure was suggested by Jardine and Sibson [2]specifically to measure dissimilarity between two classes of objects
...Here u and v are positive weights adding to unit
...P x P x w 1 P w 1 P x w 2 P w 2 x 0,1 P x wi P x wi P x i 1,2 we recover the expected mutual information measure I x,wi
... |
| 135 |
The way we interpret this hypothesis is that a term in the query used by a user is likely to be there because it is a good discriminator and hence we are interested in its close associates
...Discrimination power of an index term On p
...and in fact there made the comment that it was a measure of the power of term i to discriminate between relevant and non relevant documents
...Instead of Ki I suggest using the information radius,defined in Chapter 3 on p
... |
| 137 |
the different contributions made to the measure by the different cells
...Discrimination gain hypothesis In the derivation above I have made the assumption of independence or dependence in a straightforward way
...P xi,xj P xi,xj w 1 P w 1 P xi,xi w 2 P w 2 P xi P xj [P xi w 1 P w 1 P xi,w 2 P w 2][P xj w 1 P w 1 P xj,w 2 P w 2]If we assume conditional independence on both w 1 and w 2 then P xi,xj P xi,w 1 P xj,w 1 P w 1 P xi w 2 P xj w 2 P w 2 For unconditional independence as well,we must have P xi,xj P xi P xj This will only happen when P w 1 0 or P w 2 0,or P xi w 1 P xi w 2,or P xj w 1 P xj w 2,or in words,when at least one of the index terms is useless at discriminating relevant from non relevant documents
...Kendall and Stuart [26]define a partial correlation coefficient for any two distributions by |
| 136 |
probability functions we can write the information radius as follows:The interesting interpretation of the information radius that I referred to above is illustrated most easily in terms of continuous probability functions
...R u 1,u 2 v uI u 1 v vI u 2 v where I u i v measures the expectation on u i of the information in favour of rejecting v for u i given by making an observation;it may be regarded as the information gained from being told to reject v in favour of u i
...thereby removing the arbitrary v
...v u u 1 v u 2 that is,an average of the two distributions to be discriminated
...p x p x w 1 P w 1 p x w 2 P w 2 defined over the entire collection without regard to relevance
...There is one technical problem associated with the use of the information radius,or any other discrimination measure based on all four cells of the contingency table,which is rather difficult to resolve
... |
| 139 |
I must emphasise that the above argument leading to the hypothesis is not a proof
...One consequence of the discrimination hypothesis is that it provides a rationale for ranking the index terms connected to a query term in the dependence tree in order of I term,query term values to reflect the order of discrimination power values
...Bibliographic remarks The basis background reading for this chapter is contained in but a few papers
... |
| 120 |
convenience let us set There are a number of ways of looking at Ki
...Typically the weight Ki N,r,n,R is estimated from a contingency table in which N is not the total number of documents in the system but instead is some subset specifically chosen to enable Ki to be estimated
...The index terms are not independent Although it may be mathematically convenient to assume that the index terms are independent it by no means follows that it is realistic to do so
... |
|
|