lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Garski (JIRA)" <>
Subject [jira] Updated: (LUCENENET-288) Function.TestCustomScoreQuery
Date Thu, 19 Nov 2009 20:49:39 GMT


Michael Garski updated LUCENENET-288:

    Attachment: ExplanationComparison.txt

The root cause of the issue IS a floating point arithmetic issue.

The tests in TestCustomScoreQuery exercise a few custom function queries and then compare
the score from the scorer to the score from the explanation.  The tolerance factor in the
Java code base is used to account for floating point discrepancies as the scorer and explanation
may perform the calculation in different orders.

There are several sub-tests occurring within the test, once with the test index created that
consists of 17 documents, and others with wrapped indexes that have deleted documents in them,
changing the IDF value of the score due to the different value of MaxDocs().  It's the changed
IDF value in combination with the query boosts that throw the score differences outside the
range of the tolerance.  I modified the test to not add the document deletions and everything
passes.  I reverted the test and then modified the tests to only use query boosts of 2.0 or
less and everything passes.

I wanted to compare the results to what was happening in the JUnit tests, so I found the NUnit
case that had the greatest delta between the two scores and compared it to the same case in
JUnit (file showing the two comparisons is attached).  The differences are very minor due
to floating point precision, and I feel that altering the tolerance value of CheckHits.EXPLAIN_SCORE_TOLERANCE_DELTA
to 0.00025f, a value that is 5 times greater than in Java, is acceptable.  The attached patch
does nothing more than alter that tolerance value.

I spent yesterday afternoon and this morning going through this issue as CustomScoreQuery
is of particular importance to me as we use this internally to customize scoring for social
relevancy of a particular item.

If anyone is uncomfortable with the tolerance change feel free to investigate and propose
another solution.

> Function.TestCustomScoreQuery
> -----------------------------
>                 Key: LUCENENET-288
>                 URL:
>             Project: Lucene.Net
>          Issue Type: Bug
>            Reporter: Michael Garski
>         Attachments: ExplanationComparison.txt, LUCENENET-288.patch
> Three of the tests in TestCustomScoreQuery are not passing due to the comparison between
the explanation score and the hit score do not fall within the tolerance of +-0.00005f.  Altering
the tolerance to 0.0005f causes all of the tests to pass, so there appears to be a subtle
issue in how the scores are being calculated.
> A similar test exists in 2.4 that passes with the tolerance of 0.00005f, which to me
rules out floating point calculation differences.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message