Huge numeric discrepancies on mac_arm and other architectures

In some persistence tests, we had to relax numeric limits to prevent failure under certain architectures.

There are two possible causes:

  • Either our current measure of discrepancy is unreasonably blowing up irrelevant differences in channels with very low intensity (see below, ex #953 (closed)).
  • Or some of our computations are numerically unstable, and ought to be improved.

The problem appeared on Windows after an OS upgrade (#129 (closed)), then on Mac_arm after a security update (#597 (closed)). For the concerned tests (#597 (closed), #817 (closed)), limits were relaxed globally.

Then many more discrepancies were found on other architectures supported by the Debian science project (#952 (closed)). For these, an extra CMake option BA_3ARCH was introduced.

Hint:

  • Persistence tests are controlled by Tests/Functional/Persist/CMakeLists.txt
  • Individual test cases are implemented in Tests/Suite/Common/TestSuite.h

Discrepancies are mean relative differences as computed by DiffUtil::checkRelativeDifference

Instead, we should look at the worst data bin. For two reasons:

  • Fundamentally, we want each single computation to be correct.
  • We then could make error reports more specific, which in turn might elp in debugging.

But there is also a good reason for the current solution, namely that we should not be overly concerned about relative differences in bins that have very low absolute intensity (relative to high-intensity bins of course). Therefore we first need to invent a reasonable measure for worst-case deviations.

Edited by Joachim Wuttke