Skip to content
Snippets Groups Projects
New look: Off

Investigate difference in results when using newer OpenCV version

    • Closed Issue created by Schrödter, Tobias @t.schroedter

    When using a newer OpenCV version than 4.2. leads to failing reference tests, meaning that a significant difference in the results occurs. This prevents us from moving to a newer version of OpenCV which makes it difficult on some systems to compile PeTrack without compiling OpenCV. In particular some package managers to have this particular OpenCV version , e.g, pacman, homebrew.

    Following should be done:

    • run PeTrack for multicolor and aruco TdN test data sets
    • if possible find the origin of the differences
    • find a measure to judge results: Are the results getting better or worse with newer OpenCV version?
    • decide if a update to a newer OpenCV version is possible

    This issue can be used to store the information from the different steps.

    0 of 4 checklist items completed

    Designs

      Child items ...

      2. Activity

      • Schrödter, Tobias added discussion documentation high tooling labels

        added discussion documentation high tooling labels

      • Schrödter, Tobias
        Schrödter, Tobias @t.schroedter ·
        Author Owner

        Results from the failing tests with OpenCV 4.2.0 (our currently used version) and OpenCV 4.5.0: https://fz-juelich.sciebo.de/s/kR3jQFoiyc4AUEI

        Failing tests:

        • multiColor
        Click here for test output. 
        _______________________________________________ TestTrackerTxt.test_e2e_via_txt[multicolor] _______________________________________________
Diverging coordinates at Frame: 71 X: 9.27332 Y: 0.730025 Z: 1.68 in the test file.
At Frame: 71 X: 9.27479 Y: 0.723113 Z: 1.68 in accepted truth.
For person ID: 9 in test file and person  ID: 9 in truth file 

Diverging coordinates at Frame: 140 X: 10.8411 Y: -1.21309 Z: 1.68 in the test file.
At Frame: 140 X: 10.8394 Y: -1.20553 Z: 1.68 in accepted truth.
For person ID: 19 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 41 X: 10.6933 Y: 3.17405 Z: 1.79 in the test file.
At Frame: 41 X: 10.7062 Y: 3.17568 Z: 1.79 in accepted truth.
For person ID: 35 in test file and person  ID: 35 in truth file 

Diverging coordinates at Frame: 132 X: 8.79202 Y: -0.550019 Z: 1.79 in the test file.
At Frame: 132 X: 8.78634 Y: -0.553696 Z: 1.79 in accepted truth.
For person ID: 38 in test file and person  ID: 38 in truth file 

Diverging coordinates at Frame: 338 X: 9.29458 Y: -0.659911 Z: 1.58 in the test file.
At Frame: 338 X: 9.29858 Y: -0.650257 Z: 1.58 in accepted truth.
For person ID: 74 in test file and person  ID: 74 in truth file 

Diverging coordinates at Frame: 282 X: 11.4351 Y: -1.05126 Z: 1.79 in the test file.
At Frame: 282 X: 11.4337 Y: -1.06581 Z: 1.79 in accepted truth.
For person ID: 78 in test file and person  ID: 78 in truth file 

Diverging coordinates at Frame: 376 X: 8.64069 Y: -1.16634 Z: 1.79 in the test file.
At Frame: 376 X: 8.63428 Y: -1.15904 Z: 1.79 in accepted truth.
For person ID: 78 in test file and person  ID: 78 in truth file
        • multiColorMarkerWithAruco_dictMip36h12
        Click here for test output. 
        _________________________________ TestTrackerTxt.test_e2e_via_txt[multiColorMarkerWithAruco_dictMip36h12] _________________________________
Diverging coordinates at Frame: 0 X: -1.51035 Y: -0.304495 Z: 1.0 in the test file.
At Frame: 0 X: -1.5188 Y: -0.307958 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 1 X: -1.50984 Y: -0.3044 Z: 1.0 in the test file.
At Frame: 1 X: -1.51823 Y: -0.307843 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 2 X: -1.50898 Y: -0.304059 Z: 1.0 in the test file.
At Frame: 2 X: -1.51744 Y: -0.307524 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 3 X: -1.50759 Y: -0.303577 Z: 1.0 in the test file.
At Frame: 3 X: -1.51606 Y: -0.30698 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 4 X: -1.5054 Y: -0.303044 Z: 1.0 in the test file.
At Frame: 4 X: -1.51389 Y: -0.3065 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 5 X: -1.50221 Y: -0.302158 Z: 1.0 in the test file.
At Frame: 5 X: -1.51072 Y: -0.305634 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 6 X: -1.49833 Y: -0.30161 Z: 1.0 in the test file.
At Frame: 6 X: -1.5069 Y: -0.305089 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 7 X: -1.4944 Y: -0.300733 Z: 1.0 in the test file.
At Frame: 7 X: -1.50298 Y: -0.304101 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 8 X: -1.49061 Y: -0.299594 Z: 1.0 in the test file.
At Frame: 8 X: -1.49924 Y: -0.302813 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 9 X: -1.48642 Y: -0.296241 Z: 1.0 in the test file.
At Frame: 9 X: -1.49514 Y: -0.299374 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 10 X: -1.48188 Y: -0.291765 Z: 1.0 in the test file.
At Frame: 10 X: -1.4907 Y: -0.294833 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 11 X: -1.47609 Y: -0.287036 Z: 1.0 in the test file.
At Frame: 11 X: -1.48487 Y: -0.289997 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 12 X: -1.46822 Y: -0.282145 Z: 1.0 in the test file.
At Frame: 12 X: -1.47711 Y: -0.285082 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 13 X: -1.4583 Y: -0.277552 Z: 1.0 in the test file.
At Frame: 13 X: -1.46716 Y: -0.280484 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 14 X: -1.44706 Y: -0.272349 Z: 1.0 in the test file.
At Frame: 14 X: -1.45591 Y: -0.275285 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 15 X: -1.43399 Y: -0.26505 Z: 1.0 in the test file.
At Frame: 15 X: -1.44275 Y: -0.267889 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 16 X: -1.4172 Y: -0.257242 Z: 1.0 in the test file.
At Frame: 16 X: -1.42592 Y: -0.260078 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 17 X: -1.39555 Y: -0.252362 Z: 1.0 in the test file.
At Frame: 17 X: -1.40426 Y: -0.255187 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 18 X: -1.37046 Y: -0.249375 Z: 1.0 in the test file.
At Frame: 18 X: -1.3792 Y: -0.252225 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 19 X: -1.34318 Y: -0.246676 Z: 1.0 in the test file.
At Frame: 19 X: -1.35194 Y: -0.249523 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 20 X: -1.31401 Y: -0.242711 Z: 1.0 in the test file.
At Frame: 20 X: -1.32266 Y: -0.245548 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 21 X: -1.28319 Y: -0.238057 Z: 1.0 in the test file.
At Frame: 21 X: -1.29188 Y: -0.240904 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 22 X: -1.25086 Y: -0.231546 Z: 1.0 in the test file.
At Frame: 22 X: -1.25958 Y: -0.234474 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 23 X: -1.21634 Y: -0.222881 Z: 1.0 in the test file.
At Frame: 23 X: -1.22496 Y: -0.225885 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 24 X: -1.18283 Y: -0.212003 Z: 1.0 in the test file.
At Frame: 24 X: -1.19136 Y: -0.215045 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 25 X: -1.14954 Y: -0.198558 Z: 1.0 in the test file.
At Frame: 25 X: -1.15791 Y: -0.201729 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 26 X: -1.117 Y: -0.182511 Z: 1.0 in the test file.
At Frame: 26 X: -1.1253 Y: -0.185775 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 27 X: -1.08532 Y: -0.164017 Z: 1.0 in the test file.
At Frame: 27 X: -1.09365 Y: -0.167202 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 28 X: -1.05302 Y: -0.144277 Z: 1.0 in the test file.
At Frame: 28 X: -1.06136 Y: -0.1474 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 29 X: -1.0199 Y: -0.126887 Z: 1.0 in the test file.
At Frame: 29 X: -1.02838 Y: -0.129817 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 30 X: -0.992825 Y: -0.112782 Z: 1.0 in the test file.
At Frame: 30 X: -1.00144 Y: -0.115273 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 31 X: -0.974232 Y: -0.100153 Z: 1.0 in the test file.
At Frame: 31 X: -0.983065 Y: -0.102219 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 32 X: -0.965285 Y: -0.087649 Z: 1.0 in the test file.
At Frame: 32 X: -0.97436 Y: -0.0890937 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 33 X: -0.962654 Y: -0.0752395 Z: 1.0 in the test file.
At Frame: 33 X: -0.971943 Y: -0.0761509 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 34 X: -0.961051 Y: -0.0636771 Z: 1.0 in the test file.
At Frame: 34 X: -0.970513 Y: -0.0643178 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 35 X: -0.955301 Y: -0.0547828 Z: 1.0 in the test file.
At Frame: 35 X: -0.96482 Y: -0.0551929 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 36 X: -0.944434 Y: -0.0475362 Z: 1.0 in the test file.
At Frame: 36 X: -0.953941 Y: -0.0479194 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 37 X: -0.930154 Y: -0.0417027 Z: 1.0 in the test file.
At Frame: 37 X: -0.939638 Y: -0.042097 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 38 X: -0.913903 Y: -0.0369069 Z: 1.0 in the test file.
At Frame: 38 X: -0.923389 Y: -0.0373371 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 39 X: -0.896551 Y: -0.0315896 Z: 1.0 in the test file.
At Frame: 39 X: -0.90602 Y: -0.0321501 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 40 X: -0.877659 Y: -0.0255435 Z: 1.0 in the test file.
At Frame: 40 X: -0.887034 Y: -0.0262588 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 41 X: -0.858694 Y: -0.0178745 Z: 1.0 in the test file.
At Frame: 41 X: -0.868043 Y: -0.0187512 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 42 X: -0.838957 Y: -0.00920665 Z: 1.0 in the test file.
At Frame: 42 X: -0.848302 Y: -0.0101389 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 43 X: -0.817595 Y: -0.00129453 Z: 1.0 in the test file.
At Frame: 43 X: -0.826969 Y: -0.00228275 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 44 X: -0.796809 Y: 0.0059246 Z: 1.0 in the test file.
At Frame: 44 X: -0.806212 Y: 0.00486915 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 45 X: -0.779234 Y: 0.0129964 Z: 1.0 in the test file.
At Frame: 45 X: -0.788652 Y: 0.0119504 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 46 X: -0.762804 Y: 0.0185512 Z: 1.0 in the test file.
At Frame: 46 X: -0.772228 Y: 0.0176036 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 47 X: -0.745426 Y: 0.0240164 Z: 1.0 in the test file.
At Frame: 47 X: -0.75487 Y: 0.0231302 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 48 X: -0.73123 Y: 0.0315327 Z: 1.0 in the test file.
At Frame: 48 X: -0.740743 Y: 0.0306914 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 49 X: -0.721554 Y: 0.0382662 Z: 1.0 in the test file.
At Frame: 49 X: -0.731129 Y: 0.0374396 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 50 X: -0.712954 Y: 0.0441032 Z: 1.0 in the test file.
At Frame: 50 X: -0.722582 Y: 0.0433397 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 51 X: -0.705212 Y: 0.0483376 Z: 1.0 in the test file.
At Frame: 51 X: -0.714836 Y: 0.0475341 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 52 X: -0.698151 Y: 0.0495857 Z: 1.0 in the test file.
At Frame: 52 X: -0.707755 Y: 0.0487073 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 53 X: -0.690955 Y: 0.0459781 Z: 1.0 in the test file.
At Frame: 53 X: -0.700527 Y: 0.0449249 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 54 X: -0.682181 Y: 0.0373158 Z: 1.0 in the test file.
At Frame: 54 X: -0.691661 Y: 0.0362025 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 55 X: -0.672177 Y: 0.0292433 Z: 1.0 in the test file.
At Frame: 55 X: -0.681612 Y: 0.0280005 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 56 X: -0.663106 Y: 0.0230861 Z: 1.0 in the test file.
At Frame: 56 X: -0.672513 Y: 0.0217652 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 57 X: -0.655954 Y: 0.019642 Z: 1.0 in the test file.
At Frame: 57 X: -0.665408 Y: 0.018376 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 58 X: -0.648884 Y: 0.018407 Z: 1.0 in the test file.
At Frame: 58 X: -0.658337 Y: 0.0172154 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 59 X: -0.639035 Y: 0.016791 Z: 1.0 in the test file.
At Frame: 59 X: -0.648436 Y: 0.0155094 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 60 X: -0.624994 Y: 0.0146337 Z: 1.0 in the test file.
At Frame: 60 X: -0.634441 Y: 0.0134261 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 61 X: -0.609547 Y: 0.0130033 Z: 1.0 in the test file.
At Frame: 61 X: -0.618991 Y: 0.0118371 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 62 X: -0.592187 Y: 0.0114272 Z: 1.0 in the test file.
At Frame: 62 X: -0.601648 Y: 0.0103746 Z: 1.0 in accepted truth.
For person ID: 20 in test file and person  ID: 19 in truth file 

Diverging coordinates at Frame: 125 X: -1.1805 Y: -0.474745 Z: 1.0 in the test file.
At Frame: 125 X: -1.16415 Y: -0.4995 Z: 1.0 in accepted truth.
For person ID: 23 in test file and person  ID: 23 in truth file 

Diverging coordinates at Frame: 126 X: -1.18509 Y: -0.470721 Z: 1.0 in the test file.
At Frame: 126 X: -1.17788 Y: -0.480588 Z: 1.0 in accepted truth.
For person ID: 23 in test file and person  ID: 23 in truth file 

Different first frame for ID: 26(test) and ID: 26(truth)
First frame in truth file: 141
First frame in test file: 0
        Edited by Schrödter, Tobias
      • d.kilic mentioned in issue #218

        mentioned in issue #218

      • Schrödter, Tobias added To Do label

        added To Do label

      • Schrödter, Tobias mentioned in merge request !193 (merged)

        mentioned in merge request !193 (merged)

      • Schrödter, Tobias closed with merge request !193 (merged)

        closed with merge request !193 (merged)

      • Schrödter, Tobias mentioned in commit 7e1b1f18

        mentioned in commit 7e1b1f18

      Please register or sign in to reply