add declump tests following pyDeClump doc by barentine · Pull Request #932 · python-microscopy/python-microscopy

barentine · 2021-03-06T23:51:10Z

Addresses issue #pydeclump gap parameter doesn't work like I'd expect.

Is this a bugfix or an enhancement?
just a test
Proposed changes:

codecov · 2021-03-06T23:56:30Z

Codecov Report

Merging #932 (2e1e96f) into master (7dcb2df) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #932      +/-   ##
==========================================
+ Coverage   11.30%   11.33%   +0.02%     
==========================================
  Files         625      625              
  Lines       71915    71915              
==========================================
+ Hits         8132     8150      +18     
+ Misses      63783    63765      -18

Impacted Files	Coverage Δ
PYME/localization/FitFactories/InterpFitR.py	`78.18% <0.00%> (-2.73%)`	⬇️
PYME/localization/FitFactories/Dumbell3DFitR.py	`98.66% <0.00%> (+4.00%)`	⬆️
PYME/Analysis/points/DeClump/pyDeClump.py	`26.05% <0.00%> (+15.12%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dcb2df...2e1e96f. Read the comment docs.

David-Baddeley · 2021-03-07T20:43:47Z

These are based on an incorrect understanding of what the code should do (see #933).

The behaviour of the functions is correct (emitters within the same frame should get different labels). The IndexError in test_pydeclump_gap is due to the fact that the sig_x (dists in the test code) array is the wrong length.

fix localization setting url

…oval Fix interal hyperlink in installation.rst

…roscopy into declumptest

barentine · 2021-03-08T15:40:03Z

Hi @David-Baddeley, sorry I broke my tests with that dist length issue. It's fixed now, but my point still stands actually. pyDeClump.findClumps behavior differs from deClump.findClumps in a way which is exposed in the _gap tests in this PR

barentine · 2021-03-08T15:47:26Z

As things stand now pyDeClump.findClumps with nFrame= neither groups a gap of 0 frames nor a gap of 1 frame

In [4]:     from PYME.Analysis.points.DeClump import pyDeClump
   ...:     import numpy as np
   ...:     x = np.array([0, 0, 0])
   ...:     y = x
   ...:     t = np.array([0, 1, 3], dtype=np.int32)
   ...:     dist = 2 * np.ones(len(x))
   ...:     assigned = pyDeClump.findClumps(t.astype(np.int32), x, y, dist, 1)
   ...:     np.testing.assert_array_equal([1, 1, 2], assigned)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-4-7aab478435ec> in <module>
      6 dist = 2 * np.ones(len(x))
      7 assigned = pyDeClump.findClumps(t.astype(np.int32), x, y, dist, 1)
----> 8 np.testing.assert_array_equal([1, 1, 2], assigned)

    [... skipping hidden 1 frame]

/opt/miniconda3/lib/python3.7/site-packages/numpy/testing/_private/utils.py in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf)
    838                                 verbose=verbose, header=header,
    839                                 names=('x', 'y'), precision=precision)
--> 840             raise AssertionError(msg)
    841     except ValueError:
    842         import traceback

AssertionError: 
Arrays are not equal

Mismatched elements: 2 / 3 (66.7%)
Max absolute difference: 1
Max relative difference: 0.5
 x: array([1, 1, 2])
 y: array([1, 2, 3], dtype=int32)

while deClump.findClumps does link a gap of 0 frames but does not link a gap of 1 frame

In [5]:     from PYME.Analysis.points.DeClump import deClump
   ...:     import numpy as np
   ...:     x = np.array([0, 0, 0])
   ...:     y = x
   ...:     t = np.array([0, 1, 3], dtype=np.int32)
   ...:     dist = 2 * np.ones(len(x))
   ...:     assigned = deClump.findClumps(t.astype(np.int32), x.astype(np.float32), y.astype(np.float32), dist.astype(np.float32), 1)
   ...:     np.testing.assert_array_equal([1, 1, 2], assigned)
nPts = 3

barentine · 2021-03-08T16:00:01Z

Have pushed a commit aligning tests with the intention you mention in #933

Details

(base) Andrews-MacBook-Pro-5:points Andrew$ pytest test_clumps.py 
============================================================================ test session starts =============================================================================
platform darwin -- Python 3.7.7, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/Andrew/code/python-microscopy
collected 8 items                                                                                                                                                            

test_clumps.py ....FF..                                                                                                                                                [100%]

================================================================================== FAILURES ==================================================================================
_______________________________________________________________________ test_pydeclump_findclumps_gap ________________________________________________________________________

    def test_pydeclump_findclumps_gap():
        from PYME.Analysis.points.DeClump import pyDeClump
        import numpy as np
        x = np.array([0, 0, 0])
        y = x
        t = np.array([0, 1, 3], dtype=np.int32)
        dist = 2 * np.ones(len(x))
        assigned = pyDeClump.findClumps(t.astype(np.int32), x, y, dist, 1)
        # should group all points as there is only a 1 frame gap
>       np.testing.assert_array_equal([1, 1, 1], assigned)
E       AssertionError: 
E       Arrays are not equal
E       
E       Mismatched elements: 2 / 3 (66.7%)
E       Max absolute difference: 2
E       Max relative difference: 0.66666667
E        x: array([1, 1, 1])
E        y: array([1, 2, 3], dtype=int32)

test_clumps.py:54: AssertionError
________________________________________________________________________ test_declump_findclumps_gap _________________________________________________________________________

    def test_declump_findclumps_gap():
        from PYME.Analysis.points.DeClump import deClump
        import numpy as np
        x = np.array([0, 0, 0])
        y = x
        t = np.array([0, 1, 3], dtype=np.int32)
        dist = 2 * np.ones(len(x))
        assigned = deClump.findClumps(t.astype(np.int32), x.astype(np.float32), y.astype(np.float32), dist.astype(np.float32), 1)
        # should group all points as there is only a 1 frame gap
>       np.testing.assert_array_equal([1, 1, 1], assigned)
E       AssertionError: 
E       Arrays are not equal
E       
E       Mismatched elements: 1 / 3 (33.3%)
E       Max absolute difference: 1
E       Max relative difference: 0.5
E        x: array([1, 1, 1])
E        y: array([1, 1, 2], dtype=int32)

test_clumps.py:65: AssertionError
========================================================================== short test summary info ===========================================================================
FAILED test_clumps.py::test_pydeclump_findclumps_gap - AssertionError: 
FAILED test_clumps.py::test_declump_findclumps_gap - AssertionError: 
======================================================================== 2 failed, 6 passed in 0.45s =========================================================================

David-Baddeley · 2021-04-09T00:08:55Z

I think this is now redundant with recent changes

zacsimile and others added 2 commits March 4, 2021 16:37

Fix interal hyperlink in installation.rst

62d2f99

add declump tests following pyDeClump doc

0c560b0

add tests for c version

f58a28a

barentine mentioned this pull request Mar 7, 2021

[thing] deClump/pyDeClump.findClumps - missing doc, questionable gap parameter? #933

Closed

barentine and others added 5 commits March 7, 2021 17:06

fix localization setting url

553c726

Merge pull request python-microscopy#938 from barentine/uiloc

48cccd7

fix localization setting url

Merge pull request python-microscopy#924 from zacsimile/hyperlink-rem…

c8d04ba

…oval Fix interal hyperlink in installation.rst

Merge branch 'declumptest' of https://github.com/barentine/python-mic…

f9a4ba6

…roscopy into declumptest

fix len issue

968baa7

clarify test names and align tests with intention

2e1e96f

David-Baddeley closed this Apr 9, 2021

Navigation Menu

Uh oh!

add declump tests following pyDeClump doc - #932

add declump tests following pyDeClump doc#932
barentine wants to merge 9 commits into
python-microscopy:masterfrom
barentine:declumptest

barentine commented Mar 6, 2021

Uh oh!

codecov Bot commented Mar 6, 2021 •

edited

Loading

Uh oh!

David-Baddeley commented Mar 7, 2021

Uh oh!

barentine commented Mar 8, 2021

Uh oh!

barentine commented Mar 8, 2021 •

edited

Loading

Uh oh!

barentine commented Mar 8, 2021

Uh oh!

David-Baddeley commented Apr 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

barentine commented Mar 6, 2021

Uh oh!

codecov Bot commented Mar 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

David-Baddeley commented Mar 7, 2021

Uh oh!

barentine commented Mar 8, 2021

Uh oh!

barentine commented Mar 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

barentine commented Mar 8, 2021

Uh oh!

David-Baddeley commented Apr 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Mar 6, 2021 •

edited

Loading

barentine commented Mar 8, 2021 •

edited

Loading