TST: add regression tests for issue #12216#16019
Conversation
|
Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.
|
|
👋 Thank you for your draft pull request! Do you know that you can use |
75ecc94 to
6cd7b50
Compare
|
The FITS compression code is being heavily refactored in #15474, so let's ping @astrofrog :) |
|
Oh, I wasn't aware, thanks for noticing ! if it helps, maybe we should switch this one to draft mode for now ? |
|
The bug this PR is fixing was reported back in 5.x. Looks like this PR is good to backport, but #15474 is not till v7 at the earliest, so if Tom R is okay with this patch, we should accept it just so we do not have to wait till v7 for a fix? |
6cd7b50 to
0e4778e
Compare
|
I rebased to refresh CI. Let's actually ping @astrofrog while I'm here. |
|
@astrofrog does this need to wait for #15474 ? |
0e4778e to
8a179e8
Compare
|
Ideally yes otherwise it will not be a trivial conflict to resolve. |
|
you got it |
|
@astrofrog - do you think those tests would be useful ? |
|
I guess we might as well, though not sure if we need it to be that extensive - maybe we can just check the compression options in the compressed header that was written out? (Or maybe it wouldn't be enough to illustrate the problem?) |
|
would the following assertion match your suggestion @astrofrog ? It appears to pass on assert hdul[1].header["BITPIX"] == hdul2[1].header["BITPIX"] |
|
Yes partially but I am thinking more along the lines of checking e.g. |
|
None of these options seemed to appear in the headers I considered while looking into this. Maybe I'm not looking in the right place ? |
|
@neutrinoceros - are you opening the file disabling the compression in order to access the header? (these will just be in the compressed header): In [1]: import numpy as np
In [2]: from astropy.io import fits
In [3]: hdu = fits.CompImageHDU(np.random.random((128, 128)))
In [5]: hdu.writeto('test.fits')
In [6]: hdu2 = fits.open('test.fits', disable_image_compression=True)[1]
In [7]: hdu2.header
Out[7]:
XTENSION= 'BINTABLE' / binary table extension
BITPIX = 8 / array data type
NAXIS = 2 / number of array dimensions
NAXIS1 = 32 / width of table in bytes
NAXIS2 = 128 / number of rows in table
PCOUNT = 14585 / number of group parameters
GCOUNT = 1 / number of groups
TFIELDS = 4 / number of fields in each row
TTYPE1 = 'COMPRESSED_DATA' / label for field 1
TFORM1 = '1PB(122)' / data format of field: variable length array
TTYPE2 = 'GZIP_COMPRESSED_DATA' / label for field 2
TFORM2 = '1PB(0) ' / data format of field: variable length array
TTYPE3 = 'ZSCALE ' / label for field 3
TFORM3 = '1D ' / data format of field: 8-byte DOUBLE
TTYPE4 = 'ZZERO ' / label for field 4
TFORM4 = '1D ' / data format of field: 8-byte DOUBLE
ZIMAGE = T / extension contains compressed image
ZTENSION= 'IMAGE ' / Image extension
ZBITPIX = -64 / data type of original image
ZNAXIS = 2 / dimension of original image
ZNAXIS1 = 128 / length of original image axis
ZNAXIS2 = 128 / length of original image axis
ZPCOUNT = 0 / number of parameters
ZGCOUNT = 1 / number of groups
ZTILE1 = 128 / size of tiles to be compressed
ZTILE2 = 1 / size of tiles to be compressed
ZCMPTYPE= 'RICE_1 ' / compression algorithm
ZNAME1 = 'BLOCKSIZE' / compression block size
ZVAL1 = 32 / pixels per block
ZNAME2 = 'BYTEPIX ' / bytes per pixel (1, 2, 4, or 8)
ZVAL2 = 4 / bytes per pixel (1, 2, 4, or 8)
ZNAME3 = 'NOISEBIT' / floating point quantization level
ZVAL3 = 16.0 / floating point quantization level
ZQUANTIZ= 'NO_DITHER' / No dithering during quantization
EXTNAME = 'COMPRESSED_IMAGE' / name of this binary table extension |
8a179e8 to
60c6d1f
Compare
|
Thanks ! I cleaned up this PR and now it's all about adding my tests |
60c6d1f to
e052853
Compare
| "ZCMPTYPE", | ||
| "ZQUANTIZ", | ||
| ] | ||
| } |
There was a problem hiding this comment.
To save a few lines, maybe just for k in expected_zflags?
e052853 to
12158ed
Compare
|
Thanks! I updated the labels and also moved the milestone to match #15474 |
|
Ugh the wheel jobs got stuck. Maybe I need to prioritize fixing the labeled event for that job. |
| data = prng.random((50, 30)).astype("float32") | ||
|
|
||
| hdu = fits.CompImageHDU(**init_kwargs) | ||
| hdu.data = data.copy() |
There was a problem hiding this comment.
Indeed. It's a remnant of the initial bug report, but it's not necessary to reproduce the problem. Thanks for catching it !
Co-authored-by: Simon Conseil <[email protected]>
44e9a6c to
4d6a72f
Compare
saimn
left a comment
There was a problem hiding this comment.
Sounds good, thanks @neutrinoceros !
Description
Fixes #12216I note that #12058 was listed a potential partial solution, and I haven't dug deeply into it, but found that it didn't resolve the issue. Hopefully this approach is suitable.If not, I've listed other possible venues, see #12216 (comment)
Blocked by #15474update 2024-10-03: the bug was fixed in #15474, so I revamped this PR to only add the test suite I wrote for it.