Repository navigation
Clarify JSON marshalling of Decimal values #751
Description
About
How to marshal values of Python's Decimal type.
Status quo
Currently, the library is conveying Decimal values as strings.
crate-python/src/crate/client/http.py
Lines 105 to 106 in 7c7a885
See also
Others apparently need serialization to float.
if isinstance(obj, Decimal):
return float(obj)
Activity
surister commentedon Dec 16, 2025
To me it seems that serializing it to
floatis the best case, casting to both gives correct precisionup until one point, then
strjust adds binary floating point noise:but only up to some precision:
but in str
Thoughts?
amotl commentedon Dec 16, 2025
Yes, I think that's the whole point here: Converging to
floatis bound to IEEE 754 floating-point arithmetic precision losses, while converging tostrretains precision based on user's needs, which would be the right choice to transport numbers with fixed-point arithmetics, if the transport channel does not offer any other (dedicated) data type for that purpose. My university days are years in the past, so please correct me where you think I'm wrong.There is no floating point "noise", nor any sort of rounding errors, if you just stop using the
floatdata type, also in literal form in code.stris certainly the best representation format in ASCII which will not incur losses, that's why we are using it firsthand when marshalling to JSON.amotl commentedon Dec 16, 2025
Recap: We are running this discussion because we discovered the Singer/Meltano database adapters apparently expect
floatvalues at some spot I touched the other day when conceiving meltano-target-cratedb, most probably due to needing to satisfy a test case of target-postgres. Contrary to that need of a downstream database connector component, the crate-python driver currently serializes Python's Decimal data type tostr.NB: I will need to revisit the code base to be able to point out the spot why exactly this patch was needed. Maybe I was chasing ghosts here? @edgarrmondragon: Does anything around float/decimal serialization in Meltano database adapters ring any bell with you?
surister commentedon Dec 16, 2025
Right, had not seen that I was using a float literal instead of a string, airport jet-lag I guess. Yeah, it seems that using a
stris the best solution, is there anything that needs to be done in this repository seeing that we already serialize to string?amotl commentedon Dec 16, 2025
If some downstream applications or frameworks absolutely need
floats at this point, and can't cope by their own, the crate-python library could possibly offer configurability.edgarrmondragon commentedon Dec 18, 2025
there was python-jsonschema/jsonschema#701, but that was fixed a while back
amotl commentedon Dec 18, 2025
Thank you very much. After reading the conversation you've shared I also think this isn't an issue here. I think I will need to review the original spot at meltano-target-cratedb to recap what has been going on there about the marshalling.
Decimalvalues meltano-target-cratedb#52I will report here as soon as I have more information.