Skip to content

NumPy Float64 not being passed correctly #90

Closed

Description

@vmuriart

Possibly related to #86. When passing a numpy float64 to a dll library using PythonNet, the object type changes (not sure to what type).
I have tried to reproduce the error on the dll directly, by passing the same value with different types and it works as expected. Any ideas?

Activity

  1. vmuriart commented on Aug 19, 2015

    @vmuriart
    ContributorAuthor

    Executing the same code using Win32COM does not produce an error either.

  2. den-run-ai commented on Aug 19, 2015

    @den-run-ai
    Contributor

    can you show minimal example and how you figured out that type is changed?

    do examples on this page work for you?

    https://github.com/pythonnet/pythonnet/tree/develop

  3. vmuriart commented on Aug 20, 2015

    @vmuriart
    ContributorAuthor

    Just saw the C# examples for numpy. But I am going the other way around. I am trying to pass the numpy float64 object as an argument to the clr package. To give some context, the package is a customized dbapi wrapper.

    import clr
    import numpy as np
    clr.AddReference('Custom.DB.API')
    from Custom.DB.API import Query()
    
    q = Query('select :num from dual')
    q.parameter('num',np.float(1)) #works
    q.parameter('num',np.long(1)) #works
    q.parameter('num',np.double(1)) #breaks
    

    But the same example using Win32Com instead works without any issues.

  4. den-run-ai commented on Aug 25, 2015

    @den-run-ai
    Contributor

    @vmuriart I could replicate your problem:

    import clr
    from System import Array, Object
    import numpy as np
    
    arr3=Array[Object]([np.float(1),np.long(1)])
    list(arr3)  # returns [1.0, 1]
    

    But:

    arr4=Array[Object]([np.float(1),np.double(1)])
    

    returns:

    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-53-fee0ccf03f95> in <module>()
    ----> 1 arr4=Array[Object]([np.float(1),np.double(1)])
    
    TypeError: value cannot be converted to Object
    
  5. den-run-ai commented on Aug 25, 2015

    @den-run-ai
    Contributor

    @tonyroberts how can this issue be debugged? is this failing in the weird IL code?

  6. tonyroberts commented on Aug 25, 2015

    @tonyroberts
    Contributor

    @denfromufa just search for the error in the code (it's in converter.cs) and it might be fairly clear what's going on just from looking at the code immediately before the error. Otherwise, set a breakpoint where the error is being set, attach the visual studio debugger to your python process and run the python code and work back from there.

    In this case I imagine that the C# code is checking for an exact type match with 'float' which is failing for np.double. If that is the case, changing it to an 'isinstance' style check instead would fix it I would think.

  7. vmuriart commented on Aug 30, 2015

    @vmuriart
    ContributorAuthor

    Pardon my low level of understanding. I surveyed the code, and from what I think I understand, on runtime.cs you are creating c-type objects from the python library that correspond to Python Objects. Then the code compares incoming objects against this reference object to determine if its of the same type to then convert it?
    The numpy double isn't identifying itself as a python standard object unless I explicitly cast it to change itself into one. I'm browsing through the numpy documentation to see what type of C object it identifies itself as.
    dont know if this helps maybe.

  8. den-run-ai commented on Aug 31, 2015

    @den-run-ai
    Contributor

    to be more precise numpy double fails here:

    return new IntPtr((void*)(*((ulong*)p + n)));
    

    call stack

    Python.Runtime.dll!Python.Runtime.Runtime.PyObject_TYPE(System.IntPtr op) Line 898  C#
    Python.Runtime.dll!Python.Runtime.Runtime.PyFloat_Check(System.IntPtr ob) Line 1235 C#
    Python.Runtime.dll!Python.Runtime.Converter.ToManagedValue(System.IntPtr value, System.Type obType, out object result, bool setError) Line 311  C#
    Python.Runtime.dll!Python.Runtime.Converter.ToManaged(System.IntPtr value, System.Type type, out object result, bool setError) Line 220 C#
    

    locals

            op  {483440320} System.IntPtr
            p   0x000000001cd0b6c0  void*
            n   1   int
    
    

    auto

            (ulong*)p   0x000000001cd0b6c0  ulong*
            is32bit false   bool
            n   1   int
            p   0x000000001cd0b6c0  void*
    
    
  9. den-run-ai commented on Aug 31, 2015

    @den-run-ai
    Contributor

    to make numpy floats and integerswork with pythonnet, it is necessary to useisinstance()withnumber.Realandnumber.Integralrespectively. simply trying to compare withpython float and intdoes not work fornumpy` objects.

  10. vmuriart commented on Sep 1, 2015

    @vmuriart
    ContributorAuthor

    That sounds logical. I tried to look at how pywin32 handles the conversion, but their source isn't as navigation friendly. Have you tested to see if any of the other numpy objects break as well?

  11. den-run-ai commented on Sep 1, 2015

    @den-run-ai
    Contributor

    Yep, bunch of them break, np.float did not break because it maps to float in Python.

  12. den-run-ai commented on Sep 3, 2015

    @den-run-ai
    Contributor

    I did not find how pywin32 works, but this document shows mapping between com types and numpy types in comtypes package:

    http://pythonhosted.org/comtypes/#numpy-interop

  13. filmor commented on Aug 6, 2021

    @filmor
    Member

    This seems to be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions