Skip to content

Record return types and determine exposed modules for objects #15

Open

Description

@saulshanabrook

One thing we should add is the ability to record the return type from a function.

This would help us understand the signatures better.

Also, it could help with another issue, which is that currently we output functions/classes from the modules they are defined in, not the module they are imported from.

We should instead try to output them where they are imported from. However, if multiple libraries import from different locations, we should choose one and have the others be aliases.

One way to do that would be to record the return types from the getattr calls, so if you did something like import numpy; numpy.arange we would know the return type of the getattr is the arange function.

We currently record those getattr, but don't have the return type.

If we did, we could infer that any getattr from a module, which returns a class/fn from a different module represents an import alias.

Activity

  1. saulshanabrook commented on Jul 14, 2020

    @saulshanabrook
    ContributorAuthor

    We are currently looking at the Python bytecode stack to understand what operations are called, see:

    I think for getting the return values of functions, we might have to basically wait till the next bytecode is executed and then look at the top value of the stack to see what was left there by the execution.

  2. self-assigned this
    on Aug 18, 2020
  3. saulshanabrook commented on Nov 1, 2020

    @saulshanabrook
    ContributorAuthor

    This was merged in #78 but I had to revert that in #94 because it was causing segfaults.

    Of koalas:

    Ivy Default Cache set to: /root/.ivy2/cache
    The jars for the packages stored in: /root/.ivy2/jars
    :: loading settings :: url = jar:file:/usr/local/lib/python3.8/site-packages/pyspark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
    io.delta#delta-core_2.12 added as a dependency
    :: resolving dependencies :: org.apache.spark#spark-submit-parent-e4659a54-a70a-4cb1-9422-368fc8c06f48;1.0
    	confs: [default]
    	found io.delta#delta-core_2.12;0.7.0 in central
    	found org.antlr#antlr4;4.7 in central
    	found org.antlr#antlr4-runtime;4.7 in central
    	found org.antlr#antlr-runtime;3.5.2 in central
    	found org.antlr#ST4;4.0.8 in central
    	found org.abego.treelayout#org.abego.treelayout.core;1.0.3 in central
    	found org.glassfish#javax.json;1.0.4 in central
    	found com.ibm.icu#icu4j;58.2 in central
    downloading https://repo1.maven.org/maven2/io/delta/delta-core_2.12/0.7.0/delta-core_2.12-0.7.0.jar ...
    	[SUCCESSFUL ] io.delta#delta-core_2.12;0.7.0!delta-core_2.12.jar (91ms)
    downloading https://repo1.maven.org/maven2/org/antlr/antlr4/4.7/antlr4-4.7.jar ...
    	[SUCCESSFUL ] org.antlr#antlr4;4.7!antlr4.jar (58ms)
    downloading https://repo1.maven.org/maven2/org/antlr/antlr4-runtime/4.7/antlr4-runtime-4.7.jar ...
    	[SUCCESSFUL ] org.antlr#antlr4-runtime;4.7!antlr4-runtime.jar (11ms)
    downloading https://repo1.maven.org/maven2/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar ...
    	[SUCCESSFUL ] org.antlr#antlr-runtime;3.5.2!antlr-runtime.jar (7ms)
    downloading https://repo1.maven.org/maven2/org/antlr/ST4/4.0.8/ST4-4.0.8.jar ...
    	[SUCCESSFUL ] org.antlr#ST4;4.0.8!ST4.jar (9ms)
    downloading https://repo1.maven.org/maven2/org/abego/treelayout/org.abego.treelayout.core/1.0.3/org.abego.treelayout.core-1.0.3.jar ...
    	[SUCCESSFUL ] org.abego.treelayout#org.abego.treelayout.core;1.0.3!org.abego.treelayout.core.jar(bundle) (3ms)
    downloading https://repo1.maven.org/maven2/org/glassfish/javax.json/1.0.4/javax.json-1.0.4.jar ...
    	[SUCCESSFUL ] org.glassfish#javax.json;1.0.4!javax.json.jar(bundle) (6ms)
    downloading https://repo1.maven.org/maven2/com/ibm/icu/icu4j/58.2/icu4j-58.2.jar ...
    	[SUCCESSFUL ] com.ibm.icu#icu4j;58.2!icu4j.jar (269ms)
    :: resolution report :: resolve 2981ms :: artifacts dl 464ms
    	:: modules in use:
    	com.ibm.icu#icu4j;58.2 from central in [default]
    	io.delta#delta-core_2.12;0.7.0 from central in [default]
    	org.abego.treelayout#org.abego.treelayout.core;1.0.3 from central in [default]
    	org.antlr#ST4;4.0.8 from central in [default]
    	org.antlr#antlr-runtime;3.5.2 from central in [default]
    	org.antlr#antlr4;4.7 from central in [default]
    	org.antlr#antlr4-runtime;4.7 from central in [default]
    	org.glassfish#javax.json;1.0.4 from central in [default]
    	---------------------------------------------------------------------
    	|                  |            modules            ||   artifacts   |
    	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    	---------------------------------------------------------------------
    	|      default     |   8   |   8   |   8   |   0   ||   8   |   8   |
    	---------------------------------------------------------------------
    :: retrieving :: org.apache.spark#spark-submit-parent-e4659a54-a70a-4cb1-9422-368fc8c06f48
    	confs: [default]
    	8 artifacts copied, 0 already retrieved (15071kB/51ms)
    20/10/28 15:44:37 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
    Setting default log level to "WARN".
    To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
    
    [Stage 0:>                                                          (0 + 1) / 1]
    
                                                                                    
    20/10/28 15:44:50 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    
    [Stage 2:>                                                          (0 + 2) / 2]
    
                                                                                    
    20/10/28 15:44:54 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    
    [Stage 4:=============================>                             (1 + 1) / 2]
    
                                                                                    
    20/10/28 15:44:57 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    
    [Stage 10:=============================>                            (1 + 1) / 2]
    
                                                                                    
    20/10/28 15:45:03 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:07 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:11 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:12 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:40 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:41 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:45 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:46 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:47 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:48 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:49 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:50 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:51 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:51 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:52 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:54 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:55 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:55 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:56 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:57 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:58 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:59 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:45:59 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:46:00 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    ============================= test session starts ==============================
    platform linux -- Python 3.8.4, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
    rootdir: /usr/src/app
    plugins: cov-2.10.1, record_api-1.3.0
    collected 1201 items / 6 deselected / 1195 selected
    
    databricks/koalas/accessors.py ....
    20/10/28 15:46:01 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:46:02 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:46:09 WARN package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.
    20/10/28 15:46:12 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:46:13 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:46:13 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    databricks/koalas/base.py ................
    databricks/koalas/config.py ...
    databricks/koalas/datetimes.py ................
    databricks/koalas/extensions.py ....
    20/10/28 15:47:02 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:23 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:25 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:31 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:34 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    
    [Stage 438:>                                                        (0 + 2) / 2]
    
                                                                                    
    20/10/28 15:47:37 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:42 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:44 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:47 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:48 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:47:51 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:48:11 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
    20/10/28 15:48:11 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
    20/10/28 15:48:26 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:48:28 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:48:28 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:49:07 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:49:08 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    20/10/28 15:49:18 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
    Fatal Python error: Segmentation fault
    
    Thread 0x00007ff09248a700 (most recent call first):
      File "/usr/local/lib/python3.8/site-packages/pyspark/accumulators.py", line 239 in poll
      File "/usr/local/lib/python3.8/site-packages/pyspark/accumulators.py", line 268 in handle
      File "/usr/local/lib/python3.8/socketserver.py", line 720 in __init__
      File "/usr/local/lib/python3.8/socketserver.py", line 360 in finish_request
      File "/usr/local/lib/python3.8/socketserver.py", line 347 in process_request
      File "/usr/local/lib/python3.8/socketserver.py", line 316 in _handle_request_noblock
      File "/usr/local/lib/python3.8/socketserver.py", line 237 in serve_forever
      File "/usr/local/lib/python3.8/threading.py", line 870 in run
      File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
      File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap
    
    Current thread 0x00007ff0965a8740 (most recent call first):
      File "/usr/local/lib/python3.8/site-packages/record_api/get_stack.py", line 70 in __getitem__
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 349 in TOS
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 423 in log_called_method
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 414 in __call__
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 634 in __call__
      File "<doctest databricks.koalas.frame.DataFrame.isnull[0]>", line 1 in <module>
      File "/usr/local/lib/python3.8/doctest.py", line 1336 in __run
      File "/usr/local/lib/python3.8/doctest.py", line 1483 in run
      File "/usr/local/lib/python3.8/doctest.py", line 1844 in run
      File "/usr/local/lib/python3.8/site-packages/_pytest/doctest.py", line 287 in runtest
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 163 in pytest_runtest_call
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 256 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 310 in from_call
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 255 in call_runtest_hook
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 216 in call_and_report
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 127 in runtestprotocol
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 338 in pytest_runtestloop
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 313 in _main
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 257 in wrap_session
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 306 in pytest_cmdline_main
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 164 in main
      File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 187 in console_main
      File "/usr/local/lib/python3.8/site-packages/pytest/__main__.py", line 5 in <module>
      File "/usr/local/lib/python3.8/runpy.py", line 87 in _run_code
      File "/usr/local/lib/python3.8/runpy.py", line 194 in _run_module_as_main
    ./dev/pytest: line 56:    11 Segmentation fault      (core dumped) $PYTHON_EXECUTABLE -m pytest "$@"
    databricks/koalas/frame.py .............................................F..FF
    

    and xarray:

    ============================= test session starts ==============================
    platform linux -- Python 3.8.4, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
    rootdir: /usr/src/app, configfile: setup.cfg
    plugins: env-0.6.2, cov-2.10.1, custom-exit-code-0.3.0, hypothesis-5.38.1, record_api-1.3.0
    collected 11772 items / 2 skipped / 11770 selected
    
    xarray/tests/test_accessor_str.py ...................................... [  0%]
    .................                                                        [  0%]
    xarray/tests/test_backends.py .FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFF [  0%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFF [  1%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF...FFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFF [  2%]
    FFFFFFFFFFFFFFFFFFF.xFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFF [  2%]
    FFFFFFFFFFFFFFFFF.xFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFF [  3%]
    FFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFssFFFFFFFFFFFFFF [  3%]
    FFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFF [  4%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFF [  5%]
    FFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  5%]
    FFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  6%]
    FFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFF [  6%]
    FFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF..FFFFFFFFFF [  7%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  8%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF..FF..FFFFFFFFFFxFFFFFFFFFFFFFFFFFF [  8%]
    FFFFFFsssssssssssssssssssssssssssssssssssssssssssssssssssssssssFFFFFFFFF [  9%]
    FFFFFFFFsFxFsFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  9%]
    FFFFFFFFFFFFFFFFF                                                        [ 10%]
    xarray/tests/test_backends_api.py F                                      [ 10%]
    xarray/tests/test_backends_common.py F                                   [ 10%]
    xarray/tests/test_backends_file_manager.py FFFFFFFFFFFFFFFFFFFFFFFFFF... [ 10%]
    ..                                                                       [ 10%]
    xarray/tests/test_backends_locks.py .                                    [ 10%]
    xarray/tests/test_backends_lru_cache.py ........                         [ 10%]
    xarray/tests/test_cftime_offsets.py .................................... [ 10%]
    ........................................................................ [ 11%]
    ........................................................................ [ 12%]
    ........................................................................ [ 12%]
    ........................................................................ [ 13%]
    ........................................................................ [ 13%]
    ........................................................................ [ 14%]
    ........................................................................ [ 15%]
    ........................................................................ [ 15%]
    ........................................................................ [ 16%]
    ........................................................................ [ 16%]
    ........................................................................ [ 17%]
    ........................................................................ [ 18%]
    ........................................................................ [ 18%]
    ........................................................................ [ 19%]
    ........................................................................ [ 19%]
    ........................................................................ [ 20%]
    ........................................................................ [ 21%]
    ........................................................................ [ 21%]
    ........................................................................ [ 22%]
    ........................................................................ [ 23%]
    ........................................................................ [ 23%]
    ........................................................................ [ 24%]
    ........................................................................ [ 24%]
    ........................................................................ [ 25%]
    ........................................................................ [ 26%]
    ........................................................................ [ 26%]
    ........................................................................ [ 27%]
    ........................................................................ [ 27%]
    ..........                                                               [ 28%]
    xarray/tests/test_cftimeindex.py ....................................... [ 28%]
    ........................................................................ [ 28%]
    ........................................................................ [ 29%]
    ...........................................................FFFFFFFFFFFFF [ 30%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 30%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 31%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 32%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 32%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF............... [ 33%]
    ...............................................x........................ [ 33%]
    .................                                                        [ 33%]
    xarray/tests/test_cftimeindex_resample.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 34%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 34%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 35%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 36%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 36%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 37%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 37%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF                                  [ 38%]
    xarray/tests/test_coding.py FFFFFFFFFFFFF                                [ 38%]
    xarray/tests/test_coding_strings.py FFFFFFFFFFFFFFFFFFFFF                [ 38%]
    xarray/tests/test_coding_times.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 38%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.......................... [ 39%]
    .......................FFFFFFF.......................................... [ 40%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 40%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 41%]
    F                                                                        [ 41%]
    xarray/tests/test_combine.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.xFFF [ 41%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF                                     [ 41%]
    xarray/tests/test_computation.py FFFFFFFFFFFFFFFFFFFFF...............    [ 42%]
    xarray/tests/test_concat.py FFFFFFFF..FFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFF  [ 42%]
    xarray/tests/test_conventions.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 42%]
    FFFFFFFFFxFFFFFFFFFFFFFFFs...                                            [ 43%]
    xarray/tests/test_dataarray.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF..FFFFFF [ 43%]
    FFFF...FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF...FFFFFF [ 44%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFsFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF. [ 44%]
    .FFFF.FFFFFFFFFFFFFFFFFFFFFF....FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 45%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 46%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 46%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 47%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 47%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 48%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 49%]
    FFFFFFFFFFFFFsssssssssssssFFFFFFFFFFF                                    [ 49%]
    xarray/tests/test_dataset.py FFFFFFF.FF.F.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 49%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 50%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 50%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 51%]
    FFFFFFFFFFFFFFFFFFFFFF.....FFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 52%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 52%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 53%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 54%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 54%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 55%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 55%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 56%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF                     [ 56%]
    xarray/tests/test_distributed.py FFFFFFFFFFFFFFFFFsF                     [ 57%]
    xarray/tests/test_dtypes.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF      [ 57%]
    xarray/tests/test_duck_array_ops.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 57%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 58%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 58%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 59%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 60%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 60%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 61%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 62%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 62%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 63%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 63%]
    FFFFFFFFFFFFFFFFFFFFFFFFFFF                                              [ 64%]
    xarray/tests/test_extensions.py FFFF                                     [ 64%]
    xarray/tests/test_formatting.py FFFFFFFFFFFFFFFFF                        [ 64%]
    xarray/tests/test_formatting_html.py FFFFFFFFFFFF                        [ 64%]
    xarray/tests/test_groupby.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.F            [ 64%]
    xarray/tests/test_indexing.py FFFFFFFFFFFFFFFFFFF....................... [ 64%]
    Fatal Python error: Segmentation fault
    
    Thread 0x00007f7542a0b700 (most recent call first):
      File "/usr/local/lib/python3.8/threading.py", line 306 in wait
      File "/usr/local/lib/python3.8/queue.py", line 179 in get
      File "/usr/local/lib/python3.8/site-packages/distributed/threadpoolexecutor.py", line 51 in _worker
      File "/usr/local/lib/python3.8/threading.py", line 870 in run
      File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
      File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap
    
    Thread 0x00007f754320c700 (most recent call first):
      File "/usr/local/lib/python3.8/threading.py", line 306 in wait
      File "/usr/local/lib/python3.8/queue.py", line 179 in get
      File "/usr/local/lib/python3.8/site-packages/distributed/threadpoolexecutor.py", line 51 in _worker
      File "/usr/local/lib/python3.8/threading.py", line 870 in run
      File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
      File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap
    
    Thread 0x00007f758268a700 (most recent call first):
      File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 78 in _worker
      File "/usr/local/lib/python3.8/threading.py", line 870 in run
      File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
      File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap
    
    Current thread 0x00007f7587164740 (most recent call first):
      File "/usr/local/lib/python3.8/site-packages/record_api/get_stack.py", line 70 in __getitem__
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 349 in TOS
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 423 in log_called_method
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 414 in __call__
      File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 634 in __call__
      File "/usr/src/app/xarray/tests/test_indexing.py", line 730 in test_create_mask_error
      File "/usr/local/lib/python3.8/site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/python.py", line 1627 in runtest
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 163 in pytest_runtest_call
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 256 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 310 in from_call
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 255 in call_runtest_hook
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 216 in call_and_report
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 127 in runtestprotocol
      File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 338 in pytest_runtestloop
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 313 in _main
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 257 in wrap_session
      File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 306 in pytest_cmdline_main
      File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
      File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
      File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
      File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 164 in main
      File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 187 in console_main
      File "/usr/local/bin/pytest", line 8 in <module>
    ................................................................FFFFFFF
    

    and pandas and skimage.

    We should re-try that branch locally with one of those to track down the segfault and fix it.

  4. aktech commented on Nov 3, 2020

    @aktech
    Member

    Ah, I will take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions