I am using python-inspector as part of the oss review toolkit and I noticed that the analysis of some python projects is rather slow though it has a poetry.lock file and thus the resolution should be straight-forward.
Some debugging revealed that most of the time is spent in downloading stuff from pypi. I could greatly speed this up using async io requests using the aiohttp module. Additionally, I would suggest to replace the existing cache with aiohttp-client-cache.
I will prepare a PR to showcase the use of async io and how you can gain considerable speedups using that approach by parallelizing the downloads from pypi instead of doing them sequentially which take a while if you have a larger number of dependencies.
I am using python-inspector as part of the oss review toolkit and I noticed that the analysis of some python projects is rather slow though it has a poetry.lock file and thus the resolution should be straight-forward.
Some debugging revealed that most of the time is spent in downloading stuff from pypi. I could greatly speed this up using async io requests using the aiohttp module. Additionally, I would suggest to replace the existing cache with aiohttp-client-cache.
I will prepare a PR to showcase the use of async io and how you can gain considerable speedups using that approach by parallelizing the downloads from pypi instead of doing them sequentially which take a while if you have a larger number of dependencies.