Can codon be used on programs that use NumPy and Pandas?

[Beliavsky]

Will codon typically improve the performance of Python programs that heavily use NumPy or Pandas?

[arshajii]

Hi @Beliavsky, that's a great question -- right now Codon can call NumPy and Pandas through Python, but won't compile/optimize any of those calls, so the performance will be the same as in Python.

We are working on a Codon-native NumPy implementation that's fully compiled with added optimizations like fusion etc. The performance benefits of that will be application-specific: if you're just doing one huge matrix inverse, for example, then all the time is being spent in BLAS/LAPACK anyway so there won't be much difference, but for more complex operations--especially ones that can be fused or done more efficiently between library calls--we can potentially get much better performance. The other nice thing about that is it can be used with Codon's parallelism / GPU backend. After that, we do plan to look into Pandas, but no ETA yet -- will share more details as we make progress!

[arnoldjames98]

When I make a file that contains import numpy as np and then compile with codon build -release -exe file.py, I get the following error:

error: no module named 'numpy'

What do I need to do differently?

[chenyufei73]

same issue encountered, no module named numpy/pandas/json/dataclasses

[GaudiestTooth17]

I'm trying to use the codon.jit decorator and I'm running into this issue

[arshajii]

You can import plain Python libraries with from python import, e.g. from python import numpy as np (https://docs.exaloop.io/codon/interoperability/python) -- in JIT mode you can pass the module through pyvars: https://docs.exaloop.io/codon/interoperability/decorator#passing-globals-to-codon. Let me know if you run into any issues with this!

[alfie-nsugh]

Are there any updates on how the Codon-native NumPy implementation is coming along? I am very curious about it.