Python bindings

[jakob-keller]

Hello uom community!

My organization is running a number of Python services. At their core, they perform simple arithmetic calculations in the intersection of physics and accounting, such as adding and subtracting:

• power measurements in kW and MW
• amounts of money in € or £
• prices such as € / MW or £ / MWh

In addition, values are frequently string formatted and parsed. Nothing fancy, really.

So far, we have used custom pure-Python model classes for this purpose, since none of the Python libraries known to us satisfy our requirements in terms of performance and/or functionality. For example, astropy.units is one order of magnitude slower than our purpose built model classes.

Inspired by the spectacular success of polars, I started investigating ways to further speed up our services with the help of rust. uom caught my attention and I am now wondering whether it would be feasible to create PyO3 based Python bindings to make the power of uom available to the Python ecosystem.

• What are the thoughts of the uom core team and community?
• Are there any other Python users that would be interested in uom bindings for Python?

[adamreichold]

Just being part of the wider community, I have to admit to being somewhat sceptical: Looking at the functionality (meaning ignoring the diligently compiled list of quantities, units and associated constants), uom is primarily about new types and compile-time computation of exponents both of which are difficult to transfer to Python due to the absence of strong visibility-based encapsulation and - well - compile time.

Thinking further, if you do manage to create wrappers for all of uom's types and then start implementing operations, there is very little actual computation happening when e.g. adding two quantities in uom and I am not sure that this will suffice to pay off PyO3's call overhead (which is continued to be optimized but exists even compared to a pure Python function call nevertheless). Polars does have the distinct advantage of wrapping complex operations on large amounts of data, whole pipelines even when the lazy API is used. I am not sure a units-of-measurement use case can match this. (A fully lazy approach building up expression trees and evaluating them in a single pass sounds like too much ceremony for the use case you describe.)

So while I am sceptical, I do not want to discourage from trying out. The above is pure speculation and the proof is in the pudding as they say. I'd suggest to start with a minimal example so that you can get a feel for the necessary wrapping and actually measure its runtime costs. Such a prototype would be a much improved basis for discussion IMHO.

[jakob-keller]

Thanks for your input. I had been thinking about creating a minimal example myself and will attempt to test how it performs.

[adamreichold]

As an implementation advice, make sure to start with PyO3's 0.21 beta release if you are starting a new project. The final release is expected for next week and the new "bound" API greatly reduces the above mentioned function/method call overhead, so there is little reason to invest work into using the older "GIL ref" API provided by PyO3 0.20.x.

[jakob-keller]

OK, bear with me - these are my first steps in Rust after all :-)

I managed to get a primitive Power model to work: it is initialized in kilowatt and provides __add__() and __str__(), among others.

At first, I thought this was not going anywhere - performance was rather slow. Then I discovered the --release argument to maturin develop and performance is roughly at par if not slightly better than with my custom Python classes.

I think I might actually play around with this approach some more. The real value would probably lie in migrating some of the higher lever application code.

extern crate uom;

use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;

use uom::si::i64::Power;
use uom::si::power::{kilowatt};

#[pyclass(name = "Power")]
pub struct PyPower(Power);

#[pymethods]
impl PyPower {
    #[pyo3(signature = (in_kw = 0))]
    #[new]
    fn __new__(in_kw: isize) -> PyResult<Self> {
        match in_kw % 100 {
            0 => Ok(PyPower(Power::new::<kilowatt>(in_kw))),
            _ => {
                let message = format!("{} is not a multiple of 100", in_kw);
                Err(PyValueError::new_err(message))
            }
        }
    }

    #[getter]
    fn in_kw(&self) -> i64 {
        self.0.value / 1_000
    }

    fn __repr__(&self) -> String {
        format!("Power(in_kw={})", self.0.value / 1_000)
    }

    fn __str__(&self) -> String {
        format!("{} MW", self.0.value as f64 / 1_000_000.0)
    }

    fn __add__(&self, other: &Self) -> Self {
        Self(self.0 + other.0)
    }
}

#[pymodule]
fn quantity(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_class::<PyPower>()?;
    Ok(())
}

[iliekturtles]

@adamreichold had a really good summary. At it's core uom provides compile time safety that optimizes down to raw math operations on the underlying types (f32, f64, ...). At this point uom does provide a fairly comprehensive list of quantities and units which may be valuable even if type safety doesn't end up being feasible.

I haven't touched Python in a long long time so I won't be as much help on that side. Looking at the code you've provided so far there are a lot of untyped literals and .value references which is an escape hatch from uom's safety (most made public right now because of #251). If you do find a viable solution to your issue I'll be happy to do some review, although it likely wont be very prompt right now.