A generic API and plug-in entry-point for iotools

[mikofski]

To facilitate development and integration of external commercial weather into iotools and allow them to be developed & maintained externally, should pvlib consider the following:

1. a generic API similar to what we already use for all iotools, e.g.: data, meta = read_*() and get_*() (easier)
2. a plug-in entry-point for external tools (harder)

generic API

A generic API for iotools already exists. Publishing this in the documentation and publicizing it for external developers would help disseminate this architecture for wider use. @AdamRJensen can explain this format.

plug-in entry-point for external tools

This doesn't exist, and TBH, I don't know how to do it, but I have seen it in other apps. The basic idea is that there is something in project.toml or setup.py that allows packages on PyPI to be installed and used in collaboration with pvlib so that if for example SolarAnywhere or Solcast wanted to add an iotools plugin that followed the api, they could put it on PyPI themselves, maintain it themselves, etc., and when installed on a system with pvlib, it would become available, but we wouldn't have to maintain or update it.

A number of discussions and posts touch on this subject:

[kandersolar]

I'm not sure what the plug-in idea is envisioning. Are you thinking of some future integration between iotools and another part of pvlib, for example mc.run_model_from_api(provider='NSRDB')?

[mikofski]

No I’m thinking of something like https://github.com/pelican-plugins/render-math which is a plug-in for Pelican. It is maintained and released separately from pelican. To use it you have to first have pelican, then you just run pip install pelican-render-math and it’s instantly accessible from within pelican as if it were part of pelican as a sub-package

So analogously pvlib.iotools.solaranywhere could be a plug-in maintained and released separately from pvlib. Ditto all the satellite providers that we don’t want to be responsible for but users would like to have integrated

there’s more info about how the Pelican plugins work here: https://docs.getpelican.com/en/latest/plugins.html

[kandersolar]

Ok, thanks for explaining. I have the sense that plugins are most useful when dealing with a large code pipeline like pelican does, so I could see plugins being useful for iotools if iotools were integrated with some other part of pvlib where they were somehow used automatically (like inside a ModelChain or something). However, right now such an integration doesn't exist and the only use of iotools functions is directly by the user. If the only way an iotools function gets called is in the user's code, I think there is much less upside to having the function come from pvlib.iotools instead of cprlib (or whatever), and might just create some confusion like when the function doesn't appear in our RTD site despite ostensibly coming from pvlib.

What benefits are there to having externally distributed functions accessible via pvlib.iotools instead of their own package?

[mikofski]

You make good points