V4.y

[LAShemilt]

This is a first pass refactor to get pyscicat working with v4 of Scicat. We would like to :

• Provide an easy install that can be versioned
• mark where functions are deprecated and breaking changes occur
• have a ci workflow for testing against the v4.y backend

[dylanmcreynolds]

Thanks for this! I've never migrated a python project to use setup.cfg. I have some changes and questions.

[LAShemilt]

Now finished work on the pyscicat version for interacting with the v4 scicat backend. Please see the following important changes to pyscicat:

1) Install and Setup
I have now split up the setup.py into a minimal setup.py script and a setup.cfg. This allows for easy configuration of unique setups which may require optional dependencies. To now install these please use for example:
pip install -e ".[option]" so to use the hd5f plugins install with pip install -e ".hdf5"

2) Deprecation of methods
I have used python's deprecation package to deprecate methods that no longer exist in the v4 backend. They will still work with the old backend, and can still be called with pyscicat as normal. However, you will get a deprecation warning when they are called.

3) Endpoint routes
I had really trouble with the getter methods which was coming from urllib's urljoin. If you use a / in an endpoint definition when you join that with a base url , urljoin will treat it as an absolute path and will append it after the port number. So http://localhost:3000/api/v3 joined with /Datasets/ was becoming http://localhost:3000/Datasets/. This problem goes away when using proper routing as to not have the port in the url name, which is why I guess it has not affected production systems so far. I have removed the / at the beginning so that the append will work properly in both the test and prod cases.

4) Integration testing in github actions
I have updated the github actions to include some minor integration testing for the backend-v.4 . It uses the docker image of the last stable release from https://github.com/SciCatProject/scicat-backend-next. This runs as a service and the tests run a few methods against it to ingest and retrieve data.

Any comments and review would be greatly appreciated.

[nitrosx]

@dylanmcreynolds why don't we align the version with the current SciCat one?

[nitrosx]

One more thing: given that this PR goes against the v4.x branch, should we go ahead and merge and than work on the v4.x to make it production ready?
This merge would help me greatly with the current work that I'm doing now to refactor all our services to interact correctly with BE v4.x
@LAShemilt , @dylanmcreynolds : thoughts?

[dylanmcreynolds]

Tests are failing because of the change in how requirements are listed.

[dylanmcreynolds]

Hmm, flake8 failures now.

[dylanmcreynolds]

pytest tests are failing in this PR because somewhere along the way in these many commits, the client.py has changed dramatically.

• test_client:test_scicat_ingest fails because the returns for upload_instrument, upload_proposal and upload_sample once returned a string, now they returns a dictionary (the ones you can see in add_mock_reqeusts. I don't know who uses these methods, but they return string in the main branch.
• It looks like there are similar major changes that affect in test_client2

[dylanmcreynolds]

This call to os.environ["BASE_URL"] now blows up tests for developers that use pytest but don't have the full integration suite configured (like me). pytest picks up this module because it's name starts with test.

I don't this module using either pytest or unittest. Perhaps we could change the module name from test_integration.py to integration.py to prevent is from getting in the way of pytest?

[LAShemilt]

Please see #40