Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sub]fix: adapt to pydantic 2.x.x #758

Merged

Conversation

guilhem-barthes
Copy link
Contributor

Description

How has this been tested?

Checklist

  • changelog was updated with notable changes
  • documentation was updated

@guilhem-barthes guilhem-barthes requested a review from a team as a code owner October 6, 2023 19:34
@guilhem-barthes guilhem-barthes force-pushed the fix/adapt-poc-builder-to-pydantic-v2 branch from 1532f8d to 6459f1b Compare October 6, 2023 19:39
@guilhem-barthes guilhem-barthes changed the title docs: changelog [sub]fix: adapt to pydantic 2.x.x Oct 6, 2023
@guilhem-barthes
Copy link
Contributor Author

/e2e --benchmarks mnist,camelyon --mode=standalone

@Owlfred
Copy link

Owlfred commented Oct 6, 2023

End to end tests: ❌ FAILURE

Jobs status:

  • Camelyon / standalone-camelyon:
  • Dispatch Jobs: ✔️
  • Distributed: ⏭️
  • Documentation: ⏭️
  • MNIST / standalone-mnist:
  • Standalone / standalone-sdk:
  • SubstraFL: ⏭️

“I’m sorry, Dave. I’m afraid I can’t do that.” ― Hal, 2001: A Space Odyssey

@thbcmlowk
Copy link
Contributor

/e2e --orchestrator-mode=standalone "--refs=substra-frontend=feat/modify-failure-report,orchestrator=poc-decoupled-builder"

@Owlfred
Copy link

Owlfred commented Oct 9, 2023

End to end tests: ❌ FAILURE

Jobs status:

Oh noes.

Signed-off-by: Guilhem Barthés <[email protected]>
@thbcmlowk thbcmlowk force-pushed the fix/adapt-poc-builder-to-pydantic-v2 branch from 6459f1b to 5a160fd Compare October 11, 2023 17:08
Copy link
Contributor

@thbcmlowk thbcmlowk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@thbcmlowk thbcmlowk merged commit 4d57cff into poc-decoupled-builder Oct 11, 2023
4 checks passed
@thbcmlowk thbcmlowk deleted the fix/adapt-poc-builder-to-pydantic-v2 branch October 11, 2023 17:13
SdgJlbl pushed a commit that referenced this pull request Oct 20, 2023
thbcmlowk pushed a commit that referenced this pull request Oct 24, 2023
SdgJlbl added a commit that referenced this pull request Oct 25, 2023
* feat: decouple image builder from worker

Signed-off-by: SdgJlbl <[email protected]>

* fix: update skaffold config

Signed-off-by: Guilhem Barthes <[email protected]>

* feat: add `ServiceAccount` and modify role

Signed-off-by: Guilhem Barthes <[email protected]>

* fix: improve `wait_for_image_built`

Signed-off-by: Guilhem Barthes <[email protected]>

* feat: build image in new pod

Signed-off-by: Guilhem Barthes <[email protected]>

* chore: rename `deployment-builder.yaml` to `stateful-builder.yaml`

Signed-off-by: Guilhem Barthes <[email protected]>

* chore: rename `stateful-builder.yaml` to `statefulset-builder.yaml`

Signed-off-by: Guilhem Barthes <[email protected]>

* chore: centralize params

Signed-off-by: Guilhem Barthes <[email protected]>

* feat: create `BuildTask`

Signed-off-by: Guilhem Barthes <[email protected]>

* feat: move some values to `builder` module

Signed-off-by: Guilhem Barthes <[email protected]>

* feat: move more code to `builder`

Signed-off-by: Guilhem Barthes <[email protected]>

* fix: remove TaskProfiling as Celery task + save Entrypoint in DB

Signed-off-by: SdgJlbl <[email protected]>

* fix: extract entrypoint from registry

Signed-off-by: SdgJlbl <[email protected]>

* fix: make doc for helm chart

Signed-off-by: SdgJlbl <[email protected]>

* feat: build function at registration (#707)

<!-- Please reference issue if any. -->

<!-- Please include a summary of your changes. -->

<!-- Please describe the tests that you ran to verify your changes.  -->

- [ ] [changelog](../CHANGELOG.md) was updated with notable changes
- [ ] documentation was updated

---------

Signed-off-by: SdgJlbl <[email protected]>
Signed-off-by: Guilhem Barthes <[email protected]>
Co-authored-by: SdgJlbl <[email protected]>

* feat: share images between backends (#708)



Signed-off-by: SdgJlbl <[email protected]>

* chore: update helm worklfow

Signed-off-by: ThibaultFy <[email protected]>

* chore: add .DS_Store to gitignore

Signed-off-by: ThibaultFy <[email protected]>

* chore: rm DS_Store

Signed-off-by: ThibaultFy <[email protected]>

* chore: rm .DS_Store

Signed-off-by: ThibaultFy <[email protected]>

* [sub]fix: add missing migration poc (#728)

## Description

Add a migration missing in the poc. 
This migration alters two things:

-  modify `ComputeTaskFailureReport.logs` 
-  modify `FunctionImage.file`

This migration has been generated automatically with `make migrations`

## How has this been tested?

<!-- Please describe the tests that you ran to verify your changes.  -->

## Checklist

- [ ] [changelog](../CHANGELOG.md) was updated with notable changes
- [ ] documentation was updated

Signed-off-by: Guilhem Barthes <[email protected]>

* [sub]feat: add function events (#714)

- Substra/orchestrator#263

Add function events, used now we decoupled the building of the function
with the execution of the compute task. For that it add a status field
on the Function. It also includes another PR (merged here), to have
functions build logs working again.

In a future PR, we will change the compute task execution to avoid
having to wait_for_function_built in compute_task()

Fixes FL-1160

As this is going to be merged on a branch that is going to be merged to
a POC branch, we use MNIST as a baseline of a working model. We will
deal with failing tests on the POC before merging on main.

- [x] [changelog](../CHANGELOG.md) was updated with notable changes
- [ ] documentation was updated

---------

Signed-off-by: SdgJlbl <[email protected]>
Signed-off-by: Guilhem Barthes <[email protected]>
Signed-off-by: Guilhem Barthés <[email protected]>
Co-authored-by: SdgJlbl <[email protected]>

* [sub]fix(app/orchestrator/resources): FunctionStatus.FUNCTION_STATUS_CREATED -> FunctionStatus.FUNCTION_STATUS_WAITING (#742)

# Issue

Backend FunctionStatus are not aligned with [orchestrator
definitions](https://github.com/Substra/orchestrator/blob/poc-decoupled-builder/lib/asset/function.proto#L29-L36).
In particular, `FunctionStatus.FUNCTION_STATUS_CREATED` leading to the
following error:

```txt
ValueError: 'FUNCTION_STATUS_WAITING' is not a valid FunctionStatus
```

## Description

FunctionStatus.FUNCTION_STATUS_CREATED ->
FunctionStatus.FUNCTION_STATUS_WAITING

## How has this been tested?

Running Camelyon benchmark on
[poc-builder-flpc](https://substra.org-1.poc-builder-flpc.cg.owkin.tech/compute_plans/a420306f-5719-412b-ab9c-688b7bed9c70/tasks?page=1&ordering=-rank)
environment.

## Checklist

- [ ] [changelog](../CHANGELOG.md) was updated with notable changes
- [ ] documentation was updated

---------

Signed-off-by: Thibault Camalon <[email protected]>

* fix: builder using builder SA (#754)

* fix: builder using builder SA

Signed-off-by: Guilhem Barthés <[email protected]>

* docs: changelog

Signed-off-by: Guilhem Barthés <[email protected]>

---------

Signed-off-by: Guilhem Barthés <[email protected]>

* fix: rebase changelog

Signed-off-by: Guilhem Barthés <[email protected]>

* fix: adapt to pydantic 2.x.x (#758)

Signed-off-by: Guilhem Barthés <[email protected]>

* [sub]fix(backend/image_transfert/encoder): update pydantic method (#763)

* fix(backend/image_transfert/encoder): update pydantic method

Signed-off-by: Thibault Camalon <[email protected]>

* fix(backend/image_transfer/decoder): parse_raw -> model_validate_json

Signed-off-by: Thibault Camalon <[email protected]>

---------

Signed-off-by: Thibault Camalon <[email protected]>

* [sub]chore: upgrade chart (#765)

* chore(charts): bump chart version

Signed-off-by: Thibault Camalon <[email protected]>

* chore(charts/substra-backend/CHANGELOG): bring back unreleased section

Signed-off-by: Thibault Camalon <[email protected]>

---------

Signed-off-by: Thibault Camalon <[email protected]>

* fix: post-rebase

Signed-off-by: SdgJlbl <[email protected]>

* chore: rationalize migrations

Signed-off-by: SdgJlbl <[email protected]>

* [sub]chore(builder): waitPostgresqlInitContainer (#764)

* fix: builder using builder SA (#754)

* fix: builder using builder SA

Signed-off-by: Guilhem Barthés <[email protected]>

* docs: changelog

Signed-off-by: Guilhem Barthés <[email protected]>

---------

Signed-off-by: Guilhem Barthés <[email protected]>

* chore(charts/substra-backend/templates/statefulset-builder): add init-container waitPostgresqlInitContainer

Signed-off-by: Thibault Camalon <[email protected]>

---------

Signed-off-by: Guilhem Barthés <[email protected]>
Signed-off-by: Thibault Camalon <[email protected]>
Co-authored-by: Guilhem Barthés <[email protected]>

---------

Signed-off-by: SdgJlbl <[email protected]>
Signed-off-by: Guilhem Barthes <[email protected]>
Signed-off-by: ThibaultFy <[email protected]>
Signed-off-by: Guilhem Barthés <[email protected]>
Signed-off-by: Thibault Camalon <[email protected]>
Co-authored-by: SdgJlbl <[email protected]>
Co-authored-by: ThibaultFy <[email protected]>
Co-authored-by: Thibault Camalon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants