Update Tensorflow, gRPC, Protobuf dependencies #868

psfoley · 2023-08-01T22:03:50Z

This PR updates the Tensorflow workspace requirements to 2.13, which allows the use of significantly updated versions of the grpcio / protobuf packages

Fixes mlcommons/GaNDLF#702

Update
In addressing the broken CI tests, this PR has also brought in the following:

New KerasSerializer to address missing build function in Keras legacy optimizer
Refactoring openfl/utils and openfl/component/ca to minimize unneeded imports before fx workspace create command. This is needed because on Windows when Python modules are loaded into memory they cannot be overwritten on disk. Because we do a pip installation from code, any version differences between loaded and new packages will throw an error. This has the extra benefit of fx CLI commands now responding near instantly.
Fixing dockerization CI test by pinning pyzmq (thanks to @mansishr)
Python 3.11 support

Signed-off-by: Patrick Foley <[email protected]>

setup.py

Signed-off-by: Mansi Sharma <[email protected]>

… loaded into memory unnecessarily (breaking windows build)

…n out of memory Signed-off-by: Patrick Foley <[email protected]>

* Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]>

Squashed commit of the following: commit 5e575c6 Author: Parth Mandaliya <[email protected]> Date: Fri Sep 29 17:31:52 2023 +0530 Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions Signed-off-by: Parth Mandaliya <[email protected]> commit e2e811f Author: Parth Mandaliya <[email protected]> Date: Thu Sep 28 20:57:48 2023 +0530 Fixed flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> commit c947e11 Merge: 85e7578 11db127 Author: ParthM-GitHub <[email protected]> Date: Fri Sep 29 00:17:58 2023 +0530 Merge branch 'develop' into workspace-creator commit 11db127 Author: Amit Portnoy <[email protected]> Date: Thu Sep 28 21:34:13 2023 +0300 Update EdenPipeline in the documentation (securefederatedai#877) Signed-off-by: Amit Portnoy <[email protected]> commit 85e7578 Author: Keerti Prakash Talwar <[email protected]> Date: Sat Jul 15 01:39:53 2023 +0530 Testflow for verifying stdout redirection to Metaflow datastore (securefederatedai#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> commit 9a51a5d Author: Kevin Ta <[email protected]> Date: Fri Jul 14 12:53:08 2023 -0400 fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (securefederatedai#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 8c20e73 Author: Francis Storr <[email protected]> Date: Fri Jul 14 09:50:49 2023 -0700 Accessibility updates (securefederatedai#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes securefederatedai#848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 6fb141b Author: Parth Mandaliya <[email protected]> Date: Thu Sep 28 20:00:46 2023 +0530 Fixed flake8 issues in test/experimental directory Signed-off-by: Parth Mandaliya <[email protected]> commit cd49915 Author: Parth Mandaliya <[email protected]> Date: Thu Sep 28 18:24:02 2023 +0530 Fixed flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> commit 99c1d5d Author: Patrick Foley <[email protected]> Date: Thu Sep 21 09:37:50 2023 -0700 Migrate to Ubuntu 22.04 LTS release (supported through 2027) (securefederatedai#875) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit a0368ad Author: Patrick Foley <[email protected]> Date: Thu Sep 14 16:54:11 2023 -0700 Resolve Coverity Issues (securefederatedai#874) * Fix coverity issues * Resolve remaining coverity issues Signed-off-by: Parth Mandaliya <[email protected]> commit 34b704b Author: Mansi Sharma <[email protected]> Date: Tue Aug 29 15:35:45 2023 -0700 Add FL plan description to documentation (securefederatedai#872) * Add plan description to documentation Signed-off-by: Mansi Sharma <[email protected]> * fix indentation Signed-off-by: Mansi Sharma <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick Foley <[email protected]> --------- Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 1c7ccc6 Author: Patrick Foley <[email protected]> Date: Fri Aug 25 16:33:55 2023 -0700 Update Tensorflow, gRPC, Protobuf dependencies (securefederatedai#868) * Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 1b06e60 Author: Parth Mandaliya <[email protected]> Date: Thu Sep 28 18:00:52 2023 +0530 Added libraries required by WorkspaceBuilder Signed-off-by: Parth Mandaliya <[email protected]> commit c48c464 Author: Parth Mandaliya <[email protected]> Date: Thu Sep 28 17:06:13 2023 +0530 Added documentation and inline comments for workspace_builder Signed-off-by: Parth Mandaliya <[email protected]> commit 5ae301d Author: Parth Mandaliya <[email protected]> Date: Wed Sep 27 15:20:22 2023 +0530 Removed unnecessary files from template_workspace directory Moved print_tree to after generating python script Signed-off-by: Parth Mandaliya <[email protected]> commit 8300e84 Author: Parth Mandaliya <[email protected]> Date: Tue Sep 26 18:57:49 2023 +0530 Modified workspace_builder/build.py file to resolve a typo Signed-off-by: Parth Mandaliya <[email protected]> commit 83a6567 Author: Parth Mandaliya <[email protected]> Date: Tue Sep 26 18:53:31 2023 +0530 Modified get_collaborator_args function Signed-off-by: Parth Mandaliya <[email protected]> commit 388cd0e Author: Parth Mandaliya <[email protected]> Date: Tue Sep 26 18:38:40 2023 +0530 * Created get_collaborator_kwargs function in localruntime class which takes collaborator name as argument and returns name of private attributes function and arguments required to call it. * Changed workspace builder code used get_collaborator_kwargs function to build data.yaml file. Signed-off-by: Parth Mandaliya <[email protected]> commit 38d1015 Author: Parth Mandaliya <[email protected]> Date: Mon Sep 25 11:29:24 2023 +0530 Removing round number constant argument from 301 MNIST WATERMARKING Signed-off-by: Parth Mandaliya <[email protected]> commit 3c4fd3a Author: Parth Mandaliya <[email protected]> Date: Fri Sep 22 17:43:01 2023 +0530 Finished exporting data.yaml, plan.yaml. Workspace creator validated for 101 MNIST and 301 Watermarking with MNIST Signed-off-by: Parth Mandaliya <[email protected]> commit dba24af Author: Parth Mandaliya <[email protected]> Date: Sat Sep 16 07:03:33 2023 +0530 workspace_creator renamed to workspace_builder finding values without help of any tags Signed-off-by: Parth Mandaliya <[email protected]> commit 20068be Author: Parth Mandaliya <[email protected]> Date: Wed Sep 13 10:26:09 2023 +0530 Removed automatically generated Zone.Identifier file Added Approach2 notebook Modified creator.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 1f3750a Author: Parth Mandaliya <[email protected]> Date: Tue Sep 12 06:09:38 2023 +0530 Modifying arguments help Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 2c7720c Author: Parth Mandaliya <[email protected]> Date: Tue Sep 12 06:05:52 2023 +0530 Find aggregator, and collaborator arguments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 1b4ebfc Author: Parth Mandaliya <[email protected]> Date: Tue Sep 12 05:59:02 2023 +0530 Write flow.py, collaborator_private_attrs.py and aggregator_private_attrs.py automatically Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 0a52b4a Author: Parth Mandaliya <[email protected]> Date: Tue Sep 12 04:02:35 2023 +0530 Adding functionality to convert jupyter notebook to workspace (in progress) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit d2ad1a9 Author: ParthM-GitHub <[email protected]> Date: Wed Sep 6 18:50:41 2023 +0530 Update flow.py Signed-off-by: Parth Mandaliya <[email protected]> commit 1c53e30 Author: ParthM-GitHub <[email protected]> Date: Wed Sep 6 18:47:45 2023 +0530 Update flow.py Signed-off-by: Parth Mandaliya <[email protected]> commit 07f283f Author: ParthM-GitHub <[email protected]> Date: Wed Sep 6 13:42:26 2023 +0530 Update data.yaml Correcting a mistake train dataset assigned to test and vice versa. Signed-off-by: Parth Mandaliya <[email protected]> commit 265bf82 Author: Parth Mandaliya <[email protected]> Date: Thu Aug 31 19:31:25 2023 +0530 This commit brings following changes: 1. Added 501_pytorch_tinyimagenet_transfer_learning workspace for experimental workflow interface 2. Removed `feature_shape` from openfl/experimental/interface/cli/plan.py 3. Resolved flake8 errors in openfl-workspace/experimental directory Signed-off-by: Parth Mandaliya <[email protected]> commit 856fadb Author: Parth Mandaliya <[email protected]> Date: Wed Aug 30 20:04:56 2023 +0530 This commit brings following updates: 1. Remove unnecessary spaces from Workflow_Interface_102_Aggregator_Validation.ipynb 2. Upgraded protobuf to 3.20.3 in setup.py as per tensorboard requirements 3. Modified 101-torch-cnn-mnist workspace for n-level arguments 4. Added a federated runtime workspace for 102-aggregator-validation Signed-off-by: Parth Mandaliya <[email protected]> commit 4a68b40 Author: Parth Mandaliya <[email protected]> Date: Tue Aug 29 15:56:58 2023 +0530 Updated logs to provide exact information to user. Signed-off-by: Parth Mandaliya <[email protected]> commit 6a4ce82 Author: Parth Mandaliya <[email protected]> Date: Mon Aug 28 19:45:06 2023 +0530 Resolving last few TODO items. Signed-off-by: Parth Mandaliya <[email protected]> commit fb4effa Author: ParthM-GitHub <[email protected]> Date: Mon Aug 28 15:25:35 2023 +0530 Update workflow_interface.rst Added section on how to save best and last model from Metaflow CLI Signed-off-by: Parth Mandaliya <[email protected]> commit 52f0b52 Author: Parth Mandaliya <[email protected]> Date: Fri Aug 25 18:44:30 2023 +0530 Adding N-level arguments functionality Signed-off-by: Parth Mandaliya <[email protected]> commit 880b7a3 Author: Parth Mandaliya <[email protected]> Date: Thu Aug 24 17:56:22 2023 +0530 Moving the logs after data.yaml warning, and removing unnecessary blank lines. Signed-off-by: Parth Mandaliya <[email protected]> commit de2e333 Author: Parth Mandaliya <[email protected]> Date: Wed Aug 23 17:37:40 2023 +0530 Resolved flake8 issues, Added warning statements when data.yaml is not found or aggregator / collaborator private attributes are not found in data.yaml, Tested by running all tutorials and tests Signed-off-by: Parth Mandaliya <[email protected]> commit a2ceb05 Author: Parth Mandaliya <[email protected]> Date: Mon Aug 21 14:42:10 2023 +0530 Fixed flake8 issues and typos Signed-off-by: Parth Mandaliya <[email protected]> commit 36810fe Author: Parth Mandaliya <[email protected]> Date: Mon Aug 21 13:36:11 2023 +0530 With this commit path of template can be explicitely given, user does not have to choose from available example templates. Only applicable to experimental openfl. Signed-off-by: Parth Mandaliya <[email protected]> commit baf45f6 Author: ParthM-GitHub <[email protected]> Date: Wed Aug 16 19:01:39 2023 +0530 Updating import statement In 104 keras workspace utils.py updating import SummaryWriter statement from torch to tensorflow Signed-off-by: Parth Mandaliya <[email protected]> commit 0aa070f Author: ParthM-GitHub <[email protected]> Date: Fri Aug 11 20:03:17 2023 +0530 Update resources.py Updated get_number_of_gpus function Signed-off-by: Parth Mandaliya <[email protected]> commit 522a28c Author: ParthM-GitHub <[email protected]> Date: Fri Aug 11 18:37:39 2023 +0530 Update Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb Removing GPU parameter from collaborator decorator. Signed-off-by: Parth Mandaliya <[email protected]> commit 2f4c26e Author: Parth Mandaliya <[email protected]> Date: Fri Aug 11 14:57:50 2023 +0530 Files added required to test experimental aggregator based workflow by GitHUB actions Signed-off-by: Parth Mandaliya <[email protected]> commit 4545663 Author: ParthM-GitHub <[email protected]> Date: Fri Aug 11 11:24:25 2023 +0530 Update Workflow_Interface_VFL_Two_Party Added actor related changed to Workflow_Interface_VFL_Two_Party Signed-off-by: Parth Mandaliya <[email protected]> commit 5a89b19 Author: ParthM-GitHub <[email protected]> Date: Thu Aug 10 16:01:28 2023 +0530 Update requirements_privacy_meter.txt torch 1.13.1 requires torchvision 0.14.1 not 0.14.0 Signed-off-by: Parth Mandaliya <[email protected]> commit e5774b1 Author: Parth Mandaliya <[email protected]> Date: Thu Aug 10 13:37:24 2023 +0530 Added color escape sequences to datastore cli localruntime and workspace testcases Signed-off-by: Parth Mandaliya <[email protected]> commit 02cc8bd Author: ParthM-GitHub <[email protected]> Date: Wed Aug 9 16:41:50 2023 +0530 Update stream_redirect.py Removing unnecessary imports Signed-off-by: Parth Mandaliya <[email protected]> commit 5d9dc60 Author: Parth Mandaliya <[email protected]> Date: Tue Aug 8 16:32:09 2023 +0530 Removing unnecessary comments Signed-off-by: Parth Mandaliya <[email protected]> commit 6722bf7 Author: Parth Mandaliya <[email protected]> Date: Tue Aug 8 16:18:57 2023 +0530 Removing unnecessary deepcopies from aggregator.py file Signed-off-by: Parth Mandaliya <[email protected]> commit 9cc04a4 Author: Parth Mandaliya <[email protected]> Date: Tue Aug 8 16:16:53 2023 +0530 While taking aggregator step remove private attributes before taking checkpoint and set it back to ctx after checkpoint Signed-off-by: Parth Mandaliya <[email protected]> commit 8b64790 Author: porteratzo <[email protected]> Date: Fri Aug 4 16:48:04 2023 -0600 pinned tensorboardX (securefederatedai#870) Signed-off-by: Parth Mandaliya <[email protected]> commit 81b9ef0 Author: Parth Mandaliya <[email protected]> Date: Thu Aug 3 17:54:18 2023 +0530 Added dependecies to each tutorial notebooks Signed-off-by: Parth Mandaliya <[email protected]> commit bfa3d39 Author: Parth Mandaliya <[email protected]> Date: Thu Aug 3 14:37:30 2023 +0530 Incoporated review comments Signed-off-by: Parth Mandaliya <[email protected]> commit 07218ce Author: Parth Mandaliya <[email protected]> Date: Thu Aug 3 13:20:38 2023 +0530 Avoided deep copying the context by removing private attributes from context before checkpoint Signed-off-by: Parth Mandaliya <[email protected]> commit 8a40e91 Author: ParthM-GitHub <[email protected]> Date: Thu Aug 3 10:55:39 2023 +0530 Update collaborator.py Removing unnecessary logs Signed-off-by: Parth Mandaliya <[email protected]> commit d1bd2ad Author: ParthM-GitHub <[email protected]> Date: Thu Aug 3 10:52:05 2023 +0530 Removing unnecessary logs Removing unnecessary logs Signed-off-by: Parth Mandaliya <[email protected]> commit a04b8e8 Author: Parth Mandaliya <[email protected]> Date: Wed Aug 2 18:06:49 2023 +0530 This comming brings following changes: 1. Experimental Aggregator workflow testcases moved to test/github/experimental/workspace directory 2. Added dependecy related changes Signed-off-by: Parth Mandaliya <[email protected]> commit 54be758 Author: Parth Mandaliya <[email protected]> Date: Wed Aug 2 10:53:25 2023 +0530 Modified 104 Keras MNIST tutorial Signed-off-by: Parth Mandaliya <[email protected]> commit a7c99da Author: Parth Mandaliya <[email protected]> Date: Tue Aug 1 14:49:03 2023 +0530 This commit brings following changes: 1. Renamed Workflow_Interface_101_Keras_MNIST_with_GPU.ipynb to Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb 2. Updated documentation in Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb 3. protobuf bumped from 3.19.6 to 3.20.0 in setup.py 4. Incorporating review comments (WIP) Signed-off-by: Parth Mandaliya <[email protected]> commit edd2f55 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Jul 27 13:55:52 2023 -0700 build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (securefederatedai#774) * build(deps): bump tensorflow in /openfl-workspace/tf_2dunet Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.txt to retrigger CI * Update requirements.txt --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 32c7f34 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Jul 25 14:48:35 2023 -0700 build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (securefederatedai#772) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update sd_requirements.txt * revert to legacy SGD and install tensorflow==2.11 for workflow Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 292cb2e Author: Parth Mandaliya <[email protected]> Date: Sat Jul 29 12:03:14 2023 +0530 This commit brings following changes: 1. Removed torch dependency from "get_number_of_gpus" 2. Added keras mnist example notebook 3. Added keras mnist workspace example 4. Removed openfl-workspace/experimental/experimental-requirements.txt 5. "fx aggregator start" command will install libraries from experimental tutorials requirements_workflow_interface.txt file 6. Added testcase_datastore_cli, testcase_reference and testcase_reference_with_include_exclude 7. Updated 101 torch mnist plan.yaml and data.yaml 8. Removed aggregator start flow thread from cli/aggregator.py 9. At last changes made in federated runtime Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 4d7d995 Author: ParthM-GitHub <[email protected]> Date: Mon Jul 24 11:28:39 2023 +0530 Modified get_number_of_gpus function Modify get_number_of_gpus function Signed-off-by: Parth Mandaliya <[email protected]> commit cd7955f Author: ParthM-GitHub <[email protected]> Date: Fri Jul 21 18:01:14 2023 +0530 Update resources.py Signed-off-by: Parth Mandaliya <[email protected]> commit 69232cf Author: ParthM-GitHub <[email protected]> Date: Fri Jul 21 04:18:53 2023 +0530 Update 101 tutorial data.yaml Updated 101 tutorials data.yaml to correct a mistake by in index parameter of private attribute function. Signed-off-by: Parth Mandaliya <[email protected]> commit 023a573 Author: Parth Mandaliya <[email protected]> Date: Fri Jul 21 03:45:27 2023 +0530 Excluding 'experimental' directory from workspace templates Signed-off-by: Parth Mandaliya <[email protected]> commit 618d947 Author: Parth Mandaliya <[email protected]> Date: Fri Jul 21 03:18:15 2023 +0530 Modified 101, 102, 103, 301, 401 tutorials as per Aggregator, & Collaborator stateful collaborators Signed-off-by: Parth Mandaliya <[email protected]> commit 0deaac7 Author: ParthM-GitHub <[email protected]> Date: Fri Jul 21 02:53:49 2023 +0530 Updating 201 GPUs tutorial Updated Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb for Aggregator, and Collaborator as stateful actor Signed-off-by: Parth Mandaliya <[email protected]> commit 6eda625 Author: ParthM-GitHub <[email protected]> Date: Fri Jul 21 02:35:59 2023 +0530 Update requirements_workflow_interface.txt Updated experimental tutorials requirements.txt file Signed-off-by: Parth Mandaliya <[email protected]> commit 6c7ca1d Author: Parth Mandaliya <[email protected]> Date: Fri Jul 21 02:20:49 2023 +0530 Delete openfl/experimental/utilities/utils.py, and removed torch depency from openfl/experimental/utilities/resources.py Signed-off-by: Parth Mandaliya <[email protected]> commit e87dbcf Author: ParthM-GitHub <[email protected]> Date: Thu Jul 20 14:27:12 2023 +0530 Update requirements_workflow_interface.txt Metaflow, and Ray dependencies are moved to openfl-workspace/experimental/experimental-requrirements.txt file. Signed-off-by: Parth Mandaliya <[email protected]> commit 911a734 Author: ParthM-GitHub <[email protected]> Date: Thu Jul 20 14:12:19 2023 +0530 Update experimental-requirements.txt Added ray 2.2.0 to experimental-requirements.txt file Signed-off-by: Parth Mandaliya <[email protected]> commit defe192 Author: Parth Mandaliya <[email protected]> Date: Thu Jul 20 14:04:12 2023 +0530 Install experimental requirements while activating experimental openfl commands. Signed-off-by: Parth Mandaliya <[email protected]> commit 1154962 Author: Parth Mandaliya <[email protected]> Date: Thu Jul 20 13:45:00 2023 +0530 While activating experimental OpenFL commands, install depedencies as well. Signed-off-by: Parth Mandaliya <[email protected]> commit 3609303 Author: ParthM-GitHub <[email protected]> Date: Wed Jul 19 21:26:53 2023 +0530 FQDN switched to localhost FQDN switched to localhost Signed-off-by: Parth Mandaliya <[email protected]> commit 04e5297 Author: ParthM-GitHub <[email protected]> Date: Wed Jul 19 19:36:56 2023 +0530 Update aggregator.py Signed-off-by: Parth Mandaliya <[email protected]> commit b1b641e Author: Parth Mandaliya <[email protected]> Date: Wed Jul 19 17:07:10 2023 +0530 Modifying temp.sh for 2 collaborators. Signed-off-by: Parth Mandaliya <[email protected]> commit 1839223 Author: Parth Mandaliya <[email protected]> Date: Wed Jul 19 16:39:06 2023 +0530 plan.yaml modified in 301 watermarking, and subset of collaborators. Signed-off-by: Parth Mandaliya <[email protected]> commit 7b5d2fd Author: Parth Mandaliya <[email protected]> Date: Wed Jul 19 16:04:44 2023 +0530 Added following tutorials, and testcase: 1. Vertical FL 2. Vertical FL Two Party 3. Internal Loop Signed-off-by: Parth Mandaliya <[email protected]> commit 778294c Author: ParthM-GitHub <[email protected]> Date: Wed Jul 19 14:12:25 2023 +0530 External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> commit b67b725 Author: ParthM-GitHub <[email protected]> Date: Wed Jul 19 14:11:49 2023 +0530 External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> commit cdb8729 Author: ParthM-GitHub <[email protected]> Date: Wed Jul 19 14:10:49 2023 +0530 External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> commit f5bdb07 Author: Francis Storr <[email protected]> Date: Fri Jul 14 14:49:03 2023 -0700 Tweak link color so it’s not so aggressive (securefederatedai#865) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit b2bc10a Author: Keerti Prakash Talwar <[email protected]> Date: Sat Jul 15 01:39:53 2023 +0530 Testflow for verifying stdout redirection to Metaflow datastore (securefederatedai#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> commit 7666bfe Author: Francis Storr <[email protected]> Date: Fri Jul 14 13:09:03 2023 -0700 Accessibility color contrast fixes (securefederatedai#864) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes securefederatedai#848 Signed-off-by: Francis Storr <[email protected]> * Color contrast updates for accessibility Color contrast updates for accessibility - update generic `a` element - update color of links in the toggle-able read-the-docs panel - update the color of the text in search results - update the color of notes headers Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 7308c1e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Jul 14 12:03:04 2023 -0700 build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (securefederatedai#769) * build(deps): bump tensorflow-cpu Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow-cpu dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Use legacy RMSprop optimizer Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 3313f6e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Jul 14 11:52:42 2023 -0700 build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (securefederatedai#775) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Fixed breaking backages Signed-off-by: Patrick Foley <[email protected]> * Add quiet flag back to pip install Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit fea6684 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Jul 14 10:53:59 2023 -0700 build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (securefederatedai#770) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update Adam Optimizer import Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit d6d896e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Jul 14 10:49:08 2023 -0700 build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (securefederatedai#771) * build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * revert experimental Adam to legacy (securefederatedai#863) Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit ee05dfb Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Jul 14 10:48:30 2023 -0700 build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (securefederatedai#773) * build(deps): bump tensorflow in /openfl-workspace/keras_nlp Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](tensorflow/tensorflow@v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update RMSProp optimizer import Signed-off-by: Patrick Foley <[email protected]> * flake8 Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit f45fa04 Author: Francis Storr <[email protected]> Date: Fri Jul 14 09:50:49 2023 -0700 Accessibility updates (securefederatedai#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes securefederatedai#848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> commit 966840b Author: ParthM-GitHub <[email protected]> Date: Mon Jul 17 20:32:32 2023 +0530 Add files via upload Signed-off-by: Parth Mandaliya <[email protected]> commit 2ada448 Author: Parth Mandaliya <[email protected]> Date: Mon Jul 17 13:30:35 2023 +0530 Modified subset of collaborators testcase. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]>

* Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]>

* Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]>

* Fix flake8 error in local runtime (#764) * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#765) Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update GOVERNANCE.md Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#785) Typos Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated integrations to GaNDLF (#781) * renaming loader and runner Signed-off-by: sarthakpati <[email protected]> * updated plan to pick the new names Signed-off-by: sarthakpati <[email protected]> * new key name Signed-off-by: sarthakpati <[email protected]> * allow the ability to pass a file to `gandlf_config_dict` in addition to fully-fledged parameters Signed-off-by: sarthakpati <[email protected]> * checking this differently Signed-off-by: sarthakpati <[email protected]> * rename variable for clarity Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Introduced shard descriptor based collaborator private attributes Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding batch size for train, and test in config.yaml file Files modified: 1. config.yaml 2. mnist_shard_descriptor.py 3. Workflow_Interface_101_MNIST.ipynb 4. participants.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Introducing multiple config yaml files Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary config.yaml file. Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added collaborator private atribute delayed initialization for local_runtime Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incorporated review comments Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added multi-pricessing ray backend support and, aggregator yaml file Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated multi-processing code Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * RayExecutor class moved from participants.py to localruntime.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * RayExecytor moved from interface/pariticipants.py to runtime/local_runtime.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added Aggregator private attribute initialation in runtime Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed unnecessary import statements Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Code cleaned up, validated checkpoints manually Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Refactored, and added some new doc string Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved Flake8 instructions Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Recusrsion removal + Serialization removal integrated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incoporated Review Comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed configuration YAML files, and added functionality to initialize private attributes by calling a callback function created by end-user Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed commented code Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Implemented new approach, two example files given 1. Workflow_Interface_101_MNIST.py 2. Workflow_Interface_301_MNIST_Watermarking.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Internal Review Comments Incorporated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * No private attributes are required If private attributes are not provided, by default take an empty dictionary no need to pass a callable function. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update participants.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added a check for GPU Resource Allocation Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified error message for resource allocation Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved bug found during testing phase Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifide all the test cases, and following tutorials 1. Privacy Meter 2. FedProx Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added following test cases: 1. Workflow_Interface_101_MNIST.ipynb 2. Workflow_Interface_102_Aggregator_Validation.ipynb 3. Workflow_Interface_301_MNIST_Watermarking.ipynb 4. Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb 5. Workflow_Interface_103_Cyclic_Institutional_Incremental_Learning.ipynb Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified and Added Global_DP tutorials. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified and Added tutorial Workflow-Interface_201_Exclusive_GPUs_with_Ray.ipynb Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified documentation for Workflow_Interface_201 tutorial. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * fixed flake-8 errors Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * reverted import module code Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved merge conflicts in local_runtime.py --------- Fix flake8 error in local runtime (#764) * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Signed-off-by: Parth Mandaliya <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Parth Mandaliya <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Running a federation with GaNDLF Documentation (#794) * Initial commit of Running the federation with GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: Patrick Foley <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Fix documentation links. Change path names and templates for CI Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix breaking tests Signed-off-by: Patrick Foley <[email protected]> * Add compatible onnx version to requirements.txt file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong names in workflow file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong data path Signed-off-by: Patrick Foley <[email protected]> * Fix lint in test_gandlf.py Signed-off-by: Patrick Foley <[email protected]> * Fix lint errors Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid (#841) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix GaNDLF documentation links (#842) * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix incorrectly formatted link in docs (#839) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in local-runtime.py Integrated aggregator as stateful actor branch, tested. Signed-off-by: Parth Mandaliya <[email protected]> -------- Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump onnx in /openfl-workspace/gandlf_seg_test (#840) Bumps [onnx](https://github.com/onnx/onnx) from 1.12 to 1.13.0. - [Release notes](https://github.com/onnx/onnx/releases) - [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md) - [Commits](https://github.com/onnx/onnx/compare/v1.12.0...v1.13.0) --- updated-dependencies: - dependency-name: onnx dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Merged changes of remove-torch-dependency branch Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update setup.py Upgrading protobuf to 3.20.3 as per tensorboard requirement Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (#773) * build(deps): bump tensorflow in /openfl-workspace/keras_nlp Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update RMSProp optimizer import Signed-off-by: Patrick Foley <[email protected]> * flake8 Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (#771) * build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * revert experimental Adam to legacy (#863) Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (#770) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update Adam Optimizer import Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (#775) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Fixed breaking backages Signed-off-by: Patrick Foley <[email protected]> * Add quiet flag back to pip install Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (#769) * build(deps): bump tensorflow-cpu Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow-cpu dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Use legacy RMSprop optimizer Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility color contrast fixes (#864) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> * Color contrast updates for accessibility Color contrast updates for accessibility - update generic `a` element - update color of links in the toggle-able read-the-docs panel - update the color of the text in search results - update the color of notes headers Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tweak link color so it’s not so aggressive (#865) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (#772) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update sd_requirements.txt * revert to legacy SGD and install tensorflow==2.11 for workflow Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (#774) * build(deps): bump tensorflow in /openfl-workspace/tf_2dunet Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.txt to retrigger CI * Update requirements.txt --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Tensorflow, gRPC, Protobuf dependencies (#868) * Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add FL plan description to documentation (#872) * Add plan description to documentation Signed-off-by: Mansi Sharma <[email protected]> * fix indentation Signed-off-by: Mansi Sharma <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick Foley <[email protected]> --------- Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * GPU Added for aggregator Fixed issue in 103 Cyclic Institutional Incremental Learning tutorial Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolve Coverity Issues (#874) * Fix coverity issues * Resolve remaining coverity issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Migrate to Ubuntu 22.04 LTS release (supported through 2027) (#875) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated documentation: docs/workflow_interface.rst Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated documentation Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update workflow_interface.rst Fixing typo Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added best model and last model extraction technique in docs/workflow_interface.rst Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added GPU for aggregator Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in 103 cyclic tutorial notebook fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * Resolved merge conflicts in tests/github/experimental/testflow_datastore_cli.py Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> * Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update EdenPipeline in the documentation (#877) Signed-off-by: Amit Portnoy <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * WIP: CI Scans (#873) * Initial scans commit for bandit, hadolint, trivy Signed-off-by: Patrick Foley <[email protected]> * Address bandit scan results Signed-off-by: Patrick Foley <[email protected]> * Fix Trivy action Signed-off-by: Patrick Foley <[email protected]> * Fix linting Signed-off-by: Patrick Foley <[email protected]> * Add Coverity Badge Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#878) Signed-off-by: Parth Mandaliya <[email protected]> * initial commit * add docstrings * change importlib to import * remove unnecesary files, replace ray with ray_grouped * remove max concurency, add number of actors * Trigger CI * run tests * lint changes * flake * changed number of actors to num_actors, added docs * Fixed workflow API tests Signed-off-by: Patrick Foley <[email protected]> * lint fixes Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Grant Baker <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Francis Storr <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Signed-off-by: Amit Portnoy <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Olga Perepelkina <[email protected]> Co-authored-by: Joe Devon <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Keerti Talwar <[email protected]> Co-authored-by: KeertiX <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Francis Storr <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: kta-intel <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Sachin Gupta <[email protected]> Co-authored-by: Keerti Prakash Talwar <[email protected]> Co-authored-by: Amit Portnoy <[email protected]>

* Fix flake8 error in local runtime (#764) * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#765) Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update GOVERNANCE.md Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#785) Typos Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated integrations to GaNDLF (#781) * renaming loader and runner Signed-off-by: sarthakpati <[email protected]> * updated plan to pick the new names Signed-off-by: sarthakpati <[email protected]> * new key name Signed-off-by: sarthakpati <[email protected]> * allow the ability to pass a file to `gandlf_config_dict` in addition to fully-fledged parameters Signed-off-by: sarthakpati <[email protected]> * checking this differently Signed-off-by: sarthakpati <[email protected]> * rename variable for clarity Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Introduced shard descriptor based collaborator private attributes Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding batch size for train, and test in config.yaml file Files modified: 1. config.yaml 2. mnist_shard_descriptor.py 3. Workflow_Interface_101_MNIST.ipynb 4. participants.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Introducing multiple config yaml files Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary config.yaml file. Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added collaborator private atribute delayed initialization for local_runtime Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incorporated review comments Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added multi-pricessing ray backend support and, aggregator yaml file Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated multi-processing code Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * RayExecutor class moved from participants.py to localruntime.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * RayExecytor moved from interface/pariticipants.py to runtime/local_runtime.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added Aggregator private attribute initialation in runtime Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed unnecessary import statements Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Code cleaned up, validated checkpoints manually Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Refactored, and added some new doc string Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved Flake8 instructions Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Recusrsion removal + Serialization removal integrated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incoporated Review Comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed configuration YAML files, and added functionality to initialize private attributes by calling a callback function created by end-user Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed commented code Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Implemented new approach, two example files given 1. Workflow_Interface_101_MNIST.py 2. Workflow_Interface_301_MNIST_Watermarking.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Internal Review Comments Incorporated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * No private attributes are required If private attributes are not provided, by default take an empty dictionary no need to pass a callable function. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update participants.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added a check for GPU Resource Allocation Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified error message for resource allocation Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved bug found during testing phase Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifide all the test cases, and following tutorials 1. Privacy Meter 2. FedProx Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added following test cases: 1. Workflow_Interface_101_MNIST.ipynb 2. Workflow_Interface_102_Aggregator_Validation.ipynb 3. Workflow_Interface_301_MNIST_Watermarking.ipynb 4. Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb 5. Workflow_Interface_103_Cyclic_Institutional_Incremental_Learning.ipynb Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified and Added Global_DP tutorials. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified and Added tutorial Workflow-Interface_201_Exclusive_GPUs_with_Ray.ipynb Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified documentation for Workflow_Interface_201 tutorial. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * fixed flake-8 errors Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * reverted import module code Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved merge conflicts in local_runtime.py --------- Fix flake8 error in local runtime (#764) * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Signed-off-by: Parth Mandaliya <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Parth Mandaliya <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Running a federation with GaNDLF Documentation (#794) * Initial commit of Running the federation with GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: Patrick Foley <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Fix documentation links. Change path names and templates for CI Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix breaking tests Signed-off-by: Patrick Foley <[email protected]> * Add compatible onnx version to requirements.txt file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong names in workflow file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong data path Signed-off-by: Patrick Foley <[email protected]> * Fix lint in test_gandlf.py Signed-off-by: Patrick Foley <[email protected]> * Fix lint errors Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid (#841) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix GaNDLF documentation links (#842) * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix incorrectly formatted link in docs (#839) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in local-runtime.py Integrated aggregator as stateful actor branch, tested. Signed-off-by: Parth Mandaliya <[email protected]> -------- Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump onnx in /openfl-workspace/gandlf_seg_test (#840) Bumps [onnx](https://github.com/onnx/onnx) from 1.12 to 1.13.0. - [Release notes](https://github.com/onnx/onnx/releases) - [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md) - [Commits](https://github.com/onnx/onnx/compare/v1.12.0...v1.13.0) --- updated-dependencies: - dependency-name: onnx dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Merged changes of remove-torch-dependency branch Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update setup.py Upgrading protobuf to 3.20.3 as per tensorboard requirement Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (#773) * build(deps): bump tensorflow in /openfl-workspace/keras_nlp Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update RMSProp optimizer import Signed-off-by: Patrick Foley <[email protected]> * flake8 Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (#771) * build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * revert experimental Adam to legacy (#863) Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (#770) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update Adam Optimizer import Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (#775) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Fixed breaking backages Signed-off-by: Patrick Foley <[email protected]> * Add quiet flag back to pip install Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (#769) * build(deps): bump tensorflow-cpu Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow-cpu dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Use legacy RMSprop optimizer Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility color contrast fixes (#864) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> * Color contrast updates for accessibility Color contrast updates for accessibility - update generic `a` element - update color of links in the toggle-able read-the-docs panel - update the color of the text in search results - update the color of notes headers Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tweak link color so it’s not so aggressive (#865) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (#772) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update sd_requirements.txt * revert to legacy SGD and install tensorflow==2.11 for workflow Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (#774) * build(deps): bump tensorflow in /openfl-workspace/tf_2dunet Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.txt to retrigger CI * Update requirements.txt --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Tensorflow, gRPC, Protobuf dependencies (#868) * Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add FL plan description to documentation (#872) * Add plan description to documentation Signed-off-by: Mansi Sharma <[email protected]> * fix indentation Signed-off-by: Mansi Sharma <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick Foley <[email protected]> --------- Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * GPU Added for aggregator Fixed issue in 103 Cyclic Institutional Incremental Learning tutorial Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolve Coverity Issues (#874) * Fix coverity issues * Resolve remaining coverity issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Migrate to Ubuntu 22.04 LTS release (supported through 2027) (#875) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated documentation: docs/workflow_interface.rst Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated documentation Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update workflow_interface.rst Fixing typo Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added best model and last model extraction technique in docs/workflow_interface.rst Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added GPU for aggregator Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in 103 cyclic tutorial notebook fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * Resolved merge conflicts in tests/github/experimental/testflow_datastore_cli.py Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> * Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update EdenPipeline in the documentation (#877) Signed-off-by: Amit Portnoy <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * WIP: CI Scans (#873) * Initial scans commit for bandit, hadolint, trivy Signed-off-by: Patrick Foley <[email protected]> * Address bandit scan results Signed-off-by: Patrick Foley <[email protected]> * Fix Trivy action Signed-off-by: Patrick Foley <[email protected]> * Fix linting Signed-off-by: Patrick Foley <[email protected]> * Add Coverity Badge Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#878) Signed-off-by: Parth Mandaliya <[email protected]> * initial commit * add docstrings * change importlib to import * remove unnecesary files, replace ray with ray_grouped * remove max concurency, add number of actors * Trigger CI * run tests * lint changes * flake * changed number of actors to num_actors, added docs * Fixed workflow API tests Signed-off-by: Patrick Foley <[email protected]> * lint fixes Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Grant Baker <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Francis Storr <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Signed-off-by: Amit Portnoy <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Olga Perepelkina <[email protected]> Co-authored-by: Joe Devon <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Keerti Talwar <[email protected]> Co-authored-by: KeertiX <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Francis Storr <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: kta-intel <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Sachin Gupta <[email protected]> Co-authored-by: Keerti Prakash Talwar <[email protected]> Co-authored-by: Amit Portnoy <[email protected]> Signed-off-by: nammbash <[email protected]>

* Federated Runtime v1.0 experimental Aggregator and Collaborator in working condition Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added test, and mnist experimental workspaces working prototype Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Working on integration with Ishant's, and Keerti's code Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Second half of integration installment. Remaning: Integration of data.yaml file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary directory my_federation. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * 1. Removed unncessart files. 2. Fix `fx plan freeze` command not working issue. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added function for test workspace Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added include_exclude testcase workspace. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Default execution workspace is mnist. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * data.yaml changes are integrated with this update. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Running a federation with GaNDLF Documentation (#794) * Initial commit of Running the federation with GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: Patrick Foley <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Fix documentation links. Change path names and templates for CI Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix breaking tests Signed-off-by: Patrick Foley <[email protected]> * Add compatible onnx version to requirements.txt file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong names in workflow file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong data path Signed-off-by: Patrick Foley <[email protected]> * Fix lint in test_gandlf.py Signed-off-by: Patrick Foley <[email protected]> * Fix lint errors Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid (#841) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix GaNDLF documentation links (#842) * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix incorrectly formatted link in docs (#839) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added licensing information to all experimental workspaces Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Made changes to LocalRuntime, and tested Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Rename collaborator_name variable to name in experimental/component/collaborator/collaborator.py file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump onnx in /openfl-workspace/gandlf_seg_test (#840) Bumps [onnx](https://github.com/onnx/onnx) from 1.12 to 1.13.0. - [Release notes](https://github.com/onnx/onnx/releases) - [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md) - [Commits](https://github.com/onnx/onnx/compare/v1.12.0...v1.13.0) --- updated-dependencies: - dependency-name: onnx dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Selected collaborators functionality added, test workspace of the same is work-progress. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Selected collaborator testcases included with this commit. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Testcase private attributes workspace added with this commit. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Instead of remove private attributes from checkpoint, changed the value to string "Private attributes: Not Available." Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added / Modified types of funciton arguments, and their return types. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Aggregator shutdown related changes are done in this commit Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update testflow_subset_of_collaborators.py Updated subset of collaborators testcase Signed-off-by: Parth Mandaliya <[email protected]> * Modified subset of collaborators testcase. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add files via upload Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (#773) * build(deps): bump tensorflow in /openfl-workspace/keras_nlp Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update RMSProp optimizer import Signed-off-by: Patrick Foley <[email protected]> * flake8 Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (#771) * build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * revert experimental Adam to legacy (#863) Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (#770) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update Adam Optimizer import Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (#775) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Fixed breaking backages Signed-off-by: Patrick Foley <[email protected]> * Add quiet flag back to pip install Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (#769) * build(deps): bump tensorflow-cpu Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow-cpu dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Use legacy RMSprop optimizer Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility color contrast fixes (#864) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> * Color contrast updates for accessibility Color contrast updates for accessibility - update generic `a` element - update color of links in the toggle-able read-the-docs panel - update the color of the text in search results - update the color of notes headers Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tweak link color so it’s not so aggressive (#865) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added following tutorials, and testcase: 1. Vertical FL 2. Vertical FL Two Party 3. Internal Loop Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * plan.yaml modified in 301 watermarking, and subset of collaborators. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifying temp.sh for 2 collaborators. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update aggregator.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * FQDN switched to localhost FQDN switched to localhost Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * While activating experimental OpenFL commands, install depedencies as well. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Install experimental requirements while activating experimental openfl commands. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update experimental-requirements.txt Added ray 2.2.0 to experimental-requirements.txt file Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update requirements_workflow_interface.txt Metaflow, and Ray dependencies are moved to openfl-workspace/experimental/experimental-requrirements.txt file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Delete openfl/experimental/utilities/utils.py, and removed torch depency from openfl/experimental/utilities/resources.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update requirements_workflow_interface.txt Updated experimental tutorials requirements.txt file Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updating 201 GPUs tutorial Updated Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb for Aggregator, and Collaborator as stateful actor Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified 101, 102, 103, 301, 401 tutorials as per Aggregator, & Collaborator stateful collaborators Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Excluding 'experimental' directory from workspace templates Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update 101 tutorial data.yaml Updated 101 tutorials data.yaml to correct a mistake by in index parameter of private attribute function. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update resources.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified get_number_of_gpus function Modify get_number_of_gpus function Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Removed torch dependency from "get_number_of_gpus" 2. Added keras mnist example notebook 3. Added keras mnist workspace example 4. Removed openfl-workspace/experimental/experimental-requirements.txt 5. "fx aggregator start" command will install libraries from experimental tutorials requirements_workflow_interface.txt file 6. Added testcase_datastore_cli, testcase_reference and testcase_reference_with_include_exclude 7. Updated 101 torch mnist plan.yaml and data.yaml 8. Removed aggregator start flow thread from cli/aggregator.py 9. At last changes made in federated runtime Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (#772) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update sd_requirements.txt * revert to legacy SGD and install tensorflow==2.11 for workflow Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (#774) * build(deps): bump tensorflow in /openfl-workspace/tf_2dunet Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.txt to retrigger CI * Update requirements.txt --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Renamed Workflow_Interface_101_Keras_MNIST_with_GPU.ipynb to Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb 2. Updated documentation in Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb 3. protobuf bumped from 3.19.6 to 3.20.0 in setup.py 4. Incorporating review comments (WIP) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified 104 Keras MNIST tutorial Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This comming brings following changes: 1. Experimental Aggregator workflow testcases moved to test/github/experimental/workspace directory 2. Added dependecy related changes Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary logs Removing unnecessary logs Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update collaborator.py Removing unnecessary logs Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Avoided deep copying the context by removing private attributes from context before checkpoint Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incoporated review comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added dependecies to each tutorial notebooks Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * pinned tensorboardX (#870) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * While taking aggregator step remove private attributes before taking checkpoint and set it back to ctx after checkpoint Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary deepcopies from aggregator.py file Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update stream_redirect.py Removing unnecessary imports Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added color escape sequences to datastore cli localruntime and workspace testcases Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update requirements_privacy_meter.txt torch 1.13.1 requires torchvision 0.14.1 not 0.14.0 Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Workflow_Interface_VFL_Two_Party Added actor related changed to Workflow_Interface_VFL_Two_Party Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Files added required to test experimental aggregator based workflow by GitHUB actions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb Removing GPU parameter from collaborator decorator. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update resources.py Updated get_number_of_gpus function Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updating import statement In 104 keras workspace utils.py updating import SummaryWriter statement from torch to tensorflow Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * With this commit path of template can be explicitely given, user does not have to choose from available example templates. Only applicable to experimental openfl. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues and typos Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved flake8 issues, Added warning statements when data.yaml is not found or aggregator / collaborator private attributes are not found in data.yaml, Tested by running all tutorials and tests Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Moving the logs after data.yaml warning, and removing unnecessary blank lines. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding N-level arguments functionality Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update workflow_interface.rst Added section on how to save best and last model from Metaflow CLI Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving last few TODO items. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated logs to provide exact information to user. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following updates: 1. Remove unnecessary spaces from Workflow_Interface_102_Aggregator_Validation.ipynb 2. Upgraded protobuf to 3.20.3 in setup.py as per tensorboard requirements 3. Modified 101-torch-cnn-mnist workspace for n-level arguments 4. Added a federated runtime workspace for 102-aggregator-validation Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Added 501_pytorch_tinyimagenet_transfer_learning workspace for experimental workflow interface 2. Removed `feature_shape` from openfl/experimental/interface/cli/plan.py 3. Resolved flake8 errors in openfl-workspace/experimental directory Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update data.yaml Correcting a mistake train dataset assigned to test and vice versa. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update flow.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update flow.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding functionality to convert jupyter notebook to workspace (in progress) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Write flow.py, collaborator_private_attrs.py and aggregator_private_attrs.py automatically Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Find aggregator, and collaborator arguments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifying arguments help Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed automatically generated Zone.Identifier file Added Approach2 notebook Modified creator.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * workspace_creator renamed to workspace_builder finding values without help of any tags Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Finished exporting data.yaml, plan.yaml. Workspace creator validated for 101 MNIST and 301 Watermarking with MNIST Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing round number constant argument from 301 MNIST WATERMARKING Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * * Created get_collaborator_kwargs function in localruntime class which takes collaborator name as argument and returns name of private attributes function and arguments required to call it. * Changed workspace builder code used get_collaborator_kwargs function to build data.yaml file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified get_collaborator_args function Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified workspace_builder/build.py file to resolve a typo Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed unnecessary files from template_workspace directory Moved print_tree to after generating python script Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added documentation and inline comments for workspace_builder Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added libraries required by WorkspaceBuilder Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Tensorflow, gRPC, Protobuf dependencies (#868) * Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add FL plan description to documentation (#872) * Add plan description to documentation Signed-off-by: Mansi Sharma <[email protected]> * fix indentation Signed-off-by: Mansi Sharma <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick Foley <[email protected]> --------- Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolve Coverity Issues (#874) * Fix coverity issues * Resolve remaining coverity issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Migrate to Ubuntu 22.04 LTS release (supported through 2027) (#875) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues in test/experimental directory Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving DCO issue fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update EdenPipeline in the documentation (#877) Signed-off-by: Amit Portnoy <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tested all local-runtime & federated-runtime tutorials and testcases Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in experimental local-runtme tutorials 101, 103, 201, & 301 fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * WIP: CI Scans (#873) * Initial scans commit for bandit, hadolint, trivy Signed-off-by: Patrick Foley <[email protected]> * Address bandit scan results Signed-off-by: Patrick Foley <[email protected]> * Fix Trivy action Signed-off-by: Patrick Foley <[email protected]> * Fix linting Signed-off-by: Patrick Foley <[email protected]> * Add Coverity Badge Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#878) Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Renaming workspace_builder module to workspace_export. 2. WorkspaceBuilder class renamed to WorkspaceExport. 3. Added WorkspaceExport.export function for user which wraps generate_requirements, generate_plan_yaml, generate_data_yaml functions. 4. Merged WorkspaceExport module to `fx workspace create` command by adding new following arguments: - `--notebook` Path to jupyter notebook - `--template_workspace_dir` Output path for generated workspace 5. WorkspaceExport removed from 101, and 103 tutorials. 6. setup.py changed accordingly. Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. output_dir renamed to output_workspace also the user must pass this parameter (default=/tmp is now removed) or there will be an error. 2. template_workspace_path is not required anymore. 3. Added a new tutorials Workflow_Interface_1001_Workspace_Creation_from_JupyterNotebook.ipynb (copy of Workflow_Interface_301_MNIST_Watermarking.ipynb) explaining how to use WorkspaceExport. Signed-off-by: Parth Mandaliya <[email protected]> * In Workflow_Interface_1001 tutorial list of collaborator are changed to 5 from 2 Signed-off-by: Parth Mandaliya <[email protected]> * Uncommeted pip install libraries Signed-off-by: Parth Mandaliya <[email protected]> * Adding GPU for aggregator in 201, Global DP & Privacy Meter tutorials Signed-off-by: Parth Mandaliya <[email protected]> * Modifed documentation in 1001 tutorial Signed-off-by: Parth Mandaliya <[email protected]> * Modified documentation in 1001 tutorial Signed-off-by: Parth Mandaliya <[email protected]> * Fixing 'param' attribute not found issue. in 301, 1001 experimental tutrials and 301 mnist watermarking workspace. Sol. imagen library installs param depedency latest version and param package got an upgrde recently. Pinned param to 1.13.0 after installing imagen which fixed the issue. Signed-off-by: Parth Mandaliya <[email protected]> * VFL 2 party example Signed-off-by: Mansi Sharma <[email protected]> * participant names to lowercase and changes to instructions Signed-off-by: Mansi Sharma <[email protected]> * Resolving missing packages from watermarking examples that break python 3.9+. Fix linting, spelling. Remove experimental aggregation functions Signed-off-by: Patrick Foley <[email protected]> * Remove duplicate files in openfl.experimental Signed-off-by: Patrick Foley <[email protected]> * Trigger CI Signed-off-by: Patrick Foley <[email protected]> * Moving test to be conditional Signed-off-by: Patrick Foley <[email protected]> * Fix inconsistent use of enumerate in loops Signed-off-by: Patrick Foley <[email protected]> * Fixing tests Signed-off-by: Patrick Foley <[email protected]> * Accomodate conversion of names to lower case Signed-off-by: Patrick Foley <[email protected]> * Missing torchvision dependency in tests Signed-off-by: Patrick Foley <[email protected]> * Fix lint issues Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Francis Storr <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Signed-off-by: Amit Portnoy <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Francis Storr <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Keerti Prakash Talwar <[email protected]> Co-authored-by: kta-intel <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Amit Portnoy <[email protected]>

* Fix flake8 error in local runtime (#764) * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#765) Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update GOVERNANCE.md Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#785) Typos Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated integrations to GaNDLF (#781) * renaming loader and runner Signed-off-by: sarthakpati <[email protected]> * updated plan to pick the new names Signed-off-by: sarthakpati <[email protected]> * new key name Signed-off-by: sarthakpati <[email protected]> * allow the ability to pass a file to `gandlf_config_dict` in addition to fully-fledged parameters Signed-off-by: sarthakpati <[email protected]> * checking this differently Signed-off-by: sarthakpati <[email protected]> * rename variable for clarity Signed-off-by: sarthakpati <[email protected]> --------- Signed-off-by: sarthakpati <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Introduced shard descriptor based collaborator private attributes Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding batch size for train, and test in config.yaml file Files modified: 1. config.yaml 2. mnist_shard_descriptor.py 3. Workflow_Interface_101_MNIST.ipynb 4. participants.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Introducing multiple config yaml files Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary config.yaml file. Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added collaborator private atribute delayed initialization for local_runtime Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incorporated review comments Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added multi-pricessing ray backend support and, aggregator yaml file Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated multi-processing code Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * RayExecutor class moved from participants.py to localruntime.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * RayExecytor moved from interface/pariticipants.py to runtime/local_runtime.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added Aggregator private attribute initialation in runtime Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed unnecessary import statements Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Code cleaned up, validated checkpoints manually Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Refactored, and added some new doc string Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved Flake8 instructions Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Recusrsion removal + Serialization removal integrated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incoporated Review Comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed configuration YAML files, and added functionality to initialize private attributes by calling a callback function created by end-user Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed commented code Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Implemented new approach, two example files given 1. Workflow_Interface_101_MNIST.py 2. Workflow_Interface_301_MNIST_Watermarking.py Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Internal Review Comments Incorporated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * No private attributes are required If private attributes are not provided, by default take an empty dictionary no need to pass a callable function. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update participants.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added a check for GPU Resource Allocation Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified error message for resource allocation Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved bug found during testing phase Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifide all the test cases, and following tutorials 1. Privacy Meter 2. FedProx Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added following test cases: 1. Workflow_Interface_101_MNIST.ipynb 2. Workflow_Interface_102_Aggregator_Validation.ipynb 3. Workflow_Interface_301_MNIST_Watermarking.ipynb 4. Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb 5. Workflow_Interface_103_Cyclic_Institutional_Incremental_Learning.ipynb Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified and Added Global_DP tutorials. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified and Added tutorial Workflow-Interface_201_Exclusive_GPUs_with_Ray.ipynb Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified documentation for Workflow_Interface_201 tutorial. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * fixed flake-8 errors Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * reverted import module code Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved merge conflicts in local_runtime.py --------- Fix flake8 error in local runtime (#764) * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> * Removes unnecessary dict comprehension Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * Update README.md Signed-off-by: Parth Mandaliya <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Parth Mandaliya <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Running a federation with GaNDLF Documentation (#794) * Initial commit of Running the federation with GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: Patrick Foley <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Fix documentation links. Change path names and templates for CI Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix breaking tests Signed-off-by: Patrick Foley <[email protected]> * Add compatible onnx version to requirements.txt file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong names in workflow file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong data path Signed-off-by: Patrick Foley <[email protected]> * Fix lint in test_gandlf.py Signed-off-by: Patrick Foley <[email protected]> * Fix lint errors Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid (#841) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix GaNDLF documentation links (#842) * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix incorrectly formatted link in docs (#839) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in local-runtime.py Integrated aggregator as stateful actor branch, tested. Signed-off-by: Parth Mandaliya <[email protected]> -------- Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump onnx in /openfl-workspace/gandlf_seg_test (#840) Bumps [onnx](https://github.com/onnx/onnx) from 1.12 to 1.13.0. - [Release notes](https://github.com/onnx/onnx/releases) - [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md) - [Commits](https://github.com/onnx/onnx/compare/v1.12.0...v1.13.0) --- updated-dependencies: - dependency-name: onnx dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Merged changes of remove-torch-dependency branch Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update setup.py Upgrading protobuf to 3.20.3 as per tensorboard requirement Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (#773) * build(deps): bump tensorflow in /openfl-workspace/keras_nlp Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update RMSProp optimizer import Signed-off-by: Patrick Foley <[email protected]> * flake8 Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (#771) * build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * revert experimental Adam to legacy (#863) Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (#770) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update Adam Optimizer import Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (#775) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Fixed breaking backages Signed-off-by: Patrick Foley <[email protected]> * Add quiet flag back to pip install Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (#769) * build(deps): bump tensorflow-cpu Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow-cpu dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Use legacy RMSprop optimizer Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility color contrast fixes (#864) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> * Color contrast updates for accessibility Color contrast updates for accessibility - update generic `a` element - update color of links in the toggle-able read-the-docs panel - update the color of the text in search results - update the color of notes headers Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tweak link color so it’s not so aggressive (#865) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (#772) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update sd_requirements.txt * revert to legacy SGD and install tensorflow==2.11 for workflow Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (#774) * build(deps): bump tensorflow in /openfl-workspace/tf_2dunet Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.txt to retrigger CI * Update requirements.txt --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Tensorflow, gRPC, Protobuf dependencies (#868) * Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add FL plan description to documentation (#872) * Add plan description to documentation Signed-off-by: Mansi Sharma <[email protected]> * fix indentation Signed-off-by: Mansi Sharma <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick Foley <[email protected]> --------- Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * GPU Added for aggregator Fixed issue in 103 Cyclic Institutional Incremental Learning tutorial Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolve Coverity Issues (#874) * Fix coverity issues * Resolve remaining coverity issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Migrate to Ubuntu 22.04 LTS release (supported through 2027) (#875) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated documentation: docs/workflow_interface.rst Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated documentation Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update workflow_interface.rst Fixing typo Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added best model and last model extraction technique in docs/workflow_interface.rst Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added GPU for aggregator Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in 103 cyclic tutorial notebook fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * Resolved merge conflicts in tests/github/experimental/testflow_datastore_cli.py Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> * Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update EdenPipeline in the documentation (#877) Signed-off-by: Amit Portnoy <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * WIP: CI Scans (#873) * Initial scans commit for bandit, hadolint, trivy Signed-off-by: Patrick Foley <[email protected]> * Address bandit scan results Signed-off-by: Patrick Foley <[email protected]> * Fix Trivy action Signed-off-by: Patrick Foley <[email protected]> * Fix linting Signed-off-by: Patrick Foley <[email protected]> * Add Coverity Badge Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#878) Signed-off-by: Parth Mandaliya <[email protected]> * initial commit * add docstrings * change importlib to import * remove unnecesary files, replace ray with ray_grouped * remove max concurency, add number of actors * Trigger CI * run tests * lint changes * flake * changed number of actors to num_actors, added docs * Fixed workflow API tests Signed-off-by: Patrick Foley <[email protected]> * lint fixes Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: ParthM-GitHub <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: sarthakpati <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Grant Baker <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Francis Storr <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Signed-off-by: Amit Portnoy <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Olga Perepelkina <[email protected]> Co-authored-by: Joe Devon <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Keerti Talwar <[email protected]> Co-authored-by: KeertiX <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Francis Storr <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: kta-intel <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Sachin Gupta <[email protected]> Co-authored-by: Keerti Prakash Talwar <[email protected]> Co-authored-by: Amit Portnoy <[email protected]>

* Federated Runtime v1.0 experimental Aggregator and Collaborator in working condition Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added test, and mnist experimental workspaces working prototype Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Working on integration with Ishant's, and Keerti's code Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Second half of integration installment. Remaning: Integration of data.yaml file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary directory my_federation. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * 1. Removed unncessart files. 2. Fix `fx plan freeze` command not working issue. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added function for test workspace Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added include_exclude testcase workspace. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Default execution workspace is mnist. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * data.yaml changes are integrated with this update. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Running a federation with GaNDLF Documentation (#794) * Initial commit of Running the federation with GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update docs/running_the_federation_with_gandlf.rst Co-authored-by: Sarthak Pati <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Removed references to Intel's ownship, given it's now owned by the LF AI and Data. Signed-off-by: Patrick Foley <[email protected]> * Fix Flake8 C419 for Ubuntu CI (#800) C419 Unnecessary list comprehension passed to any()/all() prevents short-circuiting - rewrite as a generator Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Fix warnings and issues in docs (#825) * Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> * fixup! Fix warnings and issues in docs Signed-off-by: Aleksander Kantak <[email protected]> --------- Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add Logo (#827) * Add Logo * Update README.md Signed-off-by: Patrick Foley <[email protected]> * Change OpenFL documentation font to improve accessibility (#809) This replaces the font of OpenFL documents with Intel One Mono font for low vision developers. Known issues: 1. The text font within the images has not been changed. 2. Some icons that do not exist in the new font cannot be displayed properly. Fixes securefederatedai#799 Co-authored-by: Wang, Le <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Co-authored-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update unit tests to improve code coverage (#821) * Update ci config Signed-off-by: Fang, Xiaoran <[email protected]> * Add unit test for following files - openfl/federated/plan/plan.py - openfl/interface/aggregation_functions/core/adaptive_aggregation.py Signed-off-by: Fang, Xiaoran <[email protected]> * Add some test cases for databases module Signed-off-by: Fang, Xiaoran <[email protected]> * Fix bugs for databases module unittest Signed-off-by: Fang, Xiaoran <[email protected]> * Update unit tests for component module Signed-off-by: Fang, Xiaoran <[email protected]> * Restore workflow config and update some comments Signed-off-by: Fang, Xiaoran <[email protected]> * Enable save_ test case. Add yaml under test dir for unit test usage. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Remove plan to new dir. Signed-off-by: Fang, Xiaoran <[email protected]> * Add aggregator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 2 aggregator test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 aggregator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format code. Signed-off-by: Fang, Xiaoran <[email protected]> * Refactor code. Signed-off-by: Fang, Xiaoran <[email protected]> * Add collaborator start test cases. Signed-off-by: Fang, Xiaoran <[email protected]> * Add 1 collaborator test case. Signed-off-by: Fang, Xiaoran <[email protected]> * Format with flake8 Signed-off-by: Fang, Xiaoran <[email protected]> * Remove TODO comments Signed-off-by: Fang, Xiaoran <[email protected]> --------- Signed-off-by: Fang, Xiaoran <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add PyTorch linear regression example (#808) This adds a new tutorial example on distributing a linear regression task over OpenFL cluster. The model is defined by Pytorch which is able to run over both cpu (by default) and gpu. The dataset is generated by make_regression from sklearn.datasets with pre-defined parameters. Fixes #797 Co-authored-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * This prints out the hash of the CSR to disk for both the aggregator and (#813) * This prints out the hash of the CSR to disk for both the aggregator and collaborator. The user then compares and approves this hash with the hash printed out of the file to validate the CSR. In addition, a warning message is pritned if certify is run in silent mode. Fixes securefederatedai#692 Signed-off-by: Grant Baker <[email protected]> * Refactor read_csr function to use get_csr_hash Signed-off-by: Grant Baker <[email protected]> * Ask to check hashes before prompt --------- Signed-off-by: Grant Baker <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Improve workspace requirements import (#810) Remove the dump_requirement_file operation in export_ method. Fixes securefederatedai#767 Co-authored-by: Li, Qingqing <[email protected]> Co-authored-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Jiang, Jiaqiu <[email protected]> Signed-off-by: Li, Qingqing <[email protected]> Signed-off-by: Wang, Le <[email protected]> Signed-off-by: Wu, Caili <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Issue 506 Added Example using FedProx (#818) * created new ineractive_api dir to hold pytorch fedprox mnist example corrected files changed to FedProxOptimizer and ran set_old_weights for new FedProx Pytorch example renamed FedProx notebook used mode.parrameters() to get pytorch model weights got weights using state_dict changed old wieghts to list (for serialization) and fixed README input wieghts before zero_grad [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: #506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * [Enhancement: 506] Add an example that uses the FedProx optimizer in the interative_api This duplicates the MedNIST_2D example in the interative api but changes it to use the FedProx optimizer. Fixes: securefederatedai#506 Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> * Update README.md Signed-off-by: Beverly Klemme <[email protected]> * addressed comments by psfoley: corrected words in the jupyter notebook metadata and added a link to the FedProx paper in the README. Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * [Bug: 768] FX CLI: Separate create, cert gen commands (#807) This change separates existing command "fx collaborator.py generate-cert-request" command into two commands. "fx collaborator create -n {NAME} -d {DATA_PATH: optional}". "fx collaborator generate-cert-request -n {NAME}". Fixes #768 Signed-off-by: Emmanuel Jillela <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Add new tutorial example to OpenFL interactive API (#812) * Add new tutorial example to OpenFL interactive API This adds a new tutorial example on distributing a linear regression task over OpenFL cluster The model is defined by scikit-learn which is able to run over both cpu (by default) and gpu. The dataset is 1-dimensional noisy data of sinusoid with pre-defined parameters. Fixes #798 Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Grant Baker <[email protected]> Signed-off-by: Yi CAO <[email protected]> * reduced requirements.txt in workspace Signed-off-by: Beverly Klemme <[email protected]> --------- Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Co-authored-by: Yi CAO <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow in /openfl-workspace/tf_cnn_histology (#776) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * build(deps): bump tensorflow (#777) Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Update GaNDLF repo location and test CI with master branch Signed-off-by: Patrick Foley <[email protected]> * Fix documentation links. Change path names and templates for CI Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix paths Signed-off-by: Patrick Foley <[email protected]> * Fix breaking tests Signed-off-by: Patrick Foley <[email protected]> * Add compatible onnx version to requirements.txt file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong csv file name Signed-off-by: Patrick Foley <[email protected]> * Fix wrong names in workflow file Signed-off-by: Patrick Foley <[email protected]> * Fix wrong data path Signed-off-by: Patrick Foley <[email protected]> * Fix lint in test_gandlf.py Signed-off-by: Patrick Foley <[email protected]> * Fix lint errors Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid (#841) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix GaNDLF documentation links (#842) * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation * Fixed GaNDLF rst issues. Add sphinxcontrib-mermaid Signed-off-by: Patrick Foley <[email protected]> * Fix links in GaNDLF Documentation Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fix incorrectly formatted link in docs (#839) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added licensing information to all experimental workspaces Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Made changes to LocalRuntime, and tested Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Rename collaborator_name variable to name in experimental/component/collaborator/collaborator.py file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump onnx in /openfl-workspace/gandlf_seg_test (#840) Bumps [onnx](https://github.com/onnx/onnx) from 1.12 to 1.13.0. - [Release notes](https://github.com/onnx/onnx/releases) - [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md) - [Commits](https://github.com/onnx/onnx/compare/v1.12.0...v1.13.0) --- updated-dependencies: - dependency-name: onnx dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Parth Mandaliya <[email protected]> * Selected collaborators functionality added, test workspace of the same is work-progress. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Selected collaborator testcases included with this commit. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Testcase private attributes workspace added with this commit. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Instead of remove private attributes from checkpoint, changed the value to string "Private attributes: Not Available." Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added / Modified types of funciton arguments, and their return types. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Aggregator shutdown related changes are done in this commit Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update testflow_subset_of_collaborators.py Updated subset of collaborators testcase Signed-off-by: Parth Mandaliya <[email protected]> * Modified subset of collaborators testcase. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add files via upload Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp (#773) * build(deps): bump tensorflow in /openfl-workspace/keras_nlp Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update RMSProp optimizer import Signed-off-by: Patrick Foley <[email protected]> * flake8 Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_mnist (#771) * build(deps): bump tensorflow in /openfl-workspace/keras_cnn_mnist Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * revert experimental Adam to legacy (#863) Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/keras_cnn_with_compression (#770) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> * Update Adam Optimizer import Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.9.3 to 2.11.1 in /openfl-tutorials/interactive_api/Flax_CNN_CIFAR (#775) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.9.3 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.9.3...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Fixed breaking backages Signed-off-by: Patrick Foley <[email protected]> * Add quiet flag back to pip install Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow-cpu from 2.8.4 to 2.11.1 in /openfl-workspace/keras_nlp_gramine_ready (#769) * build(deps): bump tensorflow-cpu Bumps [tensorflow-cpu](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow-cpu dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Use legacy RMSprop optimizer Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility color contrast fixes (#864) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> * Color contrast updates for accessibility Color contrast updates for accessibility - update generic `a` element - update color of links in the toggle-able read-the-docs panel - update the color of the text in search results - update the color of notes headers Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tweak link color so it’s not so aggressive (#865) Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * External Loop Functionality Added External Loop Functionality Added Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added following tutorials, and testcase: 1. Vertical FL 2. Vertical FL Two Party 3. Internal Loop Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * plan.yaml modified in 301 watermarking, and subset of collaborators. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifying temp.sh for 2 collaborators. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update aggregator.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * FQDN switched to localhost FQDN switched to localhost Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * While activating experimental OpenFL commands, install depedencies as well. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Install experimental requirements while activating experimental openfl commands. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update experimental-requirements.txt Added ray 2.2.0 to experimental-requirements.txt file Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update requirements_workflow_interface.txt Metaflow, and Ray dependencies are moved to openfl-workspace/experimental/experimental-requrirements.txt file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Delete openfl/experimental/utilities/utils.py, and removed torch depency from openfl/experimental/utilities/resources.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update requirements_workflow_interface.txt Updated experimental tutorials requirements.txt file Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updating 201 GPUs tutorial Updated Workflow_Interface_201_Exclusive_GPUs_with_Ray.ipynb for Aggregator, and Collaborator as stateful actor Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified 101, 102, 103, 301, 401 tutorials as per Aggregator, & Collaborator stateful collaborators Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Excluding 'experimental' directory from workspace templates Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update 101 tutorial data.yaml Updated 101 tutorials data.yaml to correct a mistake by in index parameter of private attribute function. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update resources.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified get_number_of_gpus function Modify get_number_of_gpus function Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Removed torch dependency from "get_number_of_gpus" 2. Added keras mnist example notebook 3. Added keras mnist workspace example 4. Removed openfl-workspace/experimental/experimental-requirements.txt 5. "fx aggregator start" command will install libraries from experimental tutorials requirements_workflow_interface.txt file 6. Added testcase_datastore_cli, testcase_reference and testcase_reference_with_include_exclude 7. Updated 101 torch mnist plan.yaml and data.yaml 8. Removed aggregator start flow thread from cli/aggregator.py 9. At last changes made in federated runtime Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /tests/github/interactive_api_director/experiments/tensorflow_mnist/envoy (#772) * build(deps): bump tensorflow Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update sd_requirements.txt * revert to legacy SGD and install tensorflow==2.11 for workflow Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * build(deps): bump tensorflow from 2.8.4 to 2.11.1 in /openfl-workspace/tf_2dunet (#774) * build(deps): bump tensorflow in /openfl-workspace/tf_2dunet Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.8.4 to 2.11.1. - [Release notes](https://github.com/tensorflow/tensorflow/releases) - [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md) - [Commits](https://github.com/tensorflow/tensorflow/compare/v2.8.4...v2.11.1) --- updated-dependencies: - dependency-name: tensorflow dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.txt to retrigger CI * Update requirements.txt --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kevin Ta <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Renamed Workflow_Interface_101_Keras_MNIST_with_GPU.ipynb to Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb 2. Updated documentation in Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb 3. protobuf bumped from 3.19.6 to 3.20.0 in setup.py 4. Incorporating review comments (WIP) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified 104 Keras MNIST tutorial Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This comming brings following changes: 1. Experimental Aggregator workflow testcases moved to test/github/experimental/workspace directory 2. Added dependecy related changes Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary logs Removing unnecessary logs Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update collaborator.py Removing unnecessary logs Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Avoided deep copying the context by removing private attributes from context before checkpoint Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Incoporated review comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added dependecies to each tutorial notebooks Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * pinned tensorboardX (#870) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * While taking aggregator step remove private attributes before taking checkpoint and set it back to ctx after checkpoint Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary deepcopies from aggregator.py file Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing unnecessary comments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update stream_redirect.py Removing unnecessary imports Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added color escape sequences to datastore cli localruntime and workspace testcases Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update requirements_privacy_meter.txt torch 1.13.1 requires torchvision 0.14.1 not 0.14.0 Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Workflow_Interface_VFL_Two_Party Added actor related changed to Workflow_Interface_VFL_Two_Party Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Files added required to test experimental aggregator based workflow by GitHUB actions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Workflow_Interface_104_Keras_MNIST_with_GPU.ipynb Removing GPU parameter from collaborator decorator. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update resources.py Updated get_number_of_gpus function Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updating import statement In 104 keras workspace utils.py updating import SummaryWriter statement from torch to tensorflow Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * With this commit path of template can be explicitely given, user does not have to choose from available example templates. Only applicable to experimental openfl. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues and typos Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolved flake8 issues, Added warning statements when data.yaml is not found or aggregator / collaborator private attributes are not found in data.yaml, Tested by running all tutorials and tests Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Moving the logs after data.yaml warning, and removing unnecessary blank lines. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding N-level arguments functionality Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update workflow_interface.rst Added section on how to save best and last model from Metaflow CLI Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving last few TODO items. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Updated logs to provide exact information to user. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following updates: 1. Remove unnecessary spaces from Workflow_Interface_102_Aggregator_Validation.ipynb 2. Upgraded protobuf to 3.20.3 in setup.py as per tensorboard requirements 3. Modified 101-torch-cnn-mnist workspace for n-level arguments 4. Added a federated runtime workspace for 102-aggregator-validation Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Added 501_pytorch_tinyimagenet_transfer_learning workspace for experimental workflow interface 2. Removed `feature_shape` from openfl/experimental/interface/cli/plan.py 3. Resolved flake8 errors in openfl-workspace/experimental directory Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update data.yaml Correcting a mistake train dataset assigned to test and vice versa. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update flow.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update flow.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Adding functionality to convert jupyter notebook to workspace (in progress) Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Write flow.py, collaborator_private_attrs.py and aggregator_private_attrs.py automatically Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Find aggregator, and collaborator arguments Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modifying arguments help Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed automatically generated Zone.Identifier file Added Approach2 notebook Modified creator.py Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * workspace_creator renamed to workspace_builder finding values without help of any tags Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Finished exporting data.yaml, plan.yaml. Workspace creator validated for 101 MNIST and 301 Watermarking with MNIST Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removing round number constant argument from 301 MNIST WATERMARKING Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * * Created get_collaborator_kwargs function in localruntime class which takes collaborator name as argument and returns name of private attributes function and arguments required to call it. * Changed workspace builder code used get_collaborator_kwargs function to build data.yaml file. Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified get_collaborator_args function Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Modified workspace_builder/build.py file to resolve a typo Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Removed unnecessary files from template_workspace directory Moved print_tree to after generating python script Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added documentation and inline comments for workspace_builder Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added libraries required by WorkspaceBuilder Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update Tensorflow, gRPC, Protobuf dependencies (#868) * Update Tensorflow to latest, finally update grpcio/protobuf Signed-off-by: Patrick Foley <[email protected]> * Lint issue fix and missing tf reference Signed-off-by: Patrick Foley <[email protected]> * pyzmq version fixed * fix taskrunner tests for windows Signed-off-by: Mansi Sharma <[email protected]> * fix taskrunner test syntax for windows Signed-off-by: Mansi Sharma <[email protected]> * adding user option to workspace pip install requirements for windows Signed-off-by: Mansi Sharma <[email protected]> * fix windows CI test Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing virtual env for windows github actions Signed-off-by: Mansi Sharma <[email protected]> * testing venv for windows Signed-off-by: Mansi Sharma <[email protected]> * test venv for windows * test venv for windows * Added new KerasSerializer. Fixed other Interactive API experiments * Update taskrunner.yml * Update taskrunner.yml * Update workspace.py * Update workspace.py * Update taskrunner.yml * Remove get_model import from global namespace so dependencies are not loaded into memory unnecessarily (breaking windows build) * Refactoring and cleaning up imports to support Windows install * Fixed logger import paths * Fix missing imports * Fix native import * Fix lint errors * Fix keras optimizer patch. Remove irrelevant unit test * Format logs in UTF-8 for windows * Update interactive-kvasir.yml * Consolidate github actions python versions to single file * Update python versions * Update python versions * Update python versions * Reduce # of DataLoader workers for Pytorch Kvasir CI test * Fix Windows encoding * Fix Windows encoding and limit rounds so Github Actions CI doesn't run out of memory Signed-off-by: Patrick Foley <[email protected]> * Fix windows encoding * Fix Windows encoding --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Add FL plan description to documentation (#872) * Add plan description to documentation Signed-off-by: Mansi Sharma <[email protected]> * fix indentation Signed-off-by: Mansi Sharma <[email protected]> * Apply suggestions from code review Co-authored-by: Patrick Foley <[email protected]> --------- Signed-off-by: Mansi Sharma <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolve Coverity Issues (#874) * Fix coverity issues * Resolve remaining coverity issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Migrate to Ubuntu 22.04 LTS release (supported through 2027) (#875) Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues in test/experimental directory Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Accessibility updates (#861) * Fix incorrectly formatted link in docs Signed-off-by: Francis Storr <[email protected]> * Font styling, color contrast, other accessibility updates This update: 1. Restores Roboto and Lato fonts for most body copy, leaving Intel One Mono for code samples. 2. Adds colors (in `colors.css`) 3. Adds a new `accessibility_overrides.css` file containing CSS that improves the accessibility of the documentation and, where possible, Read The Docs. These updates remediate numerous non-conforming WCAG 2.x Level AA bugs. The use of a separate file for this hopefully makes these changes easier to manage and less likely to be accessibility overwritten in the future. Closes #848 Signed-off-by: Francis Storr <[email protected]> --------- Signed-off-by: Francis Storr <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Testflow for verifying stdout redirection to Metaflow datastore (#758) * implemented ray.wait * reverted changes back after testing * adding datastore cli test case * removed unused variables * removed stderr validation * fixed lint suggestions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Resolving DCO issue fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update EdenPipeline in the documentation (#877) Signed-off-by: Amit Portnoy <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Fixed flake8 issues Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Added weighted_average aggregation function under openfl.experimental.interface.{keras,torch}.aggregation_funtions Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Tested all local-runtime & federated-runtime tutorials and testcases Signed-off-by: Parth Mandaliya <[email protected]> * Resolving merge conflicts in experimental local-runtme tutorials 101, 103, 201, & 301 fixing FedAvg in workflow interface tutorials to be compatible with latest numpy stable release (1.24.3) (#833) * fixing FedAvg averaging in order to be compatible with numpy v1.24+ Signed-off-by: kta-intel <[email protected]> * uncommenting installations for consistency with other tutorials Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarking tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * fixing 301_MNIST_Watermarki ng tutorial FedAvg Signed-off-by: kta-intel <[email protected]> * Switching to py38 kernel and clearing cell outputs Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: kta-intel <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> * WIP: CI Scans (#873) * Initial scans commit for bandit, hadolint, trivy Signed-off-by: Patrick Foley <[email protected]> * Address bandit scan results Signed-off-by: Patrick Foley <[email protected]> * Fix Trivy action Signed-off-by: Patrick Foley <[email protected]> * Fix linting Signed-off-by: Patrick Foley <[email protected]> * Add Coverity Badge Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> * Update Hadolint threshold to flag errors only Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> * Update ROADMAP.md (#878) Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. Renaming workspace_builder module to workspace_export. 2. WorkspaceBuilder class renamed to WorkspaceExport. 3. Added WorkspaceExport.export function for user which wraps generate_requirements, generate_plan_yaml, generate_data_yaml functions. 4. Merged WorkspaceExport module to `fx workspace create` command by adding new following arguments: - `--notebook` Path to jupyter notebook - `--template_workspace_dir` Output path for generated workspace 5. WorkspaceExport removed from 101, and 103 tutorials. 6. setup.py changed accordingly. Signed-off-by: Parth Mandaliya <[email protected]> * This commit brings following changes: 1. output_dir renamed to output_workspace also the user must pass this parameter (default=/tmp is now removed) or there will be an error. 2. template_workspace_path is not required anymore. 3. Added a new tutorials Workflow_Interface_1001_Workspace_Creation_from_JupyterNotebook.ipynb (copy of Workflow_Interface_301_MNIST_Watermarking.ipynb) explaining how to use WorkspaceExport. Signed-off-by: Parth Mandaliya <[email protected]> * In Workflow_Interface_1001 tutorial list of collaborator are changed to 5 from 2 Signed-off-by: Parth Mandaliya <[email protected]> * Uncommeted pip install libraries Signed-off-by: Parth Mandaliya <[email protected]> * Adding GPU for aggregator in 201, Global DP & Privacy Meter tutorials Signed-off-by: Parth Mandaliya <[email protected]> * Modifed documentation in 1001 tutorial Signed-off-by: Parth Mandaliya <[email protected]> * Modified documentation in 1001 tutorial Signed-off-by: Parth Mandaliya <[email protected]> * Fixing 'param' attribute not found issue. in 301, 1001 experimental tutrials and 301 mnist watermarking workspace. Sol. imagen library installs param depedency latest version and param package got an upgrde recently. Pinned param to 1.13.0 after installing imagen which fixed the issue. Signed-off-by: Parth Mandaliya <[email protected]> * VFL 2 party example Signed-off-by: Mansi Sharma <[email protected]> * participant names to lowercase and changes to instructions Signed-off-by: Mansi Sharma <[email protected]> * Resolving missing packages from watermarking examples that break python 3.9+. Fix linting, spelling. Remove experimental aggregation functions Signed-off-by: Patrick Foley <[email protected]> * Remove duplicate files in openfl.experimental Signed-off-by: Patrick Foley <[email protected]> * Trigger CI Signed-off-by: Patrick Foley <[email protected]> * Moving test to be conditional Signed-off-by: Patrick Foley <[email protected]> * Fix inconsistent use of enumerate in loops Signed-off-by: Patrick Foley <[email protected]> * Fixing tests Signed-off-by: Patrick Foley <[email protected]> * Accomodate conversion of names to lower case Signed-off-by: Patrick Foley <[email protected]> * Missing torchvision dependency in tests Signed-off-by: Patrick Foley <[email protected]> * Fix lint issues Signed-off-by: Patrick Foley <[email protected]> --------- Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Parth Mandaliya <[email protected]> Signed-off-by: Patrick Foley <[email protected]> Signed-off-by: Aleksander Kantak <[email protected]> Signed-off-by: He, Dan H <[email protected]> Signed-off-by: Fang, Xiaoran <[email protected]> Signed-off-by: Klemme, Beverly <[email protected]> Signed-off-by: Baker, Grant <[email protected]> Signed-off-by: ELizabeth Simon, Neethu <[email protected]> Signed-off-by: Jillela, Emmanuel <[email protected]> Signed-off-by: Beverly Klemme <[email protected]> Signed-off-by: Elizabeth Simon, Neethu <[email protected]> Signed-off-by: Emmanuel Jillela <[email protected]> Signed-off-by: Yi CAO <[email protected]> Signed-off-by: Francis Storr <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: kta-intel <[email protected]> Signed-off-by: Mansi Sharma <[email protected]> Signed-off-by: Amit Portnoy <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Sarthak Pati <[email protected]> Co-authored-by: Prashant Shah <[email protected]> Co-authored-by: akantak <[email protected]> Co-authored-by: wangleflex <[email protected]> Co-authored-by: He, Dan H <[email protected]> Co-authored-by: xiaoranf <[email protected]> Co-authored-by: Wang, Wenjie <[email protected]> Co-authored-by: Lei5 Chen <[email protected]> Co-authored-by: Beverly Klemme <[email protected]> Co-authored-by: Emmanuel Jillela <[email protected]> Co-authored-by: Yi CAO <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Francis Storr <[email protected]> Co-authored-by: Patrick Foley <[email protected]> Co-authored-by: Kevin Ta <[email protected]> Co-authored-by: Keerti Prakash Talwar <[email protected]> Co-authored-by: kta-intel <[email protected]> Co-authored-by: porteratzo <[email protected]> Co-authored-by: Parth Mandaliya <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Mansi Sharma <[email protected]> Co-authored-by: Amit Portnoy <[email protected]>

Update Tensorflow to latest, finally update grpcio/protobuf

1b63301

Signed-off-by: Patrick Foley <[email protected]>

psfoley force-pushed the tf-1.13 branch from 5e23e2f to 1b63301 Compare August 1, 2023 22:04

Lint issue fix and missing tf reference

f345338

Signed-off-by: Patrick Foley <[email protected]>

This was referenced Aug 3, 2023

Fixing the problematic version for protobuf #869

Closed

pinned tensorboardX #870

Merged

kta-intel reviewed Aug 10, 2023

View reviewed changes

setup.py Show resolved Hide resolved

kta-intel mentioned this pull request Aug 10, 2023

Update 'grpcio' to '1.53.0' #867

Closed

mansishr and others added 3 commits August 21, 2023 10:59

pyzmq version fixed

96d598f

fix taskrunner tests for windows

c01668d

Signed-off-by: Mansi Sharma <[email protected]>

fix taskrunner test syntax for windows

baff1a0

Signed-off-by: Mansi Sharma <[email protected]>

mansishr force-pushed the tf-1.13 branch from 309c07b to baff1a0 Compare August 21, 2023 23:32

Mansi Sharma and others added 19 commits August 22, 2023 09:56

adding user option to workspace pip install requirements for windows

a253d85

Signed-off-by: Mansi Sharma <[email protected]>

fix windows CI test

e94697a

Signed-off-by: Mansi Sharma <[email protected]>

testing virtual env for windows github actions

fba30ff

Signed-off-by: Mansi Sharma <[email protected]>

testing virtual env for windows github actions

680d464

Signed-off-by: Mansi Sharma <[email protected]>

testing virtual env for windows github actions

e887ba1

Signed-off-by: Mansi Sharma <[email protected]>

testing venv for windows

c46f78c

Signed-off-by: Mansi Sharma <[email protected]>

test venv for windows

e99ad55

test venv for windows

f62cecd

Added new KerasSerializer. Fixed other Interactive API experiments

ac3087b

Merge branch 'tf-1.13' of https://github.com/psfoley/openfl into tf-1.13

c705aaa

Update taskrunner.yml

1e45246

Update taskrunner.yml

027e9c2

Update workspace.py

2df3b51

Update workspace.py

45fe7f7

Update taskrunner.yml

1c52252

Remove get_model import from global namespace so dependencies are not…

dbae5a7

… loaded into memory unnecessarily (breaking windows build)

Refactoring and cleaning up imports to support Windows install

8786895

Fixed logger import paths

ffc4310

Fix missing imports

9a7f5e8

psfoley changed the title ~~Update Tensorflow, gRPC, Protobuf dependencies~~ WIP: Update Tensorflow, gRPC, Protobuf dependencies Aug 24, 2023

psfoley added 10 commits August 24, 2023 16:32

Update interactive-kvasir.yml

293ada9

Consolidate github actions python versions to single file

ae1b6e1

Update python versions

2ac5d55

Update python versions

84dd777

Update python versions

edd6b2f

Reduce # of DataLoader workers for Pytorch Kvasir CI test

5d4de2a

Fix Windows encoding

18fb737

Fix Windows encoding and limit rounds so Github Actions CI doesn't ru…

acfa1e1

…n out of memory Signed-off-by: Patrick Foley <[email protected]>

Fix windows encoding

bfae178

Fix Windows encoding

ae69055

psfoley changed the title ~~WIP: Update Tensorflow, gRPC, Protobuf dependencies~~ Update Tensorflow, gRPC, Protobuf dependencies Aug 25, 2023

psfoley merged commit 3f7a770 into securefederatedai:develop Aug 25, 2023
22 of 23 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Tensorflow, gRPC, Protobuf dependencies #868

Update Tensorflow, gRPC, Protobuf dependencies #868

psfoley commented Aug 1, 2023 •

edited

Loading

Update Tensorflow, gRPC, Protobuf dependencies #868

Update Tensorflow, gRPC, Protobuf dependencies #868

Conversation

psfoley commented Aug 1, 2023 • edited Loading

psfoley commented Aug 1, 2023 •

edited

Loading