-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
5 changed files
with
194 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,40 +1,92 @@ | ||
# Nillion Federated Learning | ||
|
||
This repository contains the PoC for running Federated Learning on top of Nillion. It consists of two pieces of code, a Server and a Client. | ||
This repository contains the Proof of Concept (PoC) for running Federated Learning on top of Nillion. It consists of two main components: a Server and a Client. | ||
|
||
## Table of Contents | ||
|
||
1. [Introduction](#introduction) | ||
2. [Prerequisites](#prerequisites) | ||
3. [Setup](#setup) | ||
4. [Running the System](#running-the-system) | ||
5. [Regenerating gRPC](#regenerating-grpc) | ||
6. [Project Structure](#project-structure) | ||
7. [License](#license) | ||
|
||
## Introduction | ||
|
||
Federated Learning is a machine learning technique that trains algorithms across multiple decentralized edge devices or servers holding local data samples, without exchanging them. This project demonstrates how to implement Federated Learning using Nillion's secure infrastructure. | ||
|
||
## Prerequisites | ||
|
||
- Python 3.10 or higher | ||
- pip | ||
- poetry | ||
|
||
## Setup | ||
|
||
1. Install poetry: | ||
|
||
```bash | ||
pip install poetry | ||
``` | ||
|
||
2. Install project dependencies with examples extras to run the examples: | ||
|
||
```bash | ||
poetry install -E examples | ||
``` | ||
|
||
On a separate terminal: | ||
## Running the System | ||
|
||
To run the Federated Learning system, you need to start the Nillion devnet, the server, and at least two clients. | ||
|
||
1. Start the Nillion devnet: | ||
|
||
```bash | ||
# Terminal 0 | ||
nillion-devnet | ||
``` | ||
|
||
2. Start the server: | ||
|
||
```bash | ||
# Terminal 1 | ||
poetry run lr_server | ||
poetry run python3 examples/logistic_regression/server.py | ||
``` | ||
|
||
3. Start the first client: | ||
|
||
```bash | ||
# Terminal 2 | ||
poetry run lr_client 0 | ||
poetry run python3 examples/logistic_regression/client.py 0 | ||
``` | ||
|
||
4. Start the second client: | ||
|
||
```bash | ||
# Terminal 3 | ||
poetry run lr_client 1 | ||
poetry run python3 examples/logistic_regression/client.py 1 | ||
``` | ||
|
||
# Regenerating gRPC | ||
You can start additional clients by incrementing the client number on the `server.py` file. | ||
|
||
``` | ||
## Regenerating gRPC | ||
|
||
If you make changes to the `fl_service.proto` file, you need to regenerate the gRPC code: | ||
|
||
```bash | ||
poetry run python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. nillion_fl/network/fl_service.proto | ||
``` | ||
``` | ||
|
||
## Project Structure | ||
|
||
- `examples/`: Includes various example implementations (convolutional networks, logistic regression, neural networks). | ||
- `nillion_fl/`: The main package for Nillion Federated Learning (contains base client, and server implementations) | ||
- `core/`: Core components for client and server implementations. | ||
- `network/`: Contains protocol buffer definitions for network communication. | ||
- `nilvm/`: Nillion Network related components, including federated averaging implementation. | ||
- `pytorch/`: PyTorch-specific implementations for client and server. | ||
|
||
## License | ||
|
||
This project is licensed under the Apache2 License. See the LICENSE file for details. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Convolutional Neural Network Example - Nillion Federated Learning | ||
|
||
This example demonstrates how to run a convolutional neural network model using Nillion Federated Learning for CIFAR 10 dataset. | ||
|
||
## Running the Convolutional Neural Network Example | ||
|
||
1. Start the Nillion devnet: | ||
```bash | ||
# Terminal 0 | ||
nillion-devnet | ||
``` | ||
|
||
2. Start the server: | ||
```bash | ||
# Terminal 1 | ||
poetry run python3 examples/conv_net/server.py | ||
``` | ||
|
||
3. Start the first client: | ||
```bash | ||
# Terminal 2 | ||
poetry run python3 examples/conv_net/client.py 0 | ||
``` | ||
|
||
4. Start the second client: | ||
```bash | ||
# Terminal 3 | ||
poetry run python3 examples/conv_net/client.py 1 | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Logistic Regression Example - Nillion Federated Learning | ||
|
||
This example demonstrates how to run a logistic regression model using Nillion Federated Learning. | ||
|
||
## Running the Logistic Regression Example | ||
|
||
1. Start the Nillion devnet: | ||
```bash | ||
# Terminal 0 | ||
nillion-devnet | ||
``` | ||
|
||
2. Start the server: | ||
```bash | ||
# Terminal 1 | ||
poetry run python3 examples/logistic_regression/server.py | ||
``` | ||
|
||
3. Start the first client: | ||
```bash | ||
# Terminal 2 | ||
poetry run python3 examples/logistic_regression/client.py 0 | ||
``` | ||
|
||
4. Start the second client: | ||
```bash | ||
# Terminal 3 | ||
poetry run python3 examples/logistic_regression/client.py 1 | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Neural Network Example - Nillion Federated Learning | ||
|
||
This example demonstrates how to run a neural network model using Nillion Federated Learning. | ||
|
||
## Running the Neural Network Example | ||
|
||
1. Start the Nillion devnet: | ||
```bash | ||
# Terminal 0 | ||
nillion-devnet | ||
``` | ||
|
||
2. Start the server: | ||
```bash | ||
# Terminal 1 | ||
poetry run python3 examples/neural_net/server.py | ||
``` | ||
|
||
3. Start the first client: | ||
```bash | ||
# Terminal 2 | ||
poetry run python3 examples/neural_net/client.py 0 | ||
``` | ||
|
||
4. Start the second client: | ||
```bash | ||
# Terminal 3 | ||
poetry run python3 examples/neural_net/client.py 1 | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
# Neural Network Example with 5 parties - Nillion Federated Learning | ||
|
||
This example demonstrates how to run a neural network model using Nillion Federated Learning. | ||
|
||
## Running the 5-party Neural Network Example | ||
|
||
1. Start the Nillion devnet: | ||
```bash | ||
# Terminal 0 | ||
nillion-devnet | ||
``` | ||
|
||
2. Start the server: | ||
```bash | ||
# Terminal 1 | ||
poetry run python3 examples/neural_net_5/server.py | ||
``` | ||
|
||
3. Start the first client: | ||
```bash | ||
# Terminal 2 | ||
poetry run python3 examples/neural_net_5/client.py 0 | ||
``` | ||
|
||
4. Start the second client: | ||
```bash | ||
# Terminal 3 | ||
poetry run python3 examples/neural_net_5/client.py 1 | ||
``` | ||
|
||
5. Start the first client: | ||
```bash | ||
# Terminal 4 | ||
poetry run python3 examples/neural_net_5/client.py 2 | ||
``` | ||
|
||
6. Start the second client: | ||
```bash | ||
# Terminal 5 | ||
poetry run python3 examples/neural_net_5/client.py 3 | ||
``` | ||
|
||
7. Start the first client: | ||
```bash | ||
# Terminal 6 | ||
poetry run python3 examples/neural_net_5/client.py 4 | ||
``` |