Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: improve README #25

Merged
merged 10 commits into from
Jun 26, 2023
Merged

chore: improve README #25

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5550aac
Improve README
fortuna Jun 26, 2023
889b7c0
Tweak language
fortuna Jun 26, 2023
612d35d
Rearrenge
fortuna Jun 26, 2023
7cd44d7
Try align=center
fortuna Jun 26, 2023
1286992
Clean up text
fortuna Jun 26, 2023
506324d
Update README.md
fortuna Jun 26, 2023
de6fe06
Review comments
fortuna Jun 26, 2023
87c617d
Tweaks
fortuna Jun 26, 2023
d22d1d7
Add reference
fortuna Jun 26, 2023
bb9cb2a
Alignment
fortuna Jun 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,152 @@ All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.

# Cross-platform Development
jyyi1 marked this conversation as resolved.
Show resolved Hide resolved

## Building

In Go you can compile for other target operating system and architecture by specifying the `GOOS` and `GOARCH` environment variables and using the [`go build` command](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies). That only works if you are not using [Cgo](https://pkg.go.dev/cmd/cgo) to call C code.

<details>
<summary>Examples</summary>

MacOS example:
```
% GOOS=darwin go build -C x -o ./bin/ ./outline-connectivity
% file ./x/bin/outline-connectivity
./x/bin/outline-connectivity: Mach-O 64-bit executable x86_64
```

Linux example:
```
% GOOS=linux go build -C x -o ./bin/ ./outline-connectivity
% file ./x/bin/outline-connectivity
./x/bin/outline-connectivity: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=n0WfUGLum4Y6OpYxZYuz/lbtEdv_kvyUCd3V_qOqb/CC_6GAQqdy_ebeYTdn99/Tk_G3WpBWi8vxqmIlIuU, with debug_info, not stripped
```

Windows example:
```
% GOOS=windows go build -C x -o ./bin/ ./outline-connectivity
% file ./x/bin/outline-connectivity.exe
./x/bin/outline-connectivity.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
```
</details>

## Running Linux binaries

To run Linux binaries you can use a Linux container via [Podman](https://podman.io/).

### Set up podman
<details>
<summary>Instructions</summary>

[Install Podman](https://podman.io/docs/installation) (once). On macOS:
```sh
brew install podman
```

Create the podman service VM (once) with the [`podman machine init` command](https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html):
```sh
podman machine init
```

Start the VM with the [`podman machine start` command](https://docs.podman.io/en/latest/markdown/podman-machine-start.1.html), after every time it is stopped:
```sh
podman machine start
```

You can see the VM running with the [`podman machine list` command](https://docs.podman.io/en/latest/markdown/podman-machine-list.1.html):
```
% podman machine list
NAME VM TYPE CREATED LAST UP CPUS MEMORY DISK SIZE
podman-machine-default* qemu 3 minutes ago Currently running 1 2.147GB 107.4GB
```

When you are done with development, you can stop the machine with the [`podman machine stop` command](https://docs.podman.io/en/latest/markdown/podman-machine-stop.1.html):
```sh
podman machine stop
```
</details>

### Run

The easiest way is to run a binary is to use the [`go run` command](https://pkg.go.dev/cmd/go#hdr-Compile_and_run_Go_program) directly with the `-exec` flag and our convenience tool `run_on_podman.sh`:
```sh
GOOS=linux go run -C x -exec "$(pwd)/run_on_podman.sh" ./outline-connectivity
```

It also works with the [`go test` command](https://pkg.go.dev/cmd/go#hdr-Test_packages):
```sh
GOOS=linux go test -exec "$(pwd)/run_on_podman.sh" ./...
```

<details>
<summary>Details and direct invocation</summary>

The `run_on_podman.sh` script uses the [`podman run` command](https://docs.podman.io/en/latest/markdown/podman-run.1.html) and a minimal ["distroless" container image](https://github.com/GoogleContainerTools/distroless) to run the binary you want:
```sh
podman run --arch $(uname -m) --rm -it -v "${bin}":/outline/bin gcr.io/distroless/static-debian11 /outline/bin "$@"
```

You can also use `podman run` directly to run a pre-built binary:
```
% podman run --rm -it -v ./x/bin:/outline gcr.io/distroless/static-debian11 /outline/outline-connectivity
Usage of /outline/outline-connectivity:
-domain string
Domain name to resolve in the test (default "example.com.")
-key string
Outline access key
-proto string
Comma-separated list of the protocols to test. Muse be "tcp", "udp", or a combination of them (default "tcp,udp")
-resolver string
Comma-separated list of addresses of DNS resolver to use for the test (default "8.8.8.8,2001:4860:4860::8888")
-v Enable debug output
```

Flags explanation:
- `--rm`: Remove container (and pod if created) after exit
- `-i` (interactive): Keep STDIN open even if not attached
- `-t` (tty): Allocate a pseudo-TTY for container
- `-v` (volume): Bind mount a volume into the container. Volume source will be on the server machine, not the client
</details>

## Running Windows binaries

To run Windows binaries you can use [Wine](https://en.wikipedia.org/wiki/Wine_(software)) to emulate a Windows environment.
This is not the same as a real Windows environment, so make sure you test on actual Windows machines.

### Install Wine

<details>
<summary>Instructions</summary>

Follow the instructions at https://wiki.winehq.org/Download.

On macOS:
```
brew tap homebrew/cask-versions
brew install --cask --no-quarantine wine-stable
```

After installation, `wine64` should be on your `PATH`. Check with `wine64 --version`:
```
wine64 --version
```

</details>

### Run

You can pass `wine64` as the `-exec` parameter in the `go` calls.

To build:

```sh
GOOS=windows go run -C x -exec "wine64" ./outline-connectivity
```

For tests:
```sh
GOOS=windows go test -exec "wine64" ./...
```
220 changes: 56 additions & 164 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,173 +7,65 @@
[![Mattermost](https://badgen.net/badge/Mattermost/Outline%20Community/blue)](https://community.internetfreedomfestival.org/community/channels/outline-community)
[![Reddit](https://badgen.net/badge/Reddit/r%2Foutlinevpn/orange)](https://www.reddit.com/r/outlinevpn/)

This is the repository to hold the future Outline SDK as we develop it. The goal is to clean up and move reusable code from [outline-ss-server](https://github.com/Jigsaw-Code/outline-ss-server) and [outline-go-tun2socks](https://github.com/Jigsaw-Code/outline-go-tun2socks).
<p align="center">
<img src="https://github.com/Jigsaw-Code/outline-brand/blob/main/assets/powered_by_outline/color/logo.png?raw=true" width=400pt />
</p>

**WARNING: This code is not ready to be used by the public. There's no guarantee of stability.**
> **Warning**
> This code is not ready to be used by the public. There's no guarantee of stability.

Tentative roadmap:
The Outline SDK helps developers:
- Create tools to protect against network-level interference
- Add network-level interference protection to existing apps, such as content or communication apps

- Transport libraries
## Advantages

| Multi-Platform | Proven Technology | Composable |
|:-:|:-:|:-:|
| The Outline SDK can be used on Android, iOS, Windows, macOS or Linux. | The Outline Client and Server have been using the code in the SDK for years, helping millions of users in the harshest conditions. | The SDK interfaces were carefully designed to allow for composition and reuse, so you can craft your own transport. |

## Integration

The Outline SDK is written in Go. There are multiple ways to integrate the Outline SDK into your app:

- As a **Go library** ([reference](https://pkg.go.dev/github.com/Jigsaw-Code/outline-internal-sdk)), in a Go application (CLI or graphical app with frameworks like [Fyne.io](https://fyne.io/), [Wails](https://wails.io/), [Qt for Go](https://therecipe.github.io/qt/), or [Go Mobile app](https://pkg.go.dev/golang.org/x/mobile/app))
- As a native **mobile library**, using [`gomobile bind`](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile) to generate Java and Objective-C bindings for Android, iOS and macOS.
- As a **C library**, generated using the appropriate [Go build mode](https://pkg.go.dev/cmd/go#hdr-Build_modes).
- As a **side service**, built as a standalone Go binary that your main application talks to. Note that this is not possible on iOS, due to the limitation on starting sub-processes.

The Outline Client uses the mobile library approach on Android, iOS and macOS (based on Cordova) and the side service on Windows and Linux (based on Electron).


## Tentative Roadmap

The launch will have two milestones: Alpha and Beta. We are currently in pre-Alpha. Note that most of the code is not new. It's the code being used by the production Outline Client and Server. The SDK work is repackaging code from [outline-ss-server](https://github.com/Jigsaw-Code/outline-ss-server) and [outline-go-tun2socks](https://github.com/Jigsaw-Code/outline-go-tun2socks) in a way that is easier to reuse and extend.

### Alpha
The goal of the Alpha release is to make it available to potential developers early so they can provide feedback on the SDK and help shape the interfaces, processes and tools.

The code in this repository will move to https://github.com/Jigsaw-Code/outline-sdk and versions will be tagged.

Alpha tasks:

- Add Transport libraries
- [x] Generic transport client primitives (`StreamDialer`, `PacketListener` and Endpoints)
- [x] TCP and UDP client implementations
- [x] Shadowsocks client implementations
- [ ] Generic transport server primitives (TBD)
- [ ] TCP and UDP server implementations
- [ ] Shadowsocks server implementations
- [ ] Utility implementations (`ReplaceablePacketListener`, `TruncateDNSPacketListener`)

- Network libraries
- [x] Generic network primitives (TBD, something like a generic TUN device)
- [x] Implementation based on go-tun2socks

- VPN API
- [ ] VPN API for desktop (Linux, Windows, macOS)

# Cross-platform Development

## Building

In Go you can compile for other target operating system and architecture by specifying the `GOOS` and `GOARCH` environment variables and using the [`go build` command](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies). That only works if you are not using [Cgo](https://pkg.go.dev/cmd/cgo) to call C code.

<details>
<summary>Examples</summary>

MacOS example:
```
% GOOS=darwin go build -C x -o ./bin/ ./outline-connectivity
% file ./x/bin/outline-connectivity
./x/bin/outline-connectivity: Mach-O 64-bit executable x86_64
```

Linux example:
```
% GOOS=linux go build -C x -o ./bin/ ./outline-connectivity
% file ./x/bin/outline-connectivity
./x/bin/outline-connectivity: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=n0WfUGLum4Y6OpYxZYuz/lbtEdv_kvyUCd3V_qOqb/CC_6GAQqdy_ebeYTdn99/Tk_G3WpBWi8vxqmIlIuU, with debug_info, not stripped
```

Windows example:
```
% GOOS=windows go build -C x -o ./bin/ ./outline-connectivity
% file ./x/bin/outline-connectivity.exe
./x/bin/outline-connectivity.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
```
</details>

## Running Linux binaries

To run Linux binaries you can use a Linux container via [Podman](https://podman.io/).

### Set up podman
<details>
<summary>Instructions</summary>

[Install Podman](https://podman.io/docs/installation) (once). On macOS:
```sh
brew install podman
```

Create the podman service VM (once) with the [`podman machine init` command](https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html):
```sh
podman machine init
```

Start the VM with the [`podman machine start` command](https://docs.podman.io/en/latest/markdown/podman-machine-start.1.html), after every time it is stopped:
```sh
podman machine start
```

You can see the VM running with the [`podman machine list` command](https://docs.podman.io/en/latest/markdown/podman-machine-list.1.html):
```
% podman machine list
NAME VM TYPE CREATED LAST UP CPUS MEMORY DISK SIZE
podman-machine-default* qemu 3 minutes ago Currently running 1 2.147GB 107.4GB
```

When you are done with development, you can stop the machine with the [`podman machine stop` command](https://docs.podman.io/en/latest/markdown/podman-machine-stop.1.html):
```sh
podman machine stop
```
</details>

### Run

The easiest way is to run a binary is to use the [`go run` command](https://pkg.go.dev/cmd/go#hdr-Compile_and_run_Go_program) directly with the `-exec` flag and our convenience tool `run_on_podman.sh`:
```sh
GOOS=linux go run -C x -exec "$(pwd)/run_on_podman.sh" ./outline-connectivity
```

It also works with the [`go test` command](https://pkg.go.dev/cmd/go#hdr-Test_packages):
```sh
GOOS=linux go test -exec "$(pwd)/run_on_podman.sh" ./...
```

<details>
<summary>Details and direct invocation</summary>

The `run_on_podman.sh` script uses the [`podman run` command](https://docs.podman.io/en/latest/markdown/podman-run.1.html) and a minimal ["distroless" container image](https://github.com/GoogleContainerTools/distroless) to run the binary you want:
```sh
podman run --arch $(uname -m) --rm -it -v "${bin}":/outline/bin gcr.io/distroless/static-debian11 /outline/bin "$@"
```

You can also use `podman run` directly to run a pre-built binary:
```
% podman run --rm -it -v ./x/bin:/outline gcr.io/distroless/static-debian11 /outline/outline-connectivity
Usage of /outline/outline-connectivity:
-domain string
Domain name to resolve in the test (default "example.com.")
-key string
Outline access key
-proto string
Comma-separated list of the protocols to test. Muse be "tcp", "udp", or a combination of them (default "tcp,udp")
-resolver string
Comma-separated list of addresses of DNS resolver to use for the test (default "8.8.8.8,2001:4860:4860::8888")
-v Enable debug output
```

Flags explanation:
- `--rm`: Remove container (and pod if created) after exit
- `-i` (interactive): Keep STDIN open even if not attached
- `-t` (tty): Allocate a pseudo-TTY for container
- `-v` (volume): Bind mount a volume into the container. Volume source will be on the server machine, not the client
</details>

## Running Windows binaries

To run Windows binaries you can use [Wine](https://en.wikipedia.org/wiki/Wine_(software)) to emulate a Windows environment.
This is not the same as a real Windows environment, so make sure you test on actual Windows machines.

### Install Wine

<details>
<summary>Instructions</summary>

Follow the instructions at https://wiki.winehq.org/Download.

On macOS:
```
brew tap homebrew/cask-versions
brew install --cask --no-quarantine wine-stable
```

After installation, `wine64` should be on your `PATH`. Check with `wine64 --version`:
```
wine64 --version
```

</details>

### Run

You can pass `wine64` as the `-exec` parameter in the `go` calls.

To build:

```sh
GOOS=windows go run -C x -exec "wine64" ./outline-connectivity
```

For tests:
```sh
GOOS=windows go test -exec "wine64" ./...
```

- Add Network libraries
- [x] IP Device abstraction
- [x] Implementation based on go-tun2socks (LWIP)
- [ ] UDP handler to fallback to DNS-over-TCP

### Beta

The goal of the Beta release is to communicate that the SDK is ready for broader consumption, after we believe the APIs are stable enough and we have all the supporting resources in place (website, documentation, examples, etc

Beta tasks:

- Add Resources
- [ ] Website
- [ ] Bindings
- [ ] Integration documentation
- [ ] Example command-line apps
- [ ] Example graphical apps