Skip to content

Commit

Permalink
Rewrite guide.
Browse files Browse the repository at this point in the history
  • Loading branch information
danielballan committed Feb 27, 2024
1 parent 48b348b commit 0b69b42
Show file tree
Hide file tree
Showing 3 changed files with 100 additions and 143 deletions.
95 changes: 95 additions & 0 deletions docs/source/how-to/register.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Register Content in Tiled

The usage `tiled serve directory ...` is mostly for demos and small-scale use.
For more sophisticated control over this process, see

## Quickstart

The tutorial {doc}`../how-to/register` demonstrates the usage:

```
tiled serve directory [--watch] [--public] [--api-key <SECRET>] <DIRECTORY>
```

which is a shorthand for:

1. Walk a directory tree, identifying formats it recognizes, ingesting their
metadata, structure, and filepaths into a database for efficient search and
random access.
2. Start a server that uses that data.
3. Optionally, watch the directory tree for changes, and synchronizing them to
the data.

### Limitations of `tiled serve directory ...`

The shorthand is great for quickly getting started, but it has numerous
limitations.

- Tiled walks the entire directory at server startup. This can be slow.
- Tiled creates an ephemeral database (SQLite in a temporary directory)
just for this process. That work is discarded when the server shuts down.
- One database per server not horizontally scalable.
- With `--watch`, Tiled picks up files as soon as they are created, and
they may not be ready to be read yet. (Example: a partially-written HDF5
file.)
- With `--watch`, Tiled currently re-scans the entire directory from scratch
every time anything changes. This may be improved in the future, but there
are limitations to how smooth this can be.
- This can place a lot of load on a filesystem, which can be an issue for
networked file systems in particular.

When these limitations are reached, read on for a more sophisticated approach.

## Production-Scale Approach

Start a Tiled server.

```
tiled serve catalog <DATABASE_URI> -r <DIRECTORY> [--public] [--api-key <SECRET>]
```

- The `<DATABASE_URI>` may be a SQLite file like `catalog.db` or a PostgreSQL
URI like `postgres+asyncpg://<USERNAME>:<PASSWORD>:<HOST>/<DATABASE>`.
- The `<DIRECTORY>` instructs Tiled to enable an authorized clients to register
files in that directory to be served. For security reasons, nothing outside
of that directory will be possible to register. (Multiple `-r` arguments may
be used.)
- If an `--api-key` is not passed, a secure random key will be generated and
printed at server startup.

### Simple cases

As in the Quickstart, this walks the directory tree, identifies recognized
formats, and registers the metadata, structure, and filepaths.

```
tiled register http://localhost:8000 [--api-key <SECRET>] <DIRECTORY>
```

### Complex cases

Sometimes it is necessary to take more manual control of this registration
process. Using the Python client,

```py
from tiled.client import from_uri
from tiled.client.data_source import Asset, DataSource
from tiled.structures.core import StructureFamily

# You can pass the api_key in explicitly as shown here, but for security, it
# is best to set the API key as TILED_API_KEY, which from_uri(...) will
# automatically detect and use.
client = from_uri("http://localhost:8000", api_key="...")

# POST /api/v1/register/{path}
client.new(
structure_family=StructureFamily.array,
data_sources=DataSource(
assets=[Asset(data_uri="file:///...", num=1), Asset(data_uri="file:///...", num=2)],
mimetype="multipart/related;type=image/tiff",
structure_family=StructureFamily.array,
),
metadata={},
specs=[],
)
```
141 changes: 0 additions & 141 deletions docs/source/how-to/tiled-register.md

This file was deleted.

7 changes: 5 additions & 2 deletions docs/source/tutorials/serving-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ In a Python interpreter, connect with the Python client.
```python
from tiled.client import from_uri

tree = from_uri("http://localhost:8000")
client = from_uri("http://localhost:8000")
```

The ``tree`` has the same tree structure as the directory on
The ``client`` has the same tree structure as the directory on
disk, and we can slice and access the data.

```python
Expand Down Expand Up @@ -85,3 +85,6 @@ array([[1., 1., 1., ..., 1., 1., 1.],
1 2 5
2 3 6
```

The usage `tiled serve directory ...` is mostly for demos and small-scale use.
For more sophisticated control over this process, see {doc}`../how-to/register`.

0 comments on commit 0b69b42

Please sign in to comment.