forked from bluesky/tiled
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
48b348b
commit 0b69b42
Showing
3 changed files
with
100 additions
and
143 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
# Register Content in Tiled | ||
|
||
The usage `tiled serve directory ...` is mostly for demos and small-scale use. | ||
For more sophisticated control over this process, see | ||
|
||
## Quickstart | ||
|
||
The tutorial {doc}`../how-to/register` demonstrates the usage: | ||
|
||
``` | ||
tiled serve directory [--watch] [--public] [--api-key <SECRET>] <DIRECTORY> | ||
``` | ||
|
||
which is a shorthand for: | ||
|
||
1. Walk a directory tree, identifying formats it recognizes, ingesting their | ||
metadata, structure, and filepaths into a database for efficient search and | ||
random access. | ||
2. Start a server that uses that data. | ||
3. Optionally, watch the directory tree for changes, and synchronizing them to | ||
the data. | ||
|
||
### Limitations of `tiled serve directory ...` | ||
|
||
The shorthand is great for quickly getting started, but it has numerous | ||
limitations. | ||
|
||
- Tiled walks the entire directory at server startup. This can be slow. | ||
- Tiled creates an ephemeral database (SQLite in a temporary directory) | ||
just for this process. That work is discarded when the server shuts down. | ||
- One database per server not horizontally scalable. | ||
- With `--watch`, Tiled picks up files as soon as they are created, and | ||
they may not be ready to be read yet. (Example: a partially-written HDF5 | ||
file.) | ||
- With `--watch`, Tiled currently re-scans the entire directory from scratch | ||
every time anything changes. This may be improved in the future, but there | ||
are limitations to how smooth this can be. | ||
- This can place a lot of load on a filesystem, which can be an issue for | ||
networked file systems in particular. | ||
|
||
When these limitations are reached, read on for a more sophisticated approach. | ||
|
||
## Production-Scale Approach | ||
|
||
Start a Tiled server. | ||
|
||
``` | ||
tiled serve catalog <DATABASE_URI> -r <DIRECTORY> [--public] [--api-key <SECRET>] | ||
``` | ||
|
||
- The `<DATABASE_URI>` may be a SQLite file like `catalog.db` or a PostgreSQL | ||
URI like `postgres+asyncpg://<USERNAME>:<PASSWORD>:<HOST>/<DATABASE>`. | ||
- The `<DIRECTORY>` instructs Tiled to enable an authorized clients to register | ||
files in that directory to be served. For security reasons, nothing outside | ||
of that directory will be possible to register. (Multiple `-r` arguments may | ||
be used.) | ||
- If an `--api-key` is not passed, a secure random key will be generated and | ||
printed at server startup. | ||
|
||
### Simple cases | ||
|
||
As in the Quickstart, this walks the directory tree, identifies recognized | ||
formats, and registers the metadata, structure, and filepaths. | ||
|
||
``` | ||
tiled register http://localhost:8000 [--api-key <SECRET>] <DIRECTORY> | ||
``` | ||
|
||
### Complex cases | ||
|
||
Sometimes it is necessary to take more manual control of this registration | ||
process. Using the Python client, | ||
|
||
```py | ||
from tiled.client import from_uri | ||
from tiled.client.data_source import Asset, DataSource | ||
from tiled.structures.core import StructureFamily | ||
|
||
# You can pass the api_key in explicitly as shown here, but for security, it | ||
# is best to set the API key as TILED_API_KEY, which from_uri(...) will | ||
# automatically detect and use. | ||
client = from_uri("http://localhost:8000", api_key="...") | ||
|
||
# POST /api/v1/register/{path} | ||
client.new( | ||
structure_family=StructureFamily.array, | ||
data_sources=DataSource( | ||
assets=[Asset(data_uri="file:///...", num=1), Asset(data_uri="file:///...", num=2)], | ||
mimetype="multipart/related;type=image/tiff", | ||
structure_family=StructureFamily.array, | ||
), | ||
metadata={}, | ||
specs=[], | ||
) | ||
``` |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters