Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zarr v2 endpoints to Tiled #774

Open
wants to merge 51 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
ac1e632
add zarr route
genematx Jul 15, 2024
dab92e3
ENH: basic zarr functionality
genematx Aug 5, 2024
b7e39a6
ENH: map tiled chunks to zarr blocks
genematx Aug 6, 2024
784f391
MNT: Clean-up comments
genematx Aug 6, 2024
08f255d
ENH: support tables
genematx Aug 7, 2024
d10e2f1
add zarr route
genematx Jul 15, 2024
5b9e786
ENH: basic zarr functionality
genematx Aug 5, 2024
198e24b
ENH: map tiled chunks to zarr blocks
genematx Aug 6, 2024
9264d8e
MNT: Clean-up comments
genematx Aug 6, 2024
2da31d3
ENH: support tables
genematx Aug 7, 2024
09d6808
ENH: Add data type to sparse
genematx Aug 9, 2024
2ed822f
resolve conflict
genematx Aug 12, 2024
35cf1a8
ENH: support units for numpy datetime types
genematx Sep 5, 2024
a31bd94
MNT: removed unnecessary imports
genematx Sep 5, 2024
b36c6b4
ENH: add default value for units
genematx Sep 5, 2024
260e83e
ENH: update BuiltinDtype in pydantic
genematx Sep 5, 2024
0631288
ENH: update BuiltinDtype in pydantic
genematx Sep 5, 2024
4ffa2bf
ENH: add default value for units
genematx Sep 5, 2024
917319d
TST: datetime dtypes in test_array
genematx Sep 5, 2024
91f0f98
MNT: Update changelog
genematx Sep 5, 2024
b6bef55
resolve conflict
genematx Aug 12, 2024
00d0f5f
Clean-up
genematx Sep 5, 2024
ee0d2cc
MNT: gitignore alembic.ini
genematx Sep 5, 2024
24faf6f
Merge branch 'main' into add-zarr
genematx Sep 6, 2024
e68e0e5
FIX: typo in comment
genematx Sep 6, 2024
99cd7fc
ENH: Add data type to sparse
genematx Aug 9, 2024
1fbb021
FIX: assignment error
genematx Sep 6, 2024
1ec2425
MNT: clean and lint
genematx Sep 6, 2024
808ebcb
MNT: update changelog
genematx Sep 6, 2024
2e06e9f
Merge branch 'sparse-dtype' into add-zarr
genematx Sep 6, 2024
88262c3
Merge branch 'main' into datetime-units
genematx Sep 6, 2024
66eabcf
MNT: fix changelog
genematx Sep 6, 2024
57137b1
FIX: tests with COOStructure
genematx Sep 6, 2024
a435b71
Merge branch 'sparse-dtype' into add-zarr
genematx Sep 6, 2024
7e3c59a
Merge branch 'datetime-units' into add-zarr
genematx Sep 6, 2024
527dad6
FIX: typing
genematx Sep 6, 2024
4801407
BLD: add aiohttp package to server requirements
genematx Sep 6, 2024
8a7c510
MNT: clean and lint
genematx Sep 6, 2024
484c6a0
FIX: default value of units to empty string.
genematx Sep 10, 2024
c9fa03e
FIX: use None as the sentinel for the units kwarg
genematx Sep 11, 2024
045969f
ENH: use np.datetime_data to extract units
genematx Sep 11, 2024
a742bd9
TST: Fix failing authorization test -- empty password
genematx Sep 13, 2024
73b2b40
MNT: format and lint
genematx Sep 13, 2024
15a2494
Merge branch 'datetime-units' into add-zarr
genematx Sep 13, 2024
e2e315f
Merge branch 'main' into add-zarr
genematx Oct 22, 2024
24c86d5
MNT: remove deprecated PatchedStreamingResponse
genematx Oct 22, 2024
4ace292
MNT: lint
genematx Oct 22, 2024
9fd4d35
TST: add authentication tests
genematx Oct 22, 2024
8335136
FIX: ensure support for py3.8
genematx Oct 22, 2024
fc2a4ce
MNT: lint
genematx Oct 22, 2024
9c5388b
MNT: add changelog entry
genematx Oct 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ config.yml
prometheus_data
grafana_data
data
alembic.ini

tiled/_version.py

Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@ Write the date in place of the "Unreleased" in the case a new version is release

# Changelog

## Unreleased

### Added

- zarr v2 endpoints for read access

## v0.1.0b10 (2024-10-11)

- Add kwarg to client logout to auto-clear default identity.
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ tiled = "tiled.commandline.main:main"

# This is the union of all optional dependencies.
all = [
"aiohttp",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot find where this is used. Maybe this was needed in some transient state of the PR but no longer is needed.

$ git grep aiohttp
pyproject.toml:    "aiohttp",
pyproject.toml:    "aiohttp",
pyproject.toml:    "aiohttp",

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreover, since we use starlette for the server and httpx for the client, it would be somewhat odd and redundant to use aiohttp as well.

"aiofiles",
"aiosqlite",
"alembic",
Expand Down Expand Up @@ -190,6 +191,7 @@ minimal-client = [
]
# These are the requirements needed for basic server functionality.
minimal-server = [
"aiohttp",
"aiofiles",
"aiosqlite",
"alembic",
Expand Down Expand Up @@ -225,6 +227,7 @@ minimal-server = [
]
# This is the "kichen sink" fully-featured server dependency set.
server = [
"aiohttp",
"aiofiles",
"aiosqlite",
"alembic",
Expand Down
8 changes: 6 additions & 2 deletions tiled/_tests/test_protocols.py
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,13 @@ def test_sparseadapter_protocol(mocker: MockFixture) -> None:
mock_call4 = mocker.patch.object(CustomSparseAdapter, "specs")
mock_call5 = mocker.patch.object(CustomSparseAdapter, "metadata")

structure = COOStructure(shape=(2 * 5,), chunks=((5, 5),))

array = numpy.random.rand(2, 512, 512)

structure = COOStructure(
shape=(2 * 5,),
chunks=((5, 5),),
data_type=BuiltinDtype.from_numpy_dtype(array.dtype),
)
blocks: Dict[Tuple[int, ...], Tuple[NDArray[Any], Any]] = {(1,): (array, (1,))}
metadata: JSON = {"foo": "bar"}
anyslice = (1, 1, 1)
Expand Down
9 changes: 8 additions & 1 deletion tiled/_tests/test_writing.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
from ..mimetypes import PARQUET_MIMETYPE
from ..queries import Key
from ..server.app import build_app
from ..structures.array import BuiltinDtype
from ..structures.core import Spec, StructureFamily
from ..structures.data_source import DataSource
from ..structures.sparse import COOStructure
Expand Down Expand Up @@ -245,7 +246,13 @@ def test_write_sparse_chunked(tree):
"sparse",
[
DataSource(
structure=COOStructure(shape=(2 * N,), chunks=((N, N),)),
structure=COOStructure(
shape=(2 * N,),
chunks=((N, N),),
data_type=BuiltinDtype.from_numpy_dtype(
numpy.dtype("float64")
),
),
structure_family="sparse",
)
],
Expand Down
Loading
Loading