Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend dunder methods and code simplification for annotations #251

Merged
merged 19 commits into from
Aug 13, 2024

Conversation

BPdeRooij
Copy link
Contributor

This PR implements some new features, refactors the annotations code and aims to simplify it. Most importantly are the addition of AnnotatedGeometry and new dunder methods in WsiAnnotations. It is an initial idea and implementation for features, which could be useful. It would be great to further discuss these ideas and get feedback.

Changes

  • Added class AnnotatedGeometry to inheret from for Point and Polgyon classes. This simplifies code for the Point and Polygon classes. This also allows us to extend to new annotated geometries for MultiPolygon or MultiPoint with our own annotation classes.
  • Added __iadd__ and __isub__ methods for AnnotatedGeometry that raises a TypeError.
  • Modified __new__ function to be in line with shapely. This allows polygons (with holes) to be created directly as a dlup Polygon.
  • Combined annotation_type lookup into a single enum for ASAP, V7 and GeoJSON.
  • Added annotation_type to DarwinV7Metadata lookup for colors. This is useful when a class for keypoints and polygons have the same label, but different colors. In that case annotation_type is needed to retrieve the correct color.
  • Simplified _is_rectangle and added _is_alligned_rectangle for determining unrotated bounding boxes.
  • Use of Shapely affinity transformation instead of transform
  • Added a utils file with utility functions for annotations. Perhaps, DarwinV7Metadata and _get_v7_metadata can also be moved here.
  • Export z-index into geometry properties class. (Still being discussed here: z-index is not imported or exported with GeoJSONs #240)
  • Slightly improved type hinting.
  • Incorporated _ComplexDarwinPolygonWrapper directly into _parse_darwin_complex_polygon.
  • Fixed test_point_pickling in test_annotations.py by pickling dlup point instead of ShapelyPoint

Changes made to WsiAnnotations:

  • Changed _available_labels to use set instead of list for efficiency
  • Added sorting layers in place to WsiAnnotations
  • Directly use shapely.affinity.affine_transform in the read_region function.
  • Extended WsiAnnotations dunder methods:
    • Added __len__ as length of _layers
    • Extend __contains__ to check if a Point or Polygon is in layers
    • Implemented __add__, __iadd__ and __radd__ for addition, in-place addition and right side addition.
    • Added templates for methods __sub__, __isub__ and __rsub__
  • Added tests for dunder methods.

Further changes and comments:

  • Define behavior for __radd__ of lists of Polygons and Points
  • Add same functionalities for subtracting from WsiAnnotations.
  • Write more tests for (new) dunder methods
  • Move DarwinV7Metadata and _get_v7_metadata to utils file. This is not possible now because AnnotationType is added to the metadata.
  • Move other helper function and classes (such as _geometry_to_geojson, GeoJsonDict, CoordinatesDict, _is_rectangle, _is_allagined_rectangle) to util file to simplify code.
  • The WsiAnotations.copy function uses deepcopy. The test and behavior for this method are not completely in line with the shallow and deep copy operations defined in the standard python library. Another __copy__ and __deepcopy__ method could be added.

Copy link

codecov bot commented Aug 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.63%. Comparing base (548fc87) to head (faa019e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #251      +/-   ##
==========================================
+ Coverage   98.53%   98.63%   +0.09%     
==========================================
  Files          14       14              
  Lines        1091     1169      +78     
==========================================
+ Hits         1075     1153      +78     
  Misses         16       16              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@BPdeRooij BPdeRooij force-pushed the feature/extend-dunder-methods branch from 06bb573 to d1ecb26 Compare August 13, 2024 11:32
@jonasteuwen jonasteuwen self-requested a review August 13, 2024 20:10
Copy link
Contributor

@jonasteuwen jonasteuwen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work!

@jonasteuwen jonasteuwen merged commit 8d31221 into main Aug 13, 2024
9 checks passed
@jonasteuwen jonasteuwen deleted the feature/extend-dunder-methods branch August 13, 2024 22:16
jonasteuwen pushed a commit that referenced this pull request Aug 14, 2024
* Add z_index to geojson properties when importing and exporting
* Fix annotation crop for Polygons in read_region and test_annotations
* Created new AnnotatedGeometry, adhere to shapely dunder and functions
* Add helper function to utils and Enum for Darwin/ASAP annotation types
* Add darwin_annotation_type to color look up and move Darwin util function to utils
* Bugfix for WsiAnnotations.filter function
jonasteuwen pushed a commit that referenced this pull request Aug 17, 2024
* Add z_index to geojson properties when importing and exporting
* Fix annotation crop for Polygons in read_region and test_annotations
* Created new AnnotatedGeometry, adhere to shapely dunder and functions
* Add helper function to utils and Enum for Darwin/ASAP annotation types
* Add darwin_annotation_type to color look up and move Darwin util function to utils
* Bugfix for WsiAnnotations.filter function

Refactor into multiple modules

Support pickling and some other features

Extend dunder methods and code simplification for annotations (#251)

* Add z_index to geojson properties when importing and exporting
* Fix annotation crop for Polygons in read_region and test_annotations
* Created new AnnotatedGeometry, adhere to shapely dunder and functions
* Add helper function to utils and Enum for Darwin/ASAP annotation types
* Add darwin_annotation_type to color look up and move Darwin util function to utils
* Bugfix for WsiAnnotations.filter function

Removing files, improving test coverage and mypy
jonasteuwen added a commit that referenced this pull request Sep 26, 2024
* Updated workflows

* Adding C++ TIFF writer
- Bump version to 0.7
- Rebuild setup toolchain to pyproject.toml
- Remove CLI utilities and tests
- Rename dlup.types to dlup._types
- Reduce cyclic imports
- Remove color transform from internal_handler='pil' (working towards v1.0)
- Bugfixes

* Add ZSTD compression, reformat some files

* First commit with geometries

* First writing of Polygon

* Some additional changes to cast the Polygon.

* Commiting to not lose work

* Updated for initial version of WsiAnnotations

* updates

* Some updates during development

* Fix scaling issues

* Added functionality to remove polygons and points and to rebuild the RTree

* Implement complete initial version of geometry module

* We actually need to iterate over all layers, not just polys

* Implement scaling and offset functions

* Adding functionality to reindex polygons

* Created output container for results

* Sensible updates

* Add masking utility to AnnotationRegion

* Memory optimizations

* Adding sorting capabilities

* Add bounding box

* Added color LUT

* Refactor

* Extend dunder methods and code simplification for annotations (#251)

* Add z_index to geojson properties when importing and exporting
* Fix annotation crop for Polygons in read_region and test_annotations
* Created new AnnotatedGeometry, adhere to shapely dunder and functions
* Add helper function to utils and Enum for Darwin/ASAP annotation types
* Add darwin_annotation_type to color look up and move Darwin util function to utils
* Bugfix for WsiAnnotations.filter function

Refactor into multiple modules

Support pickling and some other features

Extend dunder methods and code simplification for annotations (#251)

* Add z_index to geojson properties when importing and exporting
* Fix annotation crop for Polygons in read_region and test_annotations
* Created new AnnotatedGeometry, adhere to shapely dunder and functions
* Add helper function to utils and Enum for Darwin/ASAP annotation types
* Add darwin_annotation_type to color look up and move Darwin util function to utils
* Bugfix for WsiAnnotations.filter function

Removing files, improving test coverage and mypy

* Fixing the scaling, mypy and adding tests

* minor changes in annotations

* Updating geometry module, and its tests.

* Rename modules, improve CI/CD

* Almost done with the geometry module
Strange bug remains.

* Disable a few tests, squash pylint errors

* Fixing some bugs when trees get invalidated
This was a problem when the insertation order
was not the same.

* Do an allclose rather than equal

* Weirdly enough different values on github

* Maybe a sudo install won't hurt

* Remove assert

* Restructure C++ code

* Restructure C++ code

* Refactored code.

* Refactoring

* Refactor code

* Rename functions, remove functions

* Adhere to google C++ style guide

* Maybe include stdexcept so that github understandS?

* Reducing a few compiler warnings

* Make sure mypy runs.

* Convenient helper function

* Implement XML export

* Add xsdata to requirements

* XML Schema reading now works

* Order can be 0...

* Several changes to support HaloXML

* Improve test coverage

* Improve test coverage to 90%

* Streamlining

* Trying to get better CI/CD coverage

* Fix bug in annotations_experimental.py

* Add more tests, and fix bug in process

* Updating tests, updating schema, add box

* Add more boxes functionality.

* Refactor factory methods to factory.h

* Refactor

* Code simplifications

* Some final memory improvements

* Adding boxes to the geometry module

* Improve CI/CD

* Adding deprecation warnings

* Streamline meson

* Attempt lazy initialization of the geometry collection in the region class

* Update tests for lazy evaluation

* Fixing memory error in region.h. Fix tox

* Improve workflows

* Improve workflows

* Update tox.ini

* Remove line that doesn't work on github (?!)

* Cleanup code

* Try to run tests non-parallel

* Squash errors for now

* Fixes for ubuntu

* Improving annotations to mask conversion

* Reformat

* Add Lazy Array

* Added some extra functionality to lazy array

* Added lazy evaluation for PolygonCollection

* Added lazy evaluation for PolygonCollection

* Make precommit run

* Updates to CI

* Fix mypy

* Add ROI to the GeometryCollection

* Added ROIs to DLUP XML

* Added ROIs to output xml and parsing

* move declarations to headers

---------

Co-authored-by: Bart de Rooij <[email protected]>
Co-authored-by: Jonas Teuwen <[email protected]>
Co-authored-by: JorenB <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants