2017.12.07

2017.12.07

Contributors: Albert Cardona, Andrew Champion, Chris Barnes, Tom Kazimiers

Notes

• PostgreSQL 9.6 and Postgis 2.4 are now required.

• A virtualenv update is required.

• New settings field: CROPPING_VERIFY_CERTIFICATES. This controls whether SSL
  certificates should be verified during cropping and defaults to True.

• The NODE_LIST_MAXIMUM_COUNT setting can now be set to None, which disables
  node limiting. If node count limiting isn't really needed in most cases, this
  can slighly improve query performance.

• The file UPDATE.md will now list all administration related changes for each
  release, including required manual tasks. These changes also continue to be
  part of this change log.

• Requires running of: manage.py catmaid_update_project_configuration

• Tracing data is now by default transmitted in a binary form, please make
  therefore sure your web-server applies GZIP not only to the "application/json"
  content type, but also to "application/octet-stream". For Nginx this would be
  the gzip_types setting.

Features and enhancements

Node filters:

• A new node filter called "In skeleton source" has been added. This allows to
  add a filter which only allows nodes of a particular set of skeletons,
  accessed through a skeleton source. This can be useful to look at connectivity
  of a particular set of neurons. It can of course be combined with other
  filters like volume intersection.

• New filter: Pruned arbor, which keeps the arbor proximal to the (tagged) cut
  points, discarding distal subarbors.

Connector table:

• Node filters are now supported.

• Attachment connectors are now supported.

Landmarks:

• The new "Landmark Widget" allows to create landmarks, form groups of them and
  use these groups to virtually transform skeletons from one landmark group into
  another one. This can be used e.g. to find homologues neurons. Virtual
  skeletons can currently displayed in the 3D Viewer.

• Defining landmarks and their groups is done in two stages: 1. define abstract
  landmarks and group those abstract landmarks. 2. link locations to individual
  landmarks and their presence in a group.

• Abstract landmarks represent e.g. a feature that can be found at multiple
  places in the data set, like the entry point of a particular neuron on the
  left and right side of a Drosophila brain. The left and right side of the
  brain would then be represented as groups, each one having expected abstract
  landmarks as members.

• Creating new abstract landmarks and groups can be done through the respective
  "Name" fields and "Add" buttons in the Landmarks tab. To link an abstract
  landmark to a group, click on either an existing landmark group member in the
  landmark groups table or on the "(none)" placeholder. The newly opened dialog
  will allow to associate landmarks with groups. Landmarks can be member of
  multiple groups.

• Linking locations to landmarks and the groups they are member of is done by
  using the right click context menu on either an existing location in the
  "Locations" column of the Landmarks table or the displayed "(none)"
  placeholder. Currently, the location of the active node or the center of the
  stack viewer can be linked. When using the latter option, it might be a good
  idea to enable the display of reference lines in the Settings Widget.

• To be able to use a landmark location in a transformation, the locations
  linked to landmarks need to be associated with at least one group. If for
  instance a physical location for the abstract landmark representing a
  particular neuron entry point has been found for the left side of a Drosophila
  brain, this location would be linked to the landmark itself and to the group
  representing the left side of the brain. To link the location to the group,
  right click the index number in the landmark's row in the Location table, the
  context menu will allow to add the landmark to all groups the abstract
  landmark is a member of (removal works the same way).

• Virtual skeleton transformations can be created from the "Display" tab. After
  selecting a target 3D Viewer from the drop down menu, the skeletons to create
  transformed virtual version for, have to be selected by choosing a skeleton
  source. With this done, a source and target landmark group can be selected.
  Transformations are only expect to provide reasonable results if the
  transformed skeletons are "enclosed" by the landmark group.

• The "Import tab" allows to import multiple four-column CSV files into landmark
  groups. The expected format is: Landmark name, X, Y, Z. Each file can be
  associated with one landmark group, landmarks with the same name will be
  matched.

3D viewer:

• Landmarks and landmark groups can be displayed. Similarly to volumes
  individual groups can be selected and adjusted through the "View settings"
  tab. Apart from the color and mesh-faces option for groups, it is possible to
  scale landmarks.

• All visible skeletons can now be exported as Wavefront OBJ file. Each skeleton
  will be rendered as a separate group, colors are preserved and written out as
  material file (MTL). Both files can be compressed as ZIP file on the fly and it
  is possible to generate a simple line based export as well as a mesh based
  representation using a user definable radius. The generated ZIP file can be
  used with services like augment.com.

Neuron history widget:

• A user filter can now be applied through the respective checkbox drop down
  element. If at least one user is selected, only events caused by those users
  are respected.

New connector types:

• Two new connector relations are available: attached_to and close_to. Like
  other connector relations, they can be used to link treenodes to connectors
  and can be used with the help of Alt + Click context menu when a treenode is
  selected. They are available as "Attachment" and "Close to".

• If If a connector has at least one attachment relation is viewed as
  "Attachment connector" and additional clicks with such a connector selected
  will create "Close to" edges.

• To add more information about the nature of an attachment, the connector
  should be labeled, e.g. as "vesicle".

Connectivity Widget:

• All skeletons linked through a attached_to -> connector -> close_to
  relationship are now shown in a separate table if the "Show attachments"
  option is enabled.

Export management command:

• The export management command will now also export meta-annotations when the
  --annotation option is set. All meta-annotations of annotations directly
  linked to the exported neurons are exported, too.

• Users referenced by any of the exported objects are now also exported
  automatically. This can be disabled using the --nousers options.

• User references are now stored as usernames rather than numerical IDs. This
  removes the requirement of always also exporting user models, because users
  can be mapped based on their usernames.

Import management command:

• The new --map-users option allows to map users referenced in the imported data
  to already existing users in the database. This makes updating existing
  databases easier and removes the de facto import data requirement to contain
  user models. By default --map-users is not set.

• The new --create-unknown-users option will instruct the importer to create
  new inactive user accounts for users that aren't included as objects in the
  imported data. If user mapping is enabled, accounts would only be created for
  users that can't be mapped and are also not included in the import data. By
  default --create-unknown-users is not set.

• Imported class, class instance and relation objects are skipped when
  existing objects in the target project have the same name. This makes merging
  of projects also possible for semantic data (e.g. annotations, tags). The only
  exception are neuron class instances, where two different objects are allowed
  to have the same name.

• The importer now won't keep the IDs from spatial objects in the input source,
  but will instead always create new objects. This ensures no existing data is
  replaced and allows importing different sources into a single CATMAID
  instance. If the original IDs have to be used, the --preserve-ids option
  can be used. This is done for treenodes, connects, and their links.

Miscellaneous:

• Users can now choose alternative markers for connector nodes, which do not
  obscure the object being annotated: this encourages manually annotating
  synapses etc. in a manner which is more comparable with automated detections

• Exported sub-stacks now include the Z resolution.

• Added table of reviewers vs number of nodes reviewed to the "Summary Info"
  in the Selection Table widget.

• Synapse distribution plot: global and single-axis zoom and pan are now
  supported.

• Neuron dendrogram: nodes linked to connectors (synaptic sites) can now be
  highlighted.

• Tracing layer: cached tracing data is now used by default when showing
  sub-views of previously loaded data. This is useful for e.g. quickly zooming
  in. Like with the regular node cache, views not used for one minute will be
  thrown away.

• All users can now view their user analytics plot with the help of the Project
  Statistics widget.

• The NRRD file export can now transform from FAFB v14 into template brain
  space.

• Synapse Fractions: the button "Group all ungrouped" will move all skeletons
  that are not part of a group already and not part of "others" into a new
  group. To also include skeletons in "others", use a synapse threshold of zero
  in the "Filter partners" tab.

• Selection Table: Open/Save (JSON) does now preserve the order of skeletons.

• Tracing data should load faster due to spatial query improvements and
  reduction of transferred data. The tracing layer transfers data now in binary
  by default, which can be adjusted in through the tracing layer settings.

• The catmaid_setup_tracing_for_project management command does not require a
  --user argument anymore. If not provided, the first available admin user will
  be used.

Bug fixes

• 3D Viewer: the visibility of skeletons with connector colors the same as the
  skeleton can now be controlled correctly.

• Selecting nodes explicitly with multiple stackviewers open (e.g. by
  Shift-Click in the 3D Viewer) sometimes raised an error about not being able
  to find the selected node. This is fixed now.

• Entries in the Treenode Table can now be correctly sorted by date.

• Cropping tasks can now work with HTTPS URLs.

• Graph widget: new skeletons can now be added again after individual skeletons
  have been split.

API changes

Additions

• POST /{project_id}/nodes/location:
  Get the location of multiple nodes, expects a node_ids parameter.

• GET /{project_id}/connectors/ now provides a new API to query connectors
  directly and not only links. The previous API is available as
  GET /{project_id}/connctors/links/ (see below).

• GET /{project_id}/skeletons/{skeleton_id}/neuroglancer:
  Export a morphology-only skeleton in neuroglancer's binary format.

Modifications

• GET /{project_id}/connector/user-info has been replaced with
  GET /{project_id}/connectors/user-info. Functionality is the same.

• GET /{project_id}/connectors/ has been replaced with
  GET /{project_if}/connectors/links/. Functionality is the same.

• POST /{project_id}/node/list offers a new optional parameter "format", which
  is set by default to 'json', but can be set to 'msgpack' to use msgpack binary
  encoding of the result.

Deprecations

None.

Removals

None.