Skip to content

Commit

Permalink
new project docs
Browse files Browse the repository at this point in the history
  • Loading branch information
philipbaileynar committed Jul 9, 2024
1 parent d2e176a commit cb33d4c
Showing 1 changed file with 115 additions and 0 deletions.
115 changes: 115 additions & 0 deletions content/page/documentation/project-getting-started.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
title: Project Getting Started
description: Instructions for getting new data into the Riverscapes Data Exchange
banner: true
---

The goal of this page is to help people with data to get it into the [Riverscapes Data Exchange](https://data.riverscapes.net). This page is intended for people who are new to the Riverscapes Data Exchange and have data that they would like to share with the community.

The instructions below cover several important steps, not all of which may be necessary for every situation. The phases are:

1. [Project Type Definition](#project-type-definition)
1. [Data Preparation](#data-preparation)
1. Business Logic
1. Symbology
1. Data Upload
1. Review
1. Sharing

# Project Type Definition

Is the type of project that you are working on already defined in the [Riverscapes Data Exchange](https://data.riverscapes.net)? If it is then you can skip ahead to the next section. If not, you may need to create a new project type. If you are unsure, check the list of existing project types on the Data Exchange home page.

## Defining a New Project Type

First, consider the data that you are trying to define. Think broadly and deeply about whether this is a one-off dataset, or whether you or other users will continue to use this project type for future, similar uses. Thinking ahead will increase the utility of your project type and reduce the amount of work that you need to do in the future.

If you are creating a new project type, you will need to define the following:

- **Name** - A short name for the project type. It's best if this name is less than 50 characters long. It can include spaces, but try to avoid special characters.
- **Summary** - A brief description of the project type. This should be less than 200 characters long.
- **Description** - A longer, plain text, description of the project type. This can include information about the data that is included in the project type, the purpose of the project type, and any other information that you think is relevant. Markdown syntax is supported in the description.
- **Metadata** - list of key value pairs. Useful values are:
- **Resolution** - the resolution of the data in the project type. For example, whether the individual data records represent reaches, subcatchments, or some other unit of analysis.
- **Extent** - the geographic extent of the data in the project type. For example, whether the data is limited to a specific reach, watershed, region, or whether it is global.
- **Model Status** - see Riverscapes Data Exchange [Model Statuses]() for more information.
- **Riverscapes Project Class**.
- **URL** - project a URL to an external documentation web site or a place where users can go to find more information about the project type.
- **machineName** - This is a short, unique identifier for the project type. It should be less than 20 characters long and must not include spaces or special characters. It's best, but not required, to be all lowercase.
- **State** - Whether the project type is ACTIVE or INACTIVE.

When you have determined the values for these fields, edit the file called [project_types.json](https://github.com/Riverscapes/riverscapes-tools/blob/dev/lib/riverscapes/riverscapes/projectTypeTool/projectTypes.json) in the Riverscapes Tools repository. Clone the repository, create a new branch for your changes, and then submit a pull request to the dev branch.

Consider waiting to submit your pull request until you have completed at least a draft of all the steps on this page. This will help you to understand the full scope of the project type and ensure that you have included all the necessary information.

# Data Preparation

If your data are for an existing project type, then the easiest way to prepare a new dataset is to visit the Riverscapes Data Exchange, download an existing project of the same type and then replicate the same file and folder structure in your new dataset.

If you are creating a new project type, you will need to prepare your data. This is an inherently bespoke process that will depend on the data that you have and the project type that you are creating. However, there are some general guidelines that you can follow:

- **Folder structure** - organize your data into logical, convenient folders. This will make it easier for you to find and use your data in the future. For simulation models, we typically use the folders `inputs`, `intermediates` and `outputs`. But for one-off, static datasets you might choose something more thematic. Avoid creating folders with only one or two files in them. But conversely, don't create folders with hundreds of files in them. Try to strike a balance.
- **Data Formats**. Always use open, non-proprietary data formats. The Riverscapes Consortium strives to be [FAIR](). For tabular data, use CSV or TSV. For spatial data, use [GeoPackages](https://www.geopackage.org/) or GeoTIFF. For time series data, use CSV or TSV or [SQLite](https://www.sqlite.org).
- **Coordinate Reference System**. Wherever possible, use a standard, well-known coordinate reference system. For a lot of data, this will be WGS84 EPSG 4326. For data that is specific to a region, use the appropriate UTM zone. There's no specific requirement for the coordinate reference system. Avoid unnecessary resampling of rasters. Rasters are best left in their original coordinate reference system.

## Project.rs.xml

Fundamental to each project is the `project.rs.xml` file. This file is an XML file that contains metadata about the project located in the root of the project folder. It is used by the Riverscapes Data Exchange to understand the project and to display information about the project to users.

It's easiest to copy an existing `project.rs.xml` file and then modify it to suit your project.

<Alert severity="warning">If you are starting from an existing `project.rs.xml` file, be sure to **REMOVE** the warehouse XML tag near the top of the file. New projects should not possess a warehouse tag until they are first uploaded to the Riverscapes Data Exchange.</Alert>

The `project.rs.xml` file must be valid and the easiest way to ensure this is to write it using a powerful text editor such as Visual Studio Code that has XML validation built-in. Errors will be highlighted directly in the editor.

Be sure to include a meaningful project name and a verbose, helpful description. And no one likes writing metadata, but it is vital for tracking data provenance and ensuring that your data is discoverable and reusable. Future you will be glad that you took the time to write good metadata.

## Project Bounds

The project bounds should be set to the geographic extent of the project. Use a desktop GIS to determine the outer extent of all the data in the project. Store the centroid and bounding rectangle in the project.rs.xml file.

Digitize a crude polygon around your data and then export it to GeoJSON and store this file in the root of your project folder. Make sure the GeoJSON is in [WGS84 EPSG 4326](https://spatialreference.org/ref/epsg/4326/). This step is extremely easy in QGIS.

<Alert severity="info">Avoid complex concave hulls and polygons with too many vertices. A simple rectangle or polygon with only a handul of vertices is best. A good benchmark is to keep the GeoJSON file well under 500kb.</Alert>

# Buisness Logic

Business logic is used by [Riverscapes Viewers](https://viewer.riverscapes.net) to determine how to present the project contents to users. Each project type has its own business logic file.

Business logic files are stored in the [RiverscapesXML repository](https://spatialreference.org/ref/epsg/4326/). If you are developing a new project type then follow the [instructions to develop new business logic](https://viewer.riverscapes.net/technical-reference/business-logic).

<Alert severity="info">A tip for developing business logic files is to start by saving the new business logic file directly inside your Riverscapes project folder. This is the first place that the Viewer looks for business logic files. This will allow you to test your business logic iteratively without having to upload your project to the Riverscapes Data Exchange.</Alert>

When you have developed your business logic, you can submit a pull request to the RiverscapesXML repository for review by the Riverscapes Development team to review and approve. It is recommended that you do not upload any projects until your business logic is approved.

# Symbology

The symbology used to display geospatial data in the Riverscapes Viewers is defined as a series of layer files that are also stored in the RiverscapesXML repository. Different symbology formats are used for each of the various versions of the Viewer ([Web](https://viewer.riverscapes.net/software-help/help-web/), [QGIS](https://viewer.riverscapes.net/software-help/help-qgis/), [ArcGIS](https://viewer.riverscapes.net/software-help/help-arc/) and ArcPro).

Follow the [symbology development instructions](https://viewer.riverscapes.net/technical-reference/symbology/) to develop new symbology files and then reference them in your business logic file. Note that symbology can be reused across project types. Review existing symbology and reuse it where possible.

Similar to developing business logic, the Viewer first looks for symbology files in the project folder. This allows you to test your symbology iteratively without having to upload your project to the Riverscapes Data Exchange in order to see the impacts of your changes.

Once your symbology is complete, submit a pull request to the RiverscapesXML repository for review by the Riverscapes Development team. It is recommended that you do not upload any projects until your symbology is approved.

# Data Upload

You are finally ready to upload your data to the Riverscapes Data Exchange. If you have just one or two projects to upload, you can use the [Upload feature](https://viewer.riverscapes.net/software-help/help-qgis-uploader) within the QViewer for QGIS. Use the [rscli](https://developer.riverscapes.net/dev-tools/rscli/) command line interface if you have lots of projects to upload.

You will need to have an account on the Riverscapes Data Exchange to upload data. If you do not have an account, you can [create one](https://data.riverscapes.net).

When you upload your project, you will need to provide an owner for the project. This can be an individual or an organization. Think carefully about this decision because it impacts who can edit the project in the future. You can always change the owner of a project later, but it is easier to get it right the first time.

You will also need to specify the privacy setting for the project. You can choose between public, private, and secret. Public projects are visible to everyone. Private projects are only visible to the owner. Secret projects are only visible to people who have access to the project, typically via organization membership.

# Review

After uploading your project, you should review it in the Riverscapes Data Exchange. Both the QViewer upload feature and the rscli command line interface will provide you with a URL to view your project in the Riverscapes Data Exchange. Alternatively, you can simply navigate to the [Riverscapes Data Exchange](https://data.riverscapes.net) and search for your project.

Check the name, description and apply any tags that are relevant to your project. Check that the project bounds are correct and that the project is displaying as you expect in the Web Viewer.

# Sharing

Once you are happy with your project, you can share it with the riverscapes community. Share the URL to your project with your colleagues, collaborators, and anyone else who might be interested in your work. You can also share the URL on social media or in other online forums.

We encourage you to announce the new project on the [Riverscapes community platform](https://riverscapes.net) to let others know about your work.

0 comments on commit cb33d4c

Please sign in to comment.