Example Data Sets

This repository contains open source data sets. They are intended to be used as part of an introduction to the Deephaven Community Core Engine. For more information, check out Deephaven Community Core.

Table of Contents

The following folders can be found in this repository:

• gsod - Global Surface Summary of the Day (GSOD) weather data
• iris - The iris flower data set from Ronald Fisher's 1936 paper
• metriccentury - Data recorded from a 100 km bike ride
• DeNiro - Data on Robert De Niro's movies up to 2016
• pems - Traffic flow data collected near Davis, CA.
• Taxi - Yellow taxi trip records

Description

Each folder in this repository has the following structure within:

• README - An explanation of everything about the data
• csv - A folder with all relevant data in either CSV or TSV format (if available)
• parquet - A folder with all relevant data in parquet format (if available)

Installation Instructions

The examples script image can be pulled via:

docker pull ghcr.io/deephaven/examples

To download the examples, from the root of your deephaven-core clone, run:

docker run --rm -v "$(pwd)/docker/core/data:/data" ghcr.io/deephaven/examples download

• The docker run command downloads the example data by running the examples management container.
• The -v "$(pwd)/docker/core/data:/data" argument mounts your local $(pwd)/docker/core/data path as /data in the container.
• The example data is stored to /data/examples inside the container, which is $(pwd)/docker/core/data/examples on the local filesystem.

You can run docker run again to manage the example data - for example, to download a new version. To see what options are available, run:

docker run --rm -v "$(pwd)/docker/core/data:/data" ghcr.io/deephaven/examples

Build Instructions

From the root of examples, run:

docker build -t ghcr.io/deephaven/examples docker

The docker build command builds a Docker container containing a script that helps manage the examples. That script will automate the management of the example files. It allows you to download and update the example files without directly working with the git project hosting the examples. This docker build command is only needed once; after it runs, the container is available to use on your host.