Skip to content

Commit

Permalink
Release 2.9.0
Browse files Browse the repository at this point in the history
  • Loading branch information
Jan Lienemann committed Jun 27, 2023
1 parent dd2cb0a commit b41cbd7
Show file tree
Hide file tree
Showing 100 changed files with 4,973 additions and 1,403 deletions.
398 changes: 398 additions & 0 deletions examples/00_reference/10_database_interface.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,398 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Database use with LabOne Q\n",
"\n",
"LabOne Q comes with a database interface to make it easier to store and retrieve any data or experiments. It currently supports the SQLite implementation native to Python.\n",
"\n",
"In this notebook, you will learn how to:\n",
"\n",
"* Create and connect to a database\n",
"* Save LabOne Q data to your database\n",
"* Access your stored data\n",
"* Delete data from a database \n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# 0. Python imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import datetime\n",
"import io\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# pillow is currently required for displaying images after saving them to the database\n",
"from PIL import Image as PILImage\n",
"\n",
"# convenience import for all LabOne Q software functionality\n",
"from laboneq.simple import *\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# 1. Creating and connecting to a database"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll begin by creating a new database or connecting to an existing database at the default location. This location is `./laboneq_data/data.db`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"my_db = L1QDatabase()\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also create a new database or connect to an existing database at a customized location. You'll now do this in the path you define below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"custom_db_path = \"laboneq_data/custom_database.db\"\n",
"\n",
"my_custom_db = L1QDatabase(custom_db_path)\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2. Saving a LabOne Q data object in the database"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Before saving, you can check if the database already contains any data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"[my_db.get(k, with_metadata=True) for k in my_db.keys()]\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll now create an Experiment and store it in the database, using the key `\"my_experiment\"`. Not the the key can be different than the experiment `uid`; you'll do this later on."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"exp = Experiment(\"my_experiment\", signals=[ExperimentSignal(uid=\"signal_1\")])\n",
"\n",
"my_db.store(exp, key=\"my_experiment\")\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll now store the same experiment as above using the same key. You'll now also include additional metadata for easier indexing and retrieval later. Note that this overwrites the previously stored dataset, since the same key is used."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"my_db.store(\n",
" exp,\n",
" key=\"my_experiment\",\n",
" metadata={\n",
" \"author\": \"John Doe\",\n",
" \"creation_date\": datetime.datetime.now(),\n",
" \"setup\": \"CountZero\",\n",
" },\n",
")\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, you'll store another version of the same experiment with a different key and different metadata."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"my_db.store(\n",
" exp,\n",
" key=\"my_old_experiment\",\n",
" metadata={\n",
" \"author\": \"John Doe\",\n",
" \"creation_date\": datetime.datetime(year=2021, month=4, day=20),\n",
" \"setup\": \"CountZero\",\n",
" },\n",
")\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Images from your data analysis can also be saved as part of the metadata for quick access. You'll now generate a plot using `matplotlib.pyplot` below, and then store it as part of the database metadata."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plt.plot([1, 2, 3, 4])\n",
"plt.ylabel(\"It's linear!\")\n",
"# save as png\n",
"img_buf = io.BytesIO()\n",
"plt.savefig(img_buf, format=\"png\")\n",
"image_bytes = img_buf.getvalue()\n",
"\n",
"# include the figure as part of the metadata\n",
"my_db.store(\n",
" exp,\n",
" key=\"experiment_with_image\",\n",
" metadata={\n",
" \"result_plot_png_bytes\": image_bytes,\n",
" \"creation_date\": datetime.datetime.now(),\n",
" },\n",
")\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3. Accessing the stored data by key and through metadata"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll start by checking if the database contains any data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"[my_db.get(k, with_metadata=True) for k in my_db.keys()]\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll then access the list of available keys."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"list(my_db.keys())\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll then access the metadata for each available key."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for k in my_db.keys():\n",
" print(k, my_db.get_metadata(k))\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"If you want to find and access only specific datasets, selected by metadata fields, you can do that as well. Let's say you have three experimental set-ups, named for the [Sprawl Trilogy](https://en.wikipedia.org/wiki/Sprawl_trilogy). You'd only like to access one of them, the data associated with [Count Zero](https://en.wikipedia.org/wiki/Count_Zero):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"count_zero_keys = my_db.find(metadata={\"setup_name\": \"CountZero\"})\n",
"for k in count_zero_keys:\n",
" print(k, my_db.get(k, with_metadata=True))\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You may, instead, want to perform a more general query of the metadata based on the creation date:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"new_data = my_db.find(\n",
" condition=lambda metadata: metadata[\"creation_date\"]\n",
" > datetime.datetime(year=2023, month=4, day=20)\n",
")\n",
"\n",
"for k in new_data:\n",
" print(k, my_db.get(k, with_metadata=True))\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also display an image saved in the metadata."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"im = PILImage.open(\n",
" io.BytesIO(my_db.get_metadata(\"experiment_with_image\")[\"result_plot_png_bytes\"])\n",
")\n",
"\n",
"im.show()\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# 4. Deleting data from the database"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, you may want to delete entries from your database. You can do this in the following way:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"my_db.delete(\"experiment_with_image\")\n",
"my_db.delete(\"my_experiment\")\n",
"my_db.delete(\"my_old_experiment\")\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"You've now created, accessed, and deleted data in a database with LabOne Q. \n",
"\n",
"Have more questions about incorporating LabOne Q in your experiments? Contact us at [[email protected]](mailto:[email protected])."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "develop",
"language": "python",
"name": "develop"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.0"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading

0 comments on commit b41cbd7

Please sign in to comment.