Add files via upload

deutsche-nationalbibliothek · Jan 14, 2024 · 01e1bf1 · 01e1bf1
1 parent 1275f1f
commit 01e1bf1
Showing 1 changed file with 168 additions and 0 deletions.
diff --git a/content/DNB_SRU_Tutorial_Einstieg.ipynb b/content/DNB_SRU_Tutorial_Einstieg.ipynb
@@ -0,0 +1,168 @@
+{
+  "metadata": {
+    "anaconda-cloud": {},
+    "kernelspec": {
+      "name": "python",
+      "display_name": "Python (Pyodide)",
+      "language": "python"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "python",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.8"
+    }
+  },
+  "nbformat_minor": 4,
+  "nbformat": 4,
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "source": "# DNBLab Jupyter Notebook zur SRU Abfrage",
+      "metadata": {
+        "nbpresent": {
+          "id": "d0d72cd5-034d-4b94-a4e2-54f7753cb9f0"
+        }
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": "## SRU - Schnittstellenabfrage, Datenauslieferung und Ergebnisanzeige",
+      "metadata": {
+        "nbpresent": {
+          "id": "558898c3-e162-4e46-92d5-b4c914325790"
+        }
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": "Lassen Sie uns gemeinsam folgenden Markdown *hier* einfügen:",
+      "metadata": {}
+    },
+    {
+      "cell_type": "markdown",
+      "source": "Dieses DNBLab-Tutorial beschreibt eine Beispielabfrage über die SRU-Schnittstelle mit Python. In der Jupyter Notebook Umgebung kann der dokumentierte Code direkt ausgeführt und angepasst werden. Das Tutorial umfasst die exemplarische Abfrage und Ausgabe der Daten in MARC21-xml zur weiteren Verarbeitung mit Python. ",
+      "metadata": {
+        "nbpresent": {
+          "id": "ee78241c-36e7-45e6-b015-0ef46f1e777d"
+        }
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": "## Ganz wichtig, Zwischenspeichern nicht vergessen!",
+      "metadata": {}
+    },
+    {
+      "cell_type": "markdown",
+      "source": "## Arbeitsumgebung einrichten! <a class=\"anchor\" id=\"Teil1\"></a>",
+      "metadata": {
+        "nbpresent": {
+          "id": "3c6f6024-cb7c-4b65-8646-a753069cb5a4"
+        },
+        "tags": []
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": "import urllib.parse\nfrom pyodide.http import open_url, pyfetch\nfrom js import fetch\nfrom bs4 import BeautifulSoup as soup\nimport unicodedata\nfrom lxml import etree\nimport pandas as pd",
+      "metadata": {
+        "trusted": true
+      },
+      "execution_count": 1,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": "## Abfrage über alle Datensätze... <a class=\"anchor\" id=\"Teil2\"></a>",
+      "metadata": {}
+    },
+    {
+      "cell_type": "code",
+      "source": "async def dnb_sru(query):\n    \n    base_url = \"https://services.dnb.de/sru/dnb\"\n    params = {'recordSchema' : 'MARC21-xml',\n          'operation': 'searchRetrieve',\n          'version': '1.1',\n          'maximumRecords': '100',\n          'query': query\n         }\n    \n    r = await fetch(base_url + \"?\" + urllib.parse.urlencode(params))  \n    r_text = await r.text()\n    xml = soup(r_text, features=\"xml\")\n    records = xml.find_all('record', {'type':'Bibliographic'})\n    \n    \n    if len(records) < 100:\n        \n        return records\n    \n    else:\n        \n        num_results = 100\n        i = 101\n        while num_results == 100:\n            \n            params.update({'startRecord': i})\n            r = await fetch(base_url + \"?\" + urllib.parse.urlencode(params))  \n            r_text = await r.text()\n            xml = soup(r_text, features=\"xml\")\n            new_records = xml.find_all('record', {'type':'Bibliographic'})\n            records+=new_records\n            i+=100\n            num_results = len(new_records)\n            \n        return records",
+      "metadata": {
+        "trusted": true
+      },
+      "execution_count": 2,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": "## Felder Titel und Links zu den Objekten durchsuchen... <a class=\"anchor\" id=\"Teil3\"></a>",
+      "metadata": {}
+    },
+    {
+      "cell_type": "code",
+      "source": "def parse_record(record):\n    \n    ns = {\"marc\":\"http://www.loc.gov/MARC21/slim\"}\n    xml = etree.fromstring(unicodedata.normalize(\"NFC\", str(record)))\n    \n    #idn\n    idn = xml.xpath(\"marc:controlfield[@tag = '001']\", namespaces=ns)\n    try:\n        idn = idn[0].text\n    except:\n        idn = 'fail'\n    \n    # link\n    link = xml.xpath(\"marc:datafield[@tag = '856']/marc:subfield[@code = 'u']\", namespaces=ns)\n    \n    try:\n        link = link[0].text\n        #titel = unicodedata.normalize(\"NFC\", titel)\n    except:\n        link = \"unkown\"\n        \n          # titel\n    titel = xml.xpath(\"marc:datafield[@tag = '245']/marc:subfield[@code = 'a']\", namespaces=ns)\n    \n    try:\n        titel = titel[0].text\n        #titel = unicodedata.normalize(\"NFC\", titel)\n    except:\n        titel = \"unkown\"\n        \n    \n    meta_dict = {\"idn\":idn,\n                 \"titel\":titel,\n                 \"link\":link}\n    \n    return meta_dict",
+      "metadata": {
+        "trusted": true
+      },
+      "execution_count": 15,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": "# Abfrage nach Suchwort \"Biene\" im Titel + Links zu frei verfügbaren Objekten!",
+      "metadata": {}
+    },
+    {
+      "cell_type": "code",
+      "source": "records = await dnb_sru('tit=Biene and location=onlinefree')\nprint(len(records), 'Ergebnisse')",
+      "metadata": {
+        "trusted": true
+      },
+      "execution_count": 16,
+      "outputs": [
+        {
+          "name": "stdout",
+          "text": "17 Ergebnisse\n",
+          "output_type": "stream"
+        }
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": "## Anzeige der Treffer in einer Tabelle...  <a class=\"anchor\" id=\"Teil4\"></a>",
+      "metadata": {}
+    },
+    {
+      "cell_type": "code",
+      "source": "output = [parse_record(record) for record in records]\ndf = pd.DataFrame(output)\ndf",
+      "metadata": {
+        "trusted": true
+      },
+      "execution_count": 17,
+      "outputs": [
+        {
+          "execution_count": 17,
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": "           idn                                              titel  \\\n0   1283345676  Verkauf der \"Biene\" an Franz Wagner in Leipzig...   \n1   1299731961  Die Biene und der Blitz - die eigenwillige K...   \n2   1246522276  Info Bee at UNICARagil: Support from Above for...   \n3   1213898609  Die Biene - Illustrierte Schüler*inarbeit - ...   \n4   1156534089  Aus der Wabe in die Welt: Biene macht Kultur. ...   \n5   118258280X  [Rezension zu:] Viel, Bernhard: Der Honigsamml...   \n6   1095550535  [Referentenmaterial] / Gesellschaft zur Verbre...   \n7   1263070647  Harald Weiss (Hg.): 100 Jahre Biene Maja: Vom ...   \n8   1076865429  Harald Weiss (Hg.): 100 Jahre Biene Maja: Vom ...   \n9   1046981080           Biene und Honig im pharaonischen Ägypten   \n10  1026527112  Temperature effects on the development in the ...   \n11  1081225548  Futterqualität und Rekrutierungsverhalten bei ...   \n12  1171169078                         Die Biene und ihre Zucht   \n13  1123508119                     Leben und Zucht der Honigbiene   \n14  1171169027  Melitto-Theologia : die Verherrlichung des glo...   \n15  1119154553            Die italienische Biene und ihre Zucht   \n16  1138282669                             Nektar – Biene – Honig   \n\n                                                 link  \n0   https://nbn-resolving.org/urn:nbn:de:101:1-202...  \n1   https://nbn-resolving.org/urn:nbn:de:bvb:384-o...  \n2            https://doi.org/10.18154/RWTH-2021-10614  \n3                        https://doi.org/10.18442/140  \n4   https://nbn-resolving.org/urn:nbn:de:bvb:20-op...  \n5   https://nbn-resolving.org/urn:nbn:de:hebis:30:...  \n6   https://nbn-resolving.org/urn:nbn:de:101:1-201...  \n7              https://doi.org/10.17192/ep2015.2.3594  \n8              https://doi.org/10.17192/ep2015.2.3594  \n9   https://nbn-resolving.org/urn:nbn:de:hebis:77-...  \n10  https://nbn-resolving.org/urn:nbn:de:bvb:355-e...  \n11  https://nbn-resolving.org/urn:nbn:de:hebis:30-...  \n12  https://nbn-resolving.org/urn:nbn:de:hebis:30-...  \n13  https://nbn-resolving.org/urn:nbn:de:hebis:30-...  \n14  https://nbn-resolving.org/urn:nbn:de:hebis:30-...  \n15  https://nbn-resolving.org/urn:nbn:de:hebis:30-...  \n16  https://nbn-resolving.org/urn:nbn:de:hbz:6-592...  ",
+            "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>idn</th>\n      <th>titel</th>\n      <th>link</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1283345676</td>\n      <td>Verkauf der \"Biene\" an Franz Wagner in Leipzig...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:101:1-202...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1299731961</td>\n      <td>Die Biene und der Blitz - die eigenwillige K...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:bvb:384-o...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1246522276</td>\n      <td>Info Bee at UNICARagil: Support from Above for...</td>\n      <td>https://doi.org/10.18154/RWTH-2021-10614</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1213898609</td>\n      <td>Die Biene - Illustrierte Schüler*inarbeit - ...</td>\n      <td>https://doi.org/10.18442/140</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1156534089</td>\n      <td>Aus der Wabe in die Welt: Biene macht Kultur. ...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:bvb:20-op...</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>118258280X</td>\n      <td>[Rezension zu:] Viel, Bernhard: Der Honigsamml...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:30:...</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>1095550535</td>\n      <td>[Referentenmaterial] / Gesellschaft zur Verbre...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:101:1-201...</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>1263070647</td>\n      <td>Harald Weiss (Hg.): 100 Jahre Biene Maja: Vom ...</td>\n      <td>https://doi.org/10.17192/ep2015.2.3594</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>1076865429</td>\n      <td>Harald Weiss (Hg.): 100 Jahre Biene Maja: Vom ...</td>\n      <td>https://doi.org/10.17192/ep2015.2.3594</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>1046981080</td>\n      <td>Biene und Honig im pharaonischen Ägypten</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:77-...</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>1026527112</td>\n      <td>Temperature effects on the development in the ...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:bvb:355-e...</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>1081225548</td>\n      <td>Futterqualität und Rekrutierungsverhalten bei ...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:30-...</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>1171169078</td>\n      <td>Die Biene und ihre Zucht</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:30-...</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>1123508119</td>\n      <td>Leben und Zucht der Honigbiene</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:30-...</td>\n    </tr>\n    <tr>\n      <th>14</th>\n      <td>1171169027</td>\n      <td>Melitto-Theologia : die Verherrlichung des glo...</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:30-...</td>\n    </tr>\n    <tr>\n      <th>15</th>\n      <td>1119154553</td>\n      <td>Die italienische Biene und ihre Zucht</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hebis:30-...</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>1138282669</td>\n      <td>Nektar – Biene – Honig</td>\n      <td>https://nbn-resolving.org/urn:nbn:de:hbz:6-592...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
+          },
+          "metadata": {}
+        }
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": "# Speichern der Ergebnisse als CSV-Datei!",
+      "metadata": {}
+    },
+    {
+      "cell_type": "code",
+      "source": "df.to_csv(\"SRU_Titel.csv\", index=False)",
+      "metadata": {
+        "trusted": true
+      },
+      "execution_count": 18,
+      "outputs": []
+    }
+  ]
+}