purl2sym documentation

Makefile

@@ -41,6 +41,8 @@ conf:
 dev:
 	@echo "-> Configure and install development dependencies"
 	@PYTHON_EXECUTABLE=${PYTHON_EXE} ./configure --dev
+	@echo "-> Configure and install documentation dependencies"
+	@PYTHON_EXECUTABLE=${PYTHON_EXE} ./configure --docs
 
 envfile:
 	@echo "-> Create the .env file and generate a secret key"
@@ -150,7 +152,7 @@ bump:
 
 docs:
 	rm -rf docs/_build/
-	@${ACTIVATE} sphinx-build docs/ docs/_build/
+	@${ACTIVATE} sphinx-build docs/source docs/_build/
 
 docker-images:
 	@echo "-> Build Docker services"

docs/source/index.rst

@@ -3,6 +3,12 @@ Welcome to PurlDB documentation!
 
 .. include:: ../../README.rst
 
+.. toctree::
+    :maxdepth: 2
+    :caption: Purl2Sym
+
+    purl2sym
+
 Indices and tables
 ==================
 

docs/source/purl2sym.rst

@@ -0,0 +1,21 @@
+.. _purl2sym:
+
+Purl2Sym
+============
+
+Purl2Sym collects the core package metadata along with symbols and strings
+from source code and stores them in the ``extra_data`` field of the resource.
+
+How it works
+^^^^^^^^^^^^
+
+When PurlDB receives an index request for a PURL via the ``/api/collect``
+endpoint, it fetches the archive download_url and creates a package for
+the PURL with relevant metadata. Thereafter, a scan job is scheduled which
+downloads the archive of the PURL and runs the `scan_single_package <https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html#scan-single-package>`_
+package pipeline. Thereafter, the scan job also runs the two addon pipelines:
+`collect_symbols <https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html#collect-codebase-symbols-addon>`_
+and `collect_source_strings <https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html#collect-source-strings-addon>`_
+for symbol and string collection respectively. Upon completion of the scan
+job, the package is updated with resource data along with the ``source_symbols``
+and ``source_strings`` in the ``extra_data`` field of resources.