Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Matt win build #29

Open
wants to merge 35 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
711d649
Adding in instructions for a Windows build.
Aug 25, 2021
9807848
Adding in more details for Windows build process.
Aug 25, 2021
aee4296
Slowly adding more instructions for Windows build.
Aug 26, 2021
1a3d4ef
Merge branch 'develop' into matt-win-build
Aug 31, 2021
2c20c80
Adding in some details which seem to have disappeared from the docs. …
Aug 31, 2021
0ec700c
Making more notes on Windows build as I go.
Aug 31, 2021
80ecead
Now got a basic Windows build of the docs.
Sep 1, 2021
622afe7
Removed copasi stuff.
Sep 1, 2021
56d5271
Getting documentation generated by Sphinx from Doxygen comments.
Sep 7, 2021
e86e4e5
Tidying up the Mac doc build instructions.
Sep 7, 2021
27bd96a
Continuing to beautify my notes.
Sep 7, 2021
b06aa1b
Moved SWIG instructions to bottom.
Sep 8, 2021
0c428da
Making the build documentation clearer.
Sep 8, 2021
c16e4c7
Making the build instructions clearer.
Sep 8, 2021
e8599c1
Not sure if we need this.
Sep 13, 2021
46c3b4d
Adding my rough notes so they don't get lost.
Sep 13, 2021
91c632a
Keeping track of special points needed for SWIG Python binding to wor…
Sep 16, 2021
1f3f895
Merge branch 'develop' into matt-win-build
skeating Sep 20, 2021
5cc3700
Adding C++ instructions.
Sep 20, 2021
3a8f1c7
More changes
Sep 21, 2021
6bd3954
Merge branch 'matt-win-build' of https://github.com/sbmlteam/liblx in…
Sep 21, 2021
eb46caf
More details of C and C++ with Python binding.
Sep 21, 2021
be132ba
Better to delete the entire directory.
Oct 4, 2021
54b5916
Changing swig-windows.txt into an .rst file.
Oct 5, 2021
d0daf3a
Saving the updates to the documentation.
Oct 5, 2021
040d471
Adding new file to tidy things up for Windows users.
Oct 5, 2021
56c6778
More Windows lovelyness.
Oct 5, 2021
e95173e
Tidying up the Windows example.
Oct 5, 2021
de9ac3c
So the windows example appears in contents.
Oct 5, 2021
d8e7ab9
Battling withthe documentation.
Oct 5, 2021
ce9f804
More tidying up and removal of unnecessary content.
Oct 5, 2021
66fd8da
Moved content to complete-windows-example.rst
Oct 6, 2021
f6a7138
More tidying up
Oct 6, 2021
18d08fd
Getting all the pages built.
Oct 6, 2021
51a494e
Adding details of other XML library choices to Windows build.
Oct 7, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/Doxyfile.in
Original file line number Diff line number Diff line change
Expand Up @@ -827,8 +827,11 @@ WARN_LOGFILE =
# directories like /usr/src/myproject. Separate the files or directories with
# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
# Note: If this tag is empty the current directory is searched.

INPUT = "@DOXYGEN_INPUT_DIR@" # "@DOXYGEN_INPUT_DIR@/core" "@DOXYGEN_INPUT_DIR@/model"
# CODE_SRC_DIR needs to be set as an environment variable
# e.g. set CODE_SRC_DIR=C:\Users\mattg\repos\work\CompBioLibs\liblx\src
# It is the top of the source file hierarchy.
#INPUT = "C:\Users\mattg\repos\work\CompBioLibs\liblx\src\liblx\xml" "C:\Users\mattg\repos\work\CompBioLibs\liblx\src\liblx\xml\compress" #"C:\Users\mattg\repos\work\CompBioLibs\liblx\src" #"@DOXYGEN_INPUT_DIR@" # "@DOXYGEN_INPUT_DIR@/core" "@DOXYGEN_INPUT_DIR@/model"
INPUT = "$(CODE_SRC_DIR)/liblx/xml" "$(CODE_SRC_DIR)/liblx/xml/common" "$(CODE_SRC_DIR)/liblx/xml/compress" "$(CODE_SRC_DIR)/liblx/xml/compress/test"

# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
Expand Down
3 changes: 3 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ Welcome to the LibLX API documentation
:caption: Contents:

quickstart/get-started
quickstart/complete-windows-example
quickstart/swig-windows
liblx/python-bindings
examples/examples
api

Expand Down
25 changes: 24 additions & 1 deletion docs/liblx/python-bindings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,30 @@ in your `PYTHONPATH`, e.g. `export PYTHONPATH=.:src/bindings/python`, assuming y
Then, in a Python session, you can say `from liblx import *`.


Typical build on Windows
========================
Assuming the dependencies etc are installed, you should be able to do this, in a normal Windows command prompt:

.. code-block:: bash

git clone https://github.com/sbmlteam/liblx.git
cd liblx
git checkout matt-win-build
git pull --rebase

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DWITH_PYTHON=ON -DLIBLX_DEPENDENCY_DIR=e:/Development/libSBML-dependencies/install_vs15_release_x64 C:\Users\cceagil\repos\CompBioLibs\liblx
cmake --build . --config Release

with, of course, your own values for the dependency directory and the build location (final item in long cmake command above).
This example assumes you have installed `cmake` directly, rather than using the version which comes with Visual Studio.
NB there are additional steps required; also, the first ``cmake`` command is likely to be a little different.
For more details, see `Complete Windows Example <../quickstart/complete-windows-example.html>`_.


Sample Python Session
=====================
python

Using an XML fragment like:

Expand All @@ -38,6 +59,8 @@ Using an XML fragment like:
In this example, the namespace URI is http://test.org, and `id` is an attribute with value "test1".
The elements are `annotation` and `test`. The value of the `test` element is "test2".

You will have to set your python path (either PYTHONPATH env var, or within python session) to pick up the built liblx modules.

.. code-block:: python

>>> from liblx import *
Expand Down
266 changes: 266 additions & 0 deletions docs/quickstart/complete-windows-example.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,266 @@
Complete Windows Example
========================

.. _building_cli:

Using the command line
----------------------
This is a complete Windows example, building the ``liblx`` library, the Python bindings, and the documentation (both docs such as this
one, and Doxygen-generated ones from the ``liblx`` source files). This is because I think some of the other documentation
has become a little inconsistent, as I wrote it whilst battling different Windows build issues.

We assume you have cloned the ``liblx`` repository on your local Windows computer.

The term "directory", in the following instructions, is synonymous with the Windows term "folder".

Install `Microsoft Visual Studio <https://visualstudio.microsoft.com/vs/>`_, with the Windows SDK. You can then use its version of ``cmake`` (see below).

Download the `SBML Windows dependencies <https://sourceforge.net/projects/sbml/files/libsbml/win-dependencies/>`_.
You can download all 4 and use as required. Right now there is no "version 16", the version number of my Visual Studio,
so I downloaded the highest available one (15). Make sure you check the checksums.

The most commonly downloaded one is ``libSBML_dependencies_vs15_release_x64_static.zip``, and we will use it in this example.
In this example, I unzipped it and renamed the unzipped folder to:

``C:\Users\cceagil\repos\CompBioLibs\dependencies\libSBML-Dependencies-1.0.0-b1-win64-release-static``

Use Python to create a virtual environment. Do this outside the repo you have cloned, otherwise lots of unnecessary files
will be generated in the documentation step

.. code-block:: bash

> mkdir envts && cd envts
> python -m venv myenv
> myenv\Scripts\activate
(myenv)

(Use the command ``deactivate`` if you need to exit the virtual environment.)

If you have installed ``cmake``, you can use it directly. If not, you can use the version which comes with Visual Studio.
See `Using cmake at the command line with Visual Studio <https://docs.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-160#run-cmake-from-the-command-line>`_
and `Building on the command line <https://docs.microsoft.com/en-us/cpp/build/building-on-the-command-line?view=msvc-160>`_.

**NB ensure you select the correct version of the documentation for your version of Visual Studio.**

Basically, to use the VS version of ``cmake``, you have to ``cd`` into the relevant directory on your PC,
and then execute the appropriate ``.bat`` file to update the ``PATH`` and required environment variables.
e.g.

.. code-block:: bash

(myenv) cd C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build

Running the ``set`` command (lists environment variables) reveals ``PROCESSOR_ARCHITECTURE=AMD64`` and ``Platform=x64``
(or you might be able to get this information by right-clicking on "This PC" or "My Computer", and choosing the Properties option.)
So we need to `run the file vcvar64.bat (in this case) at the command line, in our existing command
window <https://docs.microsoft.com/en-us/cpp/build/building-on-the-command-line?view=msvc-160#use-the-developer-tools-in-an-existing-command-window>`_.
Obviously, based on your computer's arhitecture you may need to select a different ``.bat`` file.

.. code-block:: bash

(myenv) vcvars64.bat
**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.11.3
** Copyright (c) 2021 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

(myenv)

After that, we need the tools to build the documentation.
For the docs, do the following:

.. code-block:: bash

(myenv) pip install sphinx_rtd_theme
(myenv) pip install breathe

You will also need to download `Doxygen <https://www.doxygen.nl/download.html>`_; also see
info about `Windows builds <https://www.doxygen.nl/manual/install.html#install_bin_windows>`_.
And then update your ``PATH`` to include the location of the Doxygen executable (``doxygen.exe``). For example:

.. code-block:: bash

(myenv) set PATH="C:\Program Files\doxygen\bin";%PATH%

Now download `GraphViz <https://graphviz.org/download/>`_, and do the same.
e.g.

.. code-block:: bash

(myenv) set PATH="C:\Program Files\GraphViz\bin";%PATH%

In the above examples, I enclosed the new part of the ``PATH`` in quotes, due to the space in "Program Files",
but this may not be necessary.

For the C/C++-Python SWIG bindings, you need to install SWIG. On my PC, the executable is
``C:\Users\cceagil\swigwin-4.0.2\swigwin-4.0.2\swig.exe``. So we again update the ``PATH``:

.. code-block:: bash

(myenv) set PATH=C:\Users\cceagil\swigwin-4.0.2\swigwin-4.0.2;%PATH%

You can check the value of the ``PATH`` if desired:

.. code-block:: bash

(myenv) echo %PATH%
C:\Users\cceagil\swigwin-4.0.2\swigwin-4.0.2;"C:\Program Files\doxygen\bin";C:\Users\cceagil\venvs\venv\Scripts;C:\Program Files\Java\jdk1.8.0_291\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C;C:\Program Files\Git\cmd;C:\Program Files\Java\jdk1.8.0_291\bin;C:\Users\cceagil\AppData\Local\Programs\Python\Python39\Scripts\;C:\Users\cceagil\AppData\Local\Programs\Python\Python39\;C:\Users\cceagil\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Graphviz\bin

Now we need to set the ``CODE_SRC_DIR`` environment variable to the top of the ``liblx`` source file hierarchy:

.. code-block:: bash

(myenv) set CODE_SRC_DIR=C:\Users\cceagil\repos\CompBioLibs\liblx\src

Finally, we create a new build directory, outside of the ``liblx`` git repo directory hierarchy. Then enter it and execute ``cmake``:

.. code-block:: bash

(myenv) mkdir build
(myenv) cd build
(myenv) cmake -DCMAKE_BUILD_TYPE=Release -DWITH_PYTHON=ON -DWITH_STATIC_RUNTIME=ON -DWITH_DOXYGEN=TRUE -DWITH_CHECK=TRUE -DLIBLX_DEPENDENCY_DIR=C:\Users\cceagil\repos\CompBioLibs\dependencies\libSBML-Dependencies-1.0.0-b1-win64-release-static C:\Users\cceagil\repos\CompBioLibs\liblx
(myenv) cmake --build . --config Release

Note, in this case, that we want a static Release build, per the version of the SBML dependency libraries we are
linking against. So both the ``CMAKE_BUILD_TYPE`` and ``--config`` options are set to ``Release``. Also, we "switch on"
the ``WITH_STATIC_RUNTIME`` option. The (rather copious) ``cmake`` output is not shown. The final item in the first ``cmake`` command
is the top level directory of the ``liblx`` cloned repository.

Run the tests to check all is well; in this case, we created a Release build:

.. code-block:: bash

(myenv) ctest -C Release
Test project C:/Users/cceagil/repos/CompBioLibs/build
Start 1: test_sbml_xml_run
1/2 Test #1: test_sbml_xml_run ................ Passed 2.25 sec
Start 2: test_python_binding
2/2 Test #2: test_python_binding .............. Passed 0.48 sec

100% tests passed, 0 tests failed out of 2

Total Test time (real) = 2.75 sec

(myenv)

If more detailed output is required, also use the ``-V`` switch - so, in this case, it would be:


.. code-block:: bash

ctest -V -C Release

Now we can look at the documentation (in the ``build/docs/sphinx/quickstart`` folder, e.g.
`Quickstart <./get-started.html>`_ and ``build/docs/sphinx/quickstart/complete-windows-example.html`` (this file)).
You should also be able to view the `API documentation <../api.html>`_, formed by Doxygen and Sphinx from the relevant comments in the source files.

You can also now use the Python bindings (wrapper) to the ``liblx`` C/C++ code. For example, from within the ``build/`` directory:

.. code-block:: bash

(myenv) cd src\bindings\python

Invoking ``dir`` should show that ``liblx.py`` is visible, and we are in the directory
``build\src\bindings\python``. To work, this needs ``_liblx.pyd``, which in our case is in the
directory ``build\src\bindings\python\Release``. To use ``liblx`` within Python, we need to update our ``PYTHONPATH`` to
include this directory. We can do this inside Python:

.. code-block:: bash

(myenv) python
>>> import sys
>>> sys.path += ["C:\\Users\\cceagil\\repos\\CompBioLibs\\build\\src\\bindings\\python\\Release"]
>>> from liblx import *


The ``liblx`` Python library can then be used as per the example
`Sample Python Session <../liblx/python-bindings.html#sample-python-session>`_.


.. _building_dont-like-cli:

I don't want to use the command line!
-------------------------------------
If you don't like the command line, you can refer to the
`detailed instructions <http://sbml.org/Software/libSBML/5.18.0/docs/cpp-api/libsbml-installation.html#detailed-windows>`_
for building ``libSBML`` on Windows (which we can adapt for building ``liblx``). Use the CMake GUI for the first
step. Then, the second command above (the build (i.e. compilation) step) can be done from within the Visual Studio
GUI. The easiest way is to locate the "solution" file, ``liblx.sln``, which should have been generated in
the ``build`` directory; navigate to it using Windows Explorer, then double-click on it to open this solution
in Visual Studio (but see below). Then, right-click on the desired target (e.g. ``ALL_BUILD``) and select the build option.

Obviously you will still have to download the required dependencies and other software referred to in the command-line build section.


.. _other_xml:

Choice of XML library
---------------------

By default, ``liblx`` is compiled with the `libXML <http://xmlsoft.org/>`_ library. However, users can choose to use another XML library instead.
The others supported at the moment are the `Expat <https://github.com/libexpat/libexpat>`_ and `Xerces <http://xerces.apache.org/xerces-c/>`_ libraries.

The required files are included in the SBML Windows dependencies, mentioned near the beginning of this page. If you wish to use another one, please refer to the table below.

.. list-table:: Choice of XML library
:widths: 20 80
:header-rows: 1

* - Preferred library
- Changes required to first ``cmake`` command above
* - libXML
- (no changes required)
* - Expat
- Add ``-DWITH_EXPAT=TRUE -DWITH_LIBXML=FALSE`` to first ``cmake`` command
* - Xerces
- Add ``-DWITH_XERCES=TRUE -DWITH_LIBXML=FALSE`` to first ``cmake`` command


.. _windows-issues:

Windows issues with the SWIG Python build
-----------------------------------------

Things should work OK, as detailed above, but this section is a record of some issues I had when battling to get a successful Windows
build, in case it helps someone.

Basically, don't use Anaconda Python to get this to work! (at least, not a Debug build).

.. code-block:: bash

set PYTHON_INCLUDE=C:\ProgramData\Anaconda3\include
set PYTHON_LIB=C:\ProgramData\Anaconda3\libs\python38.lib
-DSWIG_EXECUTABLE=C:\Users\mattg\swigwin-4.0.2\swig.exe
produces src/bindings/python/liblx.py

linker error:
LINK : fatal error LNK1104: cannot open file 'python38_d.lib' [C:\Users\mattg\build\src\bindings\python\binding_python_
lib.vcxproj]


Maybe because I specified a debug version of the dependencies???
see:
https://stackoverflow.com/questions/59126760/building-a-python-c-extension-on-windows-with-a-debug-python-installation

and:
https://stackoverflow.com/questions/17028576/using-python-3-3-in-c-python33-d-lib-not-found/45407558

It looks like we need to download a debug version of the python library. Anaconda doesn't appear to supply this.
Downloading Windows installer of Python 3.9.7 https://www.python.org/downloads/release/python-397/
Or, one can use ``#ifdef`` statements.
The installer updated the ``PATH`` (selected option to disable max ``PATH`` character limit) and appears before the
Anaconda version in the ``PATH``.

.. code-block:: bash

set PYTHON_INCLUDE="C:\Program Files\Python39\include" # location of Python.h
set PYTHON_LIB="C:\Program Files\Python39\libs\python39_d.lib" # debug library
-DPYTHON_EXECUTABLE="C:\Program Files\Python39\python.exe"
rm -rf ~/repos/work/CompBioLibs/liblx/out # delete vs cmake cache Visual Studio: Project-> cmake cache->delete cache

LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library [C:\Users\mattg
\build\src\liblx\xml\test\test_sbml_xml.vcxproj]

https://stackoverflow.com/questions/3007312/resolving-lnk4098-defaultlib-msvcrt-conflicts-with

Loading