Skip to content

Commit

Permalink
Add pybind11 and Python layers to lightning.tensor (#748)
Browse files Browse the repository at this point in the history
### Before submitting

Please complete the following checklist when submitting a PR:

- [x] All new features must include a unit test.
If you've fixed a bug or added code that should be tested, add a test to
the
      [`tests`](../tests) directory!

- [x] All new functions and code must be clearly commented and
documented.
If you do make documentation changes, make sure that the docs build and
      render correctly by running `make docs`.

- [x] Ensure that the test suite passes, by running `make test`.

- [x] Add a new entry to the `.github/CHANGELOG.md` file, summarizing
the
      change, and including a link back to the PR.

- [x] Ensure that code is properly formatted by running `make format`. 

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.


------------------------------------------------------------------------------------------------------------

**Context:**

Add both `python` and `pybind` layer to the `lightning.tensor`-`mps`
device.

1. Gates supported: Operators that have 1 or 2 wires or can be
decomposed into 1-2 wires gates. `MultiRZ` gate is not supported.
2. Obs supported: All obs supported in `lightning` state vector backends
except for `qml.Projector` and `SparseHamiltionian`
3. Both `qml.QubitStateVector` and `qml.StatePrep` are not supported.
4. `qml.state()` is not supported and the `state()` returns a
`LightningStateTensor` object represents the whole compute graph
5. `qml.BasisState` only accepts array representation of a basis state.
A `int` value can only supports up to `32` qubit system.
6. `qml.expval` is the only measurement `lightning.tensor` support.
7. Limited `gradient support` with `parameter-shift`. And a `100` qubits
SEL circuit was tested on `A100 80 GB GPU`
8. `maxBondDim` is default as `128`.
9. `pip` editable installation with `PL_BACKEND="lightning_tensor" pip
install -e .`, given that `cutensornet-cu12` is installed an `venv` env.
Detailed instructions are on[
PR#756](#756).
10. `lightning.tensor` is also tested against all the tests in `tests/`
if the tested feature is supported in `lightning.tensor`. The skipped
tests are tasks for the next step.
11. 

[SC-62440] [SC-62439] [SC-60329] [SC-60325]

**Description of the Change:**

**Benefits:**

**Possible Drawbacks:**

**Related GitHub Issues:**

---------

Co-authored-by: ringo-but-quantum <[email protected]>
Co-authored-by: Ali Asadi <[email protected]>
Co-authored-by: Mudit Pandey <[email protected]>
Co-authored-by: Vincent Michaud-Rioux <[email protected]>
Co-authored-by: Vincent Michaud-Rioux <[email protected]>
Co-authored-by: Amintor Dusko <[email protected]>
  • Loading branch information
7 people authored Jun 12, 2024
1 parent f57df46 commit 75d9c92
Show file tree
Hide file tree
Showing 47 changed files with 2,260 additions and 228 deletions.
3 changes: 3 additions & 0 deletions .github/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Release 0.37.0-dev

### New features since last release
* Implement Python interface to the `lightning.tensor` device.
[(#748)](https://github.com/PennyLaneAI/pennylane-lightning/pull/748)

* Add `inverse` support for gate operations in `lightning.tensor` in the C++ layer.
[(#753)](https://github.com/PennyLaneAI/pennylane-lightning/pull/753)

Expand Down
208 changes: 208 additions & 0 deletions .github/workflows/tests_lmps_tncuda_python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
name: Testing::Linux::x86_64::MPSTNCuda::Python
# TODO remove MPS from the workflow name and the workflow filename once exact is in.
# TODO remove the `pl_tensor_method` once exact TN is added.
on:
workflow_call:
inputs:
lightning-version:
type: string
required: true
description: The version of Lightning to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master)
pennylane-version:
type: string
required: true
description: The version of PennyLane to use. Valid values are either 'release' (most recent release candidate), 'stable' (most recent git-tag) or 'latest' (most recent commit from master)
release:
pull_request:
paths-ignore:
- .github/**
- '!.github/workflows/tests_lmps_tncuda_python.yml'
- pennylane_lightning/core/src/simulators/lightning_kokkos/**
- pennylane_lightning/core/src/simulators/lightning_qubit/**
- pennylane_lightning/core/src/simulators/lightning_gpu/**
- pennylane_lightning/core/_version.py
- pennylane_lightning/lightning_gpu/**
- pennylane_lightning/lightning_qubit/**
- pennylane_lightning/lightning_kokkos/**
push:
branches:
- master

env:
CI_CUDA_ARCH: 86
COVERAGE_FLAGS: "--cov=pennylane_lightning --cov-report=term-missing --cov-report=xml:./coverage.xml --no-flaky-report -p no:warnings --tb=native"
GCC_VERSION: 11

concurrency:
group: tests_lmps_tncuda_python-${{ github.ref }}-${{ github.event }}-${{ inputs.lightning-version }}-${{ inputs.pennylane-version }}
cancel-in-progress: true

jobs:
builddeps:
runs-on:
- self-hosted
- ubuntu-22.04
- gpu

strategy:
max-parallel: 1
matrix:
os: [ubuntu-22.04]
pl_backend: ["lightning_tensor"]
cuda_version: ["12"]

steps:
- name: Validate GPU version and installed compiler
run: |
source /etc/profile.d/modules.sh
module use /opt/modules
module load cuda/${{ matrix.cuda_version }}
echo "${PATH}" >> $GITHUB_PATH
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> $GITHUB_ENV
nvcc --version
nvidia-smi
pythontestswithMPSTNCuda:
if: ${{ !contains(fromJSON('["schedule", "workflow_dispatch"]'), github.event_name) }}
needs: [builddeps]
strategy:
matrix:
os: [ubuntu-22.04]
pl_backend: ["lightning_tensor"]
default_backend: ["lightning_qubit"]
pl_tensor_method: ["mps"]
pl_tensor_backend: ["cutensornet"]
cuda_version: ["12"]

name: Python Tests (${{ matrix.pl_backend }}, method-${{ matrix.pl_tensor_method }}, backend-${{ matrix.pl_tensor_backend }}, cuda-${{ matrix.cuda_version }})
runs-on:
- ${{ matrix.os }}
- self-hosted
- gpu

steps:
- name: Validate GPU version and installed compiler
run: |
source /etc/profile.d/modules.sh
module use /opt/modules
module load cuda/${{ matrix.cuda_version }}
echo "${PATH}" >> $GITHUB_PATH
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> $GITHUB_ENV
nvcc --version
nvidia-smi
- name: Checkout PennyLane-Lightning-Tensor-MPS-TNCuda
uses: actions/checkout@v4
with:
path: main

- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.9'

# Since the self-hosted runner can be re-used. It is best to set up all package
# installations in a virtual environment that gets cleaned at the end of each workflow run
- name: Setup Python virtual environment
id: setup_venv
env:
VENV_NAME: ${{ github.workspace }}/venv_${{ steps.setup_python.outputs.python-version }}_${{ github.sha }}
run: |
# Clear any pre-existing venvs
rm -rf venv_*
# Create new venv for this workflow_run
python --version
python -m venv ${{ env.VENV_NAME }}
# Add the venv to PATH for subsequent steps
echo ${{ env.VENV_NAME }}/bin >> $GITHUB_PATH
# Adding venv name as an output for subsequent steps to reference if needed
echo "venv_name=${{ env.VENV_NAME }}" >> $GITHUB_OUTPUT
- name: Display Python-Path
id: python_path
run: |
py_path=$(which python)
echo "Python Interpreter Path => $py_path"
echo "python=$py_path" >> $GITHUB_OUTPUT
pip_path=$(which python)
echo "PIP Path => $py_path"
echo "pip=$py_path" >> $GITHUB_OUTPUT
- name: Install required packages
run: |
cd main
python -m pip install -r requirements-dev.txt
python -m pip install ninja cmake scipy custatevec-cu${{ matrix.cuda_version }} cutensornet-cu${{ matrix.cuda_version }} openfermionpyscf
- name: Checkout PennyLane for release build
if: inputs.pennylane-version == 'release'
uses: actions/checkout@v4
with:
path: pennylane
repository: PennyLaneAI/pennylane

- name: Switch to release build of PennyLane
if: inputs.pennylane-version == 'release'
run: |
cd pennylane
git fetch --all
git checkout $(git branch -a --list "origin/v*rc*" | tail -1)
python -m pip uninstall -y pennylane && python -m pip install . -vv --no-deps
- name: Install Stable PennyLane
if: inputs.pennylane-version == 'stable'
run: |
cd main
python -m pip uninstall -y pennylane && python -m pip install -U pennylane
- name: Build and install package
run: |
cd main
rm -rf build
CMAKE_ARGS="-DPL_BACKEND=${{ matrix.default_backend }} -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \
python -m pip install . -vv
rm -rf build
rm -rf build
CMAKE_ARGS="-DPL_BACKEND=${{ matrix.pl_backend }} -DCMAKE_CXX_COMPILER=$(which g++-$GCC_VERSION)" \
python -m pip install . -vv

- name: Run PennyLane-Lightning-Tensor unit tests
if: ${{ matrix.pl_backend != 'all'}}
run: |
cd main/
DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"`
PL_DEVICE=${DEVICENAME} python -m pytest tests $COVERAGE_FLAGS
mv coverage.xml coverage-${{ github.job }}-${{ matrix.pl_backend }}.xml
- name: Upload code coverage results
uses: actions/upload-artifact@v3
with:
name: ubuntu-codecov-results-python
path: ./main/coverage-${{ github.job }}-${{ matrix.pl_backend }}.xml
if-no-files-found: error


upload-to-codecov-linux-python:
needs: [pythontestswithMPSTNCuda]
name: Upload coverage data to codecov
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Download coverage reports
uses: actions/download-artifact@v3
with:
name: ubuntu-codecov-results-python

- name: Upload to Codecov
uses: codecov/codecov-action@v4
with:
fail_ci_if_error: true
verbose: true
token: ${{ secrets.CODECOV_TOKEN }}
6 changes: 3 additions & 3 deletions .github/workflows/tests_without_binary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
runs-on: ${{ needs.determine_runner.outputs.runner_group }}
strategy:
matrix:
pl_backend: ["lightning_qubit", "lightning_kokkos", "lightning_gpu"]
pl_backend: ["lightning_qubit", "lightning_kokkos", "lightning_gpu", "lightning_tensor"]

name: Python Tests without Binary (${{ matrix.pl_backend }})

Expand Down Expand Up @@ -97,7 +97,7 @@ jobs:
python -m pip uninstall -y pennylane && python -m pip install -U pennylane
- name: Install the pennylane_lightning package
if: ${{ matrix.pl_backend == 'lightning_kokkos' || matrix.pl_backend == 'lightning_gpu'}}
if: ${{ contains(fromJson('["lightning_kokkos", "lightning_gpu", "lightning_tensor"]'), matrix.pl_backend) }}
run: |
cd main
SKIP_COMPILATION=True PL_BACKEND="lightning_qubit" python -m pip install . -vv
Expand All @@ -110,7 +110,7 @@ jobs:
cd main
python -m pip install . -vv
- name: Run PennyLane-Lightning unit tests
- name: Run PennyLane-Lightning unit tests for all backends
run: |
cd main/
DEVICENAME=`echo ${{ matrix.pl_backend }} | sed "s/_/./g"`
Expand Down
7 changes: 5 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ if(ENABLE_PYTHON)
pybind11_add_module("${PL_BACKEND}_ops" "pennylane_lightning/core/src/bindings/Bindings.cpp")

# Allow pip installation of cuQuantum & CUDA 12 libs to be accessible without setting LD_LIBRARY_PATH for lightning_gpu
if("${PL_BACKEND}" STREQUAL "lightning_gpu")
if("${PL_BACKEND}" STREQUAL "lightning_gpu" OR "${PL_BACKEND}" STREQUAL "lightning_tensor")
set(CMAKE_BUILD_RPATH_USE_ORIGIN ON)
set_target_properties("${PL_BACKEND}_ops" PROPERTIES BUILD_RPATH "$ORIGIN/../cuquantum/lib:$ORIGIN/../nvidia/cuda_runtime/lib:$ORIGIN/../nvidia/cublas/lib:$ORIGIN/../nvidia/cusparse/lib:$ORIGIN")
endif()
Expand All @@ -171,12 +171,15 @@ if(ENABLE_PYTHON)
)

target_link_libraries("${PL_BACKEND}_ops" PRIVATE ${PL_BACKEND} #simulator
"${PL_BACKEND}_algorithms"
"${PL_BACKEND}_observables"
"${PL_BACKEND}_bindings"
"${PL_BACKEND}_measurements"
)

if(NOT DEFINED PL_TENSOR)
target_link_libraries("${PL_BACKEND}_ops" PRIVATE "${PL_BACKEND}_algorithms")
endif()

set_target_properties("${PL_BACKEND}_ops" PROPERTIES CXX_VISIBILITY_PRESET hidden)
target_compile_definitions("${PL_BACKEND}_ops" PRIVATE VERSION_INFO=${VERSION_STRING})
endif()
Expand Down
31 changes: 26 additions & 5 deletions pennylane_lightning/core/_serialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ class QuantumScriptSerializer:
"""

# pylint: disable=import-outside-toplevel, too-many-instance-attributes, c-extension-no-member
# pylint: disable=import-outside-toplevel, too-many-instance-attributes, c-extension-no-member, too-many-branches, too-many-statements
def __init__(
self, device_name, use_csingle: bool = False, use_mpi: bool = False, split_obs: bool = False
):
Expand Down Expand Up @@ -86,10 +86,23 @@ def __init__(
raise ImportError(
f"Pre-compiled binaries for {device_name} are not available."
) from exception
elif device_name == "lightning.tensor":
try:
import pennylane_lightning.lightning_tensor_ops as lightning_ops
except ImportError as exception:
raise ImportError(
f"Pre-compiled binaries for {device_name} are not available."
) from exception
else:
raise DeviceError(f'The device name "{device_name}" is not a valid option.')
self.statevector_c64 = lightning_ops.StateVectorC64
self.statevector_c128 = lightning_ops.StateVectorC128

if device_name == "lightning.tensor":
self.tensornetwork_c64 = lightning_ops.TensorNetC64
self.tensornetwork_c128 = lightning_ops.TensorNetC128
else:
self.statevector_c64 = lightning_ops.StateVectorC64
self.statevector_c128 = lightning_ops.StateVectorC128

self.named_obs_c64 = lightning_ops.observables.NamedObsC64
self.named_obs_c128 = lightning_ops.observables.NamedObsC128
self.hermitian_obs_c64 = lightning_ops.observables.HermitianObsC64
Expand All @@ -98,8 +111,10 @@ def __init__(
self.tensor_prod_obs_c128 = lightning_ops.observables.TensorProdObsC128
self.hamiltonian_c64 = lightning_ops.observables.HamiltonianC64
self.hamiltonian_c128 = lightning_ops.observables.HamiltonianC128
self.sparse_hamiltonian_c64 = lightning_ops.observables.SparseHamiltonianC64
self.sparse_hamiltonian_c128 = lightning_ops.observables.SparseHamiltonianC128

if device_name != "lightning.tensor":
self.sparse_hamiltonian_c64 = lightning_ops.observables.SparseHamiltonianC64
self.sparse_hamiltonian_c128 = lightning_ops.observables.SparseHamiltonianC128

self._use_mpi = use_mpi

Expand Down Expand Up @@ -134,6 +149,8 @@ def sv_type(self):
"""State vector matching ``use_csingle`` precision (and MPI if it is supported)."""
if self._use_mpi:
return self.statevector_mpi_c64 if self.use_csingle else self.statevector_mpi_c128
if self.device_name == "lightning.tensor":
return self.tensornetwork_c64 if self.use_csingle else self.tensornetwork_c128
return self.statevector_c64 if self.use_csingle else self.statevector_c128

@property
Expand Down Expand Up @@ -286,6 +303,10 @@ def _ob(self, observable, wires_map: dict = None):
if isinstance(observable, OP_MATH_OBS):
return self._hamiltonian(observable, wires_map)
if isinstance(observable, SparseHamiltonian):
if self.device_name == "lightning.tensor":
raise NotImplementedError(
"SparseHamiltonian is not supported on the lightning.tensor device."
)
return self._sparse_hamiltonian(observable, wires_map)
return self._hermitian_ob(observable, wires_map)

Expand Down
2 changes: 1 addition & 1 deletion pennylane_lightning/core/_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@
Version number (major.minor.patch[-label])
"""

__version__ = "0.37.0-dev32"
__version__ = "0.37.0-dev33"
Loading

0 comments on commit 75d9c92

Please sign in to comment.