Adding tests for all PennyLane transforms #1215

willjmax · 2024-10-18T13:51:24Z

This PR adds unit tests for all PennyLane transforms. This PR addresses [sc-72626].

…l add support to qml.qml.CosineWindow when it's called as the first op in tapes

Co-authored-by: David Ittah <[email protected]>

…I/catalyst into cosinewindow_catalyst_support

codecov · 2024-10-18T14:03:41Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.96%. Comparing base (0bcc64c) to head (2ca613b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1215   +/-   ##
=======================================
  Coverage   97.96%   97.96%           
=======================================
  Files          77       77           
  Lines       11244    11244           
  Branches      967      967           
=======================================
  Hits        11015    11015           
  Misses        180      180           
  Partials       49       49

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-10-18T17:38:57Z

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md on your branch with:

A one-to-two sentence description of the change. You may include a small working example for new features.
A link back to this PR.
Your name (or GitHub username) in the contributors section.

paul0403 · 2024-10-18T18:46:36Z

Thanks for adding this, I will take a closer look in a bit, but first @mlxd , what's the guideline on large batches of xfailed python tests like this? Do we prefer skip over xfail, to save CI execution time?

For context, @willjmax is going over the core PL transforms and testing whether qjit works with them or not. If they don't work, a reason is noted in this testing file.

Edit: discussed this; if we know something fails, but it shouldn't, we should always aim to mark it xfail.
Skipping means we have no signal at all of whether it will pass or fail

paul0403

Thanks for adding the tests 💯 Some questions from me

paul0403 · 2024-10-21T14:46:42Z

.dep-versions

@@ -15,4 +15,4 @@ pennylane=0.39.0.dev30
 # 'runtime/Makefile' and at all GitHub workflows, using the exact
 # commit hash corresponding to the merged PR that implements the
 # desired feature.
-lightning=0.39.0-dev38
+lightning=0.39.0-dev38


Unsure why this changed; we leave an empty line at the end of all files.

paul0403 · 2024-10-21T14:50:22Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="Not JAX JIT compatible")


In cases where jax jit fails but qjit succeeds, I think we should compare qjit with plain pennylane.

paul0403 · 2024-10-21T14:51:53Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="cond does not work with qml.RY")


Is this a pennylane error or qjit error?

paul0403 · 2024-10-21T14:52:04Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="cond does not work with qml.RY")


paul0403 · 2024-10-21T14:53:09Z

frontend/test/pytest/test_transform.py

    assert expected_shape == observed_shape


+@pytest.mark.xfail(
+    reason="ValueError: Eagerly computing the adjoint (lazy=False) is only supported on single operators."


Is this a pennylane error or a qjit error?

paul0403 · 2024-10-21T14:54:34Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="Something in pyzx not JAX JIT compatible.")


Does qjit work on this and jax jit fail, or both qjit and jax jit fail?

paul0403 · 2024-10-21T14:54:58Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="JIT and QJIT return different values.")


How about qjit and plain pennylane?

paul0403 · 2024-10-21T14:55:06Z

frontend/test/pytest/test_transform.py

@@ -121,44 +166,896 @@ def qfunc(data, x, weights):
    assert expected_shape == observed_shape


-def test_split_non_commuting(backend):
-    """Test split non commuting"""
+@pytest.mark.xfail(reason="JIT and QJIT give different results.")


How about qjit and plain pennylane?

paul0403 · 2024-10-21T14:56:19Z

frontend/test/pytest/test_transform.py

+class TestMitigate:
+    """Test error mitigation transforms"""
+
+    @pytest.mark.xfail(reason="JIT and QJIT return different values.")


How about qjit and plain pennylane?

paul0403 · 2024-10-21T14:56:27Z

frontend/test/pytest/test_transform.py

+        _, observed_shape = jax.tree_util.tree_flatten(observed)
+        assert expected_shape == observed_shape
+
+    @pytest.mark.xfail(reason="JIT and QJIT return different values.")


How about qjit and plain pennylane?

obliviateandsurrender · 2024-10-24T14:22:43Z

.dep-versions

@@ -15,4 +15,4 @@ pennylane=0.39.0.dev30
 # 'runtime/Makefile' and at all GitHub workflows, using the exact
 # commit hash corresponding to the merged PR that implements the
 # desired feature.
-lightning=0.39.0-dev38
+lightning=0.39.0-dev38


Suggested change

lightning=0.39.0-dev38

lightning=0.39.0-dev38

obliviateandsurrender · 2024-10-24T14:23:40Z

frontend/test/pytest/test_transform.py

 # pylint: disable=unnecessary-lambda-assignment
+# pylint: disable=too-many-lines
+# pylint: disable=line-too-long


Could be combined together.

obliviateandsurrender · 2024-10-24T17:19:08Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="cond does not work with qml.RY")


Will this work with applying qml.RY via a function, i.e., using callable for true branch? 🤔

I guess #449 really needs to be fixed! I will take a look.

The fix is likely non-trivial, so no need to wait for it.

obliviateandsurrender · 2024-10-24T17:19:58Z

frontend/test/pytest/test_transform.py

+    assert expected_shape == observed_shape
+
+
+@pytest.mark.xfail(reason="ValueError on Lightning")


What is this error?

maliasadi and others added 16 commits September 16, 2024 15:25

Fix the issue with lowering the StatePrepBase derived gates. This wil…

c6a7701

…l add support to qml.qml.CosineWindow when it's called as the first op in tapes

removing unrelated chage

ea071c4

reverting changes

df1f279

removing line

f23d5e2

linting

f6a2c83

Update .dep-versions

fac256f

Co-authored-by: David Ittah <[email protected]>

adding cosine window test

f8f8835

Merge branch 'cosinewindow_catalyst_support' of github.com:PennyLaneA…

213d4cc

…I/catalyst into cosinewindow_catalyst_support

Merge branch 'main' into cosinewindow_catalyst_support

94c69f6

linting

b8e4022

changing qjit_capabilities to capabilities

53f77f8

Merge branch 'main' into cosinewindow_catalyst_support

1aa34f2

reverting test file

00d6145

Merge branch 'cosinewindow_catalyst_support' of github.com:PennyLaneA…

a66463d

…I/catalyst into cosinewindow_catalyst_support

batch_partial test

e5eb1a6

adding tests

59d0b54

willjmax added 3 commits October 18, 2024 10:14

removing accidentally added changes

e6906bd

codefactor

15c7143

Merge branch 'main' into qjit-transforms

2ca613b

willjmax requested review from obliviateandsurrender and soranjh October 18, 2024 17:38

willjmax requested a review from paul0403 October 18, 2024 17:52

paul0403 reviewed Oct 21, 2024

View reviewed changes

obliviateandsurrender reviewed Oct 24, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding tests for all PennyLane transforms #1215

Adding tests for all PennyLane transforms #1215

willjmax commented Oct 18, 2024

codecov bot commented Oct 18, 2024 •

edited

Loading

github-actions bot commented Oct 18, 2024

paul0403 commented Oct 18, 2024 •

edited

Loading

paul0403 left a comment

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

paul0403 Oct 21, 2024

obliviateandsurrender Oct 24, 2024

obliviateandsurrender Oct 24, 2024

obliviateandsurrender Oct 24, 2024

paul0403 Oct 24, 2024 •

edited

Loading

paul0403 Oct 24, 2024

paul0403 Oct 25, 2024

obliviateandsurrender Oct 24, 2024

		assert expected_shape == observed_shape


		@pytest.mark.xfail(reason="Not JAX JIT compatible")

		assert expected_shape == observed_shape


		@pytest.mark.xfail(reason="cond does not work with qml.RY")

		assert expected_shape == observed_shape


		@pytest.mark.xfail(reason="Something in pyzx not JAX JIT compatible.")

		assert expected_shape == observed_shape


		@pytest.mark.xfail(reason="JIT and QJIT return different values.")

		assert expected_shape == observed_shape


		@pytest.mark.xfail(reason="ValueError on Lightning")

Adding tests for all PennyLane transforms #1215

Are you sure you want to change the base?

Adding tests for all PennyLane transforms #1215

Conversation

willjmax commented Oct 18, 2024

codecov bot commented Oct 18, 2024 • edited Loading

Codecov Report

github-actions bot commented Oct 18, 2024

paul0403 commented Oct 18, 2024 • edited Loading

paul0403 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paul0403 Oct 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Oct 18, 2024 •

edited

Loading

paul0403 commented Oct 18, 2024 •

edited

Loading

paul0403 Oct 24, 2024 •

edited

Loading