Add documentation for testing

_quarto.yml

@@ -98,6 +98,8 @@ website:
             href: api/express/index.qmd
           - text: "Shiny Core"
             href: api/core/index.qmd
+          - text: "Testing"
+            href: api/testing/index.qmd
     tools:
       - icon: discord
         href: https://discord.gg/yMGCamUMnS
@@ -253,6 +255,10 @@ website:
           contents:
             - docs/modules.qmd
             - docs/module-communication.qmd
+        - section: "Testing"
+          contents:
+            - docs/testing.qmd
+            - docs/playwright-testing.qmd
         - section: "Extending"
           contents:
             - docs/custom-component-one-off.qmd

docs/playwright-testing.qmd

@@ -0,0 +1,101 @@
+---
+title: End-to-end Testing using Playwright
+editor:
+  markdown:
+    wrap: sentence
+---
+
+### What are End-to-end Tests?
+
+End-to-end testing for Shiny apps is like checking your app from start to finish, just as a user would.
+
+Imagine you're using your Shiny app. You click buttons, enter data, and see results on a graph or a dashboard. End-to-end tests mimic these actions.
+Instead of manually clicking around, we write code to do this for us. The code interacts with your app like a user, checking if everything works as expected.
+
+#### Benefits
+- End-to-end tests find issues early, like broken links or unexpected behavior.
+- As your app grows, it becomes harder to keep track of all parts. Tests help ensure nothing breaks.
+
+
+### Playwright
+
+***Playwright*** is an open-source library developed by Microsoft. It enables developers to automate browser interactions and perform end-to-end testing of web applications.
+
+Benefits of using Playwright for Shiny App testing
+
+- **End-to-End Testing**: Playwright allows you to simulate real user interactions with your Shiny app, ensuring that the reactive components and user flows work as expected.
+- **Cross-Browser Testing**: Playwright supports multiple browsers like Chromium, Firefox, and Safari(Webkit), enabling you to test your Shiny app's compatibility across different browser environments.
+- **Dynamic wait times** Playwright provides dynamic wait times, automatically waiting for elements to be ready before interacting with them, which eliminates the need for explicit waits and reduces flakiness caused by timing issues.
+
+For detailed information and guidance, check out the [official Playwright documentation](https://playwright.dev/python/).
+
+### Getting started with writing your first end-to-end test
+
+Assume you have a shiny app the doubles the slider value with the code shown below:
+
+```python
+# app.py
+from shiny import render, ui
+from shiny.express import input
+
+ui.panel_title("Hello Shiny!")
+ui.input_slider("n", "N", 0, 100, 20)
+
+
+@render.text
+def txt():
+    return f"n*2 is {input.n() * 2}"
+```
+
+If we want to test that the shiny app works for the following scenario:
+
+1. Wait for the Shiny app to finish loading
+1. Drag the slider to value as `55`
+1. Verify the output text changes to reflect the value of `n*2 is 110`
+
+The test code to test the shiny app to emulate the above scenario would be as following:
+
+```python
+# test_basic_app.py
+from shiny.playwright.controls import OutputText, InputSlider
+from shiny.run import ShinyAppProc
+from playwright.sync_api import Page
+from shiny.pytest import create_app_fixture
+
+app = create_app_fixture("remote/basic-app/app.py")
+
+
+def test_basic_app(page: Page, app: ShinyAppProc):
+    page.goto(app.url)
+    txt = OutputText(page, "txt")
+    slider = InputSlider(page, "n")
+    slider.set("55")
+    txt.expect_value("n*2 is 110")
+```
+
+#### Explanation of the test code:
+
+1. The code starts by importing necessary modules and classes. ***OutputText*** and ***InputSlider*** are classes that represent UI controls in the Shiny application.
+
+2. Defines ***test_remote_basic_app*** function with ***page*** and ***app*** parameters. *page* is an instance of the Page class from the Playwright library, which represents a single tab in a browser, and *app* is an instance of the ShinyAppProc class, which represents the Shiny app being tested.
+
+3. Navigates to the app's URL.
+
+4. Creates instances of ***OutputText*** and ***InputSlider*** for UI elements.
+
+5. Sets the slider value to `55`.
+
+6. Checks if the output text displays `n*2 is 110` as expected.
+
+![](assets/end-to-end-test-workflow.png)
+
+### Using `shiny add test` to create test files for your shiny apps
+
+`Shiny` provides a simple way to create a test file for your shiny app. Just type `shiny add test` in your terminal/console and give the **path** to the app directory along with the **name** of the test file.
+
+```bash
+shiny add test
+
+#TODO - add other lines here
+```
+

docs/testing.qmd

@@ -0,0 +1,96 @@
+---
+title: Testing
+editor:
+  markdown:
+    wrap: sentence
+---
+
+Testing Shiny apps is important to ensure the shiny app functions as expected and to catch any bugs or errors before deployment. It helps maintain code quality, user experience, and prevents potential issues from reaching the end-users.
+
+### pytest
+
+For the example below, we will use ***pytest*** as the test framework for running our unit tests. pytest is a popular, open-source testing framework for Python. It is designed to simplify the process of writing, organizing, and running tests for Python applications and libraries.
+
+More information about ***pytest*** can be found [here](https://docs.pytest.org/en/8.2.x/contents.html).
+
+::: {.callout-tip collapse="true"}
+## Auto discovery for tests
+
+pytest automatically discovers and runs tests in your project based on a naming convention (files or functions starting with `test_*.py` or ending with `*_test.py`), eliminating the need for manual test case registration. More information about test discovery can be found [here](https://docs.pytest.org/en/8.2.x/explanation/goodpractices.html#test-discovery)
+:::
+
+
+Given a shiny app that has the following code that doubles the number for any input that a user provides.
+
+```python
+# app.py
+from shiny import render, ui
+from shiny.express import input
+
+ui.panel_title("Double your amount")
+ui.input_text("txt_box", "Enter number to double it below")
+
+
+@render.text
+def txt():
+    if input.txt_box() == "":
+        return "Please enter a number"
+    # check if input is an int or float
+    try:
+        int(input.txt_box())
+    except ValueError:
+        return "Please enter a valid number"
+    return f"n*2 is {double_number(input.txt_box())}"
+
+def double_number(value: str):
+    return int(value) * 2
+```
+
+If you want to test the logic of a function that doubles a number, you can create a test file named `test_double_number.py`. This file will contain the necessary code to verify that the function works as expected.
+
+```python
+# test_double_number.py
+
+from app import double_number
+
+def test_double_number():
+    assert double_number("2") == 4
+    assert double_number("5") == 10
+    assert double_number("10") == 20
+    assert double_number("0") == 0
+    assert double_number("-5") == -10
+```
+
+To run the test, you will simply type `pytest` in your terminal. `pytest` will automatically locate the test file and run it with the results shown below.
+
+```bash
+platform darwin -- Python 3.10.12, pytest-7.4.4, pluggy-1.4.0
+configfile: pytest.ini
+plugins: asyncio-0.21.0, timeout-2.1.0, Faker-20.1.0, cov-4.1.0, playwright-0.4.4, rerunfailures-11.1.2, xdist-3.3.1, base-url-2.1.0, hydra-core-1.3.2, anyio-3.7.0, syrupy-4.0.5, shiny-1.0.0
+asyncio: mode=strict
+12 workers [1 item]
+.          [100%]
+(3 durations < 5s hidden.  Use -vv to show these durations.)
+```
+
+If the logic in the `double_number` is erroneous, and instead it triples the number, the test will catch it by showing the difference as shown below
+
+```bash
+======================================================= test session starts =======================================================
+platform darwin -- Python 3.10.12, pytest-7.4.4, pluggy-1.4.0
+configfile: pytest.ini
+plugins: asyncio-0.21.0, timeout-2.1.0, Faker-20.1.0, cov-4.1.0, playwright-0.4.4, rerunfailures-11.1.2, xdist-3.3.1, base-url-2.1.0, hydra-core-1.3.2, anyio-3.7.0, syrupy-4.0.5, shiny-1.0.0
+asyncio: mode=strict
+12 workers [1 item]
+F       [100%]
+======= FAILURES =======
+________ test_double_number ________
+
+    def test_double_number():
+>       assert double_number("2") == 4
+E       AssertionError: assert 6 == 4
+E        +  where 6 = double_number('2')
+
+```
+
+The tests mentioned earlier are suitable for testing non-reactive functions. However, when it comes to testing the reactivity of a Shiny app, we need to leverage a different approach. In this case, we can use ***Playwright*** to automate browser interactions and perform end-to-end testing of web applications.