Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTML test reports display as ai report with no test execution summary & screenshot when no --output is provided, even with no ai commands #2098

Open
1 task done
DavidREntwistle opened this issue Oct 16, 2024 · 2 comments

Comments

@DavidREntwistle
Copy link

DavidREntwistle commented Oct 16, 2024

Is there an existing issue for this?

  • I have searched the existing issues and didn't find mine.

Steps to reproduce

  1. Ensure no ai commands are within yourTestFlow.yaml file
  2. Run maestro test --format html yourTestFlow.yaml without any --output file location
  3. Observe the ~/.maestro/tests/<test-timestamp>/ai-report-yourTestFlow.html file

Actual results

HTML test reports are displaying as ai reports when no --output file location is provided:

AI suggestions for flow yourTestFlow
→ Open other report
[→ Open flow file ( yourTestFlow.yaml )]

0 possible defects found

E.g.:

  • maestro test --format html yourTestFlow.yaml = ~/.maestro/tests/<test-timestamp>/ai-report-yourTestFlow.html
  • maestro test --format html --output /testFolder/test-report.html yourTestFlow.yaml = ~/testFolder/test-report.html

Expected results

HTML test reports display the test execution summary & screenshot with and without the --output file location.
E.g.:

  • maestro test --format html yourTestFlow.yaml = ~/.maestro/tests/<test-timestamp>/yourTestFlow.html
  • maestro test --format html --output /testFolder/test-report.html yourTestFlow.yaml = ~/testFolder/test-report.html

See image example: #1750 (comment)

About app

  • Native iOS & Android app

About environment

  • MacBook Pro M2 Max - 32 GB
  • macOS Sonoma 14.7
  • arm64

Logs

N/a

Maestro version

1.39.0

How did you install Maestro?

install script (https://get.maestro.mobile.dev)

Anything else?

No response

@DavidREntwistle DavidREntwistle changed the title HTML test reports display as ai report with no test execution summary & screenshot, even with no ai commands HTML test reports display as ai report with no test execution summary & screenshot when no --output is provided, even with no ai commands Oct 16, 2024
@DavidREntwistle
Copy link
Author

ooked into this a bit more, it seems to only work when we include the output e.g. maestro test --format html --output test-report.html yourTestFlow.yaml. This then created the correct flowNameHere.html html report file instead of the ai-report-flowNameHere.html file within the ~/.maestro/tests/ path.

Updated title/details with this info now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant