Add Maestro 1.37.x support

[JampaUchoa]

Description:
Unable to use the library with the latest Maestro version (1.37.x), the latest being 1.37.9.

In the staging app, running in production mode, the isMaestro() function always returns false using version 3.0.1.

Steps to Reproduce:

1. Update the library to Maestro version 1.37.x (latest being 1.37.9).
2. Deploy the staged app in production mode.
3. Check the output of the isMaestro() function.

Expected Behavior:

• The isMaestro() function should return true when using the Maestro version 1.37.x, provided Maestro detection criteria are met.

Current Behavior:

• The isMaestro() function consistently returns false with version 3.0.1.

React Native version: 0.74.2

[joepatosh]

I have been investigating this bug. I have tracked it down to a recent change in maestro that caused the default port to change from 22087 to 7001.

See here for details: mobile-dev-inc/maestro#1816 (comment)

Easy fix (in theory)

So in theory, the fix should be as simple as changing:;

bool isReachable = [self isUrlReachable:@"http://localhost" port:22087];

to:

bool isReachable = [self isUrlReachable:@"http://localhost" port:7001];

This change does work in my testing locally. I've opened a PR with this potential fix in #17

However, I'm not convinced this will work in maestro cloud or any environment that using sharding. See below for details.

It might not be so easy...

The root cause of this problem is a recent change to maestro (#1732) that added sharding / parallel execution.

The direct bug from that change was that the default port (when not using sharding) changed from 22087 to 7001. That is the problem we see here. This problem is easy enough to handle, as described above.

However, that fix might not be durable enough to work in situations where sharding is enabled. I assume that maestro cloud uses sharding, but I do not know that for certain. My concern is that if maestro cloud uses sharding, than the fix suggested above won't work reliably when running tests in maestro cloud.

To be clear: it won't work in any environment that uses sharding. This is because when sharding is enabled, maestro picks random ports in the range (7001..7128):

private fun selectPort(effectiveShards: Int): Int =
        if (effectiveShards == 1) parent?.port ?: 7001
        else (7001..7128).shuffled().find { port ->
            usedPorts.putIfAbsent(port, true) == null
        } ?: error("No available ports found")

This is from maestro-cli/src/main/java/maestro/cli/command/TestCommand.kt

Alternative workarounds or solutions

Using --port flag

If you are running maestro directly yourself (i.e., not using maestro cloud) and you are not using sharding, you can control which port is used by using the top level --port argument. You can force maestro on port 22087 by running it like this:

./maestro --port=22087 test ../react-native-is-maestro/example

This only works when using a single shard. If using more than one shard, the --port arugment has no effect and ports in the range (7001..7128) will be used instead.

This is becasue selectPort references parent?.port:

private fun selectPort(effectiveShards: Int): Int =
        if (effectiveShards == 1) parent?.port ?: 7001
        else (7001..7128).shuffled().find { port ->
            usedPorts.putIfAbsent(port, true) == null
        } ?: error("No available ports found")

parent?.port in this case is App.port in maestro-cli/src/main/java/maestro/cli/App.kt - which can be controlled by passing arugments directly to maestro. Note that this --port arugment needs to come before the subcommand.

This works:

./maestro --port=22087 test ../react-native-is-maestro/example

This does not work:

./maestro test --port=22087 ../react-native-is-maestro/example

Using launch arguments instead (my recommended solution)

Personally, this is my preferred choice going forward. I am using launch arguments to detect if my app is running in maestro rather than checking ports.

I know this this is not directly a solution for the react-native-is-maestro, but I want to include it as an alternative option to help others who are in the same situation as me.

My reason for preferring this option is that it seems simpler and more reliable. As we can see from the discussion above:

• Relying on checking a port can be broken by changes to maestro
• The addition of sharding means we might not be able to check only a single port

On the other hand, it seems less likely that Launch Arguments will stop working due to a change in maestro.

So my solution:

I use the react-native-launch-arguments package:

import { LaunchArguments } from 'react-native-launch-arguments'

interface ExpectedArgs {
  runningInMaestro?: string
}

export const isRunningE2eTests = (): boolean => {
  const launchArgs = LaunchArguments.value<ExpectedArgs>()
  const { runningInMaestro } = launchArgs

  return !!runningInMaestro
}

I launch my app like this:

- launchApp:
    appId: 'com.myapp'
    arguments:
      runningInMaestro: 'yes'

I originally tried to use a boolean (i.e., runningInMaestro: true ) but that did not seem to work. For some reason, passing strings via arguments worked, but not booleans. I did not bother to investigate further since a string serves my purposes just fine.

This is working for me both locally and on maestro cloud.

Hope this help get some people unblocked on this!