fix: solve the test issue (#20)

.github/workflows/unit.yaml

@@ -8,7 +8,8 @@ on:
   pull_request:
 
 env:
-  EARTHENGINE_TOKEN: ${{ secrets.EARTHENGINE_TOKEN }}
+  EARTHENGINE_SERVICE_ACCOUNT: ${{ secrets.EARTHENGINE_SERVICE_ACCOUNT }}
+  EARTHENGINE_PROJECT: ${{ secrets.EARTHENGINE_PROJECT }}
 
 jobs:
   lint:

docs/usage.rst

@@ -27,11 +27,6 @@ To do so, the lib will provide a number of connection methods that will hopefull
     All the methods presented in this section will fallback to a regular ``ee.Initialize()`` if the environment parameter are not found.
     This means that you can use this plugin in your local environment without having to change anything as long as the ``ee`` module is installed and that you already run once ``ee.Authenticate()``.
 
-Private Token
-^^^^^^^^^^^^^
-
-The first method is to use a private token. This is the easiest way to connect to GEE in a CI/CD environment.
-
 .. danger::
 
     Never forget that this method can potentially expose your personal credential to GEE so take some saety precaution before starting:
@@ -41,6 +36,11 @@ The first method is to use a private token. This is the easiest way to connect t
     - make sure the account you are using will have access to all the assets you need to run your tests
     - create small tests that will run quickly to make sure you don't overload your own GEE account with concurrent tasks
 
+Private Token
+^^^^^^^^^^^^^
+
+The first method is to use a private token. This is the easiest way to connect to GEE in a CI/CD environment.
+
 First authenticate to GEE API in your local computer using ``ee.Authenticate()``.
 
 Then copy the ``credentials`` file content. This file is located in a different folder depending on the platform you use:
@@ -60,6 +60,8 @@ Here is a github action example:
 #. Then to :guilabel:`secretes and variables` -> :guilabel:`Actions`
 #. In this page, set a :guilabel:`new repository secret` with the name ``EARTHENGINE_TOKEN`` and paste the content of your ``credentials`` file in the value field.
 
+Since earthengine-api v0.1.370, it's not possible to use EE without providing a GCS project bucket. Save this value in a `EARTHENGINE_PROJECT` variable, it will be used in the method.
+
 To make the variable available in your CI environment, you will need to add the following line in your action `.yaml` file:
 
 .. code-block:: yaml
@@ -68,9 +70,16 @@ To make the variable available in your CI environment, you will need to add the
 
     env:
         EARTHENGINE_TOKEN: ${{ secrets.EARTHENGINE_TOKEN }}
+        EARTHENGINE_PROJECT: ${{ secrets.EARTHENGINE_PROJECT }}
 
     # The rest of your tests configuration
 
+When working in your local environment export a ``EARTHENGINE_PROJECT`` variable as well:
+
+.. code-block:: console
+
+    export EARTHENGINE_PROJECT=ee-community
+
 Finally you need to configure the ``pytest`` execution environment itself. Add the following line in your ``conftest.py`` file:
 
 .. code-block:: python
@@ -85,6 +94,58 @@ Finally you need to configure the ``pytest`` execution environment itself. Add t
 
 You are now ready to make API calls within your tests!
 
+Service account
+^^^^^^^^^^^^^^^
+
+.. warning::
+
+    This documentation assumes that you already have a Google cloud service account and that you have generated an API key for it. If not, please refer to Google own `documentation <https://cloud.google.com/iam/docs/keys-create-delete>` to proceed.
+
+Paste this content of the `private-key.json` in your CI/CD environment in a ``EARTHENGINE_SERVICE_ACCOUNT`` variable.
+
+Here is a github action example:
+
+.. thumbnail:: _static/github_env_var.png
+    :title: Github action environment variable setup
+
+#. First go to the :guilabel:`settings`` of your Github repository
+#. Then to :guilabel:`secretes and variables` -> :guilabel:`Actions`
+#. In this page, set a :guilabel:`new repository secret` with the name ``EARTHENGINE_SERVICE_ACCOUNT`` and paste the content of your ``credentials`` file in the value field.
+
+Currently when the earthengine-api is Initialized using a service account, the name of the associated cloud project is not detectable. It will prevent the initialization of the test folder generated from `pytest-gee`. To avoid this issue the method rely also on a ``EARTHENGINE_PROJECT`` env variable where you can set the name of your project.
+
+To make the variable available in your CI environment, you will need to add the following line in your action `.yaml` file:
+
+.. code-block:: yaml
+
+    # .github/action.yaml
+
+    env:
+        EARTHENGINE_SERVICE_ACCOUNT: ${{ secrets.EARTHENGINE_SERVICE_ACCOUNT }}
+        EARTHENGINE_PROJECT: ${{ secrets.EARTHENGINE_PROJECT }}
+
+    # The rest of your tests configuration
+
+When working in your local environment export a ``EARTHENGINE_PROJECT`` variable as well:
+
+.. code-block:: console
+
+    export EARTHENGINE_PROJECT=ee-community
+
+Finally you need to configure the ``pytest`` execution environment itself. Add the following line in your ``conftest.py`` file:
+
+.. code-block:: python
+
+    # conftest.py
+
+    import pytest_gee
+
+
+    def pytest_configure():
+        pytest_gee.init_ee_from_service_account()
+
+You are now ready to make API calls within your tests!
+
 Generate a test file tree in GEE
 --------------------------------
 

pytest_gee/__init__.py

@@ -1,7 +1,9 @@
 """The init file of the package."""
 from __future__ import annotations
 
+import json
 import os
+import tempfile
 from pathlib import Path
 from typing import Union
 
@@ -37,9 +39,48 @@ def init_ee_from_token():
         credential_file_path = credential_folder_path / "credentials"
         credential_file_path.write_text(ee_token)
 
+    project_id = os.environ.get("EARTHENGINE_PROJECT", ee.data._cloud_api_user_project)
+    if project_id is None:
+        raise ValueError(
+            "The project name cannot be detected."
+            "Please set the EARTHENGINE_PROJECT environment variable."
+        )
+
     # if the user is in local development the authentication should
     # already be available
-    ee.Initialize(http_transport=httplib2.Http())
+    ee.Initialize(project=project_id, http_transport=httplib2.Http())
+
+
+def init_ee_from_service_account():
+    """Initialize earth engine according using a service account.
+
+    The environment used to run the tests need to have a EARTHENGINE_SERVICE_ACCOUNT variable.
+    The content of this variable must be the copy of a personal credential file that you can generate from the google cloud console.
+
+    Note:
+        As all init method of ``pytest-gee``, this method will fallback to a regular ``ee.Initialize`` using the ``EARTHENGINE_PROJECT`` environment variable.
+    """
+    if "EARTHENGINE_SERVICE_ACCOUNT" in os.environ:
+        # extract the environment variables data
+        private_key = os.environ["EARTHENGINE_SERVICE_ACCOUNT"]
+        ee_user = json.loads(private_key)["client_email"]
+
+        # connect to GEE using a temp file to avoid writing the key to disk
+        with tempfile.TemporaryDirectory() as temp_dir:
+            file = Path(temp_dir) / "private_key.json"
+            file.write_text(private_key)
+            credentials = ee.ServiceAccountCredentials(ee_user, str(file))
+            ee.Initialize(credentials=credentials, http_transport=httplib2.Http())
+
+    elif "EARTHENGINE_PROJECT" in os.environ:
+        # if the user is in local development the authentication should already be available
+        # we simply need to use the provided project name
+        ee.Initialize(project=os.environ["EARTHENGINE_PROJECT"], http_transport=httplib2.Http())
+
+    else:
+        raise ValueError(
+            "EARTHENGINE_SERVICE_ACCOUNT or EARTHENGINE_PROJECT environment variable is missing"
+        )
 
 
 def wait(task: Union[ee.batch.Task, str], timeout: int = 5 * 60) -> str:

pytest_gee/plugin.py

@@ -1,6 +1,7 @@
 """A pytest plugin to build a GEE environment for a test session."""
 from __future__ import annotations
 
+import os
 import uuid
 from pathlib import Path
 
@@ -19,7 +20,15 @@ def gee_hash():
 @pytest.fixture(scope="session")
 def gee_folder_root():
     """Link to the root folder of the connected account."""
-    return Path(ee.data.getAssetRoots()[0]["id"])
+    # The credential information cannot be reached from
+    # the ee API as reported in https://issuetracker.google.com/issues/325020447
+    project_id = os.environ.get("EARTHENGINE_PROJECT", ee.data._cloud_api_user_project)
+    if project_id is None:
+        raise ValueError(
+            "The project name cannot be detected."
+            "Please set the EARTHENGINE_PROJECT environment variable."
+        )
+    return Path(f"projects/{project_id}/assets")
 
 
 @pytest.fixture(scope="session")

tests/conftest.py

@@ -1,4 +1,5 @@
 """Pytest session configuration."""
+
 import ee
 import pytest
 
@@ -7,7 +8,7 @@
 
 def pytest_configure():
     """Init GEE in the test environment."""
-    pytest_gee.init_ee_from_token()
+    pytest_gee.init_ee_from_service_account()
 
 
 @pytest.fixture(scope="session")