diff --git a/conf/gravitino-env.sh.template b/conf/gravitino-env.sh.template index 5f9e3e9d8b..a4157f42ea 100644 --- a/conf/gravitino-env.sh.template +++ b/conf/gravitino-env.sh.template @@ -10,4 +10,4 @@ # export GRAVITINO_HOME # export GRAVITINO_CONF_DIR # export GRAVITINO_LOG_DIR # Where log files are stored. PWD by default. -# export GRAVITINO_MEM # Gravitino jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxMetaspaceSize=512m +# export GRAVITINO_MEM # Gravitino jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxMetaspaceSize=512m \ No newline at end of file diff --git a/docs/gravitino-server-config.md b/docs/gravitino-server-config.md index ee3a8a4c1d..1bfeb10974 100644 --- a/docs/gravitino-server-config.md +++ b/docs/gravitino-server-config.md @@ -47,3 +47,15 @@ The following table lists the configuration items in the `gravitino.conf` file. |-----------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|---------------| | `gravitino.catalog.cache.evictionIntervalMs` | The interval in milliseconds to evict the catalog cache, default 3600000ms(1h) | `3600000` | 0.1.0 | | `gravitino.catalog.classloader.isolated` | Whether to use an isolated classloader for catalog, if it's true, all catalog-related libraries and configurations will be loaded by an isolated classloader NOT by AppClassLoader. Default value is `true` | `true` | 0.1.0 | + +## How to set up runtime environment variables + +Gravitino server also supports setting up runtime environment variables by editing the `gravitino-env.sh` file, which is located in the `conf` directory. + +### How to access Hadoop + +Currently, due to the absence of a comprehensive user permission system, Gravitino can only use a single username for +Hadoop access. Please ensure that the user starting the Gravitino server has Hadoop (HDFS, YARN, etc.) access +permissions; otherwise, you may encounter a `Permission denied` error. There are also several ways to resolve this error: +* Granting the Gravitino startup user permissions in Hadoop +* Specify the authorized Hadoop username in the environment variables `HADOOP_USER_NAME` before starting the Gravitino server. diff --git a/docs/iceberg-rest-service.md b/docs/iceberg-rest-service.md index 32cfcd6cf1..7f4c16fec5 100644 --- a/docs/iceberg-rest-service.md +++ b/docs/iceberg-rest-service.md @@ -1,5 +1,5 @@ --- -title: "How to setup Gravitino Iceberg REST server" +title: "How to set up Gravitino Iceberg REST server" date: 2023-10-18T09:03:20-08:00 license: "Copyright 2023 Datastrato. This software is licensed under the Apache License version 2." @@ -12,6 +12,8 @@ Gravitino Iceberg REST Server follows the [Iceberg REST API specification](https * Support Iceberg REST API defined in Iceberg 1.3.1, support all namespace&table interface. `Token`, `ReportMetrics` and `Config` interface are not supported yet. * Worked as a catalog proxy, supports HiveCatalog and JdbcCatalog for now. * Build with Iceberg `1.3.1`, which means the Iceberg table format version is `1` by default. +* When writing to HDFS, the Gravitino Iceberg REST server can only operate as the specified HDFS user when startup and + does not yet support proxying to other HDFS users. see *How to access Hadoop* in document *How to customize Gravitino server configurations* gravitino-server-config for more details. ## How to start the Gravitino Iceberg REST server diff --git a/docs/integration-test.md b/docs/integration-test.md index 1785a7c08b..53443654e2 100644 --- a/docs/integration-test.md +++ b/docs/integration-test.md @@ -95,7 +95,7 @@ Before running the tests, make sure Docker is installed. Additionally, the Gravitino Server and third-party data source Docker runtime environments will use certain ports. Ensure that these ports are not already in use: - Gravitino Server: Port `8090` -- Hive Docker runtime environment: Ports are `22`, `7180`, `8088`, `8888`, `9000`, `9083`, `10000`, `10002`, `50070`, and `50075` +- Hive Docker runtime environment: Ports are `22`, `7180`, `8088`, `8888`, `9000`, `9083`, `10000`, `10002`, `50010`, `50070`, and `50075` ## Debugging Gravitino Server and Integration Tests @@ -118,7 +118,7 @@ To debug the Gravitino Server and integration tests, you have two modes: `embedd - If you only debug integration test codes, You don't have to do any setup to debug directly - If you need to debug Gravitino server codes, follow these steps: - Enable the `GRAVITINO_DEBUG_OPTS` environment variable in the `distribution/package/conf/gravitino-env.sh` file to enable remote JVM debugging - - Manually start the Gravitino Server using the `./distribution/package/bin/gravitino-server.sh start` command + - Manually start the Gravitino Server using the `./distribution/package/bin/gravitino.sh start` command - Select `gravitino.server.main` module classpath in the `Remote JVM Debug` to attach the Gravitino Server process and debug it ## Running on GitHub Actions