Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated docs for postgres and mysql components #3811

Merged
merged 7 commits into from
Oct 11, 2023
Merged

Updated docs for postgres and mysql components #3811

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
893176f
Updated docs for postgres and mysql components
ItalyPaleAle Oct 11, 2023
1fb1fd4
Merge branch 'v1.12' into postgres-1.12
ItalyPaleAle Oct 11, 2023
b1b3b6d
Update mysql.md
artursouza Oct 11, 2023
e9f22d4
Update postgresql.md
artursouza Oct 11, 2023
e4ffd56
Update daprdocs/content/en/reference/components-reference/supported-b…
msfussell Oct 11, 2023
67416b3
Update postgresql.md
msfussell Oct 11, 2023
379a0ce
Update mysql.md
msfussell Oct 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 37 additions & 10 deletions daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,21 +55,27 @@ Note that you can not use secret just for username/password. If you use secret,
| `maxOpenConns` | N | Output | The max open connections. Integer greater than 0 | `"10"` |
| `connMaxLifetime` | N | Output | The max connection lifetime. Duration string | `"12s"` |
| `connMaxIdleTime` | N | Output | The max connection idel time. Duration string | `"12s"` |
| `direction` | N | Output | The direction of the binding | `"output"` |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the direction on this component. We put these into all binding to be explicit that this is useful for separation of Dapr sidecar from asking the app.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only an output binding so the direction is irrelevant. I think it causes more confusion than anything.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still tells the sidecar information that is needed. Even for output only bindings this is useful, yes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it is a no-op for output bindings, just adds confusion to users

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am referring specifically to output-only bindings. A binding that is input-only or input and output can benefit from this. But output bindings don't need this


### SSL connection

If your server requires SSL your connection string must end of `&tls=custom` for example:

```bash
"<user>:<password>@tcp(<server>:3306)/<database>?allowNativePasswords=true&tls=custom"
```
You must replace the `<PEM PATH>` with a full path to the PEM file. If you are using [MySQL on Azure](http://bit.ly/AzureMySQLSSL) see the Azure [documentation on SSL database connections](http://bit.ly/MySQLSSL), for information on how to download the required certificate. The connection to MySQL will require a minimum TLS version of 1.2.

Also note that by default [MySQL go driver](https://github.com/go-sql-driver/mysql) only supports one SQL statement per query/command.
> You must replace the `<PEM PATH>` with a full path to the PEM file. If you are using Azure Database for MySQL see the Azure [documentation on SSL database connections](https://learn.microsoft.com/azure/mysql/single-server/how-to-configure-ssl), for information on how to download the required certificate. The connection to MySQL will require a minimum TLS version of 1.2.
msfussell marked this conversation as resolved.
Show resolved Hide resolved

### Multiple statements

By default, the [MySQL Go driver](https://github.com/go-sql-driver/mysql) only supports one SQL statement per query/command.

To allow multiple statements in one query you need to add `multiStatements=true` to a query string, for example:

```bash
"<user>:<password>@tcp(<server>:3306)/<database>?multiStatements=true"
```

While this allows batch queries, it also greatly increases the risk of SQL injections. Only the result of the first query is returned,
all other results are silently discarded.

Expand All @@ -81,17 +87,37 @@ This component supports **output binding** with the following operations:
- `query`
- `close`

### Parametrized queries

This binding supports parametrized queries, which allow separating the SQL query itself from user-supplied values. The usage of parametrized queries is **strongly recommended** for security reasons, as they prevent [SQL Injection attacks](https://owasp.org/www-community/attacks/SQL_Injection).

For example:

```sql
-- ❌ WRONG! Includes values in the query and is vulnerable to SQL Injection attacks.
SELECT * FROM mytable WHERE user_key = 'something';

-- ✅ GOOD! Uses parametrized queries.
-- This will be executed with parameters ["something"]
SELECT * FROM mytable WHERE user_key = ?;
```

> [Relevant XKCD](https://xkcd.com/327/)
artursouza marked this conversation as resolved.
Show resolved Hide resolved

### exec

The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows).

The `params` property is a string containing a JSON-encoded array of parameters.

**Request**

```json
{
"operation": "exec",
"metadata": {
"sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')"
"sql": "INSERT INTO foo (id, c1, ts) VALUES (?, ?, ?)",
"params": "[1, \"demo\", \"2020-09-24T11:45:05Z07:00\"]"
}
}
```
Expand All @@ -106,7 +132,7 @@ The `exec` operation can be used for DDL operations (like table creation), as we
"start-time": "2020-09-24T11:13:46.405097Z",
"end-time": "2020-09-24T11:13:46.414519Z",
"rows-affected": "1",
"sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')"
"sql": "INSERT INTO foo (id, c1, ts) VALUES (?, ?, ?)"
}
}
```
Expand All @@ -115,13 +141,16 @@ The `exec` operation can be used for DDL operations (like table creation), as we

The `query` operation is used for `SELECT` statements, which returns the metadata along with data in a form of an array of row values.

The `params` property is a string containing a JSON-encoded array of parameters.

**Request**

```json
{
"operation": "query",
"metadata": {
"sql": "SELECT * FROM foo WHERE id < 3"
"sql": "SELECT * FROM foo WHERE id < $1",
"params": "[3]"
}
}
```
Expand All @@ -135,7 +164,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat
"duration": "432µs",
"start-time": "2020-09-24T11:13:46.405097Z",
"end-time": "2020-09-24T11:13:46.420566Z",
"sql": "SELECT * FROM foo WHERE id < 3"
"sql": "SELECT * FROM foo WHERE id < ?"
},
"data": [
{column_name: value, column_name: value, ...},
Expand All @@ -150,7 +179,7 @@ or numbers (language specific data type)

### close

Finally, the `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response.
The `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response.

**Request**

Expand All @@ -160,8 +189,6 @@ Finally, the `close` operation can be used to explicitly close the DB connection
}
```

> Note, the MySQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query.

## Related links

- [Basic schema for a Dapr component]({{< ref component-schema >}})
Expand Down
79 changes: 61 additions & 18 deletions ...docs/content/en/reference/components-reference/supported-bindings/postgresql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,9 @@ spec:
type: bindings.postgresql
version: v1
metadata:
- name: url # Required
value: "<CONNECTION_STRING>"
- name: direction
value: "<DIRECTION_OF_BINDING>"
ItalyPaleAle marked this conversation as resolved.
Show resolved Hide resolved
# Connection string
- name: connectionString
value: "<CONNECTION STRING>"
```

{{% alert title="Warning" color="warning" %}}
Expand All @@ -34,25 +33,48 @@ The above example uses secrets as plain strings. It is recommended to use a secr

## Spec metadata fields

| Field | Required | Binding support | Details | Example |
|--------------------|:--------:|------------|-----|---------|
| `url` | Y | Output | PostgreSQL connection string See [here](#url-format) for more details | `"user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca"` |
| `direction` | N | Output | The direction of the binding | `"output"` |
### Authenticate using a connection string

The following metadata options are **required** to authenticate using a PostgreSQL connection string.

| Field | Required | Details | Example |
|--------|:--------:|---------|---------|
| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"`

### Authenticate using Azure AD

Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity.

| Field | Required | Details | Example |
|--------|:--------:|---------|---------|
| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` |
| `connectionString` | Y | The connection string for the PostgreSQL database.<br>This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` |
| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` |
| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` |
| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` |

### Other metadata options

| Field | Required | Binding support |Details | Example |
|--------------------|:--------:|-----|---|---------|
| `maxConns` | N | Output | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"`
| `connectionMaxIdleTime` | N | Output | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"`
| `queryExecMode` | N | Output | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"`

### URL format

The PostgreSQL binding uses [pgx connection pool](https://github.com/jackc/pgx) internally so the `url` parameter can be any valid connection string, either in a `DSN` or `URL` format:
The PostgreSQL binding uses [pgx connection pool](https://github.com/jackc/pgx) internally so the `connectionString` parameter can be any valid connection string, either in a `DSN` or `URL` format:

**Example DSN**

```shell
user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca
user=dapr password=secret host=dapr.example.com port=5432 dbname=my_dapr sslmode=verify-ca
```

**Example URL**

```shell
postgres://dapr:[email protected]:5432/dapr?sslmode=verify-ca
postgres://dapr:[email protected]:5432/my_dapr?sslmode=verify-ca
```

Both methods also support connection pool configuration variables:
Expand All @@ -72,17 +94,37 @@ This component supports **output binding** with the following operations:
- `query`
- `close`

### Parametrized queries

This binding supports parametrized queries, which allow separating the SQL query itself from user-supplied values. The usage of parametrized queries is **strongly recommended** for security reasons, as they prevent [SQL Injection attacks](https://owasp.org/www-community/attacks/SQL_Injection).

For example:

```sql
-- ❌ WRONG! Includes values in the query and is vulnerable to SQL Injection attacks.
SELECT * FROM mytable WHERE user_key = 'something';

-- ✅ GOOD! Uses parametrized queries.
-- This will be executed with parameters ["something"]
SELECT * FROM mytable WHERE user_key = $1;
```

> [Relevant XKCD](https://xkcd.com/327/)

### exec

The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows).

The `params` property is a string containing a JSON-encoded array of parameters.

**Request**

```json
{
"operation": "exec",
"metadata": {
"sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')"
"sql": "INSERT INTO foo (id, c1, ts) VALUES ($1, $2, $3)",
"params": "[1, \"demo\", \"2020-09-24T11:45:05Z07:00\"]"
}
}
```
Expand All @@ -97,7 +139,7 @@ The `exec` operation can be used for DDL operations (like table creation), as we
"start-time": "2020-09-24T11:13:46.405097Z",
"end-time": "2020-09-24T11:13:46.414519Z",
"rows-affected": "1",
"sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')"
"sql": "INSERT INTO foo (id, c1, ts) VALUES ($1, $2, $3)"
}
}
```
Expand All @@ -106,13 +148,16 @@ The `exec` operation can be used for DDL operations (like table creation), as we

The `query` operation is used for `SELECT` statements, which returns the metadata along with data in a form of an array of row values.

The `params` property is a string containing a JSON-encoded array of parameters.

**Request**

```json
{
"operation": "query",
"metadata": {
"sql": "SELECT * FROM foo WHERE id < 3"
"sql": "SELECT * FROM foo WHERE id < $1",
"params": "[3]"
}
}
```
Expand All @@ -126,7 +171,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat
"duration": "432µs",
"start-time": "2020-09-24T11:13:46.405097Z",
"end-time": "2020-09-24T11:13:46.420566Z",
"sql": "SELECT * FROM foo WHERE id < 3"
"sql": "SELECT * FROM foo WHERE id < $1"
},
"data": "[
[0,\"test-0\",\"2020-09-24T04:13:46Z\"],
Expand All @@ -138,7 +183,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat

### close

Finally, the `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response.
The `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response.

**Request**

Expand All @@ -148,8 +193,6 @@ Finally, the `close` operation can be used to explicitly close the DB connection
}
```

> Note, the PostgreSQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query.

## Related links

- [Basic schema for a Dapr component]({{< ref component-schema >}})
Expand Down
Loading
Loading