Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.13] Add docs for SQLite name resolver #3991

Merged
merged 12 commits into from
Feb 2, 2024
Merged

Conversation

ItalyPaleAle
Copy link
Contributor

Fixes #3914

@ItalyPaleAle ItalyPaleAle requested review from a team as code owners January 31, 2024 18:34
@hhunter-ms hhunter-ms added this to the 1.13 milestone Jan 31, 2024
Copy link
Collaborator

@hhunter-ms hhunter-ms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick review

Co-authored-by: Hannah Hunter <[email protected]>
Signed-off-by: Alessandro (Ale) Segala <[email protected]>
@msfussell
Copy link
Member

msfussell commented Jan 31, 2024

@ItalyPaleAle @hhunter-ms

To help with this PR (given this is the first time we have updated name resolution in a while) there are a few other topics that need updating along with this PR.

  1. Update the configuration overview https://docs.dapr.io/operations/configuration/configuration-overview/ to include how to set the name resolution config as you have in the Configuration format section of the component in this PR. This

Configuration format

Within the Configuration YAML, set the spec.nameResolution.component property to "sqlite", then pass configuration options in the spec.nameResolution.configuration dictionary.

This is the basic example of a Configuration resource:

apiVersion: dapr.io/v1alpha1
kind: Configuration
metadata:
  name: appconfig
spec:
  nameResolution:
    component: "sqlite"
    version: "v1"
    configuration:
      connectionString: "/home/user/.dapr/nr.db"
  1. Update the Configuration resource spec https://docs.dapr.io/reference/resource-specs/configuration-schema/ to include name resolution settings above.

  2. Add a link to the https://docs.dapr.io/reference/components-reference/supported-name-resolution/ topic so that users understand how to set name resolution. In effect add this

Name resolution is configured via the [Dapr Configuration]({{< ref configuration-overview.md >}}).

  1. Update the "Pluggable service discovery" section https://docs.dapr.io/developing-applications/building-blocks/service-invocation/service-invocation-overview/#pluggable-service-discovery to include this sentence below after this sentence "Self-hosted machines can use the mDNS name resolution component. " like this

"Self-hosted machines can use the mDNS name resolution component. As an alternative to mDNS, the SQLite name resolution component can be used for running Dapr on single-node environments and for local development scenarios. Dapr sidecars that are part of the cluster store their information in a SQLite database on the local machine."

  1. Remove "provider spec" in the headings for K8s, Hashicorp and mDNS components. For example here https://docs.dapr.io/reference/components-reference/supported-name-resolution/setup-nr-consul/, here https://docs.dapr.io/reference/components-reference/supported-name-resolution/nr-kubernetes/ and here https://docs.dapr.io/reference/components-reference/supported-name-resolution/nr-mdns/

Copy link
Member

@msfussell msfussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great update. I also left additional comments on Name Resolution above ^^

| `connectionString` | Y | `string` | The connection string for the SQLite database. Normally, this is the path to a file on disk, relative to the current working directory, or absolute. | `"nr.db"` (relative to the working directory), `"/home/user/.dapr/nr.db"` |
| `updateInterval` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval for active Dapr sidecars to update their status in the database.<br>Must be at least 1s greater than `timeout`. Values with fractions of seconds are truncated (for example, `1500ms` becomes `1s`). Default: `5s` | `"2s"` |
| `timeout` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`).<br>Must be at least 1s. | Timeout for operations on the database. Integers are interpreted as number of seconds. Defaults to `1s` | `"2s"`, `2` |
| `tableName` | N | `string` | Name of the table where the data is stored. Defaults to `hosts`. | `"hosts"` |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this table created automatically by Dapr, or do you need to create this in advance?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything is automatic

| `tableName` | N | `string` | Name of the table where the data is stored. Defaults to `hosts`. | `"hosts"` |
| `metadataTableName` | N | `string` | Name of the table used by Dapr to store metadata for the component. Defaults to `metadata`. | `"metadata"` |
| `cleanupInterval` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to remove stale records from the database. Default: `1h` (1 hour) | `"10m"` |
| `busyTimeout` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to wait in case the SQLite database is currently busy serving another request, before returning a "database busy" error. Default: `800ms` (800 milliseconds) | `"100ms"` |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this relate to the timeout above? They sound like they do the same thing. When would I set this over using timeout above. Would be great to have some guidance here in the doc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's some overlap, but they behave differently.

  • busy timeout controls how locking works in SQLite. With SQLite, writes are exclusive, so every time any app is writing something, the DB is locked. If another app tries to write, it will wait up to "busy timeout" before returning the "database busy" error
  • the timeout option controls the timeout for the entire operation. For example if the query "hangs", after the DB has acquired the lock (so after busy timeout is cleared), then timeout comes into effect

For SQLite, most users should not touch this

…ame-resolution/nr-sqlite.md

Co-authored-by: Mark Fussell <[email protected]>
Signed-off-by: Alessandro (Ale) Segala <[email protected]>
@hhunter-ms
Copy link
Collaborator

To help with this PR (given this is the first time we have updated name resolution in a while) there are a few other topics that need updating along with this PR.

@msfussell @ItalyPaleAle I can do this in a separate PR

@hhunter-ms hhunter-ms mentioned this pull request Feb 2, 2024
5 tasks
…ame-resolution/nr-sqlite.md

Co-authored-by: Mark Fussell <[email protected]>
Signed-off-by: Alessandro (Ale) Segala <[email protected]>
msfussell and others added 2 commits February 2, 2024 09:46
…ame-resolution/nr-sqlite.md

Signed-off-by: Mark Fussell <[email protected]>
…ame-resolution/nr-sqlite.md

Co-authored-by: Alessandro (Ale) Segala <[email protected]>
Signed-off-by: Mark Fussell <[email protected]>
Fixed formatting
…ame-resolution/nr-sqlite.md

Signed-off-by: Mark Fussell <[email protected]>
Copy link
Member

@msfussell msfussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I did some additional editing to add clarifications on usage

| `cleanupInterval` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to remove stale records from the database. Default: `1h` (1 hour) | `"10m"` |
| `busyTimeout` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to wait in case the SQLite database is currently busy serving another request, before returning a "database busy" error. Default: `800ms` (800 milliseconds) | `"100ms"` |
| `busyTimeout` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to wait in case the SQLite database is currently busy serving another request, before returning a "database busy" error. This is an advanced setting.</br>`busyTimeout` controls how locking works in SQLite. With SQLite, writes are exclusive, so every time any app is writing the database is locked. If another app tries to write, it waits up to `busyTimeout` before returning the "database busy" error. However the `timeout` setting controls the timeout for the entire operation. For example if the query "hangs", after the database has acquired the lock (so after busy timeout is cleared), then `timeout` comes into effect. Default: `800ms` (800 milliseconds) | `"100ms"` |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to explain this (and especially I wasn't thinking my words would be used in the docs, or I'd have written something better :) ). I'd just say "This is an advanced setting"

Suggested change
| `busyTimeout` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to wait in case the SQLite database is currently busy serving another request, before returning a "database busy" error. This is an advanced setting.</br>`busyTimeout` controls how locking works in SQLite. With SQLite, writes are exclusive, so every time any app is writing the database is locked. If another app tries to write, it waits up to `busyTimeout` before returning the "database busy" error. However the `timeout` setting controls the timeout for the entire operation. For example if the query "hangs", after the database has acquired the lock (so after busy timeout is cleared), then `timeout` comes into effect. Default: `800ms` (800 milliseconds) | `"100ms"` |
| `busyTimeout` | N | [Go duration](https://pkg.go.dev/time#ParseDuration) (as a `string`) | Interval to wait in case the SQLite database is currently busy serving another request, before returning a "database busy" error.<br>This is an advanced setting that controls how long to wait if the database is locked by another transaction.<br>Default: `800ms` (800 milliseconds) | `"100ms"` |

Copy link
Collaborator

@hhunter-ms hhunter-ms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some clarification and fixing typos

Co-authored-by: Hannah Hunter <[email protected]>
Signed-off-by: Alessandro (Ale) Segala <[email protected]>
@hhunter-ms hhunter-ms merged commit c6aca39 into dapr:v1.13 Feb 2, 2024
4 checks passed
@ItalyPaleAle ItalyPaleAle deleted the sqlite-nr branch February 2, 2024 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants