Skip to content

Commit

Permalink
Added: Instructions for using Hash Cache in Server
Browse files Browse the repository at this point in the history
  • Loading branch information
Sewer56 committed Aug 4, 2024
1 parent 7a718e9 commit 5d67306
Show file tree
Hide file tree
Showing 4 changed files with 105 additions and 9 deletions.
68 changes: 68 additions & 0 deletions docs/Common/Hash-Cache/Usage-In-Server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Hash Cache Usage in Server

!!! info "This describes how the Reloaded3 server uses the hash cache for packages and game folders."

The Reloaded3 server utilizes a hash cache system to efficiently store and retrieve hash information
for both packages and games.

<!-- TODO: Move this to separate section when the File Hash Cache is implemented as separate library -->

## Package Hash Cache

For packages, the hash cache items are stored using a combination of the package name and version:

```
{PackageID}+{PackageVersion}.hashcache
```

Where:

- [{PackageID}][package-id] is the unique identifier of the package.
- [{PackageVersion}][package-version] is the semantic version of the package.

!!! example

For a package with ID `reloaded3.utility.examplemod.s56` with version `1.2.3`,
the hash cache file would be:

```
reloaded3.utility.examplemod.s56+1.2.3.hashcache
```

### Usage

When the server needs to access or update the hash cache for a specific package:

1. Construct the file name using the package name and version.
2. Try to open the file in the designated [hash cache directory for packages][location-hashcache-package].
3. If the open operation succeeds, read the [hash cache file][hash-cache-file].
4. If the file doesn't exist, create a new hash cache file for the package.

## Game Hash Cache

!!! info "Hash cache for games is tricky, because a user can have multiple versions of the same game installed."

For example, a `Steam` release of a game, and a `GOG` release of a game.

In addition, a user may have multiple computers, where the local files for a game may differ.

Therefore, the file name for the hash cache for games is stored within the [Machine Specific Info][machine-specific-info]
structure of each installed game entry.

### Usage

1. Check the hash cache directory for the game as it's loaded.
2. Compare timestamps and short hashes of files.
3. If any file is different, suggest to the user that the game folder has been modified.
4. This can be done via a [Diagnostic][diagnostic].

For game files, only the short hash and timestamp parts of the hash cache are used. Full hashes
would take too long to verify; so we simply omit them and write them as `0`.

[package-id]: ../../Server/Packaging/Package-Metadata.md#id
[package-version]: ../../Server/Packaging/Package-Metadata.md#version
[hash-cache-file]: ./File-Format.md
[location-hashcache-package]: ../../Server/Storage/Locations.md#hash-cache-files
[game-id]: ../../Server/Storage/Games/About.md
[machine-specific-info]: ../../Server/Storage/Games/About.md#machine-specific-info
[diagnostic]: ../../Server/Diagnostics.md
22 changes: 14 additions & 8 deletions docs/Server/Storage/Games/About.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,19 +128,23 @@ This structure is defined as: `HashMap<EntryID, MachineInfo>`.

`MachineInfo` is defined as:

| Type | Name | Description |
| -------------- | --------------------------------- | ------------------------------------------------------------- |
| bool | DisplayedCreateShortcutPrompt | Displayed the [Create Shortcut](#autocreateshortcuts) prompt. |
| string | FriendlyName | Friendly Name for the Machine. |
| string | MainExePath | Last path to the main executable. |
| string | MainExeHash | Original hash of the main executable. [XXH128][hashing] |
| DeploymentType | [DeploymentType](#deploymenttype) | Dictates how the loader is injected into the game. |
| Type | Name | Description |
| -------------- | --------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| bool | DisplayedCreateShortcutPrompt | Displayed the [Create Shortcut](#autocreateshortcuts) prompt. |
| string | FriendlyName | Friendly Name for the Machine. |
| string | MainExePath | Last path to the main executable. |
| string | MainExeHash | Original hash of the main executable. [XXH128][hashing] |
| string | FileHashCacheName | Name of the file containing the [File Hash Cache][file-hash-cache] in the ['Games'][file-hash-cache-location] folder |
| DeploymentType | [DeploymentType](#deploymenttype) | Dictates how the loader is injected into the game. |

The `FriendlyName` is derived from `UserName` and `MachineName`.
`MachineName` is derived as `hostname` on Linux/macOS and `ComputerName` on Windows.

Usually it's formatted as `{UserName}'s {MachineName}`.

The `FileHashCacheName` can be any name, but it is generally recommended to use `UUID v7` (timestamp + random)
for this. The name should be `{UUID}.hashcache`.

##### DeploymentType

!!! info "This defines user's preference for how the loader is injected into the game."
Expand Down Expand Up @@ -185,4 +189,6 @@ This data is cached in the [`Server Cache Files` (`Cache/Server`) folder.][adde
[store-information]: ../../../Services/Community-Repository.md#store-information
[loadout]: ../Loadouts/About.md
[hashing]: ../../../Common/Hashing.md
[loadout-specific-info]: ../../../Common/Configuration/About.md#ux-for-configuration-layers
[loadout-specific-info]: ../../../Common/Configuration/About.md#ux-for-configuration-layers
[file-hash-cache]: ../../../Common/Hash-Cache/About.md
[file-hash-cache-location]: ../Locations.md#hash-cache-files
23 changes: 22 additions & 1 deletion docs/Server/Storage/Locations.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ Also see: [Portable Installs](#portable-install).
| Temporary Files (Mods) | `Temp/{processId}`[1] | Runtime-generated temporary files. Not persisted across runs. |
| [Server Cache Files](#server-cache-files) | `Cache/Server` | Cache files tied to the Reloaded3 server. |
| [Package Cache Files](#cache-files-machine-specific) | `Cache/Package/{packageId}` | Cache files that consist of inputs that do not contain [User](#user) data. |
| [Hash Cache Files](#hash-cache-files) | `Hashes` | File hash caches. |

[1] Temporary Files should be tied to `ProcessID`, if a process with that ID is dead, they should be
auto deleted by the server. To avoid files in use accidentally being deleted, temporary files are stored
Expand Down Expand Up @@ -146,6 +147,24 @@ Loadouts

For the exact details, see the [Loadouts][loadouts-location] page.

### Hash Cache Files

!!! info "This folder stores the unmodified last states of files in packages and games."

[Documentation can be found here][hashcache-docs].

This folder has the following layout:

```
Hashes
├── Games
│   └── 24c69d40-090e-406b-9a1b-2487571a568c.hashcache
└── Packages
└── reloaded3.utility.examplemod.s56+1.2.3.hashcache
```

Details on how the hash cache files are is in the [Usage in Server][hash-cache-usage] page.

## Extra

### Multi-User Networked Systems
Expand Down Expand Up @@ -206,4 +225,6 @@ Loadouts and package configs may be shared in the future on a machine level.
[loadout-snapshot]: ./Loadouts/About.md#snapshots
[var-lib]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s08.html
[var-cache]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s05.html
[merged-files]: ../../Mods/Libraries/Merged-File-Cache/About.md
[merged-files]: ../../Mods/Libraries/Merged-File-Cache/About.md
[hashcache-docs]: ../../Common/Hash-Cache/About.md
[hash-cache-usage]: ../../Common/Hash-Cache/Usage-In-Server.md
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ nav:
- File Format: Common/Hash-Cache/File-Format.md
- Implementation Details: Common/Hash-Cache/Implementation-Details.md
- API: Common/Hash-Cache/API.md
- Hash Cache Usage in Server: Common/Hash-Cache/Usage-In-Server.md
- Hashing: Common/Hashing.md
- Files: Common/Files.md
- Mods:
Expand Down

0 comments on commit 5d67306

Please sign in to comment.