DOC Document making variants with different file extensions

en/02_Developer_Guides/14_Files/05_File_Manipulation.md

@@ -12,7 +12,7 @@ Silverstripe CMS provides a well-abstracted API for creating, manipulating, and
 
 See [images](./images/) for some image-specific manipulation methods.
 
-## Creating files in PHP
+## Creating new files in PHP
 
 When working with files in PHP you can upload a file into a [`File`](api:SilverStripe\Assets\File) dataobject
 using one of the below methods:
@@ -23,22 +23,72 @@ using one of the below methods:
 | `File::setFromStream`      | Will store content from a stream        |
 | `File::setFromString`      | Will store content from a binary string |
 
-### Upload conflict resolution
+For example:
 
-When storing files, it's possible to determine the mechanism the backend should use when it encounters
-an existing file pattern. The conflict resolution to use can be passed into the third parameter of the
+```php
+use SilverStripe\Assets\File;
+
+// Store a file named "example-file.txt".
+$fileRecord = File::create();
+$fileRecord->setFromString('This is some file content', 'example-file.txt');
+$fileRecord->write();
+```
+
+If you want to store your file in a [`DBFile`] field directly, or you don't want to use the `File` model for some reason,
+you can also use the default [`AssetStore`](api:SilverStripe\Assets\Storage\AssetStore) directly:
+
+```php
+use SilverStripe\Assets\File;
+use SilverStripe\Assets\Storage\AssetStore;
+use SilverStripe\Core\Injector\Injector;
+use SilverStripe\ORM\FieldType\DBField;
+
+// Store a file named "example-file.txt".
+$store = Injector::inst()->get(AssetStore::class);
+$result = $store->setFromString('This is some file content', 'example-file.txt');
+
+// Save a database record that points to the stored file.
+// Note that we pull the file name from the result because the asset store might have renamed it.
+$dbFile = DBField::create_field('DBFile', $result);
+$fileRecord = File::create(['Name' => $result['Filename'], 'File' => $dbFile]);
+$fileRecord->write();
+```
+
+### Storage conflict resolution
+
+When storing new files, it's possible to determine the mechanism the backend should use when it encounters
+an existing file name pattern. The conflict resolution to use can be passed into the third parameter of the
 above methods (after content and filename). The available constants are:
 
 | Constant                            | If an existing file is found then:  |
 | ----------------------------------- | ----------------------------------- |
 | `AssetStore::CONFLICT_EXCEPTION`    | An exception will be thrown         |
 | `AssetStore::CONFLICT_OVERWRITE`    | The existing file will be replaced  |
-| `AssetStore::CONFLICT_RENAME`       | The backend will choose a new name. |
+| `AssetStore::CONFLICT_RENAME`       | The backend will choose a new name  |
 | `AssetStore::CONFLICT_USE_EXISTING` | The existing file will be used      |
 
 If no conflict resolution scheme is chosen, or an unsupported one is requested, then the backend will choose one.
 The default asset store supports each of these.
 
+The conflict resolution is passed in to the `config` argument like so:
+
+```php
+use SilverStripe\Assets\File;
+use SilverStripe\Assets\Storage\AssetStore;
+
+// Store a file named "example-file.txt".
+$fileRecord = File::create();
+$fileRecord->setFromString(
+    'This is some file content',
+    'example-file.txt',
+    // If a file with that name already exists, let the file store rename this one.
+    config: ['conflict' => AssetStore::CONFLICT_RENAME]
+);
+$fileRecord->write();
+```
+
+See [file storage](./file_storage/) for more details about the way files are stored.
+
 ## Accessing files via PHP
 
 As with storage, there are also different ways of loading the content (or properties) of the file:
@@ -57,7 +107,7 @@ As with storage, there are also different ways of loading the content (or proper
 
 Silverstripe CMS has a pre-defined list of common file types. `File::getFileType` will return "unknown" for files outside that list.
 
-You can add your own file extensions and its description with the following configuration.
+You can add your own file extensions and their description with the following configuration:
 
 ```yml
 SilverStripe\Assets\File:
@@ -79,8 +129,8 @@ use SilverStripe\Assets\File;
 
 $file = File::get()->filter('Name', 'oldname.jpg')->first();
 if ($file) {
-  // The below will move 'oldname.jpg' and 'oldname__variant.jpg'
-  // to 'newname.jpg' and 'newname__variant.jpg' respectively
+    // The below will move 'oldname.jpg' and 'oldname__variant.jpg'
+    // to 'newname.jpg' and 'newname__variant.jpg' respectively
     $file->Name = 'newname.jpg';
     $file->write();
 }
@@ -94,8 +144,8 @@ use SilverStripe\Versioned\Versioned;
 
 $file = File::get()->filter('Name', 'oldname.jpg')->first();
 if ($file) {
-  // The below will immediately move 'oldname.jpg' and 'oldname__variant.jpg'
-  // to 'newname.jpg' and 'newname__variant.jpg' respectively
+    // The below will immediately move 'oldname.jpg' and 'oldname__variant.jpg'
+    // to 'newname.jpg' and 'newname__variant.jpg' respectively
     $file->Name = 'newname.jpg';
     Versioned::withVersionedMode(function () use ($file) {
         Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
@@ -104,3 +154,229 @@ if ($file) {
     });
 }
 ```
+
+## Convert a file to a different format
+
+You can use the [`manipulateExtension()`](api:SilverStripe\Assets\ImageManipulation::manipulateExtension()) method on any `File` or `DBFile` object to create a variant with a different file extension than the original.
+
+This can be very useful if you want to convert a file to a different format for the user to download or view, while leaving the original file intact. Some examples of when you might want this are:
+
+- Generating thumbnails for videos, documents, etc
+- Converting images to `.webp` for faster page load times
+- Converting documents to `.pdf` so downloaded documents are more portable
+
+### Converting between image formats
+
+Converting between image formats is the easiest example, because we can let [Intervention Image](https://image.intervention.io/v2) do the heavy lifting for us.
+
+All we need to do is tell it what extension we want to convert to and how to [handle conflicts](#storage-conflict-resolution), and if that conversion is supported it will be done.
+
+See [Supported Formats | Intervention Image](https://image.intervention.io/v2/introduction/formats) for supported formats.
+
+```php
+namespace App\Extension;
+
+use SilverStripe\Assets\Storage\AssetStore;
+use SilverStripe\Assets\Storage\DBFile;
+use SilverStripe\Core\Extension;
+
+class ImageFormatExtension extends Extension
+{
+    /**
+     * Create a variant of the image in a different format.
+     *
+     * @param string $newExtension The file extension of the formatted file, e.g. "webp"
+     */
+    public function format(string $newExtension): DBFile
+    {
+        $original = $this->getOwner();
+        return $original->manipulateExtension(
+            $newExtension,
+            function (AssetStore $store, string $filename, string $hash, string $variant) use ($original) {
+                $backend = $original->getImageBackend();
+                $config = ['conflict' => AssetStore::CONFLICT_USE_EXISTING];
+                $tuple = $backend->writeToStore($store, $filename, $hash, $variant, $config);
+                return [$tuple, $backend];
+            }
+        );
+    }
+}
+```
+
+Let's look at what's actually happening here, piece by piece.
+
+```php
+return $original->manipulateExtension($newExtension /* ... */);
+```
+
+We call the `manipulateExtension()` method and pass in the file extension we want to convert our image to. If that variant file already exists, it won't call the callback method - the asset store system won't generate the file again if it already exists.
+
+We'll be returning the result of this manipulation, which will be a `DBFile` containing all of the relevant information about our new variant.
+
+```php
+function (AssetStore $store, string $filename, string $hash, string $variant) use ($original) {
+    $backend = $original->getImageBackend();
+    // ...
+};
+```
+
+We define a callback, which will be called by `manipulateExtension()` if our variant file doesn't exist yet. This callback will be responsible for generating and storing the variant file.
+
+The parameters for the callback function are as follows:
+
+| Type | Name | Description |
+| -----| ---- | ----------- |
+| [`AssetStore`](api:api:SilverStripe\Assets\Storage\AssetStore) | store | The mechanism used to store the actual file |
+| `string` | filename | The name of the original file, including the original file extension |
+| `string` | hash | An sha1 hash of the original file content |
+| `string` | variant | A base64 encoded string with information about the variant file you're creating |
+
+We also want access to the original file record here so that we can use its [`Image_Backend`](api:SilverStripe\Assets\Image_Backend) to store the new file and do the conversion for us.
+
+```php
+$config = ['conflict' => AssetStore::CONFLICT_USE_EXISTING];
+$tuple = $backend->writeToStore($store, $filename, $hash, $variant, $config);
+```
+
+As mentioned earlier, Intervention Image will be converting the image for us. The `$backend` variable (unless you've replaced it with something else) is an instance of [`InterventionBackend`](api:SilverStripe\Assets\InterventionBackend) which implements `Image_Backend` and uses the Intervention Image API.
+
+The `$variant` variable holds information about the file conversion we want to make, so this line is just us saying "take this image, convert it to this new file type, and store the result."
+
+Notice that we're using the `CONFLICT_USE_EXISTING` [conflict resolution strategy](#storage-conflict-resolution). Our callback *shouldn't* be called if our variant file already exists, but just in case it does, we can just use the existing file instead of generating a new one.
+
+The value returned from `writeToStore()` is an associative array with information about the new variant file you've created.
+
+```php
+return [$tuple, $backend];
+```
+
+Finally, our callback returns both the information about the variant file and the `Image_Backend` object we used to generate it. Returning the `Image_Backend` here is important, because it will be used to perform any image-related manipulations we want to perform afterwards.
+
+Now we just need to apply the extension to both the `Image` and `DBFile` classes.
+
+```yml
+SilverStripe\Assets\Image:
+  extensions:
+    - App\Extension\ImageFormatExtension
+SilverStripe\Assets\Storage\DBFile:
+  extensions:
+    - App\Extension\ImageFormatExtension
+```
+
+You can use this method in PHP code or in templates on any instance of `Image` or `DBFile`. It will create a [variant](./file_storage/#variant-file-paths) with the new file extension.
+
+For example, if your page has a relation called `MyImage` to an `Image` record:
+
+```ss
+$MyImage.format('webp').ScaleWidth(150)
+```
+
+See [images](./images/) for more information about image-specific manipulation methods.
+
+### Converting between other formats
+
+Converting between other formats (including a non-image to an image) is a little bit more involved, because we have to find another library that will do the conversion for us and then store the new content.
+
+Below are two examples for these conversions - one where the file is converted to an image, and another where the file is converted to a PDF.
+
+These examples won't include performing the actual conversion from one format to another, because that would need to be handled by some third-party library. Instead, they demonstrate how to use the `manipulateExtension()` API to store the converted files as variants.
+
+```php
+namespace App\Extension;
+
+use SilverStripe\Assets\Image_Backend;
+use SilverStripe\Assets\Storage\AssetStore;
+use SilverStripe\Assets\Storage\DBFile;
+use SilverStripe\Core\Extension;
+use SilverStripe\Core\Injector\Injector;
+
+class FileConversionExtension extends Extension
+{
+    /**
+     * Create a variant of the file as an image.
+     *
+     * @param string $newExtension The file extension of the image to create, e.g. "webp"
+     */
+    public function toImage(string $newExtension): DBFile
+    {
+        /** Add some logic here to validate the conversion is supported */
+
+        $original = $this->getOwner();
+        return $original->manipulateExtension(
+            $newExtension,
+            function (AssetStore $store, string $filename, string $hash, string $variant) {
+                $tmpFilePath = /* some conversion logic goes here */;
+                $backend = Injector::inst()->create(Image_Backend::class);
+                $backend->loadFrom($tmpFilePath);
+                $config = ['conflict' => AssetStore::CONFLICT_USE_EXISTING];
+                $tuple = $backend->writeToStore($store, $filename, $hash, $variant, $config);
+                return [$tuple, $backend];
+            }
+        );
+    }
+
+    /**
+     * Create a variant of the file as a pdf.
+     */
+    public function toPdf(): DBFile
+    {
+        /** Add some logic here to validate the conversion is supported */
+
+        $original = $this->getOwner();
+        return $file->manipulateExtension(
+            'pdf',
+            function (AssetStore $store, string $filename, string $hash, string $variant) {
+                $tmpFilePath = /* some conversion logic goes here */;
+                $config = ['conflict' => AssetStore::CONFLICT_USE_EXISTING];
+                $tuple = $store->setFromLocalFile($tmpFilePath, $filename, $hash, $variant, $config);
+                return [$tuple, null];
+            }
+        );
+    }
+}
+```
+
+After applying the extension to both the `File` and `DBFile` classes, you can use these methods in PHP or in templates.
+
+```yml
+SilverStripe\Assets\File:
+  extensions:
+    - App\Extension\ImageFormatExtension
+SilverStripe\Assets\Storage\DBFile:
+  extensions:
+    - App\Extension\ImageFormatExtension
+```
+
+Okay, now lets step through those and take a look at what's going on. We'll only look at the parts that are different from the image-to-image conversion [we looked at earlier](#converting-between-image-formats).
+
+#### Converting something to an image
+
+The main difference between converting between images, and converting a non-image to an image, is that you have to get a third-party to perform the conversion for you.
+
+```php
+$tmpFilePath = /* some conversion logic goes here */;
+$backend = Injector::inst()->create(Image_Backend::class);
+$backend->loadFrom($tmpFilePath);
+```
+
+After the actual file conversion has happened, and you have the new file contents stored in some temporary location (e.g. using [`tmpfile`](https://www.php.net/manual/en/function.tmpfile.php)), we need to load that content into a `Image_Backend`. Unlike before, we don't have an existing image, so we need to get a new backend using the `Injector`.
+
+The rest is the same as when we were converting from an image - we still get Intervention Image to store the variant file for us, and we make sure to include the `Image_Backend` object in our returned value.
+
+#### Converting something to something else
+
+When the format we're converting to is *not* an image, things are a little simpler. Again, we have to perform the conversion ourselves.
+
+```php
+$tmpFilePath = /* some conversion logic goes here */;
+$config = ['conflict' => AssetStore::CONFLICT_USE_EXISTING];
+$tuple = $store->setFromLocalFile($tmpFilePath, $filename, $hash, $variant, $config);
+```
+
+Then, since we're not saving an image, we can just use the [normal asset store logic](#creating-new-files-in-php).
+
+```php
+return [$tuple, null];
+```
+
+Our new file variant isn't an image in this case, so we won't need access to the image manipulation methods provided by an `Image_Backend`. So instead, we just put `null` in its place.

en/04_Changelogs/5.2.0.md

@@ -13,6 +13,7 @@ title: 5.2.0 (unreleased)
   - [Create random passwords for new users](#create-random-passwords-for-new-users)
   - [Buttons to select all files and deselect all files](#bulk-action-buttons)
   - [New searchable dropdown fields](#searchable-dropdown-field)
+  - [Create file variants with different extensions](#file-variants)
   - [More nuanced permissions for `/dev/*` routes](#dev-route-permissions)
   - [New exception in React forms](#react-forms-exception)
   - [Other new features](#other-new-features)
@@ -174,6 +175,18 @@ A `SearchableDropdownField` will now be used when automatically scaffolding `has
 
 Previously the [`DBForeignKey.dropdown_field_threshold`](api:SilverStripe\ORM\FieldType\DBForeignKey->dropdown_field_threshold) config property was used as the threshold of the number of options to decide when to switch between auto-scaffolding a `DropdownField` and a `NumericField`. This configuration property is now used as the threshold of the number of options to decide when to start using lazy-loading for the `SearchableDropdownField`.
 
+### Create file variants with different extensions {#file-variants}
+
+A low-level API has been added which allows the creation of file variants with a different extension than the original extension. A file variant is a manipulated version of the original file - for example if you resize an image or convert a file to another format, this will generate a variant (leaving the original file intact).
+
+Some examples of when you might want this are:
+
+- Generating thumbnails for videos, documents, etc
+- Converting images to `.webp` for faster page load times
+- Converting documents to `.pdf` so downloaded documents are more portable
+
+See [file manipulation](/developer_guides/files/file_manipulation/#convert-a-file-to-a-different-format) for details about how to use this new API.
+
 ### More nuanced permissions for `/dev/*` routes {#dev-route-permissions}
 
 Previously, all `/dev/*` routes registered with [`DevelopmentAdmin`](api:SilverStripe\Dev\DevelopmentAdmin) (for example `/dev/tasks/MyBuildTask`) could only be access by administrator users, and this couldn't be configured.