Slugify Collectives and Pages

[Koc]

📝 Summary

This is a very beginning of my PR to slugify urls for Collectives and Pages. For now urls are too long and hard to understand if using cyrylic in this entities.

Here an example of some of them: http://localhost/index.php/apps/collectives/%D0%A3%D0%BA%D1%80%D0%B0%D1%97%D0%BD%D1%81%D1%8C%D0%BA%D1%96%20%D1%84%D1%83%D1%82%D0%B1%D0%BE%D0%BB%D1%96%D1%81%D1%82%D0%B8/%D0%AF%D1%80%D0%BC%D0%BE%D0%BB%D0%B5%D0%BD%D0%BA%D0%BE%20%D0%90%D0%BD%D0%B4%D1%80%D1%96%D0%B9%20%D0%9C%D0%B8%D0%BA%D0%BE%D0%BB%D0%B0%D0%B9%D0%BE%D0%B2%D0%B8%D1%87?fileId=187. Just compare it with slugified version: http://localhost/index-php/apps/collectives/ukrayinski-futbolisti-1/page-1-yurchenko-andriy-mykolayovich. Easy to read, 3 times shorter.

🖼️ Video

Screencast.from.2024-09-01.17-18-46.webm

🚧 TODO

• agreed on library which should be used for slug generation
• generate slugs for Collectives/Pages via migration
• resolve leftover fixmes
• add tests

🏁 Checklist

• Code is properly formatted (npm run lint / npm run stylelint / composer run cs:check)
• Sign-off message is added to all commits
• Tests (unit, integration and/or end-to-end) passing and the changes are covered with tests
• Documentation (README or documentation) has been updated or is not required

[mejo-]

Hey @Koc, thanks for looking into this. Definitely a good idea to slugify collectives and page titles for cleaner URLs.

Without looking into all the details, I wonder if we could implement this in a way that doesn't need the sluggified string to be persisted in the database.

Why not make calculate the slug of titles/names on demand in the backend and add them as attribute to the collectives and pages in the API responses? Then the frontend could always check for name/title match first and for slug match second. What do you think about this approach?

[Koc]

@mejo- we have a lot of pages in collectives, so I would like to reduce amount of runtime operations to speedup backend endpoints.

[Koc]

I've achieved some progress. Now it more or less works, we have even redirects if slugs have been renamed. You can observe how it works in the video that attached to PR description

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

[Koc]

@mejo- there is one more reason to persist slug in the database: we can make it editable in upcoming PRs. Sometimes page titles can be long, so we can give possibility to our users shorten them

[Koc]

For some reason migrations stop work for me (even previous). Both autoloading and dependency injection resolving not works. Any idea how to fix that?

devcontainer@5e275b290d01:/var/www/html$ ./occ migrations:migrate collectives -v

In MigrationService.php line 399:
                                                                                         
  [Exception]                                                                            
  Database error when running migration 021200Date20240725154232 for app collectives     
  Migration step 'OCA\Collectives\Migration\Version021200Date20240725154232' is unknown  
                                                                                         

Exception trace:
  at /var/www/html/lib/private/DB/MigrationService.php:399
 OC\DB\MigrationService->migrate() at /var/www/html/core/Command/Db/Migrations/MigrateCommand.php:42
 OC\Core\Command\Db\Migrations\MigrateCommand->execute() at /var/www/html/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/html/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/html/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/html/lib/private/Console/Application.php:187
 OC\Console\Application->run() at /var/www/html/console.php:87
 require_once() at /var/www/html/occ:11

In MigrationService.php line 482:
                                                                                         
  [InvalidArgumentException]                                                             
  Migration step 'OCA\Collectives\Migration\Version021200Date20240725154232' is unknown  
                                                                                         

Exception trace:
  at /var/www/html/lib/private/DB/MigrationService.php:482
 OC\DB\MigrationService->createInstance() at /var/www/html/lib/private/DB/MigrationService.php:497
 OC\DB\MigrationService->executeStep() at /var/www/html/lib/private/DB/MigrationService.php:395
 OC\DB\MigrationService->migrate() at /var/www/html/core/Command/Db/Migrations/MigrateCommand.php:42
 OC\Core\Command\Db\Migrations\MigrateCommand->execute() at /var/www/html/3rdparty/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/html/3rdparty/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/3rdparty/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/html/3rdparty/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/html/lib/private/Console/Application.php:187
 OC\Console\Application->run() at /var/www/html/console.php:87
 require_once() at /var/www/html/occ:11

migrations:migrate <app> [<version>]

[mejo-]

Thanks for the PR, @Koc, much appreciated! Sorry for taking so long with a proper review.

I have a few comments and ran into a few problems testing it, but I also have s general topic that I would like to discuss within our team:

With the PR, Collectives URLs will always include collectiveId and pageId, e.g. /apps/collectives/MyCollective/MyPage?fileId=123 will become /apps/collectives/MyCollective-1/MyPage-123 in the future. I see the advantages and I don't have strong objections, but it's a user experience change and makes the URLs less readable for collectives and pages that only had short ascii names anyway. I think this is out-weighted by the fact that URLs become way cleaner by longer and non-ascii names. The IDs are required to prevent duplicates - and they also allow cleaner URL parsing in the frontend.

@juliusknorr @max-nextcloud @jancborchardt @nimishavijay what do you think about this?

[mejo-]

Unfortunately this won't work for trashed pages and neither for trashed collectives. I guess it's cleaner to write a dedicated function that unconditionally gets the filenames for all pages from oc_collectives_pages for a specific collective and iterates over it.

Please make sure you have trashed collectives and active collectives with trashed pages when testing it 😉

[Koc]

I will double check with trashed collectives/pages but IMHO this is fine™. Code should work properly even if slugs are not generated.

[mejo-]

Well, with a collective in the collectives trash, the commented line throws a NotFoundException for me. This can be mitigated by adjusting the query to fetch collectives to only take into account non-trashed collectives:

--- a/lib/Migration/Version021500Date20240820000001.php
+++ b/lib/Migration/Version021500Date20240820000001.php
@@ -57,7 +57,9 @@ class Version021500Date20240820000001 extends SimpleMigrationStep {
                }
 
                $queryCollectives = $this->connection->getQueryBuilder();
-               $queryCollectives->select(['id', 'circle_unique_id'])->from('collectives');
+               $queryCollectives->select(['id', 'circle_unique_id'])
+                       ->from('collectives')
+                       ->where('trash_timestamp IS NULL');
                $resultCollectives = $queryCollectives->executeQuery();
 
                $queryPages = $this->connection->getQueryBuilder();

But still the commented line breaks in my tests with NotFoundException as the collective folder cannot be accessed - at least when I run the migration by calling occ upgrade on the commandline. I would have to further look into it, but I guess it's because the occ command doesn't run as a specific user, so the collective mountpoints are not accessible.

Could you check if the migration works smoothly if you have different users with collectives and pages and if you trigger the migration via running occ upgrade?

You can easily test the migrations again after the following SQL statements:

ALTER TABLE oc_collectives DROP COLUMN `slug`;
ALTER TABLE oc_collectives_pages DROP COLUMN `slug`;
DELETE FROM oc_migrations WHERE app='collectives' AND version='021500Date20240820000000';
DELETE FROM oc_migrations WHERE app='collectives' AND version='021500Date20240820000001';

Since the migration versions changed now, you might have to adjust this.

Afterwards just change the app version number in appinfo/info.xml (doesn't matter if you bump or lower it). This will make Nextcloud run the upgrade steps which will run all migrations that are not registered as already executed.

If I do this with pages in the trash, the commented line will throw a NotFoundException. This can be mitigated by the following code:

try {
	$pageInfos = $this->service->findAll($rowCollective['id'], $circle->getOwner()->getUserId());
	} catch (NotFoundException

[mejo-]

I guess that you wanted to pass the $fileId here and add it to the sluggified title in generatePageSlug?

[mejo-]

This still seems broken to me. The backend code doesn't add the page fileId to the slug, but the frontend code expects it to be there.

[Koc]

no, it's not needed. We're matching pageId and pageSlug separately on FE. This needed because slug can change and we should redirect user to a new slug based on pageId

[jancborchardt]

@Koc nice change, good for internationalization. :)

@mejo- I'm all for it too, the drawback is very very small it seems.