Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add significant performance improvements for composer v2 #608

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

xvilo
Copy link
Contributor

@xvilo xvilo commented Sep 6, 2022

Currently, Repman is quite slow on the /packages.json endpoint, this due it's requesting all JSON files for every package from S3. From composer 2.0 and onwards, the API has been changed, it does not need to have all packages available any more. But, it now may provide the providers-url and metadata-url. This is where information for a specific package can be found. Thus, it does not have to wait for all packages to be resolved AND it's able to request info about the needed packages only.

If people are still using composer 1.0 it will add back the old data, so performance impact will only be solved for people using v2. It will however, show an extra warning.

Curl request statistics for /packages.json repo endpoint:

Currently:

{
"time_redirect": 0.000000,
"time_namelookup": 0.193779,
"time_connect": 0.203508,
"time_appconnect": 0.229232,
"time_pretransfer": 0.230318,
"time_starttransfer": 19.032120,
"time_total": 23.855557,
"size_request": 107,
"size_upload": 0,
"size_download": 1269753,
"size_header": 352
}

with these changes:

{
"time_redirect": 0.000000,
"time_namelookup": 0.018870,
"time_connect": 0.022539,
"time_appconnect": 0.042854,
"time_pretransfer": 0.043032,
"time_starttransfer": 0.260770,
"time_total": 0.260906,
"size_request": 177,
"size_upload": 0,
"size_download": 3345,
"size_header": 352
}

Looking at the total request time for our internal production deployment, we're going from a request time of 24 seconds, to around 260 milliseconds.

@xvilo xvilo changed the title Better v2 implementation Draft: Add significant performance improvements for composer v2 Sep 6, 2022
@codecov
Copy link

codecov bot commented Sep 6, 2022

Codecov Report

Merging #608 (180897b) into master (f82251f) will decrease coverage by 0.01%.
The diff coverage is 96.96%.

@@             Coverage Diff              @@
##             master     #608      +/-   ##
============================================
- Coverage     99.16%   99.14%   -0.02%     
- Complexity     1910     1917       +7     
============================================
  Files           301      303       +2     
  Lines          6072     6092      +20     
============================================
+ Hits           6021     6040      +19     
- Misses           51       52       +1     
Impacted Files Coverage Δ
src/Controller/RepoController.php 98.70% <95.00%> (-1.30%) ⬇️
src/Service/Composer/ComposerEnvironment.php 100.00% <100.00%> (ø)
...rc/Service/Composer/ComposerEnvironmentFactory.php 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@xvilo xvilo changed the title Draft: Add significant performance improvements for composer v2 Add significant performance improvements for composer v2 Sep 6, 2022
@xvilo
Copy link
Contributor Author

xvilo commented Sep 6, 2022

Possible todo: add test for composer v1 and V2 package endpoint differences. Also test this with an invalid user agent. This will fix the missing coverage

@akondas
Copy link
Member

akondas commented Sep 10, 2022

I'm willing to accept this ... but can you add more tests for that? I'm not sure if i'm follow (sorry) and tests will give me more confidence 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants