Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use postgresql recursive feature for getting scheduled product #5398

Merged
merged 1 commit into from
Dec 15, 2023

Conversation

perlpunk
Copy link
Contributor

@perlpunk perlpunk commented Dec 14, 2023

For longer chains of clones it's more efficient. It also avoids a warning:

Deep recursion on subroutine "OpenQA::Schema::Result::Jobs::related_scheduled_product_id"

Issue: https://progress.opensuse.org/issues/152569

@kalikiana
Copy link
Member

For longer chains of clones it's more efiicient. It also avoids a warning:

Typo: efficient

The SQL looks correct, but I'm not sure how to tell if it's more or less efficient.

@perlpunk
Copy link
Contributor Author

perlpunk commented Dec 14, 2023

Regarding efficiency:

openqa=> explain analyze with recursive orig_id as (
            select (13062217)::bigint as orig_id, (null)::bigint as prod
            union all
select id as orig_id, scheduled_product_id as prod from jobs join orig_id on orig_id.orig_id = jobs.clone_id)
select orig_id, prod from orig_id where prod is not null order by orig_id desc;
                                                                   QUERY PLAN                                                                   
------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=854.62..854.87 rows=100 width=16) (actual time=10.769..10.770 rows=1 loops=1)
   Sort Key: orig_id.orig_id DESC
   Sort Method: quicksort  Memory: 25kB
   CTE orig_id
     ->  Recursive Union  (cost=0.00..849.28 rows=101 width=16) (actual time=0.002..10.420 rows=1099 loops=1)
           ->  Result  (cost=0.00..0.01 rows=1 width=16) (actual time=0.001..0.001 rows=1 loops=1)
           ->  Nested Loop  (cost=0.42..84.72 rows=10 width=16) (actual time=0.009..0.009 rows=1 loops=1099)
                 ->  WorkTable Scan on orig_id orig_id_1  (cost=0.00..0.20 rows=10 width=8) (actual time=0.000..0.000 rows=1 loops=1099)
                 ->  Index Scan using jobs_idx_clone_id on jobs  (cost=0.42..8.44 rows=1 width=24) (actual time=0.008..0.008 rows=1 loops=1099)
                       Index Cond: (clone_id = orig_id_1.orig_id)
   ->  CTE Scan on orig_id  (cost=0.00..2.02 rows=100 width=16) (actual time=10.757..10.764 rows=1 loops=1)
         Filter: (prod IS NOT NULL)
         Rows Removed by Filter: 1098
 Planning Time: 0.187 ms
 Execution Time: 10.811 ms
(15 rows)

And that example is a job in osd which has about 1100 ancestors.

@perlpunk
Copy link
Contributor Author

And we're using that kind of SQL already in ancestors and descendants

For longer chains of clones it's more efficient. It also avoids a warning:

    Deep recursion on subroutine "OpenQA::Schema::Result::Jobs::related_scheduled_product_id"

Issue: https://progress.opensuse.org/issues/152569
Copy link

codecov bot commented Dec 14, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e68161c) 98.36% compared to head (e56b56e) 98.37%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5398   +/-   ##
=======================================
  Coverage   98.36%   98.37%           
=======================================
  Files         389      389           
  Lines       37534    37548   +14     
=======================================
+ Hits        36922    36936   +14     
  Misses        612      612           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mergify mergify bot merged commit dcd6122 into os-autoinst:master Dec 15, 2023
36 checks passed
@perlpunk perlpunk deleted the recursive-product-id branch December 15, 2023 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants