Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zeitwerk #22488

Merged
merged 6 commits into from
Aug 23, 2023
Merged

Zeitwerk #22488

merged 6 commits into from
Aug 23, 2023

Conversation

jrafanie
Copy link
Member

@jrafanie jrafanie commented May 2, 2023

Depends on:

This PR enabled zeitwerk. It adds some acronyms/inflections that are backward compatible but required for zeitwerk. Most of the extractable content has been extracted in other PRs.

Regarding the debug flag DEBUG_MANAGEIQ_ZEITWERK:

We have a debug flag to enable logging and testing of the plugin autoload paths by eager loading them:

With no debug flag, we get not debug output and engines autoload paths are not eager loaded:

 % bin/rails zeitwerk:check
:zeitwerk is the configured autoloader. This patched file should be removed: /Users/joerafaniello/Code/manageiq/lib/extensions/as_dependencies_interlock.rb once support for the classic loader is dropped. Additionally remove all uses of AS::Depedencies.interlock.
** ManageIQ master, codename: Quinteros
Hold on, I am eager loading the application.

WARNING: The following directories will only be checked if you configure
them to be eager loaded:

  /Users/joerafaniello/Code/manageiq/app/channels
  /Users/joerafaniello/Code/manageiq/app/mailers
  /Users/joerafaniello/Code/manageiq/app/models
  /Users/joerafaniello/Code/manageiq/app/models/aliases
  /Users/joerafaniello/Code/manageiq/app/models/mixins
  /Users/joerafaniello/Code/manageiq/lib
  /Users/joerafaniello/Code/manageiq/lib/services
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-ui-classic-cc269e54618e/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-decorators-d32a9713e7e7/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-consumption-377567af9e99/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-content-e48d454fbe1d/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-api-c49eac7c3a2e/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-api-c49eac7c3a2e/lib/services
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-automation_engine-04cf2b9720ad/app/models/mixins
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-automation_engine-04cf2b9720ad/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-automation_engine-04cf2b9720ad/lib/miq_automation_engine
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-automation_engine-04cf2b9720ad/lib/miq_automation_engine/engine
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-workflows-d2bca8106496/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-vmware-e045cb18a750/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ovirt-94295f48af08/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-openstack-0cec9e559744/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-openshift-f2a2efdbf167/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-red_hat_virtualization-faf084bd29ee/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-redfish-5857bf0e3e45/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-oracle_cloud-80b43001a055/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-nuage-24aeaae5eac1/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-nsxt-025179cf5ec2/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-nsxt-025179cf5ec2/app/helpers/helper
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-lenovo-d442875edc73/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-kubevirt-c18da31236be/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-kubernetes-6acc47038ceb/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ibm_terraform-c1e38d12b021/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ibm_power_vc-af22b799ed70/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ibm_power_hmc-ba995d7c12a7/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ibm_cloud-0bd58c1075ca/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ibm_cic-9bdb24cbfa38/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-google-c6ccbb837a66/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-foreman-73663c08eb5e/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-cisco_intersight-3c74cf9a0ade/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-azure_stack-9b712689921f/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-azure-cd77aa129982/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-awx-25902304ac63/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-autosde-0671b7003471/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-ansible_tower-293a57d01191/lib
  /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-amazon-7d71bb678dc3/lib

You may verify them manually, or add them to config.eager_load_paths
in config/application.rb and run zeitwerk:check again.

Otherwise, all is good!

With debugging, we get debug logging and we eager load all autoload paths to ensure those plugins define things correctly:

% DEBUG_MANAGEIQ_ZEITWERK=true bin/rails zeitwerk:check
...
D, [2023-06-05T14:05:41.045721 #44126] DEBUG -- : [email protected]: eager load directory /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-amazon-7d71bb678dc3/app/models/manageiq/providers/amazon/storage_manager/s3/refresh_worker end
D, [2023-06-05T14:05:41.045732 #44126] DEBUG -- : [email protected]: eager load directory /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-schema-31a4f46a4493/app/models start
D, [2023-06-05T14:05:41.046027 #44126] DEBUG -- : [email protected]: constant SchemaMigration loaded from file /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-schema-31a4f46a4493/app/models/schema_migration.rb
D, [2023-06-05T14:05:41.046038 #44126] DEBUG -- : [email protected]: eager load directory /Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-schema-31a4f46a4493/app/models end
D, [2023-06-05T14:05:41.046074 #44126] DEBUG -- : [email protected]: eager load end
All is good!

Comment on lines 100 to 101
# config.load_defaults 6.1
# Disable defaults as ActiveRecord::Base.belongs_to_required_by_default = true causes MiqRegion.seed to fail validation on belongs_to :maintenacne zone
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to remind us later to try to move back to defaults?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, this is still wip and has some things I want to review as we get closer.

Comment on lines 5 to 11
# These specific directories are for code organization, not namespacing:
Rails.autoloaders.main.collapse(Rails.root.join("lib/manageiq/reporting/charting"))
Rails.autoloaders.main.collapse(Rails.root.join("lib/ansible/runner/credential"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I presume we should fix these in a follow up to be properly namespaced?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, probably... I have a few things in here that will likely need followup issues.

/Users/joerafaniello/.gem/ruby/3.0.6/bundler/gems/manageiq-providers-nuage-d87b99b29624/lib/manageiq-providers-nuage.rb
])

# requires qpid, which isn't available on my system
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to handle these "optional" requires?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I still need to figure this out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I just programmatically build a path to the file.

  # requires qpid, which is an optional dependency because LOL mac
  message_handler_path = Pathname.new(Vmdb::Plugins.paths[ManageIQ::Providers::Nuage::Engine]).join("app/models/manageiq/providers/nuage/network_manager/event_catcher/messaging_handler.rb")
  Rails.autoloaders.main.ignore(message_handler_path)

@kbrock
Copy link
Member

kbrock commented May 22, 2023

Does feel like the require and the inflection changes are non-zeitwerk friendly
Do we want to pull that out and get it in?

Or if we are about ready for the big bang, then that works too

@jrafanie
Copy link
Member Author

jrafanie commented Jun 5, 2023

Does feel like the require and the inflection changes are non-zeitwerk friendly Do we want to pull that out and get it in?

Or if we are about ready for the big bang, then that works too

Yes, @kbrock thanks... 2 weeks later, here's that PR: #22548

@jrafanie jrafanie force-pushed the zeitwerk branch 2 times, most recently from 7f97ba0 to 2724710 Compare June 5, 2023 18:18
Gemfile Outdated
@@ -9,7 +9,7 @@ require File.join(Bundler::Plugin.index.load_paths("bundler-inject")[0], "bundle
#
# VMDB specific gems
#
gem "manageiq-gems-pending", ">0", :require => 'manageiq-gems-pending', :git => "https://github.com/ManageIQ/manageiq-gems-pending.git", :branch => "master"
gem "manageiq-gems-pending", ">0", :git => "https://github.com/jrafanie/manageiq-gems-pending.git", :branch => "zeitwerk"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO, remove this once gem has been updated

@jrafanie jrafanie closed this Jun 6, 2023
@jrafanie jrafanie reopened this Jun 6, 2023
@@ -6,6 +6,7 @@
let(:missing_model) { model_directory.join("zzz_model.rb") }

before do
skip "This is currently not testable with zeitwerk!" if Rails.application.config.autoloader == :zeitwerk
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried many things to test this with zeitwerk but either can't modify the autoload paths OR, it's too late and the changes to these paths aren't used. Basically, we can't do proper setup/teardown to test this with zeitwerk. Cross repo fails in manageiq-providers-vmware if you remove the yaml autoloader, see above.

Nuage depends on qpid_proton which we can't require on mac

In dev, if we eager load the application, such as with `rake zeitwerk:check`, this file blows up
on mac because qpid_proton is an optional dependency but actively referenced in messaging_handler.rb.

Only ignore loading qpid_proton here on mac.
Add note about rails 6.1 defaults for when we start work on rails 7:
Disable defaults as ActiveRecord::Base.belongs_to_required_by_default = true
causes MiqRegion.seed to fail validation on belongs_to :maintenacne zone.
Fixes the following with zeitwerk when running the spec/lib/pdf_generator_spec.rb test:

expected file /Users/joerafaniello/Code/manageiq/lib/pdf_generator/null_pdf_generator.rb to define constant PdfGenerator::NullPdfGenerator, but didn't
We can't modify the frozen autoload paths with zeitwerk.

For now, just disable this test with zeitwerk loader.
It's far easier to debug in dev/test/containers using an ENV variable
so we'll make it so zeitwerk stdout logging and eager loading of all engines
is only done with this flag enabled.
@miq-bot
Copy link
Member

miq-bot commented Aug 21, 2023

Some comments on commits jrafanie/manageiq@17a20e8~...738b833

config/initializers/zeitwerk.rb

  • ⚠️ - 3 - Detected puts. Remove all debugging statements.

@miq-bot
Copy link
Member

miq-bot commented Aug 21, 2023

Checked commits jrafanie/manageiq@17a20e8~...738b833 with ruby 2.6.10, rubocop 1.28.2, haml-lint 0.35.0, and yamllint
4 files checked, 0 offenses detected
Everything looks fine. 🍰

@agrare agrare self-assigned this Aug 23, 2023
@agrare agrare merged commit 7ac8b82 into ManageIQ:master Aug 23, 2023
5 of 9 checks passed
@agrare agrare changed the title [WIP] Zeitwerk Zeitwerk Aug 23, 2023
@jrafanie jrafanie deleted the zeitwerk branch August 23, 2023 18:09
@Fryguy
Copy link
Member

Fryguy commented Oct 27, 2023

Skipping backport to quinteros, because it is already in the branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants