Automatic screenshot capture not working with rspec-steps

[ramunasm]

Hello there,

I am doing some smoke testing and I use rspec-steps with capybara and poltergeist.
When running the tests I found out that automatic screenshot feature only works for the first step definition, for all others I get no screenshot.

Code:

require 'capybara/rspec'
require 'capybara/poltergeist'
require 'rspec-steps'
require 'capybara-screenshot/rspec'

options = {
    :js_errors => false,
    :debug => false,
}

Capybara.register_driver :poltergeist do |app|
    Capybara::Poltergeist::Driver.new(app, options)
end

Capybara.default_driver = :poltergeist
Capybara.javascript_driver = :poltergeist
session = Capybara::Session.new(:poltergeist)

site_url = 'https://delfi.lt'

RSpec::Steps.steps "Smoke test", :type => :feature do
  it "should show the site" do
        visit(site_url)
  end
  it "should check for the element" do
        expect(page). to have_css '#some-non-existant-element'
  end
end

If I move the expect(page). to have_css '#some-non-existant-element' line to the first block, right after visit(site_url) it works just fine.

Cmd output with a run which doesn't generate a screenshot:

J:\Desktop\Smoke-tests>rspec bug.rb
.F

Failures:

  1) Smoke test should check for the element
     Failure/Error: expect(page). to have_css '#some-non-existant-element'
       expected to find css "#some-non-existant-element" but there were no match
es
     # ./bug.rb:27:in `block (2 levels) in <top (required)>'

Finished in 7.81 seconds (files took 2.13 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./bug.rb:26 # Smoke test should check for the element

Am I doing something wrong?

Thank you

[ramunasm]

I guess this is more of a feature request than a bug, as rspec-steps isn't officially supported by capybara-screenshot

In any case, here's what I have managed to find out:
It seems that rspec-steps never sets example.exception and this is why it never gets past this line:
if Capybara.page.current_url != '' && Capybara::Screenshot.autosave_on_failure && example.exception in rspec.rb file

It seems that rspec-steps is using failed_step.exception instead.
I don't have enough knowledge of Ruby to make this work myself.
Could anyone take a quick look at this?

Thank you!

[Zeragamba]

Ran into this problem as well

[optix2000]

Dug into this and it seems more of an issue with rspec-steps than capybara-screenshot.

RSpec.current_example is constantly stuck to the first step (See LRDesign/rspec-steps#34), which means that an example.exception is always that of the first example, and thus if the first example passes, example.exception is always nil.

For example, with the following spec:

require 'capybara'
require 'capybara/rspec'
require 'capybara/poltergeist'
require 'capybara-screenshot/rspec'
require 'rspec-steps'
require 'pry'

Capybara.configure do |c|
  c.javascript_driver = :poltergeist
  c.run_server = false
  c.app_host = 'http://app.lvh.me:3000'
end

RSpec::Steps.steps 'Smoke Tests', type: :feature, js: true do
  it 'Can visit page' do
    visit '/'
  end

  it 'Can do stuff' do
    binding.pry
    click_on 'Break test with nonexistant button'

    expect(page).to have_current_path('done')
  end
  it 'Can do more stuff' do
    visit '/'
  end
end

When we hit binding.pry, RSpec.current_example is wrong.

    18:   end
    19:
    20:   it 'Can do stuff' do
    21:     binding.pry
 => 22:     click_on 'Break test with nonexistant button'
    23:
    24:     expect(page).to have_current_path('done')
    25:   end
    26:   it 'Can do more stuff' do

[2] pry(#<RSpec::ExampleGroups::SmokeTests>)> RSpec.current_example
=> #<RSpec::Core::Example "Can visit page">

This propagates itself into capybara-screenshot when the example fails

    52: def after_failed_example(example)
    53:   if example.example_group.include?(Capybara::DSL) # Capybara DSL method has been included for a feature we can snapshot
    54:     Capybara.using_session(Capybara::Screenshot.final_session_name) do
    55:       require 'pry'
    56:       binding.pry
 => 57:       if Capybara.page.current_url != '' && Capybara::Screenshot.autosave_on_failure && example.exception
    58:         filename_prefix = Capybara::Screenshot.filename_prefix_for(:rspec, example)
    59:
    60:         saver = Capybara::Screenshot.new_saver(Capybara, Capybara.page, true, filename_prefix)
    61:         saver.save
    62:
    63:         example.metadata[:screenshot] = {}
    64:         example.metadata[:screenshot][:html]  = saver.html_path if saver.html_saved?
    65:         example.metadata[:screenshot][:image] = saver.screenshot_path if saver.screenshot_saved?
    66:       end
    67:     end
    68:   end
    69: end

[1] pry(Capybara::Screenshot::RSpec)> example
=> #<RSpec::Core::Example "Can visit page">
[2] pry(Capybara::Screenshot::RSpec)> example.exception
=> nil

And we see that example.exception doesn't exist as it's pointing to the first example, which succeeded.

[mattheworiordan]

@optix2000 thanks for the analysis.

I am not familiar with RSpec steps, but I assume it must expose an API to access the step exceptions? If so, do you fancy suggesting a fix with a PR perhaps?

[optix2000]

Hey @mattheworiordan,

From my cursory read of rspec-steps, I can't seem to find any API to access the actual steps, or their exceptions, so it may be non-trivial to work around this in capybara-screenshot.

Unfortunately, I don't think I'll have the time, nor am I familiar enough with RSpec to fix the issue in rspec-steps.

[mattheworiordan]

Unfortunately, I don't think I'll have the time, nor am I familiar enough with RSpec to fix the issue in rspec-steps.

Ok, that's a shame. Well if you think this becomes a big enough problem for yourself, hopefully you'll find the time and be able to make an open source contribution to this gem. I'd be very grateful, as I am sure everyone using rspec-steps would be! 🙏

[optix2000]

I may consider it as I get more familiar with Ruby and RSpec, though as it is now, everything is too magic.

I did write up a quick alternative that approximates rspec-steps available here:
https://gist.github.com/optix2000/4328ea9b4eb5b3b1e827a086a207b7e7