3865 – Fix issue where death handler was getting NilClass instead of …

…Exception (#409)

* add begin rescue to delayed job

* update death handler to notify when it's not a RetryLimitHit

We were getting some errors on Sentry where we expected the argument to be an Exception, but got NilClass. We think that was happening because some errors where happening during a delayed job (get_archive_org_status), so our handle_archiving_exceptions wasn't handling those.

Meaning they would be retried multiple times without being re-raised as RetryLater. When they hit the death handler limit_hit_exception would be nil because they would fail if original_exception.is_a?(Pender::Exception::RetryLater)

We are now handling errors inside get_archive_org_status, but we are adding the else to the death handler as an extra safety measure. We want to see what kind of errors we get for a while and then decide if this i needed or not, and what the best way to handle this is.

References: 3865

app/models/concerns/media_archive_org_archiver.rb

@@ -58,16 +58,20 @@ def get_available_archive_org_snapshot(url, key_id)
     end
 
     def get_archive_org_status(job_id, url, key_id)
-      http, request = Media.archive_org_request("https://web.archive.org/save/status/#{job_id}", 'Get')
-      response = http.request(request)
-      body = JSON.parse(response.body)
-      if body['status'] == 'success'
-        location = "https://web.archive.org/web/#{body['timestamp']}/#{url}"
-        data = { location: location }
-        Media.notify_webhook_and_update_cache('archive_org', url, data, key_id)
-      else
-        message = body['status'] == 'pending' ? 'Capture is pending' : "(#{body['status_ext']}) #{body['message']}"
-        raise Pender::Exception::RetryLater, message
+      begin
+        http, request = Media.archive_org_request("https://web.archive.org/save/status/#{job_id}", 'Get')
+        response = http.request(request)
+        body = JSON.parse(response.body)
+        if body['status'] == 'success'
+          location = "https://web.archive.org/web/#{body['timestamp']}/#{url}"
+          data = { location: location }
+          Media.notify_webhook_and_update_cache('archive_org', url, data, key_id)
+        else
+          message = body['status'] == 'pending' ? 'Capture is pending' : "(#{body['status_ext']}) #{body['message']}"
+          raise Pender::Exception::RetryLater, message
+        end
+      rescue StandardError => error
+        raise Pender::Exception::RetryLater, error.message
       end
     end
 

config/initializers/03_sidekiq.rb

@@ -13,8 +13,10 @@
     config.death_handlers << ->(job, original_exception) do
       if original_exception.is_a?(Pender::Exception::RetryLater)
         limit_hit_exception = Pender::Exception::RetryLimitHit.new(original_exception)
+        PenderSentry.notify(limit_hit_exception, { job: job, original_exception: original_exception.cause.inspect })
+      else
+        PenderSentry.notify(original_exception, { job: job })
       end
-      PenderSentry.notify(limit_hit_exception, {job: job, original_exception: original_exception.cause.inspect})
     end
   end