Orbit should retry extension registry in case of failure

[lucasmrod]

Goal

The issue with #3678 was that occasionally extension registry took more than 3s, so, Orbit, after timing out, was bringing both itself and osquery down (due to our use of oklog.Execute).

We fixed this in #3836 by increasing the timeout for Orbit's extension registry from 3s to 5m.

Zach also proposed retrying extension registering in Runner.Execute (see #3836 (comment)):
The retry would increase reliability in case of some (timeout and non-timeout) failure of the extension runner or any other issue in osquery.

We are currently just exiting the Runner.Execute function when Run fails, we should instead retry:

fleet/orbit/pkg/table/extension.go

Lines 77 to 79 in 468b67c

	if err := r.srv.Run(); err != nil {
	return err
	}

[lucasmrod]

Some bad news:

Even with the 5m timeout, I've hit the "i/o timeout error while registering extensions" in our CI/automation on Windows: https://github.com/fleetdm/fleet/actions/runs/1986053580/attempts/1

So, I think we should do both:

1. Implement Zach's proposal to retry extension registration instead of failing (this issue).
2. Create a separate issue to troubleshoot what's really going on. It's probably a bug in osquery or osquery-go (Extension cannot connect on first try when loaded automatically osquery/osquery-go#80).

[zhumo]

What's the status of this?

[lucasmrod]

Extension registering in Windows seems to be more stable in recent releases. There have been changes like this one that could have possibly help with this issue.

We haven't received reports of this issue for a while now. Let's close and re-open if need be.