Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spool drops envelopes on shutdown #3862

Open
klemmchr opened this issue Jul 25, 2024 · 8 comments
Open

Spool drops envelopes on shutdown #3862

klemmchr opened this issue Jul 25, 2024 · 8 comments
Assignees
@jjbayer @iambriccardo

Comments

@klemmchr
Copy link

klemmchr commented Jul 25, 2024 •
edited
Loading

Received this error today twice: dropped envelope: internal error

Relevant Relay logs (more below):

2024-07-25T05:04:48.780180Z ERROR relay_server::services::spooler: dropped 2 envelopes
2024-07-25T05:04:48.780581Z ERROR relay_server::utils::managed_envelope: dropped envelope: internal error tags.project_key="xxxxxxxxxxxxx" tags.has_attachments=false tags.has_sessions=false tags.has_profiles=false tags.has_transactions=false tags.has_span_metrics=false tags.has_replays=false tags.has_checkins=false tags.event_category=Some(Transaction) cached_summary=EnvelopeSummary { event_category: Some(Transaction), attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 2048, profile_chunk_quantity: 0 } recomputed_summary=EnvelopeSummary { event_category: Some(Transaction), attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 2048, profile_chunk_quantity: 0 }
Error Json 
{
    "event_id": "875c6a9ed0a24206aa86eb3ba5f5b46e",
    "project": 00000000000000,
    "release": "[email protected]+dd5bbcc27d639e8fd07091506e736ec3ffc49a86",
    "dist": null,
    "platform": "native",
    "message": "dropped envelope: internal error",
    "datetime": "2024-07-25T05:04:48+00:00",
    "tags": [
        [
            "environment",
            "production"
        ],
        [
            "event_category",
            "None"
        ],
        [
            "has_attachments",
            "false"
        ],
        [
            "has_checkins",
            "false"
        ],
        [
            "has_profiles",
            "false"
        ],
        [
            "has_replays",
            "false"
        ],
        [
            "has_sessions",
            "false"
        ],
        [
            "has_span_metrics",
            "false"
        ],
        [
            "has_transactions",
            "false"
        ],
        [
            "level",
            "error"
        ],
        [
            "logger",
            "relay_server::utils::managed_envelope"
        ],
        [
            "os",
            "Linux 5.15.153.1-1.cm2"
        ],
        [
            "os.name",
            "Linux"
        ],
        [
            "project_key",
            "xxxxxxxxxxxxxxxx"
        ],
        [
            "rust",
            "rustc 1.79.0"
        ],
        [
            "rust.name",
            "rustc"
        ],
        [
            "release",
            "[email protected]+dd5bbcc27d639e8fd07091506e736ec3ffc49a86"
        ],
        [
            "server_name",
            "ess-we-mas-sen-ca-001--muankxv-5b8f9c6954-mx2cq"
        ]
    ],
    "_meta": {
        "breadcrumbs": {
            "values": {
                "9": {
                    "message": {
                        "": {
                            "rem": [
                                [
                                    "@ip:replace",
                                    "s",
                                    22,
                                    26
                                ]
                            ],
                            "len": 35
                        }
                    }
                }
            }
        }
    },
    "_metrics": {
        "bytes.ingested.event": 5920,
        "bytes.stored.event": 6508
    },
    "breadcrumbs": {
        "values": [
            {
                "timestamp": 1721263108.641414,
                "type": "log",
                "category": "relay::setup",
                "level": "info",
                "message": "launching relay from config folder /work/.relay"
            },
            {
                "timestamp": 1721263108.641446,
                "type": "log",
                "category": "relay::setup",
                "level": "info",
                "message": "  relay mode: managed"
            },
            {
                "timestamp": 1721263108.641452,
                "type": "log",
                "category": "relay::setup",
                "level": "info",
                "message": "  relay id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            },
            {
                "timestamp": 1721263108.641459,
                "type": "log",
                "category": "relay::setup",
                "level": "info",
                "message": "  public key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            },
            {
                "timestamp": 1721263108.641464,
                "type": "log",
                "category": "relay::setup",
                "level": "info",
                "message": "  log level: info"
            },
            {
                "timestamp": 1721263108.641548,
                "type": "log",
                "category": "relay_server",
                "level": "info",
                "message": "relay server starting"
            },
            {
                "timestamp": 1721263108.652418,
                "type": "log",
                "category": "relay_server::services::outcome",
                "level": "info",
                "message": "Configured to emit outcomes as client reports"
            },
            {
                "timestamp": 1721263108.652551,
                "type": "log",
                "category": "relay_server::services::processor",
                "level": "info",
                "message": "starting 2 envelope processing workers"
            },
            {
                "timestamp": 1721263108.65481,
                "type": "log",
                "category": "relay_server::services::server",
                "level": "info",
                "message": "spawning http server"
            },
            {
                "timestamp": 1721263108.65483,
                "type": "log",
                "category": "relay_server::services::server",
                "level": "info",
                "message": "  listening on http://[ip]:3000/"
            },
            {
                "timestamp": 1721883878.778879,
                "type": "log",
                "category": "relay_server::services::server",
                "level": "info",
                "message": "Shutting down HTTP server"
            }
        ]
    },
    "contexts": {
        "Rust Tracing Fields": {
            "cached_summary": "EnvelopeSummary { event_category: None, attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 134, profile_chunk_quantity: 0 }",
            "recomputed_summary": "EnvelopeSummary { event_category: None, attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 134, profile_chunk_quantity: 0 }",
            "type": "unknown"
        },
        "Rust Tracing Location": {
            "file": "relay-server/src/utils/managed_envelope.rs",
            "line": 389,
            "module_path": "relay_server::utils::managed_envelope",
            "type": "unknown"
        },
        "device": {
            "arch": "x86_64",
            "type": "device"
        },
        "os": {
            "name": "Linux",
            "version": "5.15.153.1-1.cm2",
            "kernel_version": "#1 SMP Wed Apr 3 16:46:58 UTC 2024",
            "type": "os"
        },
        "rust": {
            "name": "rustc",
            "version": "1.79.0",
            "channel": "stable",
            "type": "runtime"
        }
    },
    "culprit": "",
    "debug_meta": {
        "images": [
            {
                "code_id": "2ba6b2a5a12925cedf6cf4b56741f95dfc76f1c9",
                "code_file": "/usr/bin/relay",
                "debug_id": "a5b2a62b-29a1-ce25-df6c-f4b56741f95d",
                "image_addr": "0x5e2b8c20b000",
                "image_size": 53344752,
                "type": "symbolic"
            },
            {
                "code_id": "6cbb2498e994ed637cb067dc8bb3f1793fd28453",
                "code_file": "linux-vdso.so.1",
                "debug_id": "9824bb6c-94e9-63ed-7cb0-67dc8bb3f179",
                "image_addr": "0x7ffe4bfbb000",
                "image_size": 3420,
                "type": "symbolic"
            },
            {
                "code_id": "0c47cec75226c7736517d5acb61e373d541a5023",
                "code_file": "/lib/x86_64-linux-gnu/libstdc++.so.6",
                "debug_id": "c7ce470c-2652-73c7-6517-d5acb61e373d",
                "image_addr": "0x7a22d0a4d000",
                "image_size": 2201728,
                "type": "symbolic"
            },
            {
                "code_id": "c3591c3e3ee3a4b11ac99e05a8d36479c3db86e9",
                "code_file": "/lib/x86_64-linux-gnu/libgcc_s.so.1",
                "debug_id": "3e1c59c3-e33e-b1a4-1ac9-9e05a8d36479",
                "image_addr": "0x7a22d0a2d000",
                "image_size": 127688,
                "type": "symbolic"
            },
            {
                "code_id": "26820458adaf5d95718fb502d170fe374ae3ee70",
                "code_file": "/lib/x86_64-linux-gnu/libpthread.so.0",
                "debug_id": "58048226-afad-955d-718f-b502d170fe37",
                "image_addr": "0x7a22d0a28000",
                "image_size": 16400,
                "type": "symbolic"
            },
            {
                "code_id": "baac66d4ed827c5168c0c72b9ee3b56ab5750a8a",
                "code_file": "/lib/x86_64-linux-gnu/libm.so.6",
                "debug_id": "d466acba-82ed-517c-68c0-c72b9ee3b56a",
                "image_addr": "0x7a22d0949000",
                "image_size": 909560,
                "type": "symbolic"
            },
            {
                "code_id": "53eaa845e9ca621f159b0622daae7387cdea1e97",
                "code_file": "/lib/x86_64-linux-gnu/libdl.so.2",
                "debug_id": "45a8ea53-cae9-1f62-159b-0622daae7387",
                "image_addr": "0x7a22d0944000",
                "image_size": 16400,
                "type": "symbolic"
            },
            {
                "code_id": "58254ca972028402bc40624f81388d85ec95f70d",
                "code_file": "/lib/x86_64-linux-gnu/libc.so.6",
                "debug_id": "a94c2558-0272-0284-bc40-624f81388d85",
                "image_addr": "0x7a22d0761000",
                "image_size": 1970000,
                "type": "symbolic"
            },
            {
                "code_id": "0e221d13cd73dac3d7b782778a5f325252903c3e",
                "code_file": "/lib64/ld-linux-x86-64.so.2",
                "debug_id": "131d220e-73cd-c3da-d7b7-82778a5f3252",
                "image_addr": "0x7a22d0c6b000",
                "image_size": 209624,
                "type": "symbolic"
            }
        ]
    },
    "environment": "production",
    "fingerprint": [
        "{{ default }}"
    ],
    "grouping_config": {
        "enhancements": "KLUv_SAYwQAAkwKRs25ld3N0eWxlOjIwMjMtMDEtMTGQ",
        "id": "newstyle:2023-01-11"
    },
    "hashes": [
        "4fbaf2c55b24194cb3fc223d07056952"
    ],
    "ingest_path": [
        {
            "version": "24.7.1",
            "public_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        }
    ],
    "key_id": "43879",
    "level": "error",
    "logentry": {
        "formatted": "dropped envelope: internal error"
    },
    "logger": "relay_server::utils::managed_envelope",
    "metadata": {
        "title": "dropped envelope: internal error"
    },
    "nodestore_insert": 1721883894.466963,
    "received": 1721883888.837159,
    "sdk": {
        "name": "sentry.rust",
        "version": "0.32.2",
        "integrations": [
            "attach-stacktrace",
            "debug-images",
            "contexts",
            "panic",
            "process-stacktrace"
        ],
        "packages": [
            {
                "name": "cargo:sentry",
                "version": "0.32.2"
            }
        ]
    },
    "timestamp": 1721883888.780667,
    "title": "dropped envelope: internal error",
    "type": "default",
    "version": "7",
    "location": null
}
@Dav1dde
Copy link
Member

Dav1dde commented Jul 25, 2024

This error is just the effect that something else went wrong and Relay couldn't process the envelope. For example a Redis outage would also generate these errors.

@klemmchr
Copy link
Author

We're not using Redis. This is our configuration:

relay:
  mode: managed
  upstream: https://xxxxxxxxxxxxxxx.ingest.de.sentry.io
  host: 0.0.0.0
  port: 3000
logging:
  level: info
limits:
  max_thread_count: 2
sentry:
  enabled: true
  dsn: https://[email protected]/xxxxxxxxxxxxx

@Dav1dde
Copy link
Member

Dav1dde commented Jul 25, 2024

I see, I thought this was a self hosted question. Best to check if you had any connectivity issues and check the Relay logs (if you have them enabled).

@klemmchr
Copy link
Author

Here the logs for the last 24 h. This error is being reported after the container received a SIGTERM and tries to gracefully shut down. For some reason it stops the outcome aggregator before it is empty and therefore drops envelopes.

2024-07-24T12:29:59.877201Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:29:59.949740Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:50:00.028640Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:50:00.125328Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:15:48.355481Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:15:48.434393Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:20:00.284677Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:20:00.355887Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:45:48.587119Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:45:48.665984Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:50:00.515736Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:50:00.584644Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:30:00.833228Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:30:00.903457Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:40:00.904281Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:40:00.981801Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:45:49.033757Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:45:49.112969Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:05:49.194788Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:05:49.266916Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:10:01.139692Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:10:01.220787Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:25:47.961866Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:25:48.032697Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:55:48.189577Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:55:48.270289Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:00:00.586073Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:00:00.669238Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:35:48.033964Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:35:48.113389Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:15:48.825797Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:15:48.889805Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:39:59.951436Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:40:00.027127Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:20:00.759097Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:20:00.831626Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:05:48.271667Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:05:48.354663Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:10:00.210559Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:10:00.282993Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:30:00.356910Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:30:00.429196Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:40:00.515360Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:25:48.891387Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:25:48.963224Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:35:48.963792Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:35:49.032396Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:50:00.983022Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:50:01.060622Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:15:49.267971Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:15:49.348711Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:15:47.894945Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:15:47.960898Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:19:59.807765Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:19:59.875920Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T12:45:48.114743Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T12:45:48.188478Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:00:00.126190Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:00:00.209185Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:25:48.435573Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:25:48.507013Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:35:48.508243Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:35:48.585459Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:40:00.430214Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T13:55:48.738258Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T13:55:48.666906Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:05:48.738638Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:05:48.824871Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:10:00.670523Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:10:00.757867Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T14:55:49.113609Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T14:55:49.194120Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:00:01.061838Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:00:01.137632Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:20:01.222620Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:20:01.309768Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:40:01.386571Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:40:01.471531Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:05:49.652952Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:05:49.743094Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:10:01.631473Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:10:01.712797Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:40:01.890022Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:40:01.958960Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:00:02.041558Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:00:02.114937Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:15:50.229821Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:25:54.759182Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:25:54.820042Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:30:02.268835Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:30:02.334416Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:45:54.892960Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:45:54.970481Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:55:54.972314Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:55:55.048312Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:05:55.049820Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:05:55.131606Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:10:02.599242Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:10:02.674959Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:40:02.820504Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:40:02.912653Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:45:55.371311Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:45:55.448943Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:05:55.990979Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:05:56.063473Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:10:03.548943Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:10:03.622779Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:35:56.219688Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:35:56.300208Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:00:03.967596Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:00:04.052118Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:05:56.456358Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:05:56.527452Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:25:56.602239Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:25:56.684131Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:30:04.222030Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:30:04.299622Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:20:04.615428Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:20:04.693558Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:45:57.234779Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:45:57.311035Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:10:05.018855Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:10:05.098053Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:25:57.555689Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:25:57.625544Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:35:57.626759Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:35:57.694559Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:40:05.237907Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:40:05.310279Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:00:05.471933Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:10:05.473570Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:10:05.548283Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:35:58.078718Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:35:58.151609Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:55:58.230229Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:55:58.302464Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:25:58.477000Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:25:58.552776Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:30:06.075337Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:30:06.149820Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:55:59.278960Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:55:59.347793Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:00:06.296474Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:00:06.363568Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:25:59.490992Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:25:59.564096Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:50:06.670037Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:50:06.740246Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:15:59.846617Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:15:59.916166Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:35:59.982005Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:36:00.053381Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:00:07.182112Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:00:07.261488Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:26:00.351390Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:26:00.424905Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:06:00.643657Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:06:00.710245Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:45:01.081329Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:45:01.144272Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:46:00.931093Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:46:00.999241Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:15:01.302236Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:15:01.382467Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:16:01.153611Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:16:01.231292Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:45:01.536245Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:45:01.604840Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:46:01.383048Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:46:01.454945Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:55:01.605727Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:55:01.684447Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:15:01.750065Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:15:01.826535Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:16:01.607637Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:16:01.674016Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:45:01.978637Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:45:02.053118Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:35:02.350446Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:35:02.426985Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:36:02.217894Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:36:02.285373Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:35:02.831963Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:35:02.910304Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:55:02.996519Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:55:03.073719Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:56:02.866721Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:56:02.955164Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:25:03.248285Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:25:03.321424Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:45:03.467687Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:45:03.393654Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:46:03.259406Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:46:03.334110Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:25:05.636136Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:25:05.714340Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:35:49.423096Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:35:49.499372Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:45:49.500617Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:45:49.579446Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:55:49.580355Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:55:49.651543Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:45:49.984804Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:45:50.073147Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:10:02.116527Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:10:02.188246Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:35:55.289800Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:35:55.369284Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:15:55.610771Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:15:55.687419Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:45:55.840170Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:45:55.917702Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:50:03.465963Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:50:03.383551Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:15:56.065134Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:15:56.146252Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:25:56.147103Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:25:56.219140Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:45:56.300977Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:45:56.374475Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:20:04.135205Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:20:04.220931Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:55:56.840317Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:55:56.921894Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:25:57.080373Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:25:57.155306Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:50:04.843395Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:50:04.942554Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:55:57.312180Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:55:57.389921Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:05:57.390523Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:05:57.477286Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:45:57.771531Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:45:57.696019Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:50:05.310990Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:50:05.376307Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:40:05.691946Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:40:05.769551Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:35:58.554276Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:35:59.208217Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:05:59.348959Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:05:59.417347Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:10:06.365137Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:10:06.441353Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:30:06.512651Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:30:06.594348Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:35:59.564572Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:35:59.636718Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:05:59.769991Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:05:59.846000Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:20:06.896885Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:20:06.968344Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:25:59.916624Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:25:59.980396Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:30:06.968706Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:30:07.035178Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:50:07.108355Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:50:07.180882Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:30:07.472096Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:30:07.399301Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:56:00.570552Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:56:00.642730Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:25:00.949096Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:25:01.013821Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:26:00.785103Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:26:00.858136Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:55:01.146030Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:55:01.220041Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:25:01.828010Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:25:01.909338Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:55:02.054519Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:55:02.131760Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:56:01.907568Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:56:01.980128Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:15:02.205815Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:15:02.272641Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:16:02.067405Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:16:02.142846Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:05:02.594800Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:05:02.667514Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:06:02.436731Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:06:02.521521Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:16:02.522962Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:16:02.632082Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:26:02.633220Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:26:02.717225Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:36:02.717968Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:36:02.793415Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:45:02.911232Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:45:02.995899Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:46:02.795282Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:46:02.865264Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:15:03.166233Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:15:03.247555Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:16:03.024649Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:16:03.095631Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:55:03.468242Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:55:03.555811Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:05:03.556249Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:05:03.632239Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:06:03.431846Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:06:03.539237Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:55:05.876786Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:55:05.955593Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:56:03.848414Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:56:03.922382Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:25:49.350195Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:25:49.421289Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:30:01.310934Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:30:01.385293Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T15:50:01.472501Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T15:50:01.549107Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:15:49.744217Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:15:49.821234Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:20:01.714348Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:20:01.802004Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:25:49.821608Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:25:49.904256Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:30:01.803516Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:30:01.889282Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:50:01.960282Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:50:02.040319Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:55:50.073611Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:55:50.152604Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:05:50.153521Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:05:50.229122Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:15:54.757627Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:35:54.821240Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:35:54.892165Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:40:02.335001Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:40:02.411588Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:00:02.491781Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:00:02.598029Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:25:55.196481Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:25:55.288293Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:30:02.751630Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:30:02.818746Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:50:02.914574Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:50:02.995429Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:55:55.449960Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:55:55.528625Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:00:02.996118Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:00:03.065679Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:20:03.148330Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:20:03.226356Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:25:55.689280Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:25:55.765412Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:30:03.227639Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:30:03.305531Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:55:55.918793Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:55:55.989799Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:20:03.624081Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:20:03.699483Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:50:03.891312Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:50:03.966282Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:55:56.376197Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:55:56.455215Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:15:56.528901Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:15:56.600324Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:35:56.755962Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:35:56.685012Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:40:04.301388Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:40:04.380450Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:05:56.922718Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:05:56.999629Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:10:04.541399Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:10:04.614321Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:35:57.156592Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:35:57.233447Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:40:04.766897Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:40:04.842436Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:30:05.169133Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:30:05.237450Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:55:57.773007Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:55:57.838743Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:20:05.549704Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:20:05.624487Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:45:58.152733Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:45:58.228572Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:50:05.770765Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:50:05.844541Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:10:05.918791Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:10:06.000028Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:15:58.395900Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:15:58.475613Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:20:06.001549Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:20:06.073939Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:40:06.151306Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:40:06.218856Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:45:59.208946Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:45:59.277759Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:50:06.220445Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:50:06.295434Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:15:59.419270Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:15:59.489839Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:55:59.703060Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:55:59.768357Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:00:06.741486Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:00:06.816139Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:10:06.817724Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:10:06.895653Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:46:00.054461Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:46:00.129774Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:56:00.131310Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:56:00.204139Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:10:07.263573Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:10:07.334366Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:16:00.281241Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:16:00.349827Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:36:00.425846Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:36:00.492738Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:40:07.472806Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:40:07.541810Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:46:00.493581Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:46:00.570140Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:04:38.778790Z INFO relay_system::controller: SIGTERM received, stopping in 10s 2024-07-25T05:04:38.778870Z INFO relay_server::services::server: Shutting down HTTP server 2024-07-25T05:04:38.778887Z INFO relay_server::services::outcome_aggregator: outcome aggregator stopped 2024-07-25T05:04:48.780180Z ERROR relay_server::services::spooler: dropped 2 envelopes 2024-07-25T05:04:48.780581Z ERROR relay_server::utils::managed_envelope: dropped envelope: internal error tags.project_key="xxxxxxxxxxxxx" tags.has_attachments=false tags.has_sessions=false tags.has_profiles=false tags.has_transactions=false tags.has_span_metrics=false tags.has_replays=false tags.has_checkins=false tags.event_category=Some(Transaction) cached_summary=EnvelopeSummary { event_category: Some(Transaction), attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 2048, profile_chunk_quantity: 0 } recomputed_summary=EnvelopeSummary { event_category: Some(Transaction), attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 2048, profile_chunk_quantity: 0 } 2024-07-25T05:04:48.780650Z ERROR relay_server::utils::managed_envelope: dropped envelope: internal error tags.project_key="xxxxxxxxxxxxx" tags.has_attachments=false tags.has_sessions=false tags.has_profiles=false tags.has_transactions=false tags.has_span_metrics=false tags.has_replays=false tags.has_checkins=false tags.event_category=None cached_summary=EnvelopeSummary { event_category: None, attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 134, profile_chunk_quantity: 0 } recomputed_summary=EnvelopeSummary { event_category: None, attachment_quantity: 0, session_quantity: 0, profile_quantity: 0, replay_quantity: 0, checkin_quantity: 0, secondary_transaction_quantity: 0, secondary_span_quantity: 0, span_quantity: 0, has_plain_attachments: false, payload_size: 134, profile_chunk_quantity: 0 } 2024-07-25T05:04:48.782667Z INFO relay_server: relay shutdown complete 2024-07-25T05:05:00.717841Z INFO relay::setup: launching relay from config folder /work/.relay 2024-07-25T05:05:00.717893Z INFO relay::setup: relay mode: managed 2024-07-25T05:05:00.717898Z INFO relay::setup: relay id: xxxxxxxxxxxxxxxxxxx 2024-07-25T05:05:00.717902Z INFO relay::setup: public key: xxxxxxxxxxxxxxxxxxxxxxxx 2024-07-25T05:05:00.717908Z INFO relay::setup: log level: info 2024-07-25T05:05:00.717929Z INFO relay_server: relay server starting 2024-07-25T05:05:00.729539Z INFO relay_server::services::outcome: Configured to emit outcomes as client reports 2024-07-25T05:05:00.729572Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:05:00.729722Z INFO relay_server::services::processor: starting 2 envelope processing workers 2024-07-25T05:05:00.730403Z INFO relay_server::services::server: spawning http server 2024-07-25T05:05:00.730427Z INFO relay_server::services::server: listening on http://0.0.0.0:3000/ 2024-07-25T05:05:00.732398Z INFO relay_server::services::outcome: OutcomeProducer started. 2024-07-25T05:05:00.732445Z INFO relay_server::services::outcome_aggregator: outcome aggregator started 2024-07-25T05:05:00.732659Z ERROR relay_server::services::health_check: Health check probe 'auth' failed 2024-07-25T05:05:00.732776Z INFO relay_server::services::relays: key cache started 2024-07-25T05:05:00.732897Z INFO relay_server::services::metrics::router: metrics router started 2024-07-25T05:05:00.732952Z INFO relay_server::services::project_cache: project cache started 2024-07-25T05:05:00.730050Z INFO relay_server::services::global_config: global config service starting 2024-07-25T05:05:00.733461Z INFO relay_server::services::global_config: requesting global config from upstream 2024-07-25T05:05:00.733481Z INFO relay_server::services::project_cache: waiting for global config 2024-07-25T05:05:00.733910Z INFO relay_server::services::project_local: project local cache started 2024-07-25T05:05:00.734002Z INFO relay_server::services::project_upstream: project upstream cache started 2024-07-25T05:05:00.874769Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:05:00.889351Z INFO relay_server::services::global_config: received global config from upstream 2024-07-25T05:35:01.014339Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:35:01.080955Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:36:00.859286Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:36:00.929897Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:56:00.999435Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:56:01.074185Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:05:01.220877Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:05:01.301291Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:06:01.074431Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:06:01.152673Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:26:01.233320Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:26:01.308331Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:35:01.461298Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:35:01.535174Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:36:01.309861Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:36:01.381944Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:56:01.455537Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:56:01.527553Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:05:01.685992Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:05:01.748643Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:06:01.528496Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:06:01.606529Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:35:01.910956Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:35:01.978024Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:36:01.753024Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:36:01.834785Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:05:02.132877Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:05:02.204635Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:06:01.981324Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:06:02.065405Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:45:02.428035Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:45:02.508756Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:46:02.287257Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:46:02.358669Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:55:02.509490Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:55:02.593762Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:05:03.074250Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:05:03.165239Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:06:02.955747Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:06:03.023896Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:26:03.097111Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:26:03.172065Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:56:03.334779Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:56:03.430439Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:26:03.625473Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:26:03.703131Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:35:05.715332Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:35:05.795154Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:36:03.703824Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:36:03.780729Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:00:01.550513Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:00:01.629558Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T16:35:49.905293Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T16:35:49.984000Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:20:02.188963Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T17:20:02.268309Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:50:02.490853Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T17:50:02.413140Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:15:55.133191Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:15:55.195802Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T18:20:02.676165Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T18:20:02.751014Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:05:55.530067Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:05:55.610020Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:10:03.066907Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:10:03.146944Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:35:55.766961Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:35:55.838302Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T19:40:03.305904Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T19:40:03.382273Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:00:03.467618Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:00:03.548500Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:30:03.699963Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:30:03.805897Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T20:40:03.807145Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T20:40:03.890292Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:10:04.053529Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:10:04.134412Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:45:56.757262Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:45:56.838541Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T21:50:04.380835Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T21:50:04.455784Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:00:04.457220Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:00:04.539552Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:15:57.000587Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:15:57.079608Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T22:30:04.694907Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T22:30:04.765947Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:00:04.944439Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:00:05.018123Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:15:57.478757Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:15:57.555157Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-24T23:20:05.099518Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-24T23:20:05.167685Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:00:05.377065Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:05:57.840189Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:05:57.914087Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:15:57.914567Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:15:57.998629Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:25:57.999542Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:25:58.077353Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T00:30:05.625614Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T00:30:05.691191Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:00:05.845056Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:00:05.918214Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T01:05:58.304270Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T01:05:58.394493Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:20:06.511032Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:20:06.442201Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:40:06.594834Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:40:06.669427Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T02:45:59.637824Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T02:45:59.701717Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T03:40:07.036467Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T03:40:07.106813Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:06:00.205008Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:06:00.280112Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:20:07.335075Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:20:07.397747Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T04:50:07.543688Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T04:50:07.615855Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:00:07.616824Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:00:07.685327Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:15:00.875530Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:15:00.947774Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T05:16:00.710450Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T05:16:00.784006Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T06:25:01.384012Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T06:25:01.460673Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:26:01.751829Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T07:26:01.674435Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:46:01.835450Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T07:46:01.906342Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:25:02.273629Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:25:02.349248Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:26:02.144238Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:26:02.216491Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T08:56:02.359894Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T08:56:02.436089Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:15:02.668838Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:15:02.754028Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T09:25:02.755120Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T09:25:02.830519Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:35:03.322855Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:35:03.392122Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T10:36:03.173177Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T10:36:03.257308Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:15:03.633429Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:15:05.634907Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:16:03.539864Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:16:03.624481Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:45:05.797070Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:45:05.875149Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T11:46:03.781515Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T11:46:03.847927Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T12:05:05.956768Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T12:05:06.033234Z INFO relay_server::services::upstream: relay successfully registered with upstream 2024-07-25T12:06:03.923942Z INFO relay_server::services::upstream: registering with upstream descriptor=https://xxxxxxxxxxxxx.ingest.de.sentry.io/ 2024-07-25T12:06:03.997459Z INFO relay_server::services::upstream: relay successfully registered with upstream

@Dav1dde
Copy link
Member

Dav1dde commented Jul 25, 2024 

2024-07-25T05:04:48.780180Z ERROR relay_server::services::spooler: dropped 2 envelopes

There were still 2 envelopes buffered in memory. This is possible if for these envelopes Relay did not or has not received a project config yet. Those envelopes cannot be processed on shutdown without the project config. It's also possibly just a bug where the spooler does not flush correctly on shutdown. Thanks for the logs, we'll have to investigate, but we also have ongoing work with the spool, with the new design this should be resolved (both cases).

@Dav1dde Dav1dde changed the title dropped envelope: internal error Spool drops envelopes on shutdown Jul 25, 2024
@getsantry
Copy link

getsantry bot commented Aug 20, 2024

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

@getsantry getsantry bot added the Stale label Aug 20, 2024
@klemmchr
Copy link
Author

Bump

@getsantry getsantry bot removed the Stale label Aug 21, 2024
@jjbayer
Copy link
Member

jjbayer commented Sep 10, 2024

@klemmchr thanks for the report. We're currently working on a complete re-implementation of the spool, including an automatic flush after shutdown: #3980

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jjbayer jjbayer

@iambriccardo iambriccardo

Labels
None yet
Projects
GitHub Issues with 👀 3
Status: No status
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Dav1dde @jjbayer @olksdr @klemmchr @iambriccardo