Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch up to spectator-cpp #3

Merged
merged 7 commits into from
Aug 8, 2024
Merged

Conversation

fvallenilla
Copy link

@fvallenilla fvallenilla commented Aug 5, 2024

Description

This change updates the fork to catch up with Netflix/spectator-cpp.

The main change that was needed was the commit that replaces invalid characters in ids, but I pulled in another couple of changes that happened before that. That fix is needed to address an issue where some IPC metrics are being silently dropped due to having invalid characters in tags.

NOTE: There are a couple departures from the upstream repo:

  1. I kept Bazel as the build system to facilitate import into proxyd
  2. I checked spectator/valid_chars.inc into git, rather than having it be auto-generated. This was also to facilitate import into proxyd.

How was this tested?

  1. Verified that spectator-cpp continues to build with Bazel
  2. Ran the unit tests
  3. Built proxyd
  4. Ran custom build of proxyd and verified that metrics with invalid tags/values were sanitized and made their way to Atlas.

The repo's GitHub actions don't run against forks, so I built a container to build and test the library based on the CI script:

FROM ubuntu:22.04

# Set environment variables
ENV CC=gcc-11
ENV CXX=g++-11
ENV LANG=en_US.UTF-8

# Install system dependencies
RUN apt-get update && \
    apt-get install -y \
    curl \
    gnupg \
    software-properties-common \
    binutils-dev \
    g++-11 \
    git \
    wget \
    && rm -rf /var/lib/apt/lists/*

# Install Bazel
RUN wget -O /usr/local/bin/bazel https://github.com/bazelbuild/bazelisk/releases/latest/download/bazelisk-linux-$([ $(uname -m) = "aarch64" ] && echo "arm64" || echo "amd64")
RUN chmod +x /usr/local/bin/bazel

# Create a directory for the project
WORKDIR /app
COPY . .

# Set the output user root for Bazel cache
ENV BAZEL_OUTPUT_USER_ROOT=/root/.cache/bazel

# Build the project
RUN bazel --batch build --jobs=2 --config asan spectator_test spectator

# Run the tests
CMD ["bash", "-c", "GTEST_COLOR=1 ./bazel-bin/spectator_test"]

cancecen and others added 6 commits August 4, 2024 17:49
This change is a companion to the fix for monotonic counter implementations in
spectatord.

Netflix-Skunkworks/spectatord#90

The original monotonic counter (`C`) was always intended to be used for the
case where a monotonic data source needs to be transformed into base units
for recording data. For example, transforming nanoseconds into seconds. This
requires division, which results in floats.

There is a valid use case for handling uints in monotonic counters, if the
data source is already in a base unit, such as bytes. Thus, a new meter type
`U` is added to spectatord which supports this use case.
It was using the existing `send` method, which formatted the value as a float,
when it needs to be formatted as a uint.
It is easy enough to break the spectatord line protocol, if any of the control
characters (`:,=`) are inserted in unexpected places. Since metric names and
tags are often programmatically generated, we want to only construct id strings
which are valid.
This file is generated by tools/gen_valid_chars.
@cancecen
Copy link
Owner

cancecen commented Aug 7, 2024

The write up is excellent, thanks for all the details.

Copy link
Owner

@cancecen cancecen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to also check in the docker run script you wrote for the tests?

README.md Outdated Show resolved Hide resolved
#include <array>
#include <fstream>

void dump_array(std::ostream& os, const std::string& name, const std::array<bool, 256>& chars) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this file useful?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That file is used to generate spectator/valid_chars.inc, which is not checked in by the upstream.

The upstream library calls this tool at build time to generate valid_chars.inc. I ended up calling it locally to generate the file.

@cancecen cancecen merged commit 6890ee0 into cancecen:main Aug 8, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants