Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[alpine images] Coolio::StatWatcher fallbacks to polling instead of using inotify #113

Open
adw1n opened this issue Feb 19, 2018 · 1 comment
Labels
stale Inactive and will be closed automatically

Comments

@adw1n
Copy link

adw1n commented Feb 19, 2018

Description

When I use the fluent/fluentd:v1.1.0 image the in_tail plugin fallbacks to polling file every 5 seconds instead of using inotify. This causes loss in timestamp precision. fluent/fluentd:v1.1.0-debian image handles watching file correctly and does not fallback to polling.

Steps to reproduce

  1. put Dockerfile, fluent.conf and write_logs.sh in one directory
    Dockerfile
FROM fluent/fluentd:v1.1.0

COPY ./fluent.conf /fluentd/etc/fluent.conf
COPY ./write_logs.sh /usr/local/bin/write-logs
RUN chmod 755 /usr/local/bin/write-logs

RUN find . -name in_tail.rb | xargs sed -i '/line.chomp!/a \ \ \ \ \ \ \ \ puts Time.now.to_datetime.iso8601(3)'

fluent.conf

<source>
  @type tail
  path /tmp/test.log
  pos_file /tmp/test.log.pos
  tag test
  path_key tailed_test
  keep_time_key true
  format none
  enable_watch_timer false
</source>

write_logs.sh

#!/bin/sh
counter=0
while true; do
	sleep 0.01
	echo $counter >> /tmp/test.log
	counter=`expr $counter + 1`
done
  1. run docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
  2. run docker exec -it inotify_problem write-logs

As you can see, logs are being polled with 5 sec frequency (convert_line_to_event function is executed every ~5 sec).

Now change fluent/fluentd:v1.1.0 to fluent/fluentd:v1.1.0-debian in the Dockerfile and run:

  1. docker stop inotify_problem; docker rm inotify_problem; docker rmi inotify_problem
  2. docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
  3. docker exec -it inotify_problem write-logs

Observe that convert_line_to_event function is executed every ~0.01 sec.

Copy link

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 7 days

@github-actions github-actions bot added the stale Inactive and will be closed automatically label Aug 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Inactive and will be closed automatically
Projects
None yet
Development

No branches or pull requests

1 participant