You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I use the fluent/fluentd:v1.1.0 image the in_tail plugin fallbacks to polling file every 5 seconds instead of using inotify. This causes loss in timestamp precision. fluent/fluentd:v1.1.0-debian image handles watching file correctly and does not fallback to polling.
Steps to reproduce
put Dockerfile, fluent.conf and write_logs.sh in one directory
Dockerfile
FROM fluent/fluentd:v1.1.0
COPY ./fluent.conf /fluentd/etc/fluent.conf
COPY ./write_logs.sh /usr/local/bin/write-logs
RUN chmod 755 /usr/local/bin/write-logs
RUN find . -name in_tail.rb | xargs sed -i '/line.chomp!/a \ \ \ \ \ \ \ \ puts Time.now.to_datetime.iso8601(3)'
fluent.conf
<source>
@type tail
path /tmp/test.log
pos_file /tmp/test.log.pos
tag test
path_key tailed_test
keep_time_key true
format none
enable_watch_timer false
</source>
write_logs.sh
#!/bin/sh
counter=0
while true; do
sleep 0.01
echo $counter >> /tmp/test.log
counter=`expr $counter + 1`
done
run docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
run docker exec -it inotify_problem write-logs
As you can see, logs are being polled with 5 sec frequency (convert_line_to_event function is executed every ~5 sec).
Now change fluent/fluentd:v1.1.0 to fluent/fluentd:v1.1.0-debian in the Dockerfile and run:
This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 7 days
Description
When I use the
fluent/fluentd:v1.1.0
image thein_tail
plugin fallbacks to polling file every 5 seconds instead of usinginotify
. This causes loss in timestamp precision.fluent/fluentd:v1.1.0-debian
image handles watching file correctly and does not fallback to polling.Steps to reproduce
Dockerfile
,fluent.conf
andwrite_logs.sh
in one directoryDockerfile
fluent.conf
write_logs.sh
docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
docker exec -it inotify_problem write-logs
As you can see, logs are being polled with 5 sec frequency (
convert_line_to_event
function is executed every ~5 sec).Now change
fluent/fluentd:v1.1.0
tofluent/fluentd:v1.1.0-debian
in the Dockerfile and run:docker stop inotify_problem; docker rm inotify_problem; docker rmi inotify_problem
docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
docker exec -it inotify_problem write-logs
Observe that
convert_line_to_event
function is executed every ~0.01 sec.The text was updated successfully, but these errors were encountered: