You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Set up the plugin with the config below. These files are very infrequent, not all of the ones created remain. So the plugin leaves some in /tmp and deletes most as it should
Expected behavior
tmp files should be removed after upload or there should be an error stating why the files are still there
It looks like this is the code creating the files. There is no way to specify a custom "temp" directory either so / keeps getting filled up if it is small
tmp = Tempfile.new("s3-")
tmp.binmode
begin
@compressor.compress(chunk, tmp)
tmp.rewind
log.debug "out_s3: write chunk #{dump_unique_id_hex(chunk.unique_id)} with metadata #{chunk.metadata} to s3://#{@s3_bucket}/#{s3path}"
put_options = {
body: tmp,
content_type: @compressor.content_type,
storage_class: @storage_class,
}
put_options[:server_side_encryption] = @use_server_side_encryption if @use_server_side_encryption
put_options[:ssekms_key_id] = @ssekms_key_id if @ssekms_key_id
put_options[:sse_customer_algorithm] = @sse_customer_algorithm if @sse_customer_algorithm
put_options[:sse_customer_key] = @sse_customer_key if @sse_customer_key
put_options[:sse_customer_key_md5] = @sse_customer_key_md5 if @sse_customer_key_md5
put_options[:acl] = @acl if @acl
put_options[:grant_full_control] = @grant_full_control if @grant_full_control
put_options[:grant_read] = @grant_read if @grant_read
put_options[:grant_read_acp] = @grant_read_acp if @grant_read_acp
put_options[:grant_write_acp] = @grant_write_acp if @grant_write_acp
put_options[:tagging] = @tagging if @tagging
if @s3_metadata
put_options[:metadata] = {}
@s3_metadata.each do |k, v|
put_options[:metadata][k] = extract_placeholders(v, chunk).gsub(%r(%{[^}]+}), {"%{index}" => sprintf(@index_format, i - 1)})
end
end
@bucket.object(s3path).put(put_options)
@values_for_s3_object_chunk.delete(chunk.unique_id)
if @warn_for_delay
if Time.at(chunk.metadata.timekey) < Time.now - @warn_for_delay
log.warn "out_s3: delayed events were put to s3://#{@s3_bucket}/#{s3path}"
end
end
ensure
tmp.close(true) rescue nil
end
end
The text was updated successfully, but these errors were encountered:
This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days
Describe the bug
I have these files on /tmp filling up disk space and they are not removed by fluentd
To Reproduce
Set up the plugin with the config below. These files are very infrequent, not all of the ones created remain. So the plugin leaves some in /tmp and deletes most as it should
Expected behavior
tmp files should be removed after upload or there should be an error stating why the files are still there
Your Environment
Your Configuration
Your Error Log
Additional context
It looks like this is the code creating the files. There is no way to specify a custom "temp" directory either so / keeps getting filled up if it is small
The text was updated successfully, but these errors were encountered: