Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect mapping of files in audit.json file #359

Open
Vishal8523 opened this issue Oct 18, 2022 · 5 comments
Open

Incorrect mapping of files in audit.json file #359

Vishal8523 opened this issue Oct 18, 2022 · 5 comments

Comments

@Vishal8523
Copy link

Versions:
winston: ^2.4.5
winston-daily-rotate-file: ^3.10.0
Node JS: v12.22.0
OS: Debian 11 Bullseye

During the logs rotate mechanism, some of the files are not deleted from the logs directory.
Upon investigation, it is found that the -audit.json file has missing files that made it to not get deleted.

Winston Settings:

const logger = new winston.Logger({
    transports: [
        new winston.transports.Console({
            level: 'info',
            colorize: true,
            prettyPrint: true,
            timestamp: true
        }),
        new winston.transports.DailyRotateFile({
            filename: Logs-%DATE%.log,
            datePattern: 'YYYY-MM-DD',
            dirname: /logs/,
            level: 'info',
            json: false,
            prettyPrint: true,
            timestamp: true,
            zippedArchive: false,
            maxSize: 100m,
            maxFiles: 7d
        })
    ]
});

The audit.json file is as on 18th Oct 2022. Here, the 17th Oct 2022 file is missing in the audit.json file and this file (Logs-2022-10-17.log) was not deleted from the logs directory.

{
    "keep": {
        "days": true,
        "amount": 7
    },
    "auditLog": "/logs/.10f0fac856acd0450d842d9db8cf031638f51968-audit.json",
    "files": [
        {
            "date": 1665546274787,
            "name": "/logs/Logs-2022-10-12.log",
            "hash": "9e4b2afc2135014d339ded5cfb4d658a"
        },
        {
            "date": 1665632751118,
            "name": "/logs/Logs-2022-10-13.log",
            "hash": "6da82625ff3ac1b2696f5cc329c6c79b"
        },
        {
            "date": 1665705699082,
            "name": "/logs/Logs-2022-10-14.log",
            "hash": "03d79ac4b065b944f801f79b9193bab9"
        },
        {
            "date": 1665792068107,
            "name": "/logs/Logs-2022-10-15.log",
            "hash": "34e6ff4052e4ca803a86b49880bf9eb0"
        },
        {
            "date": 1665878484923,
            "name": "/logs/Logs-2022-10-16.log",
            "hash": "7bb75b7fe8e6a6ab80aa092ed44a4230"
        },
        {
            "date": 1666065076652,
            "name": "/logs/Logs-2022-10-18.log",
            "hash": "88f06d511f09711d71cabb7b98e52aef"
        }
    ]
}

Requesting a resolution on the issue.

@wbt
Copy link
Collaborator

wbt commented Oct 18, 2022

At first glance, it looks like the file may have been locked by some process (e.g. an automated backup) at the time the deletion was attempted.

@Vishal8523
Copy link
Author

Vishal8523 commented Oct 19, 2022

How to avoid this situation and have proper rotation/deletion of logs ?
Considering multiple node processes are writing to the same log file

Note: Also tried removing the file manually from the audit.json but the file gets deleted.

@wbt
Copy link
Collaborator

wbt commented Oct 19, 2022

Why do you have multiple Node processes writing to the same log file? You could consider having separate processes log into separate log directories and disabling automated lockers (e.g. backup processes) if you want to increase the probability of correct rotation/deletion of logs.

@Vishal8523
Copy link
Author

It is required by the application to log only 1 file from all the Node processes. So, no much control over it.
How does one disable automated lockers ?

@wbt
Copy link
Collaborator

wbt commented Oct 20, 2022

The automated locking is a file-system feature. What would you expect the result to be if one process tried to write to a file while another process tried to delete it at the same time?
That "requirement" as stated seems inconsistent with your use of file rotation, which is going to have multiple files. You may want to consider more deeply what the source/reasoning of that might be - maybe have different processes log to different directories and something else that can read from different directories/streams to present a merged view, or coordinate logging through a single process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants