log failed mails to the db

[shawshankkumar]

comment the approach and we'll assign you the issue.

currently the failed emails are logged to the console which is not very scalable or convenient.
the plan is to log the failed emails to the database.

you have to come up with the structure.

[jatindotdev]

so, this is what i got

basically we could have a collection called logs

Where there could be two possibilities

push all the failed mail array in one go after the process is finished

Or

when we catch a failed fail, push it to the db right then

Schema could be

{
   "_id": ObjectId,
   "timestamp": string,
   "sender": string,
   "recipient": string,
   "subject": string,
   "error_message": string
}

[shawshankkumar]

I don't think adding a collection is a good idea. Any alternatives ? @jatindotdev

[jatindotdev]

I don't think adding a collection is a good idea. Any alternatives ? @jatindotdev

will get back to you after a bit of research

[sidhansukeshri]

In lights of failed emails that are logged to the console , we can catch all the required details that would be necessary to have fix the issue and we can create a table named "failed_emails" or a similar descriptive name.

Whenever an email fails to send or insert a record into the "failed_emails" table with relevant information such as the list name, user ID ,the email address, the reason for failure, and a timestamp.

Possible structure of failed_emails table

CREATE TABLE failed_emails (
    id SERIAL PRIMARY KEY,
    list_name VARCHAR(255),
    user_id INT,
    email VARCHAR(255),
    reason TEXT,
    timestamp TIMESTAMP
);

Also If not already using postgresql we can switch our database to postgresql which can parallelly implement queries quickly, it also support ip address as a data type which may help store the data of particular ip's and may help in future issues related with that particular ip address.

[HarshPatel5940]

I don't think adding a collection is a good idea. Any alternatives ? @jatindotdev

Would it be good, if we just update the target document? By just adding a failed_mails array and updating it?

[jatindotdev]

I don't think adding a collection is a good idea. Any alternatives ? @jatindotdev

Would it be good, if we just update the target document? By just adding a failed_mails array and updating it?

hmm, this sounds better only the target document is affected

[WiiWake3101]

My solution is based on the understanding of the problem using MySQL

Create a new table in the database:

This table could be named FailedEmails and could have the following fields:

• id: A unique identifier for each failed email record.
• email: The email address to which sending failed.
• subject: The subject of the email.
• body: The body of the email.
• error_message: The error message received when trying to send the email.
• timestamp: The time when the email sending failed.

Structure of table:

CREATE TABLE FailedEmails ( id SERIAL PRIMARY KEY, email VARCHAR(255), subject VARCHAR(255), body TEXT, error_message TEXT, timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP );

Whenever an email fails to send, instead of logging the error to the console, create a new record in the FailedEmails table with all the relevant information (timestamp and error_message are the important ones) and make sure to handle any errors that might occur when trying to write to the database.

I will figure out how to Error handle when an error pops

[WiiWake3101]

My solution is based on the understanding of the problem using MySQL

Create a new table in the database:

This table could be named FailedEmails and could have the following fields:

• id: A unique identifier for each failed email record.
• email: The email address to which sending failed.
• subject: The subject of the email.
• body: The body of the email.
• error_message: The error message received when trying to send the email.
• timestamp: The time when the email sending failed.

Structure of table:

CREATE TABLE FailedEmails ( id SERIAL PRIMARY KEY, email VARCHAR(255), subject VARCHAR(255), body TEXT, error_message TEXT, timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP );

Whenever an email fails to send, instead of logging the error to the console, create a new record in the FailedEmails table with all the relevant information (timestamp and error_message are the important ones) and make sure to handle any errors that might occur when trying to write to the database.

I will figure out how to Error handle when an error pops

We can also prefer MongoDB for the following reason, MongoDB offers faster query times and requires less disk space for data storage

[shawshankkumar]

I don't think adding a collection is a good idea. Any alternatives ? @jatindotdev

Would it be good, if we just update the target document? By just adding a failed_mails array and updating it?

I agree with this.

[shawshankkumar]

My solution is based on the understanding of the problem using MySQL

Create a new table in the database:

This table could be named FailedEmails and could have the following fields:

• id: A unique identifier for each failed email record.
• email: The email address to which sending failed.
• subject: The subject of the email.
• body: The body of the email.
• error_message: The error message received when trying to send the email.
• timestamp: The time when the email sending failed.

Structure of table:

CREATE TABLE FailedEmails ( id SERIAL PRIMARY KEY, email VARCHAR(255), subject VARCHAR(255), body TEXT, error_message TEXT, timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP );

Whenever an email fails to send, instead of logging the error to the console, create a new record in the FailedEmails table with all the relevant information (timestamp and error_message are the important ones) and make sure to handle any errors that might occur when trying to write to the database.

I will figure out how to Error handle when an error pops

Try reading the project code base and stack and then answer.

[jatindotdev]

I agree with this.

so can we move forward with this?

Should we update the log one at a time on fail or perform bulk writes? (ig bulk write would be better imo)

@shawshankkumar

[LordHarsh]

If we don't want to create a new collection and also don't want to affect the target document in the collection, then can create a new document in the same collection that will contain the details of all failed mails.

{
    name: %original_name% + "-failed-" + time,
    timestamp: DateTime,
    users: [{
        ...orignal_user,
        failedAt: DateTime,
        error: string
        }]
}

They way, if we want to resend emails, it will be easily possible just by re-running it and giving the name of new list.
@shawshankkumar

[WiiWake3101]

My solution is based on the understanding of the problem using MySQL

Create a new table in the database:

This table could be named FailedEmails and could have the following fields:

• id: A unique identifier for each failed email record.
• email: The email address to which sending failed.
• subject: The subject of the email.
• body: The body of the email.
• error_message: The error message received when trying to send the email.
• timestamp: The time when the email sending failed.

Structure of table:

CREATE TABLE FailedEmails ( id SERIAL PRIMARY KEY, email VARCHAR(255), subject VARCHAR(255), body TEXT, error_message TEXT, timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP );

Whenever an email fails to send, instead of logging the error to the console, create a new record in the FailedEmails table with all the relevant information (timestamp and error_message are the important ones) and make sure to handle any errors that might occur when trying to write to the database.

I will figure out how to Error handle when an error pops

We can also prefer MongoDB for the following reason, MongoDB offers faster query times and requires less disk space for data storage

Yes I have seen the repository we can go with the array way but I feel that there is a different approach and we can figure it out

[sm-sami]

If we don't want to create a new collection and also don't want to affect the target document in the collection, then can create a new document in the same collection that will contain the details of all failed mails.

{
    name: %original_name% + "-failed-" + time,
    timestamp: DateTime,
    users: [{
        ...orignal_user,
        failedAt: DateTime,
        error: string
        }]
}

They way, if we want to resend emails, it will be easily possible just by re-running it and giving the name of new list. @shawshankkumar

@LordHarsh if we are going to include one more document in the collection, how are we supposed to know it is the failed emails of a particular campaign (mailing list)? IMO updating the target document (cc: @HarshPatel5940) with a timestamp (just in case we are using the same mailing list again) makes the most sense

[LordHarsh]

If we don't want to create a new collection and also don't want to affect the target document in the collection, then can create a new document in the same collection that will contain the details of all failed mails.

{
    name: %original_name% + "-failed-" + time,
    timestamp: DateTime,
    users: [{
        ...orignal_user,
        failedAt: DateTime,
        error: string
        }]
}

They way, if we want to resend emails, it will be easily possible just by re-running it and giving the name of new list. @shawshankkumar

@LordHarsh if we are going to include one more document in the collection, how are we supposed to know it is the failed emails of a particular campaign (mailing list)? IMO updating the target document (cc: @HarshPatel5940) with a timestamp (just in case we are using the same mailing list again) makes the most sense

@sm-sami, I agree that it would be an issue if we randomly gave a new name to each failed document. However, we could name them like original_name + "-failed-" + new Date().toISOString() or something similar. This way, we can easily identify failed email documents and keep track of them. I think this method has some advantages-

• If required, can easily send emails to failed documents with less work
• No changes to orignal list/document, so same can be easily reused send mails at different occasions

What do you think?

[sm-sami]

If we don't want to create a new collection and also don't want to affect the target document in the collection, then can create a new document in the same collection that will contain the details of all failed mails.

{
    name: %original_name% + "-failed-" + time,
    timestamp: DateTime,
    users: [{
        ...orignal_user,
        failedAt: DateTime,
        error: string
        }]
}

They way, if we want to resend emails, it will be easily possible just by re-running it and giving the name of new list. @shawshankkumar

@LordHarsh if we are going to include one more document in the collection, how are we supposed to know it is the failed emails of a particular campaign (mailing list)? IMO updating the target document (cc: @HarshPatel5940) with a timestamp (just in case we are using the same mailing list again) makes the most sense

@sm-sami, I agree that it would be an issue if we randomly gave a new name to each failed document. However, we could name them like original_name + "-failed-" + new Date().toISOString() or something similar. This way, we can easily identify failed email documents and keep track of them. I think this method has some advantages-

• If required, can easily send emails to failed documents with less work
• No changes to orignal list/document, so same can be easily reused send mails at different occasions

What do you think?

@LordHarsh Yeah it does sound good, but the emails have failed for a reason, we can't just resend to all failed users again. We would have to clean before resending. Also how often do we resend failed emails (I have never resent if an email fails :p). But yes if we might resend failed mails, your idea is good.

cc: @shawshankkumar