GPL-698 Bug: Lighthouse positives file generation failed on 21st - root cause

[rl15]

Describe the bug
GPL-698 | Bug: Lighthouse positives file generation failed on 21st - root cause

RT Ticket Number
RT 702848

To Reproduce
See file 201021_0100_positives_with_locations.xlsx @ http://lighthouse-ui.psd.sanger.ac.uk/

Expected behaviour
See file 201021_1024_positives_with_locations.xlsx @ http://lighthouse-ui.psd.sanger.ac.uk/

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

[rl15]

[rl15]

[rl15]

Repair Result to 201021_0100_positives_with_locations(1)0.xml

Errors were detected in file '/Users/rl15/Downloads/201021_0100_positives_with_locations(1).xlsx'

Excel completed file level validation and repair. Some parts of this workbook may have been repaired or discarded.

[rl15]

Corrupted file 201021_0100_positives_with_locations.xlsx has 370895 rows
New file generated in UI 201021_1024_positives_with_locations.xlsx has 385184 rows

Sample data at row 370895 differ
Files out of syn at row 364461, expecting row with root id , AAC61934316 get AAC62403774

[Chris-Friend]

Suspecting that this is due to lighthouse's create report job being triggered to run twice, as per logs.

Two jobs writing to the same file at the same time may be causing corruption of the file.

[Chris-Friend]

Fairly confident that the two triggered jobs are caused by Lighthouse initialising two schedulers. This may be due to cyclic dependencies in the code, but due to my inexperience with Lighthouse/Python I'm not 100% on this

[Chris-Friend]

On the other hand, this could be an issue with the Dockerfile being configured to spin up two instances of the app (and therefore the scheduler) as outlined in the linked documentation. This seems like a similar issue to this one