Program received signal SIGILL: Illegal instruction.

[raulleoncz]

Hello,

First of all, I am sorry for the inconvenience. I have been trying to do a simulation in an acoustic-elastic medium but after some time steps I got the following error:

Program received signal SIGILL: Illegal instruction.
Backtrace for this error:
Could not print backtrace: executable file is not an executable
Could not print backtrace: executable file is not an executable
#0 0x100b32f33
#1 0x100b321db
#2 0x19990b583
zsh: illegal hardware instruction ./bin/xspecfem2D

The first time I though the simulation got unstable but after checking in the solver.log I noticed that the CFL condition was right, the number of grid points per wavelength was fine and I was using an adequate source frequency. After checking the log file I found that after 4000 time steps the 'Max absolute value of scalar field in fluid (acoustic)' and the 'Color image maximum amplitude' quickly skyrocketed to very large values:

---

Time step number 5 t = 0.0032 s out of 10000

---

We have done 4.00040001E-02 % of the total
Max norm of vector field in solid (elastic) = 0.00000000
Max absolute value of scalar field in fluid (acoustic) = 1.53421927E-02
Color image maximum amplitude = 4.0870062075555325E-003

---

Time step number 2000 t = 1.5992 s out of 10000

---

We have done 19.9919987 % of the total
Max norm of vector field in solid (elastic) = 1.48898596E-03
Max absolute value of scalar field in fluid (acoustic) = 236.648941
Color image maximum amplitude = 1.4626149786636233E-003

---

Time step number 4000 t = 3.1992 s out of 10000

---

We have done 39.9939995 % of the total
Max norm of vector field in solid (elastic) = 1718.72803
Max absolute value of scalar field in fluid (acoustic) = 19952228.0
Color image maximum amplitude = 2.1745527217967010E+022

At the end the error message says:
code became unstable and blew up (image_color_data)
Error detected, aborting MPI... proc 0

Here are some screenshots:

Has anyone had a similar problem?

Just in case I will add the log files. Thank you for your help!

solver.log

[homnath]

I would further decrease the time step. Sometimes, a few elements may cause the instabilities even if the CFL condition is satisfied.

…

--------------------------------------- Hom Nath Gharti, PhD Assistant Professor | Digital Earth Scientist https://www.digitalearthscience.com/ Department of Geological Sciences and Geological Engineering Miller Hall 314, 36 Union St Queen’s University Kingston, ON K7L 3N6, Canada Queen’s University is situated on traditional Anishinaabe and Haudenosaunee Territory.<https://www.queensu.ca/encyclopedia/t/traditional-territories>

________________________________ From: raulleoncz ***@***.***> Sent: Wednesday, May 8, 2024 9:36 PM To: SPECFEM/specfem2d ***@***.***> Cc: Subscribed ***@***.***> Subject: [SPECFEM/specfem2d] Program received signal SIGILL: Illegal instruction. (Issue #1222) Hello, First of all, I am sorry for the inconvenience. I have been trying to do a simulation in an acoustic-elastic medium but after some time steps I got the following error: Program received signal SIGILL: Illegal instruction. Backtrace for this error: Could not print backtrace: executable file is not an executable Could not print backtrace: executable file is not an executable #0 0x100b32f33 #1<#1> 0x100b321db #2<#2> 0x19990b583 zsh: illegal hardware instruction ./bin/xspecfem2D The first time I though the simulation got unstable but after checking in the solver.log I noticed that the CFL condition was right, the number of grid points per wavelength was fine and I was using an adequate source frequency. After checking the log file I found that after 4000 time steps the 'Max absolute value of scalar field in fluid (acoustic)' and the 'Color image maximum amplitude' quickly skyrocketed to very large values:

________________________________ Time step number 5 t = 0.0032 s out of 10000

________________________________ We have done 4.00040001E-02 % of the total Max norm of vector field in solid (elastic) = 0.00000000 Max absolute value of scalar field in fluid (acoustic) = 1.53421927E-02 Color image maximum amplitude = 4.0870062075555325E-003

________________________________ Time step number 2000 t = 1.5992 s out of 10000

________________________________ We have done 19.9919987 % of the total Max norm of vector field in solid (elastic) = 1.48898596E-03 Max absolute value of scalar field in fluid (acoustic) = 236.648941 Color image maximum amplitude = 1.4626149786636233E-003

________________________________ Time step number 4000 t = 3.1992 s out of 10000

________________________________ We have done 39.9939995 % of the total Max norm of vector field in solid (elastic) = 1718.72803 Max absolute value of scalar field in fluid (acoustic) = 19952228.0 Color image maximum amplitude = 2.1745527217967010E+022 At the end the error message says: code became unstable and blew up (image_color_data) Error detected, aborting MPI... proc 0 Here are some screenshots: image.png (view on web)<https://github.com/SPECFEM/specfem2d/assets/127082990/beeb3cc6-96fd-4590-9004-8ce73c03059a> image.png (view on web)<https://github.com/SPECFEM/specfem2d/assets/127082990/98748edc-c773-4f31-8479-5f11f3a75167> Has anyone had a similar problem? Just in case I will add the log files. Thank you for your help! solver.log<https://github.com/SPECFEM/specfem2d/files/15256498/solver.log> — Reply to this email directly, view it on GitHub<#1222>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABMCQ4SVO3CV623N5NXPSC3ZBLHLNAVCNFSM6AAAAABHN7BOKCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4DMNZTGQ3DEMY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[raulleoncz]

Thank you @homnath for your help, I really appreciate it.
I tried decreasing the time step and also modifying the mesh and the error is still there. Looking to the pictures, what I believe is that my simulation is having troubles dealing with the fluid-solid interface, this because the error always occurs at this point.

[homnath]

That is interesting. I noticed that you have the material properties for the solid: P-wave velocity. . . . . . . . . . . (cp) = 1.40000000E+03 S-wave velocity. . . . . . . . . . . (cs) = 1.00000000E+03 Mass density. . . . . . . . . . (density) = 1.00000000E+03 Poisson's ratio. . . . . . . . .(poisson) =-2.08333333E-02 Are those material properties realistic? A negative passion ratio is extremely rare. Best, Hom Nath

…

--------------------------------------- Hom Nath Gharti, PhD Assistant Professor | Digital Earth Scientist https://www.digitalearthscience.com/ Department of Geological Sciences and Geological Engineering Miller Hall 314, 36 Union St Queen’s University Kingston, ON K7L 3N6, Canada Queen’s University is situated on traditional Anishinaabe and Haudenosaunee Territory.<https://www.queensu.ca/encyclopedia/t/traditional-territories>

________________________________ From: raulleoncz ***@***.***> Sent: Wednesday, May 15, 2024 3:09 PM To: SPECFEM/specfem2d ***@***.***> Cc: Hom Nath Gharti ***@***.***>; Mention ***@***.***> Subject: Re: [SPECFEM/specfem2d] Program received signal SIGILL: Illegal instruction. (Issue #1222) Thank you @homnath<https://github.com/homnath> for your help, I really appreciate it. I tried decreasing the time step and also modifying the mesh and the error is still there. Looking to the pictures, what I believe is that my simulation is having troubles dealing with the fluid-solid interface, this because the error always occurs at this point. — Reply to this email directly, view it on GitHub<#1222 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABMCQ4RJIQR4BU23RG3CZNDZCOXHNAVCNFSM6AAAAABHN7BOKCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJTGI3TMNJUGE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[raulleoncz]

Actually, I am not working exactly with those properties.
I am doing some inversion examples using specfem2d as the solver. For this, I used a .xyz file to generate the necessary .bin files and after that, those bin files are the ones I am using for the simulation.
My starting configuration was this:

MODEL = default
SAVE_MODEL = binary
nbmodels = 2
1 -1 0 0 0.5 0 0 0 0 0 0 0 0 0 0
2 1 1000.0 1400.0 0 0 0 9999 9999 0 0 0 0 0 0
TOMOGRAPHY_FILE = ./DATA/Initial_data.xyz

Then, using the bin files I had to configure it this way:

MODEL = gll
nbmodels = 2
1 1 1000.0 1500.0 1000.0 0 0 9999 9999 0 0 0 0 0 0
2 1 1000.0 1400.0 0 0 0 9999 9999 0 0 0 0 0 0
TOMOGRAPHY_FILE = ./DATA/Initial_data.xyz

After a few iterations I got the above error at the same point. What I noticed is that when I changed
1 1 1000.0 1500.0 1000.0 0 0 9999 9999 0 0 0 0 0 0 to 1 1 1000.0 1500.0 0 0 0 9999 9999 0 0 0 0 0 0, specfem has no problem to do the simulation, maybe because the entire model is now acoustic. This made me think the problem is how we deal with the interfaz fluid-solid.

Thank you for your help.

[homnath]

Interesting. What happens if you provide the P and S wave speeds, which do not give the negative Poisson's ratio (less than 0.5)?

…

--------------------------------------- Hom Nath Gharti, PhD Assistant Professor | Digital Earth Scientist https://www.digitalearthscience.com/ Department of Geological Sciences and Geological Engineering Miller Hall 314, 36 Union St Queen’s University Kingston, ON K7L 3N6, Canada Queen’s University is situated on traditional Anishinaabe and Haudenosaunee Territory.<https://www.queensu.ca/encyclopedia/t/traditional-territories>

________________________________ From: raulleoncz ***@***.***> Sent: Wednesday, May 15, 2024 4:31 PM To: SPECFEM/specfem2d ***@***.***> Cc: Hom Nath Gharti ***@***.***>; Mention ***@***.***> Subject: Re: [SPECFEM/specfem2d] Program received signal SIGILL: Illegal instruction. (Issue #1222) Actually, I am not working exactly with those properties. I am doing some inversion examples using specfem2d as the solver. For this, I used a .xyz file to generate the necessary .bin files and after that, those bin files are the ones I am using for the simulation. My starting configuration was this: MODEL = default SAVE_MODEL = binary nbmodels = 2 1 -1 0 0 0.5 0 0 0 0 0 0 0 0 0 0 2 1 1000.0 1400.0 0 0 0 9999 9999 0 0 0 0 0 0 TOMOGRAPHY_FILE = ./DATA/Initial_data.xyz Then, using the bin files I had to configure it this way: MODEL = gll nbmodels = 2 1 1 1000.0 1500.0 1000.0 0 0 9999 9999 0 0 0 0 0 0 2 1 1000.0 1400.0 0 0 0 9999 9999 0 0 0 0 0 0 TOMOGRAPHY_FILE = ./DATA/Initial_data.xyz After a few iterations I got the above error at the same point. What I noticed is that when I changed 1 1 1000.0 1500.0 1000.0 0 0 9999 9999 0 0 0 0 0 0 to 1 1 1000.0 1500.0 0 0 0 9999 9999 0 0 0 0 0 0, specfem has no problem to do the simulation, maybe because the entire model is now acoustic. This made me think the problem is how we deal with the interfaz fluid-solid. Thank you for your help. — Reply to this email directly, view it on GitHub<#1222 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABMCQ4Q7PV23QHAZAYW4HMLZCPA2DAVCNFSM6AAAAABHN7BOKCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJTGQYDKMZZHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[raulleoncz]

Hello @homnath, sorry for my late response.

I have tried using different values, for example:
1 1 2600.0 3500.0 1700.0 0 0 9999 9999 0 0 0 0 0 0
2 1 1000.0 1400.0 0 0 0 9999 9999 0 0 0 0 0 0

But I got the same error. Now, I'm trying to modify the mesh, specially at the fluid-solid interface. Maybe using a more refined mesh could work.