[guide] Profiling

Profiling

Profiling PyOP2 programs

Profiling a PyOP2 program is as simple as profiling any other Python code. Let's use the advection-diffusion demo in the PyOP2 demo folder:

python -m cProfile -o adv_diff.dat adv_diff.py -m meshes/large

This will run the entire program under [cProfile](python -m cProfile -o adv_diff.dat adv_diff.py -m meshes/large) and write the profiling data to adv_diff.dat. Omitting -o will print a summary to stdout, which is not very helpful in most cases.

Creating a graph

Luckily there's a much better way of representing the profiling data using the excellent gprof2dot to generated a graph. Install from PyPI with sudo pip install gprof2dot.

Use as follows to create a PDF:

gprof2dot -f pstats -n 1 adv_diff.dat | dot -Tpdf -o adv_diff.pdf

-f pstats tells gprof2dot that it's dealing with Python cProfile data (and not actual gprof data) and -n 1 ignores everything that makes up less than 1% of the total runtime - most likely you're not interested in that (the default is 0.5).

Profiling PyOP2 from inside Fluidity

That's slightly more tricky, but not very much. Since you have no (easy) control over how Fluidity invokes the Python interpreter you need to resort to another way of using cProfile. Wrap your code inside the UFL Python input box in diamond as follows:

import cProfile
cProfile.run("""
<code>
""", 'adv_diff.'+str(time)+'.part')

Note the use of time as part of the file name, which is necessary to get an output file per time step instead of getting it overwritten each time step.

Consolidating profiles from different time steps

To aggregate the profiling data from all time steps, save the following as concat.py:

"""Usage: concat.py PATTERN FILE"""

import sys
from glob import glob
from pstats import Stats

if len(sys.argv) != 3:
    print __doc__
    sys.exit(1)
files = glob(sys.argv[1])
s = Stats(files[0])
for f in files[1:]: s.add(f)
s.dump_stats(sys.argv[2])

Use it as python concat.py 'adv_diff.*.part' adv_diff.dat and then call gprof2dot as before.