Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bench: cleanup redundant code and add Operator and XDSLOperator at same file #21

Merged
merged 61 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
687e857
mpi: Add custom topology from devito codebase
georgebisbas Jul 14, 2023
9af33a5
mpi: Add tests for Custom topology
georgebisbas Jul 14, 2023
2d33ad2
devito/mpi/distributed.py
georgebisbas Jul 14, 2023
aa7e25a
Clean benchmark
georgebisbas Jul 14, 2023
d4d0ed3
Cleanup benchmark
georgebisbas Jul 14, 2023
416f1ec
bench: cleanup
georgebisbas Jul 19, 2023
51dbc2d
bench: cleanup 3d
georgebisbas Jul 19, 2023
fe1650c
bench: more cleanup, drop redundant
georgebisbas Jul 19, 2023
479d478
bench: more cleanup
georgebisbas Jul 19, 2023
1142684
add 'set -eo pipefail' to compiler pipeline to catch errors early
AntonLydike Jul 20, 2023
8e5e7ae
Use /bin/bash for set -eo pipefail.
PapyChacal Jul 20, 2023
335b5f5
add todo.
PapyChacal Jul 20, 2023
e9c76a7
Make xDSL flow use a temp .mlir file just like the usual temp .c file.
PapyChacal Jul 20, 2023
5df0f76
operator: Add fixed for xdsloperator compilation - apply_kernel
georgebisbas Jul 21, 2023
6fc0ff7
Lower subviews.
PapyChacal Jul 21, 2023
b346498
Link to MLIR runner utils.
PapyChacal Jul 21, 2023
bd48ffa
c_runner_utils rather.
PapyChacal Jul 21, 2023
d9c4239
Merge branch 'add_custom_topology' into bench_edits
PapyChacal Jul 21, 2023
c389f96
Reverse stencil.apply inputs and try to name accordingly.
PapyChacal Jul 22, 2023
c5f9552
fix data copy, buffer play
georgebisbas Jul 26, 2023
d8b5dba
wave2d.py
georgebisbas Jul 27, 2023
a67394e
Comment out pdb.
PapyChacal Jul 27, 2023
6de1971
Fix initial buffer order.
PapyChacal Jul 27, 2023
afca66d
add canonicalize-dmp pass to dmp pipeline
AntonLydike Jul 31, 2023
f2604c6
Merge pull request #22 from xdslproject/emilien/try-fix-wave
georgebisbas Aug 2, 2023
8087f95
Add tiling.
PapyChacal Aug 1, 2023
63ab369
Add proper quoting.
PapyChacal Aug 1, 2023
e15ce97
Add dimensionality-1 tiling dimensions logic.
PapyChacal Aug 2, 2023
bacf1af
mpi: Init effort for serial modelling on wave operator
georgebisbas Aug 2, 2023
7f3b37e
mpi: wip
georgebisbas Aug 2, 2023
e54cd7b
mpi: wip
georgebisbas Aug 2, 2023
625e976
wave3d: cleanup
georgebisbas Aug 3, 2023
b73489a
wave2d: cleanup
georgebisbas Aug 3, 2023
e4db7f9
mpi-mfe: Add
georgebisbas Aug 3, 2023
ea7fe19
hacky fix for row major dmp.grid
AntonLydike Aug 3, 2023
d45c113
bench: Conditional execution heat2d
georgebisbas Aug 4, 2023
ae3d586
bench: Conditional execution heat3d
georgebisbas Aug 4, 2023
9a973cc
bench: Generalize benchmarking scripts
georgebisbas Aug 4, 2023
4b34fc5
bench: Generalize wave3d
georgebisbas Aug 4, 2023
7d7e639
wave: TryAdd example with no Operator
georgebisbas Aug 4, 2023
26abd2e
add datatest
georgebisbas Aug 4, 2023
fd312d5
setup: Add necessary data
georgebisbas Aug 5, 2023
8db89d7
bench: Load dt to XDSL
georgebisbas Aug 5, 2023
bcd7a8d
setup: Save extent
georgebisbas Aug 5, 2023
8c8858a
bench: Add so to saved data
georgebisbas Aug 5, 2023
72fb27d
bench: Add wave3d setup
georgebisbas Aug 5, 2023
722e998
bench: compress saved data
georgebisbas Aug 5, 2023
aa73e33
bench: compress properly u.data[:]
georgebisbas Aug 5, 2023
163db36
Merge pull request #23 from xdslproject/emilien/stencil-tiling
georgebisbas Aug 6, 2023
ad55988
bench: More cleanup and tiling merge
georgebisbas Aug 6, 2023
6cfe569
bench: Hide pyvista req
georgebisbas Aug 6, 2023
80c0d31
Merge pull request #24 from xdslproject/bench_edits-2
georgebisbas Aug 9, 2023
dde653e
Insert necessary boilerplate. stencil lowerings doesn't handle it.
PapyChacal Aug 9, 2023
0d73b6e
Add more sensible and resilient tile sizes.
PapyChacal Aug 4, 2023
9082d9f
Try with a arguments-minimizing pipeline.
PapyChacal Aug 4, 2023
84b5522
Improve args-minimization pipeline (by still folding all compile-time…
PapyChacal Aug 6, 2023
c694e02
Remove superfluous GPU passes.
PapyChacal Aug 7, 2023
cca08a4
Add direct .so backdoor capability, and XDSL_SKIP_CLEAN env variable …
PapyChacal Aug 7, 2023
f057a23
Use DeVito's par-tile.
PapyChacal Aug 8, 2023
baa38fb
Use the boilerplate flag to not always copy to GPU.
PapyChacal Aug 9, 2023
fa0276f
Merge pull request #26 from xdslproject/emilien/gpu-again
georgebisbas Aug 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion devito/ir/ietxdsl/cluster_to_ssa.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ def _convert_eq(self, eq: LoweredEq):
), f"can only write to offset [0,0,0], given {offsets[1:]}"

self.block.add_op(stencil.ReturnOp.get([rhs_result]))
outermost_block.add_op(func.Return.get())
outermost_block.add_op(func.Return())

return func.FuncOp.from_region(
"apply_kernel", [], [], Region([outermost_block])
Expand Down
2 changes: 1 addition & 1 deletion devito/ir/ietxdsl/xdsl_passes.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ def _op_to_func(op: Operator):
ietxdsl_functions.myVisit(i, block=block, ssa_vals=ssa_val_dict)

# add a trailing return
block.add_op(func.Return.get())
block.add_op(func.Return())

func_op = func.FuncOp.from_region(str(op.name), arg_types, [], Region([block]))

Expand Down
22 changes: 14 additions & 8 deletions devito/operator/xdsl_operator.py
georgebisbas marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@

CFLAGS = "-O3 -march=native -mtune=native"

MLIR_CPU_PIPELINE = '"builtin.module(canonicalize, cse, loop-invariant-code-motion, canonicalize, cse, loop-invariant-code-motion,cse,canonicalize,fold-memref-alias-ops,lower-affine,convert-scf-to-cf,convert-math-to-llvm,convert-func-to-llvm{use-bare-ptr-memref-call-conv},finalize-memref-to-llvm,canonicalize,cse)"'
MLIR_OPENMP_PIPELINE = '"builtin.module(canonicalize, cse, loop-invariant-code-motion, canonicalize, cse, loop-invariant-code-motion,cse,canonicalize,fold-memref-alias-ops,lower-affine,finalize-memref-to-llvm,loop-invariant-code-motion,canonicalize,cse,convert-scf-to-openmp,finalize-memref-to-llvm,convert-scf-to-cf,convert-func-to-llvm{use-bare-ptr-memref-call-conv},convert-openmp-to-llvm,convert-math-to-llvm,reconcile-unrealized-casts,canonicalize,cse)"'
MLIR_CPU_PIPELINE = '"builtin.module(canonicalize, cse, loop-invariant-code-motion, canonicalize, cse, loop-invariant-code-motion,cse,canonicalize,fold-memref-alias-ops,expand-strided-metadata, loop-invariant-code-motion,lower-affine,convert-scf-to-cf,convert-math-to-llvm,convert-func-to-llvm{use-bare-ptr-memref-call-conv},finalize-memref-to-llvm,canonicalize,cse)"'
MLIR_OPENMP_PIPELINE = '"builtin.module(canonicalize, cse, loop-invariant-code-motion, canonicalize, cse, loop-invariant-code-motion,cse,canonicalize,fold-memref-alias-ops,expand-strided-metadata, loop-invariant-code-motion,lower-affine,finalize-memref-to-llvm,loop-invariant-code-motion,canonicalize,cse,convert-scf-to-openmp,finalize-memref-to-llvm,convert-scf-to-cf,convert-func-to-llvm{use-bare-ptr-memref-call-conv},convert-openmp-to-llvm,convert-math-to-llvm,reconcile-unrealized-casts,canonicalize,cse)"'
# gpu-launch-sink-index-computations seemed to have no impact
MLIR_GPU_PIPELINE = '"builtin.module(test-math-algebraic-simplification,scf-parallel-loop-tiling{parallel-loop-tile-sizes=128,1,1},func.func(gpu-map-parallel-loops),convert-parallel-loops-to-gpu,fold-memref-alias-ops,lower-affine,gpu-kernel-outlining,canonicalize,cse,convert-arith-to-llvm{index-bitwidth=64},finalize-memref-to-llvm{index-bitwidth=64},convert-scf-to-cf,convert-cf-to-llvm{index-bitwidth=64},canonicalize,cse,gpu.module(convert-gpu-to-nvvm,reconcile-unrealized-casts,canonicalize,gpu-to-cubin),gpu-to-llvm,canonicalize,cse)"'
MLIR_GPU_PIPELINE = '"builtin.module(test-math-algebraic-simplification,scf-parallel-loop-tiling{parallel-loop-tile-sizes=128,1,1},func.func(gpu-map-parallel-loops),convert-parallel-loops-to-gpu,fold-memref-alias-ops,expand-strided-metadata,lower-affine,gpu-kernel-outlining,canonicalize,cse,convert-arith-to-llvm{index-bitwidth=64},finalize-memref-to-llvm{index-bitwidth=64},convert-scf-to-cf,convert-cf-to-llvm{index-bitwidth=64},canonicalize,cse,gpu.module(convert-gpu-to-nvvm,reconcile-unrealized-casts,canonicalize,gpu-to-cubin),gpu-to-llvm,canonicalize,cse)"'

XDSL_CPU_PIPELINE = "stencil-shape-inference,convert-stencil-to-ll-mlir,printf-to-llvm"
XDSL_GPU_PIPELINE = "stencil-shape-inference,convert-stencil-to-ll-mlir{target=gpu},printf-to-llvm"
Expand Down Expand Up @@ -137,7 +137,10 @@ def _jit_compile(self):
print("JIT Backdoor: loading xdsl file from: " + backdoor)
with open(backdoor, 'r') as f:
module_str = f.read()

source_name = os.path.splitext(self._tf.name)[0] + ".mlir"
source_file = open(source_name, "w")
source_file.write(module_str)
source_file.close()
# compile IR using xdsl-opt | mlir-opt | mlir-translate | clang
try:
cflags = CFLAGS
Expand All @@ -151,17 +154,20 @@ def _jit_compile(self):
if is_gpu:
cflags += " -lmlir_cuda_runtime "

cmd = f'xdsl-opt -p {xdsl_pipeline} |' \
# TODO More detailed error handling manually,
# instead of relying on a bash-only feature.
cmd = 'set -eo pipefail; '\
f'xdsl-opt {source_name} -p {xdsl_pipeline} |' \
f'mlir-opt -p {mlir_pipeline} | ' \
f'mlir-translate --mlir-to-llvmir | ' \
f'{cc} {cflags} -shared -o {self._tf.name} {self._interop_tf.name} -xir -'

print(cmd)
res = subprocess.run(
cmd,
shell=True,
input=module_str,
text=True,
capture_output=True,
executable="/bin/bash"
)

if res.returncode != 0:
Expand All @@ -177,7 +183,7 @@ def _jit_compile(self):
elapsed = self._profiler.py_timers['jit-compile']

perf("XDSLOperator `%s` jit-compiled `%s` in %.2f s with `mlir-opt`" %
(self.name, self._tf.name, elapsed))
(self.name, source_name, elapsed))

@property
def _soname(self):
Expand Down
41 changes: 10 additions & 31 deletions fast/diffusion_2D_wBCs.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,49 +42,28 @@
u = TimeFunction(name='u', grid=grid, space_order=so)

# Reset our data field and ICs
# u.data[:, :, :] = 0.1
init_hat(field=u.data[0], dx=dx, dy=dy, value=1.)


# u.data[0, :, int(ny/2)] = 2

a = Constant(name='a')
# Create an equation with second-order derivatives
eq = Eq(u.dt, a * u.laplace, subdomain=grid.interior)
# eq = Eq(u.dt, a * u.laplace, subdomain=grid.interior)
eq = Eq(u.dt, a * u.laplace)
stencil = solve(eq, u.forward)
eq_stencil = Eq(u.forward, stencil)

# Create boundary condition expressions
x, y = grid.dimensions
t = grid.stepping_dim

# Add boundary conditions
# bc = [Eq(u[t+1, x, y, 0], 2.)] # bottom
# bc += [Eq(u[t+1, x, y, nz-1], 2.)] # top
# bc += [Eq(u[t+1, 0, y, z], 2.)] # left
# bc += [Eq(u[t+1, nx-1, y, z], 2.)] # right

# bc += [Eq(u[t+1, x, 0, z], 2.)] # front
# bc += [Eq(u[t+1, x, ny-1, z], 2.)] # back

print(eq_stencil)

# Create an operator that updates the forward stencil point
# plus adding boundary conditions
# op = Operator([eq_stencil] + bc, subdomain=grid.interior)

# No BCs
# op = XDSLOperator([eq_stencil])
op = Operator([eq_stencil])
# print(op.ccode)
# dt = 0.00002
# import pdb;pdb.set_trace()
# Apply the operator for a number of timesteps
print(dt)
initdata = u.data[:]
op = Operator([eq_stencil], name='DevitoOperator')
op.apply(time=nt, dt=dt, a=nu)

print("Field norm is:", norm(u))
print("Devito Field norm is:", norm(u))

plot_image(u.data[0], cmap="seismic")
# Reset data and run XDSLOperator
init_hat(field=u.data[0], dx=dx, dy=dy, value=1.)
xdslop = XDSLOperator([eq_stencil], name='XDSLOperator')
xdslop.apply(time=nt, dt=dt, a=nu)

# import pdb;pdb.set_trace()
print("XDSL Field norm is:", norm(u))
34 changes: 14 additions & 20 deletions fast/diffusion_3D_wBCs.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,39 +65,33 @@ def plot_3dfunc(u):

a = Constant(name='a')
# Create an equation with second-order derivatives
eq = Eq(u.dt, a * u.laplace, subdomain=grid.interior)
eq = Eq(u.dt, a * u.laplace)

stencil = solve(eq, u.forward)
eq_stencil = Eq(u.forward, stencil)

# Create boundary condition expressions
x, y, z = grid.dimensions
t = grid.stepping_dim

# Add boundary conditions
# bc = [Eq(u[t+1, x, y, 0], 2.)] # bottom
# bc += [Eq(u[t+1, x, y, nz-1], 2.)] # top
# bc += [Eq(u[t+1, 0, y, z], 2.)] # left
# bc += [Eq(u[t+1, nx-1, y, z], 2.)] # right

# bc += [Eq(u[t+1, x, 0, z], 2.)] # front
# bc += [Eq(u[t+1, x, ny-1, z], 2.)] # back

print(eq_stencil)

# Create an operator that updates the forward stencil point
# plus adding boundary conditions
# op = Operator([eq_stencil] + bc, subdomain=grid.interior)

# No BCs
# op = XDSLOperator([eq_stencil])
op = Operator([eq_stencil])
# print(op.ccode)

# Create Operator
op = Operator([eq_stencil], name='DevitoOperator')
# Apply the operator for a number of timesteps
op.apply(time=nt, dt=dt, a=nu)
print("Devito Field norm is:", norm(u))

# Reset field
u.data[:, :, :, :] = 0
u.data[:, :, :, int(nz/2)] = 1
xdslop = XDSLOperator([eq_stencil], name='xDSLOperator')
# Apply the xdsl operator for a number of timesteps
xdslop.apply(time=nt, dt=dt, a=nu)

if args.plot:
plot_3dfunc(u)

print("Field norm is:", norm(u))
print("XDSL Field norm is:", norm(u))

# import pdb;pdb.set_trace()
18 changes: 9 additions & 9 deletions fast/nd_nwave_devito_nodamp.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,30 +102,30 @@ def plot_3dfunc(u):

# Finally we define the source injection and receiver read function to generate
# the corresponding code
src_term = src.inject(field=u.forward, expr=src * dt**2 / model.m)
op = Operator([stencil] + src_term, subs=model.spacing_map)

print(time_range)
src_term = src.inject(field=u.forward, expr=src * dt**2 / model.m)
op = Operator([stencil] + src_term, subs=model.spacing_map, name='DevitoOperator')
# Run with source and plot
# import pdb;pdb.set_trace()
op.apply(time=time_range.num-1, dt=model.critical_dt)

if len(shape) == 3:
if args.plot:
plot_3dfunc(u)

# Run more with no sources now (Not supported in xdsl)
# op = XDSLOperator([stencil])
initdata = u.data[:]

# op.apply(time=time_range.num-1, dt=model.critical_dt)
# Run more with no sources now (Not supported in xdsl)
xdslop = XDSLOperator([stencil], name='XDSLOperator')
xdslop.apply(time=time_range.num-1, dt=model.critical_dt)

if len(shape) == 3:
if args.plot:
plot_3dfunc(u)

# print(norm(u))
print(norm(u))

# todo edit data
# Reset initial data
u.data[:] = initdata

# Run more with no sources now (Not supported in xdsl)
xdslop = XDSLOperator([stencil])
Expand Down
124 changes: 0 additions & 124 deletions fast/nd_wave_devito_nodamp.py

This file was deleted.

Loading