Disable verifier in main pass manager pipeline

[Pangoraw]

This is similar to the Enzyme-JAX lit tests where the verifier is disabled between passes for some complex-related test cases.

[avik-pal]

This gives me a segfault for the NeuralOperators test case. We should add the minimal case from the linked issue as a test

julia> @code_hlo ∇fno(fno, ps, st, x)

[1490249] signal 11 (128): Segmentation fault
in expression starting at REPL[28]:1
unknown function (ip: 0x74be8c60476b)
_ZL8readBitsPKcmm at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZNK4mlir17DenseElementsAttr18IntElementIteratordeEv at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZNK4mlir17DenseElementsAttr13getSplatValueIN4llvm5APIntEEENSt9enable_ifIXoontsrSt10is_base_ofINS_9AttributeET_E5valuesrSt7is_sameIS6_S7_E5valueES7_E4typeEv at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZNK12_GLOBAL__N_111AddSimplify15matchAndRewriteEN4mlir9stablehlo5AddOpERNS1_15PatternRewriterE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZZN4mlir17PatternApplicator15matchAndRewriteEPNS_9OperationERNS_15PatternRewriterEN4llvm12function_refIFbRKNS_7PatternEEEENS6_IFvS9_EEENS6_IFNS5_13LogicalResultES9_EEEENKUlvE_clEv at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir17PatternApplicator15matchAndRewriteEPNS_9OperationERNS_15PatternRewriterEN4llvm12function_refIFbRKNS_7PatternEEEENS6_IFvS9_EEENS6_IFNS5_13LogicalResultES9_EEE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN12_GLOBAL__N_126GreedyPatternRewriteDriver15processWorklistEv at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir28applyPatternsAndFoldGreedilyERNS_6RegionERKNS_23FrozenRewritePatternSetENS_19GreedyRewriteConfigEPb at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform15ApplyPatternsOp10applyToOneERNS0_17TransformRewriterEPNS_9OperationERNS0_21ApplyToEachResultListERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform6detail20applyTransformToEachINS0_15ApplyPatternsOpERN4llvm14iterator_rangeINS4_20filter_iterator_implIPKPNS_9OperationEZNKS0_14TransformState13getPayloadOpsENS_5ValueEEUlS8_E_St26bidirectional_iterator_tagEEEEEENS_27DiagnosedSilenceableFailureET_RNS0_17TransformRewriterEOT0_RNS4_15SmallVectorImplINS0_21ApplyToEachResultListEEERSB_ at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform20TransformEachOpTraitINS0_15ApplyPatternsOpEE5applyERNS0_17TransformRewriterERNS0_16TransformResultsERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform6detail35TransformOpInterfaceInterfaceTraits5ModelINS0_15ApplyPatternsOpEE5applyEPKNS2_7ConceptEPNS_9OperationERNS0_17TransformRewriterERNS0_16TransformResultsERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform20TransformOpInterface5applyERNS0_17TransformRewriterERNS0_16TransformResultsERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform14TransformState14applyTransformENS0_20TransformOpInterfaceE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZL18applySequenceBlockRN4mlir5BlockENS_9transform22FailurePropagationModeERNS2_14TransformStateERNS2_16TransformResultsE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform15NamedSequenceOp5applyERNS0_17TransformRewriterERNS0_16TransformResultsERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform6detail35TransformOpInterfaceInterfaceTraits5ModelINS0_15NamedSequenceOpEE5applyEPKNS2_7ConceptEPNS_9OperationERNS0_17TransformRewriterERNS0_16TransformResultsERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform20TransformOpInterface5applyERNS0_17TransformRewriterERNS0_16TransformResultsERNS0_14TransformStateE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform14TransformState14applyTransformENS0_20TransformOpInterfaceE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform15applyTransformsEPNS_9OperationENS0_20TransformOpInterfaceERKNS_11RaggedArrayIN4llvm12PointerUnionIJS2_NS_9AttributeENS_5ValueEEEEEERKNS0_16TransformOptionsEbNS5_12function_refIFvRNS0_14TransformStateEEEENSG_IFNS5_13LogicalResultESI_EEE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir9transform27applyTransformNamedSequenceENS_11RaggedArrayIN4llvm12PointerUnionIJPNS_9OperationENS_9AttributeENS_5ValueEEEEEENS0_20TransformOpInterfaceENS_8ModuleOpERKNS0_16TransformOptionsE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN12_GLOBAL__N_115InterpreterPass14runOnOperationEv at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir6detail17OpToOpPassAdaptor3runEPNS_4PassEPNS_9OperationENS_15AnalysisManagerEbj at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir6detail17OpToOpPassAdaptor11runPipelineERNS_13OpPassManagerEPNS_9OperationENS_15AnalysisManagerEbjPNS_16PassInstrumentorEPKNS_19PassInstrumentation18PipelineParentInfoE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir11PassManager9runPassesEPNS_9OperationENS_15AnalysisManagerE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
_ZN4mlir11PassManager3runEPNS_9OperationE at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
mlirPassManagerRunOnOp at /mnt/.julia/artifacts/481e7bc75d2b2606c01cb87ace13a1efb182c5a9/lib/libReactantExtra.so (unknown line)
mlirPassManagerRunOnOp at /mnt/software/lux/Reactant.jl/src/mlir/libMLIR_h.jl:5853 [inlined]
run! at /mnt/software/lux/Reactant.jl/src/mlir/IR/Pass.jl:65 [inlined]
#run_pass_pipeline!#1 at /mnt/software/lux/Reactant.jl/src/Compiler.jl:252
run_pass_pipeline! at /mnt/software/lux/Reactant.jl/src/Compiler.jl:247 [inlined]
#compile_mlir!#8 at /mnt/software/lux/Reactant.jl/src/Compiler.jl:296
compile_mlir! at /mnt/software/lux/Reactant.jl/src/Compiler.jl:278 [inlined]
#6 at /mnt/software/lux/Reactant.jl/src/Compiler.jl:273 [inlined]
context! at /mnt/software/lux/Reactant.jl/src/mlir/IR/Context.jl:76
unknown function (ip: 0x74bdfd979716)
#compile_mlir#5 at /mnt/software/lux/Reactant.jl/src/Compiler.jl:271
compile_mlir at /mnt/software/lux/Reactant.jl/src/Compiler.jl:268
unknown function (ip: 0x74bdfd977ded)
jl_apply at /cache/build/builder-demeter6-6/julialang/julia-master/src/julia.h:2157 [inlined]
do_call at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:126
eval_value at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:663
jl_interpret_toplevel_thunk at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:821
jl_toplevel_eval_flex at /cache/build/builder-demeter6-6/julialang/julia-master/src/toplevel.c:943
jl_toplevel_eval_flex at /cache/build/builder-demeter6-6/julialang/julia-master/src/toplevel.c:886
eval_body at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:625
eval_body at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:539
jl_interpret_toplevel_thunk at /cache/build/builder-demeter6-6/julialang/julia-master/src/interpreter.c:821
jl_toplevel_eval_flex at /cache/build/builder-demeter6-6/julialang/julia-master/src/toplevel.c:943
jl_toplevel_eval_flex at /cache/build/builder-demeter6-6/julialang/julia-master/src/toplevel.c:886
jl_toplevel_eval_flex at /cache/build/builder-demeter6-6/julialang/julia-master/src/toplevel.c:886
ijl_toplevel_eval_in at /cache/build/builder-demeter6-6/julialang/julia-master/src/toplevel.c:994
eval at ./boot.jl:430 [inlined]
eval at ./Base.jl:130 [inlined]
repleval at /home/avikpal/.vscode-insiders/extensions/julialang.language-julia-1.127.2/scripts/packages/VSCodeServer/src/repl.jl:229
#112 at /home/avikpal/.vscode-insiders/extensions/julialang.language-julia-1.127.2/scripts/packages/VSCodeServer/src/repl.jl:192 [inlined]
with_logstate at ./logging/logging.jl:522
with_logger at ./logging/logging.jl:632 [inlined]
#111 at /home/avikpal/.vscode-insiders/extensions/julialang.language-julia-1.127.2/scripts/packages/VSCodeServer/src/repl.jl:193
unknown function (ip: 0x74bdfd97637f)
jl_apply at /cache/build/builder-demeter6-6/julialang/julia-master/src/julia.h:2157 [inlined]
jl_f__call_latest at /cache/build/builder-demeter6-6/julialang/julia-master/src/builtins.c:875
#invokelatest#2 at ./essentials.jl:1055 [inlined]
invokelatest at ./essentials.jl:1052
unknown function (ip: 0x74be40700422)
jl_apply at /cache/build/builder-demeter6-6/julialang/julia-master/src/julia.h:2157 [inlined]
do_apply at /cache/build/builder-demeter6-6/julialang/julia-master/src/builtins.c:831
#64 at /home/avikpal/.vscode-insiders/extensions/julialang.language-julia-1.127.2/scripts/packages/VSCodeServer/src/eval.jl:34
unknown function (ip: 0x74be407c52cf)
jl_apply at /cache/build/builder-demeter6-6/julialang/julia-master/src/julia.h:2157 [inlined]
start_task at /cache/build/builder-demeter6-6/julialang/julia-master/src/task.c:1202
Allocations: 194865630 (Pool: 194846171; Big: 19459); GC: 193
[1]    1490249 segmentation fault (core dumped)  julia --project=docs --threads=auto --depwarn=yes

[Pangoraw]

This IR (#238 (comment)) now fails for another reason:

loc(callsite("-":5:10 at "-":28:12)): error: op requires the same element type for all operands and results
%31 = "stablehlo.add"(%30, %29) : (tensor<10xcomplex<f64>>, tensor<10xf64>) -> tensor<10xcomplex<f64>>

that's caused by the reverse derivative of stablehlo.abs being wrong for complex numbers. Enzyme then generates the following which is valid for arith:

%31 = "arith.addf"(%30, %29) <{fastmath = #arith.fastmath<none>}> : (tensor<10xcomplex<f64>>, tensor<10xf64>) -> tensor<10xcomplex<f64>>

and then arith-raise make it to:

%31 = "stablehlo.add"(%30, %29) : (tensor<10xcomplex<f64>>, tensor<10xf64>) -> tensor<10xcomplex<f64>>

which is invalid. So we should modify the reverse rule for abs of complex numbers

[wsmoses]

so the thing is we only want to disable the verifier before arith raise and after enzyme [all other verifications are good and will stop bugs]

[Pangoraw]

should we do 3 invocations of run_pass_pipeline! with "enzyme,arith-raise" having the verifier disabled ? Or in enzyme-jax have a enzyme-hlo pass which just wraps both enzyme and arith-raise ?

[wsmoses]

I'm game for either option, up to you