Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash due to recent change in LLVM #851

Open
pdhirajkumarprasad opened this issue Sep 27, 2024 · 4 comments
Open

Crash due to recent change in LLVM #851

pdhirajkumarprasad opened this issue Sep 27, 2024 · 4 comments
Assignees
@vivekkhandelwal1 @vinayakdsci

Comments

@pdhirajkumarprasad
Copy link

for the give IR

module {
  func.func @torch_jit(  %arg2: !torch.vtensor<[1],si64>, %arg3: !torch.vtensor<[?,?,576],f32>) -> !torch.vtensor<[3,?,8,?,?],f32>  attributes {torch.onnx_meta.ir_version = 7 : si64, torch.onnx_meta.opset_version = 21 : si64, torch.onnx_meta.producer_name = "pytorch", torch.onnx_meta.producer_version = "1.12.1"} {
    %1 = torch.operator "onnx.Constant"() {torch.onnx.value = dense<-1> : tensor<1xsi64>} : () -> !torch.vtensor<[1],si64> 
    %2 = torch.operator "onnx.Concat"(%arg2, %arg2, %arg2, %arg2, %1) {torch.onnx.axis = 0 : si64} : (!torch.vtensor<[1],si64>, !torch.vtensor<[1],si64>, !torch.vtensor<[1],si64>, !torch.vtensor<[1],si64>, !torch.vtensor<[1],si64>) -> !torch.vtensor<[5],si64> 
    %3 = torch.operator "onnx.Reshape"(%arg3, %2) : (!torch.vtensor<[?,?,576],f32>, !torch.vtensor<[5],si64>) -> !torch.vtensor<[?,?,3,8,?],f32> 
    %4 = torch.operator "onnx.Transpose"(%3) {torch.onnx.perm = [2 : si64, 0 : si64, 3 : si64, 4 : si64, 1 : si64]} : (!torch.vtensor<[?,?,3,8,?],f32>) -> !torch.vtensor<[3,?,8,?,?],f32> 
    return %4 : !torch.vtensor<[3,?,8,?,?],f32>
  }
}

we are seeing crash

iree-compile: /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/include/mlir/IR/StorageUniquerSupport.h:180: static ConcreteT mlir::detail::StorageUserBase<mlir::RankedTensorType, mlir::TensorType, mlir::detail::RankedTensorTypeStorage, mlir::detail::TypeUniquer, mlir::ShapedType::Trait, mlir::ValueSemantics>::get(mlir::MLIRContext *, Args &&...) [ConcreteT = mlir::RankedTensorType, BaseT = mlir::TensorType, StorageT = mlir::detail::RankedTensorTypeStorage, UniquerT = mlir::detail::TypeUniquer, Traits = <mlir::ShapedType::Trait, mlir::ValueSemantics>, Args = <llvm::ArrayRef<long> &, mlir::Type &, mlir::Attribute &>]: Assertion `succeeded( ConcreteT::verifyInvariants(getDefaultDiagnosticEmitFn(ctx), args...))' failed.
Please report issues to https://github.com/iree-org/iree/issues and include the crash backtrace.
Stack dump:
0.	Program arguments: iree-compile --iree-input-type=auto --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=llvm-cpu --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false model.torch_onnx.mlir
 #0 0x00007ff5de9948f7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/lib/Support/Unix/Signals.inc:723:13
 #1 0x00007ff5de992b30 llvm::sys::RunSignalHandlers() /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/lib/Support/Signals.cpp:106:18
 #2 0x00007ff5de994fba SignalHandler(int) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/lib/Support/Unix/Signals.inc:413:1

command to reproduce:

iree-compile --iree-input-type=auto --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=llvm-cpu --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false t1.mlir
@ScottTodd
Copy link
Member

ScottTodd commented Sep 27, 2024 •
edited
Loading

Is there any more to that stack trace? That only shows the signal, not what caused it (beyond the assert)

@pdhirajkumarprasad
Copy link
Author

@ScottTodd hee is complete stack trace

iree-compile: /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/include/mlir/IR/StorageUniquerSupport.h:180: static ConcreteT mlir::detail::StorageUserBase<mlir::RankedTensorType, mlir::TensorType, mlir::detail::RankedTensorTypeStorage, mlir::detail::TypeUniquer, mlir::ShapedType::Trait, mlir::ValueSemantics>::get(mlir::MLIRContext *, Args &&...) [ConcreteT = mlir::RankedTensorType, BaseT = mlir::TensorType, StorageT = mlir::detail::RankedTensorTypeStorage, UniquerT = mlir::detail::TypeUniquer, Traits = <mlir::ShapedType::Trait, mlir::ValueSemantics>, Args = <llvm::ArrayRef<long> &, mlir::Type &, mlir::Attribute &>]: Assertion `succeeded( ConcreteT::verifyInvariants(getDefaultDiagnosticEmitFn(ctx), args...))' failed.
Please report issues to https://github.com/iree-org/iree/issues and include the crash backtrace.
Stack dump:
0.	Program arguments: iree-compile t1.mlir
 #0 0x00007fcf07ead8f7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/lib/Support/Unix/Signals.inc:723:13
 #1 0x00007fcf07eabb30 llvm::sys::RunSignalHandlers() /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/lib/Support/Signals.cpp:106:18
 #2 0x00007fcf07eadfba SignalHandler(int) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/lib/Support/Unix/Signals.inc:413:1
 #3 0x00007fcf015a2520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
 #4 0x00007fcf015f69fc __pthread_kill_implementation ./nptl/./nptl/pthread_kill.c:44:76
 #5 0x00007fcf015f69fc __pthread_kill_internal ./nptl/./nptl/pthread_kill.c:78:10
 #6 0x00007fcf015f69fc pthread_kill ./nptl/./nptl/pthread_kill.c:89:10
 #7 0x00007fcf015a2476 gsignal ./signal/../sysdeps/posix/raise.c:27:6
 #8 0x00007fcf015887f3 abort ./stdlib/./stdlib/abort.c:81:7
 #9 0x00007fcf0158871b _nl_load_domain ./intl/./intl/loadmsgcat.c:1177:9
#10 0x00007fcf01599e96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
#11 0x00007fcf07f4cf94 (/proj/xhdhdstaff6/dhirajp/localBuild/iree-build/lib/libIREECompiler.so+0x6578f94)
#12 0x00007fcf07f4ceb8 mlir::RankedTensorType::get(llvm::ArrayRef<long>, mlir::Type, mlir::Attribute) /proj/xhdhdstaff6/dhirajp/localBuild/iree-build/llvm-project/tools/mlir/include/mlir/IR/BuiltinTypes.cpp.inc:308:3
#13 0x00007fcf0bc79703 llvm::FailureOr<mlir::RankedTensorType> mlir::iree_compiler::IREE::Flow::(anonymous namespace)::canonicalizeSubViewParts<mlir::iree_compiler::IREE::Flow::DispatchTensorStoreOp>(mlir::iree_compiler::IREE::Flow::DispatchTensorStoreOp, mlir::RankedTensorType, llvm::SmallVector<mlir::OpFoldResult, 6u>&, llvm::SmallVector<mlir::OpFoldResult, 6u>&, llvm::SmallVector<mlir::OpFoldResult, 6u>&) /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/Dialect/Flow/IR/FlowOpFolders.cpp:558:7
#14 0x00007fcf0bc79703 mlir::iree_compiler::IREE::Flow::(anonymous namespace)::DispatchTensorStoreOpWithOffsetSizesAndStridesConstantArgumentFolder::matchAndRewrite(mlir::iree_compiler::IREE::Flow::DispatchTensorStoreOp, mlir::PatternRewriter&) const /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/Dialect/Flow/IR/FlowOpFolders.cpp:670:25
#15 0x00007fcf0c2ba3de mlir::PatternApplicator::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&, llvm::function_ref<bool (mlir::Pattern const&)>, llvm::function_ref<void (mlir::Pattern const&)>, llvm::function_ref<llvm::LogicalResult (mlir::Pattern const&)>)::$_2::operator()() const /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Rewrite/PatternApplicator.cpp:212:13
#16 0x00007fcf0c2ba3de void llvm::function_ref<void ()>::callback_fn<mlir::PatternApplicator::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&, llvm::function_ref<bool (mlir::Pattern const&)>, llvm::function_ref<void (mlir::Pattern const&)>, llvm::function_ref<llvm::LogicalResult (mlir::Pattern const&)>)::$_2>(long) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:45:12
#17 0x00007fcf0c2b741f mlir::PatternApplicator::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&, llvm::function_ref<bool (mlir::Pattern const&)>, llvm::function_ref<void (mlir::Pattern const&)>, llvm::function_ref<llvm::LogicalResult (mlir::Pattern const&)>) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Rewrite/PatternApplicator.cpp:233:9
#18 0x00007fcf0c299eae (anonymous namespace)::GreedyPatternRewriteDriver::processWorklist() /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Transforms/Utils/GreedyPatternRewriteDriver.cpp:0:0
#19 0x00007fcf0c296a50 (anonymous namespace)::RegionPatternRewriteDriver::simplify(bool*) &&::$_8::operator()() const /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Transforms/Utils/GreedyPatternRewriteDriver.cpp:874:11
#20 0x00007fcf0c296a50 void llvm::function_ref<void ()>::callback_fn<(anonymous namespace)::RegionPatternRewriteDriver::simplify(bool*) &&::$_8>(long) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:45:12
#21 0x00007fcf0c296a50 llvm::function_ref<void ()>::operator()() const /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:68:12
#22 0x00007fcf0c296a50 void mlir::MLIRContext::executeAction<(anonymous namespace)::GreedyPatternRewriteIteration, long&>(llvm::function_ref<void ()>, llvm::ArrayRef<mlir::IRUnit>, long&) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/include/mlir/IR/MLIRContext.h:275:7
#23 0x00007fcf0c296a50 (anonymous namespace)::RegionPatternRewriteDriver::simplify(bool*) && /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Transforms/Utils/GreedyPatternRewriteDriver.cpp:872:10
#24 0x00007fcf0c296a50 mlir::applyPatternsAndFoldGreedily(mlir::Region&, mlir::FrozenRewritePatternSet const&, mlir::GreedyRewriteConfig, bool*) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Transforms/Utils/GreedyPatternRewriteDriver.cpp:919:47
#25 0x00007fcf09ccb4a1 llvm::LogicalResult::failed() const /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/Support/LogicalResult.h:43:43
#26 0x00007fcf09ccb4a1 mlir::applyPatternsAndFoldGreedily(mlir::Operation*, mlir::FrozenRewritePatternSet const&, mlir::GreedyRewriteConfig, bool*) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/mlir/include/mlir/Transforms/GreedyPatternRewriteDriver.h:160:14
#27 0x00007fcf09ccb4a1 mlir::iree_compiler::IREE::Util::ApplyPatternsPass::runOnOperation() /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/Dialect/Util/Transforms/ApplyPatterns.cpp:41:16
#28 0x00007fcf0805b185 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int)::$_7::operator()() const /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Pass/Pass.cpp:0:17
#29 0x00007fcf0805b185 void llvm::function_ref<void ()>::callback_fn<mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int)::$_7>(long) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:45:12
#30 0x00007fcf0805b185 llvm::function_ref<void ()>::operator()() const /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/ADT/STLFunctionalExtras.h:68:12
#31 0x00007fcf0805b185 void mlir::MLIRContext::executeAction<mlir::PassExecutionAction, mlir::Pass&>(llvm::function_ref<void ()>, llvm::ArrayRef<mlir::IRUnit>, mlir::Pass&) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/include/mlir/IR/MLIRContext.h:275:7
#32 0x00007fcf0805b185 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Pass/Pass.cpp:520:21
#33 0x00007fcf0805b8f8 llvm::LogicalResult::failed() const /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/Support/LogicalResult.h:43:43
#34 0x00007fcf0805b8f8 llvm::failed(llvm::LogicalResult) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/Support/LogicalResult.h:71:58
#35 0x00007fcf0805b8f8 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Pass/Pass.cpp:592:9
#36 0x00007fcf0805dc19 mlir::PassManager::run(mlir::Operation*) /proj/rdi/staff/dhirajp/localBuild/iree/third_party/llvm-project/mlir/lib/Pass/Pass.cpp:0:0
#37 0x00007fcf07e00f30 llvm::LogicalResult::failed() const /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/Support/LogicalResult.h:43:43
#38 0x00007fcf07e00f30 llvm::failed(llvm::LogicalResult) /proj/xhdhdstaff6/dhirajp/localBuild/iree/third_party/llvm-project/llvm/include/llvm/Support/LogicalResult.h:71:58
#39 0x00007fcf07e00f30 mlir::iree_compiler::embed::(anonymous namespace)::Invocation::runPipeline(iree_compiler_pipeline_t) /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/API/Internal/CompilerDriver.cpp:997:7
#40 0x00007fcf07e00f30 ireeCompilerInvocationPipeline /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/API/Internal/CompilerDriver.cpp:1432:23
#41 0x00007fcf080214c8 mlir::iree_compiler::runIreecMain(int, char**)::$_2::operator()(iree_compiler_source_t*) const /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/Tools/iree_compile_lib.cc:254:11
#42 0x00007fcf08020d01 mlir::iree_compiler::runIreecMain(int, char**) /proj/xhdhdstaff6/dhirajp/localBuild/iree/compiler/src/iree/compiler/Tools/iree_compile_lib.cc:0:10
#43 0x00007fcf01589d90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#44 0x00007fcf01589e40 call_init ./csu/../csu/libc-start.c:128:20
#45 0x00007fcf01589e40 __libc_start_main ./csu/../csu/libc-start.c:379:5
#46 0x0000557cc7ddd6d5 _start (/proj/xhdhdstaff6/dhirajp/localBuild/iree-build/tools/iree-compile+0x16d5)
Abort (core dumped)

@ScottTodd
Copy link
Member

Nice, thanks. Do you also have the IR leading up to the crash?

@pdhirajkumarprasad
Copy link
Author

@ScottTodd attached is detail log contain the IR for all passes

dump.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vivekkhandelwal1 vivekkhandelwal1

@vinayakdsci vinayakdsci

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ScottTodd @vivekkhandelwal1 @vinayakdsci @pdhirajkumarprasad