[WIP]Add ncu report analyzer

[FindHao]

This PR adds a ncu report analyzer to analyze the profiled ncu report. It also adds two metrics memory_traffic and arithmetic_intensity. To avoid excessive profiling overhead, we only profile with necessary ncu metrics.

This PR is a part of operator benchmarking plan

Example commands:

python run_benchmark.py triton --op fp8_gemm --num-inputs 1  --metrics ncu_rep,memory_traffic,arithmetic_intensity

Example output:

  0%|                                                                                                                                                                                                                                                              | 0/1 [00:00<?, ?it/s]==PROF== Connected to process 1289285 (/scratch/yhao/miniconda3/envs/pta_gil/bin/python3.10)
  0%|                                                                                                                                                                                                                                                              | 0/1 [00:00<?, ?it/s]==PROF== Profiling "sm90_xmma_gemm_e4m3f16_e4m3f3..." - 0: 0%....50%....100% - 3 passes
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.34s/it]
             x_val    torch_fp8_gemm-_ncu_trace_in_task
------------------  -----------------------------------
(1024, 1024, 1024)                              success
==PROF== Disconnected from process 1289285
==WARNING== No source files were imported. Check that the target application was compiled with -lineinfo.
==PROF== Report: /scratch/yhao/tmp/tritonbench/fp8_gemm/ncu_traces/torch_fp8_gemm_0/ncu_output.ncu-rep
==PROF== Connected to process 1289431 (/scratch/yhao/miniconda3/envs/pta_gil/bin/python3.10)
  0%|                                                                                                                                                                                                                                                              | 0/1 [00:00<?, ?it/s]==PROF== Profiling "matmul_kernel" - 0: 0%....50%....100% - 3 passes
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:05<00:00,  5.25s/it]
             x_val    triton_fp8_gemm-_ncu_trace_in_task
------------------  ------------------------------------
(1024, 1024, 1024)                               success
==PROF== Disconnected from process 1289431
==PROF== Report: /scratch/yhao/tmp/tritonbench/fp8_gemm/ncu_traces/triton_fp8_gemm_0/ncu_output.ncu-rep
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:14<00:00, 14.40s/it]
             x_val    torch_fp8_gemm-arithmetic_intensity    torch_fp8_gemm-memory_traffic                                                                 torch_fp8_gemm-ncu_rep    triton_fp8_gemm-arithmetic_intensity    triton_fp8_gemm-memory_traffic                                                                 triton_fp8_gemm-ncu_rep
------------------  -------------------------------------  -------------------------------  -------------------------------------------------------------------------------------  --------------------------------------  --------------------------------  --------------------------------------------------------------------------------------
(1024, 1024, 1024)              (1.3621756724589384, 0.0)               (2150656.0, 256.0)  /scratch/yhao/tmp/tritonbench/fp8_gemm/ncu_traces/torch_fp8_gemm_0/ncu_output.ncu-rep                              (0.0, 0.0)                  (2116096.0, 0.0)  /scratch/yhao/tmp/tritonbench/fp8_gemm/ncu_traces/triton_fp8_gemm_0/ncu_output.ncu-rep

[FindHao]

@xuzhao9
Any suggestions on how we should save this data? We need to keep the metric results for each kernel, but we also need aggregated results, right? For example, the memory traffic (both read and write) for the whole operator should be the sum of all kernels' read and write traffic.

[FindHao]

@xuzhao9 @eellison
Do you think the arithmetic intensity of the whole operator can be represented as a weighted average based on execution time?