Prometheus metrics

[rsevilla87]

Type of change

• Refactor
• New feature
• Bug fix
• Optimization
• Documentation Update

Description

Scrape prometheus metrics:

• avg_cpu_usage_router_pods
• avg_cpu_usage_router_nodes
• avg_memory_usage_router_pods
• avg_memory_usage_router_nodes

The indexed documents now look like:

    "timestamp": "2023-11-15T23:17:30.447184191Z",                                                                     
    "total_avg_rps": 15516.18,                                                                                                                                                                                                                
    "rps_stdev": 1346.18,                                                                                                                                                                                                                     
    "stdev_lat": 6204.67,                                                                                              
    "avg_lat_us": 2053.89,                                                                                                                                                                                                                    
    "max_lat_us": 247897,                                                                                                                                                                                                                     
    "p90_lat_us": 1985,                                                                                                                                                                                                                       
    "p95_lat_us": 4330,                                                                                                                                                                                                                       
    "p99_lat_us": 23450,                                                                                               
    "http_errors": 0,                                                                                                  
    "read_errors": 0,                                                                                                  
    "write_errors": 0,                                                                                                                                                                                                                        
    "requests": 2793589,                                                                                                                                                                                                                      
    "timeouts": 0,                                                                                                                                                                                                                            
    "version": "issue-60@19a51244251ae8318f30e622427d92a2099166be",                                                                                                                                                                           
    "infra_metrics": {                                                                                                                                                                                                                        
      "avg_cpu_usage_router_nodes": 0.58,                                                                                                                                                                                                     
      "avg_cpu_usage_router_pods": 0.77,                                                                                                                                                                                                      
      "avg_memory_usage_router_nodes": 13500550826.67,                                                                                                                                                                                        
      "avg_memory_usage_router_pods": 131624277.33                                                                                                                                                                                            
    },                                                                                                                                                                                                                                        
    "platform": "AWS",                                                                                                                                                                                                                        
    "clusterType": "self-managed",                                                                                                                                                                                                            
    "ocpVersion": "4.13.13",                                                                                                                                                                                                                  
    "ocpMajorVersion": "4.13",      

Related Tickets & Documents

• Related Issue #
• Closes [RFE] Scrape CPU/Memory utilization during the test #60

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

• Please describe the System Under Test.
• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[krishvoor]

/lgtm