Remove redundancy in pieChart instances - JavaScript

Remove redundancy in dockerStats - JavaScript
Auto calculate current process metrics by default
Reuse disk usage metrics to avoid redundant calls
Update runbook and README.md

README.md

@@ -53,19 +53,24 @@ pyninja start
 > _By default, `PyNinja` will look for a `.env` file in the current working directory._
 </details>
 
+- **APIKEY** - API Key for authentication.
 - **NINJA_HOST** - Hostname for the API server.
 - **NINJA_PORT** - Port number for the API server.
-- **WORKERS** - Number of workers for the uvicorn server.
 - **REMOTE_EXECUTION** - Boolean flag to enable remote execution.
 - **API_SECRET** - Secret access key for running commands on server remotely.
 - **MONITOR_USERNAME** - Username to authenticate the monitoring page.
 - **MONITOR_PASSWORD** - Password to authenticate the monitoring page.
 - **MONITOR_SESSION** - Session timeout for the monitoring page.
 - **MAX_CONNECTIONS** - Maximum number of monitoring sessions allowed in parallel.
-- **SERVICE_MANAGER** - Service manager filepath to handle the service status requests.
+- **PROCESSES** - List of process names to include in the monitor page.
+- **SERVICES** - List of service names to include in the monitor page.
+- **GPU_LIB** - GPU library filepath to use for monitoring.
+- **DISK_LIB** - Disk library filepath to use for monitoring.
+- **SERVICE_LIB** - Memory library filepath to use for monitoring.
+- **PROCESSOR_LIB** - Processor library filepath to use for monitoring.
 - **DATABASE** - FilePath to store the auth database that handles the authentication errors.
 - **RATE_LIMIT** - List of dictionaries with `max_requests` and `seconds` to apply as rate limit.
-- **APIKEY** - API Key for authentication.
+- **LOG_CONFIG** - Logging configuration file path.
 
 ⚠️ Enabling remote execution can be extremely risky and a major security threat.
 So use **caution** and set the **API_SECRET** to a strong value.

docs/README.html

@@ -90,19 +90,24 @@ <h2>Environment Variables<a class="headerlink" href="#environment-variables" tit
 <div><p><em>By default, <code class="docutils literal notranslate"><span class="pre">PyNinja</span></code> will look for a <code class="docutils literal notranslate"><span class="pre">.env</span></code> file in the current working directory.</em></p>
 </div></blockquote>
 </details><ul class="simple">
+<li><p><strong>APIKEY</strong> - API Key for authentication.</p></li>
 <li><p><strong>NINJA_HOST</strong> - Hostname for the API server.</p></li>
 <li><p><strong>NINJA_PORT</strong> - Port number for the API server.</p></li>
-<li><p><strong>WORKERS</strong> - Number of workers for the uvicorn server.</p></li>
 <li><p><strong>REMOTE_EXECUTION</strong> - Boolean flag to enable remote execution.</p></li>
 <li><p><strong>API_SECRET</strong> - Secret access key for running commands on server remotely.</p></li>
 <li><p><strong>MONITOR_USERNAME</strong> - Username to authenticate the monitoring page.</p></li>
 <li><p><strong>MONITOR_PASSWORD</strong> - Password to authenticate the monitoring page.</p></li>
 <li><p><strong>MONITOR_SESSION</strong> - Session timeout for the monitoring page.</p></li>
 <li><p><strong>MAX_CONNECTIONS</strong> - Maximum number of monitoring sessions allowed in parallel.</p></li>
-<li><p><strong>SERVICE_MANAGER</strong> - Service manager filepath to handle the service status requests.</p></li>
+<li><p><strong>PROCESSES</strong> - List of process names to include in the monitor page.</p></li>
+<li><p><strong>SERVICES</strong> - List of service names to include in the monitor page.</p></li>
+<li><p><strong>GPU_LIB</strong> - GPU library filepath to use for monitoring.</p></li>
+<li><p><strong>DISK_LIB</strong> - Disk library filepath to use for monitoring.</p></li>
+<li><p><strong>SERVICE_LIB</strong> - Memory library filepath to use for monitoring.</p></li>
+<li><p><strong>PROCESSOR_LIB</strong> - Processor library filepath to use for monitoring.</p></li>
 <li><p><strong>DATABASE</strong> - FilePath to store the auth database that handles the authentication errors.</p></li>
 <li><p><strong>RATE_LIMIT</strong> - List of dictionaries with <code class="docutils literal notranslate"><span class="pre">max_requests</span></code> and <code class="docutils literal notranslate"><span class="pre">seconds</span></code> to apply as rate limit.</p></li>
-<li><p><strong>APIKEY</strong> - API Key for authentication.</p></li>
+<li><p><strong>LOG_CONFIG</strong> - Logging configuration file path.</p></li>
 </ul>
 <p>⚠️ Enabling remote execution can be extremely risky and a major security threat.
 So use <strong>caution</strong> and set the <strong>API_SECRET</strong> to a strong value.</p>

docs/README.md

@@ -53,19 +53,24 @@ pyninja start
 > _By default, `PyNinja` will look for a `.env` file in the current working directory._
 </details>
 
+- **APIKEY** - API Key for authentication.
 - **NINJA_HOST** - Hostname for the API server.
 - **NINJA_PORT** - Port number for the API server.
-- **WORKERS** - Number of workers for the uvicorn server.
 - **REMOTE_EXECUTION** - Boolean flag to enable remote execution.
 - **API_SECRET** - Secret access key for running commands on server remotely.
 - **MONITOR_USERNAME** - Username to authenticate the monitoring page.
 - **MONITOR_PASSWORD** - Password to authenticate the monitoring page.
 - **MONITOR_SESSION** - Session timeout for the monitoring page.
 - **MAX_CONNECTIONS** - Maximum number of monitoring sessions allowed in parallel.
-- **SERVICE_MANAGER** - Service manager filepath to handle the service status requests.
+- **PROCESSES** - List of process names to include in the monitor page.
+- **SERVICES** - List of service names to include in the monitor page.
+- **GPU_LIB** - GPU library filepath to use for monitoring.
+- **DISK_LIB** - Disk library filepath to use for monitoring.
+- **SERVICE_LIB** - Memory library filepath to use for monitoring.
+- **PROCESSOR_LIB** - Processor library filepath to use for monitoring.
 - **DATABASE** - FilePath to store the auth database that handles the authentication errors.
 - **RATE_LIMIT** - List of dictionaries with `max_requests` and `seconds` to apply as rate limit.
-- **APIKEY** - API Key for authentication.
+- **LOG_CONFIG** - Logging configuration file path.
 
 ⚠️ Enabling remote execution can be extremely risky and a major security threat.
 So use **caution** and set the **API_SECRET** to a strong value.

docs/_sources/README.md.txt

@@ -53,19 +53,24 @@ pyninja start
 > _By default, `PyNinja` will look for a `.env` file in the current working directory._
 </details>
 
+- **APIKEY** - API Key for authentication.
 - **NINJA_HOST** - Hostname for the API server.
 - **NINJA_PORT** - Port number for the API server.
-- **WORKERS** - Number of workers for the uvicorn server.
 - **REMOTE_EXECUTION** - Boolean flag to enable remote execution.
 - **API_SECRET** - Secret access key for running commands on server remotely.
 - **MONITOR_USERNAME** - Username to authenticate the monitoring page.
 - **MONITOR_PASSWORD** - Password to authenticate the monitoring page.
 - **MONITOR_SESSION** - Session timeout for the monitoring page.
 - **MAX_CONNECTIONS** - Maximum number of monitoring sessions allowed in parallel.
-- **SERVICE_MANAGER** - Service manager filepath to handle the service status requests.
+- **PROCESSES** - List of process names to include in the monitor page.
+- **SERVICES** - List of service names to include in the monitor page.
+- **GPU_LIB** - GPU library filepath to use for monitoring.
+- **DISK_LIB** - Disk library filepath to use for monitoring.
+- **SERVICE_LIB** - Memory library filepath to use for monitoring.
+- **PROCESSOR_LIB** - Processor library filepath to use for monitoring.
 - **DATABASE** - FilePath to store the auth database that handles the authentication errors.
 - **RATE_LIMIT** - List of dictionaries with `max_requests` and `seconds` to apply as rate limit.
-- **APIKEY** - API Key for authentication.
+- **LOG_CONFIG** - Logging configuration file path.
 
 ⚠️ Enabling remote execution can be extremely risky and a major security threat.
 So use **caution** and set the **API_SECRET** to a strong value.

docs/genindex.html

@@ -376,6 +376,8 @@ <h2 id="L">L</h2>
 <h2 id="M">M</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="index.html#pyninja.monitor.resources.map_docker_stats">map_docker_stats() (in module pyninja.monitor.resources)</a>
+</li>
       <li><a href="index.html#pyninja.models.EnvConfig.max_connections">max_connections (pyninja.models.EnvConfig attribute)</a>
 </li>
       <li><a href="index.html#pyninja.models.RateLimit.max_requests">max_requests (pyninja.models.RateLimit attribute)</a>

docs/index.html

@@ -1884,6 +1884,23 @@ <h2>Authenticator<a class="headerlink" href="#id1" title="Permalink to this head
 </section>
 <section id="module-pyninja.monitor.resources">
 <span id="resources"></span><h2>Resources<a class="headerlink" href="#module-pyninja.monitor.resources" title="Permalink to this heading">¶</a></h2>
+<dl class="py function">
+<dt class="sig sig-object py" id="pyninja.monitor.resources.map_docker_stats">
+<span class="sig-prename descclassname"><span class="pre">pyninja.monitor.resources.</span></span><span class="sig-name descname"><span class="pre">map_docker_stats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">json_data</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#pyninja.monitor.resources.map_docker_stats" title="Permalink to this definition">¶</a></dt>
+<dd><p>Map the JSON data to a dictionary.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>json_data</strong> – JSON data from the docker stats command.</p>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>Returns a dictionary with container stats.</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p>Dict[str, str]</p>
+</dd>
+</dl>
+</dd></dl>
+
 <dl class="py function">
 <dt class="sig sig-object py" id="pyninja.monitor.resources.get_cpu_percent">
 <span class="sig-prename descclassname"><span class="pre">pyninja.monitor.resources.</span></span><span class="sig-name descname"><span class="pre">get_cpu_percent</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">cpu_interval</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">float</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#pyninja.monitor.resources.get_cpu_percent" title="Permalink to this definition">¶</a></dt>

pyninja/main.py

@@ -1,4 +1,5 @@
 import logging
+import os
 import pathlib
 
 import uvicorn
@@ -150,6 +151,7 @@ def start(**kwargs) -> None:
         BASE_LOGGER.warning("Remote execution disabled")
     # Conditional endpoint based on monitor_username and monitor_password
     if all((models.env.monitor_username, models.env.monitor_password)):
+        models.env.processes.append(str(os.getpid()))
         PyNinjaAPI.routes.extend(get_all_monitor_routes(dependencies))
         PyNinjaAPI.add_exception_handler(
             exc_class_or_status_code=exceptions.RedirectException,

pyninja/monitor/resources.py

@@ -2,7 +2,6 @@
 import json
 import logging
 import os
-import shutil
 from concurrent.futures import ThreadPoolExecutor
 from typing import Dict, List
 
@@ -15,6 +14,27 @@
 LOGGER = logging.getLogger("uvicorn.default")
 
 
+def map_docker_stats(json_data: Dict[str, str]) -> Dict[str, str]:
+    """Map the JSON data to a dictionary.
+
+    Args:
+        json_data: JSON data from the docker stats command.
+
+    Returns:
+        Dict[str, str]:
+        Returns a dictionary with container stats.
+    """
+    return {
+        "Container ID": json_data.get("ID"),
+        "Container Name": json_data.get("Name"),
+        "CPU": json_data.get("CPUPerc"),
+        "Memory": json_data.get("MemPerc"),
+        "Memory Usage": json_data.get("MemUsage"),
+        "Block I/O": json_data.get("BlockIO"),
+        "Network I/O": json_data.get("NetIO"),
+    }
+
+
 def get_cpu_percent(cpu_interval: int) -> List[float]:
     """Get CPU usage percentage.
 
@@ -45,7 +65,7 @@ async def get_docker_stats() -> List[Dict[str, str]]:
         LOGGER.debug(stderr.decode().strip())
         return []
     return [
-        {key: value for key, value in json.loads(line).items() if key != "PIDs"}
+        map_docker_stats(json.loads(line))
         for line in stdout.decode().strip().splitlines()
     ]
 
@@ -62,7 +82,6 @@ async def get_system_metrics() -> Dict[str, dict]:
     return dict(
         memory_info=psutil.virtual_memory()._asdict(),
         swap_info=psutil.swap_memory()._asdict(),
-        disk_info=shutil.disk_usage("/")._asdict(),
         load_averages=dict(m1=m1, m5=m5, m15=m15),
     )
 

pyninja/monitor/routes.py

@@ -195,7 +195,10 @@ async def websocket_endpoint(websocket: WebSocket, session_token: str = Cookie(N
     session_timestamp = models.ws_session.client_auth.get(websocket.client.host).get(
         "timestamp"
     )
+    # Base task with a placeholder asyncio sleep to start the task loop
     task = asyncio.create_task(asyncio.sleep(0.1))
+    # Store disk usage information (during startup) to avoid repeated calls
+    disk_info = shutil.disk_usage("/")._asdict()
     while True:
         # Validate session asynchronously (non-blocking)
         # This way of handling session validation is more efficient than using a blocking call
@@ -225,6 +228,7 @@ async def websocket_endpoint(websocket: WebSocket, session_token: str = Cookie(N
             await websocket.close()
             break
         data = await resources.system_resources()
+        data["disk_info"] = disk_info
         try:
             await websocket.send_json(data)
         except WebSocketDisconnect: