panic: heap memory exhausted

[ianchanning]

Specifications

• Client Version: 1.33.2
• InfluxDB Version: 2
• Platform: Linux (Alpine Linux 3.18.4)

Code sample to reproduce problem

I don't know how I can reproduce this

Expected behavior

No heap memory error

Actual behavior

NodeHttpTransport.ts generates this error: panic: heap memory exhausted

Additional info

Frankly I expect you will close this bug report. I have no good way of reproducing it. But currently there are zero hits for this on the internet, so I'm adding one. This could be because we're using Alpine Linux, but no idea.

[bednar]

Hi @ianchanning,

Thank you for using our client, and I appreciate you reaching out to us with this issue.

The error message you encountered, panic: heap memory exhausted, suggests that your application may be running out of memory during its operations. This can occur for a number of reasons, such as querying a large volume of data or an inefficient use of resources within the application.

To help us better understand and resolve the issue, could you provide the following details?

1. Code Structure: How is your code organized? A general overview or specific sections related to how you use our client could be very insightful.

2. Data Volume: How much data are you querying at a time? Understanding the volume of data being processed could help identify if the issue is related to large datasets.

3. Frequency of the Issue: Does this error occur with every query you make, or is it intermittent? Knowing whether the issue is consistent or sporadic can help us pinpoint whether it's a systemic problem or triggered under specific conditions.

4. Environment Details: Information about the environment where the client is running (e.g., client environment, available memory) would also be useful.

This information will greatly assist in diagnosing the problem more accurately. If possible, a snippet of the relevant part of your code where you integrate our client and execute queries would also be helpful.

Looking forward to your response so we can work together to solve this issue.

Best Regards

[ianchanning]

Thank's for your reply... hopefully this is close to what you need. Although I don't really see how this can lead to a memory problem - it could be that we have a memory leak somewhere else and this is just the small query that hit the limit.

1. Code structure: We have a graphql-service running in a GCP cloud function
2. Data volume: Very small (KB) - it is only requesting the last() row from a flux query
3. Frequency of the issue: first time that it has occured
4. Environment Details:
   • CPU Description: AMD EPYC 7B13
   • Free Memory: 3.3 GiB
   • Memory Size: 3.8 GiB
   • Node: v20.9.0

Here are the basic methods where we call the query

  private constructor() {
    this.influx = new InfluxDB({
      url: getEnvProperty('INFLUX_URL'),
      token: process.env.INFLUX_TOKEN,
    });
    this.buckets = new BucketsAPI(this.influx);
    this.query = this.influx.getQueryApi(getEnvProperty('INFLUX_ORGANIZATION_ID'));
  }
  async getLastSensorValue({
    devEui,
    metric,
    start = new Date(1000),
    stop = new Date(),
    calibrationSensors = [],
  }: {
    devEui: string;
    metric: string;
    start?: Date;
    stop?: Date;
    calibrationSensors?: CalibrationSensorWithCalibration[];
  }): Promise<Data | undefined> {
    return new Promise<Data | undefined>((resolve, reject) => {
      const fluxQuery = `
        from(bucket: "${getBucketName()}")
          |> range(start: ${start.toISOString()}, stop: ${stop.toISOString()})
          |> filter(fn: (r) => r._measurement == "sensor")
          |> filter(fn: (r) => r["devEUI"] == "${devEui}")
          |> filter(fn: (r) => r["metric"] == "${metric}")
          |> ${group(['devEUI'])}
          ${calibrationMap(calibrationSensors)}
          |> last()
      `;
      let result: Data | undefined = undefined;
      this.query.queryRows(fluxQuery, {
        next(row, tableMeta) {
          const point = tableMeta.toObject(row);
          result = {
            value: point._value,
            time: new Date(point._time),
          };
        },
        error(error: Error) {
          reject(error);
        },
        complete() {
          resolve(result);
        },
      });
    });
  }

[bednar]

Do you have a detailed log or stack trace for a specific GCP function run that ended with panic: heap memory exhausted?

[ianchanning]

This is all I have for the stack trace, not sure if you can tell anything from it:

GraphQLError: panic: heap memory exhausted
  File "/root/node_modules/.pnpm/@[email protected]/node_modules/@influxdata/influxdb-client/src/impl/node/NodeHttpTransport.ts", line 354, col 11, in IncomingMessage.<anonymous>
  File "node:events", line 526, col 35, in IncomingMessage.emit
  File "node:domain", line 488, col 12, in IncomingMessage.emit
  File "node:internal/streams/readable", line 1408, col 12, in endReadableNT
  File "node:internal/process/task_queues", line 82, col 21, in processTicksAndRejections