Averaging high frequency readings for PID control stability

[mwiercinski]

As per this discussion the oversampling has been removed in favor of filtering. I am struggling with implementing the advice.

My setup is such that I defined a new device, let's call it "BT Raw" and used a symbolic assignment for BT in a form of (Y13+Y13[-1]+Y13[-2]+Y13[-3])/4. Unfortunately the value of the new device is not available until I press START recording. This makes PID control non-functional between the batches.

This is a minor inconvenience as I can start recording for the purpose of preheating the roaster. That said I interpret that the intended workflow is to for START to be followed by the CHARGE shortly after - is it? The unavailability of the derived signal feels unnecessarily limiting - why not calculate symbolic assignment without recording being active?

What I am trying to achieve is to average out the BT reading, which has some random noise (presumably poor electronics on the controller, I excluded the ground loop). Is there a better way of implementing it?

[roasterdave]

As per #1433 (reply in thread) the oversampling has been removed in favor of filtering. I am struggling with implementing the advice.

Artisan provides filtering already. No need to build your own. Go to menu Config>> Curves>> Filters tab. The settings in the "Curve Filter" section affect ET and BT. Here is a useful article that describes the mechanics and effects of smoothing.

I interpret that the intended workflow is to for START to be followed by the CHARGE shortly after

CHARGE can happen any time after START. If I understand what you are trying to do, you will want to click START at the beginning of your BBT.

why not calculate symbolic assignment without recording being active

The equation you show above looks backwards in time to get the values of previous samples. In order to capture and preserve those samples Artisan must be recording them.

[mwiercinski]

Artisan provides filtering already. No need to build your own. Go to menu Config>> Curves>> Filters tab. The settings in the "Curve Filter" section affect ET and BT. Here is a useful article that describes the mechanics and effects of smoothing.

To clarify, can you confirm that filtering applied from Config >> Curve >> Filters is effective for BT when used as PV for PID control? Since it's behind UI / graphing menu I was not expecting that, but if that is the case then it's awesome.

This specific part of the linked article claims otherwise:

"The smoothed curves are only computed for display purposes in Artisan"

If it is wrong, would it make sense to fix the documentation?

CHARGE can happen any time after START. If I understand what you are trying to do, you will want to click START at the beginning of your BBT.

BBT? I honestly don't have much preference how to use it. I am new to the tool, and don't know how is it intended to be used, so I've been following my intuition and random online tutorials. Before stumbling upon the issue raised in this thread, I've been pre-warming the roaster, and then only recording from the moment when charge temperature has been achieved. I know switched to constant recording now, because self-filtered PID does not work without it.

The initial goal is get PID control right, which is suffering from seemingly random noise from the probe, which now only works after START is pressed (at least in how I implemented it).

In the longer term, after getting PID to where I want it to be, I want to automate detection of achieving the thermal stability between the batches, with a goal of making loud ding to bring the operator (me) to the device to charge. I reckon this can be achieved through alarms automation - do these also require "START" to be pressed?

The equation you show above looks backwards in time to get the values of previous samples. In order to capture and preserve those samples Artisan must be recording them.

This does not sound like a hard constraint, but rather a limit of how data calculation is is implemented, right? It can keep a buffer of the values in memory (presumably of dynamic length sufficient to satisfy the computations), that are not being "recorded".

[roasterdave]

Yes, BT and ET input to the PID are smoothed per the Curve Filter settings.

The BT and ET data recorded to a profile file is not smoothed. That was the motivation behind the comment you pointed to in the blog article. When a profile file is opened in Artisan the smoothing parameters are applied before displaying the raw data. It is worth noting that BT and ET are displayed "raw" during live recording.

I suggest you hit START at the beginning of your heating or BBP. This fully activates Artisan and is the best way to get your setup performing without worrying what operates or doesn't when ON. If you don't want to see the BBP data post roast you can un-select "Show Full" under Display Filter in Config>> Curves>> Filters tab. Many roast operators always record their BBP since it is so important to achieving roast consistency. And yes, BBT was a typo.

The signal fidelity issue you mention will be an ongoing problem with taming a PID. Smoothing is not a panacea. It adds delay which can be counter productive in a system with dynamic SV. Best to solve the noise issue at or very close to the source. Noisy signals have a way of causing ongoing problems in unexpected ways. Eliminating the noise issue now will pay dividends later.

[MAKOMO]

A further limitation is the time it takes for the hardware to take a reading. The datasheets for the popular MAX6675 and MAX31855 say that their "temperature conversion times" are 220 ms and 100 ms, respectively. Of course you can request data more often, but you will not get more readings. Note that those conversion times don't include any communication (first via SPI to the microcontroller and then via serial or BLE to the client).

[mwiercinski]

The device is Kaleido M2S. I suspect the manufacturer doesn't do on-device averaging, but merely proxies the readings from the sensor verbatim. Correcting for it in Artisan is appealing, as modulo the poll limit it is very easy to do, especially with built-in filtering.

I'm going to do some more testing with a private build. If that is supportive (in terms of 1)actually works and 2) is backed by the characteristics of the hardware in the device), would you be open to a pull request to increase the poll frequency limit?

[Terracotta-6]

Kaleido M2S hardware firmware design sample rate 1.5s, The electromagnetic interference noise suffered by BT and ET is very high. It's impossible to change the control accuracy with artisan.Unless you add another hardware mix to change the control accuracy.

[mwiercinski]

I had a better look at the noise of my Kaleido, and tl;dr is that the quantitative data seems to be supportive of the claim that more samples help filtering, without increasing a delay.

I ran two set of measures, with 0.25 and 1.5 seconds respectively on two consecutive nights. On top of the bare data, I created two further datasets, applying moving window average of 6 samples for 0.25s and of 2 samples for 1.5s one. Under assumption that firmware 1.5 second-long sampling is implemented as a moving average, then the effective span of the signal considered in both filtered datasets, is 3 seconds, i.e. equal.

Note for the averaged out data, if the firmware behaviour was to be interpreted as a moving average, the 6 sample mean on 0.25 dataset should not be considered equal to 2 sample mean on 1.5s - the former would be effectively a weighed moving average, with biggest weight attributed to to measurement half-way of the window span.

I then applied moving window that for each sample subtracted a mean of samples before & after, in order to cancel the out ambient temperature change. I then plotted the resulting variations and calculated a moving RMS.

The results are following:

• 1.5s dataset is slightly noisier. This could be to the fact that it was captured on a different night, but also e.g. due to epsilon of data being lost when expecting a perfect timing of sampling with the firmware support.
• Both datasets did benefit from filtering. 0.25 second/6samples one did benefit a lot.

I suspect the assumption that firmware support that @Terracotta-6 mentions, being implemented as a moving average over 1.5 second window is false.

You can find full Colab here. The .alog files are publicly readable. Colab needs permissions to access Google Drive APIs, but if you don't feel like giving them to it, files can be uploaded manually to the session (commented out part of the first cell).

[mwiercinski]

@MAKOMO did you mean to push the change with the min_delay with this commit?

I was about to follow up with testing using a local build and higher frequency, and realised that the value has been recently changed as a part of seemingly unrelated large change.