Problems with colours in AllSky software #715
Comments
|
Can you provide us the exact rpicam-apps command line used to capture the image and the actual JPG/DNG images as well please? Without this, there's not much we can investigate I'm afraid. |
|
Hello @naushir , sorry I’m really not very experienced with Linux. What do you mean with ”rpicam-apps command line”? |
|
rpicam-apps is the application that captures the images for allskycam. The application is configured by a number of parameters on the command line. This would be useful to know exactly how the system is configured to capture images. Perhaps speak with the package authors and they can give you this information? |
|
Hello @naushir ; I will ask them, thanks. |
|
Are you able to attach the original images taken from the camera - the once you posted in the comment above don't have any metadata in them. |
|
Hope these are good?
|
|
Unfortunately these do not have metadata either :( Perhaps you can ask the authors if the original camera image is stored anywhere on the filesystem? |
|
OK, I've asked them to provide you the required information. |
|
I suspect Github may strip metadata off images. It'd be worth trying ZIPing them and attaching the archive. |
|
@babsenfred, to get the command line used to take the pictures, run: There will be one line for every image. You can attach All the images are stored in |
|
I also have the same issue, so maybe I can share my own images and command line... Here are two images taken about 60 seconds apart.
and here are the commands line used for each image.... 2024-09-13T22:23:32.191317-04:00 Allsky allsky[16016]: STARTING EXPOSURE at: 2024-09-13 22:23:32 @ 60.0 sec |
|
@jfharvey, you are using Auto White Balance, correct? |
|
No, Auto white balance is disabled for both day and night captures, with default settings for the RPI HQ camera. 
I tried to enable Auto white balance few weeks ago, but I didn't see any change. Thanks |
|
@jfharvey unfortunately I still cannot see any exif data in the images. As @6by9 said in an eariler comment, it might be GitHub stripping metadata when attaching the files. Could you zip up the image and attach the file instead please? Also, has this only recently started happening, or was the problem always visible? |
|
Hi naushir Here are the two files zipped. But as I said, I checked on my PC before uploading them, and I can't see any metedata in the pictures, but you can double check, maybe I missed something The issue started when I switched to a RPI5. On my RPI4 (using the same camera), there is no "blue image" issue. Here are the last night thumbnails. We can clearly see the issue. 
and here is the zip file |
|
@davidplowman for visibility. @jfharvey and @EricClaeys can you run If we are sure that AWB is disabled, can I suggest that we disable the ALSC algorthim and see if that fixes the issue for you. To do this, edit Thank you for your help in debugging this with us! |
|
Hello naushir, rpicam-apps build: 6202c09 16-02-2024 (17:15:39) I will check with @EricClaeys if it is possible to enable the RAW output and I will try again. I will also disable the ALSC algorithm later today and see if it is better...or not. Regards |
|
Those builds are quite old, would it be possible to update your libcamera libraries as well please? |
|
Thanks naushir I'm 100% sure I did a sudo apt update/upgrade last week, after the OS installation, but I just did another one to be sure, and now, the version is the following rpicam-apps build: d7a1a13 05-09-2024 (14:07:57) Regards |
|
@jfharvey, you can add additional command-line arguments via the EXTRA PARAMETERS setting. |
|
Hello, I also included the RAW files. Maybe you will find something useful in the metadata. Here is an excerpt from the log showing the command line used for each picture. 2024-09-17T20:20:49.579876-04:00 Allsky allsky[5081]: STARTING EXPOSURE at: 2024-09-17 20:20:49 @ 10.0 sec 2024-09-17T20:22:21.709236-04:00 Allsky allsky[5081]: STARTING EXPOSURE at: 2024-09-17 20:22:21 @ 10.0 sec |
|
@naushir, libcamera-still --output '/home/pi/allsky/tmp/image-20240917202049.jpg' --raw --timeout 1 --nopreview --width 4056 --height 3040 --shutter 10000000 --analoggain 8 --quality 100 libcamera-still --output '/home/pi/allsky/tmp/image-20240917202221.jpg' --raw --timeout 1 --nopreview --width 4056 --height 3040 --shutter 10000000 --analoggain 8 --quality 100 |
|
Thanks, these show some interesting results. image-20240917202049.dng ISP output: dcraw output: image-20240917202221.dng ISP output: dcraw output: |
|
From the above results, it seems like the blue tint is present in the RAW image as well? I assume |
|
Ive run the DNGs through RawThreapee and get the same results using the built-in AWB and get exactly the same results. It seems the blue tint is genuinely present in the RAW data. |
|
Can you share what power supply you are using on your Pi5? Is it an official Raspberry Pi supply? |
|
Thanks nausir for your help on that issue. I'm using a POE hat for the raspberry PI 5. |
|
This is pure speculation, but I wonder if this is a power supply issue causing a glitch in the sensor. Would you be able to run with a regular Raspberry Pi 5 power supply and see if it eliminates the problem? You can probably just run this indoors for the test. |
|
I use the original Raspberry power supply and also have this issue. |
|
@babsenfred is this a Raspberry Pi 5 supply (https://www.raspberrypi.com/products/27w-power-supply/)? |
|
That looks like the correct supply. If the blue tint is in the RAW data, the ISP processing is not going to be able to remove it. I'm really not sure what more to look at... It's very curious that this does not happen with Pi 4 though, the sensor runs exactly the same way on both platforms. |
|
I don't know if this is relevant but I was looking at the issues in the INDI-Allsky Github page (that is based on this build) and I couldn't find any colour issues. |
|
@jfharvey Yesterday I set the Auto-Gain and Auto-White-Balance to 'YES' in the nighttime settings. Since then I have no more 'blue' images... I see what happens tonight and I keep you posted. |
Thought I should mention that I am running Raspberry Pi 4 Model B Rev 1.5 and having the same problem with very blue images at dusk and dawn running the TJ Allsky software. |
I have had the Auto-Gain and Auto-White-Balance set to "Yes" in the nighttime settings all along and get blue images, so this seems odd. My settings are attached here in case it is helpful. Thank you! |
|
I am the author of indi-allsky and I can confirm I see the same "blue bias" behavior for the IMX477 with the latest packages on Rpi5, but only in jpeg mode. I have not observed it in DNG raw mode. What I found interesting is the blue images appeared to have an embossed/edge detection effect applied. The contours/edges are highlighted in the resulting image. Ignore the green images, that was when SCNR was disabled. Everytime I switched to JPEG mode, it resulted in blue images. Zoomed in on the effect. |
|
Would you be able to capture a DNG image that shows the error by adding a |
|
I've been running the following script to see if I can capture this problem. So far no luck. Could somebody try it out and see if they can capture a failure case and share the DNG/JPG and log file generated? #!/bin/bash
while true; do
ts=$(date "+%Y%m%d%H%M%S")
LIBCAMERA_LOG_LEVELS=IPARPI:0,RPi*:0 rpicam-still --output "image-$ts.jpg" --timeout 1 --nopreview --width 4056 --height 3040 --shutter 10s --analoggain 8 --quality 100 -r 2>&1 | tee log_$ts.txt
sleep 10
done |
|
When I disable AWB with I do NOT see the blue bias with AWB enabled nor do I see it when only RAW (no jpeg output) is enabled. Here is my command: |
|
I've used the above script with over 15000 captures over the weekend with a Pi 5 + IMX477. Sadly, not one of the captured images shows the problem. Has anybody been able to reproduce this with my script above? The AWB logs would be really interesting to inspect when it goes wrong. |
|
I think we've now managed to reporduce this problem and it is indeed an issue with AWB. The AWB does not have enough statistics to perform its search, and because the scene looks dark, it assumes indoor conditions, and sets the CT to 2500K. In fact, even the good images fail AWB - but fail in a different way, and defaults the CT to around 4000K which looks about correct. I suggest as a workaround switch to using manual WB gains. Does anyone have an estimate of the colour temperature of the night sky? It seems like for the "good" images, 4000K seems around correct, so adding It would be very useful for us to get some logs of the AWB algorithm in the "bad" case through. If somebody could run my above script overnight and get these logs, it would be incredibly useful to us. Thanks! |
|
@naushir,
Attached are 4 sets of files (dng, jpg, log). I use 2.5 for red and 2.0 for blue, but as we tell users, there is no "right" answer. It depends on their sky conditions and the dome they use to cover their camera. |
|
@EricClaeys thanks for your results. The summary of our findings is that the AWB sometimes cannot rely on the statistics zones generated from the image as the signal values may be very low.
We will consider adding a bias term to the AWB so in the latter case, the results will tend towards a user provided CT value. You can set this for your specific needs and this can give you a reasonable image in the failure case. |
|
As a test, could you add |
|
@nashir, I can upload some images in a few days if needed. |
|
The However given your use case for Allsky, I feel that perhaps manual white balance be the most robust solution. |
|
Just pitching in. I did a fresh install of Allsky and PI OS 2 weeks ago, although the rpicam build are not that new: rpicam-apps build: 49344f2 17-06-2024 (12:09:07) I use manual WB and I get the blue images :/ Do we expect the issue to be fixed on newer build of rpicam? |
|
The next update of the libcamera packages will have the following commit to address this: raspberrypi/libcamera@ea8fd63 Note that you will need to update your tuning files to provide a fallback colour temperature when the search fails to avoid the blue tint. |
What gains are you using here? Could you try targeting something like 4500K? |
Just to make sure I undestand, if I use manual WB, there should be no WB "search" and the colour temperature should not jump around as I observe? At least this is what I am used to from my mirrorless :) However this is not consistent with what I observe I guess. I have observed the blue images both for a fixed gain of 8 and autogain settings, examples are gain of 4.86, 4.33. Where can I read more about the camera tuning file, so I can aim for a color temp of 4500 K? |
|
You can read about the tuning file here. However, it doesn't tell you about specific values for specific sensors. You might do better do look at the tuning file and find the That line is telling your that the normalised red and blue values at 4660K are 0.3529 and 0.68. You should use the reciprocals of those values for the red and blue gains. Finally, when you have a DNG file, check the "As shot neutral" values in the metadata. It should be those red and blue value given above. If not, then something must have changed them. |
|
Also note that 4500K is just an educated guess on my part. If possible you should measure the actual conditions in your environment and chose an appropriate fixed colour temperature. |
Hallo all,
As discribed in the AllSky Github issues, some users experience a problem with the Rpi HQ camera.
Mostly at night, some pictures are “too blue”.
Maintainer @EricClaeys suggested to open an issue here because he thinks it is a libcamera software issue.
I’m running a fully updated Bookworm on the RPI5 with the Raspberry HQ Camera.
Attached you can see a timeline of the images taken with a 60 seconds interval.
Could this be a libcamera software issue?
Regards
The text was updated successfully, but these errors were encountered: