detect "Dissolve" transitions and "Wipe" transitions

[prominentdetail]

Would be nice to have detection for Dissolve and Wipe style transitions.

The Dissolve is similar to a fade, but instead of fading to a solid color, it fades into another scene. In addition to detecting a Dissolve, a parameter that controls how much of the Dissolve to clip out would be good too. This way you could completely remove the Dissolve if you want. 0% would be split in the middle of the Dissolve. 100% would clip out the entire transition, from the beginning of the Dissolve to the end of the Dissolve.

The Wipe is a transition where a new scene translates across the screen, overlaying the previous scene. Again, it would also be good to have a parameter to control how much of the Wipe to include or exclude. 0% would be split in the middle of the Wipe. 100% would clip out the entire transition, from the beginning of the Wipe to the end of the Wipe.

[Breakthrough]

I wonder if image contrast could be useful here, or using histograms to detect sudden changes in the image content. Edge detection might also be able to cover some of these cases. If anyone can share any research papers on this topic for any commonly used methods, if any, that would be very helpful.

[acgourley]

I would find this super useful! I found this project when attempting to solve this very problem. Unfortunately, it remains unsolved. I am encouraged to see "DissolveDetector" in the code comments as a future detector. Thanks for your all your work on this @Breakthrough - if this feature does develop I'll be the first to try it out.

I saw in 0.61 you updated the adaptive detection to use edge detail as one parameter - but when I dumped that into a CSV file I only noticed a small change in the value at my video's 0.5s crossfades, which didn't seem like a reliable signal.

[Breakthrough]

Glad you're enjoying the project. Unfortunately I don't have many cycles to tackle dissolves right now, but would be happy to accept any PRs or prototypes that seem to work. Also many thanks for the initial feedback on the edge detection. Like you said it's probably not the right choice for dissolves/wipes since the edge information doesn't change much frame-by-frame.

Initially I was thinking it would be useful for fade-outs/fade-ins, since at some point the thresholding will fail and the edges will suddenly change. For dissolves/wipes though, I've not given much thought about how to address that yet. Happy to hear any ideas people have though, or any prior art / papers to consider. (And again, prototypes/PRs are welcome too!)

[dragonflyuk]

I had a similiar problem, with a cross-dissolve, not been detected, I came up with a very simple sample the frames every n seconds. Which In my situation, worked a treat, however I can see it not working on fast moving scenes etc. Here is some code I wrote as a test, if it helps anyone else.

def detect_scene_changes(video_filename, n, threshold):
    cap = cv2.VideoCapture(video_filename)
    frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    fps = int(cap.get(cv2.CAP_PROP_FPS))

    interval = n * fps  # Number of frames to skip for the given time interval
    frame_number = 0
    result = []

    start_frame = 0
    end_frame = None

    while frame_number < frame_count:
        cap.set(cv2.CAP_PROP_POS_FRAMES, frame_number)
        ret, frame = cap.read()
        if not ret:
            break

        if frame_number % interval == 0:
            if frame_number == 0:
                prev_frame = frame
            else:
                # Calculate the percentage difference between the frames
                difference = cv2.absdiff(frame, prev_frame)
                percentage_diff = (difference.sum() / 255) / (frame.size / 3) * 100

                if percentage_diff > threshold:
                    if start_frame is None:
                        start_frame = frame_number
                    else:
                        end_frame = frame_number
                        if (end_frame == (start_frame + interval)):
                            result[-1] = (result[-1][0],end_frame / fps)
                        else:
                            result.append((start_frame / fps, end_frame / fps))
                        start_frame = end_frame

                prev_frame = frame

        frame_number += 1

    # Check if the last scene extends until the end of the video
    if start_frame is not None:
        result.append((start_frame / fps, frame_count / fps))

    cap.release()
    return result