sync audio (#1067)

* Drop late audio frames to keep sync #388 * show running architecture * Don't log failed tutk if stream is down #990 * Use valid FPS for sleep #388 * Refactor * Adjust sleep_interval #388 * Increase sleep time between frames #388 * Set larger buf size #388 * add SLEEP_INTERVAL_FPS #388 * Adjust sleep interval #388 * use genpts #388 * avoid conflicting names with errno module * refactor _audio_frame_slow * LOW_LATENCY mode * show github SHA on dev build * substream support and more refactoring #388 * div tag for jittery video in Firefox #1025 * Re-encoding audio for WebRTC/MTX * reduce sleep time for audio thread #388 * Target firefox for jittery video fix in css #1025 * Use K10050GetVideoParam for FW 4.50.4.x #1070 * Use K10006 for newer doorbell #742 and refactor * Reduce audio pipe flushing #388 * show gap when audio out of sync #388 * don't include ARCH in version * Update iotc.py * reset frame_ts on clock sync * update auth api * Restructure and cleanup * remove unneeded files * update path * Forget alarm/siren state #953 #1051 * use addon_config for Home Assistant * Additional refactoring to auth api * don't skip keyframes #388 * Update ffmpeg.py * delay audio when ahead #388 * format iotc logging so we know what cam is late * drop late video frame and speed up audio #388 * Add Floodlight V2 * Set default sample rate for all cams * Delay audio by 1 second if ahead of video #388 * tweak ffmpeg buffer #388 * Retain MQTT Discovery Message #920 * Update change log Special thanks to @carlosnasillo!
mrlt8 · Jan 11, 2024 · bbacb0f · bbacb0f
1 parent 2f36455
commit bbacb0f
Show file tree

Hide file tree

Showing 32 changed files with 1,120 additions and 1,097 deletions.
diff --git a/.github/workflows/docker-image.yml b/.github/workflows/docker-image.yml
@@ -93,19 +93,21 @@ jobs:
           TAG_NAME=${GITHUB_REF##*/v}
           if [[ $TAG_NAME =~ ^[0-9]+\.[0-9]+\.[0-9]+(-.*)?$ ]]; then
             sed -i "s/^VERSION=.*/VERSION=${TAG_NAME}/" ./app/.env
-            jq --arg VERSION "${TAG_NAME}" '.version = $VERSION' ./app/config.json > updated.json
-            mv updated.json ./app/config.json
+            jq --arg VERSION "${TAG_NAME}" '.version = $VERSION' ./home_assistant/config.json > updated.json
+            mv updated.json ./home_assistant/config.json
           fi
 
       - name: Build and push a Docker image
         uses: docker/build-push-action@v5
         with:
           builder: ${{ steps.buildx.outputs.name }}
-          context: ./app/
           push: ${{ github.event_name != 'pull_request' }}
-          file: ./app/Dockerfile.${{ matrix.dockerfile }}
+          file: ./docker/Dockerfile.${{ matrix.dockerfile }}
           platforms: ${{ steps.image_type.outputs.platforms }}
-          build-args: BUILD=${{ steps.meta.outputs.VERSION }}
+          build-args: |
+            BUILD=${{ steps.meta.outputs.VERSION }}
+            BUILD_DATE=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.created'] }}
+            GITHUB_SHA=${{ github.sha }}
           labels: |
             ${{ steps.meta.outputs.labels }}
             io.hass.name=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.title'] }}
@@ -130,13 +132,13 @@ jobs:
           TAG_NAME=${GITHUB_REF##*/v}
           if [[ $TAG_NAME =~ ^[0-9]+\.[0-9]+\.[0-9]+(-.*)?$ ]]; then
             sed -i "s/^VERSION=.*/VERSION=${TAG_NAME}/" ./app/.env
-            jq --arg VERSION "${TAG_NAME}" '.version = $VERSION' ./app/config.json > updated.json
-            mv updated.json ./app/config.json
+            jq --arg VERSION "${TAG_NAME}" '.version = $VERSION' ./home_assistant/config.json > updated.json
+            mv updated.json ./home_assistant/config.json
             echo "tag=${TAG_NAME}" >> $GITHUB_OUTPUT
           fi
       - name: Commit and push changes
         uses: stefanzweifel/git-auto-commit-action@v4
         with:
           branch: main
           commit_message: 'Bump Version to v${{ steps.version_bump.outputs.tag }}'
-          file_pattern: 'app/.env app/config.json'
+          file_pattern: 'app/.env home_assistant/config.json'
diff --git a/README.md b/README.md
@@ -53,15 +53,19 @@ You can then use the web interface at `http://localhost:5000` where localhost is
 
 See [basic usage](#basic-usage) for additional information or visit the [wiki page](https://github.com/mrlt8/docker-wyze-bridge/wiki/Home-Assistant) for additional information on using the bridge as a Home Assistant Add-on.
 
-## What's Changed in v2.6.0
+## What's Changed in v2.7.0
 
-* **NEW**: ARM 64-bit native library (#529 #604 #664 #871 #998 #1004)
-
-  The arm64 container now runs in 64-bit mode, addressing compatibility issues, particularly on Apple Silicon M1/M2/M3, when using the Home Assistant Add-on.
+* Audio sync - bridge will now try to make minor adjustments to try to keep the video and audio in sync Thanks @carlosnasillo and everyone who helped with testing! (#388).
+* Refactor for compatibility with Scrypted. Thanks @koush (#1066)
+* Use K10050GetVideoParam for FW 4.50.4.x (#1070)
+* Fix jittery video in Firefox (#1025)
+* Retain MQTT Discovery Message Thanks @jhansche! (#920) 
 
-  Resolves issues on the Raspberry Pi 4/5 running the 64-bit version of Raspbian.
+Home Assistant:
 
-* **Update**: Python 3.11 -> Python 3.12
+* Now uses `addon_config` instead of `config` [Additional info](https://developers.home-assistant.io/blog/2023/11/06/public-addon-config/) 
+  * May need to cleanup old config manually.
+* Reset alarm/siren state (#953) (#1051)
 
 
 [View previous changes](https://github.com/mrlt8/docker-wyze-bridge/releases)
@@ -109,7 +113,7 @@ The container can be run on its own, in [Portainer](https://github.com/mrlt8/doc
 | Wyze Cam V3                   | WYZE_CAKP2JFUS | ✅                                                            | 4.36.11.x |
 | Wyze Cam V4 [2K]              | HL_CAM4        | ❓                                                            | 4.52.?    |
 | Wyze Cam Floodlight           | WYZE_CAKP2JFUS | ✅                                                            | 4.36.11.x |
-| Wyze Cam Floodlight V2        | HL_CFL2        | ❓                                                            | -         |
+| Wyze Cam Floodlight V2 [2k]   | HL_CFL2        | ❓                                                            | -         |
 | Wyze Cam V3 Pro [2K]          | HL_CAM3P       | ✅                                                            | 4.58.11.x |
 | Wyze Cam Pan                  | WYZECP1_JEF    | ✅                                                            | 4.10.9.x  |
 | Wyze Cam Pan v2               | HL_PAN2        | ✅                                                            | 4.49.11.x |
@@ -212,7 +216,7 @@ WebRTC should work automatically in Home Assistant mode, however, some additiona
 
 ## Advanced Options
 
-**WYZE_EMAIL** and **WYZE_PASSWORD** are the only two required environment variables.
+All environment variables are optional.
 
 * [Audio](https://github.com/mrlt8/docker-wyze-bridge/wiki/Camera-Audio)
 * [Bitrate and Resolution](https://github.com/mrlt8/docker-wyze-bridge/wiki/Camera-Bitrate-and-Resolution)

diff --git a/app/lib.amd64 → app/lib/lib.amd64 b/app/lib.amd64 → app/lib/lib.amd64
diff --git a/app/lib.arm → app/lib/lib.arm b/app/lib.arm → app/lib/lib.arm
diff --git a/app/lib.arm64 → app/lib/lib.arm64 b/app/lib.arm64 → app/lib/lib.arm64
diff --git a/app/static/site.css b/app/static/site.css
@@ -262,4 +262,15 @@ button:not(.offline):disabled+.age {
 nav.fs-mode,
 footer.fs-mode {
   display: none;
+}
+
+/* Fix for Jittery Video in Firefox #1025 */
+@-moz-document url-prefix() {
+  body::after {
+    width: 1px;
+    height: 1px;
+    position: fixed;
+    backdrop-filter: blur(0.01rem);
+    content: "";
+  }
 }
diff --git a/app/templates/base.html b/app/templates/base.html
@@ -37,4 +37,5 @@
     {% endblock %}
 </body>
 
+
 </html>
diff --git a/app/wyzebridge/bridge_utils.py b/app/wyzebridge/bridge_utils.py
@@ -68,8 +68,11 @@ def is_livestream(uri: str) -> bool:
 
 
 def is_fw11(fw_ver: Optional[str]) -> bool:
+    """
+    Check if newer firmware that needs to use K10050GetVideoParam
+    """
     with contextlib.suppress(IndexError, ValueError):
-        if fw_ver and (fw_ver.startswith("4.51") or fw_ver.startswith("4.52")):
+        if fw_ver and fw_ver.startswith(("4.51", "4.52", "4.50.4")):
             return True
         if fw_ver and int(fw_ver.split(".")[2]) > 10:
             return True

diff --git a/app/wyzebridge/config.py b/app/wyzebridge/config.py
@@ -1,4 +1,5 @@
 from os import environ, getenv, makedirs
+from platform import machine
 
 from dotenv import load_dotenv
 
@@ -8,10 +9,15 @@
 load_dotenv()
 load_dotenv("/.build_date")
 
-VERSION: str = getenv("VERSION", "DEV")
+VERSION: str = f'{getenv("VERSION", "DEV")}'
+ARCH = machine().upper()
 BUILD = env_bool("BUILD", "local")
 BUILD_DATE = env_bool("BUILD_DATE")
-BUILD_STR = "" if BUILD == VERSION else f"[{BUILD.upper()} BUILD] {BUILD_DATE}"
+GITHUB_SHA = env_bool("GITHUB_SHA")
+BUILD_STR = ARCH
+if BUILD != VERSION:
+    BUILD_STR += f" {BUILD.upper()} BUILD [{BUILD_DATE}] {GITHUB_SHA:.7}"
+
 HASS_TOKEN: str = getenv("SUPERVISOR_TOKEN", "")
 setup_hass(HASS_TOKEN)
 MQTT_DISCOVERY = env_bool("MQTT_DTOPIC")
@@ -46,9 +52,7 @@
 makedirs(IMG_PATH, exist_ok=True)
 
 
-DEPRECATED = {
-    "DEBUG_FFMPEG",
-}
+DEPRECATED = {"DEBUG_FFMPEG"}
 
 for env in DEPRECATED:
     if getenv(env):

diff --git a/app/wyzebridge/ffmpeg.py b/app/wyzebridge/ffmpeg.py
@@ -29,11 +29,11 @@ def get_ffmpeg_cmd(
     - list of str: complete ffmpeg command that is ready to run as subprocess.
     """
 
-    flags = "-fflags +flush_packets+nobuffer -flags +low_delay"
+    flags = "-fflags +flush_packets+nobuffer+genpts -flags +low_delay"
     livestream = get_livestream_cmd(uri)
     audio_in = "-f lavfi -i anullsrc=cl=mono" if livestream else ""
     audio_out = "aac"
-    thread_queue = "-thread_queue_size 1k -analyzeduration 32 -probesize 32"
+    thread_queue = "-thread_queue_size 8 -analyzeduration 32 -probesize 32"
     if audio and "codec" in audio:
         audio_in = f"{thread_queue} -f {audio['codec']} -ac 1 -ar {audio['rate']} -i /tmp/{uri}_audio.pipe"
         audio_out = audio["codec_out"] or "copy"
@@ -60,9 +60,7 @@ def get_ffmpeg_cmd(
         + re_encode_video(uri, is_vertical)
         + (["-map", "1:a", "-c:a", audio_out] if audio_in else [])
         + (a_options if audio and audio_out != "copy" else [])
-        + ["-fps_mode", "drop", "-async", "1", "-flush_packets", "1"]
-        + ["-muxdelay", "0"]
-        + ["-rtbufsize", "1", "-max_interleave_delta", "10"]
+        + ["-max_muxing_queue_size", "1", "-max_delay", "0.1", "-rtbufsize", "32"]
         + ["-f", "tee"]
         + [rtsp_ss + get_record_cmd(uri, audio_out, record) + livestream]
     )

diff --git a/app/wyzebridge/logging.py b/app/wyzebridge/logging.py
@@ -35,6 +35,7 @@ def format_logging(handler: logging.Handler, level: int, date_format: str = ""):
     else:
         target_logger = logging.getLogger("WyzeBridge")
         logging.getLogger("werkzeug").addHandler(handler)
+        logging.getLogger("wyzecam.iotc").addHandler(handler)
         logging.getLogger("py.warnings").addHandler(handler)
 
     date_format = "%X" if not date_format and level < 20 else date_format

diff --git a/app/wyzebridge/mqtt.py b/app/wyzebridge/mqtt.py
@@ -58,6 +58,7 @@ def publish_discovery(cam_uri: str, cam: WyzeCamera, stopped: bool = True) -> No
                 "sw_version": cam.firmware_ver,
                 "via_device": f"docker-wyze-bridge v{VERSION}",
             },
+            "retain": True,
         }
 
         # Clear out old/renamed entities
@@ -276,6 +277,7 @@ def get_entities(base_topic: str, pan_cam: bool = False, rtsp: bool = False) ->
                 "command_topic": f"{base_topic}alarm/set",
                 "payload_on": 1,
                 "payload_off": 2,
+                "optimistic": False,
                 "icon": "mdi:alarm-bell",
             },
         },

diff --git a/app/wyzebridge/wyze_api.py b/app/wyzebridge/wyze_api.py
@@ -73,11 +73,13 @@ def wrapper(self, *args: Any, **kwargs: Any):
 
 
 class WyzeCredentials:
-    __slots__ = "email", "password"
+    __slots__ = "email", "password", "key_id", "api_key"
 
     def __init__(self) -> None:
         self.email: str = getenv("WYZE_EMAIL", "").strip()
         self.password: str = getenv("WYZE_PASSWORD", "").strip()
+        self.key_id: str = getenv("API_ID", "").strip()
+        self.api_key: str = getenv("API_KEY", "").strip()
 
         if not self.is_set:
             logger.warning("[WARN] Credentials are NOT set")
@@ -93,9 +95,6 @@ def update(self, email: str, password: str) -> None:
     def same_email(self, email: str) -> bool:
         return self.email.lower() == email.lower() if self.is_set else True
 
-    def creds(self) -> tuple[str, str]:
-        return (self.email, self.password)
-
     def login_check(self):
         if self.is_set:
             return
@@ -132,45 +131,48 @@ def login(self, fresh_data: bool = False) -> Optional[WyzeCredential]:
             return
         self._last_pull = time()
 
-        self.token_auth()
-        if self.auth:
+        if self.token_auth():
             return self.auth
 
         self.creds.login_check()
         try:
-            self.auth = wyzecam.login(*self.creds.creds())
+            self.auth = wyzecam.login(
+                email=self.creds.email,
+                password=self.creds.password,
+                api_key=self.creds.api_key,
+                key_id=self.creds.key_id,
+            )
         except HTTPError as ex:
             logger.error(f"⚠️ {ex}")
-            if ex.response.status_code == 403:
-                logger.error(f"Your IP may be blocked from {ex.request.url}")
-            elif resp := ex.response.text:
-                logger.warning(resp)
+            if ex.response and ex.response.status_code == 403:
+                logger.error(f"[API] Your IP may be blocked from {ex.request.url}")
+            elif ex.response and ex.response.text:
+                logger.warning(f"[API] {ex.response.text}")
             sleep(15)
-        except ValueError as ex:
-            logger.error(f"[API] {ex}")
-        except RateLimitError as ex:
+        except (ValueError, RateLimitError, RequestException) as ex:
             logger.error(f"[API] {ex}")
-        except RequestException as ex:
-            logger.error(f"[API] ERROR: {ex}")
         else:
             if self.auth.mfa_options:
                 self._mfa_auth()
             return self.auth
 
-    def token_auth(self):
+    def token_auth(self) -> bool:
+        if len(token := env_bool("access_token", style="original")) > 150:
+            logger.info("⚠️ Using 'ACCESS_TOKEN' for authentication")
+            self.auth = WyzeCredential(access_token=token)
+
         if len(token := env_bool("refresh_token", style="original")) > 150:
             logger.info("⚠️ Using 'REFRESH_TOKEN' for authentication")
             self.auth = wyzecam.refresh_token(WyzeCredential(refresh_token=token))
 
-        if len(token := env_bool("access_token", style="original")) > 150:
-            logger.info("⚠️ Using 'ACCESS_TOKEN' for authentication")
-            self.auth = WyzeCredential(access_token=token)
+        return bool(self.auth)
 
     @cached
     @authenticated
     def get_user(self) -> Optional[WyzeAccount]:
         if self.user:
             return self.user
+
         self.user = wyzecam.get_user_info(self.auth)
         return self.user
 
@@ -263,7 +265,12 @@ def _mfa_auth(self):
 
             logger.info(f'🔑 Using {resp["verification_code"]} for authentication')
             try:
-                self.auth = wyzecam.login(*self.creds.creds(), self.auth.phone_id, resp)
+                self.auth = wyzecam.login(
+                    email=self.creds.email,
+                    password=self.creds.password,
+                    phone_id=self.auth.phone_id,
+                    mfa=resp,
+                )
                 if self.auth.access_token:
                     logger.info("✅ Verification code accepted!")
             except HTTPError as ex:
@@ -397,14 +404,13 @@ def valid_s3_url(url: Optional[str]) -> bool:
     if not url:
         return False
 
-    query_parameters = parse_qs(urlparse(url).query)
-    x_amz_date = query_parameters.get("X-Amz-Date", "0")
-    x_amz_expires = query_parameters.get("X-Amz-Expires", "0")
-
     try:
-        amz_date = datetime.strptime(x_amz_date[0], "%Y%m%dT%H%M%SZ")
-        return amz_date.timestamp() + int(x_amz_expires[0]) > time()
-    except (ValueError, TypeError):
+        query_parameters = parse_qs(urlparse(url).query)
+        x_amz_date = query_parameters["X-Amz-Date"][0]
+        x_amz_expires = query_parameters["X-Amz-Expires"][0]
+        amz_date = datetime.strptime(x_amz_date, "%Y%m%dT%H%M%SZ")
+        return amz_date.timestamp() + int(x_amz_expires) > time()
+    except (ValueError, TypeError, KeyError):
         return False
 
 

diff --git a/app/wyzebridge/wyze_control.py b/app/wyzebridge/wyze_control.py
@@ -12,6 +12,7 @@
 from wyzebridge.mqtt import MQTT_ENABLED, publish_messages
 from wyzebridge.wyze_commands import CMD_VALUES, GET_CMDS, GET_PAYLOAD, PARAMS, SET_CMDS
 from wyzecam import WyzeIOTCSession, WyzeIOTCSessionState, tutk_protocol
+from wyzecam.tutk.tutk import TutkError
 
 
 def cam_http_alive(ip: str) -> bool:
@@ -128,16 +129,14 @@ def boa_control(sess: WyzeIOTCSession, boa_cam: Optional[dict]):
         pull_last_image(boa_cam, "photo", True)
 
 
-def camera_control(
-    sess: WyzeIOTCSession, uri: str, camera_info: Queue, camera_cmd: Queue
-):
+def camera_control(sess: WyzeIOTCSession, camera_info: Queue, camera_cmd: Queue):
     """
     Listen for commands to control the camera.
 
     :param sess: WyzeIOTCSession used to communicate with the camera.
     :param uri: URI-safe name of the camera.
     """
-    boa = check_boa_enabled(sess, uri)
+    boa = check_boa_enabled(sess, sess.camera.name_uri)
 
     while sess.state == WyzeIOTCSessionState.AUTHENTICATION_SUCCEEDED:
         boa_control(sess, boa)
@@ -275,6 +274,9 @@ def send_tutk_msg(sess: WyzeIOTCSession, cmd: tuple | str, log: str = "info") ->
         return _response(resp, log=log)
     except tutk_protocol.TutkWyzeProtocolError as ex:
         return resp | _error_response(cmd, tutk_protocol.TutkWyzeProtocolError(ex))
+    except TutkError as ex:
+        connected = sess.should_stream()
+        return resp | _error_response(cmd, f"[{ex.code}] {ex.name}", connected)
     except Exception as ex:
         return resp | _error_response(cmd, ex)
 
@@ -293,8 +295,9 @@ def _response(response, res=None, params=None, log="info"):
     return response
 
 
-def _error_response(cmd, error):
-    logger.error(f"[CONTROL] ERROR - {error=}, {cmd=}")
+def _error_response(cmd, error, log=True):
+    if log:
+        logger.error(f"[CONTROL] ERROR - {error=}, {cmd=}")
     return {"status": "error", "response": str(error)}