Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

endoc-284-update part1 Changes #1563

Merged
merged 3 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
240 changes: 60 additions & 180 deletions assets/images/common/audio-module.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
241 changes: 60 additions & 181 deletions assets/images/common/video-module.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion broadcast-streaming/overview/core-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: 'Core concepts'
sidebar_position: 2
type: docs
platform_selector: false
description: >
Ideas that are central to developing with Agora.
---
Expand Down
1 change: 0 additions & 1 deletion interactive-live-streaming/overview/core-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: 'Core concepts'
sidebar_position: 2
type: docs
platform_selector: false
description: >
Ideas that are central to developing with Agora.
---
Expand Down
2 changes: 1 addition & 1 deletion shared/common/_glossary.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ In real-time audio and video communication, delay refers to the time elapsed fro

### Dual-stream mode

In the dual-stream mode, the SDK transmits a high-quality and a low-quality video stream from the sender.
In the dual-stream mode, the <Vg k="VSDK" /> simultaneously transmits a higher-resolution video stream along with an additional low-resolution, low bitrate video stream.

## F

Expand Down
20 changes: 20 additions & 0 deletions shared/common/core-concepts/audio-video-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ The following diagram shows the basic workflow of using the <Vg k="COMPANY" /> S

<Vg k="COMPANY" /> uses the following basic concepts:

<PlatformWrapper platform="web">
### Track

A track contains specific audio or video information and consists of three parts: input source, filter, and output end. According to the different functions in the RTC process, tracks can be further classified into uplink tracks (the sending end) and downlink tracks (the receiving end).
Expand All @@ -26,13 +27,23 @@ Performs a series of processing operations on audio and video, including pre-pro
#### Output

Located at the end of the track, such as an encoder, or a renderer.
</PlatformWrapper>

<PlatformWrapper notAllowed="web">
### Audio module

In audio interaction, the main functions of the audio module are as shown in the figure below:

![An audio module](/images/common/audio-module.svg)

After you call `registerAudioFrameObserver`, you can obtain the raw audio data at the following observation points in the audio transmission process:
1. Obtain the raw audio data of ear monitoring through the `onEarMonitoringAudioFrame` callback.
2. Obtain the captured raw audio data through the `onRecordAudioFrame` callback.
3. Obtain the raw audio playback data of each individual stream through the `onPlaybackAudioFrameBeforeMixing` callback.
4. Obtain the raw audio playback data of all mixed streams through the `onPlaybackAudioFrame` callback.
5. Obtain the raw audio data after mixing the captured and playback audio through the `onMixedAudioFrame` callback.
(5) `onMixedAudioFrame` = (2) `onRecordAudioFrame` + (4) `onPlaybackAudioFrame`

#### Audio routing

The audio output device used by the app when playing audio. Common audio routes include wired headphones, earpieces, speakers, Bluetooth headphones, and others.
Expand All @@ -49,8 +60,17 @@ The following diagram shows the main functions of the video module in video inte

![Video module functions](/images/common/video-module.svg)

The figure shows the following observation points:

1. `POSITION_POST_CAPTURER_ORIGIN`.
2. `POSITION_POST_CAPTURER`, corresponds to the `onCaptureVideoFrame` callback.
3. `POSITION_PRE_ENCODER`, corresponds to the `onPreEncodeVideoFrame` callback.
4. `POSITION_PRE_RENDERER`, corresponds to the `onRenderVideoFrame` callback.

The APIs used by the video module are as follows:

- Enable local video collection: `enableLocalVideo`
- Local preview: `setupLocalVideo` → `startPreview`
- Video rendering shows: `setupRemoteVideo`

</PlatformWrapper>
2 changes: 1 addition & 1 deletion shared/video-sdk/develop/stream-fallback/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ When the stream fallback option is enabled, the system intelligently switches be

## Implement media stream fallback

This section explains the logic to implement the media stream callback feature:
This section explains the logic to implement the media stream fallback feature:

<ProjectImplement/>

Expand Down
2 changes: 2 additions & 0 deletions shared/video-sdk/develop/stream-raw-audio/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ Ensure that you have implemented the [SDK quickstart](../get-started/get-started
## Reference
This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

* [Audio module](../overview/core-concepts?platform=android#audio-module)

<Reference/>
</PlatformWrapper>

Expand Down
1 change: 0 additions & 1 deletion video-calling/overview/core-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: 'Core concepts'
sidebar_position: 2
type: docs
platform_selector: false
description: >
Ideas that are central to developing with Agora.
---
Expand Down
1 change: 0 additions & 1 deletion voice-calling/overview/core-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: 'Core concepts'
sidebar_position: 2
type: docs
platform_selector: false
description: >
Ideas that are central to developing with Agora.
---
Expand Down
Loading