Skip to content

Commit

Permalink
Merge pull request #28 from Yalantis/feature/156
Browse files Browse the repository at this point in the history
Updated readme file
  • Loading branch information
yala-kr authored Apr 15, 2022
2 parents 4bd116d + 7e81526 commit 765d024
Show file tree
Hide file tree
Showing 3 changed files with 137 additions and 4 deletions.
141 changes: 137 additions & 4 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,143 @@

Check this [project on dribbble] (https://dribbble.com/shots/2452050-Record-Audio-Sample)

#### [Read more about Horizon] (https://yalantis.com/blog/horizon-open-source-library-for-sound-visualization/)

<img src="blog_article_header.png" alt="example" style="width:720;height:400">

# Implementation

We decided to create our audio visualizer as the base for our audio projects. We wanted a solution that will work as the Android music visualizer. The result was this awesome equalizer concept that we called Horizon.

We implemented most of our sound analysis in C++. This language reduces the time required to fetch the spectrum of sounds. While most of our analysis code was written in C++, we still did minor calculations in Java to make our component easy to customize.

### How we draw a Bezier curve with Android Canvas

The equalizer consists of five waves. If you open the original .svg file in a vector image editor, you’ll see that the second wave, which corresponds to bass frequencies, is made of four cubic Bezier curves. A Bezier curve describes smooth curves mathematically. It’s possible to draw Bezier curves with Android Canvas. First, initialize paint and path. Then, add the Bezier curve to the path by calling the quadTo or cubitTo method:

```
path.reset();
path.moveTo(p0x, p0y);
path.quadTo(p1x, p1y, p2x, p2y);
path.moveTo(p0x, p0y);
path.close();
```

And finally, draw the path on the canvas:

```
canvas.drawPath(path, paint);
```

As you can see, drawing Bezier curves with Android Canvas is very easy, but performance is generally very poor.

### How to draw a cubic Bezier with OpenGL ES

OpenGL ES is very fast at drawing triangles, which means we need to come up with a way to split the shape we want into triangles. Since our wave is convex, we can approximate it by drawing many triangles, all of which have one vertex located at the center of the screen (0, 0).

Here’s the idea:

1. Split every Bezier curve into an even number of points nn.
2. Generate n−1n−1 triangles with vertices at (N1,N2,O), (N2,N3,O), …, (Nn−1,Nn,O).
3. Fill these triangles with color.

**Splitting the Bezier curve**

For each point on the Bezier curve, we are going to generate three attributes for three vertices. This is done with a simple method:

```
private float[] genTData() {
// 1---2
// | /
// 3
float[] tData = new float[Const.POINTS_PER_TRIANGLE * Const.T_DATA_SIZE * mBezierRenderer.numberOfPoints];
for (int i = 0; i < tData.length; i += Const.POINTS_PER_TRIANGLE) {
float t = (float) i / (float)tData.length;
float t1 = (float) (i + 3) / (float)tData.length;
tData[i] = t;
tData[i+1] = t1;
tData[i+2] = -1;
}
return tData;
}
```

Attributes of the first two vertices specify points on the curve. The attribute for the third vertex is always -1, which by our convention means that this vertex is located at (0,0)(0,0).

Next, we need to pass this data to a shader.

### Shader pipeline

We’ll use the following variables of the OpenGL Shading Language:

**Uniforms** (common for the entire wave):

* vec4 u_Color – Color of the wave
* float u_Amp – Sound level of the wave
* vec4 u_BzData – Start and end points of the Bezier curve
* vec4 u_BzDataCtrl – Two control points of the Bezier curve

**Attribute** (per individual vertex):

* float a_Tdata – interpolation coefficient tt (specifies point on the curve)

Now, given the start, end, and control points of a curve, as well as tt, we need to find the location of the point on the curve.

Let’s look at the formula for a cubic Bezier:

![Formula for a cubic Bezier](formula_cubic_Bezier.png)

It’s easy to translate this directly into GLSL:

```
vec2 b3_translation( in vec2 p0, in vec2 p1, in vec2 p2, in vec2 p3, in float t )
{
float tt = (1.0 - t) * (1.0 - t);
return tt * (1.0 - t) * p0 +
3.0 * t * tt * p1 +
3.0 * t * t * (1.0 - t) * p2 +
t * t * t * p3;
}
```

But we can do better. Let’s look at the geometric explanation of a cubic Bezier curve:

![Cubic_Bezier_curve](cubic_Bezier_curve.png)

With the help of GLSL’s mix function, we interpolate between points and almost program declaratively:

```
vec2 b3_mix( in vec2 p0, in vec2 p1,
in vec2 p2, in vec2 p3,
in float t )
{
vec2 q0 = mix(p0, p1, t);
vec2 q1 = mix(p1, p2, t);
vec2 q2 = mix(p2, p3, t);
vec2 r0 = mix(q0, q1, t);
vec2 r1 = mix(q1, q2, t);
return mix(r0, r1, t);
}
```

This alternative is much easier to read and, we think, is equivalent in terms of speed.

### Color blending

To tell OpenGL that we want screen-like blending, we need to enable GL_BLEND and specify the blend function in our onDrawFrame method before actually drawing the waves:

```
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFuncSeparate(
GLES20.GL_ONE, GLES20.GL_ONE_MINUS_SRC_COLOR,
GLES20.GL_ONE, GLES20.GL_ONE_MINUS_SRC_ALPHA
); // Screen blend mode
```

# Usage

*For a working implementation, please have a look at the Sample Project - sample*
Expand All @@ -20,14 +153,14 @@ Check this [project on dribbble] (https://dribbble.com/shots/2452050-Record-Audi

2. Initialize Horizon object with params regarding to your sound

````java
```java
mHorizon = new Horizon(glSurfaceView, getResources().getColor(R.color.background),
RECORDER_SAMPLE_RATE, RECORDER_CHANNELS, RECORDER_ENCODING_BIT);
```

3. To update Horizon call updateView method with chunk of sound data to proceed

````java
```java
byte[] buffer = new byte[bufferSize];
//here we put some sound data to the buffer
mHorizon.updateView(buffer);
Expand Down
Binary file added cubic_Bezier_curve.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added formula_cubic_Bezier.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 765d024

Please sign in to comment.