Is this an accurate algorithm for creating a population of monkey heights? Think of a crowded sidewalk in New York City. Pick any person off the street, and it may appear that their height is random. Nevertheless, it’s not the kind of random that random() produces by default. People’s heights aren’t uniformly distributed; there are many more people of about average height than there are very tall or very short ones. To accurately reflect this population, random heights close to the mean (another word for average) should be more likely to be chosen, while outlying heights (very short or very tall) should be rarer.
That’s exactly how a normal distribution (sometimes called a Gaussian distribution, after mathematician Carl Friedrich Gauss) works. A graph of this distribution is informally known as a bell curve. The curve is generated by a mathematical function that defines the probability of any given value occurring as a function of the mean (often written as μ, the Greek letter mu) and standard deviation (σ, the Greek letter sigma).
-
In the case of height values from 200 to 300, you probably have an intuitive sense of the mean (average) as 250. However, what if I were to say that the standard deviation is 3? Or 15? What does this mean for the numbers? The graphs depicted in Figure 0.2 should give you a hint. On the left is a distribution with a very low standard deviation, with the majority of the values piling up around the mean (they don’t deviate much from the standard). The version on the right has a higher standard deviation, so the values are more evenly spread out from the average (they deviate more).
@@ -309,6 +308,7 @@
A Normal Distribution of Random
Figure 0.2: Two example bell curves of a normal distribution, with a low (left) and high (right) standard deviation
+
In the case of height values from 200 to 300, you probably have an intuitive sense of the mean (average) as 250. However, what if I were to say that the standard deviation is 3? Or 15? What does this mean for the numbers? The graphs depicted in Figure 0.2 should give you a hint. On the left is a distribution with a very low standard deviation, with the majority of the values piling up around the mean (they don’t deviate much from the standard). The version on the right has a higher standard deviation, so the values are more evenly spread out from the average (they deviate more).
The numbers work out as follows: given a population, 68 percent of its members will have values in the range of one standard deviation from the mean, 95 percent within two standard deviations, and 99.7 percent within three standard deviations. Given a standard deviation of 5 pixels, only 0.3 percent of the monkey heights will be less than 235 pixels (three standard deviations below the mean of 250) or greater than 265 pixels (three standard deviations above the mean of 250). Meanwhile, 68 percent of the monkey heights will be from 245 to 255 pixels.
Here’s a sketch that calculates the magnitude of the vector between the mouse and the center of the canvas, and visualizes it as a rectangle drawn across the top of the window.
Example 2.3: Gravity Scaled by MassThe objects now fall at the same rate. I’m still basically making up the gravity force by arbitrarily setting it to 0.1, but by scaling the force according to the object’s mass, I’m making it up in a way that’s a little truer to Earth’s actual force of gravitational attraction. Meanwhile, because the strength of the wind force is independent of mass, the smaller circle still accelerates to the right more quickly when the mouse is pressed. (The online code for this example also includes a solution to Exercise 2.4, with the addition of a radius variable in the Mover class.)
Modeling a Force
Making up forces will actually get you quite far—after all, I just made up a pretty good approximation of Earth’s gravity. Ultimately, the world of p5.js is an orchestra of pixels, and you’re the conductor, so whatever you deem appropriate to be a force, well by golly, that’s the force it should be! Nevertheless, there may come a time when you find yourself wondering, “But how does it all really work?” That’s when modeling forces, instead of just making them up, enters the picture.
-
Open up any high school physics textbook and you’ll find diagrams and formulas describing various forces—gravity, electromagnetism, friction, tension, elasticity, and more. For the rest of this chapter, I’m going to consider three forces—friction, drag, and gravitational attraction—and show how to model them with p5.js. The point I’d like to make here is not that these are fundamental forces that you always need in your simulations. Rather, I want to demonstrate these forces as case studies for the following process:
-
-
Understanding the concept behind a force
-
Deconstructing the force’s formula into two parts:
-
-
How do you compute the force’s direction?
-
How do you compute the force’s magnitude?
-
-
-
Translating that formula into p5.js code that calculates a vector to be passed through a Mover object’s applyForce() method
-
-
If you can follow these steps with the example forces I’ll provide here, then hopefully when you find yourself googling atomic nuclei weak nuclear force at 3 AM, you’ll have the skills to take what you find and adapt it for p5.js.
Parsing Formulas
In a moment, I’m going to write out the formula for friction. This won’t be the first time you’ve seen a formula in this book; I just finished up the discussion of Newton’s second law, \vec{F} = M \times \vec{A} (or force equals mass times acceleration). You hopefully didn’t spend a lot of time worrying about that formula, because it’s just a few characters and symbols. Nevertheless, it’s a scary world out there. Just take a look at the equation for a normal distribution, which I covered (without presenting the formula) in “A Normal Distribution of Random Numbers”:
@@ -391,6 +379,18 @@
Parsing Formulas
When symbols are placed next to each other, this typically means multiply them. The right side of the friction formula has four elements: –, μ, N, and \hat{v}. They should be multiplied together, reading the formula as \vec{f} = -1 \times \mu \times N \times \hat{v}.
+
Open up any high school physics textbook and you’ll find diagrams and formulas describing various forces—gravity, electromagnetism, friction, tension, elasticity, and more. For the rest of this chapter, I’m going to consider three forces—friction, drag, and gravitational attraction—and show how to model them with p5.js. The point I’d like to make here is not that these are fundamental forces that you always need in your simulations. Rather, I want to demonstrate these forces as case studies for the following process:
+
+
Understanding the concept behind a force
+
Deconstructing the force’s formula into two parts:
+
+
How do you compute the force’s direction?
+
How do you compute the force’s magnitude?
+
+
+
Translating that formula into p5.js code that calculates a vector to be passed through a Mover object’s applyForce() method
+
+
If you can follow these steps with the example forces I’ll provide here, then hopefully when you find yourself googling atomic nuclei weak nuclear force at 3 AM, you’ll have the skills to take what you find and adapt it for p5.js.
Friction
Let’s begin with friction and follow the preceding steps. Whenever two surfaces come into contact, they experience friction. Friction is a dissipative force, meaning it causes the kinetic energy of an object to be converted into another form, giving the impression of loss, or dissipation.
Let’s say you’re driving a car. When you press your foot on the brake pedal, the car’s brakes use friction to slow the motion of the tires. Kinetic energy (motion) is converted into thermal energy (heat). A complete model of friction would include separate cases for static friction (a body at rest against a surface) and kinetic friction (a body in motion against a surface), but for simplicity here, I’m going to work through only the kinetic case. Figure 2.3 shows the formula for friction.
I’ll start with the basics of working with angles in p5.js, then cover several aspects of trigonometry. In the end, I’ll connect trigonometry with what you learned about forces in Chapter 2. This chapter’s content will pave the way for more sophisticated examples that require trig later in this book.
Angles
Before going any further, I need to make sure you understand how the concept of an angle fits into creative coding in p5.js. If you have experience with p5.js, you’ve undoubtedly encountered this issue while using the rotate() function to rotate and spin objects.
-
You’re most likely to be familiar with the concept of an angle as measured in degrees (see Figure 3.1). A full rotation goes from 0 to 360 degrees, and 90 degrees (a right angle) is one-fourth of 360, shown in Figure 3.1 as two perpendicular lines.
Figure 3.1: Angles measured in degrees
+
You’re most likely to be familiar with the concept of an angle as measured in degrees (see Figure 3.1). A full rotation goes from 0 to 360 degrees, and 90 degrees (a right angle) is one-fourth of 360, shown in Figure 3.1 as two perpendicular lines.
@@ -900,13 +900,13 @@
The Pendulum
You might have noticed that in Example 3.10’s spring code, I never once used sine or cosine. Before you write off all this trigonometry stuff as a tangent, however, allow me to show an example of how it all fits together. Imagine a bob hanging from an anchor connected by a spring with a fully rigid connection that can be neither compressed nor extended. This idealized scenario describes a pendulum and provides an excellent opportunity to practice combining all that you’ve learned about forces and trigonometry.
A pendulum is a bob suspended by an arm from a pivot (previously called the anchor in the spring). When the pendulum is at rest, it hangs straight down, as in Figure 3.18. If you lift up the pendulum at an angle from its resting state and then release it, however, it starts to swing back and forth, tracing the shape of an arc. A real-world pendulum would live in a 3D space, but I’m going to look at a simpler scenario: a pendulum in the 2D space of a p5.js canvas. Figure 3.19 shows the pendulum in a nonresting position and adds the forces at play: gravity and tension.
When the pendulum swings, its arm and bob are essentially rotating around the fixed point of the pivot. If no arm connected the bob and the pivot, the bob would simply fall to the ground under the influence of gravity. Obviously, that isn’t what happens. Instead, the fixed length of the arm creates the second force—tension. However, I’m not going to work with this scenario according to these forces, at least not in the way I approached the spring scenario.
-
Instead of using linear acceleration and velocity, I’m going to describe the motion of the pendulum in terms of angular acceleration and angular velocity, which refer to the change of the arm’s angle \theta relative to the pendulum’s resting position. I should first warn you, especially if you’re a seasoned physicist, that I’m going to conveniently ignore several important concepts here: conservation of energy, momentum, centripetal force, and more. This isn’t intended to be a comprehensive description of pendulum physics. My goal is to offer you an opportunity to practice your new skills in trigonometry and further explore the relationship between forces and angles through a concrete example.
Figure 3.19: A pendulum showing \theta as the angle relative to its resting position
+
Instead of using linear acceleration and velocity, I’m going to describe the motion of the pendulum in terms of angular acceleration and angular velocity, which refer to the change of the arm’s angle \theta relative to the pendulum’s resting position. I should first warn you, especially if you’re a seasoned physicist, that I’m going to conveniently ignore several important concepts here: conservation of energy, momentum, centripetal force, and more. This isn’t intended to be a comprehensive description of pendulum physics. My goal is to offer you an opportunity to practice your new skills in trigonometry and further explore the relationship between forces and angles through a concrete example.
To calculate the pendulum’s angular acceleration, I’m going to use Newton’s second law of motion but with a little trigonometric twist. Take a look at Figure 3.19 and tilt your head so that the pendulum’s arm becomes the vertical axis. The force of gravity suddenly points askew, a little to the left—it’s at an angle with respect to your tilted head. If this is starting to hurt your neck, don’t worry. I’ll redraw the tilted figure and relabel the forces F_g for gravity and T for tension (Figure 3.20, left).
This is about as simple as a particle can get. From here, I could take the particle in several directions. I could add the applyForce() method to affect the particle’s behavior (I’ll do precisely this in a future example). I could also add variables to describe color and shape, or load a p5.Image to draw the particle in a more interesting way. For now, however, I’ll focus on adding just one additional detail: life span.
Some particle systems involve an emitter that serves as the source of the particles. The emitter controls the initial settings for the particles: position, velocity, and more. It might emit a single burst of particles, a continuous stream of particles, or some variation thereof. The new feature here is that particles born at the emitter can’t live forever. If they did, the p5.js sketch would eventually grind to a halt as the particles add up to an unwieldy number over time. As new particles are born, old particles need to be removed, creating the illusion of an infinite stream of particles without hurting the performance of the sketch.
There are many ways to decide when a particle is ready to be removed. For example, it could “die” when it comes into contact with another object or when it leaves the frame of the canvas. For now, I’ll choose to give particles a lifespan variable that acts like a timer. It will start at 255 and count down to 0 as the sketch progresses, at which point the particle will be considered dead. Here’s the added code in the Particle class:
With lifespan ranging from 255 to 0, it can conveniently double as the alpha transparency for the circle representing the particle. This way, when the particle is dead, it will have literally faded away.
With the addition of the lifespan property, I’ll need one more method, one that can be queried (for a true or false answer) to determine whether the particle is alive or dead. This will come in handy when I write a separate class to manage the list of particles. Writing this method is pretty easy: I just need to check whether the value of lifespan is less than 0. If it is, return true; otherwise, return false:
Figure 7.27: Example scenarios for death and birth in the Game of Life
With the elementary CA, I visualized many generations at once, stacked as rows in a 2D grid. With the Game of Life, however, the CA is in two dimensions. I could try to create an elaborate 3D visualization of the results and stack all the generations in a cube structure (and in fact, you might want to try this as an exercise), but a more typical way to visualize the Game of Life is to treat each generation as a single frame in an animation. This way, instead of viewing all the generations at once, you see them one at a time, and the result resembles rapidly developing bacteria in a petri dish.
-
One of the exciting aspects of the Game of Life is that some known initial patterns yield intriguing results. For example, the patterns shown in Figure 7.28 remain static and never change.
Figure 7.28: Initial configurations of cells that remain stable
+
One of the exciting aspects of the Game of Life is that some known initial patterns yield intriguing results. For example, the patterns shown in Figure 7.28 remain static and never change.
The patterns in Figure 7.29 oscillate back and forth between two states.
The factorial() function calls itself within its own definition. It may look a bit odd at first, but it works, as long as a stopping condition exists (in this case, n <= 1) so the function doesn’t get stuck calling itself forever. (I’m using <= instead of === as a safeguard against infinite recursion, but I should probably include additional error checking to manage noninteger or negative inputs to be more mathematically accurate.)
-
Figure 8.7 illustrates the steps that unfold when factorial(4) is called.
+
The factorial() function calls itself within its own definition. It may look a bit odd at first, but it works, as long as a stopping condition exists (in this case, n <= 1) so the function doesn’t get stuck calling itself forever. (I’m using <= instead of === as a safeguard against infinite recursion, but I should probably include additional error checking to manage noninteger or negative inputs to be more mathematically accurate.) Figure 8.7 illustrates the steps that unfold when factorial(4) is called.
Figure 8.7: Visualizing the process of calling the recursive factorial() function
diff --git a/content/10_nn.html b/content/10_nn.html
index e8703f8e..61693b32 100644
--- a/content/10_nn.html
+++ b/content/10_nn.html
@@ -518,12 +518,11 @@
Putting the “Network” in Neur
Figure 10.10: Data points that are linearly separable (left) and data points that are nonlinearly separable, as a curve is required to separate the points (right)
Now imagine you’re classifying plants according to soil acidity (x-axis) and temperature (y-axis). Some plants might thrive in acidic soils but only within a narrow temperature range, while other plants prefer less acidic soils but tolerate a broader range of temperatures. A more complex relationship exists between the two variables, so a straight line can’t be drawn to separate the two categories of plants, acidophilic and alkaliphilic (see Figure 10.10, right). A lone perceptron can’t handle this type of nonlinearly separable problem. (Caveat here: I’m making up these scenarios. If you happen to be a botanist, please let me know if I’m anywhere close to reality.)
-
One of the simplest examples of a nonlinearly separable problem is XOR (exclusive or). This is a logical operator, similar to the more familiar AND and OR. For A AND B to be true, both A and B must be true. With OR, either A or B (or both) can be true. These are both linearly separable problems. The truth tables in Figure 10.11 show their solution space. Each true or false value in the table shows the output for a particular combination of true or false inputs.
Figure 10.11: Truth tables for the AND and OR logical operators. The true and false outputs can be separated by a line.
-
See how you can draw a straight line to separate the true outputs from the false ones?
+
One of the simplest examples of a nonlinearly separable problem is XOR (exclusive or). This is a logical operator, similar to the more familiar AND and OR. For A AND B to be true, both A and B must be true. With OR, either A or B (or both) can be true. These are both linearly separable problems. The truth tables in Figure 10.11 show their solution space. Each true or false value in the table shows the output for a particular combination of true or false inputs. See how you can draw a straight line to separate the true outputs from the false ones?
The XOR operator is the equivalent of (OR) AND (NOT AND). In other words, A XOR B evaluates to true only if one of the inputs is true. If both inputs are false or both are true, the output is false. To illustrate, let’s say you’re having pizza for dinner. You love pineapple on pizza, and you love mushrooms on pizza, but put them together, and yech! And plain pizza, that’s no good either!
Now that I’ve finished the think() method, the real challenge can begin: teaching the bird to win the game by consistently flapping its wings at the right moment. This is where the GA comes back into the picture. Recalling the discussion from Chapter 9, three key principles underpin Darwinian evolution: variation, selection, and heredity. I’ll revisit each of these principles in turn as I implement the steps of the GA in this new context of neural networks.
Variation: A Flock of Flappy Birds
A single bird with a randomly initialized neural network isn’t likely to have any success at all. That lone bird will most likely jump incessantly and fly way off-screen, or sit perched at the bottom of the canvas awaiting collision after collision with the pipes. This erratic and nonsensical behavior is a reminder: a randomly initialized neural network lacks any knowledge or experience. The bird is essentially making wild guesses for its actions, so success is going to be rare.
-
This is where the first key principle of GAs comes in: variation. The hope is that by introducing as many different neural network configurations as possible, a few might perform slightly better than the rest. The first step toward variation is to add an array of many birds (Figure 11.4).
Figure 11.4: A population of birds, each with unique neural networks, navigating through the pipes in the neuroevolution process
+
This is where the first key principle of GAs comes in: variation. The hope is that by introducing as many different neural network configurations as possible, a few might perform slightly better than the rest. The first step toward variation is to add an array of many birds (Figure 11.4).
// Population size
let populationSize = 200;
// Array of birds
@@ -900,13 +900,13 @@ 
diff --git a/content/04_particles.html b/content/04_particles.html
index e84ec303..458e2946 100644
--- a/content/04_particles.html
+++ b/content/04_particles.html
@@ -66,7 +66,8 @@ 
diff --git a/content/08_fractals.html b/content/08_fractals.html
index 27153ed0..b9d17fb1 100644
--- a/content/08_fractals.html
+++ b/content/08_fractals.html
@@ -115,8 +115,7 @@ 
diff --git a/content/11_nn_ga.html b/content/11_nn_ga.html
index cb1a05ff..ae5f08a4 100644
--- a/content/11_nn_ga.html
+++ b/content/11_nn_ga.html
@@ -278,11 +278,11 @@ 