Particle tutorial audio spectrum sampler

From PopcornFX
Jump to navigation Jump to search
! IMPORTANT ! This is a PopcornFX v1 page. PopcornFX v2 documentation can be found here
For PK-Fx Editor version : 1.4.0 and above
Main page: Particle tutorials

In this tutorial, we'll see:

  • How to setup a sound player action to play a sound upon FX instantiation.
  • How to setup a sound "backdrop" in the editor.
  • How to use the spectrum sampler and control particles based on the audio spectrum.
Tuto4 preview1.jpg

Final result

Sound player

The spectrum sampler samples the audio spectrum of any audio channel in your game/application.
Therefore, in this tutorial, before sampling any audio, we'll need to make the realtime editor play some audio.

There are two ways to do this:

  • create a sound node in the particle treeview to play a sound.
  • configure the editor environment to play a background sound, pretty much like setting un a backdrop for the effect.

We'll first see how to create a sound node.

Right-click on the "Particle System" node in the particle treeview, and select "New Sound" in the popup menu:

Tut04 03.jpg

Select the newly created node, and type the path of your audio file in the "SoundPath" field. The particle tutorials pack comes with an example audio track you can use: "Sounds/ip2x-electric_techno_(grotesque_No_more).mp3"

As soon as you validate with the 'Enter' key, the sound should play.
Now, note that by default, when a new effect is created, the editor is in loop mode, and loops the effect. this will loop the sound too.

Tut04 04.jpg

The default effect loops every two seconds, so if your sound is longer than this, it will play over itself. you probably don't want that.
Select the "Editor Properties" node in the particle treeview, and set "loop delay" to whatever the length of your audio sample is.

(for the example audio file, it's ~220 seconds)

Tut04 05.jpg

Also don't forget to set the particle spawner duration to the same value so it loops correctly:

Tut04 06.jpg

Once you're done you can reset the whole effect with the respawn button to start from a clean state.

Audio backdrop

Setting up an audio backdrop makes the music independent from the FX, it will only be editor-side, and only a tool for edition, like the backdrops. Therefore, you do not have to change the spawner duration or anything, both will be completely independent.

The audio backdrop is available from HH-Fx designer v1.1.0.14460

To setup an audio backdrop, unroll the EditorProperties node, and select the "Audio Environment" node:


Then simply type the path of the sound in the node properties panel. The audio will start looping as soon as you validate with enter, or reload the editor.

EditorProperties Audio.png

Basic spectrum sampling

To create a spectrum sampler, right click on the 'Samplers' node of the layer in the particle treeview, and select "New Sampler" > "CParticleSamplerSpectrumImpl"

By default, if it is the first sampler, it will be named "Sampler_0"

Tut04 01.jpg

Go in the physics evolver and disable the gravity for the moment.
Then we'll go in the spawn script to setup the position using the sampler.

Like all other samplers, the spectrum sampler has sampling functions available in-script.

The most straightforward version to use of the 'sample' function is one that only takes a sampling cursor as a parameter. It is a value between 0.0 and 1.0 (included), that tells the sampler where you want to sample the audio spectrum. 0.0 means full-bass, 1.0 means full-treble, and values in-between mean frequencies in between:

Tut04 SpectrumCursors.png

We're going to add to the script:

float cursor = rand(0,1); // random sampling position
Position = float3(cursor, Sampler_0.sample(cursor) * 200, 0);

therefore, the components of Position will be:

x = cursor // places the particle on the x-axis depending on where it will be sampled in the spectrum. particles sampling at full-bass will be at x=0, particles sampling at full-treble will be at x=1
y = Sampler_0.sample(cursor) * 200 // samples the spectrum at "cursor" and scales the sampled values to make the movement visible (the returned values are pretty small so most of the time they need scaling)
z = 0

Tut04 08.jpg

We'll quickly boost up the particle count to visualize the spectrum more clearly. (here, the spawner is set to a rate of 300 particles/second)

Tut04 09.jpg

You will note that the spectrum we get isn't very reactive. This is because the spectrum is sampled at spawn-time, the particles are placed on the spectrum, and then they just sit there for their entire life, even if the spectrum levels change (and they do, unless the particle life is ridiculously small).

For a more dynamic effect, we're going to need to sample the spectrum in an evolve script.

We could reuse Position.x as a sampling cursor, but this will be unlikely feasible in a real-world effect, and you will most likely need to keep track of the particle sampling location on the spectrum.

Therefore, we're going to create a new particle field named "Cursor", of type "float", where we'll store the sampling cursor:

Tut04 10.jpg

The spawn script becomes:

Cursor = rand(0,1);
Position = float3(Cursor*2-1,0,0);

The Cursor*2-1 instead of just Cursor is there to make the effect bigger, and centered on the spawn point. The particle positions will be distributed along the 'x' axis in the [-1,1] range.

Quick scripting note: we also could have used swizzling and written:

Cursor = rand(0,1);
Position = (Cursor*2-1).x00;

Tut04 11.jpg

Now, create an evolve script, we're going to sample the spectrum, and at each update, force the position to snap to the current spectrum value:

Position = Position.x0z + Sampler_0.sample(Cursor)._0x0 * 300;

Here, we're masking out the 'y' component of the position, and forcing it to the spectrum level scaled by 300. It is equivalent to writing:

Position = float3(Position.x, Sampler_0.sample(Cursor) * 300, Position.z);

The particles will immediately snap and follow the spectrum curve:

Tut04 12.jpg

If you select the spectrum sampler, you can now play around with the values and see what they do. For example, when selecting 'Point' filtering and bringing the convolution slider up to 0.7 :

! WARNING ! for hh-fx designer versions below, when changing the spectrum filtering mode, the modification won't be dynamically taken into account.
you need to select the script nodes to trigger the change. If you don't, it will still be correctly saved in the effect file, but you won't see it until a scene reset or until you re-launch the fx editor window

Tut04 13.jpg

Linear filtering:

Tut04 14.jpg

Cubic filtering:

Tut04 15.jpg

Particle Coloring

We are now going to see how to color the particles based on the spectrum levels. It is pretty easy, it all boils down to manipulating the sampled level and using it to index a color curve.

First, bring the convolution down a bit to show more details on the spectrum, and boost the particle spawn rate to 5000/s :

Tut04 16.jpg

Next, add a Color field in the particle fields, and create a new float4 curve sampler, and setup whatever color gradient you like:

Tut04 17.jpg

Now, we're going to tell the evolve script to sample our color curve using the spectrum values.

Change the evolve script from this:

	Position = Position.x0z + Sampler_0.sample(Cursor)._0x0 * 300;

to This:

	float	spectrum = Sampler_0.sample(cursor);	// store the sampled value in a temporary local variable to avoid sampling the spectrum twice
	Position = Position.x0z + (spectrum * 150)._0x0;
	Color = Sampler_1.sample(spectrum * 500).xyz1;	// Sampler_1 is the name of our new color curve, and we use a .xyz1 swizzle to force alpha to one.

As soon as you hit Ctrl+S in the script editor, you should see the particles being colored depending on the spectrum level.
The higher the spectrum, the more it will sample the color on the right hand-side of the curve. the lower, the more it will sample on the left.

Tut04 18.jpg

Animating particles

Next, we're going to see how to improve the particle animation. We'd like to make the particles rise from the ground plane, up to the spectrum line, and stay on the line.

first we'll restore acceleration in the physics evolver, and set the ConstantAcceleration field to something like "0; 10; 0" to make the particles go upwards.

After validating this new acceleration value, you won't see any change. That's normal, because we are actually snapping the y component of the positions to the spectrum in the evolve script, and it is run after the physics evolver, so it overrides the simulated positions.

What we'll need to do is find a way to clamp the particle positions to the spectrum line.

If we go in the evolve script, we have that line:

	Position = Position.x0z + (spectrum * 150)._0x0;

if we change it to:

	Position = Position.x0z + (Position.y)._0x0;

we'll see the particles rising from the ground plane and going upwards. (note that it is strictly equivalent to Position = Position;, therefore to not touching the positions at all).
to clamp the particles to the spectrum level, we can use the min() function. This function takes two values (scalar or vector), and returns the smallest of the two.

	Position = Position.x0z + min(Position.y, spectrum * 150)._0x0;

This will actually say "don't touch the position 'y' coordinate, unless it is above 'spectrum * 150', in which case, set it to 'spectrum * 150'"

We can also move the spectrum line up a bit for better visualization, by adding a small offset to the vertical limit:

	Position = Position.x0z + min(Position.y, spectrum * 150 + 0.1)._0x0;

Tut04 20.jpg

Adding a size curve to make the particles smaller between the top and bottom lines can look nicer:

Tut04 21.jpg

We can also play around a bit with the velocity. giving an initial velocity to particles that's higher when the spectrum level is higher will produce interesting dynamic "ripple" patterns as the music is pulsating.
In the spawn script, add something like:

	Velocity = pow(Sampler_0.sample(Cursor) * 100, 0.5)._0x0 * 2;

Tut04 22.jpg

We'll now combine two coloring patterns together for a more complex effect.

We currently have the pattern "sample the color curve with a cursor that depends on the spectrum level based on the 'x' particle coordinate"
This gives us a horizontal mapping of the spectrum.

If we sample the spectrum again using this time the 'y' particle coordinate, we'll get a vertical mapping of the spectrum.

To see this in action, remplace the Color computation in the evolve script from this:

	Color = Sampler_1.sample(spectrum * 500).xyz1;

to this:

	Color = (Sampler_1.sample(spectrum * 500).xyz1 +
		 Sampler_1.sample(Sampler_0.sample(Position.y * 0.7) * 800));	// scale the position Y coordinate down a bit to make it fit nicely

This will combine the two spectrum color mappings into one, and give a dual gradient:

Tut04 25 1.jpg

We can take this further and combine the horizontal gradient with a radial gradient instead of a vertical one:

the length() function takes a vector and returns its length. Therefore, if we subtract the particle position from a given point in space, and pass the result to the 'length()' function, we will get the distance of the particle to that point. We can use that value to sample the spectrum.

Particles on the point will sample the spectrum at cursor=0, particles in a circle 0.2 units away from the point will sample the spectrum at cursor=0.2, etc...

for example, with the 2D point located at x=0 and y=0.25, with the distance scaled by 0.4:

	Color = (Sampler_1.sample(spectrum * 500).xyz1 +
		 Sampler_1.sample(Sampler_0.sample(length(Position.xy - float2(0, 0.25)) * 0.4) * 800));

Tut04 26.jpg

Tweaking the colors a bit:

Tut04 27.jpg

Final details

Finally, we're going to stretch the particles along the vertical axis depending on the spectrum levels.
We could use the velocity vector, but we want more control over the stretch length. Create a new particle field named 'Axis', of type float3, with transform filter = rotate (there is a preset for it):

Tut04 28.jpg

Go in the billboard renderer and change the BillboardMode to 'VelocityCapsuleAlign' or 'VelocitySpheroidalAlign' or 'VelocityAxisAlign'

Then, in the evolve script, set the axis to float3(0, some_value_based_on_spectrum, 0);

for example:

	float	cursorCombined = spectrum * 500 + Sampler_0.sample(length(Position.xy - float2(0, 0.25)) * 0.4) * 800;	// this is just a factorization of what we had before in the 'Color' assignment
	Color = Sampler_1.sample(cursorCombined).xyz1;
	Axis = float3(0, saturate(pow(cursorCombined * 0.5, 2)), 0);

Tut04 29.jpg

Change the size curve to make a more "shadowy" effect on the bottom, bringing the particle size to nearly zero:

Tut04 30.jpg

Just a quick reminder before the end: you can use the InteractiveMode for quick script constants tweaks.
(to activate, click the small gears icon in the top left toolbar, above the realtime viewport)

Once activated, press the 'Ctrl' key in the script editor. It will highlight all the values that can be interactively tweaked:

Tut04 31.jpg

With the 'Ctrl' key still down, right mouse-click on any of these highlighted constants (here, the 0.25 value of the radial mapping 'y' center point)
A small slider will pop underneath the value, and clicking and dragging inside this slider will dynamically change the value and give you the opportunity to make quicker tweaks. Here, we just brought down the center point a bit:

Tut04 32 1.jpg

Taking it further

All this is of course not limited to linear mapping of cursors/Positions along a single axis, you can construct more complex patterns. Here are just a few quick examples.
If we change the spawn script from this:

	Cursor = rand(0,1);
	Position = float3(Cursor*2-1,0,0);

to this:

	Position = rand(float3(-1,0,-1), float3(1,0,1));	// spawn the particles on a horizontal square with a side 2 units long.
	Cursor = saturate(length(Position)*0.3);		// compute the cursor based on the distance of the particle to the origin of the effect.

and in the evolve script, from this:

	Position = Position.x0z + (min(Position.y, spectrum * 150 + 0.5))._0x0;
	float	cursorCombined = spectrum * 500 + Sampler_0.sample(length(Position.xy - float2(0,-0.13)) * 0.4, 0.2, spectrumFilter.Linear) * 800;

to this:

	Position = Position.x0z + (min(Position.y, spectrum * 150))._0x0;	// removed the offset (+ 0.5) from the base spawn plane
	float	cursorCombined = spectrum * 500;				// removed the secondary radial effect for more visual clarity

It will give the following effect:

Tut04 33.jpg

The spectrum sampling cursor can of course be anything. For example, a texture sampler:

	Cursor = Sampler_2.sample(Position.xz * 0.5 + 0.5).x;

Tut04 34 1.jpg

Or a procedural turbulence field:

	Cursor = saturate(Sampler_3.samplePotential(Position).x * 0.5 + 0.5);

Tut04 35.jpg

Previous tutorial : Particle tutorial fire
Next tutorial : Trails