Particle tutorial audio spectrum sampler
|For PK-Fx Editor version : 1.4.0 and above
Main page: Particle tutorials
In this tutorial, we'll see:
The spectrum sampler samples the audio spectrum of any audio channel in your game/application.
Therefore, in this tutorial, before sampling any audio, we'll need to make the realtime editor play some audio.
There are two ways to do this:
- create a sound node in the particle treeview to play a sound.
- configure the editor environment to play a background sound, pretty much like setting un a backdrop for the effect.
We'll first see how to create a sound node.
Right-click on the "Particle System" node in the particle treeview, and select "New Sound" in the popup menu:
Select the newly created node, and type the path of your audio file in the "SoundPath" field. The particle tutorials pack comes with an example audio track you can use: "Sounds/ip2x-electric_techno_(grotesque_No_more).mp3"
As soon as you validate with the 'Enter' key, the sound should play.
Now, note that by default, when a new effect is created, the editor is in loop mode, and loops the effect. this will loop the sound too.
The default effect loops every two seconds, so if your sound is longer than this, it will play over itself. you probably don't want that.
Select the "Editor Properties" node in the particle treeview, and set "loop delay" to whatever the length of your audio sample is.
(for the example audio file, it's ~220 seconds)
Also don't forget to set the particle spawner duration to the same value so it loops correctly:
Once you're done you can reset the whole effect with the respawn button to start from a clean state.
Setting up an audio backdrop makes the music independent from the FX, it will only be editor-side, and only a tool for edition, like the backdrops. Therefore, you do not have to change the spawner duration or anything, both will be completely independent.
The audio backdrop is available from HH-Fx designer v18.104.22.16860
To setup an audio backdrop, unroll the EditorProperties node, and select the "Audio Environment" node:
Then simply type the path of the sound in the node properties panel. The audio will start looping as soon as you validate with enter, or reload the editor.
Basic spectrum sampling
By default, if it is the first sampler, it will be named "Sampler_0"
Go in the physics evolver and disable the gravity for the moment.
Then we'll go in the spawn script to setup the position using the sampler.
Like all other samplers, the spectrum sampler has sampling functions available in-script.
The most straightforward version to use of the 'sample' function is one that only takes a sampling cursor as a parameter. It is a value between 0.0 and 1.0 (included), that tells the sampler where you want to sample the audio spectrum. 0.0 means full-bass, 1.0 means full-treble, and values in-between mean frequencies in between:
We're going to add to the script:
float cursor = rand(0,1); // random sampling position Position = float3(cursor, Sampler_0.sample(cursor) * 200, 0);
therefore, the components of Position will be:
x = cursor // places the particle on the x-axis depending on where it will be sampled in the spectrum. particles sampling at full-bass will be at x=0, particles sampling at full-treble will be at x=1 y = Sampler_0.sample(cursor) * 200 // samples the spectrum at "cursor" and scales the sampled values to make the movement visible (the returned values are pretty small so most of the time they need scaling) z = 0
We'll quickly boost up the particle count to visualize the spectrum more clearly. (here, the spawner is set to a rate of 300 particles/second)
You will note that the spectrum we get isn't very reactive. This is because the spectrum is sampled at spawn-time, the particles are placed on the spectrum, and then they just sit there for their entire life, even if the spectrum levels change (and they do, unless the particle life is ridiculously small).
For a more dynamic effect, we're going to need to sample the spectrum in an evolve script.
We could reuse
Position.x as a sampling cursor, but this will be unlikely feasible in a real-world effect, and you will most likely need to keep track of the particle sampling location on the spectrum.
Therefore, we're going to create a new particle field named "Cursor", of type "float", where we'll store the sampling cursor:
The spawn script becomes:
Cursor = rand(0,1); Position = float3(Cursor*2-1,0,0);
Cursor*2-1 instead of just
Cursor is there to make the effect bigger, and centered on the spawn point. The particle positions will be distributed along the 'x' axis in the [-1,1] range.
Quick scripting note: we also could have used swizzling and written:
Cursor = rand(0,1); Position = (Cursor*2-1).x00;
Now, create an evolve script, we're going to sample the spectrum, and at each update, force the position to snap to the current spectrum value:
Position = Position.x0z + Sampler_0.sample(Cursor)._0x0 * 300;
Here, we're masking out the 'y' component of the position, and forcing it to the spectrum level scaled by 300. It is equivalent to writing:
Position = float3(Position.x, Sampler_0.sample(Cursor) * 300, Position.z);
The particles will immediately snap and follow the spectrum curve:
If you select the spectrum sampler, you can now play around with the values and see what they do. For example, when selecting 'Point' filtering and bringing the convolution slider up to 0.7 :
! WARNING ! for hh-fx designer versions below 22.214.171.12476, when changing the spectrum filtering mode, the modification won't be dynamically taken into account.
you need to select the script nodes to trigger the change. If you don't, it will still be correctly saved in the effect file, but you won't see it until a scene reset or until you re-launch the fx editor window
We are now going to see how to color the particles based on the spectrum levels. It is pretty easy, it all boils down to manipulating the sampled level and using it to index a color curve.
First, bring the convolution down a bit to show more details on the spectrum, and boost the particle spawn rate to 5000/s :
Next, add a Color field in the particle fields, and create a new float4 curve sampler, and setup whatever color gradient you like:
Now, we're going to tell the evolve script to sample our color curve using the spectrum values.
Change the evolve script from this:
Position = Position.x0z + Sampler_0.sample(Cursor)._0x0 * 300;
float spectrum = Sampler_0.sample(cursor); // store the sampled value in a temporary local variable to avoid sampling the spectrum twice Position = Position.x0z + (spectrum * 150)._0x0; Color = Sampler_1.sample(spectrum * 500).xyz1; // Sampler_1 is the name of our new color curve, and we use a .xyz1 swizzle to force alpha to one.
As soon as you hit Ctrl+S in the script editor, you should see the particles being colored depending on the spectrum level.
The higher the spectrum, the more it will sample the color on the right hand-side of the curve. the lower, the more it will sample on the left.
Next, we're going to see how to improve the particle animation. We'd like to make the particles rise from the ground plane, up to the spectrum line, and stay on the line.
first we'll restore acceleration in the physics evolver, and set the ConstantAcceleration field to something like "0; 10; 0" to make the particles go upwards.
After validating this new acceleration value, you won't see any change. That's normal, because we are actually snapping the y component of the positions to the spectrum in the evolve script, and it is run after the physics evolver, so it overrides the simulated positions.
What we'll need to do is find a way to clamp the particle positions to the spectrum line.
If we go in the evolve script, we have that line:
Position = Position.x0z + (spectrum * 150)._0x0;
if we change it to:
Position = Position.x0z + (Position.y)._0x0;
we'll see the particles rising from the ground plane and going upwards. (note that it is strictly equivalent to
Position = Position;, therefore to not touching the positions at all).
to clamp the particles to the spectrum level, we can use the
min() function. This function takes two values (scalar or vector), and returns the smallest of the two.
Position = Position.x0z + min(Position.y, spectrum * 150)._0x0;
This will actually say "don't touch the position 'y' coordinate, unless it is above 'spectrum * 150', in which case, set it to 'spectrum * 150'"
We can also move the spectrum line up a bit for better visualization, by adding a small offset to the vertical limit:
Position = Position.x0z + min(Position.y, spectrum * 150 + 0.1)._0x0;
Adding a size curve to make the particles smaller between the top and bottom lines can look nicer:
We can also play around a bit with the velocity. giving an initial velocity to particles that's higher when the spectrum level is higher will produce interesting dynamic "ripple" patterns as the music is pulsating.
In the spawn script, add something like:
Velocity = pow(Sampler_0.sample(Cursor) * 100, 0.5)._0x0 * 2;
We'll now combine two coloring patterns together for a more complex effect.
We currently have the pattern "sample the color curve with a cursor that depends on the spectrum level based on the 'x' particle coordinate"
This gives us a horizontal mapping of the spectrum.
If we sample the spectrum again using this time the 'y' particle coordinate, we'll get a vertical mapping of the spectrum.
To see this in action, remplace the Color computation in the evolve script from this:
Color = Sampler_1.sample(spectrum * 500).xyz1;
Color = (Sampler_1.sample(spectrum * 500).xyz1 + Sampler_1.sample(Sampler_0.sample(Position.y * 0.7) * 800)); // scale the position Y coordinate down a bit to make it fit nicely
This will combine the two spectrum color mappings into one, and give a dual gradient:
We can take this further and combine the horizontal gradient with a radial gradient instead of a vertical one:
length() function takes a vector and returns its length. Therefore, if we subtract the particle position from a given point in space, and pass the result to the 'length()' function, we will get the distance of the particle to that point. We can use that value to sample the spectrum.
Particles on the point will sample the spectrum at cursor=0, particles in a circle 0.2 units away from the point will sample the spectrum at cursor=0.2, etc...
for example, with the 2D point located at x=0 and y=0.25, with the distance scaled by 0.4:
Color = (Sampler_1.sample(spectrum * 500).xyz1 + Sampler_1.sample(Sampler_0.sample(length(Position.xy - float2(0, 0.25)) * 0.4) * 800));
Tweaking the colors a bit:
Finally, we're going to stretch the particles along the vertical axis depending on the spectrum levels.
We could use the velocity vector, but we want more control over the stretch length. Create a new particle field named 'Axis', of type float3, with transform filter = rotate (there is a preset for it):
Go in the billboard renderer and change the BillboardMode to '
VelocityCapsuleAlign' or '
VelocitySpheroidalAlign' or '
Then, in the evolve script, set the axis to float3(0, some_value_based_on_spectrum, 0);
float cursorCombined = spectrum * 500 + Sampler_0.sample(length(Position.xy - float2(0, 0.25)) * 0.4) * 800; // this is just a factorization of what we had before in the 'Color' assignment Color = Sampler_1.sample(cursorCombined).xyz1; Axis = float3(0, saturate(pow(cursorCombined * 0.5, 2)), 0);
Change the size curve to make a more "shadowy" effect on the bottom, bringing the particle size to nearly zero:
Just a quick reminder before the end: you can use the InteractiveMode for quick script constants tweaks.
(to activate, click the small gears icon in the top left toolbar, above the realtime viewport)
Once activated, press the 'Ctrl' key in the script editor. It will highlight all the values that can be interactively tweaked:
With the 'Ctrl' key still down, right mouse-click on any of these highlighted constants (here, the
0.25 value of the radial mapping 'y' center point)
A small slider will pop underneath the value, and clicking and dragging inside this slider will dynamically change the value and give you the opportunity to make quicker tweaks. Here, we just brought down the center point a bit:
Taking it further
All this is of course not limited to linear mapping of cursors/Positions along a single axis, you can construct more complex patterns. Here are just a few quick examples.
If we change the spawn script from this:
Cursor = rand(0,1); Position = float3(Cursor*2-1,0,0);
Position = rand(float3(-1,0,-1), float3(1,0,1)); // spawn the particles on a horizontal square with a side 2 units long. Cursor = saturate(length(Position)*0.3); // compute the cursor based on the distance of the particle to the origin of the effect.
and in the evolve script, from this:
Position = Position.x0z + (min(Position.y, spectrum * 150 + 0.5))._0x0; float cursorCombined = spectrum * 500 + Sampler_0.sample(length(Position.xy - float2(0,-0.13)) * 0.4, 0.2, spectrumFilter.Linear) * 800;
Position = Position.x0z + (min(Position.y, spectrum * 150))._0x0; // removed the offset (+ 0.5) from the base spawn plane float cursorCombined = spectrum * 500; // removed the secondary radial effect for more visual clarity
It will give the following effect:
The spectrum sampling cursor can of course be anything. For example, a texture sampler:
Cursor = Sampler_2.sample(Position.xz * 0.5 + 0.5).x;
Or a procedural turbulence field:
Cursor = saturate(Sampler_3.samplePotential(Position).x * 0.5 + 0.5);