Note: I use the term “subnormal”, but a lot of people use the term “denormal” or “de-normal” or “denormalized number”. Any term is acceptable, but know that they all mean the same thing.
What Is A Subnormal?
The short answer: They are VERY small numbers. Small as in a decimal point followed by 38 zeroes and then some non-zero digits.
The long answer: In the float32 decimal representation, the first bit is the sign bit, the next 8 bits are the exponent bits and the last 23 bits are the mantissa bits. The number, then, is [sign * 2^exponent * mantissa]. The exponent has a lower limit of -127. The mantissa, while only 23 bits long can actually store 24 bit information. This is done with the use of an “implied leading one”. So instead of mantissas of 0 to 1, you actually have mantissa values of 1 to 2. The CPU knows to add a leading one to the value. In order to allow values lower than what the -127 exponent would allow, the processor can go into “denormal mode,” effectively ignoring the leading one. While accuracy at those low levels may be maintained, the CPU cannot perform it math as normal (pun intended). Instead, it uses subnormal-specific instructions that are sometimes 100 times slower than normal.
Why Are Subnormals Bad?
As stated before, they require A LOT of CPU resources. While any good DSP will strive for utmost accuracy, these values represent signal levels below -800dBFS. They simply aren't significant enough to warrant the extreme CPU demands to deal with them.
What Causes Subnormals?
The most common cause of subnormals are delay lines where the signal is attenuated by multiplication (or division) and fed back into itself. This happens obviously in in echo/delay/reverb effects, but more often than not, it occurs in IIR filters (which have one or more “feedback arms”). Consider the following example (a simple 1st order Highpass Filter):
- Code: Select all
out = in – in1 + (out*coeff);
in1 = in;
The feedback arm (out*coeff) will constantly multiply the result of the previous instance by the coefficient (a positive value less than 1). This is perfectly fine when a signal is present, but what happens when the input signal disappears? It keeps getting smaller and smaller and smaller and smaller, approaching but never reaching 0. Considering most audio is processed tens of thousands of time per second, it isn't long before you reach that 1e-038 threshold. When this happens the CPU must enter denormal mode.
How Can I Test For Subnormals?
The easiest way is to just look at the CPU meter at the bottom of Synthmaker. If you notice that it is higher after you release a note (or send audio through) than when a signal is present, then you have a subnormal problem. This may be harder to see on newer, faster CPUs (most subnormal spikes on my quad-core i5 only register at about 5%), but if you're looking for it, you'll see it. I always like to check my work on my atom machine. That thing will jump to 100% CPU if subnormals are present.
There are other ways with some analytic tools floating around, but often subnormal values are “trapped” in feedback loops in the code blocks which they originate, so the output might not necessarily show the presence of the subnormals.
I Know I Have Subnormals, How Do I Track Them Down?
First, examine anything that has a feedback loop in it. 99% of the time, that's where they'll be: Delays, reverbs, filters, etc. To test, simply disconnect the suspected module (or comment out the block of code) and see if you notice any performance improvement after notes have been released. If so, you've found the culprit. If not, keep looking!
It's best to look in the “mono” section of your project (with the blue connectors). This section is always active and therefore very prone to subnormals. While subnormals will show up in the poly section, they are not as obvious (and not as detrimental) because the poly sections “shut off” when the ADSR envelope closes.
Okay, I've Tracked Them Down, How Do I Deal With Them?
There are three common ways to deal with subnormals. We'll call them the “noise”, “redundant math”, and “test and flush” methods.
The Noise Method
The Noise method introduces very low level noise near the beginning of the audio path. This doesn't really get rid of them, it just makes sure they are never present to begin with. It ensures that there is always a signal present so the feedback arms never get the chance to enter subnormal levels. The “denormal remover” module in the SM toolbox has this option, but it uses a “white noise” routine to do so. Personally, I believe this to be overkill, so here is a little piece of code that you can paste into a “code” or “assembler” block to achieve the same results. It simply produces a low level signal at Nyquist that should be sufficient to prevent denormals, while still being way below the audible level. This method has the advantage of being easy to implement. You simply insert near the front of your audio path and it should take care of the problem. You might run into problems if you happen to attenuate this signal before it reaches the problem area, though. So you may need to play with the position of the code in the schematic path or the level of the noise. The downside of this method is it does introduce noise into the schematic and may or may not affect the sound you're after (depending on how sensitive you are to that kind of thing).
- Code: Select all
streamin in;
streamout out;
float hold;
stage(0){
hold = 1e-024;
}
stage(2){
hold = -1 * hold;
out = in + hold;
}
- Code: Select all
streamin in;
streamout out;
float negOne=-1.0;
float level=1e-024;
stage0;
movaps xmm0,level;
movaps hold,xmm0;
stage2;
movaps xmm0,hold;
mulps xmm0,negOne;
movaps hold,xmm0;
addps xmm0,in;
movaps out,xmm0;
The Redundant Math Method
The second method the deal with subnormals is to simply perform a redundant math operation. The theory behind this is if you have a subnormal value and you add it to, say, 1 and then immediately subtract 1, the subnormal value is “eaten up” in the operation. To implement that in our earlier HPF example you would do something like...
- Code: Select all
temp = in – in1 + (out*coeff);
out = temp + 1 – 1;
in1 = in;
The trick here is to find an appropriate value to add and subtract. In much the same way that subnormals are “eaten up” so may other values that are drastically different from the value you choose. Some have suggested that a value of 1e-012 is a good starting point. But all situations are different and you may need to experiment with different add/subtract values. I used to use this method all the time because it is pretty easy to implement, but rarely do anymore because it can seriously affect the “resolution” of your filters.
The Test and Flush Method
The third and final method I'll discuss is my preferred method. It involves simply testing for values that may be approaching the subnormal level and flushing them to zero. I personally have come to favor the threshold value of 1e-036 as it is pretty insignificant to begin with but leaves a little “wiggle room” for a few more multiplications before you start getting subnormals. To implement it in our HPF example, I would do something like...
- Code: Select all
temp = in – in1 + (out*coeff);
out = temp&(abs(temp)>1e-036);
in1 = in;
By using the AND logic with the condition that the absolute value of the output is higher than my chosen threshold, it ensures that any value approaching (or having reached) subnormal will automatically become a zero and any value higher than the threshold will go through unchanged. In assembly, you could do the same thing like this...
- Code: Select all
//define these values
float subNorm=1e-036;
int mask=2147483647;
//value to test is in xmm0
movaps xmm7,xmm0;
andps xmm7,mask;
cmpps xmm7,subNorm,6;
andps xmm0,xmm7;
movaps out,xmm0;
An important thing to point out... if you are using higher order filter (like biquads or similar) that have multiple feedback arms in series, you don't need to perform these actions for every value passed to the next instance. Generally, if you wipe out subnormals in the first feedback arm, that should effectively take care of them for the subsequent ones.
Okay, I hope that helps people understand subnormals a little better and I hope some of the examples I gave provide a good foundation of tools and tricks to help optimize your projects. Anybody who pays attention probably knows that I struggled with a lot of these issues when I was first starting with SM and had some crazy notions of how to handle these situations. Thanks to many people on here (trogluddite(!) , aliasant, infuzion come to mind) I think I've gotten the hang of it. Hopefully, this thread will provide a one-stop place for people in the same situation.
If I have mis-stated or omitted anything, please chime in so I can correct it. Any other comments or additions as always are welcomed and encouraged!
-cyto
. like pall, i often wondered myself about this behaviour with the higher CPU after 
