Pitch Extraction

[martinvicanek]

Hi,

I have implemented a pitch extraction method based on ASDF (Average Squared Difference Function). It is a port of a project I first did with SE.

Compared to zero crossing algorithms ASDF is more CPU demanding but also more robust (no octave jumps). The accuracy is not bad either (1 cent). Latency is small enough to allow real time applications like guitar synth, pitch correction etc.

Here is a demo guitar synth using my pitch extractor. Actually there are two synths in there: one is alanlog and the other is MIDI controlled. The synths are far less sophisticated than Trog's Grrrr and the GUI is quite awkward, it is basically to demonstrate the pitch extraction thing.

The MIDI interface is a bit sluggish, which is probably due to the Tick 100 limitation, but still OK for real time performance.

Since this is my first SM project, I am looking forward to receiving comments and suggestions from the gurus

I'd like to thank BertAnt with whom I had an inspiring discussion over at the SE forum and who encouraged me to attempt the MIDI conversion.

Martin

[RunBeerRun2]

Works very well, low on latency and fast tracking. Tracks my whole guitar's range, on a bass only down to the E which is the guitar's low E(7th fret A string).

I've exported it for MIDI output use, in Reaper on my dual core it runs at a nice minimal 2.6% cpu.

Thanks, it's really cool!

[oddson]

Tracks my whole guitar's range, on a bass only down to the E which is the guitar's low E(7th fret A string).

I think it's bottom end is set in the filter value it starts with...

I still don't get how it works... ASDF is clearly an auto-correlation method (and I get that part) but I can't quite figure how it's implemented here (although i just now opened it up!)


It looks like you check four candidates but I'm not sure how that gets you a specific MIDI let alone track pitch bend:S

[martinvicanek]

Thanks for your comments, guys.

Yes, the option to set the pitch detection range is there but clearly it has to be accessible from the GUI. I'll post an update soon.

ASDF is indeed very similar to autocorrelation based pitch detection. Basically one compares the original audio signal with a delayed signal. They will be equal for periodic input if the delay time matches the period. Now if you have an array of delays tuned to semitones, all you need to do is pick the best match. In my scheme I use 4 octaves packed in Mono4 for efficiency, so in the first step I have 4 candidates, one from each octave.

The next step is to pick from these four. Note that there is an ambiguity because you will have matches for multiples of the period (at the fundamental frequency, 1/2 the frequency, 1/3 etc.). To pick the right candidate I add a little bias to give preference to the higher octaves.

Once you have the best matching semitone, you can estimate pitch bending from the two neighbors' values.

One nice bonus is that ASDF gives you a measure for voiced (= periodic) vs. unvoiced input. That can be used for a gate function to prevent the device from trying to extract a single pitch when in fact there is none (try playing chords!).

Hope that explains it a bit.
Martin

[trogluddite]

Welcome to the forum, Martin.
Thanks for the schematic, and the good explanation too. I've been looking at correlation type pitch-detecting for a while, trying to improve the tracking on GRRRrrr, but gave up because I couldn't find a way get a reasonable CPU load - yours uses about a quarter of the CPU of my best efforts.
I'll definitely be playing with this some more, and having a poke around inside for some inspiration - so thanks again for sharing.
Had a quick look through the code, and it looks as if there could be some reasonable CPU savings still to be had. The SM code for accessing arrays is particularly inefficient - so you should see some good savings in the ASDF module by replacing the 14 item arrays with discrete variables - untidy, I know, but since your loops are already unrolled, quite easy to do.

[oddson]

...Now if you have an array of delays tuned to semitones, all you need to do is pick the best match...

Holy crap... I'm starting to get it but that's massively daunting -- all those fractional delays!

It would take me ages just to understand how you've done this...

But I do have a question from what little of this I think I'm starting to understand:

Why do you need to calc delays in stage 2? Aren't these values constants. Even at ~10 Hz that's a lot of calculation.

[martinvicanek]

Thanks Trog, I replaced the indexed arrays by fixed variables, not sure if it makes a huge difference. Anyway, I updated the file at the top of this thread. It also has a range selector for guitar or bass.

Oddson, the delay constants are calculated in stage(2) so the user can switch the tonal range without having to restart. At hop(4096) it does not hurt. The number crunching part is really the fractional delay calculation itself (yes, they have to be fractional) plus the lowpass for each semitone. However, it turns out that it is perfectly feasible (using Mono4 and hop(8) though).

Martin

[BertAnt]

Thanks Trog, I replaced the indexed arrays by fixed variables, not sure if it makes a huge difference. Anyway, I updated the file at the top of this thread. It also has a range selector for guitar or bass.

Oddson, the delay constants are calculated in stage(2) so the user can switch the tonal range without having to restart. At hop(4096) it does not hurt. The number crunching part is really the fractional delay calculation itself (yes, they have to be fractional) plus the lowpass for each semitone. However, it turns out that it is perfectly feasible (using Mono4 and hop(8) though).

Martin

hello martin, it's really cool that you've finally done your amazing code in SM ! for me it'll be much easier to understand (i hope ) because there's no 'language gap' anymore like with the one you did in SE

edit : it's confusing as well apparently i see that you've integrate the LPF with the delay in one code..

[martinvicanek]

Hey Bert,

good to meet you here!
Yes, I stuffed all I could in code. I just love the SM code module!
The reason why I haven't broken it down in smaller parts was mainly to avoid unnecessary I/O (better performance?). I tried to use comments to make it readable for others, but I guess it is always harder to read other people's code than your own. If you have specific questions I'll be happy to respond.

Martin

[TomC]

Since this is my first SM project, I am looking forward to receiving comments and suggestions from the gurus

A division is MUCH more expensive (CPU-wise) than a multiplication.
After a quick look I've found several places where you can remove them:

- Bandpass module: instead of feeding "SampRat" into the code-primitive use "1/SampRat" and replace the "whatever/samplerate" statements by "whatever*samplerate" statements. I recommend to rename "samplerate" appropriate (I used "invsamplerate") to avoid confusion.

- ASDF module: do the same for "basefreq". Additionally you use about one zillion divisions with the variable "semi". Do the same here, replace "float semi=1.059463094359;" by "float invsemi=0.9438743126819565;" (that's the result of 1/1.059463094359) and use multiplications instead. If the original "semi" is not used anymore after that you can delete its definition.

I only had a quick look, so if there are more avoidable divisions somewhere else you know now what to do
You'll save tons of CPU-cycles this way.

Tom

[TomC]

Oddson, the delay constants are calculated in stage(2) so the user can switch the tonal range without having to restart. At hop(4096) it does not hurt.

At higher sampling rates (where your CPU is fighting anyway) it WILL hurt.
Compute the delays in GREEN and update the results only when the user actually changes the tonal range. That's only ONE single computation per tonal range change.

Tom

[martinvicanek]

Many thanks, Tom! Yes, divisions are expensive. I tried not to use them in the "inner loop" (the part of the code that is executed frequently) but you are right, they could be avoided at the other places, too. Thank you!

Martin

[RunBeerRun2]

I like the new bass guitar range, works pretty well!

I added an E0 on the note selector, my 5-string tuned down to F# tracks well w/it.

[martinvicanek]

RunBeerRun, good to hear that it works for you, too! If you really like bass, you can also mix the original sound with an octave down synthesized sound. Actually it was the search for a decent octave effect that started the whole project.

[mccy]

Great! Tested it today.
Couldn't decide if I like Grrrr better or thisone, but thisoe has Midi . Tried it with Saxophone-Playing and I think I will use it occasionally to transfer saxplaying to midi when I don't wnt to use a keybord. It works really well!

All one would need now is a decent pitch modifyer to get some autotuner and Midi Harmonizer stuff. There was one in the "Adern" Modular Package for Scope... Is there something usable for Synthmaker?

Martin

EDIT: Found Ssambeans and MyCos and the reaktor Stuff. Very impressed. A harmonizer shouldn't be a problem. I once tried to build an autotuner on Scopes SDK, so that could be harder, although I think SMs possibilities are superior.