Pitch Extraction

[martinvicanek]

Hey Martin,

good to know it works for sax! Actually I am not entirely satisfied with the MIDI conversion - it is not as responsive as the analog synth. I suspect it has to do with the stream-to-green conversion, which apparently cannot be done faster in SM than using the tick100 module.

I have a pitch shifter with autotune function ready, I just need a quiet moment to put it up. Bear with me, or should I say "stay tuned"?

Martin

[mccy]

GRRRRREAT!
I've done a quick MIDI shifter (which did not need any of your modules of course, but it was the first step to build an autotuner I thought ...) (It's fun to play ones own voice as a harmony to your own singing!!!), but faild with the autotuner, my first quick build just jumped wildly arround which was a quite funny effect .

P.S. I've read somewhere, that someone did a module to devide the Tick100 into a faster one (even a 1000?)... can't remember exacly if it worked well or not.

1000 Thanks

Martin

EDIT: Know now, what I did wrong in the autotuner. I'll give it a second try, even if it will be worthless compared to a solution of a real coder.

[trogluddite]

Tick100 into a faster one (even a 1000?)

There's no way to do it internally in SM, though many people have tried using lots of different methods - there just seems to be an inherent limit to how many you can generate.
Nix has been developing a very sneaky method, by sending fast MIDI events into SM plugins from an external 'Tick Generator' made using MAX/MSP. You can see his post HERE. It is very promising, but it does require some extra software, and setting up some special routing in SM/sequencer to make it work.

[nix]

It seems to go up to 22050 hz, which is the limit of MAX /msp 'bangs'.
Let me know how many hz you need and I'll bang(hehe) out an.exe if you want to pursue this avenue.
I'll make it adjustable actually soon.
It is very close to sample accurate on my machine(actually, I'll do some tests and post up), but it failed miserably on AngularMomentum's system.

[mccy]

Here comes a combination of your and MyCos Work...
You can play your own voice autotuned via keyboard in realtime as a second voice to your (untuned) voice.

Have fun!

Martin

MIDI_AUTO_pitcher.osm

MyCos and your work combined in a realtime MIDI AUTOTUNE PLAYER.

(195.94 KiB) Downloaded 265 times

EDIT: Trying a faster version: Tried it with the fast Power Module, but didn't get it to work. Is there a MONO fast Power module anywhere? I bet it's a problem with poly... ...EDIT: Ah, got it, there is a stream power , should work, will try later...

[mccy]

Fast version...

MIDI_AUTO_pitcherFAST.osm

(197.24 KiB) Downloaded 299 times

[mccy]

EASY AUTOMATIC TUNER, working great !!!

MIDI_PITCHER_EASYtuner.osm

(265.61 KiB) Downloaded 332 times

[martinvicanek]

Nice integration job, mccy!

[trogluddite]

Here's the results (so far) of some experiments I've been doing using a variation on Martin's correlation method.
In this case it is a 'mutated' idea stolen from the way FFTs work. Rather than auto-correlating with a delayed copy of the original signal, the audio is correlated with a bank of quadrature oscillators - i.e. the combined correlation with both the sine and cosine function of the desired frequencies.
I chose this because the quadrature oscillators can be programmed with a very small chunk of code (as long as the sines/cosines are calculated in green) - it's using about 2-3% of my Q6700 CPU to scan 8 octaves of semitones, despite using no hopping at all.

Quadrature Experiments v3A.osm

(106.2 KiB) Downloaded 288 times

So far, I'm just using the correlates to drive a funky frequency display - I need more time to study martin's clever code for deriving the root frequency using the weightings and interpolation.
You'll also see that there is an audio output. As the design is based around FFT ideas, I wondered whether the inverse function could be calculated to reconstruct the original signal. It works surprisingly well, despite not using sub-harmonics of the sample rate (as a true FFT would do). It can even make a very crude polyphonic pitch shifter - though a bit noisy (I think because the quadratures are not locked in phase to the sample rate?)
Still needs some work - the quadrature oscillators are free running, so the accumulation of rounding errors eventually makes it unstable (watch your speakers/ears!!) - when you see the Freq display start to get a bit random, you can press the PANIC button to reset the oscillators.
The main module has 3 other inputs...
-Base Frequency. The lowest freq shown on the display
-Averaging. Scale up the time that the correlation is averaged over - can get some nice 'harmonic resonator' sounds with long settings.
-Scale. Magnifies the display.

Next thing I'm looking into is using the proper FFT sub-harmonics - I think a very efficient sin/cos lookup-table should be possible, knowing that the table would always be indexed in integer numbers of samples, which ought to solve the instability problems. Still not as efficient as a true FFT, don't think, but at least it could be done all in streams.

[martinvicanek]

Amazing stuff, Trog!

As to your free running oscillators, a possible remedy could be to use Goertzel's algorithm or some sliding-window variation of it? Actually I am experimenting with this method to build a chord analyzer. So far I have a setup for three octaves, so you clearly beat me there.

Surprising how well the re-synthesis works for zero pitch shift (less so for nonzero shifts). Did you realize that when you feed in a sine you get a distribution of detected frequencies with considerable side lobes? This is inherent to all short-time FT (physicists call it uncertainty principle: you can't determine both frequency and time simultaneously). My guess is that this is the reason for the "noisiness" in pitch shifting.

Oh, the method I use for interpolating the root frequency is quite straight forward: take the semitone delay with the smallest autocorrelation. Evaluate also the autocorrelation for the two neighbors. Assume that the autocorrelation is a quadratic function of delay, i.e. draw a parabola through the three points. Now find the minimum to locate the best matching frequency. It is not a big deal to work out the maths, only slightly complicated by the progressive spacing between semitones in the delay time domain.

[trogluddite]

Thanks for the tips and ecouragement - the Goertzel algorithm looks very economical, I shall have to give it a try.

After some tests with sine sweeps etc., I managed to work out empirically what was going on with the side bands - having played around with FM and ring modulation so much in the past, I feel a bit dumb for not recognising the problem! I suppose that the big issue is that the side bands retain their linear difference from the detection/reconstruction frequencies (i.e. they are frequency shifted rather than pitch shifted).

I'm hoping that a little extra tuition from a maths grad' friend will get me a bit more up to speed with the equations - resonating quadrature filters just about makes sense to my engineer brain, but most of the articles on FFT, wavelets etc. that I've read look like secret codes of the illuminati to me!.
Still, the empirical experiments are keeping me entertained with their strange 1950's sci-fi noises in the meantime!

P.S.) Interesting too, to see Heisenberg come up in a DSP context - I'd always associated the uncertainty principle with quantum physics. I always felt that we are too much a society of 'specialists', so I'm always fascinated by these little examples of the 'holostic-ness' of science and technology.

[BertAnt]

As to your free running oscillators, a possible remedy could be to use Goertzel's algorithm or some sliding-window variation of it? Actually I am experimenting with this method to build a chord analyzer. So far I have a setup for three octaves, so you clearly beat me there.

thats what i have in mind too, using single goertzel as a detector for each semitone instead of doing FFT for the whole audio, can make the polyphonic pitch extraction much easier... it's pretty much the same concept with the previous multitap cancellation delay i guess, only with goertzel, there shouldn't be problems with polyphony

i've done some tryouts too in reaktor, but so far i found sometimes it still responds too long on low notes, which makes it difficult to play in realtime... but the 'sensitivity' of it is very promising, it can detect notes within a chord quite accurately...

[martinvicanek]

Polyphonic pitch extraction would be very cool, but I doubt it can be done in real time. Consider this:

We can do monophonic pitch extraction by autocorrelation (or AMDF, it does not matter at this level). After one cycle the signal starts folding back on itself, and after two cycles the correlation is complete, so by that time we can be sure what the fundamental frequency is. Two cycles for a low E string (guitar) are 24ms, which is just about what you can still tolerate for live performances. So this works OK, provided the signal is actually monophonic.

Now if you go for poly pitch extraction, you have to somehow decompose the signal into single frequencies (STFFT, Goertzel, again it does not matter). For semitone resolution, you would have to analyze 17 cycles (two adjacent semitones differ by 6% in their frequencies). So now we are talking about 200ms! If you apply a windowing function (there are good reasons to do that), the figure gets even worse by a factor of 2 to 3. (You'd still have to separate harmonics from real notes, but that is another consideration, not a fundamental latency limit.)

I find myself trapped in this general reasoning, so I'd love to see someone prove me wrong. At any rate, even a near-real time poly pitch extraction is useful, if not for live performance then perhaps for chord analysis.

[trogluddite]

Thanks to Martin's Hilbert port, I managed to fix my oscillator instability problem.
Doubt it's very technically accurate, but by using the Hilbert to 'complexify' the input, I can correlate using only one sine wave per frequency band. Not quite as CPU lite as before, but much better than having to generate both sine and cosine for every band, and a least it's now stable!!
I've posted it as a 'Graphics' project (HERE) - as Martin has pointed out, the time constants are just too long for pitch extraction with 1-semi resolution at low frequencies, but it works fast enough to make a pretty waterfall style frequency plot.

[BertAnt]

Polyphonic pitch extraction would be very cool, but I doubt it can be done in real time....

what i'm thinking is, if only we could 'model' a proper sensor for 'each' semitones (like 1 channel of the cancellation delay for example) there should be no problem of that excessive latency, given that the sensor works fast and accurate enough.

that way, there's no need to decompose the signal like you said...the sensor would recognize what note present in the audio instantly. and seems like, 1 goertzel per semitones should do fine in theory, considering that it only reacts if the audio contains its frequency (from what was said on wiki : the Goertzel algorithm looks at specific, predetermined frequencies.)

dont know how to achieve it in SM though, but in reaktor you could manipulate polyphony easily so that you could use an 48 voices component instead of using 48 of them for example - which is a huge advantage in CPU usage.