Hello pianovintage, welcome to the forum,
DigitalWhiteByte wrote:envelope generator
This is the key problem with your schematic - a simple ADSR envelope module following the filter should get your samples playing in time.
A little background info about poly signals may help you to understand this...
The 'MIDI to Poly' and envelope generators have a special 'hidden' relationship inside the SynthMaker engine.
The stock envelope generators have special code that tells the poly streams how to behave when voices are started and ended. For example, this allows SM to know when a voice is finished at the end of a long release envelope, so that the sound is not 'killed' until the envelope has completed.
In this case, it also has some control over the counters inside the sample players - particularly, when to reset to the first sample of the file, and when to stop counting.
Without this, changes to the pitch etc. are only correctly recognised each time the sample player resets to the first sample - so the timing problem is caused by each note starting at some random point inside the wave file, and then having to wait until the next reset of the sample loop before the new pitch takes effect - leading to those random time delays.
It also means that SM does not know when to stop playing the sample - hence the extra high CPU due to lots of overlapping voices that are still playing even though you may not be able to hear them (a pitch of zero stops the counter, but the code for the counter is still running, but 'frozen' in position).
An envelope with very fast attack and decay will also help to prevent nasty clicks and pops caused by the sample starting and stopping too quickly.
Hope that helps to get your design working.