Strange.

I figured it will be tricky, because I don't measure time for speed, but just velocity based on refresh speed. Depending on how far the mouse has traveled between frames, this speed is being added into a brush size value multiplier that gets capped at 1.0. In each loop this value get's diminished by being multiplied with 0.95, making it shrink again fairly rapidly, but not instantly.
See the interface of Verve?
See the dials?
See any words show up as you hover with your mouse above a dial?
Go through each one of them until you see MIN.SIZE! If you want to get there faster, see the section for the brush parameters? See the BIGGEST DIAL of the brush interface? Hover over that and it should display the text "SIZE". See the NEXT BIGgest dial next to it? Hover over it and you should see "MIN SIZE"!
If you cannot see or find any of them, continue to work as you have been working, because you're doing amazing paintings, but don't expect any useful answers from me, because I have no idea how this software even runs on your machine or how you do your magic with it, haha.

Watch out that you don't have 2 instances of Verve running at the same time! I don't know what that would do?!
Anyway, your mouse type should not have any effect on it. You should record a tiny video where you start a stroke slowly and then gradually move faster. I'd be curious to see that. And leave the interface up so the settings show! That might help me understand better, you know.