macOS or Windows ? On macOS with "Use Hardware Acceleration" activated, we're using AVFoundation player. For better accuracy, don't use it. In other cases we use FFMPEG Player and this should not happen.
I remember we fixed something about it but I don't remember if it's in 3.xxx or 4.xxx.
You can also try re-encoding your movies in HAP codec, that will make it more reactive. Here: https://hap.video/
What codec are you using ?