[bandoneon]

When in annotation mode, go to the menu on the right, disable touch mode.

Thanks! I suppose you mean the option to "Enable touch input in stylus more" right?
That helps, but then I cannot pan and zoom at all when annotating, which is also a problem. Another slight inconvenience is that I also cannot use three-finger tap in order to exit annotation mode.
I was hoping it would be possible to combine touch and stylus input, while having palm rejection as well...