[rwstout]

If you just swap the file, the annotations should remain. At least at android.
You could try. To Make a copy from your song with annotations and swap the file there. But there should be no difference. 

As a side note: you should swap the via the swap file dialog.
Long press a song -> press the three dots-> swap file. 
If you  swap the file with in the "file" dialog of the song you will loose your annotations.

You are right: that's what I did (Long press a song -> press the three dots-> swap file).
The problem seems to be the syncing.
When I upsync the swapped library to Dropbox (sync annotations=off), and then downsync it to the voice-library on my tablet (Samsung  A9+) (sync annotations=off), I get the swapped PDF, but the annotations that I made on the tablet are gone for that song.