[sciurius]

[Retry -- First attempt crashed but despite numerous "Auto Saving" I couldn't get the text of the posting back. How is this supposed to work?]

Some results from the new beta.

When the synch completes, server (and client) displays a dialog "Exit" or "Continue". When selecting "Continue" there is not much to do. The server is non-responsive (cannot be connected from a client) and cannot be restarted (button is greyed out). Toggling Wifi/Bluetooth and back re-enables the start button.

Song file names are now relative, great!! I can copy/paste the database to another system and it works.
Audio files are not relative, even if they neatly reside under the MSPro storage. Work in progress?

A series of test runs.

1. Just a synch to get the libraries, er, synched.
   
2. Repeat. A number of songs (22) seems to require updating although nothing was changed on either end.
   
3. Repeat. Everything in synch.
   
4. Update song "As I Roved Out" on the server and run the synch. A number of songs (18 out of the 22 songs of the 2nd run) again require updating although nothing was changed in their content or metadata.
   
5. Repeat. Everything in synch.
   
6. Repeat. The same 18 songs seem to require updating although nothing was changed on either end.
   
7. Repeat. The same 18 songs etc..
   
8. Repeat. Everything in synch.
   

Logfiles attached: log4c.txt (ebbreviated and commented) and log4.txt (full log).

[sciurius]

Would it be interesting to have an option complementary to "find missing files" that shows files in the MSPro storage location that are not referred from the database (and allow to remove them)?

[BRX]

For a start I tested the backup after installing the beta of the companion to the desktop and also the appbundle for MSP.

I tried to restore the backup made with the old beta.

My custom storage location is X:\dropbox\noten as I explained

With the restore I got the message that I have a custom storage location that MSP has no access to and was asked to choose a location.

I tried choosing the location above again.

Now I got the message (on W10!) "You must select the following folder: /storage/emulated/0......
Please try again or select cancel to use the default storage location.

After closing the message with OK and the choosing cancel MSP nevertheless restored the library to 

X:\dropbox\noten

without a folder structure. All the sheets had been restored to the root of "noten". The links for audio still point to the root of "noten" as well without respecting the folder structure.

I repeated the procedure with a backup created with the companion beta with the exact same result.

On the Android device I backed up from was still the last beta though.

I then tried to copy the db directly hoping the relative paths would work. I installed the new beta on the Android tablet and run MSP once. I said it upgraded the database.
But the size was the same as the database that was saved by the companion for the backup. Interestingly MSP on W10 couldn't load the database I copied from the companion directory for the first try and restored an automatical backup.

The second try with the copied db from Android worked as such. But it didn't find the sheets and reported  /storage/xxxx-xxxx/Noten/sheets/sheet.pdf does not exist.

Is there some fault in my thinking or my workflow? I haven't tried the sync yet and want to get backup and the direct copy to work before I have to run through a sync.

[Zubersoft]

Johan - I released a version 2.2.6 that adjusts the audio file paths as well. It was just a complete oversight on my part. Thanks for bringing that up. As far as the merging, I'm guessing there are probably differences in the number of calculated pages on your two devices for those chord pro/text based songs, and that's causing it to merge every time. I'll have to see what I can do about that.  With the continue button, if you click that, the server will go back to waiting for a connection from a client, so you wouldn't have to click the button again. If the client is not detecting the server after hitting continue, or the server is not accepting the connection, that is a problem I will have to address then. With the find missing files thing, I'm not sure how useful that would be in practice as most people probably don't have a mix of files with some under the storage location and some not. If you guys disagree, I can look into adding something for that, but I think there are probably higher priority things to address first.

BRX - When restoring a backup in either version (Android or Windows 10) it tries to honor the storage location in the backup file if you check the "Restore settings" checkbox. With Windows 10, permission has to be granted to access the storage location if a folder was picked outside of the storage location. Unfortunately, I obviously didn't handle the case where an Android backup file is restored on Windows 10, and the storage location can't be used for that, so I will add code to handle that. You should uncheck the "Restore settings" option and try again. I will say that your folder structure will only be restored if you created that library backup with your storage location set to a folder that all of your songs are under. So if you have /storage/xxx-xxxx/Noten as your storage directory, and all of your song files are under there, then Windows 10 should preserve your subdirectories. If your storage location is set to some other folder and you are managing your own files, then the folders won't be preserved on Windows. In short, set your storage location to that folder even if you are managing your own files otherwise the subdirectories can't be preserved. If your backup file on Android was created with a different storage location, you'll have to recreate it (and include settings in the backup file so the storage location can be extracted during the restore). Just to clarify, even if you don't restore settings when restoring the backup file, it will still extract the storage location so it can try to preserve subdirectories. So including the settings with the backup file is important if you care about your directories being preserved.

Did you set your storage location to /storage/xxxx-xxxx/Noten before updating to the latest version? If not, none of your paths are going to be relative in the database. They will be absolute and non-transferrable to Windows 10. I did release v2.2.6, so if you set your storage location properly and then install that, it should fix the database paths to be relative.

Mike

[sciurius]

Great job, thanks!

As for the reformatting hypothesis, I'm afraid have my doubts. Both tablets have a 10.1" screen with 800x1280 px, density 149 ppi.
Initially I restored a msb from the server on the client, so there are no differences in global settings.
Also, several of the files are PDF documents.
But I cannot shed any light on this. I'll keep my eyes open.

Don't bother about the missing file thing, as you already noted there are many other things more important. I wrapped up a script that does the job for me.

BTW did you know that recent Firefox complains about downloading a apk? It stops before completing the download with a cryptic message that "this files is not normally downloaded". Requires some extra mouse actions to get it.

[sciurius]

Ran some synchs with 2.2.6 and I don't see the symptom anymore. So lets assume you fixed it .

[sciurius]

I played a bit with synch to cloud and to backup, but these do not seem to be useful yet, is that correct?

For example: synch to an empty drive folder seems okay, a subsequent synch (with no changes inbetween) is painfully slow (approx. 10 songs per minute, gradually slowing down to less). It merges quite a lot of songs (again: there were no changes at all).
Also, it is not clear what the 'client' and 'server' roles are in this case.
EDIT: Killed the synch after 5 hours and only 21% progress.

Synch to a backup file quickly leads to a java crash.

Unfortunately, in contrast to synch to device, these functions do not write a log file so it is hard to report details.

[Zubersoft]

Thanks for trying to test them out. I'm surprised the sync to cloud folder was so slow for you though. It definitely was not that slow for me. I'll try again using your library, and hopefully I'll see the same behavior as you. I'll also try the sync to backup flie with the one you provided me, but I'm pretty sure I did that before and didn't experience any kind of crash... 

Thanks,
Mike

[BRX]

Hey Mike, I'm a bit confused right now, what to do and what to expect, so

When restoring a backup in either version (Android or Windows 10) it tries to honor the storage location in the backup file if you check the "Restore settings" checkbox. With Windows 10, permission has to be granted to access the storage location if a folder was picked outside of the storage location.

Yes. The permission was asked each time when I tried a restore. I chose the storage location on W10 x:\dropbox\noten before the backup and restore so I didn't expect to have to grant access again. Also I couldn't just pick OK but had to navigate away from the storage location (which was displayed in the file picker) and back to it to choose it with OK.

Unfortunately, I obviously didn't handle the case where an Android backup file is restored on Windows 10, and the storage location can't be used for that, so I will add code to handle that. 

The code isn't in the last beta yet, right? At least with my last try the message with the Android storage location popped up again.

You should uncheck the "Restore settings" option and try again. I will say that your folder structure will only be restored if you created that library backup with your storage location set to a folder that all of your songs are under. So if you have /storage/xxx-xxxx/Noten as your storage directory, and all of your song files are under there, then Windows 10 should preserve your subdirectories. If your storage location is set to some other folder and you are managing your own files, then the folders won't be preserved on Windows. In short, set your storage location to that folder even if you are managing your own files otherwise the subdirectories can't be preserved. If your backup file on Android was created with a different storage location, you'll have to recreate it (and include settings in the backup file so the storage location can be extracted during the restore). Just to clarify, even if you don't restore settings when restoring the backup file, it will still extract the storage location so it can try to preserve subdirectories. So including the settings with the backup file is important if you care about your directories being preserved.

Well, do I understand correctly: To restore the folder structure under W10 I have to create also a xxxx-xxxx/storage/noten path on W10 to use the backups from Android? 
That's less than ideal. Especially if I want to have it in a dropbox folder under Windows.

I had the expectation that relative paths would mean sheets from e.g. xxxx-xxxx/storage/noten/sheets/sheet.pdf on Android and x:\dropbox\noten\sheets\sheet.pdf on W10 would be saved as sheets/sheet.pdf in the database and the storage folder for the full path will be added by the respective MSP app on the devices according to the settings.

I restored an backup (made with beta companion) from Android to W10 today again with the latest betas. With and without settings it restored without folder structure (which was expected after your explanations above).

May I summarize what I'd like to be able to do and kind of expected?

I want to have my Android storage folder under "xxxx-xxxx/Noten/" on the external SD card and on W10 under "x:\dropbox\noten" in a Dropbox folder obviously.

A backup with companion or from within the app should be restored under the Win10 storage location with the exact folder structure of the subdirectories in the location.
Ideally a simple copy of the database from Android to W10 (maybe manually or as a simple option in the library sync) should be able without adjusting of paths (i.e. if all the audio and PDF files are there and in identical relative paths).

Will this be possible with the current or coming adjustments?

Also I noticed another side problem.

I indeed forgot at first to set the MSP storage location to "xxxx-xxxx/Noten/" on the external SD card. I thought I had because I always had my sheets under this location and imported from there (managing my files myself). When I changed the storage location now the option to expose the database vanished.

I understand why. The database has to be writable and sadly there are millions of problems to write to externals SD cards. The same problem goes for the editing of chordpro files, I guess (which I never use, so no big problem there for me).

But I'd really like to have the database still exposed on an unrooted device so I can still use my own manual automatic sync with Dropbox/Dropsync I mentioned a few times already.

Could you leave the option to expose the database even if the storage location is now on the external SD card? It would be sufficient if the current database would still be present in the default Android/data/com.zubersoft/files directory (cited from memory), maybe as copy from the private folders similar to a backup?

[Zubersoft]

BRX.


No, I haven't released a fix for that yet. I'll work on it today.

As far as the Windows 10 side, no, you don't have replicate the exact storage path from Android. What's going to happen is that the Android storage location is going to removed from each path, and whatever remains is going to be recreated under the storage location in Windows 10. 

You should already be able to copy the database over if all of your paths are relative in the database after installing the latest beta version. You'll find that out pretty quick if you drop the database on Windows and your songs don't load correctly. If you didn't update the storage location before updating with the latest beta, then it's going to be much more difficult to fix now. The easiest thing would be to adjust the database version with a sql editor, then restart the app and let it update again. 

I'll look into allowing expose database to be set to true even if an SD card is being used. I'll have to update the description to specify that if an SD card is used, the default application data directory is used instead.

Mike

[Zubersoft]

Johan - I'm testing with your library and the song Blue suede shoes - Carl Perkins doesn't have a file in it, which caused errors in my code because it's not supposed to be possible to have a song without a file in it.  Do you know how that happened? 

Thanks,
Mike

UPDATE:
I think it's a problem with blank files due to the code I added for relative paths.

[sciurius]

Mike, I really can't tell... There's some much experimenting I've deen doing these days. Just blame it on me .

[BRX]

If you didn't update the storage location before updating with the latest beta, then it's going to be much more difficult to fix now. The easiest thing would be to adjust the database version with a sql editor, then restart the app and let it update again. 

Well, the first time I updated with the "relative path beta" I hadn't updated the storage location to the SD card. I think I changed it though before updating to the newest beta. After each install of a new beta I got the message about the database adjustment.

I'm not familiar with sql editors and editing of database. Which editor do you recommend and what do I have to change.

I guess just changing some paths with the manual path correction utility from the MSP Android settings are not sufficient?

BTW, the mobilesheets.db that is saved by the companion during a backup under 

"C:\Users\user\AppData\Roaming\Zubersoft\MobileSheetsProCompanion\mobilesheets.db"

is that the copied and unchanged db from the tablet the backup is from (in my case the Android one)?

If so then the simple copy to "C:\Users\user\AppData\Local\Packages\41730Zubersoft.MobileSheets_ys1c8ct2g6ypr\LocalState\mobilesheets.db" didn't work either. MSP reported an error about not readable/loadable database and using the backup instead.

[Zubersoft]

I've released version 2.2.7 of the beta.  There are a few important changes including:

1) Song groups (artist, albums, setlists, collections, etc) are now only merged at the end and only if the "Merge groups" setting is checked. They are no longer included as part of the "Song metadata" setting. This ensures that the song groups maintain the same ordering on each device. Previously, the approach I was using would result in the groups having different orderings, making it seem like there were differences between the client/server when there should have been none.

2) I fixed an issue with setlist song notes where removing a song from a setlist was not removing the setlist song notes. Unfortunately, if setlist notes were previously added for a song, those notes may still be lingering in the database and can create merge conflicts when there are none.

3) Fixed some issues with merging where files that are used for multiple songs resulted in some unnecessary processing.

4) Fixed issue with the same preference file being included in backups multiple times

5) Fixed issue with the status log text not correctly clearing and resuming scrolling

6) Fixed issue for windows 10 where the tab entry count wouldn't update after certain actions (restoring a backup, clearing the library, etc)

Hopefully with the latest version, you can merge multiple times and see no differences. 

Thanks,
Mike

[sciurius]

Results for 2.2.7.

Server is set top update client only.
Client has a clean install of MSPro 2.2.7, no database.
Both systems are set to not manage files, storage location is ext SDcard, expose database.

First synch: all files are transferred from server to client.

Second synch: no updates, nothing transferred.

Third synch: 17 files are updated (metadata) on the client.

Fourth synch: no updates, nothing transferred.

Fifth synch: the same 17 files are updated (metadata) on the client.

And so on.

Logs + backup set have been sent via mail.