[JBlyth]

Hi Mike,

I have tried the Sync between my Win 10 and Android devices three times, and it has failed at the 23% mark each time (for 101 songs). I have attached the logs. The first attempt crashed MSPro on the Server end.....

This time it is failing on a different song to last time, so perhaps the attachment of an MP3 file is not the issue.

I have attached a fourth log.txt file from when I tried to sync between two Win10 devices, which failed at the 1% mark. Perhaps the Win10 to Win10 is not available yet???? I noticed that I had to do a direct connection as the Server didn't automatically show up on the Client device.

Cheers
John

[BRX]

Some additional info.

I had a look at the log.txt. And to my surprise it has logged comparisons up to songs with the letter "G" (and that's where I approximately canceled the process). So only the GUI and the progress bar didn't show any progress and seemed to be stuck. Not ideal but maybe I'll leave the process for a few hours unwatched and see if it finishes somehow the next try.

How does the sync proceed with source files already in the right place (but without a song reference in the db)? I hope it doesn't try to transfer the source file anyway but compares/skips similarily like in the normal backup? Also, will it leave other sheetfiles (not referenced in the db) alone? I'm asking since I keep a lot of other sheet files which aren't necessary in MSP (other formats or not included for other reasons) in the same main sheets directory like the MSP source files.

I'm trying my first db backup with the PC-companion (always refrained from it because of the size of the database (60k entries) and source files). And it's a really slow progress.
So I'd really like the option to "filter" and select the library sync data (by collection, album, setlist and so on) to reduce the time (and also to ease sharing parts of the library).

Also I'd really advocate the option to exclude the source files and audio files for the same reasons (which I'd like to maintain manually). The risk that a sloppy usage results in some orphaned entries is well worth the gain in time for the backup and sync of large libraries/dbs and parts of them.

[BRX]

OK, the normal backup is a bit OT, but still related and since I brought it up here already.

I've tried to backup my library (without audio) twice. It took a while but my (source) Android tablet reported that the backup was successful and (only) offered the option to disconnect from the PC companion what I did.

But the PC companion then asked if I really want to cancel the backup. The first time I said yes and the (4 GB) backup file was deleted. 

I repeated the process and after the disconnect canceled the cancel. But it's stuck with the screen "Transferring data from tablet" and the last song (though showing 100 % progress).

What am I doing wrong? Or is it a bug?

[Zubersoft]

JBlyth - thanks for the logs. It looks like a threading issue when annotations are synchronized due to the fact that on Windows, some of the data for Annotations has to be handled on the UI thread. I'll get a fix in place for that.

BRX - With a library as large as you are working with, it's possible that the status box filled up with so much text that it couldn't append any more. That's not supposed to happen, as there isn't supposed to be a fixed limit on how much text you can add in one of those components, but I do know that some manufacturers did add their own limitations. I may have to try putting a hard limit in place, or at least provide a way to clear the box if needed, but I don't love either approach as I want users to be able to have access to the results of the merge if needed. The progress bar shouldn't have been stuck unless the UI thread itself was stuck because it couldn't append any more text to the status box.

MSP first checks before sending files between the devices if a file already exists in the destination location. If it exists, then it does a checksum comparison to ensure that the file exactly matches what the other device has. If it's an exact match, then it will not be sent. If there is a difference, the existing file will be overwritten and replaced.

Adding conditions to what data is backed up by the companion (or in MobileSheetsPro itself) is not something I want to do. It adds significant complexity not only to the back-up process, but to the restore process as well, as I can no longer guarantee that songs that are referenced in the database will be found in the backup file. I really don't want to modify the backup/restore code and the backup file format to have to deal with that.  I'm open to the idea of not including audio in the backup file, as I used to support that in the original MobileSheets, but it does mean that users that enable the copying of audio files in the storage settings (meaning the audio files are managed in the storage location) could potentially find themselves without any audio at all after restoring their backup. The same concerns apply to the merging - adding the ability to filter what songs are going to be a part of the merge on each device means that I can no longer just rely on what's found in the database. Instead, each tablet would have to instruct the other about what songs are going to be participating in the merge, and then I have to add a lot checks for situations where you might say song A on tablet 1 is participating in the merge but that same exact song on tablet 2 is not.  While this is something I could potentially support in the future, it would involve having to add an entirely new screen for selecting what songs you want included in the merge, unless I made it very simple and did something like a multi-selection dropdown where you could pick collections or setlists that you want to limit the merge to. If that's sufficient for what you want, then I can add that to the list of future enhancements.

If all you want is to create a backup of the database itself without backing up song files or audio files, then you don't need me to make any changes. Just enable the "Expose Database" setting in the storage, and you can copy mobilesheets.db anywhere you want. This is something I only want advanced users doing, because I don't want angry emails from customers who didn't include files in their backup and lost everything because of it. There is always a balance between giving my power users what they want, and making sure that the larger majority of users with simpler use case scenarios aren't given ample opportunities to shoot themselves in the foot.

As far as the PC companion not finishing the backup, I have heard of people encountering that issue, but I've always struggled to reproduce it myself. It usually means the companion app is waiting for some additional bytes to be sent over the network, but either the tablet never sends them or the companion app doesn't receive them. If the progress shows 100%, you can safely terminate the process in task manager, and the backup file should be perfectly fine. 

Thanks,
Mike

[BRX]

Adding conditions to what data is backed up by the companion (or in MobileSheetsPro itself) is not something I want to do. It adds significant complexity not only to the back-up process, but to the restore process as well, as I can no longer guarantee that songs that are referenced in the database will be found in the backup file.

No, you misunderstood. I don't expect something like that in the backup (backup is and always should be a complete backup IMO). And you can exclude audio already in the current backup feature, BTW.

I was just talking about the library sync which I'd like to be able to limit to certain collections, setlists etc. And if you end up with some orphaned entries with that you can always do another sync with files or use the repair utility in the preferences.

If all you want is to create a backup of the database itself without backing up song files or audio files, then you don't need me to make any changes. Just enable the "Expose Database" setting in the storage, and you can copy mobilesheets.db anywhere you want. 

I know. That's what I've been doing almost from the beginning. But since I've now 2 Android deviceds and 2 W10 devices it's not so easy any more to correct the paths. So I was hoping for something like the "expose database" way from within the library sync but with an automatic path adjustment that works equally for all devices. Or if you can improve your path adjusting utility so it works on W10 for Android db and vice versa, that would be fine as well of course.

If the progress shows 100%, you can safely terminate the process in task manager, and the backup file should be perfectly fine. 

OK, good to know. I copied the backup file before canceling the companion. That should do it then, too.

MSP first checks before sending files between the devices if a file already exists in the destination location. If it exists, then it does a checksum comparison to ensure that the file exactly matches what the other device has. If it's an exact match, then it will not be sent. If there is a difference, the existing file will be overwritten and replaced.

That's what I was hoping. It's important to me that doesn't delete "orphaned source sheets" for some reason because it doesn't find a reference in the database.

Thanks again for your time and effort.

[Zubersoft]

Okay, thanks for the clarification. I do plan on adding a "fix path" type utility to the Windows 10 version, but it will be limited to folders under the storage location, as that is all the app is granted access to. 

So if I do add the option to filter only on songs belonging to certain collections or setlists, I think the merge would have to be confined to merging songs in that list that exist on both tablets, and adding songs that exist on the server over  to the client. The filter would only apply to what exists on the server device, so that would cause other songs on the client device that might not exist on the server to be ignored. Is that basically what you want? When merging groups (collections, setlists, genres) between the two devices, how am I then supposed to handle songs that only exist on one device but were ignored because of the filter? Should I add blank songs on the target device? Should I not allow groups to be merged when filtering songs like that? Should I only merge groups where they both contain only songs in that filtered list?  Already I can see this making things much messier.

Thanks,
Mike

[BRX]

I do plan on adding a "fix path" type utility to the Windows 10 version, but it will be limited to folders under the storage location, as that is all the app is granted access to. 

That should be sufficient in general. But then the audio files can't be in a separate folder outside of the storage location. Or is it possible to set the whole partition (D:/) as storage location and still have explicitly D:/sheets and D:/audio as the directories MSP uses and needs access to?

The filter would only apply to what exists on the server device, so that would cause other songs on the client device that might not exist on the server to be ignored. Is that basically what you want? 

Yes, I think that's completely sufficient. I do think this feature will mostly be used to supply the data from a "master" to "slaves". (And it doesn't make much sense to sync setlists both ways anyway) And if indeed a merge of a collection or album is needed, then you could simply reverse the roles of server and client for a second sync, right? Don't make this messier than I wanted. :-)

[Zubersoft]

I have released v2.2.4 of the beta for Android and Windows 10. This round of updates fixed the issue on Windows 10 when annotations are synchronized and adds an additional output message for each file that is sent from the server to the client so that it's easier to tell how far along it is.

Mike

[Zubersoft]

BRX - if you set the entire D drive as the storage location, then MobileSheets would be able to access any folders under that, including the two you listed. So that is certainly one way to handle it.

Mike

[BRX]

Good. We discussed having to folders for sheets and audio elsewhere already. But would it be a problem for the library sync if there's only one folder to choose in the folder settings? What happens if the storage location (i.e. the whole partition) is chosen for the folder settings to include an audio sync.

And a bit related: I just noticed that MSP on Android didn't find a sheet for a song anymore since it's spelling changed from capitals to small letters. I know that Android is case sensitive of course. 
But because of the interchange between Windows and Android libraries it should be case insensitive for the libary sync and for the database in general.

[Zubersoft]

BRX - why did the spelling change? Did the library merge do that? It should have updated the database if it did. I'm confused how that happened. On some Android devices, the file system is case-insensitive. On others, it's case-sensitive. So I can't really know beforehand what I'm dealing with. It's easier if I just do case-sensitive comparisons across the board, and if I flag some things as different that aren't, it will just result in a few more things being processed than necessary. It shouldn't create errors though, whereas if I try to play games with case-sensitivity in each direction, I'm likely to introduce bugs.

Thanks,
Mike

[BRX]

I thought about that too and I think it's no fault of MSP or Android. 

I sometimes replace PDFs with better scans. I'm doing this on my Windows desktop inside my Dropbox folder which then is synced to my other sheets devices.

So there was a change under Windows. With different devices and different OS to be synced and the different case sensitivities for Windows and Android/Linux I thought it would be sensible to go for case insensitivity in general. (But of course it's never so easy as I think).

[BRX]

Back to the storage folder under W10.

I tried to restore my Android backup to the root of E:\

It reported that some files couldn't be written to the location when in fact nothing was.
I then changed the storage location to E:\dropbox\Noten

MSP reported to have to move the old files (though there weren't any) to the new location and pretended to do that for a while. The database still had paths to e:\ after that though.

The I restored the backup to the new storage location. MSP worked then fine with database and sheet (I think I saw a few wrong paths related to " ' " in the songtitle. Will report later when I've figured it out). I think MSP also respected files that were already at the location.

But, with the current MSP, can I solve this problem? I want to have the sheets in my dropbox folder E:\dropbox\Noten. I want my audio files to reside outside the dropbox in E:\Audio. I guess a backup and/or syncing the sheets and audio from my Android device won't work currently, right?

Do you have any suggestions?

Or can you give MSP the storage location E:\ for the general access and allow other folders as sheets and audio folder which are respected by sync/backup and accessible by MSP under W10?

[Zubersoft]

If you want to be able to set your storage location to E:, you really need to have an empty library to start with, and you can't restore a library backup unless that backup already mapped all of the files/folders to the E: drive.  The reason is that, if the backup file wasn't created with all of the files/folders somewhere under the storage location you have chosen (and you must NOT restore settings from the backup as that would override your storage location potentially), then it will have to figure out where to copy those files/folders to. It tries to restore subdirectories correctly based upon the storage location that was used in the backup file (i.e. it strips the storage location from the file paths and then figures out if those folders and files can be created under the current storage location), but this may not always be possible.  If you are starting with an empty library, and you set the storage location to E:, and then you import your files from folders under that storage location, MobileSheets should recognize that it doesn't need to copy any files as they are already under the storage location. Some users have reported issues in the past with using the root folder of a drive as their storage location though, and I haven't had time to look into this more, so I don't know if it's still a problem. It's probably safer if you create a single folder under E: such as MobileSheets, and then under that you have your audio folder and your dropbox folder.  You definitely cannot have E:\dropbox\Noten as your storage location and your audio files in E:\Audio, as MobileSheets will not be granted access to the Audio folder by the OS. The only way to get around that would be for me to add a new option in the settings to allow users to add a list of folders they want MobileSheets to have access to, and I would have to persist access to those folders. That is a can of worms though when dealing with backup files and multiple devices.

Mike

[JBlyth]

Hi Mike,

I was able to successfully complete a perfect Sync between my Win 10 and Android devices with the new version, which hasn't been possible for a while. 

It also worked Win10 to Win 10 although the Song Title formatting from the Library did not get passed it seems.

Mp3 files failed to be Synced to the Android device, but if I understand it correctly, the Android version doesn't play mp3 files. Is that correct? Is it difficult to make the Android version play mp3 files from within MS? 

If it was possible, this would make Win 10 and Android devices completely compatible for Syncing purposes and would be a decided benefit. I sometimes use MS to directly play a backing track if the keyboard I am playing on won't play midi or mp3 files....

Cheers
John