• 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Information for beta testing library sync feature
#91
A couple of test runs.

Server is Samsung GT-N8010, LineageOS 14.1 (Android 7.0). MSPro does not manage song files. Config is to update client only.
Client is Asus Zenpad Z300M, Android 7.1. MSPro manages song files.

First run: The status of the client is arbitrary, so many files are updated from server to client.
Second run, immedeately after first: A number of files are, again, updated.
Third run, immedeately after second: A number of files are, again, updated.

For example, take a look at "----- Set 1 -----.pdf", a file that definitely didn't change since its creation.

Also, watch "The_Parting_Glass.cho". There are 4 (different!) files on the server:

-rwxrwx--x 1 root sdcard_rw 991 2018-05-23 19:35 Folkmuziekgroep_Drenthe/The_Parting_Glass.cho
-rwxrwx--x 1 root sdcard_rw 908 2017-07-28 21:07 September/The_Parting_Glass.cho
-rwxrwx--x 1 root sdcard_rw 897 2017-06-16 15:59 SingAlong/The_Parting_Glass.cho
-rwxrwx--x 1 root sdcard_rw 941 2018-05-01 12:10 Songs/The_Parting_Glass.cho

However, they all end up in a single file on the client:

-rw-rw---- 1 u0_a132 sdcard_rw 991 2018-06-25 13:21 The Parting Glass/The_Parting_Glass.cho

Log files (each file includes all 3 runs) attached.

EDIT: Much to my surprise (and astonishment) I found out that when restoring a full backup, the collapsing of songs with identical names into a single song file also happens.


Attached Files
.zip   logs.zip (Size: 363.97 KB / Downloads: 1)
Johan
johanvromans.nl — hetgeluidvanseptember.nl — mojore.nl -- howsagoin.nl
Samsung Galaxy Note S7FE (T733) 12.4", Android 13.0, AirTurn Duo & Digit (Gigs).
Samsung Galaxy Note S4 (T830) 10.5", Android 10.0 (maintenance and backup).
Samsung A3 (A320FL), Android 8.0.0 (emergency).
Reply
#92
In the dialog "Select song to use": can you please also show the file size?
Johan
johanvromans.nl — hetgeluidvanseptember.nl — mojore.nl -- howsagoin.nl
Samsung Galaxy Note S7FE (T733) 12.4", Android 13.0, AirTurn Duo & Digit (Gigs).
Samsung Galaxy Note S4 (T830) 10.5", Android 10.0 (maintenance and backup).
Samsung A3 (A320FL), Android 8.0.0 (emergency).
Reply
#93
I've added the file size to the output after each file (i.e. MyFile.pdf (456 KB). As far as the backup file issue, if you use the MobileSheetsPro storage location as the basis for all your folders, then MobileSheetsPro will be able to use your subdirectories appropriately. If you do not, then MobileSheetsPro won't be able to know which subdirectories should be part of the destination path. If you restore to the original file locations, this should have tried to create all your original files and folder paths, so I'm surprised that collapsed all the files into one.

As far as the merge collapsing the files into one, that should have only happened if for some reason it couldn't use and recreate the original subdirectories. I'll have to see if I can reproduce what you are seeing.

Thanks,
Mike
Reply
#94
(06-26-2018, 02:34 PM)Zuberman Wrote: I've added the file size to the output after each file (i.e. MyFile.pdf (456 KB).

Are we talking about the "Select song to use" dialog? It already displays a.o. the creation date. The file size in bytes would give another clue which of the alternatieves would be the one to use.

Quote:As far as the backup file issue,

Note that I take full responsibility for what happens to me, since I persist in using a non-preferrred directory layout.

Basically, all my files are in folders named after the collection, on the SD card, outside of MSPro. MSPro is set to not copy the songs, so they are retained in their original locations.

  /storage/ABCD-0123/Partituren/Collection1/Song1.cho
  /storage/ABCD-0123/Partituren/Collection1/Song2.cho
  /storage/ABCD-0123/Partituren/Collection2/Song1.cho
  /storage/ABCD-0123/Partituren/Collection2/Song3.cho
  ...

There are several songs with identical names in different collections.

I make a backup and relocate the folder names. In the new backup the locations become:

  /storage/3210-DCBA/Partituren/Collection1/Song1.cho
  /storage/3210-DCBA/Partituren/Collection1/Song2.cho
  /storage/3210-DCBA/Partituren/Collection2/Song1.cho
  /storage/3210-DCBA/Partituren/Collection2/Song3.cho
  ...

The folder /storage/3210-DCBA/Partituren/ exists on the target device.
However, when I restore this backup, the songs are restored as:

   [MSPro Storage location]/Song1/Song1.cho
   [MSPro Storage location]/Song2/Song2.cho
   [MSPro Storage location]/Song3/Song3.cho
   
or

   [MSPro Storage location]/Song1.cho
   [MSPro Storage location]/Song2.cho
   [MSPro Storage location]/Song3.cho

even though I select "restore to original file locations".

When I relocate to /storage/3210-DCBA/Android/data/com.zubersoft.mobilesheetspro/files/Partituren/Collection1/Song1.cho it works.

Quote:As far as the merge collapsing the files into one, that should have only happened if for some reason it couldn't use and recreate the original subdirectories. I'll have to see if I can reproduce what you are seeing.

It could be a subtile permissions issue. Don't bother too much.
Johan
johanvromans.nl — hetgeluidvanseptember.nl — mojore.nl -- howsagoin.nl
Samsung Galaxy Note S7FE (T733) 12.4", Android 13.0, AirTurn Duo & Digit (Gigs).
Samsung Galaxy Note S4 (T830) 10.5", Android 10.0 (maintenance and backup).
Samsung A3 (A320FL), Android 8.0.0 (emergency).
Reply
#95
I have restructured my library to bring everything under the MSPro storage location. The structure is now:

  [MSPro Storage location]/Partituren/Collection1/Song1.cho
  [MSPro Storage location]/Partituren/Collection1/Song2.cho
  [MSPro Storage location]/Partituren/Collection2/Song1.cho
  [MSPro Storage location]/Partituren/Collection2/Song3.cho

I made a new full backup and transferred it to the target device.

With "Restore to original file locations" I get:

  [MSPro Storage location]/Song1.cho
  [MSPro Storage location]/Song2.cho
  [MSPro Storage location]/Song3.cho

With "Restore to MSPro location" I get:

  [MSPro Storage location]/Song1/Song1.cho
  [MSPro Storage location]/Song2/Song2.cho
  [MSPro Storage location]/Song3/Song3.cho

In either case, there is only one "Song1.cho" left.

If I first relocate the backup (change the source library path to the target library path), a restore to original file locations gives:

  [MSPro Storage location]/Partituren/Collection1/Song1.cho
  [MSPro Storage location]/Partituren/Collection1/Song2.cho
  [MSPro Storage location]/Partituren/Collection2/Song1.cho
  [MSPro Storage location]/Partituren/Collection2/Song3.cho

Hurray!

Of much more interest is what happens when I clear the library on the target device, and use the new library sync facility.

It just works...! Perfect!
Johan
johanvromans.nl — hetgeluidvanseptember.nl — mojore.nl -- howsagoin.nl
Samsung Galaxy Note S7FE (T733) 12.4", Android 13.0, AirTurn Duo & Digit (Gigs).
Samsung Galaxy Note S4 (T830) 10.5", Android 10.0 (maintenance and backup).
Samsung A3 (A320FL), Android 8.0.0 (emergency).
Reply
#96
I modified 'As I Roved Out' on the PC and copied (rsync) the updated file to MSPro.
On MSPro, I removed all annotations from this song.
Then I started a sync. When it came to the modified song, it crashed:

06-26-2018 16:05:04 - Sending database file to connected device
06-26-2018 16:05:04 - Sending 1851392 bytes for file /storage/C443-17EE/Android/data/com.zubersoft.mobilesheetspro/files/mobilesheets.db
06-26-2018 16:05:23 - 1851392 bytes successfully sent for file: /storage/C443-17EE/Android/data/com.zubersoft.mobilesheetspro/files/mobilesheets.db
06-26-2018 16:05:31 - File successfully received and written to /storage/emulated/0/Android/data/com.zubersoft.mobilesheetspro/cache/temp_mobilesheets.db
06-26-2018 16:05:32 - Comparing song 'Are You There, Mo-ri-ar-i-ty?'
...a lot of compares...
06-26-2018 16:05:34 - Comparing song Arthur McBride
06-26-2018 16:05:34 - Comparing song As I Roved Out
06-26-2018 16:05:34 - Comparing song As I Roved Out
06-26-2018 16:05:34 - Merging 051_As_I_Roved_Out.cho
06-26-2018 16:05:34 - Error: java.lang.NullPointerException: Attempt to read from field 'android.graphics.Rect com.zubersoft.mobilesheetspro.b.ac.e' on a null object reference
06-26-2018 16:05:34 - Error: java.lang.NullPointerException: Attempt to read from field 'android.graphics.Rect com.zubersoft.mobilesheetspro.b.ac.e' on a null object reference
06-26-2018 16:05:34 - java.lang.NullPointerException: Attempt to read from field 'android.graphics.Rect com.zubersoft.mobilesheetspro.b.ac.e' on a null object reference
at com.zubersoft.mobilesheetspro.synclibrary.aw.a(LibraryMerger.java:2175)
at com.zubersoft.mobilesheetspro.synclibrary.aw.a(LibraryMerger.java:702)
at com.zubersoft.mobilesheetspro.synclibrary.aw.w(LibraryMerger.java:318)
at com.zubersoft.mobilesheetspro.synclibrary.l.a(BaseTransport.java:732)
at com.zubersoft.mobilesheetspro.synclibrary.m.run(Unknown Source)
at java.lang.Thread.run(Thread.java:761)

(Note there are multiple songs with title "As I Roved Out".)

The log file contains the sync mentioned in posting #95, a subsequent sync (that did not update anything), and the crashing sync.
FWIW, the first sync was via WiFi, the other syncs via BlueTooth.
Johan
johanvromans.nl — hetgeluidvanseptember.nl — mojore.nl -- howsagoin.nl
Samsung Galaxy Note S7FE (T733) 12.4", Android 13.0, AirTurn Duo & Digit (Gigs).
Samsung Galaxy Note S4 (T830) 10.5", Android 10.0 (maintenance and backup).
Samsung A3 (A320FL), Android 8.0.0 (emergency).
Reply
#97
(06-25-2018, 02:07 PM)JBlyth Wrote: [Edited]
It also worked Win10 to Win 10 although the Song Title formatting from the Library did not get passed it seems.

Mp3 files failed to be Synced to the Android device....

Hi Mike,

I tried the Sync again from Win 10 to Android, but this time I turned ON the "Copy Audio Files" in the Storage Options, and the MP3 file came across successfully this time.  I also used the "Always use data from the server" option, so I am not sure which change allowed the MP3 file to be received on my Android tablet

I also realised from another post mentioning MP3 files that MSPRO does actually play them happily. I don't know where I got the idea that it didn't....

Are ALL SETTINGS supposed to be updated with a Sync? I noted that the "Start Scrolling When Song Loads" setting didn't get updated, just as the Song Title Formatting didn't the previous Sync. Is this by design or is it a bug????

Cheers
John
Reply
#98
Yes, all song settings should be updated (but not application settings). So scroll settings should be synchronized, but song title formatting is not synchronized as that's an application setting, not a per-song setting.

Mike
Reply
#99
Johan,

I'm investigating the latest issue you described, but I wanted to say that I've made some significant improvements to the handling of directories when restoring backups. I now try to preserve the directories on Android and Windows 10 that are found in the backup file. There is one caveat for this: the file paths must start with the storage location in order for this to work. Otherwise I don't know which parts of the path to preserve. If you believe that information could be inferred from the file paths, I'm happy to discuss ideas for an algorithm for that. I imagine what may happen is that it could end up creating a larger number of subdirectories under the storage location than is required, so the question is whether that is better or worse than having the file moved to the storage location with a new folder. To test all this, I took your backup file, and changed the storage location that was extracted from the backup file to /storage/[sd card]/Partituren in the debugger, and verified that it recreated all of your original folders. So it works much better than before if you can have one parent folder for everything.

I should mention that the same holds true for synchronizing devices. You either need to have all files under the storage location, or alternatively, you have to set up folder mappings so that each device knows how to recreate the necessary subdirectories.

Thanks,
Mike
Reply
(06-28-2018, 02:51 PM)Zuberman Wrote: ... but I wanted to say that I've made some significant improvements to the handling of directories when restoring backups.

Great! Are these improvements already in the beta download?

Quote:The file paths must start with the storage location in order for this to work.

This is reasonable. The 'new' androids (well, since several years ago) impose restrictions on access of the storage devices so I'm starting to get rid of the idea of file placement all over the place while still expecting all tools to know how to deal with that.

Since I can access Android/data/com.zubersoft.mobilesheetspro/files from outside (adb, ssh, rsync) there's no reason for me to put the files elsewhere. My goal is to make music, not to make your life hard  Big Grin .
Johan
johanvromans.nl — hetgeluidvanseptember.nl — mojore.nl -- howsagoin.nl
Samsung Galaxy Note S7FE (T733) 12.4", Android 13.0, AirTurn Duo & Digit (Gigs).
Samsung Galaxy Note S4 (T830) 10.5", Android 10.0 (maintenance and backup).
Samsung A3 (A320FL), Android 8.0.0 (emergency).
Reply
I'm going to release a new beta version with those changes as soon as I finish the sync to folder functionality, which is very close to being done.
Reply
The sync to folder functionality is finished, but I'm going to work on switching everything to relative paths before the next release. Hopefully I can get it done tomorrow without too much effort if just modifying the database code is enough. That way you guys can test out the relative paths before a public release, as I want to be sure there are no unexpected problems from making that switch, as it will impact every user.

Mike
Reply
Version 2.2.5 of the beta is now available. There are quite a few changes in this release, including:

1) Added the sync to cloud folder option. I will cover this in additional detail further below.
2) Changed all paths in the database to be relative to the storage location when possible with forward slashes. This means if all files are relative to the storage location, you should be able to drop an Android database on Windows (and vice versa) and if the files/folders are all available, it will just work without any additional changes.
3) Differences in song/setlist note settings are now ignored if there are no notes
4) Added fixes to handle songs that use the same file multiple times
5) Added fixes for restoring library backups to preserve subdirectories
6) Added code to properly delete files if a one-way merge requires it
7) Modified behavior when prompt user is selected with a one way merge so that a prompt is now shown with "do nothing" on one of the sides.
8) Fixed merge errors related to chord pro and text files having a different number of pages on each device due to screen size differences.
9) (Windows 10 only) Fixed link point radius sizes to ignore scaling factors
10) (Windows 10 only) Keyboard focus is now assigned to the main library list automatically
11) (Windows 10 only) Added code to correctly handle UTF-16 or UTF-8 with BOM text/chord pro documents in the editor
12) (Windows 10 only) Added fixes for uploading files to Dropbox and Drive to ensure the correct file size is used
13) (Windows 10 only) Fixed issue where audio player would occasionally not play an audio track due to loading the track twice in quick succession
14) (Windows 10 only) Fixed issue on the group editor screen where the entries could not be seen if dark theme or night mode was enabled

A beta version of the companion has also been released at www.zubersoft.com/mobilesheets/downloads/MobileSheetsProCompanion.exe that has the changes for relative paths. It would be great if some testers could test that out as well just make sure I didn't miss anything.

With the new sync to folder option, you have to pick Dropbox or Google Drive, and then select a folder. If you have never synchronized to that folder before, MobileSheets will prompt you to upload your library to that folder. It doesn't matter if you already have files and folders there - you still have to let MobileSheets upload your entire library, as it compiles important information about every file (like hashcodes), sets up the folder structure it needs (it will try to replicate what is on the device if everything is relative to the storage location) and updates the database that is uploaded to reference the paths to the files in the cloud folders. After you've done this initial sync, future syncs will download the database and hashcode file and then quickly look for differences between all of the songs and files in the libraries. Using this precompiled list of hashes prevents MobileSheets from having to recalculate the hash for every file in the cloud folder (meaning the entire file has to be downloaded) to see if any changes have been made to it since the last sync. If the last modified timestamp changes, then the hash has to be recalculated (in case you upload a new version of a file). Handling the sync to folder option this way was the only way I could come up with to handle common usage scenarios while still keeping the sync as fast as possible (without having to download the data for every file every time).  For users that want to use this functionality please note: you CANNOT replace the database in the cloud folder with one from a device. That will break the synchronization functionality completely, as all of the file paths in the database would be incorrect. You can drop in updated versions of song files if needed (or new files to be imported), and the next sync will capture these, but the database and hashcode file must not be edited in any way.

Thanks,
Mike
Reply
I had a successful one-way Sync between my Win10 and Android device. Previously, I reported that the Scrolling "Automatic start" setting wasn't passed by the Sync, but I think somehow I had ticked that inadvertently. Certainly the setting was correctly updated on the Sync this time.

However, when I tried doing a folder Sync, the Dropbox Captcha seemed to go into a permanent loop and never verified. I am not sure if this is a problem with WIN10 MSPRO or DropBox itself....

Cheers
John
Reply
I encountered that as well, so I used the log in with Google option, which did work. I don't know what's going on with the Dropbox connection suddenly. I'll see if I can find any answers.

Mike
Reply




Users browsing this thread:
5 Guest(s)


  Theme © 2014 iAndrew  
Powered By MyBB, © 2002-2024 MyBB Group.