[itsme]

My saved filters get lost frequently because user_filters.xml is replaced with a newly created empty file of 403 bytes size. Probably it's not an issue of MobileSheets alone but is caused by some kind of interaction between MobileSheets, GoogleDrive Sync and my specific workflow. Alas I cannot reproduce it reliably so I'm asking for advice what I could try or investigate to narrow down the cause.

I use MobileSheets on two Win10 devices, a detachable that I use in tablet mode on stage and a laptop that I use as my usual work PC to edit songs and so on. The MobileSheets version usually is the same, most of the time the latest released version. Song files are in an identical folder structure within the MobileSheets storage location (d:\HerbertsGoogleDrive\_MobileSheets\) which is synchronized between both devices by the GoogleDrive sync program. I use several MobileSheets databases that I switch from time to time and that I copy manually between both devices every now and then. Losing the user filters seems not to be connected neither with switching the database nor with copying a database file.

The issue is not new, it happens for a long time, since many MobileSheets versions. I lived with the workaround to overwrite the empty user_filters.xml  with a correct one from an older backup, but I'm getting more and more annoyed by that issue so I decided to try to get it fixed.

Any idea what I should investigate? What causes MobileSheets to create a new, empty user_filters.xml (d:\HerbertsGoogleDrive\_MobileSheets\user_filters.xml in my case)?

[Zubersoft]

This is going to be a tough one. We need to identify what series of actions results in the file being erased and an empty file being put in its place, but if it's not easy to reproduce it, that's going to make it difficult. Let me ask this - can we remove Google Drive sync from the picture for a little bit to see if the issue occurs without that? We just need to identify whether that has any impact at all. If the problem happens even without Google Drive Sync in the loop, then I can focus my testing on just MobileSheets itself in a controlled environment.

I'll review the code, but I would imagine the only thing that might cause an empty file to be generated is if MobileSheets tries to write to the file but that fails for some reason.

Mike

[itsme]

I'm pretty sure that GoogleDrive sync is a main part of the problem and the issue will be gone as soon as I run MobileSheets without it.
I don't change my user filters very often, so I don't expect that MobileSheets tries to write into user_filters.xml every time it is accessed

Looking into my MobileSheets folder I find this:
   
When I started this thread, I replaced the empty file with a correct one from a backup. I copied that correct file to user_filters_2024-03-14.xml to simplify the workaround of copying and copied the wrong file to user_filters_empty.xml. Note that the time stamp of user_filters_empty.xml matches the time when I started this thread.
user_filters.xml is 403 bytes again, the time stamp is 2024-03-06 8:24, so it happened again yesterday morning.
The <somenumber>user_filters.xml with 0 bytes size make me think that the sync is involved somehow.

GoogleDrive keeps a history of the last 100 versions of a file
   
I downloaded the last saved versions until correct and incorrect changes occurred
   
The local time stamps don't help, they show when I downloaded the files.
But GoogleDrive_FileVersions shows that the last correct change _097 -> _098 happened 2024-03-15, _098 -> _099 on 2024-03-17 saved an empty file
From 2024-03-15 to 2024-03-17 I was away from home, so I am sure that only my Detachable was in use. I cannot tell if on my main laptop, that I left at home (switched off or in standby) MobileSheets had stayed open.
GoogleDrive's activity list shows the time stamps and makes me think that unnecessary file changes took place
   

[itsme]

Comparing downloaded file versions shows me that user_filters.xml also is written when MobileSheet's "search" function is used:
   
No idea, if that has any effect on the issue. It just explains that MobileSheets writes user_filters.xml more often than I thought before.

[Zubersoft]

MobileSheets will update user_filters.xml anytime MobileSheets is minimized (or right before it is closed) to ensure the filters are saved if the application is suspended in the background (which Windows can do whenever it decides it wants to, at least with the Microsoft Store version of MobileSheets). This way if a user automatically loads the last used filters, they are properly persisted and available. I'm guessing something is occasionally going wrong with this on your device that is causing the filters to be saved with only the contents of the temporary filter, without all your other saved filters. Figuring out when/how that occurs is going to be quite difficult though. There is code that tries to open the existing filters file to edit it, but if this fails, it will generate a new file with only the temporary filter. I'm guessing this is the code that is triggering. So something is causing MobileSheets to fail to read the existing user_filters.xml file. We need to figure out how and when this occurs.

Mike

[Zubersoft]

I am going to make a change for the next update on Windows and we can see if it helps. Right now, one thing I do is check for a non-zero file size to make sure the existing user_filters.xml file is valid. However, it's possible this is problematic in some scenarios. So instead of doing that check, I'll just make sure that the document has some of the expected elements in it, and if it does, then it's valid.

Mike

[itsme]

This issue still bothers me. 
I'm on Win10 and manage my files myself, so I have set my MobileSheets folder to d:\HerbertsGoogleDrive\_MobileSheets\
My MobileSheets folder with all the song files is synchronized by the GoogleDrive desktop program.
Databases are c:\Users\her\AppData\Local\Packages\41730Zubersoft.MobileSheets_ys1c8ct2g6ypr\LocalState\
They are not synchronized, I copy them manually when it's required.
Filters are stored in d:\HerbertsGoogleDrive\_MobileSheets\user_filters.xml

What I'm thinking about:
Shouldn't user_filters.xml be stored with the databases and other program-wide setting files like default.xml or libraries.xml in LocalState, so that they are not touched by the GoogleDrive desktop program?
annotation_favorites.xml and stamplists.json are also stored in d:\HerbertsGoogleDrive\_MobileSheets\ like user_filters.xml, but if they had a similar issue I wouldn't notice it because I don't use features that need them.

[Zubersoft]

So I assumed most users would prefer to have most of the application files stored in their storage location. I can't store the database in the storage location due to UWP SQLite library limitations, otherwise I would be storing it in the storage location as well. While I understand why, in this scenario, you'd prefer me not to do that, I could certainly see in other situations that users would prefer the opposite. So I'm not sure exactly what I should do here. Changing the behavior is also not going to be great - if I change the code to look for the file in the default application directory instead of the storage location, then users with a custom storage location would lose all of the contents of that file. So then I'd have to add update code that moves the files from the storage location to the default application data directory upon updating to the new version but only if the two are different. I'm just worried some issue might occur during all of this on top of the extra work it creates. I'm not opposed to making the change though.

Mike

[Zubersoft]

After thinking for it a bit, I'll just make the change. It's not a big deal and it's probably best to be consistent with having all the .xml files in one folder.

Mike

[BRX]

I like it. It had me bothered a bit, too.

[Zubersoft]

I'm only making this change on Windows though, to be clear.

Mike

[Swiss1000]

There is a reason for the "loss" of data: If you include the "Settings" file or restore "settings" and the path is not the same between the previous and downloaded ones, the data seems to be lost. In reality, either the library does not have the same name or the path has become different. And the user who "restores" loses his data. 

If it could help...

PS : despite Itsum's opposition: for me, restoring settings should absolutely be separate from the backup file.

[Zubersoft]

This post has absolutely nothing to do with that at all. Nowhere in here did we mention backup files or issues with multiple libraries. This problem was purely due to the location the files were saved in relation to the use of an automatic cloud synchronization service. I understand that you have your opinions on the topic of settings in relation to multiple libraries, but please do not post and derail other threads that are completely unrelated. I also disagree that settings should be separate from backup files - that would just cause users to always have to manage two separate backup mechanisms everytime they want to back things up, and it doesn't make sense at all for users that only have a single library (which is the vast majority of users). If you want to discuss this more, we can do so back in the other thread.

Mike

[itsme]

Many thanks for trying to help, but this reported issue is not at all related to backup & restore

[Swiss1000]

This post has absolutely nothing to do with that at all. Nowhere in here did we mention backup files or issues with multiple libraries. This problem was purely due to the location the files were saved in relation to the use of an automatic cloud synchronization service. I understand that you have your opinions on the topic of settings in relation to multiple libraries, but please do not post and derail other threads that are completely unrelated. I also disagree that settings should be separate from backup files - that would just cause users to always have to manage two separate backup mechanisms everytime they want to back things up, and it doesn't make sense at all for users that only have a single library (which is the vast majority of users). If you want to discuss this more, we can do so back in the other thread.

Mike

OK I understand, but a refexion should be managed about "what setting are machine or hardware installation" and what settigs could be shared in anny other tablet ?
I I restored a database in Windows 11 and that chaged my private directory it could be hazzardus isn't it ?