[FiddlerJeanne]

If it helps, I have both the original file on one tablet and the restored file on another tablet. I don't think it's related to specific files because I restored once, some files were missing. I restored it again (no new backup) and different files were missing. I restored again, and now I don't know if any files are missing. It just happens that the original missing files are in a couple setlists of stuff I'm working on right now, which is why I noticed them.

If you want copies of anything, I'll be happy to provide (but I'm new to this so specific instructions would be a good idea).

[itsme]

This shows once again that there is a real need for a function within MSP that checks if all files that are referenced from songs are available and readable.

[GraemeJ]

The new missing files finder is worth its weight in gold!  In my case, it found 32 missing files in 31 directories!  Most of these, I would have been unlikely to find on a 'using regularly' basis, they would only have been identified at the last minute - by which time it would have been too late, of course.  

I noticed that it would also find missing audio files as well as music charts - that's also very useful.*

I'll spend some time this evening sorting out and replacing all the missing stuff and then make a new backup.  It will be interesting to see if anything goes missing after a restore.  Good practice would indicate that running the missing file finder, immediately following a restore, would be the best idea (maybe, this could be included as part of the restore command?).

* Not sure if this is of any significance, but I did notice that an audio file, reported as missing from one chart, was still working with another chart that the same audio file was linked to.  So, the file was present, just the pointer to it was absent/corrupted.

[Rey_a]

I agree. This a very usefull tool.
I ran it on my production tablet and it says everything is ok.
I was just a bit surprised (worried) though, that it came to that conclusion with almost lightning speed. About half a second to verify some 1750 scores (no mp3 and such).

I will probably do a test an my backup machine and remove a file deliberately and see if it detects it ...

[sciurius]

I will probably do a test an my backup machine and remove a file deliberately and see if it detects it ...

Tried it already. Works .

[Rey_a]

So did I. Yes it does work. Unbelievable at that speed.
Great!

[FiddlerJeanne]

Nice. It found no missing files on either tablet. I did restore the database again yesterday, so looks like the bug may be fixed also (I hope). Yay!

[GraemeJ]

Perhaps I should expand slightly on my missing files report.

1 - Nearly all the missing charts were items that I would rarely, if ever, be likely to use. So, although reported as missing today, they could well have been lost some time ago, perhaps at the rate of only one or two per backup/restore. I doubt very much if they all went in one go.

2 - As a beta tester, I have obviously been using the backup and restore features an awful lot more than the average user - sometimes three or four cycles in a day. Although files should not be lost, this does mean that I have been exposed to any potential problem more than most would be. Also, it may be that coding errors (since picked up and rectified) resulted in files being lost.

3 - Four or five of the files reported as missing were associated with Placeholder pages (I've been experimenting with these recently). As I don't have any charts or audio associated with these, they're not so much missing as never there in the first place . However, it is possible to link files to a Placeholder - and I'm sure some people might do that - so, although slightly misleading for people like me, the report is accurate.

[Zubersoft]

Thanks for testing the feature out guys! I'm glad it's working well for everyone. I forgot to include a check to ignore blank files, so I'll go ahead and add that. The audio file problem you ran into is interesting Graeme - I'm going to have to do further testing to see if that behavior is specific to audio files.

Thanks,
Mike

[GraemeJ]

Update on missing file/s thingy:

I took my earliest MSP backup, from 27th March 2015, loaded it into Companion and set about extracting the missing files. These number 24 in total, disregarding the Placeholders, which contained no files anyway.

All of the missing files were in the backup, so I extracted them in the usual way, this gave me a set of directories, each one containing the PDF file for the piece concerned (there was one chordpro file as well, not usually used, but I was just checking it out).   I then moved the individual PDF's to another directory, from where I intend to do a batch import into MSP (having first deleted the file references in MSP itself).

NOTE: The function to batch delete missing file references in the missing file search tool is a good idea, but there is no option to retain anything one might want to keep - in my case, the Placeholder pages - so I have to delete each individual reference on the tablet from the library page.

Of the 24  files that should have existed, only 11 were actually there, the other directories were empty. If nothing else, it would appear this is a problem that has existed for some little time and not something that has suddenly popped up.

Checking the backup in Companion, the charts that were missing certainly appeared to be there, as there was a number in the 'Pages' column for every one of them.  Of course, that's as far as I can go, since it is not currently possible to view the pages in Companion, but Companion certainly believes these files exist - even if that's not true.

My next move was to import the recovered files, using the batch import tool in Companion.  I had to rename each one, as MSP still believed the original file existed. There was a problem with the chordpro file, see the screenshot.  Then I trawled through the library and deleted every title that reported a missing file.  Used the missing file tool again and there were a couple where the missing file was an audio file and I wasn't sure what to do about them.  My Placeholder files were still there as well.  In the end I made a note of the songs concerned and used the delete command to get rid of everything. 

I replaced the Placeholders and another run of the tool reported nothing missing (other than the Placeholders not having any associated files) so it would seem that everything on the tablet was hunky-dory.

Using Companion, I made a completely new backup, including the audio files. After the backup completed, I closed down MSP and Companion.

I fired up Companion again and loaded the new backup file.  This seemed OK, from what little one can tell in this mode.  I started MSP again and set Companion to do a restore of the backup I had just made.  Then I made myself a stiff whisky and waited for it to complete.

A couple of stiff ones later, the backup had completed and I ran the missing files tool - A FILE WAS MISSING [Image: angry.gif] !  Admittedly, an audio file, but missing never the less.  It was definitely there before I started all this nonsense.

It's late - three o'clock in the morning - I can do no more tonight.

Two screen shots attached - one from the tablet, showing the missing file it reported after I had gone through all the above stuff (you can disregard the Placeholders, they have no associated files) and the other showing an error when trying to import a chordpro file into MSP via Companion.

[Zubersoft]

Thanks for the update Graeme, and testing all that out. I believe I've found the reason for the chordpro error. As for the missing audio file, that is definitely concerning. I will try to do some more testing with audio files in backups to see if I can replicate errors there. It will be important to find out if your backup file contains the audio file, or if the audio file is missing from the backup file. The only way to know for sure is to manually transfer that backup over to your tablet, and restore it from there. If the audio file is missing after restoring it, then we know there is some kind of issue with backing up from the companion that needs to be addressed. If the audio file is present, then we know that there is an issue with restoring backups from the companion that I need to look into.

Thanks,
Mike

[GraemeJ]

I transferred my last backup to the tablet and then did a restore from there. Then I checked for missing files.

The GOOD news - no missing files at all.

I then did another restore of the same backup file, but using Companion. Checked for missing files and the *same* MP3 is missing.

The BAD news - this is obviously a problem related to Companion.

I am now doing another restore from the file on the tablet - I'll report back when that's completed.

I will say one thing, this would have been a near impossible task before Mike introduced the missing file finder.

NB: While fiddling around with all this stuff, I noticed there seemed to be quite a lot of duplicated files. I was aware there were some and I had put that down to my actions when beta testing, but there seem to be many more than I would have thought were for that reason. I'm wondering if this is another Companion thing? I will find and delete all the duplicates and then keep my eye open for any more that crop up, but that's a long job and will have to wait for another day.

[GraemeJ]

Update:

The restore from the tablet worked perfectly, no missing files. Got to be Companion.

[jfree]

What is the 'missing files finder' app? I went to open a song and it was missing yesterday. I've only backed up to ext card on the tab and dropbox. I'm not even 100% certain this was after a restore. I believe it was from the card. I haven't noticed any others missing yet. I've got about 850 songs so who knows when it may show itself if there are others.

[GraemeJ]

You'll find the missing file finder in the menu - Settings | Other Settings | Diagnostics