[trimbolim]

Greetings,

I'm getting started with digitizing my big band's library. We'd like to keep the library platform-agnostic, but I'm personally pursuing MSP on a chromebook to try to maximize value. I've had very good results so far.

We've got an existing library database web app that we can easily customize and extract extensive metadata from. We number our charts and have about 300 active charts in the book.

I'm about 50% done with scanning the book for my own instrument (Readdle Scanner Pro is impressive.) Initially, I'm going with one pdf file per chart, per instrument. I've been naming the files after the titles which makes things simple when importing to MSP but now I'm exploring the most efficient way to get the rest of the metadata (nicely formatted title, composer, arranger, etc.) from our existing database into MSP. From what I've seen, it looks like CSVs are the way to go for that, but I'm not 100% sure. Most of the docs and forum threads show CSVs being used to index fakebooks. I've tried that out with a few fakebooks and I understand that application. One awkwardness with the CSV method is the required field for "pages." In my case, I don't really need that since I'm not managing a large file (just one chart per file), I just want to import metadata. Can I use a wildcard like "*", or just leave it as "1-"?

As a possible workflow, I'm thinking:

- Scan the chart and name it after the chart number i.e. 131.pdf
- Have a like-named CSV (131.csv) that I generate from the database using these columns and values
  - Title: In the Mood
  - Composers: Glenn Miller
  - Song id: 131
  - Genre: Swing Dance
  - Collections: Big band library
  - Pages: 1-?

You see from the pages field that I've got a problem to solve. The guitar part has 2 pages and the piano part has 3. If I didn't have to worry about the pages field, I could make a global MSP metadata extract function in our web app for the whole band. It would make a zip file of 300 CSVs with all the metadata. I could just tell everyone to name their pdfs consistently and importing the CSVs would make their pdfs instantly searchable from various angles.

I could maintain a count of pages per chart per instrument in the database, but it just makes the whole scheme more fragile without any benefit. I'd like to find a way around it.

I've already got a set list builder in the web app, so exporting mss files for set lists looks trivial. I just have to settle on the actual pdf file management scheme.

Thanks for reading and for any insignts you can share.

Matt Trimboli

[itsme]

@Matt:
The keyword for genre is 'Genres'
When the column title keyword is unknown for MSP, the column is simply ignored. This allows columns with additional information, just in case

all kewords except 'Title' are plural and can take multiple values e.g.

Title;Custom Groups;Pages;Source Types;Keys;Composers;Years;Signatures;Albums
Ain't We Got Fun;;284-285;LS|MSCZ|2 S.|Text;Eb;Kahn|Egan Whiting;1921;4/4;CJB Fakebook 2.2

@Mike:
Pages: 1- 
imports only the first page
Pages: 1-99
seems to import the song correctly with all available pages, but page turning past the last available page causes an infinite 'blue circle' wait cursor (tested on Win10)
funny: when this wait cursor is displayed, page turning backwards goes correctly to the last page
it could be auto-corrected during import to the total number of pages in the PDF

[Zubersoft]

I can look into supporting 1-* for the CSV import where I replace any * that is found with the number of pages in the file. That shouldn't be difficult to do. I'll also add code to fix the page order to remove invalid pages.

Mike

UPDATE: 
Both changes are complete and will be included with version 3.6.7.

[trimbolim]

Thanks for the quick testing and feature update! I have finished scanning my 300 charts and did a few test imports of handmade CSVs (with precise page specifications) so that I can better understand the import workflow. I noticed that "batch import" doesn't seem to support importing CSVs. I had assumed that I would use that. Is there an option to avoid 300 individual CSV import actions in the UI?

And before I get any further down this road, I want to confirm that CSVs are the most appropriate method to load metadata for my use case. I feel like I might be abusing a feature meant for fakebooks.

Thanks so much,

Matt

[Zubersoft]

Unfortunately, no, there is no batch import option for CSV files as the current implementation allows user to pick which songs they actually want to import from the CSV instead of just importing the entire thing. I know some users want to be able to have CSV files where each entry can be linked to a separate file, so adding support for that is on my list. This would also support potentially concatenating all 300 CSV files you have into one that you could just import that one file to create all the songs. For now, there isn't a way to avoid having to select all 300 CSV files individually. I can certainly understand the desire to use CSV files primarily to populate metadata, so I would like to support a workflow for that, but I won't have a solution in the short term for this.

Mike

[BRX]

It's a pretty old request from several users including me. That's no criticism, Mike, since I know how full your plate is. But it's nice to see this request pop up again and I want to remind all MSP users who'd like that feature that the more often a feature is requested the more he moves it up on the todolist.

So here's a chance to give a +1  for this feature.

[hari]

Is there a current option to do this already? I have thousands of songs in a database. Generating the CSV files would be no problem, but how could I batch import those in some automated fashion?

[Geoff Bacon]

As Mike said earlier in this thread, there is no current way to import multiple files with a single .csv file - the .csv file only identifies each piece within a multi piece file.

Assuming you can generate a .csv file for a single file, I don't see why you couldn't use that to import the single song along with any metadata that you can generate from your database (I haven't tried it).
While this should do what you want for a single song, I've no idea whether MSP allows you to import multiple .csv files, with associated songs, at the same time.

Suggest you experiment with 3 songs.

Another option, but not so nice might be to merge several of your pdf's into a single file and generate a single .csv for them.
This would reduce the number of .csv files you would have to import if MSP can only handle one at a time.
Ideally, do this for songs that don't change.

Again, experiment to see what works best for you

Sciurius has some tools (https://github.com/sciurius/MSPro-Tools) that can manipulate the database - I've no idea whether these would be useful to you.

Geoff

[hari]

Thanks for your feedback @Geoff. I'm now using `pdftk` in my export script to cat only the specific pages out of larger PDF files with multiple songs to work around the lacking mass-csv import. That way I can use the "populate metadata from directories" option in the batch import. This at least imports my internal id from the database (custom), Title and Artist for each PDF automatically.