MobileSheets Forums

Full Version: CSV Merge
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I'd love to be able to merge CSV data into MSP. 

My workflow are:

1.
- design charts in iRealPro
- export playlist to a Linux box
- process using irealcvt to images
- process using irealcvt and scripts to extract key, tempo and style
- collect all data in a CSV file


2.
- write scores in MuseScore
- convert to PDF 
- extract key, tempo and style 
- collect all data in a CSV file

From here I transfer the directories, including images, PDFs, associated audio files and the CSV file, to my tablet. I'm then able to import all songs into collections ignoring duplicates but am then faced with the manual tasks of:

- merging in meta data that has changed
- removing songs that are no longer referenced in the CSV file
- manually referencing audio files 

I've tried combining all songs into a single PDF file and importing the metadata from the CSV file. This works but creates duplicates if performed again meaning that the collection needs to be removed first with the resulting loss of historic set lists.

I'm perfectly capable of writing code to modify the database to my requirements but would like to know if CSV merging is on the development time line or whether it's possible to assist the development team in adding this. 

Kind regards, 
Terry.
Sorry for the long delay in responding Terry. I haven't really given CSV merging too much thought to be honest. I guess in some ways it's similar to the Sync Library features. I can look into this next year after the iOS version is released. If a lot of users will find this functionality useful, I can raise the priority for when it's implemented. As far as offering your assistance, do you mean with establishing the requirements and design, or do you mean with the actual coding?

Thanks,
Mike
(07-02-2019, 02:31 PM)Zubersoft Wrote: [ -> ]Sorry for the long delay in responding Terry. I haven't really given CSV merging too much thought to be honest. I guess in some ways it's similar to the Sync Library features. I can look into this next year after the iOS version is released. If a lot of users will find this functionality useful, I can raise the priority for when it's implemented. As far as offering your assistance, do you mean with establishing the requirements and design, or do you mean with the actual coding?

Thanks,
Mike

Hi Mike,

Thank you for your response.  I've progressed my ideas on this subject and, while I've solved my immediate problems by directly editing your database using a simple scripted SQL stream rather than a CVS file, I believe I may be able to be of some use to you.

My thinking now is that MSP should export an API for manipulating the database and perhaps performing other tasks.  My database edits, especially adding new songs, are quite extensive and I'll need to check your schema after every database version change.  An API would present a consistent interface and remove that need and would make user code contributions much simpler; I could easily contribute the CVS merging code without touching your code for instance.  Using an API would also allow other apps to interact with MSP and I'm currently developing one that would benefit from this.  Integration into Tasker would also be possible and would bring MPS into the world of automation.

I worked in the software industry for 24 years and am now working as a musician while dabbling in software tools to make my life easier.  I'm open to collaboration and would happily assist with the design or coding if it would help get the job done. 

Kind regards,
Terry.
Terry,

Having an API would be wonderful in terms of the contributions MobileSheetsPro users could make to extending the application and even support automation (as you have described). The main difficulty is in figuring out how to handle this well in Android across all supported OS versions (as some are more restrictive than others). I haven't done a lot of research into implementing a plugin-based architecture with an Android application, and the little I've seen points to using services with AIDL. I'm not really sure how well that will fit in with the way MobileSheetsPro works, and my fear is that in trying to support something like that (which would be a considerable amount of work), it could have a negative impact on the application as a whole, as I could introduce bugs or performance issues. The question is how much the majority of users will benefit from being able to add plugins, because I would be spending a lot of time and effort working on that instead of other features which have also been requested. If plugins become commonplace, it also means that users could start encountering bugs and issues due to plugins, but they would come to me for support (greatly increasing the complexity of offering support, which is already a considerable time investment). 

I also would need to see how easy it is to support plugins in a C# UWP application. From what I've seen so far (https://social.msdn.microsoft.com/Forums...=wpdevelop) it points to having to write code to support communication with external applications. So it seems to be similar to Android.

Perhaps if we clearly define what should be possible through IPC with an external application, it may be possible to add support for a more limited set of functions related solely to the database. It still would be a large effort I imagine, so I'm not sure where it will fit into the schedule priority wise. It really depends upon how much demand there is for something like this in order for it to be worked before other things that are commonly requested.

Thanks,
Mike
Mike, 

Thank you for your thoughtful reply. 

I agree that AIDL is overkill and too much reliance would be placed on the quality of the plugins. 

I've recently moved my iRealPro code onto android and am now happily synchronising playlists directly to your database. I've utilised irealcvt inside a Termux environment to generate the images and Tasker as the glue between the applications - an export from iRealPro lands me in MS with an updated Collection that matches my playlist, including Keys, Composers, Style, Tempo and some MIDI commands. This is working well without an API and the database interaction is likely too complicated for one. I'll have to live with database version changes. 

However, another application that I'm porting needs a lot more configuration for each song and there isn't the wiggle room in your database to squeeze in my data. That's not a problem but I'll need to write a separate app and the interaction between the apps won't be pretty. I'd love to be able to quickly switch from one to the other maintaining state. 

How about using Android Activities? A few published Activities inside MS would allow the app to be opened straight to a Song, Collection etc. Being able to broadcast calls to Activities from Smart Buttons or on receipt of MIDI commands would allow connections to all number of other apps. To be able to talk to anything from MS I'm currently limited to MIDI, a file export or a URL. Whilst it's possible to declare Intents to receive the latter two, a broadcast activity with arguments filled by variable expansion would be significantly cleaner and more flexible. As I said before, connection to Tasker and other automation tools opens up all sorts of possibilities and this would be a first step.

I'm about to start on the MIDI part of this project and wondered whether your implementation of  Google MIDI connections is limited to USB and BTLE or do you allow inter-app software routing. I guess I'll find out soon enough. 

Kind regards, 
Terry.
Terry,

If you want to open MobileSheetsPro and have it load a song or setlist, that should be pretty easy for me to support. I would just require a few flags (to specify that a song/setlist should be loaded and which of the two to load) and the ID of the song/setlist. I could also support loading based upon name if that would help in any way.

If you use the Google MIDI library under Settings->MIDI Settings->Configure MIDI Connections, then you can utilize virtual MIDI ports to allow inter-app software routing. This has been tested with the Metronome by Soundbrenner.

Mike
Mike, 

Thank you. Launching straight to a Song/Collection/Setlist by ID would be fantastic and would allow me to implement an app to the side of MS with clean interactions. I'll have your database open anyway so have no need to address by name but I imagine you'll implement it for completeness anyway. Shout if you'd like it beta tested. 

I'm pleased to hear that inter-app MIDI works and I'll be utilising that functionality soon. I don't have a feel for Android MIDI latencies yet but I'm hoping they're not too bad. 

Kind regards, 
Terry.