Page 1 of 3

New SID to MIDI conversion tool

Posted: 08/08/2017 - 6:29
by M3wP
Heyas!

I'm pleased to announce that very soon I will be uploading my new SID to MIDI conversion tool onto GitHub.

If there is anyone else interested in it, I will expedite the process.

So far, it has these features:

- Very accurate instrument/patch detection (complex waveform shaping detected)
- Very accurate timing reproduction (customisable BPM and Division)
- Allows for complex MIDI mapping (Drum Mode, Note Mode, Chord Mode)
- Can output dumps for playback on SID emulation of each instrument/patch or just the filters
- Open source!

It is part of my XSID file tool-chain, much of which is already up on GitHub. I have created the XSID tools to manage SID files outside of the emulation. An XSID file is already very much like a MIDI file but is for playback on a SID emulator and allows for playback without the emulation of the CPU and CIAs.

I must apologise for the name since I became aware that there is already a C64 composition tool with that name. I just don't have a better one at the moment.

Also, I must also apologise for the fact that the tools are currently only available on Windows. In theory, I could compile Linux and MacOS versions but I am missing a few technical skills required for translating the LibReSIDFP and LibSIDPlay library wrappers I've created to GCC and also the time required for actually doing it. I am investigating but it will take a while. I'm very open to help with this.

Let me know your thoughts!


Daniel.

Re: New SID to MIDI conversion tool

Posted: 08/08/2017 - 11:30
by Sanjo Ride
Hi M3Wp
This sounds great, I wrote my own tools that "listen" to the sid memory locations in Vice and control my DAW through LoopBE but there are obvious drawbacks to this approach such as no real midi noteOff's and tempo sync'ing problems.

Really looking forward to trying your converter. Any idea how soon ?

Re: New SID to MIDI conversion tool

Posted: 08/08/2017 - 19:52
by M3wP
Heya!

Well, I thought I might as well upload it now. I had to do the documentation and I wanted to more exhaustively test and add a few features but it functions so I put it up.

You'll find it at: https://github.com/M3wP/XSID.

At the moment, you will need to use the SID Convert tool to convert the SID files into XSID files. Once I have refactored everything into libraries, I will include this functionality in the XSID To MIDI application.

At the moment, pulse width and effect changes are not output as controller information in the MIDI files. I will get to that shortly but its only for reference anyway.

Please let me know about your experience with using the tools and if you find any problems. Any other contributions would also be greatly appreciated.


Daniel.

Re: New SID to MIDI conversion tool

Posted: 08/08/2017 - 20:36
by Sanjo Ride
Nice.

It all seems to build ok in Rad studio 10.1 Berlin. I'll have a play and get back to you.

Re: New SID to MIDI conversion tool

Posted: 08/08/2017 - 21:06
by M3wP
Hey, cool. You're a Pascal (Delphi) programmer?

If you're reading the source code, I apologise for the general lack of comments. I'm still getting to that... :wink:

Everything should be fairly easy to understand though, I've tried to name everything with meaningful names (well, except local variables - I'm a bit of a shocker with those).

FYI, the Pascal code has no problems compiling to 64bit however I can't say the same for the C/C++ libraries that are used. I haven't tried those as yet.

I did try to compile with FPC/Lazarus some conversions and I don't know if it was because I was using the 64bit compiler or not but the LZMA units didn't work there. I have to try to track down the cause. I have started my own conversion of the most recent LZMA units but I have a bug in my code which I'm having trouble with so I'm not using them as yet.


Daniel.

Re: New SID to MIDI conversion tool

Posted: 09/08/2017 - 11:46
by Sanjo Ride
"Hey, cool. You're a Pascal (Delphi) programmer?"

Yeah .. We're a dying breed mate :P

"f you're reading the source code, I apologise for the general lack of comments. I'm still getting to that... "

No Probs, I'm not one to judge, a lot of the time comments are pointless and people dont bother to read them anyway .. Kind of like all the Docs I write.

A did a conversion of ninja by rob hubbard. Works well it plays in FL Studio and all the tracks are nicely separated.

I did have to turn off runtime range checking though, otherwise it hangs when loading the XSid file.

Excellent work mate.

Re: New SID to MIDI conversion tool

Posted: 09/08/2017 - 18:15
by M3wP
Yeah, I used to write heaps of comments but it got to be a waste of time.

Oh I forgot to upload that change. The problem is in the LZMA units. You need to have range checking off in one of them. I'll check which and upload it. I'll also check that its not on in the Release build.

Thanks for your feedback.

Re: New SID to MIDI conversion tool

Posted: 09/08/2017 - 18:55
by Sanjo Ride
Sorry mate I should have mentioned to save you some time.

The unit is TLZMADecoder.pas

In particular this function :

function TLZMADecoder2.DecodeNormal(const rangeDecoder:TRangeDecoder):byte;

is trying to pass back a byte result that is > 256

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 4:00
by M3wP
Yeah, I already knew that but thank you very much. I just turn range checking off for the whole unit.

By the way, I got a 64 bit version working. I'm not sure if I'll upload the changes right now because my MSVC skills are poor and I'm not sure I did it in the best way there. I'll have to see. Also, there is no real benefit as far as I can tell. In fact, the output may be worse because there is MMX code in the ReSID resampler that has no 64 bit compatible (SSE or "pure") equivalent. I have to disable the MMX code for the 64 bit version because it is not compatible.

I've also just now implemented Pulse Width output and Effect flagging. I'm not quite happy with the results at the moment because I'll need to use different controllers (the general purpose ones instead of sound control ones). I'll post again when I've uploaded the changes. I've also made it so that you can flag an instrument for suppression when doing a dump all.

I'm considering allowing you to name each instrument and output that name with the MIDI track. Would you find this helpful?


Daniel.

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 5:09
by Sanjo Ride
Cool!
I agree that the 64bit version probably doesn't have any major benefit (That I can think of at least)

As for the naming of the instruments, for myself personally, I prefer to do that in FL studio as the name is overwritten when I replace the midi track with a VST, but someone else might like that feature.

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 5:40
by M3wP
Okay...

I've been using the tool a bit now and I must say, the results really are much more accurate than from SID2MIDIw which I've used in the past. I'm rather enjoying it.

And...

From the update log:

- Instruments are numbered from 1 instead of 0 to match MIDI track numbers.
- Instrument number included in more dumps.
- You can now name an instrument and that name is output to MIDI.
- You can optionally output Pulse Width change data to controllers (one or two, fine and/or coarse).
- You can optionally output Effect change data to a controller.
- You can now suppress an instrument in a dump all to MIDI.
- Double clicking the instrument list accesses the mapping for the selected instrument.
- A memory leak regarding the SMF files has been fixed.
- Project loading stops when an invalid instrument mapping is found.


The Pulse Width data can be output in one of two ways, single controller or double controller (or just not at all...). For single controller, the Pulse Width is scaled down to the range 0..127 and output on controller 16. For double controller, the scaled range is 0..16383 and is output on controller 16 for coarse (high bits) and 17 for fine (low bits).

The Effect data is output on controller 18. 0 means no effects, 32 for just Oscillator Sync, 64 for just Ring Modulation, 96 for both. I wanted to output them on two different controllers but I ran out (my synth uses a lot of them and most of them are already predefined).

I decided on allowing you to name the instruments and output that name to the MIDI track using text type '03' (track name) if you've given them one. The default is to name the instrument by the channel assigned. The name is displayed in the instrument list now, too.

You can get to the mapping by double clicking now! Woo hoo!

And I fixed a memory leak... Etc...


Daniel.

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 8:41
by M3wP
Whoops, there was a bug or two. I've fixed them now. Hopefully it all works as it should.

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 12:33
by Sanjo Ride
I'll Git a new version at some point and try it out.

My knowledge of midi is not as extensive as yours by the looks of things so I might have some quizzies.

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 14:52
by M3wP
Hey, no problem. I've downloaded a few guides and technical notes and use them as reference. I've also got my synth manual which is invaluable.

Initially I was trying to use millisecond timing using SMPTE (frames/subframes) instead of the Division but my DAW (Sonar) wouldn't accept it and neither would DirectMusic Producer (which I still use occasionally to screen oddities from MIDIs I get in the wild). I'm still a little frustrated by it but never mind. Higher Division values (say 480 or 960) seem to be just fine anyway.

I've decided that I'm not happy with the pitch bend calculation. I'm going to have a go at fixing it. I'm pretty sure its rounding in the wrong direction for negative values and I want to have a residual from the roundings.


Daniel.

Re: New SID to MIDI conversion tool

Posted: 10/08/2017 - 15:27
by M3wP
Hmm...

It seems that there is probably not a lot of difference in general. The difference is only going to be a couple of cents at most which is below the threshold of hearing. They say that even trained audiophiles can only hear the difference of 3 cents or so.

Anyway, the calculation is more correct now and it will make some difference at lower pitch bend range values. I'll upload the change later.