Now listening to: L’univers De La Mer


Dominique Guiot – L’univers De La Mer (1978)

Final (?) notes on programming the NSX-39


The Pocket Miku has turned into much more of a learning experience than I’d expected it to be. The regular MIDI sections are pretty much the same as for the other instruments I have, such as the Korg Kaosillator and the A-300 MIDI controller keyboard, as well as the software synth packaged with Java. One of the big hurdles was in trying to read incoming messages from the Miku and aligning them with data requests made by Java to obtain the contents of the lyrics and command slots.  Then there was that whole thing with SysEx messages. Since I don’t use a sequencer app on my PC (yet), system exclusive messages aren’t something that I’d encountered before. As mentioned in an earlier post, SysEx is used to access hardware properties that you can’t normally get to via regular MIDI messages, although it’s possible for the manufacturer to make them available as ChangeControl options (if so, CC is faster than SysEx). This means that SysEx messages are device specific, and songs that you write that use SysEx for one instrument won’t sound the same if you run them on something else.

One of the bigger learning opportunities came when I decided to reverse engineer the command slots. The Pocket Miku has 128 command slots, which are blocks of memory that hold instructions to be executed when the associated event is encountered. Examples include playing “Hello” on power-up, initializing a lyrics string when the “A”-“E”-“I”-“O”-“U” buttons are pressed, or doing something customized by the user. You can see the contents of all of the command slots by sending multiple “Command Slot # Requests”.

Let’s start out simply programming a command slot first. The best example is from the Perfect Guide, which is to add a “keypressed beep” to the SHIFT key (which I’ve discussed before).

The SHIFT key is button SW7, and the command slot for SW7 pressed is #17 (0x11), CS_SW7D. So this is the command slot we’re going to write to. To program the NSX-39, we need to use SysEx (0xF0). First, we need the SysEx header for accessing the NSX-39 features:

F0 – Start of SysEx message
43 79 09 11 – Header for programming the NSX-39

After this portion, we specify the instruction to execute. 0x01 will request the version number, 0x0B lets us request specific command slot strings. To SET a command slot, the command is 0x0C (without the “0x” part).

0C – Request command slot # contents
11 – CS_SW7D – slot # for SHIFT pressed

I’ve finally got a handle on the next portion. In MIDI, bit 7 (the most-significant bit of an 8-bit byte) specifies whether the byte is data, or a message status byte. The MIDI instruction for NOTE ON is 0x90 (1001 0000). Putting a NOTE ON instruction in the middle of the SysEx string will result in an error – “Illegal SysEx character. Only F0 and F7 are allowed.” To get around this, we need to convert the instruction from three 8-bit bits to four 7-bit bytes with padded leading 0’s.

Specifically, for our SHIFT key, I want to use NOTE ON to play the open triangle from the percussion instrument set on channel 9 of the general MIDI instrument sound bank, and I’ll randomly pick a velocity of 0x65. Since the triangle is key 81 (0x51) on the keyboard, the MIDI message would be

99 – Note on for channel 9
51 – Open triangle
65 – velocity (volume)

99 51 65

To convert this, turn it into binary:
10011001 01010001 01100101
reform as 7 bit bytes with leading 0 padding:
0000100 1100101 0100010 1100101
add leading 0’s to each byte to make 8-bit bytes again:
00000100 01100101 00100010 01100101
Then convert back to hex:
04 65 22 65

To complete the SysEx string, add 0xF7 as “end of SysEx message”.

The full message would be (in Java, no spaces):
F0 43 79 09 11 0C 11 04 65 22 65 F7

If we’re using sequencer software like Acid Studio or GarageBand, we’d just activate the SysEx box and type this message in as-is. In Java, we need to use a SysEx object kind of as follows:

SysexMessage sysexMessage = new SysexMessage();
sysexMessage.setMessage(byteMsg, byteMsg.length);
nsxRcvr.send(sysexMessage, -1);

Ok, so far so good.
Let’s get the string back. As mentioned above, we need to use SysEx to talk to the NSX-39, and the command we want to send is 0x0B – Request Command Slot.

F0 – Start of SysEx message
43 79 09 11 – NSX-39 header
0B – Request command slot contents
11 – Desired slot # (CS_SW7D = SHIFT pressed)
F7 – End of SysEx message

F0 43 79 01 00 0B 11 F7

This will cause the NSX-39 to come back with:
F0437909111B04652265F7

If we add white space to make it more readable, and divide it into lines with comments:
F0 – Start of SysEx
43 79 09 11 – NSX-39 header
1B – Command Slot contents (note there’s no slot number)
04 65 22 65 – The contents of the returned slot
F7 – End of SysEx

For amusement value, let’s look at another response string returned from the NSX-39:
F0437909111B0A0801000A060000F7

F0 – Start of SysEx
43 79 09 11 – NSX-39 header
1B – Command Slot contents (unknown slot)
0A 08 01 00 – Command 1
0A 06 00 00 – Command 2
F7 – End of SysEx

This anonymous slot executes two commands. From this example we can see that the commands are always 4 bytes each. We’ll have to look at the table on page 23 of the NSX39MIDIGuide.pdf to get the meaning of these commands:

0A 08 01 00 – Activate DOREIMI mode (play “Do Re Mi” on the keyboard)
0A 06 00 00 – Turn off NSX-1 compatibility mode

I’ll tell you now that this is slot 0, CS_MIDI_INIT, which activates on power-up, after the NSX-1 initialization completes. That is, whenever I turn on the Pocket Miku, the NSX-1 chip initializes, and then CS_MIDI_INIT is activated. The CS_MIDI_INIT command slot contains “DOREMI Mode On” and “NSX-1 Compatibility Off”. That’s it. (Although, when CS_MIDI_INIT finishes running, it activates command slot CS_MIDI_INIT2, which in turn calls CS_MIDI_INIT3, and you could put more commands in those 2 slots if you wanted.)

If you’ve got the MIDI Guide pdf open, then look at the rest of the commands. Most of them are fully-spelled out and start with 0x08 – 0x0F. The exceptions are DIRECT_MIDI_1 – DIRECT_MIDI_3 and DIRECT_MIDI_OUT_1 – DIRECT_MIDI_OUT_3. The first set sends direct data to the NSX-1 from the NSX-39. The second set is used to treat the NSX-39 as a MIDI controller by sending the data directly to the USB port.

The importance of these exceptions lies in the fact that the data is 7-bit encoded. If we go back to the contents of the CS_SW7D slot, we had obtained:

04 65 22 65

According to the NSX-39 MIDI Guide, this is a DIRECT_MIDI_3 command. To decode it, convert it to binary:
00000100 01100101 00100010 01100101
Take the right-most 7 bits:
1100101
Trim the remaining trailing 0 (and the space) and add those 7 bits back:
00000100 01100101 001000101100101
Remove the 8 right-most bits and call this D2:
D2 = 01100101 = 0x65
This leaves us with:
00000100 01100101 0010001
Take the right-most 6 bits:
0010001
Trim the trailing 0 and space and add those 6 bits back:
00000100 01100101010001
Cut the right-most 8 bits and call them D1:
D1 = 01010001 = 0x51 = 81
This leaves us with:
00000100 011001
Take the right-most 5 bits:
11001
Trim the trailing 0 and space and add those 5 bits back:
0000010011001
Take the right-most 8 bits and call them D0:
D0 = 10011001 = 0x99 = NOTE ON channel 9

Putting D0 D1 D2 in order:
99 51 65 = Note On, channel 9, Open Triangle, Velocity = 101

Granted, we could have predicted this just by looking at the encoded data, but knowing how to decode everything in the return strings means that I can now write encoder-decoders in Java, and have all of the command slots in human-readable form within a matter of a few seconds. This is important when looking at things like the string for CS_SW6U (VIBRATO button released):

CACCO_START, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 32)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 28)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 1E)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 14)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 0A)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 05)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 04)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 03)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 02)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 01)], TIME_WAIT 000013, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 00)], TIME_TAIL, DIRECT_MIDI [CONTROLCHANGE Ch 0 (MODULATION 00)], CACCO_END_TIME 0F

Try reading that as pure hex code. (As a hint, “cacco” is the Japanese word for “parenthesis” or “bracket”.)

I may be the only person who cares at all about this. I whined some more on the U.S. Yamaha forums, and was told that the NSX-1 chip is only available in the Asian region, not in the U.S., and that I have to try finding the documentation from a server in Asia somewhere. So, this isn’t really on anyone’s radar, if they’re not Japanese. Now that I have my Java applet running, I’ll put everything away to focus on something else (maybe video games).

Programming the Pocket Miku, Details, Part 6


Most of what I’d done in the last blog entry was to summarize the translations of Polymoog’s MIDI guide, which brings me to page 16 of that guide. Most of what follows concerns manipulating the pointer to the phoneme character strings that Miku “sings” when you touch the stylus to the carbon keyboard. You can issue these commands either as a “command” (0D) or as an instruction string placed in one of the command slots (0C) to be implemented when a particular button action occurs (pressing “A”, or releasing SHIFT + “U”).

F0 43 79 09 11 0C … F7, or
F0 43 79 09 11 0D … F7

These character pointer operations include setting the pointer (MOJI_SET_KASHI = 09 05 00), incrementing the pointer (KASHI_POS_INC = 09 01 00 00), selecting the pointer position (0 to 63: KASHI_POS_SEL = 09 02 00 nn, where nn is 00 to 3F), selecting the lyrics slot (KASHI_SLOT_SEL = 09 03 00 0n, where n is 0 to F), and phoneme character entry (MOJI_SEL = 09 04 00 nn, where nn is 00 to 7F from the character table on page 10 of Polymoog’s MIDI guide). (Japanese translations appear below.)

Page 17 gets into some of the other commands from the command table on page 23, such as NOTEON, NOTEOFF, ADV_PB (pitch bend), VOL_UP, VOL_DOWN, etc. They work as you’d expect them to as regular MIDI messages.

F0 43 79 09 11 0D 0A 02 00 00 F7 – Like pressing the Volume Up key on the front panel.

VOL_SET: Set volume level (0-8)
VOL_UP: Increase volume one level
VOL_DOWN: Decrease volume one level

KEY_UP: With NOTEON or REVOICE,
KEY_DOWN: changes the keyboard up or down one octave.

TUNING_STOP: Lets you calibrate the carbon keyboard.
TUNING_START:
TUNING_TOGLE:

NSX1_GOKAN_OFF: Turns NSX-1 compatibility mode on and off.
NSX1_GOKAN_ON:
NSX1_GOKAN_TOGLE:

MODE_DOREMI: Puts keyboard into Do-Rei-Mi mode.
MODE_NORMAL: Miku sings from the lyrics tables.

MIDI_RESET: (Panic) Resets the phoneme chip.

ROM_RESET: (Initialize) Returns the Flash ROM settings to the defaults .

Section 3-6 on page 20 discusses the concept of CACCO, which is the Japanese word for “brackets” or “parentheses”. This mode is used to assign a string of unrelated commands to a specific command slot as if they were within a pair of parentheses. Once CACCO_START is encountered, the NSX-39 will execute the subsequent commands until encountering CACCO_END.

The example is to assign certain commands to the SHIFT button when pressed (CS_SW7D = 11).

0C 02 00 00 – CACCO_START (command list start)
09 02 00 00 – KASHI_POS_SEL(0) (reset lyrics list character pointer to position 0)
0C 03 00 0A – CACCO_END (command list end)
0C 02 00 00 – CACCO_START (command list start)
09 05 00 00 – MOJI_SET_KASHI (set pointer to play current character)
08 01 00 00 – REVOICE (replay the character)
09 01 00 00 – KASHI_POS_INC (point to the next character in the lyrics list)
0C 03 00 05 – CACCO_END (command list end)

From these commands we can see that when you press the SHIFT key, Miku will sing the first character in the currently selected lyrics slot.

The full SysEx message would be (all one line, no comments):

F0 43 79 09 11 // Start of SysEx message (fixed)
0C 11 // Change Command Slot for CS_SW7D
0C 02 00 00 // As given above
09 02 00 00
0C 03 00 0A
0C 02 00 00
09 05 00 00
08 01 00 00
09 01 00 00
0C 03 00 05
F7 // End of SysEx message (fixed)

Section 3-7 (page 21) describes timing. You can specify a wait time (TIME_WAIT = 0D nn nn nn, where nn can be from 00 to 7F). A value of 500 is roughly 1 second. You can cancel wait time with TIME_CANCEL (0C 05 00 00), and specify a TIME_TAIL (0C 04 00 00), which lets the timing buffer clear out. The pattern would look like:

CACCO_START > A > B > TIME_TAIL > C > D > CACCO_END

The example on page 22 is to assign a string to VOL UP + VOL DOWN + “A” (panic reset). (All one line, no comments.)

F0 43 79 09 11 // Start of SysEx message (fixed)
0C 32 // Change the command slot for CS_SW89_1D (Up + Dn + A)
04 65 12 66 // Direct MIDI 3 command mode to play a music note (see below)
0C 02 00 00 // CACCO_START
0D 00 00 C4 // Wait 200 ms
0A 07 00 00 // MIDI_RESET
0C 04 00 00 // TIME_TAIL
0A 07 00 00 // MIDI_RESET resets even with Cancel
0C 03 00 0F // CACCO_END
F7 // End of SysEx message (fixed)

Note: The 04 65 12 66 portion is a repeat of the 7-bit direct MIDI format, which takes “99 49 66” (Note Number 49 from the Drum Kit sound bank ON with a velocity of 0x66) and converts it to “04 65 12 66”.

Everything else is tables. Page 23 shows the command table. Translations:
“kashi” is “song lyric” (used to refer to the current lyrics string).
“Gokan” is “compatibility mode”.
“cacco” is “parentheses”.

Pages 24 to 27 describe the operations of the Command Slots. These are addresses within the Flash ROM pointing to instructions to execute when specific events occur, such as pressing or releasing one of the front panel buttons, power up (kidou), turning audio on/off, or activating the USB port. The slots themselves are listed in the tables on pages 28 to 30. I think most of this is self-explanatory so I’m not going to translate the tables. If there are enough requests for it, though, I’ll reconsider.

Programming the Pocket Miku, Details, Part 4


XG
XG is something that Yamaha came up with, and refers to Yamaha EXtended General MIDI. This is actually the easiest way to change Reverb, Chorus and Variation, although most of these effects don’t do much that I can tell (Reverb is the most noticeable). To determine what each address changes, you need to refer to the subsequent XG Parameter Change Table.

First, look at the subtable for the XG system. Address 00 00 7E is “XG System ON”.
You need this one.

XG Parameter Changes
F0 43 1n 4C hh mm ll dd F7

F0 43 – Fixed.
1n – n = channel number (0-F). Doesn’t seem to matter which number you use.
4C – Fixed.
hh mm ll – High, middle, low address. You get this from the other tables.
dd – 1 or 2 bytes of data, determined by the address you pick.
F7 – Fixed.

Turn on the XG system:
F0 43 10 4C 00 00 7E 00 F7 // In java, no spaces!

Now, if you go to the XG Parameter Change Tables, look for the second one labeled “Effect 1”. This is for Reverb. The tables for Chorus and Variation will follow the same pattern. The first three columns give the address to write data to, and the first row is the master effect setting. For the first row, you specify which Reverb preset you want (Room 1, Room 2, Canyon) using 2 bytes. The subsequent rows access specific parameters (wet/dry, delay, filter cutoff feedback, etc.) The presets are given in the Reverb Block Effect Type table, and the Effect Parameter Lists tables describe which parameters are available for which preset (that is, if you use Room 1 you get one set of parameters, if you use White Room or Tunnel you get a different set).

Example:

Use Reverb Room 1:
F0 43 10 4C 02 01 00 02 16 F7

F0 43 – Fixed
10 – 1 + channel number
4C – Fixed.
02 01 00 – Program reverb block
02 16 – Use the Room 1 preset
F7 – End of message

Once you’ve selected the Reverb preset (Room 1), you can change Wet/Dry. Look in the table for “Hall1/2 … Room1/2 … Stage1/2…” Wet/Dry is parameter number 10. Go back to the XG table for Effect1. Parameter number 10 is at address “02 01 0B” and takes one byte of data.

Change Wet/Dry for Room 1 to 0x2D:
F0 43 10 4C 02 01 0B 2D F7  // In Java, no spaces!

As mentioned above, so far Reverb is the only effect that I can noticeably detect changing, and Wet/Dry and Feedback are the two parameters that have equally big impacts. I think I have an advantage, using Java over a commercial sequencer, in that I’ve put controls in the applet to let me interactively change parameters like Wet/Dry with a slider, so I can tell immediately what some parameter does without having to hand enter individual new values into a data entry field.

eVocaloid:
The last remaining XG system option is specifically for controlling the Pocket Miku. For this, we need to go to the Gakken document written by a guy calling himself Polymoog. The NSX-39 MIDI Guide can be found on the Otona no Kagaku website. Like the Pocket Miku Perfect Guide, this one is all in Japanese, but most of what’s in the Perfect Guide was lifted directly from Polymoog’s document.

Note that the Pocket Miku isn’t a Yamaha product, and everything specific to the Miku was created by Gakken. The following SysEx messages are Miku-specific and not described in the Yamaha documentation.

The eVocaloid has command slots that you can reprogram. You’re writing to Flash memory so that your changes will remain in effect after you turn the machine off, and you can use the Miku as a MIDI controller (sending to MIDI OUT).

I really wish there were more examples and more data tables. Some of the examples are for attaching a “keypress” sound to the buttons, checking whether the buttons are on or off, and changing the “lyrics” for what Miku sings. But the examples are kind of incomplete.

From the Yamaha MIDI spec:

Changing Phonetic symbols:

F0 43 79 00 50 1m dd … F7

F0 43 79 00 50 – Fixed.
1m – 0=replace, 1=append
dd – List of phonetic symbols (ASCII only, space is [SP], comma is [,])

That’s all we get from Yamaha, so you can see that it’s a bit lacking. Polymoog’s guide and the Perfect Guide are a little better, but not by too much.

Reprogramming one of the 15 voice slots:

F0 43 79 09 11 0A 01 0D0D0D43062973 F7

F0 – Start of SysEx string
43 79 09 11 – Reprogram Pocket Miku
0A – Enter lyrics mode
01 – Slot number (Button “A” on the front panel. See below.)
0D0D0D43062973 – Data values for the sounds “Ge ge ge no Kitaro”
F7 – End of SysEx string

(Slot number: 01-15. 1-5 = “A”-“O”; 6-10 = SHIFT+”A”-“O”; 11-15= VIBRATO+”A”-“O”)

The hex codes for the Japanese lyrics can be found on page 10 of Polymoog’s guide.

Other examples from the Perfect Guide/Polymoog’s document:

Return command slot to default:
F0 43 79 09 11 0C nn 00 F7 (“nn” = command slot number)
Assign triangle instrument voice to the Shift key:
F0 43 79 09 11 0C 11 04 652265 F7

(0C 11 = Program command slot 11)
(04 65 22 65 -> 65 06 65 converts to “play triangle”. 8-bit encoding converted to 7-bit.)

This last example is the trickiest to explain. The SysEx command “0C” needs to have the data bytes converted to 7-bits long each. Because bytes are normally 8 bits, you need to write out the data on a piece of paper, using only the lower 7 bits of each byte, and then rewrite them as 8-bit bytes. So, while the data string contains “04652265”, the original data was “99 51 65”. “0x99” = “MIDI Note ON for Channel 9”. “0x51” = “Open Triangle” from the Drum Kit instruments table. “0x65” = Note ON velocity of 101.
Reading button status (Button “A” is ON):
F0 43 79 09 11 20 00 01 F7

20 = Read buttons.
00 01 – 000000ih 0gfedcba

“A” = SW1, “O” = SW5, Vibrato = SW6, Shift = SW7, Vol Up = SW8, Vol Down = SW9

In the example, “00 01” means that the “A” button (AKA SW1) is turned on. “00 00” is “all buttons Off”. “00 04” would mean the SHIFT button is pressed.

“0304” should be interpreted as “Volume Up + Volume Down + U”, which is the same as “Turn on General MIDI”.

youtube video direct link.

Video of the Java app in action.

Bokaro P ni Naritai, vol. 3


(Images used for review purposes only.)

I want to be a Vocaloid Producer, vol. 3, 1,500 yen, plus tax.
One reason I expect that this series is going to stop coming with the DVD-ROMs is that their instructions for cutting the proof-of-purchase seals, for use in getting the full-version serial numbers at the end of the series run, states that the seals are on the inside of the hard cardboard cover sheets for volumes 1-3, and then on a different sheet for the remaining volumes (meaning that the packaging is going to change). Since the “training course” relies so heavily on movie files and editing the supplied demo songs, I’d expect that the files for the later volumes would have to be provided some other way to make up for the absence of the DVDs, or, that they’d all be zipped onto the third DVD. However, the DVD-ROM that came with this issue just has the regular demo files, plus copies with the edits already made so you can compare what you did against what you were supposed to do. That, and there’s a “pick-up artist song” by Hachiko-P, and the next MMD model, this time for “Satchan” (Speaker). Vol. 4 will hit Kyushu around the 23rd, so I’ll find out then what’s going on.

The movie files have been covering the usage of the different apps in small bits and pieces. For Vocaloid, we’re taught how to change the lengths of different vocal components and add new phonemes to make Rana sound more natural. Unfortunately, my version of Vocaloid Lite seems to be slightly different than what’s in the video, and I couldn’t insert one of the phonemes as demonstrated. And in fact, part of the starting demo file already had some of the edits in place before I opened it.


(Pick-up Artist interview with Hachiko-P.)

With Singer-Song Writer, we’re treated to more of the mixer functionality, specifically, adding reverb, and playing with compression and the maximizer-limiter. SSW is very powerful, yes, but so far all the work has been on modifying finished songs. I’d like to go into the process of composing music from scratch pretty soon.

The really impressive package, though, is Miku-Miku Dance. The lesson this time is to take the Rana model from volume 1 and change the pose to match the volume 3 cover. There’s a lot of twisting and pulling on specific joints, but the connections are designed so that the rest of the model stretches or bends in a fairly natural way in response. You don’t have to adjust every single joint individually, which simplifies the operations a lot. The video is 8 minutes long, but because I was pausing it to work on the model, the entire lesson took an hour. When you’re done, you’re instructed to import the bird-like speaker creature, Satchan, and fix its pose on your own. As kind of a cheat, rather than adjust Rana’s fingers to make a fist or curve them somewhat, we’re given “pose” files to import for both the right and left hands.

Again, the magazine mirrors the movies on how to use Vocaloid, SSW and MMD. Additionally, there’s the interview and introduction of Hachiko-P’s Vocaloid songs, some joking between Rana and Robo-Panda about what reverb is, and a couple pages dedicated to some of the other Vocaloid products. I’m looking forward to the next issue to see what else MMD can do.

A few days ago, I got to wondering if I could find English user manuals online, given that both Vocaloid and MMD are running on my PC in English. Turns out that Yamaha has been packaging the English PDF with Vocaloid Lite, and that’s found by going to the help menu. I haven’t located anything for SSW, but there’s a dedicated website called Learn MMD that has a number of tutorials for Miku-Miku Dance in English, which makes up for the lack of a manual for MMD on the DVD-ROM.

About Japanese phonemes:
The Japanese alphabet is comprised of consonant-vowel combinations, rather than the smaller components we see in English. That is, we have “ah”, “ee”, “ooh”, “eh”, “oh”, “kah”, “key”, “coo”, “keh”, “co”, “sah”, “she”, “sue”, “seh”, “sew”, etc. You can double up on some of the consonants to emphasize the “t” or “s” sounds, and you can lengthen the time you hold the vowels. Usually, a lengthened vowel (“to” compared with “tou”) is just one sound, rather than separating them into two individual sounds (“to” + “u”). This is relevant to Vocaloid because it doesn’t recognize certain sounds. You can extend a note, so that the vowel part sounds like it’s longer, but many times you need to enter “to” + “u” as two individual letters, as if they’re separate notes. On top of this, Vocaloid doesn’t recognize kanji, so all of the lyrics have to be entered in the phonetic alphabetic system called hiragana. Having a Japanese wordprocessor like NJ-Star, or the Japanese IME character entry software is important here.

Programming the Pocket Miku, Details, Part 3


SysEx

System Exclusive messages are not part of the standard MIDI spec, and are specific to each given machine. That means that if you use SysEx messages in the sequencer in your song, they won’t have the same effect if you play the song on anything other than the Pocket Miku.

SysEx messages are like the other MIDI messages in Java, in that you use “setMessage()” to prep the data you want to send to the synth. But you use an object of the SysexMessage class to do it:

String s = “F0….F7”; // NO SPACES BETWEEN CHARACTERS!!!

byte[] byteMsg        = new byte[nLengthInBytes];
SysexMessage sysexMessage = new SysexMessage();

for (int i = 0; i < nLengthInBytes; i++) {
byteMsg[i] = (byte) Integer.parseInt(s.substring(i * 2, i * 2 + 2), 16);
}

sysexMessage.setMessage(byteMsg, byteMsg.length);
nsxRcvr.send(sysexMessage, -1);

byteMsg is a byte string, meaning that you have to convert the SysEx message from a regular character string to a byte array. I stole the byte conversion code from the jsresources page.

NSX-1 SysEx messages have several patterns, but the rules are that they all have to start with “F0” and end with “F7”. In between those two markers you will probably have a 3-byte address (Most-Significant Byte, Middle-Significant Byte and Least-Significant Byte), a SysEx “type” identifier, and one or two bytes of data. In some cases, data is variable length, as when you’re programming the eVocaloid “song” string in a lyrics slot.

Yamaha divides SysEx into: Universal Real Time Messages, Universal Non-Real Time Messages, XG messages and eVocaloid messages.

Universal Real Time Messages (URTM):

The first three messages duplicate some of the earlier parameter messages: changing master volume and master fine and coarse tuning.

After this, we get Reverb, Chorus, After-Touch and Control Change.

The format for Reverb is:

F0 7F 0N 04 05 01 01 01 01 01 PP VV … F7

F0 7F – Fixed.
0N – N is the channel number.
04 05 01 01 01 01 01 – Fixed.
PP – Parameter number (00 = Reverb Type, 01 = Reverb Time, etc.)
VV – Value (00 = Room S, 01 = Room M, etc., for Reverb Type)
VV – Value (0-127 = 0-11s, for Reverb Time)
F7 – Fixed.

The format for Chorus is:

F0 7F 0N 04 05 01 01 01 01 02 PP VV … F7

All that changes is the the last “01” becomes “02”. Plus, there’s a few more parameters available (Chorus Type, Mod Rate, Mod Depth, Feedback, etc.)

Channel Pressure (Aftertouch):

F0 7F 0N 09 01 0M PP RR … F7

0M – M = Channel number (0-15).
PP – Parameter number (00 = Pitch, 01 = Filter Cutoff, 02 = Amplitude, etc.)
VV – 00 to 7F, effect depends on the parameter selected.

Control Change:

F0 7F 0N 09 03 0M CC PP RR … F7

Key-Based Instrument Control:

F0 7F 0N 0A 01 0M KK CC VV … F7

To be honest, I haven’t done anything with Control Change or Key-Based Instrument Control because they look like too much work. While they both require a “CC” value (the controller number), the only place where controller numbers are defined is in the table for Key-Based Instrument Control (0x07 = Volume, 0x0A = Pan, 0x5B = Reverb, 0x5D = Chorus).
Universal Non-Real Time Messages

Now we get to the messages that really matter. The first and foremost is turning General MIDI On and Off.

As mentioned previously, several of the Control Change and NRPN messages only work with GM ON (such as Reverb and Drum Cutoff Frequency). You can turn GM on and off from the Pocket Miku front panel by pressing Volume Up and Down, and “U” simultaneously (you’ll hear a little “click” sound if you do it right. This is not convenient if you want to change modes in the middle of a song.

The better way is to send a SysEx message:

F0 7E 00 09 01 F7  // GM On

F0 7E 00 09 02 F7  // GM Off

Note that while the MIDI spec shows “F0 7F XN”, “X” is ignored, and I find that “N=0” works fine.

I haven’t bothered with Scale/Octave Tuning. It just seems easier to use the sequencer or keyboard to shift scales.

 

Programming the Pocket Miku, Details, Part2


RPN

RPN refers to Registered Parameter Number. NRPN is Non-Registered Parameter Number. “Registered” means that it’s part of the standard MIDI spec and will work the same on all synthesizers. “Non-Registered” parameters only work as intended for the given machine. That is, the NSX-1 NRPN messages are NSX-1 specific. Both sets of parameters take effect faster than if you program them using SysEx, and they duplicate some of the SysEx parameters.

Registered Parameter Numbers are:

Pitch Bend Sensitivity (changes the range of pitch bend)
Master Fine Tuning (changes the pitch of all notes a small amount)
Master Coarse Tuning (changes the pitch of all notes a lot)
RPN null (nothing)

NRPN

The NSX-1 Non-Registered Parameter numbers duplicate many of the Control Change parameters. That is, Vibrato Rate, Vibrato Depth and Vibrato Delay work the same as Control Change Vibrato Rate, Time and Depth do. Note that NRPN includes identical parameters specific to the Drum Kit instruments, but on a per-note basis. When you’re using the drum kit (either the sound bank or GM channel 9), each keyboard key plays a different drum instrument. When you play a drum note, you need to specify that note number as part of the NRPN Drum parameter data if you want to change Drum Filter or Drum Pitch.

Example:

Say you’re pressing the keyboard key that corresponds to MIDI note number 60 while using the Drum Kit sound bank. If you want to change the Drum Low Pass Filter Cutoff Frequency in Java, you do the following:

myMsg1.setMessage(ShortMessage.CONTROL_CHANGE + channel, 0x63, nrpnMsb);
myMsg2.setMessage(ShortMessage.CONTROL_CHANGE + channel, 0x06, dataMsb);
myMsg3.setMessage(ShortMessage.CONTROL_CHANGE + channel, 0x62, nrpnLsb);
myMsg4.setMessage(ShortMessage.CONTROL_CHANGE + channel, 0x26, dataLsb);
nsxRcvr.send(myMsg1, -1);
nsxRcvr.send(myMsg2, -1);
nsxRcvr.send(myMsg3, -1);
nsxRcvr.send(myMsg4, -1);

Where:

0x63 is the Control Change number for setting the NRPN MSB byte, and “nrpnMsb” contains 0x14 for controlling the Drum Cutoff Frequency.
0x06 is the CC number for setting the NRPN data MSB byte, and “dataMsb” represents the cutoff frequency (0-127).
0x62 is the CC number for setting the NRPN LSB byte. “nrpnLsb” is the keyboard key number (60 in this example).
0x26 is the CC number for setting the NRPN data LSB byte. “dataLsb” has no effect and can be left at 0x00.

If you play more than one drum instrument, you need to change “nrpnLsb” to that note number every time you call Drum Low Pass Filter Cutoff Frequency.

NRPN parameters that are not available through Control Change include: Low Pass Filter Cutoff Frequency, Low Pass Filter Resonance, Drum Low Pass Filter Cutoff Frequency, Drum Low Pass Filter Resonance, and everything that starts with 0x70.

0x70

The 0x70 NRPN parameters only apply to eVocaloid (channel 0). Vibrato Type and Delay work the same as for General MIDI. Portamento Timing is a single parameter that ranges from 0x00 (fast) to 0x7F (slow). Seek isn’t included in the Perfect Guide so I don’t know what that does. White Noise adds a little hiss to the eVocaloid voice. Phoneme Connect Type has three modes: 0x00 = Fixed 50ms delay between phonemes; 0x01 = Minimum connect times; 0x02 = Note velocity changes time between phonemes. (Start of Phonetic Symbol, Phonetic Symbol and End of Phonetic Symbol aren’t included in the Perfect Guide.)

Note that anything not included in the Pocket Miku Perfect Guide is not otherwise documented, and seems to be deactivated in the Pocket Miku itself.

In the Control Section above, we have Reverb, Chorus and Variation parameters. For some reason, chorus and variation have very little impact on the note played, as far as I can tell. Reverb is the most noticeable of the three, and I can sense some change in chorus, but I have yet to figure out if variation is doing anything at all. This is relevant in that it plays a big part in the following SysEx tables.

 

Programming the Pocket Miku, Details, Part 1


The Miku, AKA – the NSX-39, is based on the Yamaha NSX-1 chip, which is the heart of the Vocaloid products, and includes the eVocaloid system. It has a USB connector, so you can plug it into your PC or Mac, and run it using any sequencer software you like (the Perfect Guide book focuses mainly on Garage Band). Most of the examples in the Perfect Guide revolve around tweaking the sequencer to customize it for the Miku, or showing how to send SysEx messages within the body of a song. But, if you have Java, you can create your own applet to make the Miku do what you want as a regular synthesizer.

The starting point is the Yamaha MIDI spec doc. I wish it was more instructional, but it’s basically just a bunch of code tables. The first table in the Yamaha doc has the channel messages. Since the Yamaha MIDI spec is in English, there’s no point in repeating all the codes here. The important things to note are the Control Change and Mode Messages, which both use the “Bn” status byte (where “n” is the MIDI channel you want to control, 0-15. The Pocket Miku can play 16 instruments simultaneously, with channel 0 assigned to eVocalod and channel 9 is the internal General MIDI drum kit.)

RAS

The Vocaloid system apparently (I’m still trying to understand this, since the documentation is all in Japanese) divides speech phonemes up into “heads”, “bodies” and “tails”, combined with “joints” and “shots” (as mentioned in the product brief). These phoneme components can be mixed and matched as part of Yamaha’s “Real Acoustic Sound” (RAS). I mention this because the Yamaha MIDI doc shows which MIDI options apply to which operating mode, including GM (General MIDI), eVocaloid and RAS. But, the Pocket Miku Perfect Guide Book only shows the GM and eVocaloid modes. So far, I haven’t found any documentation on how to program the RAS mode and I’m thinking that Miku doesn’t support it.

The other reason for mentioning RAS is that in the MIDI doc, there are three sound banks, which you can select using Control Change: “Normal” (0x00), “Drum Kit” (0x7F) and “RAS” (0x08). The Perfect Guide only gives the “Normal” and “Drum Kit” banks. The “Normal” bank contains the standard 128 instrument voices that all MIDI devices support. The “Drum Kit” bank is actually the percussion voices available on channel 9 of the “Normal” bank.

CONTROL_CHANGE:

To select sound banks in Java:

ShortMessage myMsg1 = new ShortMessage();
ShortMessage myMsg2 = new ShortMessage();
int channel = 1;  // 0 to 15
myMsg1.setMessage(ShortMessage.CONTROL_CHANGE + channel, 0x00, msb);  // MSB = 0x00 or 0x7F.
myMsg2.setMessage(ShortMessage.CONTROL_CHANGE + channel, 0x20, 0x00);
nsxRcvr.send(myMsg1, -1);
nsxRcvr.send(myMsg2, -1);

To select specific instrument voices within the selected bank:

myMsg1.setMessage(ShortMessage.PROGRAM_CHANGE + channel, n, 0);  // n = 0x00 – 0x7F
nsxRcvr.send(myMsg1, -1);

As mentioned above, the NSX-39 supports all 16 MIDI channels, with numbering starting at 0 (the Perfect Guide starts numbering at 1). Channel 0 is for the eVocaloid; 9 is the Drum Kit.

To select the channel, look closer at the above code:

channel = 0; // For channel 0.

CC List:

The other CC parameters may need a little description. Note that not all of them apply to General MIDI or eVocaloid. Also, you need to turn General MIDI ON (AKA: NSX-1 compatibility mode) to get things like Portamento to activate.

Modulation: Varies the note pitch.
Portamento Time: Time needed to sweep from one note to the next.
Panpot: Pans stereo sound between the left and right speakers.
Expression: Acts like a volume control.
Sustain (Damper): How long the note plays after Note Off.
Portamento: Steps to sweep from one note to the next.
Sostuneto: Switches “sustain note” on and off.
Soft Pedal: Switches “soften note” on and off.
Harmonic Control: Varies the amount of harmonics added to the note.
Release Time: Changes the ADSR release time.
Attack Time: Changes the ADSR attack time.
Decay Time: Changes the ADSR decay time.
Brightness: Adds higher frequencies to the note.
Vibrato Rate: Cycle rate for the vibrato effect.
Vibrato Depth: Size of pitch variation applied to the note.
Vibrato Delay: Time after the note starts that vibrato is applied.
General Purpose Control: RAS only, not included in the Perfect Guide.
Portamento Control: Key to apply portamento to.
Effect 1 Depth: Applies reverb (echo) to the note.
Effect 3 Depth: Applies a “chorus” effect to the note.
Effect 4 Depth: Applies a “variation” effect to the note.
RPN Increment: Increments the current RPN parameter.
RPN Decrement: Decrements the current RPN parameter.
NRPN LSB: Allows you to change NRPN parameters.
NRPN MSB: Allows you to change NRPN parameters.
RPN LSB: Allows you to change RPN parameters.
RPN MSB: Allows you to change RPN parameters.

Comments:

Some of the above parameters only work for General MIDI, when General MIDI is turned on, and that’s something you do through SysEx (System Exclusive) messages, which I’ll get to later.  I’ll cover RPN and NRPN messages next time.

 

Programming the Pocket Miku


I’ve had the “Singing Keyboard Synthesizer Pocket Miku” for months now, and I’ve played with it a bit as a handheld synth keyboard off and on when I’ve had the time (which hasn’t been all that often). Recently, I decided to write a Java app for accessing the different features available via MIDI commands, and I’ve kind of run into a wall regarding documentation. That is, there doesn’t seem to be anything useful in English.

The Miku, also known as the NSX-39, is based on the Yamaha NSX-1 chip, which is used in a number of other products, including a Vocaloid-style Speak and Spell game in Japan. While Yamaha has posted the NSX-1 timing sheet, product brief and MIDI spec online, they haven’t made the NSX-1 User’s Manual publicly available. The MIDI spec is the only one in English, and while it’s good for things like the Control Change and SysEx messages, it lacks example usages, and there’s nothing for specifically controlling the NSX-39.

Gakken has a go-to author for synth-specific issues who goes by the name Polymoog. Poly wrote a MIDI guide for the NSX-39 which is very detailed, but is in Japanese. He also makes reference to a number of spec docs hosted on Google docs that aren’t publicly viewable. I’ve tried requesting access to them, but haven’t gotten a response back. I’ve also tried asking on the Yamaha Synth forums whether anyone has these docs and, again, haven’t gotten an answer back.

Now, I’m willing to accept that the NSX series synths are only available in Japan, and that the demand for English documentation is very low. But, it would be nice to have all of the Japanese docs so I can at least get a better shot at figuring out what’s going on, on my own.

Fortunately, I do have the Pocket Miku Perfect Guide, which includes a few of the data tables, and some example SysEx messages. Unfortunately, a lot of the information was lifted directly from Polymoog’s MIDI guide, and contains the same typos Poly made.  So, I can get working about 50% of what I want to accomplish, but I have no idea what else the Miku is capable of, and I don’t know why the other 50% isn’t functioning correctly.

This brings me to my point. I’ve been compiling a lot of notes on the NSX-39 Control Change and SysEx messages, and I’m slowly translating, for myself, the key operations of the NSX-39 from Polymoog’s guide. I’m going to start up a new series of blog entries here, alternating with the backgammon series (due to wrap up in a few weeks), consisting of these notes. What I’m really hoping is that this will help connect me with other English-speaking NSX-39 owners, and we can get something of a community started up. Worst case, I’ll just have my notes available here in the blog as back up if my laptop ever gets fried.

Electronic Google


I’ve been wanting to write a post on this for a while, so I might as well do it and get it out of the way.

Google will occasionally run special images or animations on their main search page to commemorate specific people, events or dates. Two that they’ve done in the past that are appropriate to be mentioned here are for Les Paul, and the Moog Synthesizer. Click on the image to go to the google page, play with them and have fun. (Although, the page for the Les Paul link may have been taken down already.)