Programming the Pocket Miku, Details, Part 6

Most of what I’d done in the last blog entry was to summarize the translations of Polymoog’s MIDI guide, which brings me to page 16 of that guide. Most of what follows concerns manipulating the pointer to the phoneme character strings that Miku “sings” when you touch the stylus to the carbon keyboard. You can issue these commands either as a “command” (0D) or as an instruction string placed in one of the command slots (0C) to be implemented when a particular button action occurs (pressing “A”, or releasing SHIFT + “U”).

F0 43 79 09 11 0C … F7, or
F0 43 79 09 11 0D … F7

These character pointer operations include setting the pointer (MOJI_SET_KASHI = 09 05 00), incrementing the pointer (KASHI_POS_INC = 09 01 00 00), selecting the pointer position (0 to 63: KASHI_POS_SEL = 09 02 00 nn, where nn is 00 to 3F), selecting the lyrics slot (KASHI_SLOT_SEL = 09 03 00 0n, where n is 0 to F), and phoneme character entry (MOJI_SEL = 09 04 00 nn, where nn is 00 to 7F from the character table on page 10 of Polymoog’s MIDI guide). (Japanese translations appear below.)

Page 17 gets into some of the other commands from the command table on page 23, such as NOTEON, NOTEOFF, ADV_PB (pitch bend), VOL_UP, VOL_DOWN, etc. They work as you’d expect them to as regular MIDI messages.

F0 43 79 09 11 0D 0A 02 00 00 F7 – Like pressing the Volume Up key on the front panel.

VOL_SET: Set volume level (0-8)
VOL_UP: Increase volume one level
VOL_DOWN: Decrease volume one level

KEY_DOWN: changes the keyboard up or down one octave.

TUNING_STOP: Lets you calibrate the carbon keyboard.

NSX1_GOKAN_OFF: Turns NSX-1 compatibility mode on and off.

MODE_DOREMI: Puts keyboard into Do-Rei-Mi mode.
MODE_NORMAL: Miku sings from the lyrics tables.

MIDI_RESET: (Panic) Resets the phoneme chip.

ROM_RESET: (Initialize) Returns the Flash ROM settings to the defaults .

Section 3-6 on page 20 discusses the concept of CACCO, which is the Japanese word for “brackets” or “parentheses”. This mode is used to assign a string of unrelated commands to a specific command slot as if they were within a pair of parentheses. Once CACCO_START is encountered, the NSX-39 will execute the subsequent commands until encountering CACCO_END.

The example is to assign certain commands to the SHIFT button when pressed (CS_SW7D = 11).

0C 02 00 00 – CACCO_START (command list start)
09 02 00 00 – KASHI_POS_SEL(0) (reset lyrics list character pointer to position 0)
0C 03 00 0A – CACCO_END (command list end)
0C 02 00 00 – CACCO_START (command list start)
09 05 00 00 – MOJI_SET_KASHI (set pointer to play current character)
08 01 00 00 – REVOICE (replay the character)
09 01 00 00 – KASHI_POS_INC (point to the next character in the lyrics list)
0C 03 00 05 – CACCO_END (command list end)

From these commands we can see that when you press the SHIFT key, Miku will sing the first character in the currently selected lyrics slot.

The full SysEx message would be (all one line, no comments):

F0 43 79 09 11 // Start of SysEx message (fixed)
0C 11 // Change Command Slot for CS_SW7D
0C 02 00 00 // As given above
09 02 00 00
0C 03 00 0A
0C 02 00 00
09 05 00 00
08 01 00 00
09 01 00 00
0C 03 00 05
F7 // End of SysEx message (fixed)

Section 3-7 (page 21) describes timing. You can specify a wait time (TIME_WAIT = 0D nn nn nn, where nn can be from 00 to 7F). A value of 500 is roughly 1 second. You can cancel wait time with TIME_CANCEL (0C 05 00 00), and specify a TIME_TAIL (0C 04 00 00), which lets the timing buffer clear out. The pattern would look like:


The example on page 22 is to assign a string to VOL UP + VOL DOWN + “A” (panic reset). (All one line, no comments.)

F0 43 79 09 11 // Start of SysEx message (fixed)
0C 32 // Change the command slot for CS_SW89_1D (Up + Dn + A)
04 65 12 66 // Direct MIDI 3 command mode to play a music note (see below)
0C 02 00 00 // CACCO_START
0D 00 00 C4 // Wait 200 ms
0A 07 00 00 // MIDI_RESET
0C 04 00 00 // TIME_TAIL
0A 07 00 00 // MIDI_RESET resets even with Cancel
0C 03 00 0F // CACCO_END
F7 // End of SysEx message (fixed)

Note: The 04 65 12 66 portion is a repeat of the 7-bit direct MIDI format, which takes “99 49 66” (Note Number 49 from the Drum Kit sound bank ON with a velocity of 0x66) and converts it to “04 65 12 66”.

Everything else is tables. Page 23 shows the command table. Translations:
“kashi” is “song lyric” (used to refer to the current lyrics string).
“Gokan” is “compatibility mode”.
“cacco” is “parentheses”.

Pages 24 to 27 describe the operations of the Command Slots. These are addresses within the Flash ROM pointing to instructions to execute when specific events occur, such as pressing or releasing one of the front panel buttons, power up (kidou), turning audio on/off, or activating the USB port. The slots themselves are listed in the tables on pages 28 to 30. I think most of this is self-explanatory so I’m not going to translate the tables. If there are enough requests for it, though, I’ll reconsider.

NSX-39, a first look

If you’re not familiar with Vocaloid, I suggest that you read the wiki article. It’s a very popular franchise in Japan, and in part that popularity revolves around the characters created to represent the various singing voices that the Vocaloid software can produce. As an example, Miku, one of the first, and one of the main “lead singers”, is consistently used in product tie-ins with FamiMart convenience stores.

As I’ve mentioned here recently, Gakken had announced the upcoming release for their NSX-39 Pocket Miku “singing synthesizer”, through their Otona no Kagaku line of kits. To an extent, the NSX-39 is kind of an extension of the SX-150 ribbon controller synthesizer. The concept is that rather than just having musical instrument voices like a piano or flute, you have the human voice singing specific sounds. The original Vocaloid software was written around the idea of music being created based on transcribing written lyrics. What the NSX-39 does is provide a USB port for downloading new lyrics to the kit, and then offering up to 10 operating modes for playing those sounds back as music. The user interface is a ribbon controller and stylus, with the ribbon split into 2 parts. One is a regular continuous ribbon as with the SX-150, while the other has distinct piano-style keys printed in it, resembling the keyboard printed on Miku’s stockings in the character art.

As can be seen in the below photo, the face panel has Vibrato, shift, A, E, I, O, U, and Volume Up and Down buttons. There are stereo headphone and USB jacks at the back, and an Off/Battery/USB switch. So, you can run the NSX off USB power if you don’t want to use up batteries (3 AAA’s). All of the face panel buttons do double duty if you use the shift key with them. The normal mode is to assign each of the keyboard keys to the Japanese versions of “do re mi fa so la ti do”. The A-O buttons switch the sounds to just that letter across the entire keyboard. Shift A-O activates a pre-programmed sentence that sequences every time you touch a note with the stylus. Shift-A gives you “Ko-ni-chi-wa-A-ri-ga-to” (Hello, thank you) playing one syllable per note press. Vibrato warbles the sound currently being played, and Shift-Vibrato returns the unit to Do-Re-Mi mode. Volume Up and Down changes speaker volume in 5 steps, while Shift-Volume Up/Down changes the keyboard up/down 1 octave. Vibrato-Volume Up/Down is pitch bend. Vibrato and A-O lets you switch to user presets.

(NSX-39 with Miku sticker applied.)

Pressing Volume Up and Down and A at the same time is “Panic” mode (defaults reset).¬† Up and Down and U switches between NSX-1 mode. Up and Down and O turns tuning on and off. Up+Down+Shift reinitializes the unit. At the moment, I can’t tell the difference between NSX-1 and NSX-39 modes, or Panic and Initialize.

(FamiMart version of Miku.)

The box contains the fully-assembled NSX-39, a 16-page user’s manual, 2 stickers you can put on the face plate (a picture of Miku, or the definitions of the different buttons in English) and an insert sheet advertising the real Vocaloid software system. The first 3 pages of the manual look at the history of electronic speech, from the Speak-and-Spell up to Vocaloid. The rest of the manual is split up into Operations; Using the online app or a sequencer; and Troubleshooting.

In general, the NSX-39 is a toy. It’s nice to play around with and make¬† sounds for a while, but it has the same appeal as the Korg Monotron, and about the same play lifespan. As-is. To expand its repertoire and keep it more interesting for a longer time, Gakken has added some nice features. First, if you go to the app page, you can access a second webpage that lets you type in new “sounds-as-words” into the interface and automatically store them in the NSX-39 (it has to be connected and turned on at the time) for later
playback. When you plug the unit into your computer (PC or Mac), the OS autodetects the NSX-39 and downloads the proper USB drivers for it. The app only runs under Google Chrome (which I hate) and only accepts Japanese character entry (what’s called Hiragana). This means that you can’t use the app for English entry, since the Vocaloid software isn’t tuned to English. If you can read and write Hiragana, it’s great.

(Miku loves FamiMart)

The second nice thing added by Gakken is MIDI support. You can run the NSX-39 from a PC-based sequencer through the USB port (if you have sequencer software), and you can even use it as a MIDI controller. I haven’t tried this out yet (the Pocket Miku just hit the stores in my region of Japan two days ago). But, I’m thinking that I’ll be able to interface it with my K-Gater Java app sometime by the end of April or beginning of May when my schedule frees up. If K-Gater works, I’ll try writing a Java-based sequencer to include the Kaossilator Pro. (To interface with my other gear, I’ll need to buy a PC-to-MIDI connector converter, which I don’t have yet.) The manual lists the supported MIDI operations, but it’s just in one small paragraph; it’s not in table format. So it may take some experimenting to get the codes to work right.

One thing about the USB connection is that you need Micro-B. Fortunately, when I bought my new Nikon CoolPix digital camera, it came with a Micro-B USB cable, so I already had that available when it came time to plugging the NSX-39 into my laptop.

Comments: The Pocket Miku is an interesting novelty item to add to your rig. If you use your laptop as a converter, you can drive the unit from a regular MIDI keyboard rather than having to use the ribbon cable. Or, you can use regular sequencer software and have it playing while you compose songs regularly through the computer. The three things I wish were different are: Firefox browser support for the app; support for English phonemes; and the ability to choose a voice other than Miku’s. Miku is supposed to sound like a 16-year-old girl, and she gets rather piercing when singing higher notes. I’d like to have a lower, male voice in order to get bass sounds. But, maybe that will be a sequel kit.

Pocket Miku Singing Keyboard
Released Apr. 3rd, 2014, 4,980 yen (w/o tax).
Ships fully assembled; not a kit.
Accompanied by a 16-page B5-sized user’s manual.
Requires 3 AAA batteries when not connected to a computer.
Requires a Micro-B USB cable to connect to a computer (PC or Mac).