More news on the Soundplane to CV module.
The second phase was hardware design and prototyping.
There are many embedded processors and I did not want to get too deeply into the hardware design process until after I was reasonably sure that the chosen one could handle the task. I recall some 8-bit MCU USB peripherals are limited to a maximum of 256 bytes for isochronous packets, while the Soundplane uses two endpoints of 388 bytes each. The proof of concept phase mentioned last week convinced me that the TM4C1294 can handle the full bandwidth, which makes sense given the 32-bit ARM architecture.
Cortex-A chips - the opposite end of the spectrum - are significantly more expensive, pull vastly more power from an already challenged modular power supply, and are really only necessary when you need serious graphics processing and full operating system support. You will not find a Cortex-A chip for as little as $7, so I am happy with the Cortex-M4F.
The next task after settling on a chip was to arrange all of the external connections to fit the specific GPIO ports. With 128 pins, 90 of which are GPIO, there are many interfacing options. Starting with a system design at a block diagram level, I grouped similar control signals together on the same port to simplify the firmware, and with so many GPIO options I was also able to move functions around the chip to locate them near subsystem groups on the PCB. Despite the 32-bit nature of ARM, the 15 ports are all 8 bits or less, which is a little different from, e.g., the XMOS chip architecture.
A side effect of designing the system first is that many of the expansion boards went through layout and prototyping before the main board. The advantage of handling them first was that they were very quick to design and I could work on the main board while waiting for the expansion boards to come back from fabrication and assembly. Future installments will discuss those.
I then completed the main board PCB design and reviewed it several times before beginning layout. You will see in the photo that it has the requisite Euro rack power connector. Peripherals include a rotary encoder, potentiometer, button, several LED indicators, and the obvious USB A Host port. Besides the many anticipated expansion connectors such as octal MIDI I/O, there are also a couple of quad synchronous serial port headers. Those SSI peripherals may never be used, but they would allow for serious high-speed data connections to other subsystems.
Basic firmware is up and running on this main board and communicating with the Soundplane. A lot more work is needed on the firmware, however, to implement all of the subsystems and get them working together. Obviously, besides the foundational basics, the first test will be to track at least one touch and generate a set of Gate and X+Y+Z Control Voltages that bring Soundplane performance gestures to life on a modular.
There is no particular schedule for future installments of these reports, but I think there should be time for me to put out a few more in the coming weeks.
The TT code has been open source all along for the purpose of maximizing compatibility with as many platforms as possible. Randy is currently working on optimizations for that code which should help, as you say.
The BBB/Bela Cortex-A9 seems to be geared towards a higher-level Linux system with higher latency, while offering easier application development due to the well-known platform. This Cortex-M4 platform will not be based on Linux, and so the latency should be as low as possible for hardware - unfortunately, low-level firmware is a bit more difficult to master although the performance makes it worth the effort.
Yes, there is a bit of news... enough that I'll break it into two installments.
The first phase was research and proof of concept.
The Soundplane poses a bit of a challenge because it requires isochronous USB, and quite a bit more payload than some embedded processors can handle. The project got started with a Texas Instruments LaunchPad : the EK-TM4C1294XL. The chosen processor is an ARM Cortex M4, and it seems to have what is needed. There is floating point support and the ability to host the Soundplane.
You'll see in the photo that an adaptor was needed from the tiny USB connector on the LaunchPad to a USB A socket as needed by the Soundplane. A quick bit of development in Code Composer Studio resulted in firmware that can power up and communicate with the Soundplane, and it doesn't seem to be dropping any packets. In addition, crude touch detection seems to indicate that there are no lost packets, so full bandwidth is working.
You'll also see a tiny SMD capacitor soldered to the through-hole expansion pads. This was necessary to handle the current draw of the Soundplane at 230 mA, which is almost half the limit of USB.
After the first phase, significant time was spent designing custom hardware for a modular environment with precision D/A for CV outputs.
Stay tuned for the second phase.
Dangerous Prototypes is also a good site with lots of open-hardware and open-source projects. It's basically Hack-a-Day for electronics design.
Does the Soundplane application support the full MIDI Channel Mode 4 specification?
Specifically, I'm asking about the parameter in the Channel Mode Message (#126) for Mono Mode On which takes the number of voices as the second byte (with 0 meaning "all" voices). This command would presumably be sent after the Channel Mode Message (#124) for Omni Off.
I was away from this thread for a long time, but it is great to see the two of you helping each other out on these platforms. I will soon be working on a Cortex M4 implementation running a lot slower than 1 GHz, so I may need to focus on optimization. Thanks for sharing all of your notes as you worked!
As for bypassing running a laptop, that's the point: Performance would certainly be stand-alone, i.e., without a laptop. Just the Soundplane and your synth of choice, with only the The Soundplane-to-CV in between. It would have on-board non-volatile memory for your custom settings, and that could be easily loaded from a USB Memory Stick. SDHC card support would require dedicated hardware, increasing the cost and panel real estate, so it's doubtful that particular format would be supported.
As for bypassing running a modular synth, I'm not exactly sure that I follow. I assume you mean that you'd like to use a standard Analog synth, even if you don't already have a modular. There should be no issue with that, technologically, although there does need to be some sort of case, even if it's a small modular rack or just a metal box. We'll keep the options open as much as possible, within reason.
Are there any settings that you might need to change on the fly during a performance? Anything that a few customizable presets wouldn't cover? Keep the ideas coming!
And, to state what's probably obvious to most people here, your 2-in 4-out low-latency system could be used to build a DIY 4x2 pressure sensitive surface. I'm sure the AM335x has sufficient power to calculate 2,000 multi-point FFT conversions per second.
Yes, constant latency is more easily ignored than jitter.
I was concerned about FS because the Soundplane is FS. So, it seems like the MTT would be forced to add latency.
Really cool project!
Sorry for the detailed questions, but do the USB Multi-Transaction Translator hubs add any latency when performing their feature? Seems like it would be unavoidable, although High Speed frame latency is only 1/8th Full Speed latency. Hopefully the total latency is only 1 ms + 0.125 ms = 1.125 ms.
Meanwhile, I assume that your STM32F7 Discovery Kit firmware must implement High Speed USB, otherwise the Multi-TT hub would not really provide any advantage when connected there.
You may not actually need as high as 125 kHz sampling rate.
With only 16 carriers and a target 500 Hz update rate instead of 1 kHz, you could possibly sample as low as 31.25 kHz or 32 kHz. Then you'd only need 32 frequency bins from a 64-point FFT, and that might be easier to perform with limited DSP capabilities. Your bandwidth might be one quarter that of the 8x64 Soundplane, so perhaps you don't need the reduction we attained by converting to the frequency domain. Instead of the Soundplane's 768 kB/s USB bandwidth, you might be able to get by with 384 kB/s of time domain information. You can cut that down to 192 kB/s via FFT. Of course, this assumes 12-bit samples. The Soundplane uses ADC chips that operate at 1 MHz. If you use 24-bit audio ADC chips, you'll need 8 of them (or an 8-channel CODEC), but the 24-bit data will double your bandwidth needs unless you dither down the bit depth before sending over USB.
In any case, you'll still need isochronous USB packets, whether time domain or frequency domain, in order to guarantee that the USB Host will allocate enough bandwidth to guarantee delivery of the data stream without dropouts.
Sorry for all the numbers - I hope I didn't do any of the math incorrectly.
p.s. A CODEC may simplify things, since you could have sine wave outputs and ADC inputs on a single chip with a single sample rate. However, I don't really know of any 16-out, 8-in CODEC parts. Designing around a lower sample rate may offer more converter chip options for your project. You might get better results around 60 kHz than 30 kHz, though, with the understanding that this would double your bandwidth requirements.
Sounds like fun! Keep us posted.
The paper never mentions whether the haptic vibrations reduce the sensitivity of the Soundplane by the nature of them being placed between the surface and the sensor. I'd be curious to know how much of the haptic vibration is picked up by the Soundplane's sensors, given that the haptic frequency is bandpass-filtered to 40 Hz - 400 Hz, and the Soundplane has the ability to read vibrations up to 500 Hz.
The closing remarks do admit that more research is needed, so I'd be curious to read a followup paper.
Wow! I had not seen Juglans before.
The piece has such a wide palette of sounds as well as dynamic range. I watched the video before reading the followup comments, and I couldn't tell that you were selecting sounds. I can see how that might not be your preferred way to play. Let us know if you find a different solution - or if you have any more videos to share.
Keep us posted, thetechnobear, and maybe even create a new thread (separate from HD MIDI) when your Axoloti supports 14-bit CC. If it works well, I hope the offical Soundplane client will support 14-bit CC out of the box.
I've used the environment in Logic Studio Pro to both generate and respond to 14-bit CC as a test, but I've not actually checked whether any of my synths support it. Does Zebra?
Considering that this is one of those chicken-and-egg situations, maybe if you add support for 14-bit CC to Axoloti, more people would become aware of it and think about adding support elsewhere.
The Livid Instruments Ohm64 and family support 14-bit CC generation, but it's difficult to find a control surface that's accurate enough.
HD MIDI sounds like a bold departure, but I think we've yet to tap the limits of 1983 MIDI.
Of the many things you have to tackle, you could get a start by looking for USB Host isochronous examples, such as would allow you to connect a USB Audio interface and stream audio. If you can get that going on your STM32F4, then you could probably easily host the Soundplane with a few modifications. Actually, the Soundplane requires much less complex support than an audio interface, although it's using about 3 times the bandwidth of 24-bit stereo audio. That's (a)
Of course, processing touches is the next step after that, and you have the client source as a guide for that (b).
How is HD MIDI different? We have 14-bit Pitch Bend (X), and at least the potential for 14-bit CC for expression (Y) and pressure (Z). I didn't realize that the Soundplane client didn't produce 14-bit CC already, so I'm wondering whether anyone has added that feature. It should be relatively easy to add.
Testing could be done with something like the Encore Expressionist MIDI-to-CV, which has 16-bit D/A ranging from -3V to +10V. What hardware synths out there support 14-bit CC modulation?
Hi Scott, can we get an update on your progress? I'm probably not the only one who is interested to read about what you were able to get working, and whether you've upgraded to the STM32F4 and floating point.
One limitation of putting CV outputs on the Soundplane is the power supply. USB provides +5 V, but modular racks typically provide ±15 V and sometimes as much as ±18 V. There would be a serious drain on the USB power if it were boosted high enough to interface with modular, not to mention the noise created by a switching power supply needed for boosting voltage like that. Some of the better CV interfaces provide pitch CV from -3 V to +10 V to span many octaves, so we'd like to match that range with the Soundplane for maximum flexibility. Restricting pitch to 0V and +5 V would be rather limiting.
So, yeah, there are space and engineering issues like the power supply and processing that would make it very challenging to squeeze everything in the Soundplane case. It's much easier to start with a modular power supply and bring it down to 5 V than to go the other way.
Thanks for the idea, though. It's fun to consider approaching things from a different angle.
Thank you for the details of your patches, ngwese. This seems similar to what I needed to do with the Matrix-12 to open up the envelope. Oberheim allows multiple CV sources to be mixed in the VCA input.
If you care to describe how you use slew and/or lag, that might be helpful.
The computer-based configuration side of this modular system could be open source, just like for the Soundplane.
I'm curious: How many open source products actually have compelling features available from a third party? Can you list them? I'd like to get a sense of the community's skill set.
There will be expansion for more touches. If you're willing to spend extra, then at least ten full touches will be possible (40 CV outputs), probably in sets of two touches each. I doubt that it makes sense to go beyond that.
Some form of configuration for the outputs will be possible (range, precision), but the minimal interface may require some or all voltage configuration to be done on a computer and then stored in the module for regular use. Presets will be available so that you're not stuck with just one setup when you don't have a computer around.
I'm fairly certain that Gate or Trigger or Velocity Gate will be possible in firmware for those who need different things. That way, you won't need to derive a gate or use a comparator unless you really want extra outputs without buying another expander.
This module will be a USB Host so that it can communicate with the Soundplane (a USB Device). The USB-MIDI features you're discussing are USB Device features. Believe it or not, 95% of the chips out there can only do one at a time, so it's either Host or Device, but not both. Another factor is that you need one kind of connector to be a Host, and another to be a Device. The only universal USB connector is OTG (On The Go), which is the tiny connector found on cell phones. It (OTG) seems like a horrible interface to put on a module, so that pretty much rules out USB-MIDI features. I think there are plenty of options in that area anyway, so please continue to have fun with your CVPal! I see the Soundplane as something parallel to that, rather than a replacement.
Early mockups based on actual board layouts look like 6HP might fit, but that might be tight. I'm fairly certain that 8HP will work. This is for a minimum of 2 touchs, which is 8 CV outputs, grouped as Gate with X, Y, and Z. Thanks for chiming in, ngwese and others.
Thanks, shedacq. That page looks like it might be specific to Windows 10. Looking around at some of the chapters, there are references to Windows 8. I'll take a further look when I have some time.