ForumsHardware ← Soundplane : raw matrix processing


Im trying to get my head around how the soundplane software works, and Id like to understand the principles of how it determines touch x/y/z from the raw matrix.
This is really just 'out of interest' as I love to know how things tick :)

as the source code is open source, I can work through that, but I wondered if there is something a bit higher level.

In particular, are there some details of the basic maths involved, e.g. what maths techniques are used to go from the matrix (which i assume is a 2d pressure values).
(I can then get the specifics from the internet :))

and/or perhaps a max patch that does something similar?

I guess ideally, Id like to try modeling the process in Max, just so i can a feel for it.

note: I know Randy has spent a lot of time refining the touch data, eliminating noise etc, so i know it won't be as good… but more an understanding of the principles involved.
(its this refinements, that make me think perhaps the C++ code might make it tricky to see the basic process)


so i assume I'm after peak detection algos for 2D arrays (3D grid), and looking for one thats not a brute force search thru the grid for maxima (with respect to neighbours).

I've also had a quick check of the DIY projects here, I guess part of this may be a starting point. (just need to remove the bit about converting audio signal, as this is already done in the soundplane)

Q. is the matrix from OSC completely raw, or has it had the calibration data applied to remove 'noise' , if the former, I might look at the source, to see if I can 'optionally' output the later.

I'm glad to help you play with this stuff. It is not documented anywhere apart from the C++ code, though I hope that is very readable. One reason it's not documented better is that the algorithm is fairly often in flux as I try and do a better job. In particular there will be a big change coming within a few months as I do a rewrite to make adjacent touches work better. If this is a big step up, then spending some time documenting things will make sense at that point.

Some algorithms are really clean and profound, this one not so much. I am using quite a few techniques to get smooth yet fast touches and increase the sensitivity. It's like a grab bag of everything that helped. So explore and feel free to ask questions along the way.

The basic interesting idea is that it's not just peak picking, but distance comparing the 2D images from a template representing the ideal touch. This helps pick light touches out of noise.

The matrix output over OSC is calibrated to set levels but otherwise unfiltered, if memory serves.

I support your efforts to get something going in Max, but in my experience that is a pretty bad environment for this kind of task. Or rather, flow-based programming is not a good fit. If I had to do something in Max I would use javascript in max.

I realize this only scratches the surface but feel free to ask more questions.

fair enough, will check out the code.

Max, agreed, I just thought it might be easier to visualize whats going on.
anyway, once I looked through the code i will have a better idea.

A visualizer would be pretty easy to do in Max. I believe I already have one in my Max examples folder that came with the Soundplane distribution.