· or — how to a designer can do pattern recognition using machine learning

4 min readJun 22, 2018

As an industrial designer I like to make things by just gluing other people’s work together. In ‘better’ fields than mine they call this “standing on the shoulders of giants”, I call it “I can google things and make it work so pay me please”.

Now, one of the big thing I have been playing with is Machine Learning. It’s super cool, it’s the wave of the future, and it’s a lot of math for designers like me (well I have a bachelors in mechanical engineering but I traded math for people skills so I am a designer now).

Here is a link to an article I am going to write explaining what machine learning actually is and how a designer can use it: but for now the reason for why it’s not there.

Steal the first thing: the wekinator

The wekinator is a super cool project because it lets the dirty paws of designers and artists all over the statistically driven world of machine learning without them needing to do any math. There are a ton of cool examples, but today we are going to focus on the “ Creation of systems for gesture analysis and feedback”. Full credit for the initial work and amazing vision goes to the creator Dr. Rebecca Fiebrink, she is amazing.

Step 1: download and install shit

install everything. If you have any problems I feel bad for you son, I got 99 problems but Googling usually solves them.

(optional) Step 2: have it all fall to shit

well for me the example programs would not start, luckily like any designer with experience grabbing things off an arduino I have processing and every example has a little source folder with the PDE file that you need to open, install all the required scripts and then you can run it. If you need help with this, ask bellow because I also got a bit lost.

But for now I am going to assume you can run the examples!

Step 3: cry at the beauty, cry at the lack thereof

Okay, you want to the following scripts:

the wekinator itself
Inputs\Mouse_ForDTW_2Inputs
Outputs\Processing_TriggerText_1DTW

Yeah, things like color, font choice, UX design can be solidly classified as Maker or just plain bad. But that’s for another time, for now we’re going to input the settings on the wekinator

Step 4: set up the wekinator (you’ll mess up, that’s okay)

So, let’s set things up, first you stop, collaborate, and listen (and tell wekinator to start listening)

Then you’re going to tell it the number of inputs (2 as it says on out beautiful black input program).

Then switch it to dynamic time warping which is wekinator talk for Time Series Analysis.

And finally tell it we have 2 gesture types (dot and dash?)

Step 5: let’s jam (and by jam I mean train)

We’re going to start out by training something simple, gesture 1 is going to be a ‘ –’ while gesture 2 is going to be a ‘|’ (these are nice and easy to differentiate).

So just draw a ‘–’ (but probably good to draw an em dash so it’s nice and clear). Just draw it once. The wekinator should now show you this:

if you accidentally clicked a bunch, just press the minus until you’re back at 1.

Now to switch to training gesture 2, just click on the top of the black mouse input program (so you don’t accidentally train it), press 2 on your keyboard so it;s training number 2, and draw a ‘|’.

Now gesture 2 should have a one, and the party can start (if this is messed up, google, try again, or just ask down in the comments).

Step 5: C wekinator, C wekinator run, run wekinator run!

First, switch the mouse program to running, then press the big ‘run’ button on the wekinator and start running!

wow, such click, such drag

So, what is the wekinator doing during this time?

would be great if windows gamebar would record the screen or I was getting payed so I cared enough to get OBS up and running but we live with what we can!

It vorks! It’s vorking! Make sure you change the Match threshold on the bottom if the green circle isn’t lighting up. You can see nicely the probability of what it classifies. See if you can trick it!

You should be able to see the amazing output:

Step 6: conclusions and future work

You might not be too impressed about our binary classifier, I mean I am not, but it is the start of something cool. You just got a machine to recognize something. In the next week weeks and articles, I am going to describe and play around with different inputs and different outputs (especially the out put) but it’s all based of this same idea: give the wekinator all the information, train it, and then just let it run and do all the logic.

Tell me what you think, share your projects and processes, and see you soon!