Connected Devices

So this is how we imagined this, as I wrote few posts earlier – lights will turn on to indicate where should you place your fingers, then when you play the chord, it will load the next one and light up next group of LEDs to show you next chord. App on your phone is sending commands to ukulele what chord to show.

How we tackled it.. we:

Made the User scenario movie

Decided what the components we need and made system diagram.. (previous post 🙂 )

Figured out what’s the best way to connect and control 48 LEDs:






Program modes, chords and animations:



Ok its working, lets make it smaller.. Design the double sided circuit board:



All the soldering we did by hand, so we had to check what was working and what needs to be fixed:


Finaly we made the PhoneGap app to communicate with our LED matrix driver:




And test it!

We’ll light all LEDs to see if they are all functioning; Then we send command to play animation – light up random LEDs..


And and see it connected, with the app 🙂


This is system diagram how we envisioned building it:

0.After turning on the device attached on ukulele (for the feedback – LED on ukulele turns green), it starts with opening application which connects to ukulele via Bluetooth. UkuleLED is now connected and ready to receive commands (color of the LED on ukulele is turns blue).

1. User selects lesson inside the app, which then calculates what are the LEDs that need to light up, and sends to command to the Arduino (Arduino EZLink for serial Bluetooth communication).

3. When the LEDs are light up, app is listening for the user to play the chord. Until that happens, LEDs stay light up.

4. When the chord is played, app immediately sends the next chord command for next group of LEDs to turn on (of course, all previously turned on LEDs are being turned off ).

— it continues till the user shuts down the device down, or connection is lost (which is again displayed on ukulele’s LED).



We both ukulele in parts, this how cute that is:


This is how and where we are planing to add our circuit board with LEDs, between fret-board and neck. Arduino, 2 LED drivers and EZlink will come somewhere at the end of the neck.

ukulele structures-01

Bill of materials (for now) looks like this:

$40 Grizzly H3125 Ukulele Kit Amazon
$17 56 leds
🙂 circuit Board floor
$7 Arduino Pro Mini 328 – 5V/16MHz Sparkfun
$25 Bluefruit EZ-Link – Bluetooth Serial Link & Arduino Programmer – v1.3 Adafruit
$36 2x adafruit 24 channel 12- bit PWM LED drivers Adafruit
$2 RGB LED Adafruit
$1 switch – Adafruit

Project with Ma Tan and Seiya. Here is the post Ma Tan posted, I’m just copying for my records 🙂


When we try to learn how to play string instruments like ukulele or guitar, we often have to go back and forth between the text book or website and the instrument, which is really time-consuming and tired. This is one of the biggest common bottle-neck for beginners because it prevents them from moving forward the next step where they can actually enjoy playing music so they can’t reach the real fun.
But we believe we can solve the problem and lead beginners to that fun phase with less stress by teaching the chord right on the finger board instead of textbooks or websites.
Displaying where to put your fingers with LEDs embedded on the finger board, and sensing which note you played by making a closed circuit with the conductive string and the metal fret you pressed.
Use scenario:
 VIMEO LINK  explaining all. **Wish this wordpress alows vimeo **
connection ways:
 We have 3 different ways to detect the different note of Ukulele:
1) Analog inputs
unnamed (1)
2) Sound Recognition and analysis
unnamed (2)
3) 56 press sensors
unnamed (3)
Phone app wireframe:

I don’t play with it any more.

Place I live in is small enough with all the furniture in. Maneuvering Sphero in space like that is more frustrating then fun. I keep on hitting furniture heh  Controls are great, need to emphasize that. It’s super responsive and with all special tricks you can do makes it even more endearing. It’s the space you need. I feel you need at least a 10 x 10 feet patch of open space, at least. I imagine playing with it in a park would be awesome (Imagine the same for dogs there). For curiosity sake, I tasted it in one of the hallways we have at ITP, it can go pretty far without disconnecting (30 feet).

I never used ramps that are included. Sphero is plastic and weights just enough to seem not proper for the wooden floor I have. Aside the loud sound I suppose it would make, I don’t dare to test its endurance-mostly ’cause its not mine.

I downloaded more apps, so I have more things to write here. There are literally lots of them. I planed to picked few to test. All of a sudden I was downloading 5 at the time, because they all looked so exciting. Even thought the ratings were not all the best, they looked so exciting – like the Sphero packaging box.

Virtual reality looks so good, but get frustrating fast because Sphreo is always leaving the screen. I go to shoot a zombie on the side and off it goes, I then have to chase it with my phone (I don’t have bigger screen then IPhone and Kindle is not supported). The same with Sharky game. I tried being closer and further and lowering the speed, still my score never passed 0 🙂

Much much more fun is using Sphero as a game controller. You hold Sphero in your hand and play the game by tilting it and rotating. Games vary from really simple – just rotating to more complex actions like – combining rotating and tilting. These ones require some practising, and keep me engaged a bit longer then the simple ones. You get, for example,  a spaceship that is always firing, so gameplay is mostly moving and aiming. Maybe it’s the right time to say I’m not a game person?

The one I enjoyed the most is called Color grab, where your reflexes are tested to see if you can pick up the Sphero when it is a particular randomly chosen color. It is less screen based, and you compete with other person, in person (?). Screen is arbiter there –  to give commands, choose color and set your score. When I was reviewing which apps to download, I found this video on youtube and that shows how the game go.

It’s the coolest toy I had in my life that I won’t find time to use. Kids and pets thought…



So I have a toy. Cheerful, colorful, fast little toy that doesn’t mind my (bad) experimental handling. And that’s the best part –  it’s a toy, I shouldn’t be afraid trying new things with it. What keeps me a bit away from it is the battery. It takes three times longer to charge it ready to play. Then you have an hour of battery life.. but not the whole hour of my day to spend playing with it. There are lots of different apps, for different games and I’ve downloaded a couple. For curiosity’s sake.. and because I was googling again, and this is important feature I should be aware of already, how to put Sphero to sleep. Turning on and off things should be intuitive, right? Weather reading from device itself, box or even *even* even* the manual. “leave sphero to charge and it will go asleep”. Hm.. but I want it inside a box, now what?

Might be a good way of forcing you to keep it on the charger, so it simply stays charged and ready for next play – yeah. I still wish there is a way of taping it, shaking it, some similar way of interacting with it like when you are waking it up. Because, I was lending it to classmate.

And imagine, there is an app Sphero sleep  🙂 And this is so much easier than opening the Sphero app, waiting for it to connect, select “Just Drive”, open settings, THEN put Sphero to sleep. This is one of the features I never use. The app should be one button inside the main Sphero’s app. It’s ridiculously complicated for command that often (worse then shutting down windows computer 🙂 ).

Rarely I use the tips – game like tasks they offer inside an app. I’m sure this is amazing for kids or game-lovers, I’m interacting with it when I take short brakes from work. Phone is always at hand rich, so does the Sphero. Short ride around the place, changing colors, see how far it can go without loosing the signal; are my maneuvers getting better, can I drive it around the wall and manage to bring it back without seeing it?

IMG_0383Oh that’s great. Few relaxing minutes from work. I’m seeing it almost like a pet. Wish it can jump out of the charger by itself – controlled with an app ( without an actual physical interaction ).

A robot- ball, yay! It was a ‘Christmas day’ on a Connection Devices class – everyone got a device to live with for 3 weeks.

** now I’m going to break it down a bit. I’ll sum the experience describing the appeal of the object, packaging, interface and interaction – both lovely and frustrating moments  **

Lucky enough, I was first one to pick. Box itself was exciting!

image (1) imageimage (3)image (4)


I think it has all it needs, but not too much. Took me few seconds to glance over all sides, to can’t wait to open and try it. Oh I just with the battery is charged, but lets see. A tiny instruction inside, all in illustrations and less then 10 steps, wohoo! Is that like a single minute? 🙂

image (1) image (2) image (3) image (4)In 5 seconds I should know if my ball is charged, right? That’s what it says. And here comes the frustration:

??Wait, it doesn’t explain flashing blue, just the constant blue. Let me try to wake her up. No, doesn’t work. Ok, back to charging. Charger is blinking? ball says nothing? Did I set it properly? Heavy part is at the bottom. Then of course, I’m trying to adjust it, rotate it a bit there, a bit over-there; I’m lifting it and putting back.. lifting it a bit.. oh! it started glowing in all colors, when I lift it a bit, but for a few seconds and nothing else. It goes asleep again, or what?

Instruction is so readable, simple and short, but I wish it says what are the indicators of charging, or what all that different colors in the ball means. And here comes Google, and that’s a lapse, from UX view, or at least mine.

How do I charge Sphero? I found it fast, it’s even in FAQs, but was I suppose to google before even trying it?

**Three hours after this and charging, everything worked just fine. The ball woke up, and everything was so logical. Interface app is fairly simple, and for now, it’s rolling 🙂 I will comment more on the interface later..

There are so many more apps for Sphero and soon I will be trying them. Anything worth mentioning, I’ll mention here.

A coffee cup, for people always having one in their hand; Navigates with simple light signals (for left, right and forward).

It also vibrates when telling you to stop, along with flashing red lights. When you reach destination, vibrates in different manner, followed by green lights 🙂


upcoming turning: flashing yellow for 10 sec, then green light turns on

turn : green light on

forward : flashing green

turn 180, go backwards : coffee lid turns 180 degrees

stop : “hysteric” vibration with flashing red lights 🙂

stop, you’ve reached destination : ‘pleasant’ vibration followed with green flashing lights