tappr.tv 11.0 on iPadOS, iOS
After 13 years, 11 versions, and over 5,000 dances, created by a global community of 100,000 dancers, tappr.tv is finally getting its first show! In “deeje Presents”, with 10 episodes and 40 dances, I’ll give you an inside tour of the first decade of digital dancing.
The show launches on International Dance Day, April 29th 2023, and ends the week of July 4th, U.S. Independence Day. Each episode will premiere via app store events and a lengthier episode viewers guide published here.
To kick things off, let’s start with a brief recap of tappr.tv
Visualizing Music
In the late 1990s I had a vision of visualizing music. Not just for my own visuals, I saw a space where anyone could create visuals to their favorite music. The VC pitch could have been “Animusic for consumer generated content”.
Emboldened by my vision, I quit Apple, lived off AOL stock, and started building an animation system. Turns out, visualizing music is just a fancy way of saying “dancing” and input devices of the day (e.g. mice and keyboards) just can’t dance. With that realization, I pivoted into screenplay visualization. (That’s a whole ‘nother story for another time.)
The iPad
I spent the first decade of the 21st century as a freelance iOS developer, sometimes getting paid, and sometimes taking equity. In 2010, Apple announced the iPad, and I knew instantly it was the right device to dance on. I immediately started working on tappr.tv, with basic wands in v1 at the end of 2010, followed by tracks in v2, dances in v3, and publishing and community in v4 by the end 2011. It was a heady time, to say the least.
After that, the community took off on its own. Not like a rocket, but slow and steady, new users found the app, liked and rated dances, found their way into studio mode, recorded their own dances, and tapped publish.
It was amazing! Soon there was 100 dances, and then 1,000.
To this day, every time a new dance is published my heart skips a beat of joy.
Architecture
One thing I pride myself on is architectural considerations. I spent a ton of time thinking about motion capture and playback aka mocap. I designed a system to capture and pass thru all the motion sensor data on your mobile device over time. Sensor data includes touch inputs, gyroscopes, and accelerometers. My previous experiences with QuickTime guided my designs around a clock interface to drive capture, process, record, and render systems, strung together in ad-hoc streams.
Today I might have built this system on top of Combine and AVFoundation. But as of now, I have a mobile mocap system that is screen agnostic, input agnostic, render agnostic, very efficient, and bug free. I am not inclined to refactor it right now.
Refactoring
That’s not to say that I haven’t had refactoring challenges. In fact, I’ve had 4 crucial refactors in the last decade.
The Stage UI
The first was moving from a crude UI with a squiggly font, to iOS 7 and its modern focus on depth, deference, and clarity. I wrote a whole article about that “ Scaling tappr.tv from 3.5” to 12.9" “
The big win here was implementing the “Stage UI”, where the navigation elements appears on panels that behave like curtains, covering and revealing a stage where dances occur. The curtains are “frosted”, so you can still see the blurred dance even when you’re looking up dance or dancer info.
CloudKit
The second big refactor was moving from Parse to self-hosted Parse to CloudKit for the backend. I was able to abstract out the server endpoints into a nice singleton class, and then fuse data from both backends. In subsequent release, I was able to migrate all the data over to CloudKit, without app call sites knowing any difference. Eventually Parse was retired completely.
The big win here is that I no longer have server/hosting costs to worry about. As long as there are people dancing in tappr.tv, the content from the community is preserved thru their iCloud accounts.
Apple Music vs Spotify
tappr.tv doesn’t deal with music directly, and isn’t possible without the Music Player API in iOS. That API is probably one of the oldest in all of iOS, with influences tracing back to the iPod. With it, I can find a song by title and artist, start and stop playback, and get and set the current playback time.
At first, all of the streaming music services (i.e. Spotify, Rhapsody, et al) offered iOS integration libraries with basic music playback, and I abstracted an interface to plug them all in and let users decide which to enable. Unfortunately, one by one, most of those streaming services died or strayed away from integration APIs (scowling at you, Spotify), while at the same time Apple Music emerged and fully embraced third party developers.
As such, tappr.tv is now available exclusively for Apple Music subscribers, who can watch just about every dance available.
SceneKit
The fourth, and perhaps most significant, refactor was from Cocos3D (an open source 3D scene graph API) to SceneKit. Because I wanted dances to retain their original appearances, I also had to dive into Metal to reimplement the particle emitter system from Cocos3D.
This refactoring paid off handsomly when ARKit was announced. I had AR playback implemented in a weekend!
Touring
On top of the mocap architecture, I implemented splitters, network senders, and network receivers, which allows tappr.tv to have a number of exciting multi-device experiences.
For instance, I can have a central iPad playing a recorded dance thru a projector and loudspeaker. Then I have a set of other iPads locked in wand mode sending motion data from IRL dancers, back to the central device which receives and renders them in realtime on top of the recorded dance. Basically, you’re dancing with strangers! I put this setup in a dome with several bright projectors, curated a “playlist” of dances, and took this show on the road! It’s been to IllVille@BurningMan and the Santa Cruz MAH Light Festival several times over.
AR and VR
The promise of AR and VR has been tantalizing us for years now, and in 2018 I integrated with the emerging Google Cardboard ecosystem, as well as the Bridge 6DOF VR headset from Occipital. Both were f’ing amazing experiences for tappr.tv dances, and the 6DOF was particularly delightful. Unfortunately, Google got bored with VR, and Bridge was soon cancelled.
ARKit is still implemented, and it makes a great demo just holding your device up and walking around a virtual dance IRL space.
deeje Presents
Over the years, tappr.tv has seen its dancer community cross 100,000, and its published dances pass 5,000.
And I’ve watched every one of them.
Sometimes, people kinda figure it out, but the published dance is just a little bit of random noodling across the screen.
And sometimes, people really get it. They play back their favorite song dozens of times to record multiple wand tracks, and produce truly stunning visuals. When a dance has 2–3 full tracks, it’s great. When they have 4 or more, they are usually awesome
And that’s how I rate them: 3 stars for good, 4 stars for great, and 5 stars for awesome. After 13 years, I’ve rated over 500 dances with 4 stars or better!
After a while, I realized I needed a better way to surface all this great content. Thus the concept of a tappr.tv “show” was born. The first show “deeje Presents” is personally-curated and hand-coded, has 10 episodes, and each episode has a theme and 4 dances. The idea is you can watch an episode per week, in between whatever other entertainment you’ve queued up. It is available on iPadOS, iOS, and tvOS.
I hope you enjoy the show!
Originally published at https://blog.deeje.tv.
