What if telepresence could be more meaningful and less present?


I had the pleasure of being involved with this project from pre-seed to current Series A hunting.

The fine folks at LeafLabs, who did some serious hardware/firmware for Project Ara, Google's modular phone, decided to use their knowledge and experience to launch a separate startup.

The idea is simple – teleconferencing is stuck mostly on small devices, looks crappy, typically isn't very smart, and doesn't particularly foster collaboration in a deep way. (Yes, I know there are several enterprise-level attempts, and smaller startups working in this space). Ultimately, the specific pitch is to make telepresence feel as "there" as a window in a wall, particularly between remote teams.

What if there were a wall mounted device, with stellar optics, that runs Android (with collaborative apps - task management, drawing, Pinterest, etc), and is smart enough to go away when you don't need it?

This is how it would feel.



That is the final pitch video, and I'll get into some details about the story and filming process shortly. But first let's rewind to what happened between the initial idea and what you just saw.

An idea like this needs time and money to develop. So a little under a year ago, we first had several conversations about the typical pain points of telepresence, what an ideal experience would be like, and then distilled the major points into five illustrations to convey the idea to investors.

I worked with my friend Miles Yoshida on the illustrations. He's worked for Jeff Koons in NY, and sculpts and draws like Michelangelo.




We settled on these things: the setup needs to be easy, the devices should be smart enough to automatically pair and tile, information should be transferrable from one to the other, large interactive task management would be easy and useful, and augmented content, like speaker information and transcription, would be very welcome. Looking back at these, it feels good to see how much of this made into the video a year later.

So guess what? They pitched it, and got the funding!


The Real Work Begins


I have to point out that it was amazing to work with the people at LeafLabs, they have an awesome combination of hardcore MIT engineering skills and business acumen. And I can't speak fully to their efforts during this phase, but I'll share what I do know.

They got to work on building a screen with DSLR quality optics (in fact the final video was shot on the same optics as the device). And they got the video streaming at 4K resolution at 10Mbit speeds, which is approximately entry level home cable speeds, and is an impressive feat in itself. The combination of great optics and super high resolution is something that is hard to describe.

Prior to this, I hadn't given the quality of Facetime or Google Hangouts much thought. The experience is similar to when Apple devices became Retina resolution, once you see the older version it looks terrible! The picture is super sharp, the depth of field super creamy, the whole thing is many steps forward into reality. It looks like what teleconferencing looks like in science fiction movies.







The product was named Portal, and the startup branch of Leaflabs came to be known as Eco. So while Eco was working on the technology we started to explore the nature of the interactions.

Since the initial target deployment is between remote teams, and setting up a "Skype session at 2pm" is not like spontaneous run-ins near the coffee machine, the idea is to have the Portals be always on. This presented some challenges. It's natural to have an "oh that's creepy" reaction to the idea of always on, so we had to think quite a bit as to how to make that work in a good way.

In addition to that, in the spirit of a "window," they made the (I think excellent) decision to ditch the picture in picture that is common for teleconferencing. It's safe to say most people spend just as much time looking at themselves as they do the other person when that's there.

So we did several months of UX thinking about privacy modes, the necessary interactions and feedback, physical gestures, and the bare minimum of UI elements necessary to accomplish these things, ideally with no learning curve. The image above is a compilation of some of the thinking about interface elements and their behavior.

Then it came time for an investor demo last spring. They had built two working prototypes and placed them on separate floors of the building, one in their space and one in the woodshop. In addition to the prototype demos they wanted to do a quick teaser look and feel video that demonstrated the interactions and explored the vibe and voice of the product. Check it out:




And that went well again! We all returned to our work. Eco made additional progress – the device now runs Android (and has touch!) – and we now needed a video to communicate the product to the outside world, where there would be no prototype present. And we needed to tell a story.


The Story


The story is of a startup who sends Brooke to start a new office somewhere far away. She starts alone in an empty room, sets up a desk, gets wistful when the door rings and she's got a package. She opens it up, puts it on the wall, and "hello!" Not alone anymore!

She begins to collaborate with her remote team, makes a friend, invites her to join the new office. Two become many. The team uses the Portal to seamlessly collaborate and have funny moments. They are loosely building a product like Portal itself (it's meta like that). Drawing optics diagrams, deciding what should go into "art mode," and managing tasks. We decided that being too specific about what they're making might feel like a video about that product, so we kept it loose.




This shoot was a doozy! We story boarded every shot. Decided to use only real people, no actors. Everyone in the video is either from LeafLabs/Eco or friends from the building (the lawyer is our friend Rex and is a real lawyer). The CEO is AJ Meyer, the founder of LeafLabs/Eco. The guy in the hat is my old friend from art school, Gerard Patawaran and we have thrown multiple art shows together, so I naturally asked him to help with the shoot (preproduction, storyboarding, staging, cinematography). We did screen tests of everyone we could get our hands on, and asked them to look lonely/wistful and interact with an imaginary portal on the wall. Hannah Verlin from LeafLabs was instrumental in making sure we touched on all the necessary points.

We needed to have something like ten Portals, so we went and green screened everything, shot every interaction from behind the characters and inside the device (x2 to include the other side). I did all of the editing, compositing and a chunk of the animation. Jason Tucker and Michael Degen are these ballers that did practically all of the UI design, rigging, and a large amount of the animation. That awesome task management scene is 100% Jason.

Just in case there are any haters who look at the video and think "psshh, that's not real," whatever. The prototypes are real, and the image on them looks exactly like that. You can nitpick on the lack of bezel if you'd like :)

Actually to be perfectly honest, if I felt like we were over promising in a way that Eco couldn't deliver, I would not have done it.


The Music


I'll end with an wonderful lucky moment. At some point during the shoot, I was up at night stressing about the music. I mistakenly hadn't considered it in the budget and our timeline wouldn't allow for it anyway. We needed something beautiful that felt right and stock music tends to have that generic hand clappy vibe and I don't think I could handle that. Then I remembered that three years ago I had taken the bones of a piano track to my brother's place in Los Angeles and we jammed on it with him and a cellist and that it had been quite beautiful. I looked in Dropbox, and there it was. And you know what it was called? The Leaf. 

Amazing. This whole project is one of the most fun and challenging things I have had the pleasure of doing. Some additional behind the scenes photos below.

If you have a product that needs communicating, shoot me an email!



On the Set

In Other News

Slow Dance


Eco Portal


Patient Alpha




Harvard GSD


Movie Work

2005 - PRESENT