Have managed to catch a quick break and as such, I’ve decided to upload the teaser video we presented as ISMAR 2011 for our poster session. Given the recent progress and developments we have made in the past few months, we will be able to post a more detailed video once we clear up all of our obligations and responsibilities (read: once we get some more papers submitted and published).

Enjoy,

-Nick D.

I’ve finally managed to schedule in some time to update the blog. Unlike previous blog updates (which were little more than changes to the front-end template), I’ve decided to start from scratch and transfer the content to this new incarnation. This was necessary as the old site had some obsolete, modified code that was causing problems throughout the entire site. Given these bugs, I figured it was time to start fresh. I’ve also been optimizing the website to render well on mobile platforms, like the iPhone and iPad. There’s still much work to be done, but the site’s in good enough shape that I feel comfortable publishing the changes.

In making these changes, I’ve implemented a new section that’s near and dear to me: the Portfolio section. Most of the pictures I’ve uploaded here have been shown via other platforms (like Facebook and Twitter), but I’ve decided to consolidate them onto my site for two reasons: 1) I don’t like the intellectual property policies of Facebook and 2) Fotomoto.com.

Fotomoto.com is a website that allows artists to upload their works and they will do everything, from printing to delivery of your images. If you have a set of photographs or images you’re proud of and would like someone to distribute them, Fotomoto’s the way to go. And no, I’m not being paid by them to say this.

In other news, I’ve updated the Beyond the Surface page to take into the account the latest videos made for the Microsoft Research Summit that took place in July 2010. The next phase now is to come up with some useful test cases and validate these designs. After which, we will go through another round of paper writing. Also, I’ve begun a new personal project called Project Jupiter. The details on Project Jupiter are pretty scarce, for the moment. Once I progress further with it, I’ll be sure to update the page.

Enjoy!

I’ve decided to start a new personal project called “Project Jupiter”. My goal for this project is to create an inexpensive augmented reality-enabled cyborg suit. Quite lofty and ambitious, I agree, but I am confident that we have reached a point where a system like this is quite doable.

Surprisingly, I was inspired by a movie I saw over the summer called “Scott Pilgrim Vs the World“. Without revealing much of the movie, the bit that inspired me most was that the characters were living, in effect, in a pseudo-augmented world, populated by sights and sounds associated with video games. Afterward, I found myself seriously pondering what it would be like to live life like that. And thus, I’ve decided to start this project, to explore how a system could be designed such that its user would be immersed in a kind of video game world.

Stay tuned for more updates as they come.

Pardon my absence, but for the past month I’ve either been traveling, moving to and fro New York to Sacramento, CA, busy with my new internship at Intel while at the same time working on my research project. While I’ll elaborate on these adventures in upcoming blog posts (as well as update and implement a new feature on this site), I just wanted to share a video of something really interesting and related to the field of Augmented Reality.

While going out with a friend last Thursday, trying to disconnect from work and research, we stepped outside and saw this beautifully orchestrated spatial augmented reality presentation made for the World of Color presentation at Disneyland. It was made by the kind folks of Obscura Digital, based in San Francisco, New York and Tokyo.

Enjoy

We’ve updated the tracking system of Beyond the Surface. Right now, we’re using NaturalPoint’s Optitrack trackers. These trackers triangulate the positions of markers placed on rigid bodies. The benefit of this tracking technology is that it doesn’t require for the camera to recognize any markers. Thus, we are able to get very close to the virtual models, close enough that the following videos would have been impossible under marker tracking technologies.

There are two versions of the same scene (Flatiron and surrounding buildings in Manhattan). One shows the models in higher detail, but is recorded from screen shots and as such, doesn’t show how fluid the system is. The second video was recorded from an external camera facing the monitor. It is of noticeable lower resolution, but it demonstrates how the system responds in real-time.

Enjoy!

Beyond the Surface – Flatiron building – CamStudio version from Nicolas Dedual on Vimeo.

Beyond the Surface – Flatiron Building – External Camera Version from Nicolas Dedual on Vimeo.

Considering that right now I’m in the midst of temporarily relocating to California, wrapping up research and packing, it seems rather insane to sit down and write a blog post. However, when the Muse speaks, I’ve learned that it is important to listen and, at the very least, make notes.

I went to Rome in mid May, for among other reasons, to celebrate my birthday. Being by myself afforded me with the time to internalize many things and appreciate much in my life. It’s rather unusual for me to share my insights with the world but I felt compelled to do so.

Rome was, is, and will always be an amazing city. From the Campidoglio to the Palatine, from Trastevere and Testaccio to the Vatican, Rome will always leave you with a sense of awe, inspiration and with blisters on your toes from all the walking and dancing that’s there to be had. Al least, I have blisters and hopefully you will too should you ever visit it :-D. I spent a lot of my time on this trip around the ruins of Ancient Rome. It was exhilarating to know that one was standing on top of thousands of years of history. Think of it: perhaps at that very spot, thousands of years ago, a Roman citizen could have witnessed the cremation of Julius Caesar, or seen Hadrian’s triumphant return from Jerusalem, or just stood aghast as the Goths ransacked the Eternal City. Rome is teeming with stories to tell those who are willing to listen.

Though, not all stories need be ancient. While I was contemplating the view of the Roman Forum as I sat perched atop the Palatine Hill, I was mesmerized by the whispers of jazz that emanated from the Basilica of Maxentius and Constantine. What the song was, I do not know (though I know it was clearly sung in English). However, the melody was soft and serene, soulful and forceful, and the singer sang with a pain in her soul that was palpable. I could not help but be moved by the music and the setting, and I scribbled down my thoughts and recorded what little I could of the moment. Perhaps later on, I will expand these thoughts and commit them to pen and paper.

At this moment in my life, it would be a shame if I went to Rome and did not have any “Dolce Vita” moments to share. Rest assured, I did :-D . During the weekend, I met two fellow travelers at the Colosseum, who were basking in the sun and enjoying the view, whilst taking pictures and chit-chatting. After joining in their conversation, they invited me to come along with them in their plans for the day. Which, in fact, were no plans at all save to get some food, Peroni’s (Italian beer) and just walk about Rome. The Germans have a word called “Wanderlust” to describe the strong desire one has to wander around, which is exactly what we did. Throughout the course of the day, we played music in front of the Forum with whatever instruments we found, made nonsensical hand gestures that described in detail the woes of traveling through an area affected by an active volcano, embraced the history around us and made stories up whenever we were not aware of what the actual history of the place was. All the while imbibing from the drink of Dionysus, found cheaply at the local supermarket. Looking back, I must say it was great to be able to relax and enjoy the moment with good company. While I did not partake excessively in the consumption of alcohol, I was able to appreciate why the god of wine and drink is also known as the “Liberator”.

And yet, at the end of it all, after one of our party left us at the Spanish Steps, and the remainder of us sat in awe at the beauty of Vatican City, I had to admit that, while this moment was great, an existence based on just enjoying these kind of moment is a sad and empty life. For example, if the only way feels one is able to “liberate” oneself is solely through the use of alcohol, well… those are just excuses now, aren’t they? In the end, alcohol can never force you to do anything: it just gives you a short blanket of plausible deniability that will never truly cover up your actions. At the end of it all: there is no substance to a life purely dedicated to pleasure seeking. There is nothing that one can reflect upon and say with pride “I did this and it has made the world better”. Then again, a life without any sort of pleasure is a cruel and hypocritical life. I believe Kahlil Gibran said it best: (see inset for quote)

Pleasure is a freedom-song, But it is not freedom.
It is the blossoming of your desires,
But it is not their fruit.
It is a depth calling unto a height, But it is not the deep nor the high. It is the caged taking wing,
But it is not space encompassed.
Ay, in very truth, pleasure is a freedom song.
And I fain would have you sing it with fullness of heart; yet I would not have you lose your hearts in the singing.

I would encourage you to read the rest of the poem here, as I believe that the entire poem, not just this quote, is apropos to the topic at hand.

Like a flash, it dawned on me that for the past year of my life, I have been living a lie. I have been trying to live and enjoy a life of the pleasures of the “here and now”. Perhaps this has been so because I’ve felt I’ve been missing out on the things typically done by people of my age. And in part that is true: for some time I’ve wondered what it was like to enjoy the moment without worrying about its repercussions, how it will affect my life later on in the future.

While that lesson is valid, it has been learned at the expense of losing a big part of who I am. As I was strolling along the river Tiber, the night after my “Dolce Vita” moment, I realized this. I realized that for several years now (and large part over the last year), I had lost myself in myself. Before I left for Rome, a great friend told me that she didn’t see in me that spark, that fire that burnt deep within me in years prior, and that she hoped that I would regain it once more. I think that in Rome I was able to rekindle that fire in me. Though right now it is a small fire, with time and dedication, I hope to make it grow higher. Only time will tell.

Speaking of friends, I met up with a good friend and colleague while I was in Rome. One night, as we were having dinner, he said in the conversation the strangest, yet most poignant thing: “Everybody is expendable”. I’m not so sure how I feel about that, or whether I agree in whole or in part. Sure, you could argue that some people in one’s life are expendable (like someone who cheats on you, or leaves you alone while “visiting” someone else, or casually forgets to send you a birthday wish), but I think I disagree. I don’t think anyone is really expendable: everyone has a lesson to teach us, everyone has a purpose in one’s life. The trick is to embrace the lesson they have for us. To imply that someone can be tossed aside is to refuse to acknowledge the effect (whether good or bad) this individual has had in your life. And, really, the only person you’re fooling is yourself.

And as the dinner carried on, I caught myself thinking about friendship and what it means to be a friend. I had not seen or heard from my friend in several years, yet he welcomed me with open arms when he knew I was in Rome, took me out to dinner, showed me amazing sights for my birthday and, in the end, appreciated me for myself and also the bond we shared. I again remembered something I had chosen to forget: friends are not a convenience. You should never consider a friend those you reach out to you only when they’re lonely and need someone to talk to, or whenever they need your help. Friends are those people whom you are proud to be associated with, who will tell you the truth especially when it hurts, who look out for your best interest when you yourself are not able or willing to. “Friends are the family we choose for ourselves”, as Edna Buchanan said, and like all decisions, it should not be taken lightly or abused upon.

There is far more to tell about Rome and about myself than I can write in a blog post. I do not know what this new year in my life has in store for me. I’m sure I’ll have good moments and not-so-good ones. But, if I can remember and keep prescient the lessons I’ve learned in Rome, I will be my best by my standards. And in the end, that’s really all one can ever hope.

I share this because, in my opinion, it is very cool.

Last year, I was fortunate enough to visit Ireland once again. This time around, as I was staying in County Ennis, I made sure to see the Cliffs of Moher.

It is a place of breathtaking beauty that nature saw fit to share with the Irish. I hope that the following picture captures some of the essence of what I saw.

However, this post isn’t about the Cliffs themselves, but their visitor’s center.

The Visitor Center of the Cliffs of Moher strives to show not only the geological processes that brought about the cliffs, but it also highlights some of the legends associated with the Cliff, its history, how precarious the entire ecosystem is and how global warming, if unchecked, will lead Ireland back into another Ice Age.

And they did all of this using multi-touch projection tables, and CAVE-like environments. In all, the Visitor’s center executed brilliantly how technology can complement storytelling and serve a greater function than just looking cool for the sake of looking cool.

To give you a taste of what I saw, I was able to record the CAVE-like setup as it was showing a video of a seagull flying around the cliffs. When I was there, I felt I was immersed in the scene, even though there was no head tracking and, at times, I could see my own shadow. Still, I hope you will enjoy this video as much as I did:

Let’s Go from the Whiteboard: Supporting Transitions in Work through Whiteboard Capture and Reuse

This paper was presented by Stacy Branham from Virginia Tech. It’s a system that not only captured the contents you write on the whiteboard, but also allows you to share, reuse, tag, and remotely access this information. You can see ReBoard in action at this link . One of the main problems that ReBoard addresses is that the information written on whiteboards tends to be ephemeral or, worse, fixed in concrete lest we ever forget that spark of inspiration, effectively blocking that space on the whiteboard from ever being used. By transferring the contents of the whiteboard to other mediums like the web (and even Chumby’s) and, more importantly, allowing for that information to be tagged and searched, it allows for that information to exist beyond the whiteboard in ways that are useful to the user.

Deception and Magic in Collaborative Interaction

This paper was presented by Joe Marshall from the University of Nottingham. His paper showed how augmented reality and the magic of deception can be used to enhance the user experience and extend traditional HCI techniques. A brief demo video of the application he developed can be seen here. Aside from the fact that I love magic, I  agree wholeheartedly that the art of magic has a lot to offer the HCI community. Given that most user dynamics with a computer are built upon an inherent layer of trust, it is quite interesting to explore how this level of trust can be improved and, at times, manipulated by the machine.

Another reason why I love this presentation was because it had, in my opinion, the best slide one can expect from in any conference:

Demos and exhibits

The CHI conference grants the HCI community with the opportunity to see some of the best HCI products and research that the community has to offer. Though I was not able to see all of the exhibits, from the ones I saw these were, in my opinion, the best.

Pinch-the-sky

The team at Microsoft Research, headed by Andy Wilson and Hrvoje Benko demoed their Pinch-the-sky dome. This video does a far better job at explaining its function than I could. I should note that Hrvoje Benko is an alumna of the Computer Graphics and User Interface lab at Columbia University!

Robots

The individuals at Aldebaran Robotics, based in France, showcased their Nao robots. When I first saw him, I immediately thought of Mega Man, the 80s video game character. Here are some of the pictures of Nao in action

ARDomino game

Last, but certainly not least, we have the ARDomino game that the Computer Graphics and User Interfaces laboratory at Columbia University demoed at CHI2010. The object of the game is simple: Shoot all of your colored dominoes out of the table. The trick is that the dominoes are only visible through the game display. I’ve made a quick edit of some people playing the ARDomino game.

ARDomino Game demonstrated at CHI2010 from Nicolas Dedual on Vimeo.

CHI2010 was an amazing experience that this brief blog post cannot begin to summarize. If you ever get an opportunity to simply attend, I highly recommend it. You’ll then be able to see (and hopefully meet) those who are shaping the future of computing.

Enjoy!

Beyond the Surface: Sneak preview from Nicolas Dedual on Vimeo.

We propose research into the design and development of hybrid user interfaces [Feiner and Shamash, 1991] that put the multiuser tabletop user interface afforded by Microsoft Surface into a rich context of additional displays and devices, ranging from hand-held, to head-worn, to stationary. The idea is to supplement the display and interaction volume established by and anchored on Surface with the capability to interact both above and around it. Our goal in this proposal is to build on previous work to create novel software infrastructure, user interface techniques, and example applications that emphasize their usability through real-world scenarios. Together, these will support collaborative interaction inside a computational volume, within many users and their devices are located—displaying, using, and exchanging data. We propose to do this by using a combination of software infrastructure including Microsoft Surface, Windows 7, and Goblin XNA (our system for research in 3D user interfaces and games, built on top of XNA Game Studio 3.0 and distributed at http://goblinxna.codeplex.com).

For example, hand-held devices such as smartphones and UMPCs can be tracked on Surface to present additional information. These devices can also be tracked in 3D position and orientation above Surface using a range of possible approaches (e.g., a sub-Surface camera, as employed in Second Light; optical tracking by a camera in the hand-held device itself, viewing known material displayed on Surface; or the use of a variety of 3D tracking systems currently available in our lab). See-through head-worn displays can also be supported and tracked similarly, augmenting the 3D volume above Surface in stereo, customized to the information needs of the individual user. Additional stationary displays can increase the amount of shared space available. One domain that we propose to use as a test case is that of urban visualization, creating a vivid, manipulable representation of an urban environment that can be simultaneously explored and modified by multiple users. Here, we will leverage our ongoing work on situated visualization of the site of Columbia’s planned northern campus expansion in Manhattanville.