The art of storytelling can get lost in the digital age, where the restless intersecting of images, text, and flashy graphics can work wonders or simply add non-conceptual noise to an already crowded field. But the bright side to this distracting picture is the excitement of making road maps into a new storytelling world and inventing methods where none exist yet—that’s what the researchers inside the future cinema lab, led by caitlin fisher, ma/93, are trying to do
The idea behind the word “lab” is one of experimentation where failure is welcome. After all, you can figure out good new ideas by eliminating the bad ones. It’s a liberating premise—and a central principle of the Future Cinema lab. Mess up a few times, and what comes after can make headway in the developing field of augmented reality (AR).
That perpetual state of possibility is what keeps Caitlin Fisher, MA/93, in motion. She is the Canada Research Chair in Digital Culture at York University in Toronto, where the Future Cinema lab is housed. Fisher set it up in 2004 within York’s faculty of fine arts, using grant money from the Canada Research Chair program, which gives the university $100,000 annually for five years to support the work of “exceptional emerging researchers.”
The lab is a large room with a tracking system built into the ceiling, which allows for the transposition of physical movement into a virtual scene. Images can be projected onto the curtain of mist of the nightclub-worthy fog screen. There are all manner of computer screens on walls and tabletops, a futuristic-looking bit of headgear ready to use and a number of low-tech telling props, including an old barber’s chair and a stack of suitcases. The room looks simultaneously futuristic and as plain as a high school drama classroom, with its repurposed castoffs. Those discards are key in bridging the narrative from offline to online and back again.
That’s because the physical objects hold radio-frequency identification tags and other digital markers that act as gateways to take them into the digital world. Using a handheld screen with a camera—something like an iPad 2—the “reader” (or experimenter) can enter the virtual space and and use the sub-layers of enhanced visuals, sound and text available in the software to explore the new forms of storytelling. Sometimes augmented reality users describe the effect as inanimate objects having secrets, so the user can poke into the action and see what pops up randomly through use of the software.
That sense of magic is akin to the Cinema of Attractions, when early filmmakers realized the image itself could represent more than the story behind it. They discovered they had new tools to show and not just tell, Fisher says. She speaks quickly and excitedly, jumping around to related arguments and ideas. You instinctively step closer to make sure you don’t miss anything as you try to follow her trajectories. It’s as if she’s speaking like an AR piece, where every word is a springboard to a new conceptual avenue.
“It’s like film at the beginning of the last century. The rules are still unwritten,” she says. “I’m interested in this moment as a pioneering movement, where conventions are yet to be established.”
Projects coming out of the Future Cinema lab play with narrative convention by telling and retelling stories in non-linear ways. Student Geoffrey Alan Rhodes created an interactive story called “52 Card Psycho.” He placed a marker on each playing card of a deck to mark 52 different images from the shower scene in the Alfred Hitchcock film Psycho. The user can rearrange the cards to create a new storyline to question the idea of narrative. The artistry is in the remaking of the montage with every play.
Often, in AR, researchers start with a conventional narrative and then tinker with it, using a suite of electronic linkage tools tied to real-word props, which act as markers for a new kind of storytelling. The tinkerer becomes the tailor of the tale.
Other iterations include tabletop theatre, where a series of inanimate objects holds different digital codes and each item tells a different part of the story. A piece on pioneering filmmaker Georges Méliès casts him as the guide to AR’s “fantastic future” and is projected onto the fog screen once the reader activates the picture with cards and a code-reading device.
The lab’s augmented reality imprint, Future Stories Press, will be launching a series of hybrid physical-digital titles this year, placing markers on the work of Governor General’s Award-winning illustrator Wallace Edwards (Alphabeasts) to open up new digital worlds that are still co-constituent with the physical world.
The key in transmuting these ideas is to be limber with the tools of the trade. To that end, Fisher and her colleagues created SnapDragonAR, a drag-and-drop authoring software that has practically no learning curve.
It was especially useful, since most students came to the lab without programming experience. “Students had amazing ideas that couldn’t be executed with existing tools and their skill sets,” she says, “but they should be part of the storytelling conversation.”
That conversation is ongoing, and keeping those ideas up in the air is a central tenet of the Future Cinema lab—to train a generation to experiment with their own stories in ways that go beyond the flashy tricks of computer-enhanced advertising. Ease of technology and the ability to get ideas up and out there means researchers can get to the bigger questions surrounding AR technology.
“How do we imagine future interactivity?” Fisher asks. “The central questions are around what’s emotionally compelling.”
Aside from emotional appeal, the ability to visualize huge amounts of information and make it intelligible is increasingly important. Just think of your own life. Do you have 5,000 family photos on your hard drive?
“People have so much digital data that we’ll be increasingly telling stories of our archives,” Fisher says. “You have to imagine how ideas go together to make visual patterns from vast quantities of information. Can you make it into a sculpture you can walk through? How much faster could you communicate information using a chart rather than showing people the raw data? How we condense information or show it in a picture actually matters.”
Fisher champions the power of knowledge visualization and relates a tale of a six-sided digital cave created by a student at Duke University in Durham, North Carolina, around patent development and directorships in southern California. Rather than reading about it in a book, you would walk right into the argument and the webs of influence that determine its outcome. There would be a name and a location where that person worked and if you followed the lines you would see where the money came from. “You got a sense of how big it was and who was connected to whom,” Fisher says.
Her own doctoral thesis at York, completed in 2000, was a structure of 40,000 links—a thought sculpture that was a result of 10 years of focus on hypertext and how varying ideas went together. It speaks to the changing nature of academia in departments with a technological edge, where students need to be both theorists and practical experimenters. These days, many doctoral projects have no print component. “People get caught in the idea that the dissertation is not mutable,” Fisher says.
And why shouldn’t it be? Changing forms of literacy are occurring all the time with rising technical enhancements. There have been countless articles breathlessly carrying on about amazing technologies that will change the way we read books, and enough cranks out there to remind us that a book is a form that does not need improving upon.
“Everybody has ideas whether people’s literacies are changing for better or for worse,” Fisher says. “I think it’s fair to say that they’re changing—and people’s literacies change by having challenging works that call them to change.” She says new technologies create new ways of reading, noting that with tabletop theatre AR set-ups, stories are told in no particular order. People create the narrative as they go in a choose-your-own-adventure style of interactivity.
It creates new ways of knowing according to Anne Balsamo at the Annenberg School for Communication and Journalism at the University of Southern California, Los Angeles. She argues that tinkering is a gateway skill for developing the technological imagination.
For example, back in 2003, Duke University issued a challenge: If a student could dream up an educational use for the iPod, the university would give the student—and everyone in the related class—one of the devices.
Nearly every faculty benefited from the experiment. Some listened to downloads of T.S. Eliot or organic chemistry lectures. A group of biomedical engineering students adjusted the iPod’s signal processing so that they could put a bud in one ear and a stethoscope in the other and diagnose heart conditions by assessing the patient being examined, in real time, against samples in the audio library of cardiac arrhythmias from the National Institutes of Health.
Of the experiment, author Cathy N. Davidson said, in The Times Higher Education, “It asked the professoriate to admit that students knew something key about novel interactive technologies and their potential for enhanced learning.”
Fisher sees an interesting intersection between hands-on learning and big-picture thinking.
“Older divisions around thinkers versus makers, and ways to knowledge, are breaking down all over. It’s partly because of new tools and partly because of new sensibilities.” Working in augmented reality serves both needs—it’s spatial and tactile so people theorize through making.
“We’re at one of those moments in technology where it’s old and new,” Fisher says. “It’s not an accident that my student’s piece on Méliès is about early cinema, because, as you try your ways, you take inspiration from the people who’ve had the capacity and the luxury to be able to experiment.”