C/AMIGObot: Creating virtual soundscapes with robotic senses

C/AMIGObot: Creating virtual soundscapes with robotic senses first appeared in Ryerson’s INNOVATION NEWSLETTER

Jason with robot

Jason Nolan has been immersed in virtual reality (VR) in its various permutations for decades, long before its most recent incarnations of high-definition headsets and experiences.

“VR and augmented reality (AR) are most often associated with immersive visual environments, but AR/VR environments run the gamut from text-based simulations that have been around since the early 1980s to vibro-tactile hardware and the present trend in the form of VR glasses,” said Nolan, a professor with Ryerson’s School of Early Childhood Studies. “My present AR/VR project focuses on an interactive environment-sensing robot that we are calling ‘C/AMIGObot: A Creative Autonomous Mobile Interactive Generative-music Object roBot,’ which generates sound based on data from over 20 sensors.” These sensors can detect environmental information such as proximity to objects and people, ambient noise, environmental factors, and light intensity.

Nolan’s cross-disciplinary team is finishing the second prototype of the C/AMIGObot and hopes to begin field testing in the new year to assess how this method of “sonifying” spaces might influence our perception and understanding of the physical spaces around us. He is the director of the Responsive Ecologies Lab and the Experiential Design and Gaming Environments Lab, where the project is housed.

C/AMIGObot’s virtuality is perceived through auditory stimulation of space and participants, rather than through sight. C/AMIGObot takes the data that its sensors collect and uses it to generate ambient sound that in turn represents spaces virtually. “All of this information is processed into data that can then be assigned to various elements of music synthesis such as various generators and oscillators, and circuits,” says Nolan. “This would enable the general public or musicians to create music with the data generated by physical spaces, micro-environmental conditions, and how the individuals move in and about the space.”

Potential uses for the C/AMIGObot run the gamut from helping children to understand their learning environments, to modifying the perception of institutional spaces, to giving musicians tools to rethink how musical compositions represent and interact in mixed-reality (AR/VR) spaces.

Nolan is autistic, and this project is centred on Nolan’s curiosity about how young children explore and physically engage with sensory information as the foundation for their learning. The project is heavily influenced by the British musician and producer Brian Eno, and his ideas and work in generative and ambient music. Nolan believes that moving beyond an “ocular-centric perspective” offers new research, design and learning opportunities.

“Though I primarily see C/AMIGObot as a learning tool to encourage people to re-think how we perceive spaces, I look forward to supporting new ways of interacting with and through the spaces in which we live,” said Nolan.

Phantom Compass Collaboration with RE/Lab


phantomcompass logo

Award-winning video game company partners with Ryerson University to develop advanced Virtual Reality experiences

Innovative game developers partner with Ryerson University’s RE/Lab to explore the application of VR in special needs education, health care, and more.

TORONTO, September 26, 2016 — Phantom Compass, the award-winning game developer behind fantasy pinball mashup Rollers of the Realm (available on Steam and PlayStation Network) and upcoming ‘80s car combat tribute Auto-Age: Standoff, today announced breakthroughs in understanding of VR game design through research collaboration with Ryerson University’s Responsive Ecologies Lab (RE/Lab).

Read more

nice joystick hack

JW in @ryersonu @ReLabRye using our 3D printed joystick jig & handmade cable to use @proloquo2go via @get_tecla

Jig and joystick hack
Jig and joystick hack

originally tweeted.

Here are some close ups of the joystick and the 3d printed jig to turn the 8 way switch into a 4 way switch so it could be used with the @proloquo2go & @get_tecla

4 way joystick hack with jig
side view
4 way joystick hack with jig
4 way joystick hack with jig
8way joystick viewed from bottom
8way joystick viewed from bottom

Is tech overpowering the classroom?

I’m quoted in the @OttawaCitizen re: Is tech overpowering the classroom?. And of course it is.

I was asked a series of questions:

  1. What is the value of getting young kids using technology, such as tablets, at a young age?
  2. Is there any evidence to suggest they might be better off reading, or exercising?
  3. Is there an age that you would consider “too young” to be using tablets and computers?
  4. If kids are going to be using tablets, what sort of activities are good for them to do

Here’s my response that didn’t make it in:

I’m very skeptical of the need for children to be using digital technology at a young age. Children should be engaged in physical, active, multisensory exploration of the world around them, in a manner that is open ended and full of unknowable outcomes. There is no digital technology that can provide children with as rich a learning experience as their own interaction with the world around them. Of course, there are instances where digital technology can provide children with opportunities that they can’t engage in on their own, but these are specific and rather limited. Cell phones and iPads can be used to allow children to document their explorations and creations. Taking a picture of a sand castle or snow angel, or recording a special event is a wonderful way to document children’s experiences in a way that they can explore and reflect on a later date. Sometimes children are so engaged that they do not end up with a clear memory of what was going on. Digital technology can help with that. And even before an experience or an event, digital technology, in the form of videos, can help a child understand an upcoming experience and gain a form of mental preparedness of what is going to happen. Whether this is a doctor’s visit, or going camping, or going to a new school, being mentally prepared for new event can help a child to avoid being overwhelmed, and also be able to make the most of experience. Another very important use of digital technology comes in the form of its ability to help disabled children engage more fully in the world around them; a world that is rarely designed with their needs in mind. Any form of technology, digital or mechanical, this can help extend a disabled child’s ability to move, explore, communicate, or share their experiences helps to ameliorate, even in a small way, the challenges they face in everyday life. And the better these tools are in helping children engage in communicate, the greater the chance that the able-bodied community will see them as valued members of our community. Finally, digital technology is a great way to maintain and nurture family bonds when the child is growing up far away from grandparents and extended family members. It can help in the maintenance of first languages and cultures that could quickly become dim memories if there wasn’t videos and web chats to help maintain the ties. The use of digital technologies by young children should never be about the age of the child. Every child’s developmental path and individual needs and curiosities are different. Parents and teachers must take responsibility in carefully observing each child in order to decide to the best of their ability whether digital technology is increasing a child’s opportunities for exploration and learning about the world around them, or distraction that merely keeps quiet and occupied.

Public art by Kenneth Emig installed on Adàwe crossing

ken's balls
Kenneth’s installation
RE/Lab artist in residence Kenneth Emig has just had his latest work installed on Adàwe crossing. Here’s the press release from the City of Ottawa:

Public art by Kenneth Emig installed on Adàwe crossing
Public art was installed this past weekend on Adàwe crossing, the new pedestrian and cycling bridge spanning the Rideau River between Donald Street and Somerset Street East. The artwork entitled A View from Two Sides created by local artist Kenneth Emig is an integral part of the bridge design.

The artwork features two reflective stainless steel spheres, each 1.5 metres in diameter suspended at eye level on the extended observation areas of the bridge. Each sphere presents the observer with an ever-changing panoramic view that includes the sky, river, shores, bridge, pedestrians and cyclists.

The result is an unlimited variation of visual possibilities, providing residents and visitors with the perfect place to pause and take in the natural beauty of the area.

Kenneth Emig’s art practice intersects form, light, sound, movement and technology. He aims to provoke awareness of our own perceptions and understandings of our world. He is currently Artist in Residence at the Responsive Ecologies Laboratory at Ryerson University. In 2004, Emig was awarded a City of Ottawa public art commission for his light-based installation entitled The Second Story, on permanent display at the Eva James Memorial Community Centre.

This artwork and other commissions were made possible by the City of Ottawa’s Public Art Policy, which allows for one percent of funds from municipal projects to be set aside for public art to enhance the space and make art accessible to everyone.

Adàwe crossing was officially opened on December 4, 2015 and was named for the Algonquin term meaning “to trade”. The name is also symbolic of the history of the river, the indigenous heritage of the area and the bridge’s ability to link the communities on either side.

For more information:
Public art for Rideau River Crossing

Installation d’une œuvre d’art public de Kenneth Emig sur la passerelle Adàwe
Une œuvre d’art public a été installée en fin de semaine dernière sur la passerelle Adàwe, le nouveau pont pour piétons et cyclistes qui enjambe la rivière Rideau entre la rue Donald et la rue Somerset Est. Cette œuvre, intitulée Une vue de deux côtés, créée par Kenneth Emig, artiste local, fait partie intégrante de la conception du pont.

L’œuvre se compose de deux sphères réfléchissantes en acier inoxydable, dont chacune fait 1,5 mètre de diamètre, suspendues au niveau des yeux sur les aires d’observation du pont. Chaque sphère présente à l’observateur une vue panoramique en perpétuel changement qui embrasse le ciel, la rivière, les berges, le pont, les piétons et les cyclistes.

Il en résulte une gamme illimitée de possibilités visuelles, qui offre aux résidents et aux visiteurs un endroit parfait pour faire une pause et contempler la beauté naturelle du lieu.

L’art de Kenneth Emig combine forme, lumière, son, mouvement et technologie. Cet artiste vise à nous faire prendre conscience de nos propres perceptions et de la manière dont nous comprenons notre univers. Il est actuellement artiste en résidence au Responsive Ecologies Laboratory de l’Université Ryerson. En 2004, Emig a créé une œuvre d’art public commandée par la Ville d’Ottawa, une installation lumineuse intitulée The Second Story, maintenant exposée en permanence au Centre commémoratif Eva-James.

Une vue de deux côtés ainsi que d’autres œuvres d’art commandées ont été réalisées grâce à la Politique d’art public de la Ville d’Ottawa. Cette politique prévoit l’affectation de un pour cent des fonds liés aux projets municipaux à des œuvres d’art public, afin d’embellir l’espace public et de mettre l’art à la portée de tous.

La passerelle Adàwe a été officiellement inaugurée le 4 décembre 2015. Son nom vient du terme algonquin qui signifie « échanger ». Ce nom symbolise aussi l’histoire de la rivière, l’héritage autochtone de cette région et la capacité du pont à relier les milieux de vie situés de chaque côté.

Renseignements :
Œuvre d’art public pour la passerelle sur la rivière Rideau

Making sense, sensing space: an inter-sensory walk & workshop

Trinity ChurchTrinity Square Park

On Monday November 9th, Responsive Ecologies Lab (RE/Lab) RA Melanie McBride will facilitate an inter-sensory talk and walk focused on the urban ambiances and ecologies in and around the RE/Lab with special guest Dr Milena Droumeva of Simon Fraser University. Drawing on applied and critical perspectives from sensory studies, acoustic ecology, critical pedagogy and smell walking/mapping, this workshop introduces practices of ‘sensory inquiry’ and ‘DIY sense making’  as a means of relating with, representing and constructing inter-sensory ambiances. Through a short inter-sensory walk involving listening, smelling, touching and (maybe) tasting different ambiances of place and (urban) space, participants will gain an understanding of more embodied, critical, inclusive and socioculturally situated approaches to studying the senses. Participants will be introduced to transdisciplinary techniques of engaging with and representing sensory, spatial, material and sociocultural ambiances that exceed traditional ethnographic methods, through an emphasis on non-visual stimuli such as smell, sound, touch, temperature, movement and bodily state.

Workshop facilitators

Dr. Milena MilenaDroumeva is the Glenfraser Professor in Sound Studies at Simon Fraser University. An experienced sound researcher, multimodal ethnographer and soundwalking enthusiast, Milena has been recording and listening for over 14 years in Vancouver and Europe as part of her urban soundscapes and mobile media cultures research. You can find her musings on sound and other material goodies at http://natuaural.com follow @ambientfluff



MelVentMelanie McBride
is a doctoral candidate in Communications and Culture at York University where she is researching inter-sensory learning and communications practices involving smell (and, increasingly, taste). Melanie’s developing practices of ‘sensory inquiry‘ are informed by her sensory ethnographic field work in Canada and France and her background as an inner city educator. Melanie is also a researcher with Ryerson’s RE/Lab, where she has created a DIY smell lab.  Find her online at:
https://www.flickr.com/photos/multisensorymel/ https://soundcloud.com/multisensorymel

Co-Facilitator: Daniel Harley
Daniel Harley is a Research Assistant at the Synaesthetic Media Lab (SynLab) at Ryerson University, studying interactive narratives for tangible and embodied technologies. He also has a background in music, with over ten years of experience working as a violinist in the Greater Toronto Area.