Category Archives: Core Faculty

C/AMIGObot: Creating virtual soundscapes with robotic senses

C/AMIGObot: Creating virtual soundscapes with robotic senses first appeared in Ryerson’s INNOVATION NEWSLETTER

Jason with robot

Jason Nolan has been immersed in virtual reality (VR) in its various permutations for decades, long before its most recent incarnations of high-definition headsets and experiences.

“VR and augmented reality (AR) are most often associated with immersive visual environments, but AR/VR environments run the gamut from text-based simulations that have been around since the early 1980s to vibro-tactile hardware and the present trend in the form of VR glasses,” said Nolan, a professor with Ryerson’s School of Early Childhood Studies. “My present AR/VR project focuses on an interactive environment-sensing robot that we are calling ‘C/AMIGObot: A Creative Autonomous Mobile Interactive Generative-music Object roBot,’ which generates sound based on data from over 20 sensors.” These sensors can detect environmental information such as proximity to objects and people, ambient noise, environmental factors, and light intensity.

Nolan’s cross-disciplinary team is finishing the second prototype of the C/AMIGObot and hopes to begin field testing in the new year to assess how this method of “sonifying” spaces might influence our perception and understanding of the physical spaces around us. He is the director of the Responsive Ecologies Lab and the Experiential Design and Gaming Environments Lab, where the project is housed.

C/AMIGObot’s virtuality is perceived through auditory stimulation of space and participants, rather than through sight. C/AMIGObot takes the data that its sensors collect and uses it to generate ambient sound that in turn represents spaces virtually. “All of this information is processed into data that can then be assigned to various elements of music synthesis such as various generators and oscillators, and circuits,” says Nolan. “This would enable the general public or musicians to create music with the data generated by physical spaces, micro-environmental conditions, and how the individuals move in and about the space.”

Potential uses for the C/AMIGObot run the gamut from helping children to understand their learning environments, to modifying the perception of institutional spaces, to giving musicians tools to rethink how musical compositions represent and interact in mixed-reality (AR/VR) spaces.

Nolan is autistic, and this project is centred on Nolan’s curiosity about how young children explore and physically engage with sensory information as the foundation for their learning. The project is heavily influenced by the British musician and producer Brian Eno, and his ideas and work in generative and ambient music. Nolan believes that moving beyond an “ocular-centric perspective” offers new research, design and learning opportunities.

“Though I primarily see C/AMIGObot as a learning tool to encourage people to re-think how we perceive spaces, I look forward to supporting new ways of interacting with and through the spaces in which we live,” said Nolan.

Is tech overpowering the classroom?

I’m quoted in the @OttawaCitizen re: Is tech overpowering the classroom?. And of course it is.

I was asked a series of questions:

  1. What is the value of getting young kids using technology, such as tablets, at a young age?
  2. Is there any evidence to suggest they might be better off reading, or exercising?
  3. Is there an age that you would consider “too young” to be using tablets and computers?
  4. If kids are going to be using tablets, what sort of activities are good for them to do

Here’s my response that didn’t make it in:

I’m very skeptical of the need for children to be using digital technology at a young age. Children should be engaged in physical, active, multisensory exploration of the world around them, in a manner that is open ended and full of unknowable outcomes. There is no digital technology that can provide children with as rich a learning experience as their own interaction with the world around them. Of course, there are instances where digital technology can provide children with opportunities that they can’t engage in on their own, but these are specific and rather limited. Cell phones and iPads can be used to allow children to document their explorations and creations. Taking a picture of a sand castle or snow angel, or recording a special event is a wonderful way to document children’s experiences in a way that they can explore and reflect on a later date. Sometimes children are so engaged that they do not end up with a clear memory of what was going on. Digital technology can help with that. And even before an experience or an event, digital technology, in the form of videos, can help a child understand an upcoming experience and gain a form of mental preparedness of what is going to happen. Whether this is a doctor’s visit, or going camping, or going to a new school, being mentally prepared for new event can help a child to avoid being overwhelmed, and also be able to make the most of experience. Another very important use of digital technology comes in the form of its ability to help disabled children engage more fully in the world around them; a world that is rarely designed with their needs in mind. Any form of technology, digital or mechanical, this can help extend a disabled child’s ability to move, explore, communicate, or share their experiences helps to ameliorate, even in a small way, the challenges they face in everyday life. And the better these tools are in helping children engage in communicate, the greater the chance that the able-bodied community will see them as valued members of our community. Finally, digital technology is a great way to maintain and nurture family bonds when the child is growing up far away from grandparents and extended family members. It can help in the maintenance of first languages and cultures that could quickly become dim memories if there wasn’t videos and web chats to help maintain the ties. The use of digital technologies by young children should never be about the age of the child. Every child’s developmental path and individual needs and curiosities are different. Parents and teachers must take responsibility in carefully observing each child in order to decide to the best of their ability whether digital technology is increasing a child’s opportunities for exploration and learning about the world around them, or distraction that merely keeps quiet and occupied.

testing out the stomp interface

Farheen’s testing out the stomp interface of the @ReLabRye AAC device for Ontario Accessibility Innovation Showcase #OntarioAIS. I posted a picture of kamran grinding down some plexiglass for one of the devices in a previous post.

Farheen testing the stomp interface...
Farheen testing the stomp interface…

The AAC (Augmented and Alternative Communication) is something that our undergraduate research assistants have been working on all summer. They started with an open source ACC unit developed by Jim Wroten‘s Morse2Go.org as a replication project (to confirm the best hardware for the implementation) and a platform to develop new input switches, and design a new 3D printed case to integrate the speaker in a single unit. Though the hardware is the best for the code they developed, we decided to search around for what we thought might be a better input method. We settled upon the H4 (4 button huffman code) method developed by Scott Mackenzie and his grad students at York University.

Presently, we’re working to finalize input methods, such as this unique stop interface, to help people realize that input methods can be anything, and that only through observation and careful communication with an individual can needs of that person be understood, and the best possible input device developed with them. I doubt anyone would need a stomp interface, but it does get you thinking!

We have not developed a standalone H4 version based on Jim’s hardware yet, but we should have it done by the end of the summer. The next step will be to port the code over to new open-source hardware with the goal of simplifying the design and cost while increasing the number of languages that this can be used with to any language that can be represented using roman characters.

innovation.ca talking about our lab director :)

RE/Lab Director with some of his cardboard creations
RE/Lab Director with some of his cardboard creations

Canadian Foundation for Innovation ran a story last week about work started in the EDGE Lab, and continuing in the RE/Lab: More than just a cardboard box: Using this cheap and lightweight material, researchers are helping parents design custom adaptations for their children with special needs. The foundation has funded both labs and has been a great supporter of our work.

Check out the great video they did as well.

The two designs in the images will be presented at the Para PanAm Games Accessibility Showcase.