Lecturers

• Matthias Kappeler
• Marcial Koch

Students / Groups

• group 1: Carina, Elena, Lea https://www.notion.so/9444aad910eb4a3ba8d5c7a7b9fcaedf?v=0a540ab9e5b943bea09ba8224c9429b4 
• group 2: Matilda, Sonja, Loïc, Mo https://modob.notion.site/43b4fa42044643c3abd0638d72c45172?v=7e25177a63484efc8547b9e7d7ab2e9e
• group 3: Beni, Lars, Luis  https://conscious-starflower-0cb.notion.site/bf35a1bd756b4343a6770d795d4dc62f?v=a66ba16a2b2c48439a74fa30e94cd86c
• group 4: PJ, Tanja, Luke https://miro.com/app/board/uXjVPzphJdU=/?share_link_id=798777318418
• group 5: Audrey, Lyv, Matthias https://www.tumblr.com/sonic-interaction-2023

Tools

• Teleconferencing Zoom
• DAW Ableton Live 11, Download 90 Days Trial Version
• DAW Reaper, free to use

Attendance

Grading

Grades will be based on the following criteria:

• 40% Use of sounds as a means of perceptual design / interaction design and the corresponding development through user studies / field trials.
• 40% Quality of the mockup demos and the sounds used in them
• 20% Designing (including content) of the video and the documentation including presentation of your findings from the design research process.
  

Also see: Grading Scale and Evaluation Criteria

Final delivery of the module

All deliverables have to be uploaded to the filer until Monday 23.01.2023 at 23:59 PM.

• Written Documentation. It will be done in your blogs. Keep your blog up to date with your daily process (including sound snippets, short videos and photos). We also base our mentoring on the blogs. Export your documentation as PDF and upload it to the filer (to be stored in a file labelled “Documentation”)
• A short video (~60sec) of the project (to be stored in a file labelled “Video”). The video should be created with your mockup prototypes, as easily staged as possible. Here "faking" (using sound design to replace not properly working functions) is explicitly allowed. Make sure that your concept can be understood and the visuals and audio are of good quality. 
• Text file including the project title, short description, project description and contributors (in a file to be labeled “Texts”)
• Approx. 10 representative images of the project (to be stored in a file labeled “Images”)
• Mockup project files (to be stored in a file labeled “Prototype”)
• The used sound assets
• Ableton Live project (all sounds embedded, standalone file)
• Any scripts and technical drawings

Please take a look at the IAD documentation guidelines for submission.

Expenses

All expenses must be approved in advance by the lecturers.
Please be aware that your expenses can only be refunded if you submit the complete documents no later than 4 weeks after the end of the module.

Feedback

There will be an overall feedback after the final presentation.

Sound is an integral part of any film. Some even say that the quality of a moving picture is 50% sound. Iconic sound design as seen in Jurassic Park or Star Wars have created their own worlds and defined a genre. Sound creates reality, makes things believable but also influence the emotion of a scene enormously. That's what we want to focus on in this exercise.

You are free to decide if you want to work on your own or in teams of two. 

What to do

• Pick one of the film scenes that we have provided for you (links down below)
• Define for yourself what the content of the scene is.
• Think of how you want to set the mood of the scene through sound. This can be any of the following or something else:
  • happy / uplifting
  • romantic
  • danger / suspense
  • sad / dark
  • funny / comedic
  • …
• What kind of "foley onscreen-sounds" do you need? 
• What is happening off-screen?
• Plan what you need and put your sounds on a graph like the example below. Do this with both moods.
• Take your scene into any DAW (digital audio workstation) or NLE (nonlinear editing software) and create a soundtracks from scratch.

original sound (or adr) and voice over text

The voice over text for film 2 and the transcript of film one scene 1 can be part of your project but don't have to be. 

You can use any sound you can find on the internet. If you find sounds that are not in the public domain or free of licence you can use them for the screening in class but have to be really careful when putting it on the web. Self recorded sounds are much appreciated.

ZHdK SFX filer: smb://filerav.ad.zhdk.ch/SFX

With ZHdK VPN you have access to the "Prosoundeffect" Soundlibrary

Spitfire Audio Free VST

The goal is the systematic, creative exploration of sound and its importance for the design of interactions and affordances.

This is done by working on defined creative hypotheses in a small "design research" process.

• The design drafts and experimental prototypes are used for gaining insights into tonal potentials of meaning and “Affordances” through the use of “Wizard-of-Oz” prototypes.
• In the second week you will keep experimenting with user studies to further develop your sonic concepts and at the same time start building a functional prototype with sensors and sound feedback.
• A special focus lies on design methods that understand sound as a means and material of interaction design.
• The continuous valuation of design hypotheses with small “User Studies” und Feldversuchen. Because there are barely any rules about Sonic Interaction Design, but a lot of space for experiments and design research that we want to explore.

in short:

• To understand the influence of sound on interaction with an object.
• To understand the influence of sound on perception of virtual and physical rooms.
• To discuss and deepen the meaning of concrete design variants in this regard.

Process

• Develop a purpose for your installation          
  • What functionalities/services are possible? (what is its function?)
  • What kind of processes are going on inside? 
  • What's the intention?
• How should the interaction feel? Is there an emotional arc?
• Create a sonic moodboard
• Develop multiple scenarios of how the can be approached, touched, moved and how the room/system reacts to it. 
  • Proximity
  • Elevation
  • Loudness
  • Brightness
  • Acceleration
  • Orientation
  • ...
• Build a Wizzard-of-Oz setup that allows you to test your ideas
• Validate your concept trough user testing 
  • invite a number of people (one after the other)
  • play the interaction sounds in sync with the test-users actions
  • record the user test (video + audio)
  • ask the users what worked for them and what didn't. How did they feel? (document)
  • analyse the recordings 
  • define lessions learned (what worked, what didn't)
  • improve your concept and your sounds
  • do this iteration as often as possible!
• Plan your functional prototype
  • what kind of sensor-data do you need?
  • what kind of logic do you need to program?
• Start building your functional prototype and keep evaluating your concept and sounds through user tests while you do it! 
• Do the same usertests: review - improve - iterations with your functional prototype

• Create an object that allows to explore the sonic relation between object and room
• The object can have a screen but doesn't have to. The touch interface mustn't be the primary way to interact. 
• Orientation and acceleration of the object need to have a meaning to the room. The room reacts when these parameters change. 
• The room is equipt with a 4 channel setup, but you can also work in stereo.
• Create a immersive sound experience. 

Kick-Off

• Introduction into the module "Sonic Interaction Desgin"
  
• Warmup: Sonic Charade
  

Introduction Exercise 1: Evoking Emotion Through Sound in Film

Presentation Exercise 1: Evoking Emotion Through Sound in Film

The order is determined in class. The group, which are presenting a new movie for the first time, shows the one with the original sound first.

Kick-Off Project Sonification

• Introduction into the project "Sonification"

For intermediate presentation 1: Concept:

• Develop a basic idea
• Create sonic moodboards
• Create first mockups to test the ideas

Short Mentoring

Have a look on the google calendar for your timeslot. If someone is unable to attend a meeting, please swap the timeslots among the class so that there are no gaps.

We expect first thoughts regarding a concept. Bring different ideas out of a brainstorming.

Workshop Fast Sonic Prototype via OSC

Morning: Self preparatory

• Download and install trial version Ableton Live Suite 11 (90 Days)
• Install the plugin „Connection Kit“ for Ableton
• Look at the Ressources in Paul on the bottom -> especially go over the Tutorials and Synthesizer & Sound terms
• If you have an Iphone, install GyrOSC (cost 1 CHF on the Appstore, but past students used it also after the module for fast prototyping) and Syntien. If you have an Android or another phone type, I would recommend this alternatives: OSC Controller (worked good last year), Multisense OSC (caused problems, but they may have fixed it)


• Download the folder OSC-Processing-Ableton Live
• Follow the Tutorial OSC-Processing-Ableton Live

Afternoon: Questions and Exploration

• Smartphone Sensor Data via OSC (GyrOSC, Syntien)
• OSC Mapping & Routing via Processing
• OSC In & Controller Mapping in Ableton

• Sound Expression Exercise in Groups: Use at least 2 sensor data from your phone as input and create a short performance

Workshop "Ambisonic" in Ableton Live

Morning: Self preperatory

• Install the plugin „Envelop“ for Ableton


• Download the folder Ableton Live-4Channel Example
• Follow the Tutorial Ableton Live-4Channel

Afternoon: Questions and Exploration

• 4-Channel Routing in Ableton

Intermediate Presentation 1 and peer review: Concept

Expectation:
We expect a complete concept with first experiments and conclusion on their findings.

Questions that might appear:

• What are the goals of the project?
• Are those realisitc in this timeframe?

Mentorings

Look at the calendar for your timeslot. If someone is unable to attend a meeting, please swap the timeslots among the class so that there are no gaps.

Intermediate Presentation 2 and peer review: Prototype

The order is determined in class.

• Current state of your experiment
• Process so far
• Results from user testings
• Further steps

Feedback in class

Mentorings

Look at the calendar for your timeslot. If someone is unable to attend a meeting, please swap the timeslots among the class so that there are no gaps.

Final Presentation

Look at the calendar for your timeslot.
The final presentation should be no longer than 15 minutes and is structured as follows:

• 5 minutes Demo: Demonstrate your final version of the concept / object to the group. The functionally implemented elements can be combined with Wizard-Of-Oz Mockup elements if necessary.
• 10 minutes Presentation: Show us the development of your concept. What did you learn? What went well? How would you proceed?

If you have a good and/or creative idea to use your 15 minutes in a different way that suits your project better - feel free to do so!

General Sonic Interaction Ressources

Prototyping

Useful Protocols for Prototyping

• Open Sound Control (OSC)
  
• Musical Instrument Digital Interface (MIDI)

Prototyping Tools

• TouchOSC - OSC Interfaces for Tablets and Smartphones (OSC & MIDI)
• GyrOSC - Transmits Smartphone Sensor Data over OSC
• Holon.ist - Create Musical Mappings from physical events, locations and ioT data (OSC & MIDI)
• OSCulator - Routing Application für OSX (OSC)

Prototyping Methods

• Wizard of OZ
• Ivestigative Tools

Synthesizer and Sound terms

• Learning Synth by Ableton
  
• Learning Synth Playground by Ableton

Ableton Live 11

General Tutorials, More be found here.

Plugins & Packs

• Connection Kit - send or receive OSC/MIDI data
• Ableton specific Plugin Database
• KVR Audio Plugin Database