EN

Sound 3 Image and Movement | INTERCUL

Training Manual for Trainers: Teaching Computer Vision for Audio Generation

Objective Equip trainers with a step-by-step guide to effectively teach converting visual information into audio through computer vision analysis, enabling participants to create responsive sonic environments driven by imagery and movement.

Structure of the Training Session

  1. Preparation (approx. 10 minutes)
  • Objective: Set up computer vision and audio synthesis

•     Steps:

  1. Research computer vision tools (OpenCV, MediaPipe, or commercial solutions).
  2. Prepare visual content with varied movement and color
  3. Set up audio synthesis software and MIDI/OSC communication protocols.Checklist:
  • Camera or video input device
  • Computer vision software with tracking capabilities
  • Audio synthesis tools (Ableton, SuperCollider, Pure Data)
  • Visual analysis to MIDI/OSC conversion

2.    Introduction to Vision-to-Sound Conversion (approx. 15 minutes)

  • Objective: Demonstrate how visual elements can drive musical

•     Steps:

  1. Explain computer vision basics: object detection, motion tracking, color
  2. Show examples of installations where visuals control sound
  3. Demonstrate different approaches: pixel-to-pitch, motion-to-rhythm, color-to-timbre. Trainer Tip: Use contrasting visual examples to clearly show different sonic responses.

3.   Hands-on Practice (approx. 30 minutes)

  • Objective: Create a basic image-to-sound conversion

•     Steps:

  1. Set up basic motion detection or color
  2. Map visual parameters to simple audio synthesis
  3. Experiment with different visual input sources and musical Trainer Tip: Start with obvious mappings (brightness to volume) before exploring abstract relationships.

4.   Advanced Features and Creative Use Cases (approx. 15 minutes)

  • Objective: Explore complex vision analysis and musical

•     Steps:

  1. Demonstrate advanced tracking: pose estimation, facial expression
  2. Show multi-layered visual analysis creating complex musical
  3. Explore real-time video processing for live performance

5.   Wrap-Up and Feedback (approx. 10 minutes)

  • Objective: Consolidate vision-to-sound

•     Steps:

  1. Review computer vision techniques and audio mapping
  2. Share resources for further exploration of creative coding and visual
  3. Encourage experimentation with unconventional visual

Post-Training Follow-Up

  • Provide access to recordings, cheat sheets, or tool
  • Schedule optional Q&A sessions or office

    Trainer Tip:     Encourage a collaborative group where participants can share projects and solutions.