Generate Sound Sculpture 03

Generate Sound Sculpture 03

Work created in 2025/5/31

The core of this work lies in capturing the viewer’s own image in real time and utilizing it as material for both three-dimensional modeling and sound generation. The camera system, activated by clicking the startButton, is an innovative mechanism that incorporates the very act of viewing the artwork into the creative process itself.
The viewer is not merely a passive observer but is positioned as a co-creator of the work. The video data captured by the camera is processed by Three.js and applied in real time as a texture to a 3D form. As a result, the three-dimensional sculpture displayed on the screen becomes a unique entity that reflects the viewer’s appearance and environment.
By mapping the video as a texture onto the three-dimensional object, the work acquires a strongly self-referential character. The viewer witnesses the process in which their own image is distorted, transformed, and abstracted. This experience symbolically expresses the instability and fluidity of self-representation in the digital age.
In contrast to the fixity of traditional portraits and self-portraits, the “portrait” in this work is in constant flux. Even small movements by the viewer in front of the camera ripple across the surface of the 3D object and influence the sound as well. This interaction suggests the plasticity of identity in contemporary society.
The music generated based on the video data functions not as mere background music but as an acoustic translation of the viewer’s physical presence. The sound elements processed by Tone.js are linked to parameters such as the brightness, saturation, and motion of the video, converting the viewer’s visual presence into an auditory experience.
Through this conversion process, corporeality—often lost in digital environments—is recovered through the alternative sensory channel of sound. By “hearing” their own presence as sound, the viewer gains a sense of reality within the digital space.
As an interactive work utilizing a camera, this piece also raises important questions about the concept of privacy. While the viewer’s image is used to generate the artwork, issues regarding how that data is processed, stored, or deleted are closely related to the handling of personal information in today’s digital society.
Within the context of art, the act of providing one’s image carries a different meaning from the routine collection of images by social media or surveillance cameras. Here, the viewer consciously agrees to have their image “transformed into art.” This voluntary participation can be understood as a form of active self-expression in the digital age.
While built on a technical foundation of real-time video processing with WebRTC, 3D rendering with Three.js, and audio synthesis with Tone.js, the work succeeds in offering a poetic experience without foregrounding its technical complexity. This balance demonstrates an ideal relationship between technology and art.
For the viewer, the technical processes are rendered transparent, and only the resulting visual and auditory experiences are perceived. This “transparent technology” approach indicates the maturity required for media art to provide a pure artistic experience.
The real-time generated sound and visuals possess a uniqueness that cannot be reproduced by recording or playback. Even if the same viewer engages with the work again, subtle differences in lighting, clothing, facial expression, and posture will create an entirely different experience.
This uniqueness shares the aesthetics of live performance and improvisational music. By intentionally creating an unrepeatable experience in an era where digital technology enables easy duplication, the work reaffirms the value of “here and now” in art.
This piece presents a new way of art-making that leverages personal devices and network infrastructure as creative environments. Transcending the physical constraints of museums and galleries, the viewer’s personal space is instantly transformed into an art studio. This model offers important insights for the future of digital art distribution and experience design.