abstract

Draw Portraits by Music: A Music based Image Style Transformation

Authors:

Siyu Jin,

Jingyan Qin,

Wenfa LiAuthors Info & Claims

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Pages 4399 - 4400

https://doi.org/10.1145/3394171.3416340

Published: 12 October 2020 Publication History

Get Access

Abstract

"Draw portraits by music", an interactive work of art. Compared with music visualization and image style conversion, it's AI's imitation of human synaesthetic. New portraits gradually appear on the screen and are synchronized with music in real-time. Users select music and images as the main interactive contents, the parameters of the music are used as the dynamic expression of human emotions, and the new pixel generation process of the image is regarded as the result of emotions affecting humans.

Supplementary Material

MP4 File (3394171.3416340.mp4)

This interactive work of art stimulates the new direction of visual and auditory creation is a cooperation of computer science and art.

Download
26.93 MB

References

[1]

Lunke, K., & Meier, B. (2019). Creativity and involvement in art in different types of synaesthesia. British journal of psychology, 110(4), 727--744. https://doi.org/10.1111/bjop.12363

Crossref

Google Scholar

[2]

Li, T. & Ogihara, M. (2004, October). Music artist style identification by semi- supervised learning from both lyrics and content. In Proceedings of the 12th annual ACM international conference on Multimedia(pp. 364--367). https://doi.org/10.1145/1027527.1027612

Digital Library

Google Scholar

[3]

Wu, X., Xu, B., Qiao, Y., & Tang, X. (2012, October). Automatic music video generation: cross matching of music and image. In Proceedings of the 20th ACM international conference on Multimedia (pp. 1381--1382). https://doi.org/10.1145/2393347.2396495

Crossref

Google Scholar

[4]

Wu, Y. Qiao, X. Wang, and X. Tang. Cross matching of music and image. In ACMMM, 2012. https://doi.org/10.1145/3123514.3123517

Digital Library

Google Scholar

[5]

Li, W. H. (2010, June). Musical box: draw it yourself. In Proceedings of the 9th International Conference on Interaction Design and Children (pp. 293--295). https://doi.org/10.1145/1810543.1810595

Crossref

Google Scholar

[6]

Subramaniam, A., & Barthet, M. (2017, August). Mood Visualiser: Augmented Music Visualisation Gauging Audience Arousal. In Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences (pp. 1--8).https://doi.org/10.1145/3123514.3123517

Crossref

Google Scholar

[7]

Voong, M., & Beale, R. (2007, April). Music organisation using colour synaesthesia. In CHI'07 extended abstracts on Human Factors in Computing Systems (pp. 1869--1874). https://doi.org/10.1145/1240866.1240913

Crossref

Google Scholar

[8]

Li, W. H. (2008). Music painter (Doctoral dissertation, Massachusetts Institute of Technology).

Google Scholar

[9]

DiPaola, S., & Arya, A. (2006, July). Emotional remapping of music to facial animation. In Proceedings of the 2006 ACM SIGGRAPH symposium on Videogames (pp. 143--149). https://doi.org/10.1145/1183316.1183337

Digital Library

Google Scholar

[10]

Miyazaki, R., Fujishiro, I., & Hiraga, R. (2004). comp-i: A System for Visual Exploration and Editing of MIDI Datasets.

Google Scholar

Cited By

View all

Yan GYao CZhang CWang JHu YYing F(2022)MusicCollage: A Music Composition Tool for Children Based on Synesthesia and a Genetic AlgorithmDistributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity10.1007/978-3-031-05431-0_15(205-216)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-05431-0_15

Index Terms

Draw Portraits by Music: A Music based Image Style Transformation
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models
  2. Visualization
    1. Visualization theory, concepts and paradigms

Recommendations

Music/lyrics composition system considering user's image and music genre
SMC'09: Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics

This paper proposes a music/lyrics composition system consisting of two sections, a lyric composing section and a music composing section, which considers user's image of a song and music genre. First of all, a user has an image of music/lyrics to ...
Affective interacting art
ICEC'10: Proceedings of the 9th international conference on Entertainment computing

This paper to study the potential of expressing ink-and-wash painting through interaction, and present a direction that can coincide with modern paintings by developing ink-and-wash painting from a traditional aspect through analyzing the theories and ...
Analysis about Application of Learning System into Interactive Digital Art Implementation
ISPAW '11: Proceedings of the 2011 IEEE Ninth International Symposium on Parallel and Distributed Processing with Applications Workshops

The rapid growth of science and technology influences to Digital Arts. Digital art is no more unfamiliar genre in art field. We can see many kinds of digital arts in our surroundings. High technology makes digital art more brilliant and various. ...

Comments

Information & Contributors

Information

Published In

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

October 2020

4889 pages

ISBN:9781450379885

DOI:10.1145/3394171

General Chairs:
Chang Wen Chen
Chinese University of Hong Kong, Shenzhen, China
,
Rita Cucchiara
UNIMORE, Italy
,
Xian-Sheng Hua
Alibaba Group, China
,
Program Chairs:
Guo-Jun Qi
Futurewei Technologies, USA
,
Elisa Ricci
UNITN & Fondazione Bruno Kessler, Italy
,
Zhengyou Zhang
Tencent, China
,
Roger Zimmermann
National University of Singapore, Singapore

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Check for updates

Author Tags

Qualifiers

Abstract

Conference

MM '20

Sponsor:

SIGMM

MM '20: The 28th ACM International Conference on Multimedia

October 12 - 16, 2020

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
187
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yan GYao CZhang CWang JHu YYing F(2022)MusicCollage: A Music Composition Tool for Children Based on Synesthesia and a Genetic AlgorithmDistributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity10.1007/978-3-031-05431-0_15(205-216)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-05431-0_15

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Music/lyrics composition system considering user's image and music genre

Affective interacting art

Analysis about Application of Learning System into Interactive Digital Art Implementation