WO2023107063A1 - System that plays music according to the emotion in the sentence read with the eye tracking system - Google Patents

System that plays music according to the emotion in the sentence read with the eye tracking system Download PDF

Info

Publication number
WO2023107063A1
WO2023107063A1 PCT/TR2022/051411 TR2022051411W WO2023107063A1 WO 2023107063 A1 WO2023107063 A1 WO 2023107063A1 TR 2022051411 W TR2022051411 W TR 2022051411W WO 2023107063 A1 WO2023107063 A1 WO 2023107063A1
Authority
WO
WIPO (PCT)
Prior art keywords
music
eye tracking
book
emotion
tracking system
Prior art date
Application number
PCT/TR2022/051411
Other languages
French (fr)
Inventor
Eyüp Burak Ceyhan
Mertkan SİNOPLU
Original Assignee
Bartin Üni̇versi̇tesi̇
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bartin Üni̇versi̇tesi̇ filed Critical Bartin Üni̇versi̇tesi̇
Publication of WO2023107063A1 publication Critical patent/WO2023107063A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Definitions

  • This invention relates to the eye tracking system that enables the user to listen to the music related to the emotions in the part of the book being read, and the system that plays music according to the emotion in the sentence read.
  • audio books As another study on audiobooks, Philips (2007) mentioned in his research that with the advent of portable cassette players, the first steps of audiobooks have been taken and people's interest in this type of book is increasing day by day. In addition, there is a subgenre of audio books called audio books, and he emphasized that these books are specially made for visually impaired people.
  • This invention is a system that plays music according to the emotion in the sentence read with the eye tracking system, and its feature is that it is a new technology.
  • the invention in order to realize all the objectives mentioned above and which will emerge from the detailed description below; With the help of eye tracking devices that are placed on the top of the physical book or the mobile or desktop platform where the book is read, or that can be worn like glasses, the reader listens to the predetermined music related to the feeling of the section being read. In addition, the user can choose the music for certain feelings before starting the book. In this way, it is thought that the pleasure of the user from reading books will be increased and the rate of reading books, which is very low today, can be increased.
  • Eye tracking systems can be examined in two categories as software and hardware.
  • eye tracking devices follow the physical movements of the eye with infrared rays (Akgiil and Giine ⁇ , 2019).
  • the necessary calibrations are made and the place where the eye is looking is calculated.
  • the developed system is planned to be implemented as two different versions.
  • the device can be worn as glasses and eye tracking can be performed and the desired music can be played without the need for a different system thanks to the speakers or headphones on the device.
  • the device is attached to the book and eye tracking is performed through this device.
  • music related to speakers and headphones can be played on the device that can be attached to the book.
  • the user can buy whichever product he or she finds more suitable.
  • readers with hearing difficulties can also use the developed system efficiently.
  • the aim of the invention is to make the reader listen to the previously determined music related to the feeling of the part being read with the help of eye tracking devices placed on the upper part of the physical book or the platform where the book is read or worn in the form of glasses.
  • the user can choose the music for certain feelings before starting the book. In this way, it is aimed to increase the pleasure of the reader from reading books.
  • since it is a system in which books and music will be combined it is thought that people who do not normally have the habit of reading books may also be interested in the system that is the subject of the invention and the rate of reading books can be increased. It is thought that the music used in the movies will directly give the audience the feeling of the movie, and the music to be used for the book will help to give the emotions of the book to the reader, and since it is much more interesting, its commercial value will be high.
  • Figure 1 The subject of the invention is the schematic view of the operation of the system.
  • the system developed in the subject of the invention can be automatically optimized for different books.
  • the subject of the invention is the device that can perform eye tracking by wearing glasses and the desired music plays music without the need for a different system thanks to the speakers or headphones on the device.
  • the device subject to the invention can be attached to the book and eye tracking can be done through this device.
  • the invention can be put on the book and the music related to the speaker and headphones is played on the device.
  • the invention is a system that enables the user to listen to the music related to the emotions in the part of the book being read.
  • the basic steps in the invention are as follows:

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Headphones And Earphones (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

This invention relates to the eye tracking system that enables the user to listen to the music related to the emotions in the part of the book being read, and the system that plays music according to the emotion in the sentence read and its feature is; choosing music (10) for the desired sections of predetermined books, transferring music from different sources to the system and saving (20) it externally / internally matching (30) the music recorded or sent with the book according to the feelings of the relevant book sections, for a physical book or a system developed for mobile or desktop platforms, the user choosing (40) one of two different versions of the eye tracking system placed on the glasses or on the top of the book, it includes the steps of following the eye with the developed eye tracking system and making the music listen by the user (50) matched with the part being read.

Description

SYSTEM THAT PLAYS MUSIC ACCORDING TO THE EMOTION IN THE SENTENCE READ WITH THE EYE TRACKING SYSTEM
Technical Field:
This invention relates to the eye tracking system that enables the user to listen to the music related to the emotions in the part of the book being read, and the system that plays music according to the emotion in the sentence read.
State of the Art:
Today, there are various applications for gaining the habit of reading books. In audiobook systems, which is one of these applications, the book is previously recorded with sound by a narrator and then explained to the listener without a visual.
Yasaci (2019) stated in his study that audiobooks are a rapidly developing method in the book industry. He also mentioned that audiobooks attract people who do not read books to gain the habit of reading.
As another study on audiobooks, Philips (2007) mentioned in his research that with the advent of portable cassette players, the first steps of audiobooks have been taken and people's interest in this type of book is increasing day by day. In addition, there is a subgenre of audio books called audio books, and he emphasized that these books are specially made for visually impaired people.
An example of another system in the current system is glasses with headphones and mixed reality technology. In a study of these glasses, Montuwy et al. (2019) mentioned the use of mixed reality and speaker-assisted glasses for navigation purposes for elderly people in their research. They also stated that bone conduction headphones are used for people with sensory loss. It has been stated that this technology provides convenience for elderly people both visually and audibly.
Montuwy, A., Cahour, B., & Dommes, A. (2019). Using Sensory wearable devices to navigate the city: effectiveness and user experience in older pedestrians. Multimodal Technologies and Interaction, 3(1), 17.
Philips, D. (2007). Talking books: The encounter of literature and technology in the audio book. Convergence, 13(3), 293-306.
Yasaci, S. (2019). The development of reading habits in the context of digitalization in Turkey: Audiobooks example (Doctoral dissertation, Marmara University (Turkey).
In audiobook technology, which is the prior art mentioned above, books are narrated audibly. For this reason, it does not attract the attention of users who want to read the book instead of listening to it. Thanks to the developed system, it is thought that the feelings of certain parts of the book will be given to the readers with music, and the pleasure from reading will increase, and thus the rate of reading will increase. Mixed reality and speaker glasses, which are another technique, do not have eye tracking technology. This is an important disadvantage in literature. In wearable eye tracking technology, which is another similar technique, the device works dependent on a computer. However, the fact that these systems are dependent on another device for eye tracking poses a disadvantage due to both cost and space occupancy. Also, previous techniques do not support speakers or headphones.
Consequently, there is a need for new technology that can overcome the disadvantages mentioned above. Description of the Invention:
This invention is a system that plays music according to the emotion in the sentence read with the eye tracking system, and its feature is that it is a new technology.
The invention in order to realize all the objectives mentioned above and which will emerge from the detailed description below; With the help of eye tracking devices that are placed on the top of the physical book or the mobile or desktop platform where the book is read, or that can be worn like glasses, the reader listens to the predetermined music related to the feeling of the section being read. In addition, the user can choose the music for certain feelings before starting the book. In this way, it is thought that the pleasure of the user from reading books will be increased and the rate of reading books, which is very low today, can be increased.
Eye tracking systems can be examined in two categories as software and hardware. In terms of hardware, eye tracking devices follow the physical movements of the eye with infrared rays (Akgiil and Giine§, 2019). In software, the necessary calibrations are made and the place where the eye is looking is calculated. The developed system is planned to be implemented as two different versions. In the first version, the device can be worn as glasses and eye tracking can be performed and the desired music can be played without the need for a different system thanks to the speakers or headphones on the device. In the second version, the device is attached to the book and eye tracking is performed through this device. As with the glasses-style device, music related to speakers and headphones can be played on the device that can be attached to the book. The user can buy whichever product he or she finds more suitable. In addition, with the bone conduction headphone option, readers with hearing difficulties can also use the developed system efficiently.
The aim of the invention is to make the reader listen to the previously determined music related to the feeling of the part being read with the help of eye tracking devices placed on the upper part of the physical book or the platform where the book is read or worn in the form of glasses. In addition, the user can choose the music for certain feelings before starting the book. In this way, it is aimed to increase the pleasure of the reader from reading books. In addition, since it is a system in which books and music will be combined, it is thought that people who do not normally have the habit of reading books may also be interested in the system that is the subject of the invention and the rate of reading books can be increased. It is thought that the music used in the movies will directly give the audience the feeling of the movie, and the music to be used for the book will help to give the emotions of the book to the reader, and since it is much more interesting, its commercial value will be high.
Description of the Figures:
The invention will be described with reference to the accompanying figures, so that the features of the invention will be more clearly understood and appreciated, but the purpose of this is not to limit the invention to these certain regulations. On the contrary, it is intended to cover all alternatives, changes and equivalences that can be included in the area of the invention defined by the accompanying claims. The details shown should be understood that they are shown only for the purpose of describing the preferred embodiments of the present invention and are presented in order to provide the most convenient and easily understandable description of both the shaping of methods and the rules and conceptual features of the invention. In these drawings.
Figure 1 The subject of the invention is the schematic view of the operation of the system.
The figures to help understand the present invention are numbered as indicated in the attached image and are given below along with their names.
Description of References:
10. Choosing Music 20. Saving
30. Matching
40. Choosing
50. Listening by the User
Description of The Invention:
Invention; Choosing music (10) for the desired sections of predetermined books, transferring music from different sources to the system and saving (20) it externally / internally matching (30), the music recorded or sent with the book according to the choosing (40) of the relevant book sections, physical book or two different platforms, mobile or desktop software or hardware is developed for eye-tracking system from one version to be selected by the user paired with eye tracking eye tracking system was developed to be listening by the user (50) and read the chapter includes the process steps.
The system developed in the subject of the invention can be automatically optimized for different books. The subject of the invention is the device that can perform eye tracking by wearing glasses and the desired music plays music without the need for a different system thanks to the speakers or headphones on the device.
The device subject to the invention can be attached to the book and eye tracking can be done through this device.
The invention can be put on the book and the music related to the speaker and headphones is played on the device.
Detailed Description of The Invention:
The invention is a system that enables the user to listen to the music related to the emotions in the part of the book being read. The basic steps in the invention are as follows:
1- Choosing music (10) for the desired chapters of the predetermined books, 2- Transferring music from different sources to the system and saving (20) it extemally/intemally,
3- Matching (30) the music recorded or sent with the book according to the feelings of the book chapters,
4- Choosing (40) one of two different eye tracking system versions developed for books and other mobile and desktop platforms,
5- Following the eye with the developed eye tracking system and listening by the user (50) matched with the reading section,
6- Automatic optimization of the developed system for different books.
The successful implementation of these steps ensures the operation of the system that plays music appropriate to the emotion of the book section being read. Thanks to this system, it is thought that the enjoyment of reading books and the rate of reading will increase.

Claims

CLAIMS - The invention is related to the system that plays music according to the emotion in the sentence read with the eye tracking system and its feature is; choosing music (10) for the desired sections of predetermined books, transferring music from different sources to the system and saving (20) it externally / internally matching (30) the music recorded or sent with the book according to the feelings of the relevant book sections, for a physical book or a system developed for mobile or desktop platforms, the user choosing (40) one of two different versions of the eye tracking system placed on the glasses or on the top of the book, it includes the steps of following the eye with the developed eye tracking system and making the music listen by the user (50) matched with the part being read. - As mentioned in Claim 1, it is a system that plays music according to the emotion in the sentence read with the eye tracking system, and its feature is; it is characterized by the fact that the developed system can be automatically optimized for different books. - As mentioned in Claim 1, it is an alternative arrangement of the system that plays music according to the emotion in the sentence read with the eye tracking system, and its feature is; the device is characterized by eye tracking by wearing glasses and playing the desired music without the need for a different system thanks to the speakers or headphones on the device. - As mentioned in Claim 1, it is an alternative arrangement of the system that plays music according to the emotion in the sentence read with the eye tracking system, and its feature is; it is characterized by the fact that the device is attached to the book and eye tracking is performed through this device.
7 - As mentioned in Claim 1 or Claim 4, it is an alternative arrangement of the system that plays music according to the emotion in the sentence read with the selftracking system, and its feature is; it is characterized by the playing of music related to speakers and headphones on the device that can be attached to the book. - As mentioned in Claim 1, it is an alternative arrangement of the system that plays music according to the emotion in the sentence read with the eye tracking system, and its feature is; it is characterized by the playing of relevant music by bone conduction headphones, which can be used optionally to help users with hearing difficulties use the system more efficiently.
8
PCT/TR2022/051411 2021-12-06 2022-12-05 System that plays music according to the emotion in the sentence read with the eye tracking system WO2023107063A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
TR2021/019123 2021-12-06
TR2021/019123A TR2021019123A2 (en) 2021-12-06 2021-12-06 THE SYSTEM THAT PLAYS MUSIC ACCORDING TO THE EMOTION IN THE SENTENCE READ WITH THE EYE TRACKING SYSTEM

Publications (1)

Publication Number Publication Date
WO2023107063A1 true WO2023107063A1 (en) 2023-06-15

Family

ID=85114350

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/TR2022/051411 WO2023107063A1 (en) 2021-12-06 2022-12-05 System that plays music according to the emotion in the sentence read with the eye tracking system

Country Status (2)

Country Link
TR (1) TR2021019123A2 (en)
WO (1) WO2023107063A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006100645A2 (en) * 2005-03-24 2006-09-28 Koninklijke Philips Electronics, N.V. Immersive reading experience using eye tracking
US20150082136A1 (en) * 2013-09-18 2015-03-19 Booktrack Holdings Limited Playback system for synchronised soundtracks for electronic media content
EP3040812A1 (en) * 2014-12-29 2016-07-06 Immersion Corporation Systems and methods for generating haptic effects based on eye tracking

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006100645A2 (en) * 2005-03-24 2006-09-28 Koninklijke Philips Electronics, N.V. Immersive reading experience using eye tracking
US20150082136A1 (en) * 2013-09-18 2015-03-19 Booktrack Holdings Limited Playback system for synchronised soundtracks for electronic media content
EP3040812A1 (en) * 2014-12-29 2016-07-06 Immersion Corporation Systems and methods for generating haptic effects based on eye tracking

Also Published As

Publication number Publication date
TR2021019123A2 (en) 2021-12-21

Similar Documents

Publication Publication Date Title
Szarkowska Text-to-speech audio description: towards wider availability of AD
Fryer An introduction to audio description: A practical guide
Jones et al. ONTRACK: Dynamically adapting music playback to support navigation
Taylor et al. Fitting and dispensing hearing aids
Lopez et al. Seeing films through sound: Sound design, spatial audio, and accessibility for visually impaired audiences
Stahl The roaring navigator: a group guide for the zoo with shared auditory landmark display
CN110992970A (en) Audio synthesis method and related device
CN110915240B (en) Method for providing interactive music composition to user
Sodnik et al. Spatial auditory human-computer interfaces
Yang et al. Audio augmented reality: A systematic review of technologies, applications, and future research directions
WO2020255767A1 (en) Information processing system, information processing method, and recording medium
Warren et al. Navigation via continuously adapted music
Indans et al. Towards an audio-locative mobile application for immersive storytelling
Morgan Categorical and dimensional ratings of emotional speech: Behavioral findings from the Morgan emotional speech set
Gunn On recording performance or speech, the cry, and the anxiety of the fix
Lilliestam Research on music listening: From typologies to interviews with real people
Önen The Voice as a Narrative Element in Documentary Films
Ferziger et al. Audio description in the theater: assessment of satisfaction and quality of the experience among individuals with visual impairment
Wang Radio Culture in Cold War Hong Kong
Huwiler Radio drama adaptations: An approach towards an analytical methodology
WO2023107063A1 (en) System that plays music according to the emotion in the sentence read with the eye tracking system
Hennig Natural user interfaces and accessibility
Lopez et al. The Design of an Audio Film: Portraying Story, Action and Interaction through Sound.
Frith et al. Make a sound garden grow: Exploring the new media potential of social soundscaping
Wormnæs et al. Audio-described educational materials: Ugandan teachers’ experiences

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22904826

Country of ref document: EP

Kind code of ref document: A1