EP3610366B1 - Display apparatus and controlling method thereof - Google Patents
Display apparatus and controlling method thereof Download PDFInfo
- Publication number
- EP3610366B1 EP3610366B1 EP18843147.2A EP18843147A EP3610366B1 EP 3610366 B1 EP3610366 B1 EP 3610366B1 EP 18843147 A EP18843147 A EP 18843147A EP 3610366 B1 EP3610366 B1 EP 3610366B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- type information
- rendering
- weight
- volume level
- display apparatus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 23
- 238000009877 rendering Methods 0.000 claims description 190
- 230000005236 sound signal Effects 0.000 claims description 116
- 230000005540 biological transmission Effects 0.000 claims description 28
- 230000000694 effects Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000000344 soap Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000012780 transparent material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/60—Receiver circuitry for the reception of television signals according to analogue transmission standards for the sound signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
- H04R1/025—Arrangements for fixing loudspeaker transducers, e.g. in a box, furniture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
- H04R2499/10—General applications
- H04R2499/15—Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S1/00—Two-channel systems
- H04S1/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/13—Aspects of volume control, not necessarily automatic, in stereophonic sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- Apparatuses and methods consistent with exemplary embodiments of the present disclosure relate to a display apparatus and a controlling method thereof and, more particularly, to a display apparatus which outputs a rendered sound signal and a controlling method thereof.
- an electronic apparatus having a speaker such as a display apparatus and an audio output apparatus applies various rendering techniques to contents to satisfy expectation of a user at the time of listening to a sound signal.
- a display apparatus performs rendering so that the contents of the news, current affairs, and documentary genres are to be emphasized so as to emphasize the voice transmission power, and the rendering is performed so that the movie emphasizes three-dimensional feeling, music plays a high reproduction band, and sports emphasizes presence feeling. Accordingly, a user can listen to the sound signal rendered differently according to different genres of contents, and receive various user experiences.
- the current display apparatus maintains a fixed rendering policy for each genre of content, there is a problem that a user's listening level according to the current volume level of the display apparatus is not considered. For example, even at low volume levels, the content of the sports genre is rendered to emphasize the presence only, so that the user could not properly listen to the narrator's voice.
- Patent publication US 2008/0153537 A1 describes dynamically learning a user's response via user-preferred audio settings in response to different noise environments.
- Patent publication KR1020160079577A describes a method and apparatus for reproducing content using metadata.
- Exemplary embodiments address at least the above problems and/or disadvantages and other disadvantages not described above. Also, the exemplary embodiments are not required to overcome the disadvantages described above, and may not overcome any of the problems described above.
- the present disclosure includes one or more exemplary embodiments that may address and/or solve the above-mentioned needs, and it is an object of the one or more exemplary embodiments of the present disclosure to provide a display apparatus which renders a sound signal based on a genre of a content and a volume setting level of a display apparatus to provide a user with an optimized sound, and a controlling method thereof.
- the processor may, based on the set volume level exceeding the predetermined range, render the sound signal based on the first rendering type information, and based on the set volume level being less than the predetermined range, render the sound signal based on the second rendering type information.
- the processor is configured to, based on the set volume level being within the predetermined range and approaching to a lower limit of the predetermined range relative to an upper limit of the predetermined range, set the second weight to be greater than the first weight, and based on the set volume level being within the predetermined range and approaching to the upper limit of the predetermined range relative to the lower limit of the predetermined range, set the first weight to be greater than the second weight.
- the information regarding the predetermined range of each content genres is stored in the storage, and wherein the processor identifies the predetermined range which corresponds to a genre of the content based on the stored information.
- the processor may obtain the weight based on the set volume level and a hearing level, and the hearing level is obtained based on at least one of an average viewing distance of a user to the display apparatus, a content viewing time, a size of the sound signal, and an ambient noise level.
- Each of the first rendering type information and the second rendering type information is parameter information to adjust at least one of a voice transmission power, a low frequency range, a reproduction band, a presence, and a sense of space.
- the genre of the displayed content may be included in the content or obtained based on the sound signal.
- the rendering may include, based on the set volume level exceeding the predetermined range, rendering the sound signal based on the first rendering type information, and based on the set volume level being within the predetermined range, rendering the sound signal based on the second rendering type information.
- the rendering may include, based on the set volume level being within the predetermined range and approaching to a lower limit of the predetermined range relative to an upper limit of the predetermined range, setting the second weight to be greater than the first weight, and based on the set volume level within the predetermined range and approaching to the upper limit of the predetermined range relative to the lower limit of the predetermined range, setting the first weight to be greater than the second weight.
- the rendering may include storing information regarding the predetermined range of each content genres, wherein the rendering further comprises identifying the predetermined range which corresponds to a genre of the content based on the stored information.
- the rendering may include obtaining the weight based on the set volume level and a hearing level, and wherein the hearing level is obtained based on at least one of an average viewing distance of a user to the display apparatus, a content viewing time, a size of the sound signal, and an ambient noise level.
- the genre of the displayed content may be included in the content or obtained based on the sound signal.
- FIG. 1 is a view illustrating an example of a display apparatus according to an exemplary embodiment.
- a display apparatus 100 may be implemented as a digital TV having at least one speaker.
- the present disclosure is not limited thereto and can be implemented in various types of devices having display functions such as a PC, a mobile phone, a tablet PC, a portable multimedia player (PMP), a portable multimedia player (PDA), and a navigation device.
- PMP portable multimedia player
- PDA portable multimedia player
- the various embodiments of the present disclosure are applicable to an electronic apparatus including at least one speaker unit such as an audio apparatus, a sound bar, a home theater system, and a room speaker without a display function.
- the speaker unit has a function of converting electric pulses into sound waves, and can be realized as, for example, a dynamic type according to a principle and a method of converting electric signals into sound waves.
- the present disclosure is not limited thereto, and it may be implemented as an electrostatic type, a dielectric body type, a magnetostrictive type, or the like within the scope of the present disclosure.
- the display apparatus 100 is implemented as a digital TV for convenience of description.
- the display apparatus 100 receives various genres of contents according to an embodiment.
- the contents of various genres refer to kinds of contents such as movies, soap operas, sports, news, and music.
- the genres of contents may be represented by characteristics, type, category, etc. of contents, but are generally referred to as the genres of contents.
- the display apparatus 100 acquires rendering type information based on genres of contents and volume level which is currently set in the display apparatus 100, and outputs a sound signal based on the acquired rendering type information.
- FIG. 2 is a block diagram illustrating a configuration of a display apparatus according to an exemplary embodiment.
- the display apparatus 100 includes a display 110, a storage 120, a processor 130, and a speaker 140.
- the display 110 provides various content screens that can be provided through the display apparatus 100.
- the content screen includes various contents such as an image, a video, a text, music, etc., an application execution screen including various contents, a graphic user interface (GUI) screen and so on.
- GUI graphic user interface
- the display 110 may be implemented as various types of displays such as a liquid crystal display, an organic light-emitting diode, a liquid crystal on silicon (LCoS), a digital light processing (DLP) and the like.
- the display 110 may be implemented as a transparent display that is made of a transparent material and displays information.
- the display 110 may be implemented as a touch screen having a mutual layer structure with the touch pad. In this case, the display 110 may be used as a user interface in addition to an output device.
- the storage 120 stores various data, programs, or applications for driving and controlling the display apparatus 100.
- the storage 120 stores rendering type information for each content genre according to an embodiment of the present disclosure.
- the rendering type information is a parameter information for providing various effects to the sound signal of the content.
- the rendering type information may include information on parameter setting values for emphasizing at least one of a sound transmission power, a reproduction band, a low frequency range, a sense of space, and presence.
- the sound signal may be a digital sound signal, but is not limited thereto.
- a plurality of types of rendering information are matched and stored in one genre.
- the first rendering type information and the second rendering type information may correspond to the sports genre.
- the first rendering type information may be information used when the volume level set in the display apparatus 100 exceeds an upper limit (hereinafter referred to as a first level) of a predetermined volume level range.
- the second rendering type information may be information used when the volume level set in the display apparatus 100 is less than a lower limit (hereinafter referred to as a second level) of a predetermined volume level range.
- the rendering type information matched by genre is not limited to two, and two or more rendering type information may be matched.
- the storage 120 may store rendering type information corresponding to a sub-genre, as well as rendering-type information per genre.
- the storage 120 may store a plurality of rendering type information corresponding to various sub-genres such as horror, melodrama, action, war, and SF in the movie genre.
- the volume level set on the display apparatus means a volume level corresponding to the volume set by the user among the volume levels that the display apparatus 100 can provide.
- the plurality of volume levels that can be provided in the display apparatus 100 may differ depending on the characteristics of the display apparatus 100, for example, a manufacturer, a type, model, and the like. That is, different volume levels (for example, sound pressure level (SPL)) may be mapped to the same volume (for example, 10) for each characteristic of the display apparatus 100.
- SPL sound pressure level
- the dBSPL value corresponding to the current volume of the display apparatus 100 is referred to as the volume level currently set in the display apparatus.
- the processor 130 controls the overall operation of the display apparatus 100.
- the processor 130 may include one or more of a digital signal processor (DSP), a central processing unit (CPU), a controller, an application processor (AP), or a communication processor (CP)), an ARM processor, or may be defined by a corresponding term.
- DSP digital signal processor
- CPU central processing unit
- AP application processor
- CP communication processor
- ARM processor ARM processor
- the processor 130 obtains the first and second rendering type information corresponding to the genre of the content displayed on the display 110 from the storage 120.
- the present disclosure is not limited thereto, and it is needless to say that at least one rendering type information corresponding to the genre of the content can be obtained from an external server, metadata regarding a content, and so on.
- the processor 130 renders the sound signal of the content based on at least one of the first rendering type information and the second rendering type information corresponding to the genre of the content displayed on the display 110.
- the processor 130 applies a weight according to the currently set volume level to at least one of the first rendering type information and the second rendering type information, to obtain third rendering type information, and render the sound signal based on the third rendering type information.
- the first and second weights may be respectively applied and summed to the first rendering type information and the second rendering type information corresponding to the genre of content, and the third rendering type information can be obtained.
- the predetermined volume level range is not limited to 40 dBSPL to 80 dBSPL, and the predetermined volume level range may be set differently according to a user setting, manufacturer's setting, viewing time, content genre, and the like.
- the lower and upper limits of the volume level range set at 40 dBSPL and 80 dBSPL, respectively can be adjusted to reflect the human auditory system, which is relatively sensitive at evening and early morning hours.
- the predetermined volume level range can be adjusted from 35dBSPL to 75dBSPL.
- the processor 130 when a volume level which is set in the display apparatus 100 exceeds a predetermined volume level range, performs rendering of a sound signal of a content based on one of first rendering type information and second rendering type information corresponding to the displayed genres of contents.
- the processor 130 may render the sound signal based on the first rendering type information corresponding to the content genre.
- the first rendering type information may include a rendering parameter setting value to emphasize the genre characteristic of the corresponding content in the audio signal of the corresponding content.
- the genre characteristic may mean a sound effect that the user generally expects in the genre of the content.
- the user may desire to be provided with a sound effect reflecting genre characteristics of the content. Accordingly, in the present disclosure, when the setting volume level of the display apparatus 100 exceeds the first level, that is, the volume level of the predetermined threshold value, the sound signal is rendered based on the first rendering type information, and the desired effect can be provided at the corresponding volume level.
- the first rendering type information may include a parameter setting value for rendering the sound signal to be emphasized such that the sense of space and the presence are relatively emphasized relative to the voice transmission power.
- the first rendering type information corresponding to the sports genre includes information on parameter setting values for emphasizing the presence and sense of space relative to the voice transmission power, the reproduction band, and the low frequency range as shown in Table 1 below.
- the user can be provided with other sound effects such as a commentary voice due to a high volume setting while receiving a sound signal emphasizing presence and sense of space by a certain level or more.
- Table 1 Voice transmission power Reproduction band Low frequency range Sense of space Presence 3 2 3 4 5
- the processor 130 renders a sound signal based on the second rendering type information if the set volume level of the display apparatus 100 is less than a lower limit of a predetermined volume level range (for example, 40 dBSPL).
- the second rendering type information includes a rendering parameter setting value for emphasizing a sound effect that falls below a predetermined level when the setting volume is lowered below a second level in the genre of the content.
- the set volume level of the display apparatus 100 is less than a lower limit of the predetermined volume level range, it means that the user is viewing the content with a relatively low volume.
- some effects from among various effects of the sound signal of the content may not be provided to a user properly.
- the user can not recognize 'dialogue', 'narrator's voice' and so on, which correspond to the voice transmission power, due to the low volume of the display apparatus 100.
- the sound signal is rendered based on the fixed rendering type information for emphasizing the genre characteristic, at least one of the effects of the sound signal (for example, at least one of the voice transmission power, presence, sense of space, reproduction band, and low frequency range) can be further lowered.
- the processor 130 renders a sound signal based on the second rendering type information to emphasize a sound effect which is declined in the corresponding content genre.
- the sports genre content may be rendered with the sound signal being emphasized relative to the presence and the sense of space.
- the second rendering type information corresponding to the sports genre includes information on the parameter setting values for emphasizing the voice transmission power relative to the presence, the sense of space, the reproduction band, and the low frequency range as shown in Table 2 below.
- Table 2 Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 2 2 2 2 2 2
- the processor 130 is configured such that if the set volume level of the display apparatus 100 is within a predetermined volume level range, the first and second weights are summed up and applied to each of the first and second rendering type information corresponding to the content and the third rendering type information can be obtained.
- the processor 130 when the set volume level approaches to a lower limit (that is, the second level) than an upper limit (that is, the first level) of the predetermined volume level range, may set the second weight to be greater than the first weight.
- the processor 130 may apply a weight of 0.75 to the second rendering type information and a weight of 0.25 to the first rendering type information.
- the third rendering type information can be obtained as Table 3 below. [Table 3] Voice transmission power Reproduction band Low frequency range Sense of space Presence 4.5 2 2.25 2.5 2.75
- the first weight may be set to be higher than the second weight.
- the processor 130 may apply a weight of 0.75 to the first rendering type information and a weight of 0.25 to the second rendering type information. That is, the weight can be calculated based on the following Equation 1. SPL max ⁇ SPL est SPL min
- SPL max indicates a upper limit of the predetermined volume level range
- SPL min indicates the currently set volume level or hearing level which is currently set in the display apparatus 100
- SPL min indicates a lower limit in the predetermined volume level.
- the processor 130 may render the sound signal by using a voice emphasis filter (or center sound rendering filter), sound effect emphasis filter (or background sound rendering filter), music emphasis filter and so on, when rendering the sound signal to emphasize at least one of the voice transmission power, reproduction band, low frequency range, sense of space, and presence.
- various filters such as a hi-pass filter, a low-pass filter, a band-pass filter, a mask filter, a head-related transfer function (HRTF) filter, and so on, can be used.
- the sound signal for which the processor 130 performs rendering can be a plurality of channel signals (for example, a stereo signal).
- the present disclosure is not limited thereto, and the sound signal may be included in the sound content itself or may be a part of the video content. According to an exemplary embodiment, any signal which can be divided into a center signal and an ambient signal is applicable.
- the processor 130 may divide a stereo signal into a center (or a primary, hereinafter referred to as a first sound signal) and an ambient (reverberation, applause, wind, various background sounds) signal (hereinafter referred to as a second sound signal).
- the processor 130 when voice is detected from the first sound signal, may apply the voice emphasis filter to the first sound signal. Accordingly, rendering can be performed so that the first sound signal is comparatively emphasized than the second sound signal and the voice transmission power can be improved.
- the weight of the voice emphasis filter is increased and the weight of the background sound emphasis filter is reduced because the importance of the dialogue transmission increases.
- the volume level is equal to or higher than the first level
- the weight of the background sound emphasis filter can be increased and the weight of the voice emphasis filter can be reduced.
- it is also possible to emphasize the voice transmission power by increasing the gain of the mid-range voice with respect to the low / high band gain.
- the processor 130 may divide the sound signal into the bands of low sound, middle sound, and high sound, and enhance the signal components corresponding to the low sound and high sound.
- it is possible to enhance the low-frequency range by emphasizing the low-frequency range of the mid / high range.
- the processor 130 may increase the gain of the sound effect (or background sound) in order to enhance the presence of the signal.
- the processor 130 may apply a sound effect emphasis filter to the second and third sound signals to detect a sound effect in the second sound signal, thereby rendering the second sound signal.
- a high-frequency range emphasis filter may be applied to the sound signal.
- the display apparatus 100 can output a sound signal emphasizing the sound effect and the presence.
- the processor 130 may emphasize the presence by rendering the second sound signal including the crowded sound of the audience among the sound signals of the sports genre contents to be emphasized.
- the processor 130 may use at least one of a head-related transfer function (HRTF) filter and a crosstalk cancellation to emphasize the sense of space of a sound signal.
- HRTF head-related transfer function
- the two-channel crosstalk cancellation may render the sound signal output through the L channel speaker based on the transmission path of the sound signal to be transmitted only to the left ear of the user, and the sound signal output through the R channel speaker (Right ear).
- the processor 130 identifies a predetermined volume level range for each genre. Since the predetermined volume level range may be different for each genre, the processor 130 may identify a predetermined volume level range corresponding to the genre of the content. For example, the movie and music genres range from a predetermined volume level of 40 dBSPL to 85 dBSPL, and the sports and news / TV show genres range from a predetermined volume level of 40 dBSPL to 79 dBSPL. Meanwhile, a predetermined volume level range for each content genre is pre-stored in the storage 120. However, the present disclosure is not limited thereto, and it is needless to say that a predetermined volume level range can be set by setting of an external server, metadata of contents, user and manufacturer.
- a predetermined volume level range (for example, a default volume level range) may be set regardless of the genre.
- the default volume level range may be 40 dBSPL (sound pressure level) to 80 dBSPL.
- the predetermined volume level range may be adjusted according to the viewing time.
- the lower and upper limits of a predetermined volume level range may be adjusted to be relatively small at a specific time zone (e.g., evening and early morning hours) when the human listening system becomes sensitive.
- the volume level range set at 40dBSPL to 80dBSPL can be adjusted to 35dBSPL-75dBSPL at dawn when the human listening system becomes sensitive.
- the processor 130 identifies a genre of a content based on information of a genre included in a content and a sound signal. For example, the processor 130 may identify a genre of a content based on genre information (for example, metadata) included in a content itself.
- genre information for example, metadata
- the processor 130 can identify the genre of the content based on the sound signal of the content. For example, it is possible to identify a frequency change amount in units of a predetermined section (hereinafter, referred to as a detection section) of a sound signal, and to identify a genre corresponding to the confirmed frequency change amount.
- the predetermined detection interval unit may be at least one of at least one audio frame unit, at least one audio scene unit, and at least one audio chunk unit.
- the processor 130 obtains a weight based on the volume level and the listening level currently set in the display apparatus 100.
- the third rendering type information is obtained by acquiring the weight using the listening level together with the set volume level, and applying the obtained weight to the first rendering type information and the second rendering type information.
- the listening level is obtained based on at least one of the average viewing distance of the user to the display apparatus 100, the content viewing time, and the size of the sound signal of the content.
- the processor 130 may set the second weight which is applied to the second rendering type information corresponding to the low level as a comparatively greater than the first weight. A detailed description thereof will be given in FIG. 8 .
- the speaker 140 outputs the rendered sound signal.
- the speaker 140 may be implemented as at least one speaker unit.
- the speaker 140 may include a plurality of speakers for multi-channel reproduction.
- the speaker 140 may include a plurality of speakers serving as channels for outputting mixed sound signals.
- a speaker for at least one channel may be implemented as a speaker array including a plurality of speaker units for reproducing different frequency bands.
- FIG. 3 is a block diagram to describe a detailed configuration of a display apparatus according to an exemplary embodiment.
- the display apparatus 100 includes a display 110, a storage 120, a processor 130, a speaker 140, a content receiver 150, a communicator 160, a remote control signal receiver 170, and an inputter 180.
- the detailed description of the components shown in FIG. 3 will be omitted for the parts overlapping the components shown in FIG. 2 .
- the processor 130 renders a sound signal of a content based on the rendering type information which is stored in the storage 120.
- the processor 130 may include a CPU, a ROM (or non-volatile memory) storing a control program for controlling the display apparatus 100, RAM (or volatile memory) used as a storage area for storing data or corresponding to various operations performed in the display apparatus 100.
- the CPU by accessing the storage 120, performs booting using the O / S stored in the storage 120. By using various programs, contents, and data stored in the storage 120, the CPU performs various operations.
- the storage 120 may be implemented as an internal memory such as a ROM or a RAM included in the processor 130, or may be implemented as a memory separate from the processor 130.
- the storage 120 may be implemented as a memory embedded in the display apparatus 100 or a memory which can be attached to / detached from the display apparatus 100 depending on the purpose of data storage.
- data for driving the display apparatus 100 may be stored in a memory embedded in the display apparatus 100
- data for an extension function of the display apparatus 100 may be stored in the memory which can be attached to and detached from the display apparatus 100.
- the memory embedded in the display apparatus 100 may be implemented in the form of a nonvolatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD), and a memory which can be attached to or detached from the sound output apparatus 100 can be implemented as a memory card (for example, a micro SD card, a USB memory or the like) and an external memory (for example, a USB memory) connectable to a USB port, or the like.
- a memory card for example, a micro SD card, a USB memory or the like
- an external memory for example, a USB memory
- the content receiver 150 may be implemented as a tuner for receiving broadcast content, but it is not limited thereto and may be implemented as various types of communication modules capable of receiving various external contents such as a Wi-Fi module, a USB module, and an HDMI module.
- the content receiver 150 may include a tuner, a demodulator, an equalizer, a demultiplexer, and the like.
- the content receiver 150 tunes the broadcast channel under the control of the processor 130, receives the broadcast signal desired by the user, demodulates and equalizes the received broadcast signal, and then converts the broadcast signal into a video signal, a sound signal, and additional data, etc.
- the demuxed video signal is provided to the processor 130.
- the processor 130 performs various image processing such as noise filtering, frame rate conversion, and resolution conversion on the provided video signal, and generates a frame to be output on the screen.
- the demuxed sound signal is provided to the processor 130.
- various processes such as decoding or amplification of a sound signal, noise filtering, and the like can be performed. That is, the processor 130 performs rendering on the sound signal in accordance with various embodiments of the present disclosure.
- the communicator 160 transmits / receive a sound signal.
- the communicator 160 may receive a sound signal through streaming or downloading from an external apparatus (e.g., a source device), an external storage medium (e.g., universal serial bus (USB)), an external server (e.g., a webhard) through communication methods such as an AP-based Wi-Fi, a Bluetooth, a Zigbee, a wired / wireless local area network (LAN), a wide area network (WAN), Ethernet, IEEE (Institute of Electrical and Electronics Engineers) 1394, high definition multimedia interface (HDMI), universal serial bus (USB), mobile high-definition link (MHL), advanced encryption standard (AES) / European broadcasting union (EBU), optical, coaxial, and so on.
- an external apparatus e.g., a source device
- an external storage medium e.g., universal serial bus (USB)
- an external server e.g., a webhard
- communication methods such as an AP-based
- the remote control signal receiver receives a remote control signal transmitted from a remote controller.
- the remote control signal receiver 170 may be configured to include a light receiver for receiving infra red (IR) signal, or may be configured to receive a remote control signal by performing communication according to a wireless communication protocol such as a remote controller and Bluetooth or Wi-Fi.
- IR infra red
- the inputter may be implemented by various buttons provided on the main body of the display apparatus 100.
- the user can input various user commands through the inputter 180 such as a turn-on / turn-off command, a channel conversion command, a volume control command, a menu confirmation command and so on.
- the inputter controls rendering according to various embodiments of the present disclosure in response to a user's input to an inputter.
- the processor 130 may render the sound signal based on the first and second rendering type information.
- the processor 130 may render the sound signal based on any one of the first and second rendering type information. For example, when the rendering-off command is input, the processor 130 may render the sound signal based only on the first rendering type information.
- predetermined volume level range by genres of contents and rendering type information will be described in a greater detail.
- FIG. 4 is a view to describe rendering type information of a sports genre content according to an exemplary embodiment.
- the rendering type information of the sports genre is divided into the first rendering type information and the second rendering type information.
- the display apparatus 100 renders a sound signal based on the first rendering type information when the currently set volume level of the display apparatus 100 exceeds an upper limit of a predetermined volume level range.
- the first rendering type information if performing rendering of the sound signal of the sports genre content, rendering is performed such that the presence is relatively emphasized relative to the remaining voice transmission power, reproduction band, low frequency range, and sense of space.
- the display apparatus 100 renders an sound signal based on the second rendering type information when the currently set volume level of the display apparatus 100 is less than a lower limit of a predetermined volume level range. According to the second rendering type information, the rendering is performed so that the voice transmission power is relatively emphasized compared to the remaining reproduction band, low frequency, sense of space, and presence.
- the display apparatus 100 applies the different weights to the first and second rendering type information to acquire the third rendering type information. Further, it is possible to render the sound signal based on the obtained third rendering type information.
- the content of the sports genre has a predetermined volume level range of 40 dBSPL to 79 dBSPL, and if the currently set volume level of the display apparatus 100 is 40 dBSPL to 79 dBSPL, the display apparatus 100 applies the first weight to the first rendering type information and the second weight to the second rendering type information to obtain the third rendering type information.
- the first weight can be calculated based on Equation 2 as shown below. 1 ⁇ 79 dBSPL ⁇ SPL est 40 dBSPL
- SPL est is a currently set volume level or listening level of the display apparatus 100.
- the second weight can be calculated based on Equation 3 as shown below. 79 dBSPL ⁇ SPL est 40 dBSPL
- SPL est is a currently set volume level of the display apparatus 100.
- FIGS. 5-7 are views to describe rendering type information and a predetermined volume level range according to contents of various genres.
- the rendering type information of a movie genre content is divided into the first rendering type information and the second rendering type information.
- FIG. 4 shows the first rendering type information of a movie genre according to an exemplary embodiment of the present disclosure.
- Table 4 Voice transmission power Reproduction band Low frequency range Sense of space Presence 4 4 5 5 4
- the display apparatus 100 When the currently set volume level exceeds an upper limit of the predetermined volume level range, the display apparatus 100 renders a sound signal based on rendering type information as shown in Table 4.
- the display apparatus 100 when the currently set volume level of the display apparatus 100 is less than a lower limit of a predetermined volume level range, the display apparatus 100 renders a sound signal based on the rendering type information as shown in FIG. 5 .
- the display apparatus 100 when the currently set volume level of the display apparatus 100 is within the predetermined volume level range, the display apparatus 100 applies the first and second weights to the first rendering type information and the second rendering type information to acquire the third rendering type information, and render the sound signal based on the third rendering type information. For example, if the predetermined volume level range of the movie genre is 40 dBSPL to 85 dBSPL, and the currently set volume level of the display apparatus 100 is within 40 dBSPL to 85 dBSPL, then the display apparatus 100 render the sound signal based on the obtained third rendering type information.
- the rendering type information of the news / TV show genre is divided into the first rendering type information and the second rendering type information.
- Tables 6 and 7 below show the first and second rendering type information of the news / TV show genre according to an exemplary embodiment of the present disclosure.
- [Table 6] Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 4 3 3 3
- [Table 7] Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 3 2 2 2 2
- the display apparatus 100 when the currently set volume level of the display apparatus 100 exceeds an upper limit of the predetermined volume level range, the display apparatus 100 renders a sound signal based on the first rendering type information as shown in Table 6. Further, if the currently set volume level of the display apparatus 100 is less than a lower limit of the predetermined volume level range, the display apparatus 100 renders a sound signal based on the second rendering type information as shown in Table 7.
- the display apparatus 100 applies the first and second weights to the first and second rendering type information, respectively, to acquire the third rendering type information, and render the sound signal based on the third rendering type information. For example, if the predetermined volume level range of the news / TV show genre is 40dBSPL to 79dBSPL and the current set volume level of the display apparatus 100 is within 40dBSPL to 79dBSPL, then the display apparatus 100 renders a sound signal based on the third rendering type information.
- FIGS. 8 and 9 the first and second rendering type information of a music genre content can be shown in FIGS. 8 and 9 .
- Table 8 Voice transmission power Reproduction band Low frequency range Sense of space Presence 4 5 5 3 3
- Table 9 Voice transmission power Reproduction band Low frequency range Sense of space Presence 3 5 5 2 2
- the predetermined volume level range of the music genre is 40 dBSPL to 85 dBSPL, and if the currently set volume level of the display apparatus 100 is within 40 dBSPL to 85 dBSPL, the display apparatus 100 renders a sound signal based on the acquired third rendering type information.
- the various embodiments of the present disclosure are not limited to the specific values for the rendering type information and the predetermined volume level range described in FIGS. 4 to 7 and Tables 1 to 9, and can be set differently according to various purposes of the present disclosure.
- the display apparatus 100 may have various rendering type information according to a setting of the user or a setting of the manufacturer of the display apparatus 100. It is needless to say that the genre of the contents can be more subdivided and diversified than the above-mentioned example.
- FIG. 8 is a view to describe a listening level of a user according to an exemplary embodiment.
- the display apparatus 100 calculates a weight based on the currently set volume level and listening level.
- the display apparatus 100 outputs the sound signal according to the volume level currently set in the display apparatus 100, the level of the sound signal recognized by the user's hearing system may be different from the set volume level.
- the size of the sound perceived by the user differs depending on the frequency of the sound signal. That is, the intensity of the physically measured sound does not match the size of the sound actually perceived through the human auditory system. For example, if you hear 70dB sound at 50h and 70dB at 200 Hz, the human auditory system will perceive 70dB sound at 200 Hz to be louder. The sound of 60dB at 1 kHz and 70dB at 200 Hz will be recognized as the same size.
- the display apparatus 100 determines the loudness based on the frequency of the sound signal and the set volume level of the display apparatus 100.
- loudness refers to the amount of subjective sound perceived by a human auditory system when a sound is delivered to the human ear.
- the user's perceived level (loudness) of the output sound signal may vary depending on the average viewing distance of the user to the display apparatus 100, the content viewing time, the size of the sound signal, the surrounding noise level, etc.
- the sound signal is constant in size because it is mixed with an average of -23 loudness k-weighted relative to full scale (LKFS), while other game contents and Internet contents have different average sizes of a sound signal depending on the content, and may not be constant within one content.
- the display apparatus 100 calculates a weight based on the listening level according to the size (e.g., loudness) of the subjective sound perceived by the human auditory system.
- the display apparatus 100 detects the ambient noise level and obtain the third rendering type information by further considering the detected ambient noise level. For example, if the ambient noise level exceeds a predetermined level, the volume level currently set on the display apparatus 100 may be different from the user's perception level. For example, the sound signal can be recognized at a level lower than the set volume level. Accordingly, the display apparatus 100 obtains the third rendering type information by setting the second weight to a value relatively greater than the first weight.
- the present disclosure is not limited thereto, but it is needless to say that the third rendering type information can be obtained by setting the first weight to a relatively greater value than the second weight.
- the display apparatus 100 acquires the third rendering type information in consideration of the viewing time. For example, in a particular time period (e.g., evening and early morning hours) when the human auditory system is sensitive to sound signals, a first weight that is applied to the first rendering type information and a second weight that is applied to the second rendering type information can be calculated differently from the daytime time zone. For example, in the evening and early morning hours, the first weight may be set to a relatively greater value than the second weight to obtain third render type information.
- the present disclosure is not limited to this, and it is also possible to set the second weight to a relatively large value.
- FIG. 9 is a flowchart to describe a controlling method of a display apparatus according to an exemplary embodiment.
- the first rendering type information and the second rendering type information based on the volume level are stored for each content genre shown in FIG. 9 . According to the controlling method of the display apparatus in which the first rendering type information and the second rendering type information based on the volume level are stored for each content genre shown in FIG. 9 , the first rendering type information and the second rendering type information corresponding to the genre of the content are obtained in step S910.
- the sound signal of the content is rendered in step S920.
- step S930 the rendered sound signal is output in step S930.
- step S920 when the volume level currently set in the display apparatus 100 is within a predetermined range, the sound signal of a content is rendered based on the third rendering type information which is obtained by applying a weight according to a set volume level to at least one of the first rendering type information and the second rendering type information.
- the sound signal is rendered based on the first rendering type information. If the set volume level is less than a lower limit of the predetermined range, it is possible to render the sound signal based on the second rendering type information.
- the first weight is applied to the first rendering type information and the second weight is applied to the second rendering type information to obtain the third rendering type information based on the set volume level, in step S920.
- the second weight value is set to a value greater than the first weight value when the set volume level approaches the lower limit relatively to the upper limit of the predetermined range in step S920. If it is determined that the set volume level is within a predetermined range, the first weight can be set to a value greater than the second weight when the set volume level approaches the upper limit relatively to the lower limit of the predetermined range in step S920.
- the display apparatus 100 stores information on a predetermined range for each content genre.
- the predetermined range corresponding to the genre of the content can be identified based on the stored information.
- a weight can be obtained based on the set volume level and the listening level.
- the listening level may be obtained based on at least one of the average viewing distance of the user to the display apparatus, the viewing time, the size of the sound signal, and the ambient noise level.
- each of the first rendering type information and the second rendering type information is a parameter information for adjusting at least one of a voice transmission power, a low frequency range, a reproduction band, a sense of space and presence.
- the genre information can be included in a content or obtained based on a sound signal.
- the methods according to various exemplary embodiments can be implemented only by software upgrade or hardware upgrade with respect to the existing display apparatus.
- the aforementioned various exemplary embodiments can be performed at an embedded server in a display apparatus or an external server of a display apparatus.
- the various embodiments described above can be implemented in a recording medium that can be read by a computer or a similar device using software, hardware, or a combination thereof.
- the embodiments described herein may be implemented by the processor itself.
- embodiments such as the procedures and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more of the functions and operations described herein.
- Computer instructions for performing the processing operations according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium.
- Computer instructions stored in such non-volatile computer-readable media may cause a particular device to perform processing operations according to various embodiments described above when executed by a processor.
- the non-transitory computer-recordable medium is not a medium configured to temporarily store data such as a register, a cache, or a memory but an apparatus-readable medium configured to semi-permanently store data.
- the above-described various applications or programs may be stored in the non-transitory apparatus-readable medium such as a compact disc (CD), a digital versatile disc (DVD), a hard disc, a Blu-ray disc, a universal serial bus (USB), a memory card, or a read only memory (ROM), and provided therein.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- Health & Medical Sciences (AREA)
- Circuit For Audible Band Transducer (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Television Receiver Circuits (AREA)
- Stereophonic System (AREA)
Description
- Apparatuses and methods consistent with exemplary embodiments of the present disclosure relate to a display apparatus and a controlling method thereof and, more particularly, to a display apparatus which outputs a rendered sound signal and a controlling method thereof.
- Recent development of electronic technology has led to development and spread of various types of electronic devices. In particular, an electronic apparatus having a speaker such as a display apparatus and an audio output apparatus applies various rendering techniques to contents to satisfy expectation of a user at the time of listening to a sound signal.
- For example, a display apparatus performs rendering so that the contents of the news, current affairs, and documentary genres are to be emphasized so as to emphasize the voice transmission power, and the rendering is performed so that the movie emphasizes three-dimensional feeling, music plays a high reproduction band, and sports emphasizes presence feeling. Accordingly, a user can listen to the sound signal rendered differently according to different genres of contents, and receive various user experiences.
- However, since the current display apparatus maintains a fixed rendering policy for each genre of content, there is a problem that a user's listening level according to the current volume level of the display apparatus is not considered. For example, even at low volume levels, the content of the sports genre is rendered to emphasize the presence only, so that the user could not properly listen to the narrator's voice.
- Patent publication
US 2008/0153537 A1 describes dynamically learning a user's response via user-preferred audio settings in response to different noise environments. Patent publicationKR1020160079577A - Exemplary embodiments address at least the above problems and/or disadvantages and other disadvantages not described above. Also, the exemplary embodiments are not required to overcome the disadvantages described above, and may not overcome any of the problems described above.
- The present disclosure includes one or more exemplary embodiments that may address and/or solve the above-mentioned needs, and it is an object of the one or more exemplary embodiments of the present disclosure to provide a display apparatus which renders a sound signal based on a genre of a content and a volume setting level of a display apparatus to provide a user with an optimized sound, and a controlling method thereof.
- According to an exemplary embodiment, there is provided a display apparatus as set out in the accompanying claims.
- The processor may, based on the set volume level exceeding the predetermined range, render the sound signal based on the first rendering type information, and based on the set volume level being less than the predetermined range, render the sound signal based on the second rendering type information.
- The processor is configured to, based on the set volume level being within the predetermined range and approaching to a lower limit of the predetermined range relative to an upper limit of the predetermined range, set the second weight to be greater than the first weight, and based on the set volume level being within the predetermined range and approaching to the upper limit of the predetermined range relative to the lower limit of the predetermined range, set the first weight to be greater than the second weight.
- The information regarding the predetermined range of each content genres is stored in the storage, and wherein the processor identifies the predetermined range which corresponds to a genre of the content based on the stored information.
- The processor may obtain the weight based on the set volume level and a hearing level, and the hearing level is obtained based on at least one of an average viewing distance of a user to the display apparatus, a content viewing time, a size of the sound signal, and an ambient noise level.
- Each of the first rendering type information and the second rendering type information is parameter information to adjust at least one of a voice transmission power, a low frequency range, a reproduction band, a presence, and a sense of space.
- The genre of the displayed content may be included in the content or obtained based on the sound signal.
- According to an exemplary embodiment, there is provided a controlling method of a display apparatus as set out in the accompanying claims.
- The rendering may include, based on the set volume level exceeding the predetermined range, rendering the sound signal based on the first rendering type information, and based on the set volume level being within the predetermined range, rendering the sound signal based on the second rendering type information.
- The rendering may include, based on the set volume level being within the predetermined range and approaching to a lower limit of the predetermined range relative to an upper limit of the predetermined range, setting the second weight to be greater than the first weight, and based on the set volume level within the predetermined range and approaching to the upper limit of the predetermined range relative to the lower limit of the predetermined range, setting the first weight to be greater than the second weight.
- The rendering may include storing information regarding the predetermined range of each content genres, wherein the rendering further comprises identifying the predetermined range which corresponds to a genre of the content based on the stored information.
- The rendering may include obtaining the weight based on the set volume level and a hearing level, and wherein the hearing level is obtained based on at least one of an average viewing distance of a user to the display apparatus, a content viewing time, a size of the sound signal, and an ambient noise level..
- Each of the first rendering type information and the second rendering type information is parameter information to adjust at least one of a voice transmission power, a low frequency range, a reproduction band, a presence, and a sense of space.
- The genre of the displayed content may be included in the content or obtained based on the sound signal.
- According to various embodiments of the present disclosure, it is possible to provide a user with a certain level of listening ability by rendering different characteristics, from among various characteristics of a sound signal, to be emphasized according to the genre of the content and the volume setting value of the display apparatus.
- The above and/or other aspects will be more apparent by describing certain exemplary embodiments, with reference to the accompanying drawings, in which:
-
FIG. 1 is a view illustrating an example of a display apparatus according to an exemplary embodiment. -
FIG. 2 is a block diagram illustrating a configuration of a display apparatus according to an exemplary embodiment. -
FIG. 3 is a block diagram to describe a detailed configuration of a display apparatus according to an exemplary embodiment. -
FIG. 4 is a view to describe rendering type information of a sports genre content according to an exemplary embodiment. -
FIGS. 5-7 are views to describe rendering type information and a predetermined volume level range according to contents of various genres. -
FIG. 8 is a view to describe a listening level of a user according to an exemplary embodiment. -
FIG. 9 is a flowchart to describe a controlling method of a display apparatus according to an exemplary embodiment. - Exemplary embodiments are described in greater detail below with reference to the accompanying drawings.
- In the following description, like drawing reference numerals are used for like elements, even in different drawings. The matters defined in the description, such as detailed construction and elements, are provided to assist in a comprehensive understanding of the exemplary embodiments. However, it is apparent that the exemplary embodiments may be practiced without those specifically defined matters. Also, well-known functions or constructions are not described in detail since they would obscure the description with unnecessary detail.
- A singular expression includes a plural expression, unless otherwise specified. It is to be understood that the terms such as "comprise" or "consist of" are used herein to designate a presence of characteristic, number, step, operation, element, component, or a combination thereof, and not to preclude a presence or a possibility of adding one or more of other characteristics, numbers, steps, operations, elements, components or a combination thereof.
-
FIG. 1 is a view illustrating an example of a display apparatus according to an exemplary embodiment. - Referring to
FIG. 1 , adisplay apparatus 100 may be implemented as a digital TV having at least one speaker. However, the present disclosure is not limited thereto and can be implemented in various types of devices having display functions such as a PC, a mobile phone, a tablet PC, a portable multimedia player (PMP), a portable multimedia player (PDA), and a navigation device. Further, it goes without saying that the various embodiments of the present disclosure are applicable to an electronic apparatus including at least one speaker unit such as an audio apparatus, a sound bar, a home theater system, and a room speaker without a display function. Here, the speaker unit has a function of converting electric pulses into sound waves, and can be realized as, for example, a dynamic type according to a principle and a method of converting electric signals into sound waves. However, the present disclosure is not limited thereto, and it may be implemented as an electrostatic type, a dielectric body type, a magnetostrictive type, or the like within the scope of the present disclosure. Hereinafter, it is assumed that thedisplay apparatus 100 is implemented as a digital TV for convenience of description. - The
display apparatus 100 receives various genres of contents according to an embodiment. Here, the contents of various genres refer to kinds of contents such as movies, soap operas, sports, news, and music. The genres of contents may be represented by characteristics, type, category, etc. of contents, but are generally referred to as the genres of contents. - Meanwhile, according to an embodiment of the present disclosure, the
display apparatus 100 acquires rendering type information based on genres of contents and volume level which is currently set in thedisplay apparatus 100, and outputs a sound signal based on the acquired rendering type information. Hereinafter, various embodiments of the present disclosure will be described with reference to the drawings. -
FIG. 2 is a block diagram illustrating a configuration of a display apparatus according to an exemplary embodiment. - According to
FIG. 2 , thedisplay apparatus 100 includes adisplay 110, astorage 120, aprocessor 130, and aspeaker 140. - The
display 110 provides various content screens that can be provided through thedisplay apparatus 100. Here, the content screen includes various contents such as an image, a video, a text, music, etc., an application execution screen including various contents, a graphic user interface (GUI) screen and so on. - Meanwhile, the
display 110 may be implemented as various types of displays such as a liquid crystal display, an organic light-emitting diode, a liquid crystal on silicon (LCoS), a digital light processing (DLP) and the like. Also, thedisplay 110 may be implemented as a transparent display that is made of a transparent material and displays information. Meanwhile, thedisplay 110 may be implemented as a touch screen having a mutual layer structure with the touch pad. In this case, thedisplay 110 may be used as a user interface in addition to an output device. - The
storage 120 stores various data, programs, or applications for driving and controlling thedisplay apparatus 100. - In particular, the
storage 120 stores rendering type information for each content genre according to an embodiment of the present disclosure. Here, the rendering type information is a parameter information for providing various effects to the sound signal of the content. For example, the rendering type information may include information on parameter setting values for emphasizing at least one of a sound transmission power, a reproduction band, a low frequency range, a sense of space, and presence. Here, the sound signal may be a digital sound signal, but is not limited thereto. - According to an embodiment, a plurality of types of rendering information are matched and stored in one genre. For example, the first rendering type information and the second rendering type information may correspond to the sports genre. Here, the first rendering type information may be information used when the volume level set in the
display apparatus 100 exceeds an upper limit (hereinafter referred to as a first level) of a predetermined volume level range. In addition, the second rendering type information may be information used when the volume level set in thedisplay apparatus 100 is less than a lower limit (hereinafter referred to as a second level) of a predetermined volume level range. However, the rendering type information matched by genre is not limited to two, and two or more rendering type information may be matched. In addition, thestorage 120 may store rendering type information corresponding to a sub-genre, as well as rendering-type information per genre. For example, thestorage 120 may store a plurality of rendering type information corresponding to various sub-genres such as horror, melodrama, action, war, and SF in the movie genre. - Here, the volume level set on the display apparatus means a volume level corresponding to the volume set by the user among the volume levels that the
display apparatus 100 can provide. The plurality of volume levels that can be provided in thedisplay apparatus 100 may differ depending on the characteristics of thedisplay apparatus 100, for example, a manufacturer, a type, model, and the like. That is, different volume levels (for example, sound pressure level (SPL)) may be mapped to the same volume (for example, 10) for each characteristic of thedisplay apparatus 100. Hereinafter, for convenience of description, the dBSPL value corresponding to the current volume of thedisplay apparatus 100 is referred to as the volume level currently set in the display apparatus. - The
processor 130 controls the overall operation of thedisplay apparatus 100. Theprocessor 130 may include one or more of a digital signal processor (DSP), a central processing unit (CPU), a controller, an application processor (AP), or a communication processor (CP)), an ARM processor, or may be defined by a corresponding term. - In particular, the
processor 130 obtains the first and second rendering type information corresponding to the genre of the content displayed on thedisplay 110 from thestorage 120. However, the present disclosure is not limited thereto, and it is needless to say that at least one rendering type information corresponding to the genre of the content can be obtained from an external server, metadata regarding a content, and so on. - The
processor 130 according to an embodiment of the present disclosure renders the sound signal of the content based on at least one of the first rendering type information and the second rendering type information corresponding to the genre of the content displayed on thedisplay 110. Here, if the set volume level is within a predetermined volume level range, theprocessor 130 applies a weight according to the currently set volume level to at least one of the first rendering type information and the second rendering type information, to obtain third rendering type information, and render the sound signal based on the third rendering type information. - For example, if the predetermined volume level range is 40dBSPL to 80dBSPL and the set volume level of the
display apparatus 100 is 50dBSPL, the first and second weights may be respectively applied and summed to the first rendering type information and the second rendering type information corresponding to the genre of content, and the third rendering type information can be obtained. - Meanwhile, the predetermined volume level range is not limited to 40 dBSPL to 80 dBSPL, and the predetermined volume level range may be set differently according to a user setting, manufacturer's setting, viewing time, content genre, and the like. As an example, at evening and early morning hours, the lower and upper limits of the volume level range set at 40 dBSPL and 80 dBSPL, respectively, can be adjusted to reflect the human auditory system, which is relatively sensitive at evening and early morning hours. For example, at evening and early morning hours, the predetermined volume level range can be adjusted from 35dBSPL to 75dBSPL.
- In the meantime, the
processor 130, when a volume level which is set in thedisplay apparatus 100 exceeds a predetermined volume level range, performs rendering of a sound signal of a content based on one of first rendering type information and second rendering type information corresponding to the displayed genres of contents. - For example, if the set volume level exceeds 80 dBSPL, the
processor 130 may render the sound signal based on the first rendering type information corresponding to the content genre. The first rendering type information may include a rendering parameter setting value to emphasize the genre characteristic of the corresponding content in the audio signal of the corresponding content. Here, the genre characteristic may mean a sound effect that the user generally expects in the genre of the content. - When a user views content with a relatively high volume, in general, the user may desire to be provided with a sound effect reflecting genre characteristics of the content. Accordingly, in the present disclosure, when the setting volume level of the
display apparatus 100 exceeds the first level, that is, the volume level of the predetermined threshold value, the sound signal is rendered based on the first rendering type information, and the desired effect can be provided at the corresponding volume level. - For example, in the case of sports genre content, the first rendering type information may include a parameter setting value for rendering the sound signal to be emphasized such that the sense of space and the presence are relatively emphasized relative to the voice transmission power. For example, the first rendering type information corresponding to the sports genre includes information on parameter setting values for emphasizing the presence and sense of space relative to the voice transmission power, the reproduction band, and the low frequency range as shown in Table 1 below. In this case, the user can be provided with other sound effects such as a commentary voice due to a high volume setting while receiving a sound signal emphasizing presence and sense of space by a certain level or more.
[Table 1] Voice transmission power Reproduction band Low frequency range Sense of space Presence 3 2 3 4 5 - The
processor 130 according to an exemplary embodiment of the present disclosure renders a sound signal based on the second rendering type information if the set volume level of thedisplay apparatus 100 is less than a lower limit of a predetermined volume level range (for example, 40 dBSPL). Here, the second rendering type information includes a rendering parameter setting value for emphasizing a sound effect that falls below a predetermined level when the setting volume is lowered below a second level in the genre of the content. - If the set volume level of the
display apparatus 100 is less than a lower limit of the predetermined volume level range, it means that the user is viewing the content with a relatively low volume. In this case, some effects from among various effects of the sound signal of the content may not be provided to a user properly. For example, it can be assumed that the user can not recognize 'dialogue', 'narrator's voice' and so on, which correspond to the voice transmission power, due to the low volume of thedisplay apparatus 100. Even in this case, if the sound signal is rendered based on the fixed rendering type information for emphasizing the genre characteristic, at least one of the effects of the sound signal (for example, at least one of the voice transmission power, presence, sense of space, reproduction band, and low frequency range) can be further lowered. - In order to solve such a problem, if the set volume level of the
display apparatus 100 is less than a lower limit of the predetermined volume level range, theprocessor 130 renders a sound signal based on the second rendering type information to emphasize a sound effect which is declined in the corresponding content genre. For example, when the set volume level is less than 40 dBSPL, the sports genre content may be rendered with the sound signal being emphasized relative to the presence and the sense of space. Thus, the user can easily listen to the voice of the narrator even in a small volume. That is, the second rendering type information corresponding to the sports genre includes information on the parameter setting values for emphasizing the voice transmission power relative to the presence, the sense of space, the reproduction band, and the low frequency range as shown in Table 2 below.[Table 2] Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 2 2 2 2 - Hereinbelow, when the currently set volume level of the
display apparatus 100 is within a predetermined volume level range, various exemplary embodiments of acquiring the third rendering type information based on the first rendering type information and the second rendering type information will be described. - The
processor 130 according to an exemplary embodiment of the present disclosure is configured such that if the set volume level of thedisplay apparatus 100 is within a predetermined volume level range, the first and second weights are summed up and applied to each of the first and second rendering type information corresponding to the content and the third rendering type information can be obtained. For example, theprocessor 130, when the set volume level approaches to a lower limit (that is, the second level) than an upper limit (that is, the first level) of the predetermined volume level range, may set the second weight to be greater than the first weight. For example, if the predetermined range is 40 dBSPL to 80 dBSPL and the set volume level of thedisplay apparatus 100 is 50 dBSPL, theprocessor 130 may apply a weight of 0.75 to the second rendering type information and a weight of 0.25 to the first rendering type information. For example, in the case of the sports genre content, if the first and second render type information are added after the weight of 0.25 is applied to the first render type information and the weight of 0.75 is applied to the second render type information, the third rendering type information can be obtained as Table 3 below.[Table 3] Voice transmission power Reproduction band Low frequency range Sense of space Presence 4.5 2 2.25 2.5 2.75 - Also, when the set volume level of the
display apparatus 100 approaches the upper limit (first level) relative to the lower limit (i.e., the second level) of the predetermined volume level range, the first weight may be set to be higher than the second weight. For example, if the predetermined volume level range is 40 dBSPL to 80 dBSPL and the set volume level of thedisplay apparatus 100 is 70 dBSPL, theprocessor 130 may apply a weight of 0.75 to the first rendering type information and a weight of 0.25 to the second rendering type information. That is, the weight can be calculated based on the following Equation 1. - Here, SPLmax indicates a upper limit of the predetermined volume level range, SPLmin indicates the currently set volume level or hearing level which is currently set in the
display apparatus 100, and SPLmin indicates a lower limit in the predetermined volume level. - The
processor 130 according to an exemplary embodiment of the present disclosure which is not part of the claimed invention, may render the sound signal by using a voice emphasis filter (or center sound rendering filter), sound effect emphasis filter (or background sound rendering filter), music emphasis filter and so on, when rendering the sound signal to emphasize at least one of the voice transmission power, reproduction band, low frequency range, sense of space, and presence. In this case, various filters such as a hi-pass filter, a low-pass filter, a band-pass filter, a mask filter, a head-related transfer function (HRTF) filter, and so on, can be used. Here, the sound signal for which theprocessor 130 performs rendering can be a plurality of channel signals (for example, a stereo signal). However, the present disclosure is not limited thereto, and the sound signal may be included in the sound content itself or may be a part of the video content. According to an exemplary embodiment, any signal which can be divided into a center signal and an ambient signal is applicable. - In a case where the voice emphasis filter is to be applied according to an embodiment, when the input sound signal is a stereo signal, the
processor 130 may divide a stereo signal into a center (or a primary, hereinafter referred to as a first sound signal) and an ambient (reverberation, applause, wind, various background sounds) signal (hereinafter referred to as a second sound signal). - The
processor 130, when voice is detected from the first sound signal, may apply the voice emphasis filter to the first sound signal. Accordingly, rendering can be performed so that the first sound signal is comparatively emphasized than the second sound signal and the voice transmission power can be improved. - For example, when the set volume level of the
display apparatus 100 is equal to or lower than the second level, the weight of the voice emphasis filter is increased and the weight of the background sound emphasis filter is reduced because the importance of the dialogue transmission increases. When the volume level is equal to or higher than the first level, the weight of the background sound emphasis filter can be increased and the weight of the voice emphasis filter can be reduced. In addition, it is also possible to emphasize the voice transmission power by increasing the gain of the mid-range voice with respect to the low / high band gain. - In order to emphasize the reproduction band of the sound signal, the
processor 130 according to an embodiment of the present disclosure may divide the sound signal into the bands of low sound, middle sound, and high sound, and enhance the signal components corresponding to the low sound and high sound. In addition, it is possible to enhance the low-frequency range by emphasizing the low-frequency range of the mid / high range. - In addition, the
processor 130 may increase the gain of the sound effect (or background sound) in order to enhance the presence of the signal. For example, theprocessor 130 may apply a sound effect emphasis filter to the second and third sound signals to detect a sound effect in the second sound signal, thereby rendering the second sound signal. Also, a high-frequency range emphasis filter may be applied to the sound signal. Accordingly, thedisplay apparatus 100 can output a sound signal emphasizing the sound effect and the presence. For example, theprocessor 130 may emphasize the presence by rendering the second sound signal including the crowded sound of the audience among the sound signals of the sports genre contents to be emphasized. - Also, the
processor 130 may use at least one of a head-related transfer function (HRTF) filter and a crosstalk cancellation to emphasize the sense of space of a sound signal. For example, the two-channel crosstalk cancellation may render the sound signal output through the L channel speaker based on the transmission path of the sound signal to be transmitted only to the left ear of the user, and the sound signal output through the R channel speaker (Right ear). - Meanwhile, the
processor 130 identifies a predetermined volume level range for each genre. Since the predetermined volume level range may be different for each genre, theprocessor 130 may identify a predetermined volume level range corresponding to the genre of the content. For example, the movie and music genres range from a predetermined volume level of 40 dBSPL to 85 dBSPL, and the sports and news / TV show genres range from a predetermined volume level of 40 dBSPL to 79 dBSPL. Meanwhile, a predetermined volume level range for each content genre is pre-stored in thestorage 120. However, the present disclosure is not limited thereto, and it is needless to say that a predetermined volume level range can be set by setting of an external server, metadata of contents, user and manufacturer. In addition, a predetermined volume level range (for example, a default volume level range) may be set regardless of the genre. As an example, the default volume level range may be 40 dBSPL (sound pressure level) to 80 dBSPL. In addition, the predetermined volume level range may be adjusted according to the viewing time. As an example, the lower and upper limits of a predetermined volume level range may be adjusted to be relatively small at a specific time zone (e.g., evening and early morning hours) when the human listening system becomes sensitive. For example, the volume level range set at 40dBSPL to 80dBSPL can be adjusted to 35dBSPL-75dBSPL at dawn when the human listening system becomes sensitive. - The
processor 130 according to an exemplary embodiment identifies a genre of a content based on information of a genre included in a content and a sound signal. For example, theprocessor 130 may identify a genre of a content based on genre information (for example, metadata) included in a content itself. - In a case where the genre information included in the content does not exist, the
processor 130 can identify the genre of the content based on the sound signal of the content. For example, it is possible to identify a frequency change amount in units of a predetermined section (hereinafter, referred to as a detection section) of a sound signal, and to identify a genre corresponding to the confirmed frequency change amount. Here, the predetermined detection interval unit may be at least one of at least one audio frame unit, at least one audio scene unit, and at least one audio chunk unit. - The
processor 130 according to an embodiment of the present disclosure obtains a weight based on the volume level and the listening level currently set in thedisplay apparatus 100. The third rendering type information is obtained by acquiring the weight using the listening level together with the set volume level, and applying the obtained weight to the first rendering type information and the second rendering type information. Specifically, the listening level is obtained based on at least one of the average viewing distance of the user to thedisplay apparatus 100, the content viewing time, and the size of the sound signal of the content. For example, as the user's average viewing distance to thedisplay apparatus 100 increases, the user's listening level is lower than the volume level currently set in thedisplay apparatus 100, and thus, theprocessor 130 may set the second weight which is applied to the second rendering type information corresponding to the low level as a comparatively greater than the first weight. A detailed description thereof will be given inFIG. 8 . - The
speaker 140 outputs the rendered sound signal. In this case, thespeaker 140 may be implemented as at least one speaker unit. In one example, thespeaker 140 may include a plurality of speakers for multi-channel reproduction. For example, thespeaker 140 may include a plurality of speakers serving as channels for outputting mixed sound signals. In some cases, a speaker for at least one channel may be implemented as a speaker array including a plurality of speaker units for reproducing different frequency bands. -
FIG. 3 is a block diagram to describe a detailed configuration of a display apparatus according to an exemplary embodiment. - Referring to
FIG. 3 , thedisplay apparatus 100 includes adisplay 110, astorage 120, aprocessor 130, aspeaker 140, acontent receiver 150, acommunicator 160, a remote control signal receiver 170, and an inputter 180. The detailed description of the components shown inFIG. 3 will be omitted for the parts overlapping the components shown inFIG. 2 . - The
processor 130 renders a sound signal of a content based on the rendering type information which is stored in thestorage 120. - The
processor 130 according to one embodiment of the present disclosure may include a CPU, a ROM (or non-volatile memory) storing a control program for controlling thedisplay apparatus 100, RAM (or volatile memory) used as a storage area for storing data or corresponding to various operations performed in thedisplay apparatus 100. - The CPU, by accessing the
storage 120, performs booting using the O / S stored in thestorage 120. By using various programs, contents, and data stored in thestorage 120, the CPU performs various operations. - Here, the
storage 120 may be implemented as an internal memory such as a ROM or a RAM included in theprocessor 130, or may be implemented as a memory separate from theprocessor 130. In this case, thestorage 120 may be implemented as a memory embedded in thedisplay apparatus 100 or a memory which can be attached to / detached from thedisplay apparatus 100 depending on the purpose of data storage. For example, data for driving thedisplay apparatus 100 may be stored in a memory embedded in thedisplay apparatus 100, and data for an extension function of thedisplay apparatus 100 may be stored in the memory which can be attached to and detached from thedisplay apparatus 100. The memory embedded in thedisplay apparatus 100 may be implemented in the form of a nonvolatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD), and a memory which can be attached to or detached from thesound output apparatus 100 can be implemented as a memory card (for example, a micro SD card, a USB memory or the like) and an external memory (for example, a USB memory) connectable to a USB port, or the like. - The
content receiver 150 may be implemented as a tuner for receiving broadcast content, but it is not limited thereto and may be implemented as various types of communication modules capable of receiving various external contents such as a Wi-Fi module, a USB module, and an HDMI module. For example, thecontent receiver 150 may include a tuner, a demodulator, an equalizer, a demultiplexer, and the like. Thecontent receiver 150 tunes the broadcast channel under the control of theprocessor 130, receives the broadcast signal desired by the user, demodulates and equalizes the received broadcast signal, and then converts the broadcast signal into a video signal, a sound signal, and additional data, etc. - The demuxed video signal is provided to the
processor 130. Theprocessor 130 performs various image processing such as noise filtering, frame rate conversion, and resolution conversion on the provided video signal, and generates a frame to be output on the screen. - The demuxed sound signal is provided to the
processor 130. In theprocessor 130, various processes such as decoding or amplification of a sound signal, noise filtering, and the like can be performed. That is, theprocessor 130 performs rendering on the sound signal in accordance with various embodiments of the present disclosure. - The
communicator 160 transmits / receive a sound signal. For example, thecommunicator 160 may receive a sound signal through streaming or downloading from an external apparatus (e.g., a source device), an external storage medium (e.g., universal serial bus (USB)), an external server (e.g., a webhard) through communication methods such as an AP-based Wi-Fi, a Bluetooth, a Zigbee, a wired / wireless local area network (LAN), a wide area network (WAN), Ethernet, IEEE (Institute of Electrical and Electronics Engineers) 1394, high definition multimedia interface (HDMI), universal serial bus (USB), mobile high-definition link (MHL), advanced encryption standard (AES) / European broadcasting union (EBU), optical, coaxial, and so on. - The remote control signal receiver (not shown) receives a remote control signal transmitted from a remote controller. The remote control signal receiver 170 may be configured to include a light receiver for receiving infra red (IR) signal, or may be configured to receive a remote control signal by performing communication according to a wireless communication protocol such as a remote controller and Bluetooth or Wi-Fi.
- The inputter (not shown) may be implemented by various buttons provided on the main body of the
display apparatus 100. The user can input various user commands through the inputter 180 such as a turn-on / turn-off command, a channel conversion command, a volume control command, a menu confirmation command and so on. - Meanwhile, the inputter according to an embodiment of the present disclosure controls rendering according to various embodiments of the present disclosure in response to a user's input to an inputter. In one example, if the user input through the inputter corresponds to rendering-on, the
processor 130 may render the sound signal based on the first and second rendering type information. In addition, when the user input corresponds to the rendering-off, it is possible to render the sound signal based on any one of the first and second rendering type information. For example, when the rendering-off command is input, theprocessor 130 may render the sound signal based only on the first rendering type information. - Hereinbelow, the predetermined volume level range by genres of contents and rendering type information will be described in a greater detail.
-
FIG. 4 is a view to describe rendering type information of a sports genre content according to an exemplary embodiment. - As illustrated in
FIG. 4 , the rendering type information of the sports genre is divided into the first rendering type information and the second rendering type information. - The
display apparatus 100 renders a sound signal based on the first rendering type information when the currently set volume level of thedisplay apparatus 100 exceeds an upper limit of a predetermined volume level range. According to the first rendering type information, if performing rendering of the sound signal of the sports genre content, rendering is performed such that the presence is relatively emphasized relative to the remaining voice transmission power, reproduction band, low frequency range, and sense of space. - In addition, the
display apparatus 100 renders an sound signal based on the second rendering type information when the currently set volume level of thedisplay apparatus 100 is less than a lower limit of a predetermined volume level range. According to the second rendering type information, the rendering is performed so that the voice transmission power is relatively emphasized compared to the remaining reproduction band, low frequency, sense of space, and presence. - If the current set volume level of the
display apparatus 100 is within the predetermined volume level range, thedisplay apparatus 100 applies the different weights to the first and second rendering type information to acquire the third rendering type information. Further, it is possible to render the sound signal based on the obtained third rendering type information. For example, as shown inFIG. 4 , the content of the sports genre has a predetermined volume level range of 40 dBSPL to 79 dBSPL, and if the currently set volume level of thedisplay apparatus 100 is 40 dBSPL to 79 dBSPL, thedisplay apparatus 100 applies the first weight to the first rendering type information and the second weight to the second rendering type information to obtain the third rendering type information. Herein, the first weight can be calculated based on Equation 2 as shown below. - Here, SPLest is a currently set volume level or listening level of the
display apparatus 100. -
- Here, SPLest is a currently set volume level of the
display apparatus 100. -
FIGS. 5-7 are views to describe rendering type information and a predetermined volume level range according to contents of various genres. - As illustrated in
FIG. 5 , the rendering type information of a movie genre content is divided into the first rendering type information and the second rendering type information.FIG. 4 below shows the first rendering type information of a movie genre according to an exemplary embodiment of the present disclosure.[Table 4] Voice transmission power Reproduction band Low frequency range Sense of space Presence 4 4 5 5 4 - When the currently set volume level exceeds an upper limit of the predetermined volume level range, the
display apparatus 100 renders a sound signal based on rendering type information as shown in Table 4. - In addition, when the currently set volume level of the
display apparatus 100 is less than a lower limit of a predetermined volume level range, thedisplay apparatus 100 renders a sound signal based on the rendering type information as shown inFIG. 5 .[Table 5] Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 2 4 2 2 - Also, as described above, when the currently set volume level of the
display apparatus 100 is within the predetermined volume level range, thedisplay apparatus 100 applies the first and second weights to the first rendering type information and the second rendering type information to acquire the third rendering type information, and render the sound signal based on the third rendering type information. For example, if the predetermined volume level range of the movie genre is 40 dBSPL to 85 dBSPL, and the currently set volume level of thedisplay apparatus 100 is within 40 dBSPL to 85 dBSPL, then thedisplay apparatus 100 render the sound signal based on the obtained third rendering type information. - As illustrated in
FIG. 6 , the rendering type information of the news / TV show genre is divided into the first rendering type information and the second rendering type information. Tables 6 and 7 below show the first and second rendering type information of the news / TV show genre according to an exemplary embodiment of the present disclosure.[Table 6] Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 4 3 3 3 [Table 7] Voice transmission power Reproduction band Low frequency range Sense of space Presence 5 3 2 2 2 - As described above, when the currently set volume level of the
display apparatus 100 exceeds an upper limit of the predetermined volume level range, thedisplay apparatus 100 renders a sound signal based on the first rendering type information as shown in Table 6. Further, if the currently set volume level of thedisplay apparatus 100 is less than a lower limit of the predetermined volume level range, thedisplay apparatus 100 renders a sound signal based on the second rendering type information as shown in Table 7. - Also, if the currently set volume level of the
display apparatus 100 is within the predetermined volume level range, thedisplay apparatus 100 applies the first and second weights to the first and second rendering type information, respectively, to acquire the third rendering type information, and render the sound signal based on the third rendering type information. For example, if the predetermined volume level range of the news / TV show genre is 40dBSPL to 79dBSPL and the current set volume level of thedisplay apparatus 100 is within 40dBSPL to 79dBSPL, then thedisplay apparatus 100 renders a sound signal based on the third rendering type information. - As illustrated in
FIG. 7 , the first and second rendering type information of a music genre content can be shown inFIGS. 8 and 9 .[Table 8] Voice transmission power Reproduction band Low frequency range Sense of space Presence 4 5 5 3 3 [Table 9] Voice transmission power Reproduction band Low frequency range Sense of space Presence 3 5 5 2 2 - According to an embodiment of the present disclosure, the predetermined volume level range of the music genre is 40 dBSPL to 85 dBSPL, and if the currently set volume level of the
display apparatus 100 is within 40 dBSPL to 85 dBSPL, thedisplay apparatus 100 renders a sound signal based on the acquired third rendering type information. - However, the various embodiments of the present disclosure are not limited to the specific values for the rendering type information and the predetermined volume level range described in
FIGS. 4 to 7 and Tables 1 to 9, and can be set differently according to various purposes of the present disclosure. For example, thedisplay apparatus 100 may have various rendering type information according to a setting of the user or a setting of the manufacturer of thedisplay apparatus 100. It is needless to say that the genre of the contents can be more subdivided and diversified than the above-mentioned example. - Hereinbelow, an exemplary embodiment of calculating a weight based on a listening level of a user will be described in a greater detail.
-
FIG. 8 is a view to describe a listening level of a user according to an exemplary embodiment. - As illustrated in
FIG. 8 , thedisplay apparatus 100 calculates a weight based on the currently set volume level and listening level. - Although the
display apparatus 100 outputs the sound signal according to the volume level currently set in thedisplay apparatus 100, the level of the sound signal recognized by the user's hearing system may be different from the set volume level. - According to the equal loudness contours showing the relationship between the sound pressure level and the frequency of the sound signal, even when the sound signal has the same decibel, the size of the sound perceived by the user differs depending on the frequency of the sound signal. That is, the intensity of the physically measured sound does not match the size of the sound actually perceived through the human auditory system. For example, if you hear 70dB sound at 50h and 70dB at 200 Hz, the human auditory system will perceive 70dB sound at 200 Hz to be louder. The sound of 60dB at 1 kHz and 70dB at 200 Hz will be recognized as the same size.
- The
display apparatus 100 according to an embodiment of the present disclosure determines the loudness based on the frequency of the sound signal and the set volume level of thedisplay apparatus 100. Here, loudness refers to the amount of subjective sound perceived by a human auditory system when a sound is delivered to the human ear. - For example, the user's perceived level (loudness) of the output sound signal may vary depending on the average viewing distance of the user to the
display apparatus 100, the content viewing time, the size of the sound signal, the surrounding noise level, etc. For example, in the case of general live TV programs, the sound signal is constant in size because it is mixed with an average of -23 loudness k-weighted relative to full scale (LKFS), while other game contents and Internet contents have different average sizes of a sound signal depending on the content, and may not be constant within one content. Accordingly, thedisplay apparatus 100 calculates a weight based on the listening level according to the size (e.g., loudness) of the subjective sound perceived by the human auditory system. - The
display apparatus 100 according to an exemplary embodiment of the present disclosure detects the ambient noise level and obtain the third rendering type information by further considering the detected ambient noise level. For example, if the ambient noise level exceeds a predetermined level, the volume level currently set on thedisplay apparatus 100 may be different from the user's perception level. For example, the sound signal can be recognized at a level lower than the set volume level. Accordingly, thedisplay apparatus 100 obtains the third rendering type information by setting the second weight to a value relatively greater than the first weight. However, the present disclosure is not limited thereto, but it is needless to say that the third rendering type information can be obtained by setting the first weight to a relatively greater value than the second weight. - In addition, according to an embodiment of the present disclosure, the
display apparatus 100 acquires the third rendering type information in consideration of the viewing time. For example, in a particular time period (e.g., evening and early morning hours) when the human auditory system is sensitive to sound signals, a first weight that is applied to the first rendering type information and a second weight that is applied to the second rendering type information can be calculated differently from the daytime time zone. For example, in the evening and early morning hours, the first weight may be set to a relatively greater value than the second weight to obtain third render type information. However, the present disclosure is not limited to this, and it is also possible to set the second weight to a relatively large value. -
FIG. 9 is a flowchart to describe a controlling method of a display apparatus according to an exemplary embodiment. - According to the controlling method of the display apparatus in which the first rendering type information and the second rendering type information based on the volume level are stored for each content genre shown in
FIG. 9 , the first rendering type information and the second rendering type information corresponding to the genre of the content are obtained in step S910. - Based on at least one of the first rendering type information and the second rendering type information, the sound signal of the content is rendered in step S920.
- Then, the rendered sound signal is output in step S930.
- In step S920, when the volume level currently set in the
display apparatus 100 is within a predetermined range, the sound signal of a content is rendered based on the third rendering type information which is obtained by applying a weight according to a set volume level to at least one of the first rendering type information and the second rendering type information. - In the rendering step of S920, when the set volume level exceeds an upper limit of the predetermined range, the sound signal is rendered based on the first rendering type information. If the set volume level is less than a lower limit of the predetermined range, it is possible to render the sound signal based on the second rendering type information.
- If the set volume level is within the predetermined range, the first weight is applied to the first rendering type information and the second weight is applied to the second rendering type information to obtain the third rendering type information based on the set volume level, in step S920.
- If it is determined that the set volume level is within a predetermined range, the second weight value is set to a value greater than the first weight value when the set volume level approaches the lower limit relatively to the upper limit of the predetermined range in step S920. If it is determined that the set volume level is within a predetermined range, the first weight can be set to a value greater than the second weight when the set volume level approaches the upper limit relatively to the lower limit of the predetermined range in step S920.
- In addition, the
display apparatus 100 stores information on a predetermined range for each content genre. In the rendering of the S920, the predetermined range corresponding to the genre of the content can be identified based on the stored information. - Also, in the rendering step of S920, a weight can be obtained based on the set volume level and the listening level. Here, the listening level may be obtained based on at least one of the average viewing distance of the user to the display apparatus, the viewing time, the size of the sound signal, and the ambient noise level.
- Meanwhile, each of the first rendering type information and the second rendering type information is a parameter information for adjusting at least one of a voice transmission power, a low frequency range, a reproduction band, a sense of space and presence.
- In addition, the genre information can be included in a content or obtained based on a sound signal.
- The methods according to various exemplary embodiments can be implemented only by software upgrade or hardware upgrade with respect to the existing display apparatus.
- In addition, the aforementioned various exemplary embodiments can be performed at an embedded server in a display apparatus or an external server of a display apparatus.
- Meanwhile, the various embodiments described above can be implemented in a recording medium that can be read by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. According to software implementation, embodiments such as the procedures and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more of the functions and operations described herein.
- Meanwhile, computer instructions for performing the processing operations according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. Computer instructions stored in such non-volatile computer-readable media may cause a particular device to perform processing operations according to various embodiments described above when executed by a processor.
- The non-transitory computer-recordable medium is not a medium configured to temporarily store data such as a register, a cache, or a memory but an apparatus-readable medium configured to semi-permanently store data. Specifically, the above-described various applications or programs may be stored in the non-transitory apparatus-readable medium such as a compact disc (CD), a digital versatile disc (DVD), a hard disc, a Blu-ray disc, a universal serial bus (USB), a memory card, or a read only memory (ROM), and provided therein.
- The foregoing exemplary embodiments and advantages are merely exemplary and are not to be construed as limiting the inventive concept. The exemplary embodiments may be readily applied to other types of device or apparatus. Also, the description of the exemplary embodiments is intended to be illustrative, and not to limit the scope of the inventive concept, and many alternatives, modifications, and variations will be apparent to those skilled in the art.
Claims (13)
- A display apparatus (100) comprising:a speaker (140);a storage (120) configured to store first rendering type information and second rendering type information based on a volume level of each content genre;a display (110); anda processor (130) configured to:render a sound signal of a content displayed on the display (110) based on at least one of the first rendering type information and the second rendering type information corresponding to a genre of the content, andoutput the rendered sound signal through the speaker (140),characterised in thatthe processor (130) is further configured to, based on a set volume level in the display apparatus being within a predetermined range, render the sound signal based on third rendering type information,wherein a weight is obtained by summing up a first weight and a second weight, the first weight and the second weight being according to the set volume level, and the third rendering type information is obtained by applying the weight to each of the first and the second type information.
- The display apparatus of claim 1, wherein the processor (130) is configured to:based on the set volume level exceeding an upper limit of the predetermined range, render the sound signal based on the first rendering type information, andbased on the set volume level being less than a lower limit of the predetermined range, render the sound signal based on the second rendering type information.
- The display apparatus of claim 1,
wherein the processor (130) is configured to:based on the set volume level being within the predetermined range and approaching to a lower limit of the predetermined range relative to an upper limit of the predetermined range, set the second weight to be greater than the first weight, andbased on the set volume level being within the predetermined range and approaching to the upper limit of the predetermined range relative to the lower limit of the predetermined range, set the first weight to be greater than the second weight. - The display apparatus of claim 1, wherein information regarding the predetermined range of each content genres is stored in the storage (120), and
wherein the processor (130) identifies the predetermined range which corresponds to a genre of the content based on the stored information. - The display apparatus of claim 1, wherein the processor (130) obtains the weight based on the set volume level and a hearing level, and
wherein the hearing level is obtained based on at least one of an average viewing distance of a user to the display apparatus, a content viewing time, a size of the sound signal, and an ambient noise level. - The display apparatus of claim 1, wherein each of the first rendering type information and the second rendering type information is parameter information to adjust at least one of a voice transmission power, a low frequency range, a reproduction band, a presence, and a sense of space.
- The display apparatus of claim 1, wherein the genre of the displayed content is included in the content or obtained based on the sound signal.
- A controlling method of a display apparatus in which first rendering type information and second rendering type information based on a volume level of each content genre is stored therein, the method comprising:obtaining the stored first rendering type information and second rendering type information corresponding to a genre of an input content (S910);rendering a sound signal of the content displayed on a display (110) based on at least one of the first rendering type information and the second rendering type information (S920); andoutputting the rendered sound signal (S930),wherein the rendering comprises:
characterised in that
based on a set volume level in the display apparatus being within a predetermined range, rendering the sound signal based on third rendering type information, wherein a weight is obtained by summing up a first weight and a second weight, the first weight and the second weight being according to the set volume level, and the third rendering type information is obtained by applying the weight to each of the first and the second type information. - The method of claim 8, wherein the rendering comprises:based on the set volume level exceeding an upper limit of the predetermined range, rendering the sound signal based on the first rendering type information, andbased on the set volume level being less than a lower limit of the predetermined range, rendering the sound signal based on the second rendering type information.
- The method of claim 8, wherein the rendering comprises, based on the set volume level being within the predetermined range and approaching to a lower limit of the predetermined range relative to an upper limit of the predetermined range, setting the second weight to be greater than the first weight, and based on the set volume level within the predetermined range and approaching to the upper limit of the predetermined range relative to the lower limit of the predetermined range, setting the first weight to be greater than the second weight.
- The method of claim 8, further comprising storing information regarding the predetermined range of each content genres , wherein the rendering further comprises identifying the predetermined range which corresponds to a genre of the content based on the stored information.
- The method of claim 8, wherein the rendering comprises obtaining the weight based on the set volume level and a hearing level, and
wherein the hearing level is obtained based on at least one of an average viewing distance of a user to the display apparatus, a content viewing time, a size of the sound signal, and an ambient noise level. - The method of claim 8, wherein each of the first rendering type information and the second rendering type information is parameter information to adjust at least one of a voice transmission power, a low frequency range, a reproduction band, a presence, and a sense of space.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020170101264A KR102409376B1 (en) | 2017-08-09 | 2017-08-09 | Display apparatus and control method thereof |
PCT/KR2018/008820 WO2019031767A1 (en) | 2017-08-09 | 2018-08-03 | Display apparatus and controlling method thereof |
Publications (3)
Publication Number | Publication Date |
---|---|
EP3610366A1 EP3610366A1 (en) | 2020-02-19 |
EP3610366A4 EP3610366A4 (en) | 2020-04-29 |
EP3610366B1 true EP3610366B1 (en) | 2023-10-04 |
Family
ID=65271815
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18843147.2A Active EP3610366B1 (en) | 2017-08-09 | 2018-08-03 | Display apparatus and controlling method thereof |
Country Status (5)
Country | Link |
---|---|
US (1) | US10318234B2 (en) |
EP (1) | EP3610366B1 (en) |
KR (1) | KR102409376B1 (en) |
CN (1) | CN111095191B (en) |
WO (1) | WO2019031767A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10602300B1 (en) * | 2019-01-27 | 2020-03-24 | Philip Scott Lyren | Switching binaural sound from head movements |
WO2020215204A1 (en) * | 2019-04-23 | 2020-10-29 | 李修球 | Intelligent device and volume control method therefor |
CN113727240B (en) * | 2020-05-26 | 2024-06-14 | 南宁富联富桂精密工业有限公司 | Sound playing adjusting method and portable device |
CN112601170B (en) * | 2020-12-08 | 2021-09-07 | 广州博冠信息科技有限公司 | Sound information processing method and device, computer storage medium and electronic equipment |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20160079577A (en) * | 2014-12-26 | 2016-07-06 | 삼성전자주식회사 | Method and apparatus for reproducing content using metadata |
Family Cites Families (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100808154B1 (en) | 2001-06-23 | 2008-02-29 | 엘지전자 주식회사 | Method for selecting video?audio of digital TV |
JP2006186920A (en) | 2004-12-28 | 2006-07-13 | Pioneer Electronic Corp | Information reproducing apparatus and information reproducing method |
KR100731533B1 (en) * | 2005-12-14 | 2007-06-25 | 주식회사 대우일렉트로닉스 | Method for auto setting video and audio mode of digital television |
CN101501625B (en) * | 2006-03-06 | 2017-08-01 | 雷蛇(美国)有限公司 | Enhanced 3D sound |
KR101260941B1 (en) | 2006-06-14 | 2013-05-06 | 삼성전자주식회사 | Receiving apparatus and method for providing sound effect based on genre information of content in digital multimedia broadcasting system |
JP4306708B2 (en) | 2006-09-25 | 2009-08-05 | ヤマハ株式会社 | Audio signal processing device |
US20080153537A1 (en) * | 2006-12-21 | 2008-06-26 | Charbel Khawand | Dynamically learning a user's response via user-preferred audio settings in response to different noise environments |
JP4851387B2 (en) * | 2007-05-08 | 2012-01-11 | シャープ株式会社 | Sound reproduction apparatus and sound reproduction method |
JP2010021953A (en) * | 2008-07-14 | 2010-01-28 | Sharp Corp | Video/audio output device and television receiver |
JP4844622B2 (en) * | 2008-12-05 | 2011-12-28 | ソニー株式会社 | Volume correction apparatus, volume correction method, volume correction program, electronic device, and audio apparatus |
KR101672210B1 (en) | 2010-01-21 | 2016-11-04 | 엘지전자 주식회사 | Audio system and method of operating the same |
JP5917518B2 (en) | 2010-09-10 | 2016-05-18 | ディーティーエス・インコーポレイテッドDTS,Inc. | Speech signal dynamic correction for perceptual spectral imbalance improvement |
US20120084653A1 (en) * | 2010-10-01 | 2012-04-05 | Sony Corporation | System and method for content rendering control |
US9258665B2 (en) * | 2011-01-14 | 2016-02-09 | Echostar Technologies L.L.C. | Apparatus, systems and methods for controllable sound regions in a media room |
KR20120124136A (en) * | 2011-05-03 | 2012-11-13 | 주식회사 바이오사운드랩 | Regeneration apparatus of multimedia with hearing protection |
CN102999492A (en) * | 2011-09-08 | 2013-03-27 | 百度在线网络技术(北京)有限公司 | Method and device for adjusting volume of media file |
JP5853813B2 (en) * | 2012-03-27 | 2016-02-09 | 船井電機株式会社 | Audio signal output device and audio output system |
US9031244B2 (en) | 2012-06-29 | 2015-05-12 | Sonos, Inc. | Smart audio settings |
JP2014030140A (en) * | 2012-07-31 | 2014-02-13 | Yamaha Corp | Ambient noise estimation device, sound volume adjusting device, ambient noise estimation method, and ambient noise estimation program |
SG11201504368VA (en) * | 2012-12-04 | 2015-07-30 | Samsung Electronics Co Ltd | Audio providing apparatus and audio providing method |
US9571054B2 (en) | 2013-02-28 | 2017-02-14 | Rovi Guides, Inc. | Systems and methods for dynamically adjusting volume based on media content |
CN107093991B (en) | 2013-03-26 | 2020-10-09 | 杜比实验室特许公司 | Loudness normalization method and equipment based on target loudness |
US9565497B2 (en) * | 2013-08-01 | 2017-02-07 | Caavo Inc. | Enhancing audio using a mobile device |
AU2015234454B2 (en) * | 2014-03-24 | 2017-11-02 | Samsung Electronics Co., Ltd. | Method and apparatus for rendering acoustic signal, and computer-readable recording medium |
JP2016066832A (en) | 2014-09-22 | 2016-04-28 | ヤマハ株式会社 | Acoustic control device, server device, and program |
US9837100B2 (en) * | 2015-05-05 | 2017-12-05 | Getgo, Inc. | Ambient sound rendering for online meetings |
KR20170030384A (en) | 2015-09-09 | 2017-03-17 | 삼성전자주식회사 | Apparatus and Method for controlling sound, Apparatus and Method for learning genre recognition model |
KR20170058839A (en) * | 2015-11-19 | 2017-05-29 | 한국전자통신연구원 | A sound object control apparatus and method based on additional visual object |
GB2550877A (en) * | 2016-05-26 | 2017-12-06 | Univ Surrey | Object-based audio rendering |
-
2017
- 2017-08-09 KR KR1020170101264A patent/KR102409376B1/en active IP Right Grant
-
2018
- 2018-07-26 US US16/046,038 patent/US10318234B2/en active Active
- 2018-08-03 EP EP18843147.2A patent/EP3610366B1/en active Active
- 2018-08-03 CN CN201880044195.6A patent/CN111095191B/en active Active
- 2018-08-03 WO PCT/KR2018/008820 patent/WO2019031767A1/en unknown
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20160079577A (en) * | 2014-12-26 | 2016-07-06 | 삼성전자주식회사 | Method and apparatus for reproducing content using metadata |
Also Published As
Publication number | Publication date |
---|---|
CN111095191B (en) | 2024-02-20 |
KR102409376B1 (en) | 2022-06-15 |
EP3610366A1 (en) | 2020-02-19 |
WO2019031767A1 (en) | 2019-02-14 |
KR20190016823A (en) | 2019-02-19 |
CN111095191A (en) | 2020-05-01 |
EP3610366A4 (en) | 2020-04-29 |
US10318234B2 (en) | 2019-06-11 |
US20190050193A1 (en) | 2019-02-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3610366B1 (en) | Display apparatus and controlling method thereof | |
US11727948B2 (en) | Efficient DRC profile transmission | |
CN110036654B (en) | Electronic device and power control method thereof | |
US10848901B2 (en) | Sound output apparatus and method for processing sound signal based on dynamic range corresponding to volume set | |
US20120230525A1 (en) | Audio device and audio system | |
US9847767B2 (en) | Electronic device capable of adjusting an equalizer according to physiological condition of hearing and adjustment method thereof | |
JP2020109968A (en) | Customized audio processing based on user-specific audio information and hardware-specific audio information | |
EP3142384A1 (en) | System and method for enhancing virtual audio height perception | |
US20170118571A1 (en) | Electronic apparatus and sound signal adjustment method thereof | |
GB2550877A (en) | Object-based audio rendering | |
CN110996143B (en) | Digital television signal processing method, television, device and storage medium | |
KR101672210B1 (en) | Audio system and method of operating the same | |
US10992273B2 (en) | Electronic device and operation method thereof | |
KR20200021854A (en) | Electronic apparatus, method for controlling thereof and recording media thereof | |
JP2009283997A (en) | Voice output device, program, and recording medium | |
KR102521415B1 (en) | Video Providing System and Method of Controlling Video Apparatus | |
US20190090057A1 (en) | Audio processing device | |
KR20100031422A (en) | Image processing apparatus and control method thereof | |
WO2023215405A2 (en) | Customized binaural rendering of audio content | |
TW202143219A (en) | Audio control system for sound channel of loudspeaker mainly including a main audio and video processing unit, a main loudspeaker unit, an auxiliary processing unit, an auxiliary loudspeaker unit and a remote-control unit | |
CN113438572A (en) | Sound box and system | |
CN114339541A (en) | Method for adjusting playing sound and playing sound system | |
KR20150079514A (en) | Audio apparatus and method for transmitting audio signal and Audio system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20191111 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20200401 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04N 5/60 20060101ALI20200326BHEP Ipc: H04R 3/00 20060101ALI20200326BHEP Ipc: G06F 3/01 20060101ALI20200326BHEP Ipc: H04R 5/04 20060101ALI20200326BHEP Ipc: G06F 3/16 20060101AFI20200326BHEP Ipc: H04S 1/00 20060101ALI20200326BHEP |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20211013 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20230512 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602018058826 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20231004 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1618424 Country of ref document: AT Kind code of ref document: T Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240105 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240204 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240204 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240105 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240104 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240205 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240104 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231004 |