CN111034215A

CN111034215A - Stereo service device and driving method thereof and computer readable recording medium

Info

Publication number: CN111034215A
Application number: CN201880050835.4A
Authority: CN
Inventors: 金智宪
Original assignee: Digital Audio Co Ltd
Current assignee: Digital Audio Co Ltd
Priority date: 2017-09-22
Filing date: 2018-08-31
Publication date: 2020-04-17
Anticipated expiration: 2038-08-31
Also published as: KR20190034486A; KR102057684B1; KR102070360B1; CN111034215B; KR20190034487A

Abstract

The present invention relates to a stereo service apparatus, a driving method thereof, and a computer-readable recording medium, and a stereo service apparatus according to an embodiment of the present invention may include: a storage unit that matches and stores HRTF data relating to a physical characteristic of a user and sound source environment (3D) data relating to a sound source environment; and a control unit that extracts an HRTF candidate group from the (stored) HRTF data based on a user test result for matching the sound, and sets at least one data having a similarity equal to or greater than a reference value in the extracted HRTF candidate group as personalized data for a specific user.

Description

Stereo service device and driving method thereof and computer readable recording medium

Technical Field

The present invention relates to a stereo service apparatus, a method of driving the same, and a computer readable recording medium, and more particularly, to a stereo service apparatus, a method of driving the same, and a computer readable recording medium, which allow a user to listen to music, etc. through, for example, 3D headphones, etc., in consideration of physical characteristics inherent to the user and an actual sound source environment.

Background

The mono-originated sound technology is now being developed as stereo (2D) technology as if listening in real-life, apart from pure stereo (2D). 3D sound technology has long been used in the field of motion pictures. In the field of computers such as games, it is also used as a tool for improving the sense of heaviness. Is an important element for doubling the reality of three-dimensional information contained in images and videos.

The stereo technology is a technology for allowing a listener who is not located in a sound source generation space to feel the sense of direction, the sense of distance, and the sense of space as in the sound source generation space. If stereo technology is utilized, the listener can get the sensation as if listening in the field. Stereophonic techniques for providing a listener with a three-dimensional spatial and directional sensation have been studied for decades. However, since the 21 st century, word processors have been developed at a high speed, many acoustic devices have been developed, and attention has been paid to stereo technology while the realization thereof has been increased.

Research on the three-dimensional audio technology is continuously being conducted, wherein processing audio signals using an "Individualized head related transfer function (Individualized HRTF)" brings about a research result that most realistic audio can be played. In a conventional audio signal processing method using a head-related transfer function, an impulse response can be obtained by recording an audio signal by placing a microphone in a real human ear or an ear having a model of a human figure (for example, Torso), and if the impulse response is applied to the audio signal, the position of the audio signal in a three-dimensional space can be perceived. Here, the head-related transfer function shows a transfer function generated between the sound source and the ear of the person, which has not only a feature that the function value differs depending on the orientation and height of the sound source, but also a physical property such as the shape/size of the head of the person, the shape of the ear, and the like. I.e. each person has a fixed head-related transfer function.

However, since head-related transfer functions (i.e., non-individualized HRTFs) measured by various models (e.g., dummy heads) have been used in three-dimensional audio signal processing so far, there is a problem in that it is difficult to provide the same three-dimensional sound effects to persons whose physical characteristics are different from each other.

In addition, the conventional multimedia playing system does not have a module that can adapt a head-related transfer function according to the physical signs of the user according to the individual user, and therefore, the three-dimensional audio signal having the optimal sense of presence cannot be provided to the individual user.

Disclosure of Invention

An object of an embodiment of the present invention is to provide a stereo service apparatus that allows a user to listen to music or the like through, for example, 3D headphones or the like, considering the user's inherent physical characteristics and the actual sound source environment, a driving method of the apparatus, and a computer-readable recording medium.

The stereo service apparatus according to an embodiment of the present invention includes: a storage section that matches and stores Head Related Transfer Function (HRTF) data related to physical characteristics of a user and sound source environment (3D) data related to a sound source environment of the user; and a control unit that extracts a HRTF data candidate group related to a user from the stored HRTF data based on the stored sound source environment data matched with a sound source environment test result provided by the user, and sets one data selected from the extracted candidate group as personalized HRTF data of a specific user.

The storage section stores sound source environment data matched with each HRTF data, and each sound source environment data may be associated with a plurality of signals obtained by dividing frequency characteristics and time difference characteristics of an arbitrary signal into a plurality of sections.

The control unit may extract sound source environment data associated with the plurality of signals, which matches the sound source environment test result, as a candidate group.

The control unit may perform a pulse test for learning a time difference (ITD), a sound pressure difference (ILD), and a spectrum queue by an acoustic output device of a user in order to obtain the sound source environment test result.

The control unit may use game application software for issuing a specific impulse sound source to the user via the sound source output device to grasp the sound source position for the impulse test.

The control unit measures the similarity between the HRTF data of the extracted candidate group and the stored HRTF data, and sets a candidate having the largest similarity measurement value as the personalized HRTF data of the user.

The stereo service apparatus may further include: and a communication interface part for providing the set personalized data to a stereo output device of the user when the user sends a request.

The control part can control the communication interface part to apply the set personalized data to convert the audio or video which the user wants to play, thereby providing the streaming media service.

In addition, a method for driving a stereo service apparatus according to an embodiment of the present invention includes, as a method for driving a stereo service apparatus including a storage unit and a control unit, the steps of: matching and storing Head Related Transfer Function (HRTF) data related to the body characteristics of the user and sound source environment (3D) data related to the sound source environment of the user in a storage part; and a control unit for extracting a HRTF data candidate group related to a user from the stored HRTF data based on the stored sound source environment data matched with a sound source environment test result provided by the user, and setting one data selected from the extracted candidate group as specific user-customized HRTF data.

The storage step is used for storing sound source environment data matched with each HRTF data, and each sound source environment data is related to a plurality of signals obtained by dividing the frequency characteristic and the time difference characteristic of any signal according to a plurality of intervals.

The setting step may extract sound source environment data on the plurality of signals matching the sound source environment test result as a candidate group.

The setting step may include the steps of: pulse tests for learning a time difference of tone (ITD), a tone pressure difference (ILD), and a spectral queue are performed through a user's acoustic output device in order to obtain the sound source environment test result.

The setting step may include the steps of: for the impulse test, game application software is used which issues a specific impulse sound source to a user through the sound source output device to grasp the sound source position.

The setting step may measure a similarity between the HRTF data of the extracted candidate group and the stored HRTF data, and set a candidate having a largest similarity measurement value as the personalized HRTF data of the user.

The driving method of the stereo service apparatus may further include the steps of: when the user sends a request, the communication interface part provides the set personalized data to a stereo output device of the user.

The setting step may include the steps of: and controlling the communication interface part so as to convert the audio or video which the user wants to play by applying the set personalized data, thereby providing the streaming media service.

In addition, according to a computer-readable recording medium of an embodiment of the present invention, as a computer-readable recording medium including a program for executing a stereo service method, the stereo service method performs the steps of: matching and storing Head Related Transfer Function (HRTF) data related to physical characteristics of a user and sound source environment (3D) data related to a sound source environment of the user; and extracting a HRTF data candidate group related to the user from the stored HRTF data based on the stored sound source environment data matched with the sound source environment test result provided by the user, and setting one data selected from the extracted candidate group as personalized HRTF data of the specific user.

According to the embodiments of the present invention, not only can a matched stereo sound source reflecting the physical characteristics inherent to the user be provided, but also an acoustic output of an environment similar to the actual sound source environment can be realized, and therefore even users having different physical characteristics can enjoy the same three-dimensional acoustic effect by their own stereo headphones or the like.

In addition, even if the user does not purchase a product having a module additionally provided on the product, such as a stereo headphone, in order to enjoy the sound effect, the user can use the optimum sound service by simply providing application software on his/her own sound output apparatus.

Drawings

Fig. 1 is a diagram illustrating a stereo service system according to an embodiment of the present invention.

Fig. 2 is a block diagram illustrating a configuration of the stereo service apparatus of fig. 1.

Fig. 3 is a block diagram illustrating another configuration of the stereo service apparatus of fig. 1.

Fig. 4 and 5 are diagrams for explaining a stereo sound system that changes according to frequency characteristics.

FIG. 6 is a graph showing frequency characteristics of 0 to 30 degree angle differences.

Fig. 7 is a graph showing the results of the arithmetic processing performed with intermediate variation values of 5 degrees, 15 degrees, 20 degrees, and 25 degrees.

Fig. 8 is a graph showing a sharp frequency response change.

Fig. 9 is a graph showing the pulse response characteristics of the actual auditory change by the 1/3 octave smoothing process.

Fig. 10 is a diagram for explaining directivity and spatiality under the condition of natural reflected sound.

Fig. 11 is a diagram for explaining ITD matching.

Fig. 12 is a diagram for explaining ILD matching.

Fig. 13 is a diagram for explaining spectrum alignment matching.

Fig. 14 is a diagram for explaining a stereo service process according to an embodiment of the present invention.

Fig. 15 is a flowchart illustrating a driving procedure of the stereo service apparatus according to an embodiment of the present invention.

Description of the reference symbols

100: stereo output device 110: communication network

120. 120': stereo service apparatus 200: stereo personalized processing part

210. 330: the storage unit 300: communication interface

310: the control unit 320: stereo personalized execution part

Detailed Description

The stereo service device includes: a storage section that matches and stores Head Related Transfer Function (HRTF) data related to physical characteristics of a user and sound source environment (3D) data related to a sound source environment of the user; and a control unit that extracts a HRTF data candidate group related to a user from the stored HRTF data based on the stored sound source environment data matched with a sound source environment test result provided by the user, and sets one data selected from the extracted candidate group as personalized HRTF data of a specific user.

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

As shown in fig. 1, a stereo service system 90 according to an embodiment of the present invention includes a stereo output apparatus 100, a communication network 110, and a part or all of a stereo service apparatus 120.

Here, "include a part or all" means that the stereo output apparatus 100 itself has a module (e.g., H/W, S/W) for providing the service of the present invention, and is operated as a stand alone (stand alone) or configured to omit the communication network 110, so that the stereo output apparatus 100 and the stereo service apparatus 120 perform direct (e.g., P2P) communication, or further, a part of the components like the stereo service apparatus 120 are incorporated in a network apparatus (e.g., an AP, a switching apparatus, etc.) in the communication network 110, and the like, and the description is made in all of the included forms in order to sufficiently understand the present invention.

The stereo output device 100 includes various devices that output only audio or audio and video together, such as speakers, earphones, headphones, an MP3 Player, a PMP (Portable Multimedia Player), a mobile phone (e.g., a smart phone), a DMB Player, a smart tv, and a home theater. In embodiments of the invention, 3D headphones may be assumed.

The stereo output device 100 may include a program or application software capable of outputting a sound that has been personalized to a specific user when a product is out of stock. Accordingly, the user can set the optimal sound conditions for himself by running the assumed corresponding application software of the stereo output apparatus 100. For this reason, the user sets acoustic conditions specific to the user himself/herself, taking into account the actual sound source environment in which the user mainly moves, while reflecting the body characteristics inherent to the user, such as a head related transfer function (hereinafter, referred to as "HRTF"). The sound conditions may be used when the sound source of the song or the like that the user wants to run changes.

Needless to say, the stereo output apparatus 100 may be connected to the stereo service apparatus 120 of fig. 1 via a stereo playback apparatus, i.e., a terminal apparatus such as a smartphone, and may perform an operation for setting the sound conditions as described above. Further, after obtaining and storing the program or data related to the set conditions in the stereo output apparatus 100, it is possible to listen to the audio running using the stored data in the optimum environment. Here, the so-called "optimal environment" includes at least an environment created by individualized HRTF data. Of course, as for the process, the stereo output apparatus 100 provides an audio file desired by the user to the stereo service apparatus 120, or the stereo service apparatus 120 runs the corresponding audio file to obtain a streaming service, whichever way.

As described above, since the stereo output device 100 and the stereo service device 120 can be linked in a plurality of forms, the embodiment of the present invention is not particularly limited to any one form. However, in the case of acquiring a streaming service, since the service may be not smooth when a load of the communication network 110 is generated, it is preferable to reflect an optimal sound condition and operate it as much as possible after storing a specific audio file (for example, a music file) in the stereo output apparatus 100. The detailed examples will be described later.

The communication network 110 includes both a wired network and a wireless network. For example, wired internet and wireless internet may be used or linked as the communication network 110. Here, the wired network includes an internet network such as a cable network or a public telephone network (PSTN), and the wireless communication network includes a CDMA, WCDMA, GSM, EPC (Evolved Packet Core), LTE (Long term evolution), wireless broadband (Wibro) network, and the like. Of course, the communication network 110 according to the embodiment of the present invention is not limited to this, and may be used, for example, in a cloud disk computer network in a cloud disk computer environment, a 5G network, or the like as an access network of a new-generation mobile communication system to be realized in the future. For example, if the communication network 110 is a wired communication network, the communication network may be connected to an exchange of a telephone Station in the communication network 110, or the like, but in the case of a wireless communication network, the communication network may be connected to an SGSN or a GGSN (Gateway GPRS Support Node) used by a communication company to process data, or may be connected to various relays such as a BTS (Base Station Transmission network), a NodeB, and an e-NodeB to process data.

The communication network 110 includes an Access Point (AP). The access point comprises a small base station like a femto base station or pico base station mostly located in a building. Here, femto or pico base stations are classified according to the number of stereoscopic output devices 100 that can be provided at the maximum in the classification of small base stations. Of course, the access point includes a near field communication module for performing near field communication such as the stereo output device 100 and the internet of things and WI-FI. The access point may utilize TCP/IP or RTSP (Real-Time Streaming Protocol) for wireless communication. Here, the short-range communication can be performed by various standards such as bluetooth, internet of things, infrared ray (IrDA), UHF (Ultra High Frequency) and VHF (Very High Frequency) RF (Radio Frequency) and Ultra-wideband communication (UWB), in addition to WI-FI. Thus, the access point extracts the location of the packet and stores the highest communication path for the extracted location, along which the packet may be delivered to the next device, such as the stereo service device 120. The access points may share multiple lines in a general network environment, and may include routers, repeaters, and the like, for example.

The stereo service apparatus 120 provides a personalized stereo service to the user of the stereo output apparatus 100. Here, the "personalized stereo service" is to provide stereo on the basis of a setting value most similar to the physical characteristics of the user and the actual sound source environment specific to each user. More precisely, the setting values may be set so as to reflect the physical characteristics of the user selected in consideration of the actual sound source environment. Accordingly, for example, in the case where the stereo service apparatus 120 is a server providing a music service, audio data is processed on the basis of the corresponding setting values and supplied to the stereo output apparatus 100. According to an embodiment of the present invention, the stereo service apparatus 120 may change an internal element such as a sound field of the audio signal itself based on a corresponding setting value (e.g., personalized HRTF data), or may change an external element such as hardware (e.g., an equalizer, etc.) outputting the audio signal, whichever is.

It is further observed that the stereo service apparatus 120 according to the embodiment of the present invention can be connected to and operate with the stereo output apparatus 100 in various forms. For example, when requesting to download application software in order to utilize a service according to an embodiment of the present invention, the stereo output apparatus 100 may provide the corresponding application software. Here, the application software helps to select sample data that best matches the physical characteristics (or sound source environment) of the user according to the input information (e.g., test results) of the user among the stored matching sample data (e.g., general KRTF data of about 100). For this, for example, it is possible to find and use the most similar value by matching 100 sample data with a game APP in which a user listens to a specific impulse sound source and grasps the position of the sound source, finding an expected HRTF in the process, measuring the approximation with 100 models. As a result, the selected personalized data is finally provided to the user after the sound source is adjusted (or modified).

Of course, the stereo service apparatus 120 may perform the above operation after the stereo output apparatus 100 is connected to the stereo service apparatus 120 by the operation of the application software. In other words, the stereo service device 120 selects HRTFs personalized to a specific user in sample data based on matching information received through the stereo output device 100 such as a smart phone by means of an interface with the user, and provides a personalized stereo service based on the HRTFs.

For example, the stereo service apparatus 120 provides the stereo output apparatus 100 with the corresponding selected data, whereby the stereo output apparatus 100 can output audio by modifying, e.g., adjusting (scaling), the audio signal according to the corresponding data when running a music file stored internally or provided from the outside. In the case where the stereo service apparatus 120 is an apparatus for providing music services, when a specific music file is provided, the stereo output apparatus 100 may be operated by converting the music file based on data of a specific user and providing the converted music file in the form of a file. Furthermore, the stereo service device 120 may provide a service to the stereo output device 100 in the form of streaming media according to the converted audio with personalized HRTF data of a specific user.

As described above, the stereo service apparatus 120 according to the embodiment of the present invention can operate in various forms and stereo output apparatuses 100, and of course, the operation can be performed together with all the operations described above. This is determined by the intention of the system designer at any rate, and therefore, the embodiment of the present invention is not particularly limited to a certain form.

Further, the stereo service device 120 includes a DB (Data Base) 120 a. The DB120a of the stereo service device 120 stores not only sample data for the HRTF data personalized for the user setting, but also personalized HRTF data personalized for the user setting using the sample data. Of course, the HRTF data may be matched to sound source environment data that enables knowing the actual sound source environment of a specific user and stored. Or, when the HRTF data are stored separately, specific individualized and specific HRTF data can be found, and then sound source environment data specific to a specific individual can be found and combined with each other, however.

As shown in fig. 2, the stereo service 120 according to the first embodiment of the present invention includes a stereo personalization processing part 200 and a part or all of a storage part 210, and the meaning of "including a part or all" herein is the same as the above.

The stereo personalization processing unit 200 sets personalized sound data according to a user. Here, the personalized sound data includes HRTF data related to physical characteristics of each user, and may further include sound source environment data related to an actual sound source environment of each user matched to the HFTR data.

The stereo personalization processing section 200 searches for data suitable for a specific user from a plurality of sample data stored in the storage section 210 by using an interface with the user (for example, touch input, voice input, or the like) as input information, and sets the found corresponding data as data specific to the individual user. When providing audio service, the user uses the corresponding setting data to change the audio.

Of course, the stereo personalization processing unit 200 provides the audio output device 100 of fig. 1 with data suitable for a specific user person as described above, and the audio output device 100 can use the corresponding data, and thus the embodiment of the present invention is not particularly limited to any one of the embodiments.

The storage unit 210 may store various data or information processed by the stereo personalization processing unit 200. Here, the storage includes temporary storage. For example, the provision and storage of sample data for the personalization process may be obtained from the DB (120a) of fig. 1. Also, the stereo personalization processing unit 200 may provide corresponding sample data when requested.

In addition, the storage unit 210 may match and store the personalized HRTF data and sound source environment data with the user identification information according to the user using the provided sample data. In addition, the data stored in the stereo personalization processing unit 200 may be provided and stored in the DB120a of fig. 1 according to a request.

The stereo personalization processing unit 200 and the storage unit 210 in fig. 2 are not much different from the stereo service apparatus 120 in fig. 1 in content, and are replaced with the above-described contents.

As shown in fig. 3, the stereo service apparatus 120' according to another embodiment of the present invention includes a communication interface part 300, a control part 310, a stereo personalization performing part 320, and a part or all of a storage part 330.

Here, "include a part or all" means that a part of the components of the analog storage section 330 may be omitted and the components may be combined with other components of the analog control section 310, and the description will be made in a form of all inclusion in order to facilitate a sufficient understanding of the invention.

The communication interface part 300 may provide application software for the stereo service according to the embodiment of the present invention according to a user request. In addition, the communication interface 300 functions to connect services when the audio output device 100 such as a smartphone connected to a 3D headphone runs application software, for example. In this process, the communication interface 300 receives the user identification information ID and transmits the user identification information ID to the control unit 310.

The communication interface unit 300 receives input information of a user, which is an HRTF personalized according to a user selection and sound source environment data related to a sound source environment in which the user is located, and transfers the input information to the control unit 310.

Furthermore, the communication interface 300 may provide the HRTF data and sound source environment data personalized to the audio output device 100 according to the user, or may provide the audio sound source reflecting the dependent data in a streaming form or in a file form. For example, a specific song may be transformed and provided in a form that conforms to the physical characteristics and actual environment of the user.

The control unit 310 controls all operations of the communication interface unit 300, the stereo personalization execution unit 320, and the storage unit 330 that constitute the stereo service apparatus 120'. For example, the control part 310 may perform an operation of searching for specific user-customized data matching the input information by operating the stereo personalization performing part 320 according to the user's request and the user input information received through the communication interface part 300. More specifically, the control unit 310 may cause the program in the stereo personalization execution unit 320 to run, and may provide the input information provided by the communication interface unit 300 to the stereo personalization execution unit 320.

Further, the control section 310 may control the communication interface section 300 so that HRTF data (and sound source environment data) set for each user is obtained from the stereo personalization execution section 320, temporarily stored in the storage section 330, and then stored in the DB120a of fig. 1. It is of course preferable at this time to store the matched user identification information together.

The stereo personalized execution part 320 performs an operation of setting HRTF data and sound source environment data personalized for a specific user as fully described above, and more specifically, for example, may also perform an operation of searching for personalized HRTF data from the sound source environment data and converting audio based on the set data. Actually, the audio conversion may include, as the correction operation, an operation of setting data to various characteristics according to the frequency or time of converting the basic audio.

The contents related to the storage unit 330 are not much different from those of the storage unit 210 of fig. 2, and are replaced with the above-described contents.

The details of the communication interface 300, the control unit 310, the stereo personalization execution unit 320, and the storage unit 330 in fig. 3 are not much different from those of the stereo service apparatus 120 in fig. 1, and therefore, the above-described contents are used instead.

In addition, the control section 310 of fig. 3 may include a CPU (Central processing unit) and a memory as still another embodiment. Here, the CPU may include a control circuit and an arithmetic circuit (ALU), an analysis section, and a registration storage. The control circuit is related to control operation, the arithmetic circuit can carry out various data arithmetic operations, and the analysis part can help the control circuit to analyze the command language of the machine language. The registration store is associated with the data store. Most importantly, the memory may include a Random Access Memory (RAM), and the control unit 310 may rapidly increase the operation speed by storing the program stored in the stereo personalization execution unit 320 in the internal memory and then operating the same when the stereo service apparatus 120' is initially driven.

Fig. 4 and 5 are diagrams for explaining the stereo sound which changes according to the frequency characteristics, and fig. 6 is a diagram showing the frequency characteristics with an angular difference of 0 to 30 degrees. Fig. 7 is a graph showing the results of the arithmetic processing using intermediate change values of 5 degrees, 15 degrees, 20 degrees, and 25 degrees, fig. 8 is a graph showing a rapid frequency response change, fig. 9 is a graph showing the pulse response characteristics of the actual auditory change by the 1/3 octave smoothing processing, and fig. 10 is a graph for explaining the directivity and the spatiality under the natural reflex sound condition.

Fig. 4 to 10 are views pertaining to a description of a 3D filtering (e.g., α filtering) operation for generating the same sound source environment data as the embodiment of the present invention, the sound source environment data may be stored after being matched with HRTF data, although it may be stored in advance, and it is preferable that the sound source environment data is stored after being matched with the respective HRTF data, according to the embodiment of the present invention.

The α filtering according to the embodiment of the present invention is divided into a frequency characteristic change (or distortion) and a time difference characteristic change, the frequency characteristic change is performed in accordance with a predetermined octave band standard after reducing the peak frequency band of a specific frequency by a predetermined dB, and the time difference characteristic change is performed in the form of an original sound (or fundamental sound) + a predetermined time interval +1 reflected sound + a predetermined time interval +2 reflected sounds + a predetermined time interval +3 reflected sounds.

The reason why the frequency characteristic is changed in the embodiment of the present invention is as follows. Although a complete personalized HRTF should have thousands of directional function values, adapting it to an actual sound source has real difficulties. Therefore, as shown in fig. 4, for example, the sum of the sum and the sample data is made uniform for a 30-degree angle sound source corresponding to 30 channels, and the median is filtered at the middle of each direction and each point (for example, 5-degree unit). Fig. 4 (a) shows the upper 9 channels, (b) shows the middle 12 channels, and (c) shows the lower 9 channels and 2 LFE (Low Frequency Effect) channels. In fact, as shown in fig. 5, one recognizes the 3-dimensional sound source at a more detailed angle.

In addition, in order to make the frequency characteristics vary, power level addition and subtraction may be used in the embodiment of the present invention. Fig. 6 shows frequency characteristics of the angular difference of 0 to 30 degrees, and fig. 7 shows a graph in which the arithmetic processing is performed with intermediate variation values of 5 degrees, 15 degrees, 20 degrees, and 25 degrees.

The sharp frequency change is different from the actual human auditory characteristic, and therefore, in the embodiment of the present invention, in order to obtain a frequency change value similar to the human auditory characteristic, the sharp change value is smoothed according to the 1/3 octave band standard. Fig. 8 shows the impulse response characteristic of a sharp change, and fig. 9 shows the impulse response characteristic of an actual auditory change by 1/3 octave smoothing processing.

α filters characteristic changes that are observed in association with time difference characteristic changes and require real-time conversion of sample data having time differences in a precision angle of 5 degrees in accordance with a standard of 30 degrees, and in this case, according to an embodiment of the present invention, the time difference characteristic changes can be made in such a manner that a change value according to each direction is applied in 1 sample unit in EX-3D Binaural (binary renderer) Software (SW), whereby the sound source can be naturally moved when positioned in real time in accordance with the longitude and latitude standard, and the clarity can be maintained.

Further observing the time difference characteristic variation, human beings hear sounds in a space where natural reflected sounds exist, rather than listening to sounds in a anechoic chamber, and can naturally recognize the directivity and the spatial feeling under the natural reflected sound conditions. Therefore, by adding natural initial reflected sound reflected in space to the HRTF to form a head-related impulse response (HRIR), three-dimensional spatial audio can be improved. Fig. 10 shows the formation of HRIRs from reflected tones.

α the temporal characteristic variation leads to natural angle variation and frequency characteristic variation when matching the HRTF of an individual, thereby making the quality and panning accuracy of the sound source improved, furthermore, the temporal characteristic variation mixes HRTF and BRIR (Binaural Room Impulse Response) for realizing natural three-dimensional spatial audio, thereby making the sound source varied to be played and propagated after being similar to the actual human auditory characteristic, in any case.

Fig. 11 is a diagram for explaining ITD matching, fig. 12 is a diagram for explaining ILD matching, and fig. 13 is a diagram for explaining spectral alignment matching.

Referring to fig. 11 to 13, in an embodiment of the present invention, operations including ITD matching, ILD matching, and spectral queue matching may be performed for personalized filtering. For matching, by impulse test, for example, the best data is found out among 100 model data, that is, the expected HRTF is found out therefrom and the similarity to 100 models is measured, and the most similar value is found out.

The ITD matching is performed to identify a direction from a time difference of a sound source reaching both (side) ears of a human by analyzing the time difference, and thus, a sound source time difference (ITD) is found. Therefore, since the time difference between the sound source and the ears is generated according to the size of the human head for ITD matching, the difference of 0.01ms to 0.05ms is minimum for the sound source at the left and right 30 degrees angle, which is important for the phase separation of the front sound, and for time difference matching, matching is performed in units of one sample (0.002ms) from 6 samples to 18 samples with 48000 samples as a reference for correcting the digital delay. For the matching analysis, the distinctive pulse sound source is listened to by taking one sample as a unit, and the sound source which is most vivid in listening is selected. As a result, ITD matching matches the phases of the sound sources, and makes the sound phase intelligibility and transient (initial) response clear, thereby clarifying the sound phase transition of the sound sources in the three-dimensional space. In the conventional ITD, when the sound is not matched with each user, the sound is disturbed, thereby generating a marginal sound (metallic sound) and transmitting an unpleasant sound. Fig. 11 illustrates signals provided to a user for ITD matching according to an embodiment of the present invention.

In addition, the ILD matching aims at finding a difference in sound pressure Intensity (ILD) because the size of sound reaching both ears is one of the important clues for three-dimensional direction recognition. The sound reaching both ears has a size of at least 20dB to at most 30dB in the front left and right 30-degree angle. An Impulse Response (IR) sound source generating a difference is divided into 10 steps to be listened to by a listener through impulse sound (source), and the direction of the sound source is recognized, so that matching is performed with respect to responses close to 30 degrees on the left and right. By matching the ILD to enable prediction of the head size and emitted sound of an individual, the accuracy of audio clarity and direction identification may be improved by adapting the HRTF optimized for the individual. Figure 12 illustrates signals provided to a user for ILD matching according to an embodiment of the present invention.

Further, the spectral alignment matching aims at finding a spectral alignment because the inherent frequency response of each angle is different for positions where the geometry of ITDs and ILDs is not distinguished, that is, the front, rear, top, bottom, and 360-degree directions in the front angle, and the front, rear, top, bottom, and 360-degree directions are used for identifying the position of the sound source. Listening to 10 pulse sound sources with different frequency characteristics, identifying relevant angles in front, back, above and below, and listening to the pulse sound sources, wherein the person is assigned with the highest accuracy to match the spectrum queue. Since the HRTF using the conventional analog head does not match the spectral alignment of the individual listener, it is difficult to recognize the front phase, the upper phase, the rear phase, and the lower phase. FIG. 13 illustrates signals provided to a user for spectral queue matching according to an embodiment of the present invention.

According to an embodiment of the present invention, the ITD, ILD and spectral queue search for personalized sample data of a specific user by letting the user listen to a specific impulse sound source (or test sound source) and grasp the game APP of the sound source, for example, in a manner matching 100 sample data, and then provide the sound source to be played by each user based on the sample data.

For convenience of description, referring to fig. 1 and 14 together, the media player application (MediaPlayer App)1400 and the Native Runtime (Native Runtime)1410 of fig. 14 correspond to, for example, an operation unit of the audio output device 100 of fig. 1, and the 3D Engine unit (EX-3D Engine)1420 and the 3D Server (EX-3D Server)1430 of fig. 14 may correspond to the stereo Server 120 and the DB120a (or a third party Server) of fig. 1.

As viewed from fig. 14, the 3D engine part 1420 may receive user information by means of an interface with a user and store the same in the 3D server 1430(S1400, S1401).

In addition, the 3D engine part 1420 may perform the following operations: input information (e.g., ITD, ILD, spectral queue information) using a test sound source is received by means of an interface with a user and HRTF data personalized for a specific user is set using the same (S1402, S1403, S1404). More specifically, the 3D engine section 1402 may decide the user HRTF by matching the user identification information (S1403). Of course, in this process, generalized 100 HRTF sample data may be used. In order to form data related to the sound source environment, for example, the HRIR section 1423b adds a natural initial reflected sound reflected in space to the HRTF to form HRIR, thereby forming a value for enhancing a three-dimensional spatial audio (S1404), and forms a time difference of the sound source in the sound phase externalizing section 1423D (in combination with the HRTF of the user) by using a set value to allow the user to listen to the HRTF, thereby making it possible to transmit data personalized for the specific user.

Through the process, if the selection process of the HRTF data of the specific user is finished, the 3D engine portion 1420 provides the specific user with personalized HRTF data changed according to output characteristics of the audio or video including the audio, i.e., the converted image, when the user desires to play the audio (e.g., music).

In fig. 14, for example, when the audio output device 100 of fig. 1 plays an audio file acquired through various paths (e.g., a media source 1401, an external receiver 1403, etc.), the decoder 1405 decodes the compressed file and plays the file, and at this time, the 3D engine portion 1420 changes the audio to be played based on the HRTF data personalized for a specific user and reflects the physical characteristics of the user, and at this time, plays the audio in a state similar to the sound source environment where the user is located, thereby maximizing the effect of listening to the music.

For convenience of explanation, referring to fig. 1 and 15 together, the stereo service apparatus 120 according to the embodiment of the present invention matches Head Related Transfer Function (HRTF) data related to physical characteristics of a user and sound source environment data related to a sound source environment and stores S1500.

Further, the stereo service device 120 extracts an HRTF data candidate group related to the user from the stored HRTF data based on (already) stored sound source environment data matching the sound source environment test result provided by the user, and sets one data selected from the extracted candidate group as specific user-customized HRTF data S1510.

For example, the stereo service device 120 matches 100 sample data by the actual environment where the user is located, i.e., by the game APP that lets the user listen to a specific impulse sound source and grasp the sound source position in order to know the HRTF of the user. In other words, in a state where HRTF data and sound source environment data are matched and stored, input information of a user inputted through a test using a pulsed sound source is extracted as a specific user HRTF candidate group based on sound source environment data matched through the corresponding input information, and HRTFs having the highest similarity among the extracted candidate groups, that is, a reference value or more are used as HRTF data of the user. Of course, as in the embodiment of the present invention, the extracted candidate group may be compared with the stored HRTF data again, and the similarity may be measured and the measurement result may be used.

For example, assume that 5 candidate groups are selected at a time. In this case, a method of comparing the HRTF data with a preset reference value is employed to find the HRTF data having the highest similarity among the candidate groups. Alternatively, a method of comparing between candidate groups and sequentially excluding specific HRTF data may be used. As such, in the embodiments of the present invention, the method for finally finding the HRTF data corresponding to the specific user may be various, and therefore, the embodiments of the present invention are not particularly limited to a certain manner.

Further, the description has been given of the operation in which all the components constituting the embodiment of the present invention are combined into one or combined, and the present invention is not limited to the embodiment. In other words, all the components may be selectively combined with one or more components and operated within the scope of the object of the present invention. Further, all the components may be implemented by one piece of independent hardware, but some or all of the components may be selectively combined to be implemented by a computer program having program modules that execute some or all of the functions combined in one or more pieces of hardware. Codes and code segments constituting the computer program may be easily derived by those skilled in the art of the present invention. The computer program is stored in a non-transitory computer readable recording medium (non-transitory computer readable media) readable by a computer, and is read and executed by the computer, thereby implementing the embodiment of the present invention.

Here, the non-transitory readable recording medium refers to a medium that stores data in a semi-permanent manner, and is readable by a machine, and is not a medium that stores data for a short time like a register, a buffer, a memory, or the like. Specifically, the program may be stored in a non-transitory readable recording medium such as a CD, a DVD, a hard disk, a blu-ray disc, a USB, a memory card, a ROM, or the like, and provided.

Although the preferred embodiments of the present invention have been illustrated and described, the present invention is not limited to the specific embodiments described above, and various modifications can be made by those skilled in the art to which the present invention pertains without departing from the gist of the present invention claimed in the claims.

Industrial applicability

While the present invention has been described with reference to the embodiments shown in the drawings, the embodiments are illustrative, and it is to be understood that various changes and equivalents may be made therein by those having ordinary skill in the art. Therefore, the true technical scope of the present invention needs to be determined by the technical idea of the scope of the appended claims.

Claims

1. A stereo service apparatus, comprising:

a storage section that matches and stores Head Related Transfer Function (HRTF) data related to physical characteristics of a user and sound source environment (3D) data related to a sound source environment of the user; and

and a control unit that extracts a HRTF data candidate group related to a user from the stored HRTF data based on the stored sound source environment data matched with a sound source environment test result provided by the user, and sets one data selected from the extracted candidate group as personalized HRTF data of a specific user.

2. The stereo service apparatus according to claim 1,

the storage unit is configured to store sound source environment data that matches each of the HRTF data, and each of the sound source environment data is associated with a plurality of signals obtained by dividing frequency characteristics and time difference characteristics of an arbitrary signal into a plurality of sections.

3. The stereo service apparatus according to claim 2, wherein,

the control unit extracts sound source environment data associated with the plurality of signals, which match the sound source environment test result, as a candidate group.

4. The stereo service apparatus according to claim 1,

the control unit performs a pulse test for learning a time difference (ITD), a sound pressure difference (ILD), and a spectrum queue by an acoustic output device of a user in order to obtain the sound source environment test result.

5. The stereo service apparatus according to claim 4,

the control unit uses game application software for issuing a specific impulse sound source to the user via the sound source output device for the impulse test, thereby grasping the sound source position.

6. The stereo service apparatus according to claim 1,

7. The stereo service apparatus according to claim 1, further comprising:

and a communication interface part for providing the set personalized data to a stereo output device of the user when the user sends a request.

8. The stereo service apparatus according to claim 7,

the control part controls the communication interface part so as to apply the set personalized data to convert the audio or video which the user wants to play, thereby providing the streaming media service.

9. A method for driving a stereo service apparatus including a storage unit and a control unit, the method comprising the steps of:

matching and storing Head Related Transfer Function (HRTF) data related to the body characteristics of the user and sound source environment (3D) data related to the sound source environment of the user in a storage part; and

the control unit extracts a HRTF data candidate group related to a user from the stored HRTF data based on the stored sound source environment data matched with a sound source environment test result provided by the user, and sets one data selected from the extracted candidate group as personalized HRTF data of a specific user.

10. The driving method of the stereo service apparatus according to claim 9, wherein,

11. The driving method of the stereo service apparatus according to claim 10, wherein,

the setting step extracts sound source environment data associated with the plurality of signals, which match the sound source environment test result, as a candidate group.

12. The driving method of the stereo service apparatus according to claim 9, wherein the setting step comprises the steps of:

pulse tests for learning a time difference of tone (ITD), a tone pressure difference (ILD), and a spectral queue are performed through a user's acoustic output device in order to obtain the sound source environment test result.

13. The driving method of the stereo service apparatus according to claim 12, wherein the setting step comprises the steps of:

for the impulse test, game application software is used which issues a specific impulse sound source to a user through the sound source output device to grasp the sound source position.

14. The driving method of the stereo service apparatus according to claim 9, wherein,

the setting step measures the similarity between the HRTF data of the extracted candidate group and the stored HRTF data, and sets a candidate with the largest similarity measurement value as the personalized HRTF data of the user.

15. The driving method of the stereo service apparatus according to claim 9, further comprising the steps of:

when the user sends a request, the communication interface part provides the set personalized data to a stereo output device of the user.

16. The driving method of the stereo service apparatus according to claim 15, wherein the setting step comprises the steps of:

and controlling the communication interface part to convert the audio or video which the user wants to play by applying the set personalized data, thereby providing the streaming media service.

17. A computer-readable recording medium as a computer-readable recording medium including a program for executing a stereo service method, the stereo service method performing the steps of:

matching and storing Head Related Transfer Function (HRTF) data related to physical characteristics of a user and sound source environment (3D) data related to a sound source environment of the user; and

and extracting a HRTF data candidate group related to the user from the stored HRTF data based on the stored sound source environment data matched with the sound source environment test result provided by the user, and setting one data selected from the extracted candidate group as personalized HRTF data of the specific user.