KR20210010404A - Apparatus for adjusting output sound source and method thereof - Google Patents

Abstract

The present invention provides an operating method of a device for adjusting sound source output, operated by at least one processor. The method includes the steps of: calculating a threshold by collecting external noise for a certain period of time through one or more interlocking microphones, and calculating the average size of the collected external noise; preprocessing according to the input of a sound analysis model learned for the external noise if the external noise collected in real time has a value greater than the threshold; determining whether the sound generated within a critical distance is a human voice or a vehicle driving sound by inputting the preprocessed external noise into the learned sound analysis model; and outputting to the generated sound to the sound source output device which is interlocked to corresponding external noise when the external noise is determined as a human voice or the vehicle driving sound, wherein when a sound source content is output from the sound source output device, the sound source content is output by combining the voice of the person or the driving sound of the vehicle.

Description

Sound source output control device and its method {APPARATUS FOR ADJUSTING OUTPUT SOUND SOURCE AND METHOD THEREOF}

It relates to a sound source output control apparatus and method for automatically controlling the sound source output based on external noise.

Recently, as various small devices have become popular, the cases of watching music, video and DMB in daily life are increasing. In order to use these devices, earphones, headphones, etc., formed in a structure that block external noise so that they can concentrate on the corresponding content, are worn.

However, since it is not possible to completely block external noise, the user increases the volume of earphones and headphones worn, and if maintained for a long time, it may develop into a problem of hearing loss.

In addition, by being blocked from external noise and focusing on the content, an accident may occur because the user misses or ignores surrounding situations that must be recognized, resulting in social loss.

In order to solve this problem, when a sudden loud noise occurs outside while the user is wearing the earphone, research is being conducted to adjust the volume of the earphone in response to the corresponding noise. Since there is a high possibility that this is the situation after the occurrence of, the user cannot secure time to respond to the situation.

Accordingly, there is a need for a technology that accurately classifies noise generated in the surroundings and controls the volume of the earphone worn by the user or outputs external noise to the earphone so that the user can recognize the situation and secure time to respond.

The problem to be solved is to provide a technology for outputting a combination of sound source content and the recognized human voice or vehicle driving sound when a human voice or vehicle driving sound is recognized among external noises in a situation in which sound source content is output.

According to an embodiment of the present invention, as an operating method of a sound source output control device operated by at least one processor, external noise is collected for a predetermined period of time through one or more interlocked microphones, and the average magnitude of the collected external noise is calculated. To calculate the threshold value, if the external noise collected in real time has a value greater than the threshold, pre-processing the external noise according to the input of the learned sound analysis model, and inputting the pre-processed external noise into the learned sound analysis model Determining whether the sound generated within the critical distance is a human voice or a driving sound of a vehicle, and when the external noise is determined as a human voice or a driving sound of a vehicle, outputting the corresponding external noise to a sound source output device interlocking And, when the sound source content is being output from the sound source output device, the sound source content and the voice of the person or the driving sound of the vehicle are combined and output.

Collecting human voices spoken at various locations and driving sounds of a driving vehicle. Data set by selecting only sounds generated within a critical distance based on distance characteristics of the sound signal from among human voices or vehicle driving sounds Constructing, and learning a first sound analysis model that determines the voice of a person uttering within a critical distance using the data set and a second sound analysis model that determines the driving sound of a vehicle generated within the critical distance. It may further include a step.

The microphone may represent a polyhedral microphone having a microphone sensor attached to the center of each surface to receive sound for each microphone sensor.

In the learning step, when external noise is received through a plurality of microphones, only sounds within a critical distance are classified based on the time difference reached for each microphone from the position of the generated sound, and the sound within the classified critical distance is It can be learned to determine whether the voice of the vehicle or the driving sound of the vehicle.

The learning step is based on the dispersion characteristic (g _d ) of the sound according to the distance of the sound signal (S(t)) when the sound source sound signal (S _o (t') is propagated by t time as shown in the following equation). It is possible to learn to calculate a sound dispersion characteristic G _d (W) according to a distance in a domain and select only sounds generated within a critical distance based on the calculated sound dispersion characteristic.

Here, S _o (t') means a value set as the signal value of the sound source, S _o (w) is the sound source sound signal in the frequency domain, and S(W) is the sound signal S(t) in the frequency domain. Show.

In the step of calculating the threshold, the collected external noise is converted into a digital signal, and the average size is calculated based on the converted digital signals. Then, the threshold may be calculated by adding an error range for the calculated average size.

In the outputting of the sound source output device, if the external sound volume value is set based on the recognized human voice or the driving sound of the vehicle, the volume of the sound source content may be lowered based on the external sound volume value.

According to another embodiment of the present invention, as a program executed by a computing device and stored in a computer-readable storage medium, external noise is collected for a certain time through one or more interlocked microphones, and the average of the collected external noise The step of calculating the threshold value by calculating the size, deleting if the external noise collected in real time has a value less than the threshold, and preprocessing the external noise according to the input of the learned sound analysis model if it has a value greater than the threshold. The step of inputting the external noise into the learned sound analysis model to determine whether the sound generated within a critical distance is a human voice or a driving sound of a vehicle, and when the external noise is determined as a human voice or a driving sound of a vehicle, the corresponding external noise is determined. It includes instructions for executing the step of outputting to an interlocking sound source output device.

According to the embodiment, by recognizing and providing the voice of a person uttering at a close range of the user using the sound source content while blocking external noise or the driving sound of a vehicle approaching at a close range, the user can easily recognize changes in the surrounding situation. Accidents can be prevented.

In addition, according to an embodiment, the volume of the sound source output control device is automatically adjusted so that communication is possible without directly controlling the sound source content that the user is using to communicate with people around him or to pay attention to situations in which noise occurs. It provides user convenience by controlling.

1 is a block diagram showing a system including a sound source output control apparatus according to an embodiment of the present invention.
2 is an exemplary view for explaining a method of analyzing sounds within a critical distance through a tetrahedral microphone according to an embodiment of the present invention.
3 is an exemplary diagram showing a frequency domain according to a distance according to an embodiment of the present invention.
4 is a block diagram showing a sound source output control apparatus according to an embodiment of the present invention.
5 is an exemplary view showing a sound analysis model according to an embodiment of the present invention.
6 is a flowchart illustrating a method of controlling a sound source output according to an external sound according to an embodiment of the present invention.
7 is a hardware configuration diagram of a computing device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the embodiments of the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated. In addition, terms such as "... unit", "... group", and "module" described in the specification mean units that process at least one function or operation, which can be implemented by hardware or software or a combination of hardware and software. have.

In the specification, external noise includes all sounds generated from outside, such as human voices or car sounds.

In the specification, the sound source output device refers to an earphone or a headset that can be attached to the user's body and outputs sound source content. Hereinafter, an embodiment is implemented in a situation in which the sound source output device is mounted by the user.

In the specification, the sound source output control device is mounted on a terminal that provides sound source content to the sound source output device, but is not limited thereto and may be implemented in a form embedded in a sound source output device such as an earphone or a headset. Through this, the sound source output control device may control the volume, playback or stop of sound source content output from the sound source output device. Here, it is not limited by the type of earphone or headset, wired or wireless, or pairing format.

In the specification, the voice of a person or the driving sound of a vehicle refers to a sound generated within a critical distance indicated by the user in a short distance or a sound traveling toward the user, but is not limited thereto, and the bicycle clock sound, the bicycle driving sound, It may also include sounds from various means of transportation such as motorcycles and electric scooters.

In the specification, sound source and sound are used with the same meaning, and sound source content refers to content using sound, and includes all music, songs, lectures, lectures, and the like.

1 is a block diagram showing a system including a sound source output control apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the system interlocks with one or more microphones 10 to the sound source output control device 100 and collects external noise through the microphone 10. In addition, when the sound source output control device 100 analyzes the external noise and recognizes a human voice or a driving sound of a vehicle at a short distance, the sound source output control device 100 outputs the corresponding sound through the linked sound source output device 20. The sound source output device 20 is a device that outputs a sound source in close contact with an ear, such as an earphone or a headset, and represents a pair of output devices connected by wire or wirelessly.

In this case, the sound source output control apparatus 100 may use one or more learned artificial neural networks to determine whether the external noise collected by the microphone 10 is a human voice generated at a critical distance or a driving sound of a vehicle.

In addition, when the sound source output control apparatus 100 recognizes a human voice or a driving sound of a vehicle within a critical distance from external noise, if there is currently output content, the volume of the content is lowered. In this case, the threshold distance indicates a preset short distance from the user's location and can be changed and set based on the user's use environment.

In addition, the sound source output control device 100 may output external noise received from the microphone 10 through the sound source output device 20 as it is. In detail, the sound source output control device 100 may combine the sound source content being output with the external noise sound and output it through the sound source output device 20, and at this time, the external noise sound is output louder than the volume of the sound source content being output. Can be controlled to do.

The sound source content output here is sound source content stored in the terminal or sound source content received in real time from the external server 30 connected via a wired or wireless network.

In other words, the terminal in which the sound source output control device 100 is embedded is connected to the external server 30 through a network to transmit and receive data.

Here, the networks are 3GPP (3rd Generation Partnership Project) networks, LTE (Long Term Evolution) networks, WIMAX (World Interoperability for Microwave Access) networks, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network). , Wide Area Network (WAN), Personal Area Network (PAN), Blues (Bluetooth) network, satellite broadcasting network, analog broadcasting network, Digital Multimedia Broadcasting (DMB) network, and the like, but are not limited thereto.

Through this, the sound source output control device 100 may be connected to an external device or an external server 30 through a network through a separate communication module to transmit and receive data, but to the external device or an external server 30 through a terminal. It is connected and can transmit and receive data. This configuration can be easily designed and changed by the user in the future.

On the other hand, the sound source output control device 100 performs control such as whether or not to play sound source content, volume control, etc., and is a device that is connected to the equipped microphone 10 and the sound source output device 20 through a network, a smart terminal. , Smart watch, etc., but is not limited to a specific terminal.

Hereinafter, it is assumed that the sound source output control device 100 is embedded in the terminal, but the description is not limited thereto, and the sound source output device 20 is embedded in the sound source output device 20 or the terminal and the sound source output device 20 are not limited thereto. It can be built into the connecting line.

Hereinafter, a configuration of a tetrahedral microphone for classifying a critical distance of sounds received through the microphone 10 or a secured data set will be described in detail using FIGS. 2 and 3.

Hereinafter, a configuration using a tetrahedral microphone or a distance characteristic of a sound signal will be described in detail so that only sounds within a critical distance can be classified.

2 is an exemplary diagram for explaining a method of analyzing sounds within a critical distance through a tetrahedral microphone according to an exemplary embodiment of the present invention, and FIG. 3 is an exemplary diagram showing a frequency domain according to distance according to an exemplary embodiment of the present invention. to be.

As shown in FIG. 2, the microphone 10 may be implemented as a tetrahedral microphone, and microphone sensors are arranged at points respectively located at the midpoints of the slopes to receive voice from four microphone sensors (represented by circles). .

At this time, the time difference for receiving sound from the plurality of microphone sensors varies according to the distance from the sound source generation point.

For example, when the distance between the microphone 10 and the sound source is within a critical distance (near field), the time to reach each microphone sensor from the location of the sound source appears differently.

As shown in FIG. 2, as the distances reached to each microphone sensor (mic ₁ , mic ₂ , mic ₃ ) are applied differently as the distance is within the threshold distance, as a result, the time delay is reduced by comparing the time received for each microphone sensor. Occurs.

Specifically, the microphone sensor, even based on the distance (x) of the (mic _2), the mic ₃ which received sound closer than a microphone sensor (mic ₂₎ the distance value between the sound source is located in closer distance by more than x d ₂ , In mic ₁ , which receives sound at a greater distance than the microphone sensor (mic ₂ ), the distance to the sound source is located at a distance by d ₁ more than x.

Due to the difference in distance between the sound source and the sound source, there is a difference in time for each microphone sensor to receive the sound source.

Therefore, in the case of a near field within a critical distance, the microphone sensor ((mic ₁ , The time difference value between mic ₂ ) and the microphone sensor ((mic ₂ , The time difference value between mic ₃ ) appears large, whereas in the case of a far field, the microphone sensor ((mic ₁ , The time difference value between mic ₂ ) and the microphone sensor ((mic ₂ , The time difference between mic ₃ ) appears relatively insignificant.

For example, in the case of near field, the microphone sensor ((mic ₁ , The time difference between detecting the sound source between mic ₂ ) is about 10 seconds, and the microphone sensor ((mic ₂ , The difference in time when the sound source is detected between mic ₃ ) appears as about 20 seconds. In the case of a short distance, the time delay value between the microphone sensors is more than the threshold value.

On the other hand, in the case of the far field, for example, the microphone sensor ((mic ₁ , The time difference between detecting the sound source between mic ₂ ) is about 10 seconds, and the microphone sensor ((mic ₂ , The difference in time when the sound source is detected between mic ₃ ) appears as about 11 seconds. As such, in the case of a long distance, a difference appears in the time delay value between the microphone sensors below the threshold value.

As shown in Equation 1 below, a time delay (1) for a sound source located within a critical distance is represented by using a microphone sensor (mic _1, mic ₂ , mic ₃ ), and a time delay (2) for a sound source located above the threshold distance (2 ) Can be represented.

[Equation 1]

Wherein τ ₁₂ is the time difference to be received sound from the mic ₁ and the time difference to be received sound from the mic _2, length, τ ₂₃ is mic ₂ and mic ₃ d ₁ is in the distance to the sound source position and a mic _1, except the distance to the sound source position and mic ₂ , d ₂ is the distance from the sound source position to mic ₂ , excluding the distance from the sound source position and mic ₃ , and c is the speed of the sound signal.

And d is the distance between the sound source and each microphone sensor, l is the distance between the microphone sensors.

In this way, based on the time difference for the same sound input through the plurality of microphone sensors of the tetrahedral microphone 10, the sound source output control device 100 determines whether the location of the external noise received in real time is within a threshold distance or greater than the threshold distance. Can be classified.

In addition, the sound source output control apparatus 100 may secure various data sets according to various distances between the microphone 10 and the location where the voice or the driving sound of the vehicle is generated, and use it to learn an artificial neural network.

On the other hand, the sound source output control apparatus 100 may be implemented in a form of a polyhedral microphone, not a tetrahedral microphone, or a plurality of forms with each microphone having a predetermined distance.

In addition, in the case of a non-polyhedral microphone, only sounds within a critical distance may be used to learn an artificial neural network based on a sound signal for a distance characteristic.

4A is a graph showing sound dispersion characteristics according to distance, and (b) shows human sounds at a distance of 1m (1m PN) and human sounds at a distance of 6m (6m PN).

When the sound source sound signal S ₀ (t') is generated and propagated by t, the sound signal is S(t), and the dispersion characteristic according to the distance is as shown in Equation 2 below.

[Equation 2]

Here, g _d represents the dispersion characteristic according to the distance of the sound signal S(t), and this dispersion characteristic is expressed as G _d (w) in the frequency domain, and can be expressed as the ratio of the sound signal of the sound source to the sound signal propagated for t time. .

In other words, So(w) denotes a sound source sound signal in the frequency domain, and S(W) denotes a sound signal S(t) in the frequency domain.

The sound source output control apparatus 100 may train the artificial neural network in consideration of distance characteristics of such sound signals when sound sources generated at various locations are secured as a data set.

In other words, the sound source output device 100 may train an artificial neural network by selecting only signals within a critical distance based on a difference between a dispersion characteristic for sound generated at a critical distance and a dispersion characteristic for sound generated outside the critical distance. .

Hereinafter, using Figs. 4 and 5, an artificial neural network that determines the voice of a person located at a critical distance or the driving sound of a vehicle is trained, and a sound source that controls the sound source output according to the determined result using the learned artificial neural network. The output control device will be described in detail.

4 is a block diagram showing a sound source output control apparatus according to an embodiment of the present invention.

As shown in FIG. 4, the sound source output control apparatus 100 includes a collection unit 110, a preprocessor 120, a learning unit 130, a recognition unit 140, and a control unit 150.

For the sake of explanation, the collection unit 110, the preprocessor 120, the learning unit 130, the recognition unit 140, and the control unit 150 are referred to as names, but these are computing devices operated by at least one processor. . Here, the collection unit 110, the preprocessor 120, the learning unit 130, the recognition unit 140, and the control unit 150 may be implemented in one computing device or distributedly implemented in a separate computing device. . When distributed in a separate computing device, the collection unit 110, the preprocessor 120, the learning unit 130, the recognition unit 140, and the control unit 150 may communicate with each other through a communication interface. It suffices that the computing device is a device capable of executing a software program written to carry out the present invention.

The collection unit 110 collects external noise in real time through one or more interlocked microphone sensors and converts it into a digital signal.

In addition, the preprocessor 120 calculates an average size based on the digital signals converted for a preset time.

Here, the meaning of the average size refers to the amount of noise that is generally generated, and indicates a value that is difficult to discriminate as a specific sound such as a human voice or a vehicle sound.

Therefore, the preprocessor 120 may set a value obtained by adding the threshold value to the average magnitude value of the digital signals as the threshold value. The preprocessing unit 120 continuously detects digital signals greater than the threshold value at a specific period, at a set time interval, but the threshold value is repeated N times (N is a natural number) when it is not recognized as a human voice or a driving sound of a vehicle. Can be reset.

Accordingly, if the digital signal converted in real time after the threshold value is set is less than the threshold value, the preprocessor 120 ignores the digital signal, and if it is greater than the threshold value, the digital signal is transmitted to analyze the digital signal. For example, the preprocessor 120 may transmit the digital signal to the learning unit 130 when the corresponding digital signal is received as a data set, and may transmit the actual data to the recognition unit 140.

In addition, the preprocessor 120 may perform a pre-processing operation such as dividing the digital signal into a plurality of sections or converting it into a plurality of matrix forms according to an input format of an interlocked sound analysis model.

In other words, the preprocessor 120 may extract a plurality of characteristic values for a corresponding digital signal through a Mel-Frequency Cepstral Coefficient (MFCC) in order to input into an interlocked sound analysis model. Then, the extracted feature values are input to a sound analysis model for recognizing human speech. This feature value extraction can be set and changed according to the type of sound analysis model.

In addition, when different types of sound analysis models are used, the preprocessor 120 may convert the corresponding digital signal into each input format for each type of sound analysis model.

The learning unit 130 trains one or more sound analysis models based on a plurality of data sets. Here, the data set refers to a kind of learning data for learning the sound analysis model, and the digital signal collected through the same process as the collection unit 110 and the preprocessor 120 is It can be secured by tagging the driving sound.

The sound analysis model is an artificial intelligence model, which means an artificial neural network, but it is not necessarily limited. Linear regression, logistic regression, decision tree, support vector machine ( support vector machine), or the like, or a reinforcement learning model, a convolutional neural network (CNN), a deep recurrent neural network (RNN), a deep autoencoder, etc. It is not limited to one (hereinafter, a sound analysis model and an artificial intelligence model will be mixed and described).

The learning unit 130 may train an artificial neural network that determines a human voice from the collected digital signal or an artificial neural network that determines a driving sound of a vehicle from the digital signal. In other words, a first sound analysis model for determining a person's voice and a second sound analysis model for determining a driving sound of a vehicle may be trained, respectively.

The learning unit 130 may be implemented as a separate computing device, and may store one or more artificial neural networks for which training has been completed in an interworking database or retrain the artificial neural network according to a predetermined period.

For example, when the learning unit 130 is implemented in a separate computing device, the artificial neural network that has been learned is stored in the sound source output control device 100 through a wired or wireless network, and retrained in a predetermined period or in real time. The internal weight value for the artificial neural network can be updated.

In addition, the learning unit 130 may train the first sound analysis model and the second sound analysis model in consideration of a time delay characteristic of sound sources collected from a plurality of microphone sensors based on a distance to a location where sound is generated.

In other words, the learning unit 130 trains the sound analysis model to determine whether the sound is a human voice or a vehicle driving sound by classifying only the sound generated at a location within a critical distance.

Accordingly, the learning unit 130 detects only the voice within the desired threshold distance or the driving sound of the vehicle in the model learning process, and if it is classified as a sound generated outside the threshold distance, the corresponding sound is converted to noise even if it is a voice or driving sound of the vehicle. Can be taught to be classified.

As described above with reference to FIGS. 2 and 3, the learning unit 130 learns to determine whether the sound source is a human voice generated within a threshold distance or a driving sound of a vehicle, including distance information from the generated sound source.

The recognition unit 140 applies the digital signal for the actual external noise received from the preprocessor 120 to the learned artificial neural network to determine whether the digital signal is a human voice or a driving sound of a vehicle.

When using an artificial neural network for which one or more learning has been completed, the recognition unit 140 first determines whether the driving sound of the vehicle is sequentially based on a certain criterion through a specific artificial neural network, and if it is not the driving sound of the vehicle, it corresponds to the human voice. Can be determined.

For example, when prioritizing whether a user is in a dangerous situation, it is possible to first determine whether it is a driving sound of a vehicle, and then determine a human voice. Certain criteria for sequentially determining such an artificial neural network can be easily changed later.

The recognition unit 140 determines the sound of the driving sound of the vehicle moving at a low speed to the user's position through the learned artificial neural network (sound analysis model), or the driving sound of the vehicle determines the sound of a sudden stop or brake in the vicinity of the user, or In addition to the driving sound of the vehicle, various sounds such as clock sound can be discriminated. Accordingly, the discrimination of the driving sound of the vehicle can be confirmed through each independent artificial neural network or through a single artificial neural network.

Alternatively, the recognition unit 140 may simultaneously input a plurality of sound analysis models and recognize what sound means the corresponding digital signal based on result data obtained from each sound analysis model.

For example, the recognition unit 140 obtains a result value of whether the corresponding noise is a human voice or a driving sound of a vehicle through the learned sound analysis model, or if the corresponding noise is a human voice or a driving sound of a vehicle, the volume is adjusted. You can get the result of lowering.

The controller 150 may control the volume of the sound source content provided according to the recognition result of the recognition unit 140 to be adjusted or paused.

For example, when the corresponding digital signal is a human voice or a driving sound of a vehicle, the control unit 150 controls the volume of the sound source content to be lowered, and combines the sound source content with the human voice or the driving sound of the vehicle and outputs it together. can do.

In this case, a human voice or a driving sound of a vehicle may be output as it is, and the sound source content may be output with a difference in volume so as to distinguish and recognize a sound source content and a human voice or a driving sound of the vehicle.

For example, the controller 150 may output a louder volume of a human voice or a driving sound of a vehicle than the sound source content.

In addition, the control unit 150 may output the corresponding external noise as it is, or select and output only the voice of the person or the driving sound of the vehicle.

In addition, when both the voice of the person and the driving sound of the vehicle are determined, the control unit 150 may temporarily stop the provided sound source content to output the voice of the person and the driving sound of the vehicle.

In detail, the control unit 150 generates a combination (S) of sound source content and human voice or vehicle driving sound through Equation 3 below.

[Equation 3]

S = a * S ₁ + (1-a)* S ₂

Here, S ₁ denotes a sound source content being played, and S ₂ denotes a sound received from the outside. In addition, a is a value from 0 to 1, and when S ₂ is a voice of a person or a driving sound of a vehicle, a has a value less than 1, and in other cases, a=1. Here, a can be applied as a preset value according to a voice of a person or a driving sound of a vehicle, and the setting of the a value can be easily changed and designed later.

On the other hand, if it is not a human voice or a driving sound of a vehicle, the control unit 150 can estimate it as a simple noise. If it is estimated as a simple noise, the control unit 150 counts the simple noise while deleting the corresponding sound, and accumulates the corresponding count. If the number of times matches the threshold, you can reset the threshold.

As described above, the control unit 150 can automatically output a combination of sound source content and human voice or vehicle driving sound, and if the human voice or vehicle driving sound is not recognized in real time, before adjusting the volume of the corresponding sound source content. Can be reset to

5 is an exemplary view showing a sound analysis model according to an embodiment of the present invention.

As shown in FIG. 5, by using a plurality of data sets in the sound analysis model, iterative learning is performed to determine the human voice or the driving sound of the vehicle, respectively.

At this time, the sound source output control device 100 secures data about various people's voices and voices spoken toward the user at different distances within a critical distance based on the user as a data set, and the sound analysis model A (the first sound analysis model ) By repetitive learning, it is possible to analyze the characteristic value of the voice data and control the volume to be lowered when the corresponding voice data speaks to a person.

Specifically, input in the form of a matrix to the sound analysis model A, cause convolution that does not take into account the past time and separate convolution for efficient calculation are performed simultaneously, and gate activation , GA) operations are independently performed. Independently calculated G.A. The process of combining the result values into one array through concatenation (Concat) is repeated.

In the section marked with +, an operation is performed to add the values inputted through the Residual Connect for each matrix index, and in the last step, the input data is calculated by Log Softmax to determine whether the corresponding input data is human voice. It is possible to determine whether it is noise, and finally obtain a result value of lowering the volume.

On the other hand, the sound source output control apparatus 100 secures as a data set the driving sounds of various vehicles and the data collected from the vehicle driving toward the user at different distances within a critical distance based on the user, and the sound analysis model B (the second By repeatedly learning (a sound analysis model), a characteristic value of the voice data is analyzed to determine whether the corresponding voice data is a driving sound of a vehicle, and finally, a result value of lowering the volume may be obtained.

In detail, a plurality of sound sample points are input to the sound analysis model B to determine whether the vehicle is driving sound through repeated learning.

In this way, the sound source output control apparatus 100 may determine a human voice or a driving sound of a vehicle using a sound analysis model optimized for each.

Meanwhile, the sound analysis model A and the sound analysis model B are one embodiment and are not necessarily limited to using the corresponding neural network.

6 is a flowchart illustrating a method of controlling a sound source output according to an external sound according to an embodiment of the present invention.

As shown in FIG. 6, the sound source output control apparatus 100 collects learning data including a human voice or a driving sound of a vehicle (S110).

The sound source output control apparatus 100 may collect a voice uttered within a threshold distance or a driving sound of a vehicle moving toward a central region.

For example, a voice spoken toward a central region or a driving sound of a vehicle moving toward the central region may be collected within a region having a constant diameter.

The sound source output control apparatus 100 uses a delay time difference of sound received by a microphone sensor mounted on each side by using a tetrahedral microphone, or by using a dispersion characteristic of a sound signal according to a distance characteristic from the sound source to a voice within a critical distance. Or, it is possible to classify the driving sound of the vehicle.

Next, the sound source output control device 100 learns one or more artificial neural networks based on the collected data set (S120).

The sound source output control apparatus 100 performs repetitive learning to classify data input with an accuracy equal to or higher than a threshold value as a human voice or a driving sound of a vehicle.

On the other hand, depending on the situation, it is possible to generate a learned artificial neural network by collecting voices for a specific person in addition to general human voices or driving sounds of a vehicle or various external sounds such as bicycles or electric scooters other than vehicles as a data set.

In this way, artificial neural networks can be trained to collect and recognize data sets that are most suitable according to the user's situation.

After the artificial neural network is trained, it can be started immediately at step S130 except for steps S110 and S120.

Next, the sound source output control apparatus 100 calculates a threshold value through the average magnitude of external noise for a predetermined time (S130).

The sound source output control apparatus 100 may calculate a threshold value by adding an estimated error range to the average noise calculated for the collected external noise. Here, the error range can be easily changed and designed later.

Then, the sound source output control device 100 collects the external noise of the sound source output device in real time (S140). The sound source output control device 100 collects external noise in real time through an interlocked microphone and converts it into a digital signal.

Then, the sound source output control apparatus 100 checks whether the magnitude of the external noise has a value greater than a threshold value (S150).

At this time, the sound source output control apparatus 100 estimates that the external noise is less than the threshold value, ignores it, and returns to step S140 or S130.

When the amount of the external noise has a value greater than the threshold, the sound source output control apparatus 100 recognizes a human voice or a driving sound of a vehicle from the external noise through the learned artificial neural network (S160).

The sound source output control apparatus 100 may determine whether an external noise is a voice of a person uttering toward the user or a driving sound of a vehicle approaching the user through the learned artificial neural network.

In this case, when the sound source output control device 100 is not all of a human voice or a driving sound of a vehicle, the corresponding external noise may be ignored and the process returns to step S140 or S130.

When a human voice or vehicle driving sound is recognized, the sound source output control apparatus 100 outputs the recognized human voice or vehicle driving sound through the sound source output device (S170).

The sound source output control device 100 is a sound source output device worn by a user and outputs a corresponding person's voice or a vehicle driving sound. In this case, when the sound source content is being output from the sound source output device, the sound source output control device 100 may control to lower the volume of the sound source content and output a corresponding person's voice or vehicle driving sound.

In addition, the sound source output control device 100 is automatically terminated when the user ends the use of the sound source output device or is detached from the user's body.

7 is a hardware configuration diagram of a computing device according to an embodiment of the present invention.

As shown in FIG. 7, the hardware of the computing device 200 may include at least one processor 210, a memory 220, a storage 230, and a communication interface 240, and may be connected through a bus. have. In addition, hardware such as an input device and an output device may be included. The computing device 200 may be equipped with various software including an operating system capable of driving a program.

The processor 210 is a device that controls the operation of the computing device 200 and may be various types of processors 210 that process instructions included in a program. For example, a CPU (Central Processing Unit) or an MPU ( Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), and the like. The memory 220 loads a corresponding program so that instructions described to perform the operation of the present invention are processed by the processor 210. The memory 220 may be, for example, read only memory (ROM), random access memory (RAM), or the like. The storage 230 stores various types of data, programs, etc. required to execute the operation of the present invention. The communication interface 240 may be a wired/wireless communication module.

According to an embodiment, by recognizing and providing a voice of a person occurring at a close distance of a user using sound source content or a driving sound of a vehicle approaching at a close distance, it is possible to prevent accidents that may occur by easily recognizing changes in surrounding conditions.

In addition, according to the embodiment, communication is possible without controlling the sound source content used by the user to have a conversation or pay attention to a certain situation.

In addition, since the characteristic values of sound corresponding to dozens of coefficients are extracted using MFCC for external noise and the corresponding characteristic values are input to the artificial neural network, the human voice or vehicle can be accurately calculated with a relatively small amount of artificial neural network calculation. Can determine the driving sound of

The embodiments of the present invention described above are not implemented only through an apparatus and a method, but may be implemented through a program that realizes a function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (14)

A method of operating a sound source output control device operated by at least one processor,
Collecting external noise for a certain period of time through one or more interlocked microphones, calculating the average size of the collected external noise to calculate a threshold,
If the external noise collected in real time has a value greater than the threshold, preprocessing the external noise according to the input of the learned sound analysis model,
Inputting the pre-processed external noise into the learned sound analysis model to determine whether the sound generated within a critical distance is a human voice or a driving sound of a vehicle, and
If the external noise is determined as the voice of the person or the driving sound of the vehicle, outputting the external noise to a sound source output device interlocking with the external noise,
Including,
When the sound source content is being output from the sound source output device, the method of outputting a combination of the sound source content and the voice of the person or the driving sound of the vehicle. In claim 1,
Collecting voices of a person igniting at various locations and driving sounds of a driving vehicle,
Constructing a data set by selecting only the sound generated within a critical distance based on the distance characteristic of the sound signal from among the human voice or the driving sound of the vehicle, and
Learning a first sound analysis model for determining the voice of a person speaking within a critical distance using the data set and a second sound analysis model for determining a driving sound of a vehicle generated within the critical distance,
Operation method further comprising a. In paragraph 2,
The microphone is a method of operation representing a polyhedral microphone having a microphone sensor attached to the center of each surface to receive sound for each microphone sensor. In paragraph 3,
The learning step,
In the case of receiving external noise through a plurality of microphones, only sounds within a critical distance are classified based on the time difference reached for each microphone from the location of the generated sound, and human voice or vehicle for the classified sounds within the critical distance. How to learn to determine whether the driving sound of the. In paragraph 2,
The learning step,
As shown in the following equation, when the sound source sound signal (S _o (t') is propagated by t time, based on the dispersion characteristic (g _d ) of the sound according to the distance of the sound signal (S(t)), An operating method of calculating sound dispersion characteristics G _d (W) and learning to select only sounds generated within the threshold distance based on the calculated sound dispersion characteristics.

Here, (S _o (t') means a value set as the signal value of the sound source sound signal, S _o (w) is the sound source sound signal in the frequency domain, and S(W) is the sound signal in the frequency domain (S( t) In paragraph 2,
The step of calculating the threshold is
An operating method of converting the collected external noise into a digital signal, calculating the average size based on the converted digital signals, and calculating the threshold value by adding an error range for the calculated average size. In paragraph 6,
The step of outputting to the sound source output device,
If an external sound volume value is set based on the recognized voice of the person or the driving sound of the vehicle, the operation method of controlling the volume of the sound source content to be lowered based on the external sound volume value. A program executed by a computing device and stored in a computer-readable storage medium,
Collecting external noise for a certain period of time through one or more interlocked microphones, calculating the average size of the collected external noise to calculate a threshold,
If the external noise collected in real time has a value less than the threshold value, delete it, and if it has a value greater than the threshold value, preprocessing the external noise according to the input of the learned sound analysis model,
Inputting the pre-processed external noise into the learned sound analysis model to determine whether the sound generated within a critical distance is a human voice or a driving sound of a vehicle, and
If the external noise is determined as the voice of the person or the driving sound of the vehicle, outputting the external noise to a sound source output device interlocking with the external noise,
A program containing instructions to execute. In clause 8,
Collecting voices of a person igniting at various locations and driving sounds of a driving vehicle,
Constructing a data set by selecting only the sound generated within a critical distance based on the distance characteristic of the sound signal from among the human voice or the driving sound of the vehicle, and
Learning a first sound analysis model for determining the voice of a person speaking within a critical distance using the data set and a second sound analysis model for determining a driving sound of a vehicle generated within the critical distance,
The program further comprises. In claim 9,
A program that collects the external noise in real time using a polyhedral microphone that has a microphone sensor attached to the center of each surface to receive sound for each microphone sensor. In claim 10,
The learning step,
A program that classifies only sounds within a critical distance based on the time difference reached for each microphone sensor from the location of the generated sound, and learns to determine whether the classified sound within the critical distance is a human voice or a vehicle driving sound . In claim 9,
The learning step,
A program that learns to select only the sound generated within the threshold distance according to the dispersion characteristic in the frequency domain represented by the ratio of the sound signal of the sound source to the external noise to the sound signal propagated by t time based on the dispersion characteristic of sound according to distance . In claim 9,
The step of outputting to the sound source output device,
When the sound source content is being output from the sound source output device, a program for outputting a combination of the sound source content and the voice of the person or the driving sound of a vehicle. In claim 13,
The step of outputting to the sound source output device,
A program for determining the volume of the combined sound by controlling the volume of the sound source content to be lowered based on the external sound volume value when the external sound volume value is set based on the recognized human voice or the driving sound of the vehicle .

Images