KR20230144822A - Neural network learning unit and improved system and method for providing sound source maps inclulding the same - Google Patents

Abstract

The present invention discloses a sound source map providing system. More specifically, the present invention is a deep learning-based sound source map that provides a sound source map that displays the intensity and location of one or more sound sources existing on a flat or curved surface in the form of an image using sound signals measured with a plurality of microphones. It relates to a provision system and method.
According to an embodiment of the present invention, in a system that provides an acoustic map using an ultrasonic microphone array, at least two encoders with different frequencies are installed in the deep learning model introduced for analysis, thereby differentiating the low and high frequencies. Spatial aliasing phenomenon can be minimized.

Description

Neural network learning unit and improved sound source map providing system including the same {NEURAL NETWORK LEARNING UNIT AND IMPROVED SYSTEM AND METHOD FOR PROVIDING SOUND SOURCE MAPS INCLULDING THE SAME}

The present invention relates to a sound source map providing system, and in particular, a deep learning method that provides a sound source map that displays the intensity and location of one or more sound sources existing on a flat or curved surface in the form of an image using sound signals measured with a plurality of microphones. It is about a sound source map providing system.

A sound source map refers to an image that calculates the locations and intensities of multiple sound sources and displays them on a flat or curved surface where multiple sound sources whose locations and intensities are unknown are believed to exist.

Figure 1 is a schematic diagram showing a beam forming method for providing a conventional sound source map. In relation to the technology for generating a conventional sound source map, according to the beamforming method, the measurement plane of the measurement device A microphone array is formed by arranging a number of microphones, and the microphone array predicts the location and intensity of each sound source on the sound source plane from the measurement signals received in the x, y, and z directions to create a beamforming map. can be created.

In particular, in this beam forming method, the location and intensity of sound sources are calculated from sound pressure values acquired using a microphone array, and the map is completed by matching them on a plane.

However, according to the above-described beam forming method, there is a problem in that spatial aliasing occurs in a frequency section where the spacing between microphones that detect a sound source is larger than a half wavelength.

The above-described spatial aliasing refers to a phenomenon in which peak values appear not only at the location of the actual sound source on the sound source map but also at other locations, making it appear as if a sound source is not actually present. This spatial aliasing is also called a ghost source.

To solve this problem, deep learning technology in the field of artificial intelligence has been applied, and various solutions have been proposed to overcome the ghost sound source phenomenon described above.

Meanwhile, as one of the existing sound source map production methods, there were deconvolution methods to improve the resolution of the sound source map by beamforming methods, such as time delay and sum beamforming, but these existing methods It is known that spatial aliasing problems still occur at high frequencies even if . In other words, even if the latest deep-learning technology is applied as well as the general beam forming method, there is a limit to solving the problem of the appearance of many ghost sound sources due to the occurrence of spatial aliasing.

Public Patent Publication No. 10-2022-0032382 (Publication date: 2022.03.15.)

The present invention was created to solve the above-mentioned problems. The present invention is a system that detects one or more sound sources using a microphone array and provides a sound source map that maps the intensity and location on a plane, and provides a sound source map by spatial aliasing. There is a challenge in improving the decline in map accuracy.

To this end, the present invention has the task of providing a system that configures parts of the deep learning model differently for lower and higher frequency bands based on the spatial Nyquist frequency.

In order to solve the above-described problem, the neural network learning unit according to a preferred embodiment of the present invention is composed of a plurality of layers and includes one or more main features extracted from the first frequency band from input data through a convolution operation. A first encoder for generating a map, a second encoder for generating a feature map composed of a plurality of layers and including one or more key features extracted from the input data in a second frequency band through a convolution operation, the input data A switch that selects the output of one of the first and second encoders according to the shape, and, depending on the selection of the switch, a feature map output from the first or second encoder is input and restored to generate an output image. May include a decoder.

The first and second frequency bands may be divided into high and low bands, respectively, based on the spatial Nyquist frequency wavelength for the input data.

The plurality of layers may each include one or more convolutional layers, a pooling layer, a batch normalization layer, and a non-linearization layer.

The first and second convolutional encoders may be set to have different kernel sizes of the convolutional layers.

During learning, the switch may select the first encoder when spatial aliasing does not exist in the input data, and select the second encoder when spatial aliasing exists in the input data.

In addition, in order to solve the above-described problem, an improved sound source map providing system including a neural network learning unit according to a preferred embodiment of the present invention provides a sound source map from sound signals received from a microphone array in which a plurality of microphones are arranged at regular intervals. A system for providing a sound source map, comprising: an image generator that generates a beamforming image through a beamforming process from an acoustic signal received from the microphone array; a target map generator that generates a target map using the beamforming image; It includes a neural network learning unit that receives the target map, performs learning, and generates a sound source map that images the location and intensity of the sound source included in the analysis image according to one or more key features included in the image based on the set deep learning model. can do.

The neural network learning unit is composed of a plurality of layers, and is composed of a first encoder that generates a feature map including one or more main features extracted in a first frequency band from the beamforming image through a convolution operation, and a plurality of layers. a second encoder that generates a feature map including one or more key features extracted in a second frequency band from the beamforming image through a convolution operation, the first and second encoders according to the shape of the beamforming image; It may include a switch for selecting one of the outputs, and a decoder for generating the sound source map by restoring the feature map output from the first or second encoder according to the selection of the switch.

The first and second frequency bands may be divided into high and low bands, respectively, based on the spatial Nyquist frequency wavelength for the beamforming image.

The switch may select the first encoder when spatial aliasing does not exist in the beamforming image, and select the second encoder when spatial aliasing exists in the beamforming image.

According to an embodiment of the present invention, in a system that provides an acoustic map using an ultrasonic microphone array, at least two encoders with different frequencies are installed in the deep learning model introduced for analysis, thereby differentiating the low and high frequencies to determine the space Aliasing phenomenon can be minimized.

Accordingly, a wider area can be referenced in the convolution operation of the deep learning model during learning, allowing the spacing between sensors to be set narrower when designing the microphone array, and thus building a system by reducing the number of microphones required for the microphone array. It has the effect of reducing costs.

Figure 1 is a schematic diagram showing a conventional beam forming method for providing a sound source map.
Figure 2 is a diagram showing the structure of an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention.
Figure 3 is a diagram illustrating a method of generating a target map by an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention.
Figure 4 is a diagram showing the network structure of the neural network learning unit included in the improved sound source map providing system according to an embodiment of the present invention.
Figure 5 is a diagram showing a machine learning method of an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention.
Figure 6 is a diagram showing a method of providing a sound source map in an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification and include each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Additionally, unless otherwise defined, all terms used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

In the following description, the "neural network learning unit" and the "improved sound source providing system" of the present invention and each component thereof are executable by a known microprocessor and are stored in a readable and writable recording medium and stored on a computing device. It can be mounted.

In the following description, the term referring to the “improved sound source map providing system including a neural network learning unit” of the present invention may be abbreviated as “sound source map providing system” or “system” for convenience of description.

Hereinafter, a neural network learning unit and an improved sound source map providing system including the same according to an embodiment of the present invention will be described with reference to the drawings.

Figure 2 is a diagram showing the structure of an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention, and Figure 3 is a diagram showing the structure of an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention. This is a diagram illustrating the goal map creation method.

Referring to Figure 2, a sound source map providing system 100 according to an embodiment of the present invention, an image generator 110 that generates a beamforming image through a beam forming process from the sound signal received from the microphone array 10, A target map generator 120 that generates a target map using a beamforming image, receives the target map, performs learning, and includes it in the analysis image according to one or more key features included in the image based on a set deep learning model. It may include a neural network learning unit 130 that generates a sound source map that images the location and intensity of the sound source.

In detail, the image generator 110 receives an acoustic signal from a sound source and generates a beamforming image through beam forming technology such as delay-and-sum beamforming. can do.

Here, the microphone array 10 is composed of a plurality of microphones arranged left and right. The overall width of the microphones arranged affects the sound signals of low-band frequencies, and the spacing of each microphone affects the sound signals of high-band frequencies. gives.

Accordingly, the designer of the conventional microphone array 10 must design the array configuration differently, such as arranging the spacing between microphones of the microphone array 10 to be narrow, taking into account the spatial aliasing phenomenon that may occur in the acoustic signal, etc., but the present invention According to the improved sound map providing system 100, by generating a sound source map through an appropriate encoder according to the sound signal, it is possible to cover both low-band and high-band frequencies without changing the design of the microphone array. do.

The target map generator 120 generates a target map used as learning data based on the acoustic signal and inputs it to the neural network learning unit 130 to perform deep learning. To this end, the target map generator 120 may be composed of a plurality of modules according to their functions.

For example, the target map generator 120 includes a grid generation module 121 that generates a grid with a predetermined range of intervals around the sound source. The result value is maximum at the location of the sound source, and depending on the distance to the sound source, A result value calculation module 123 that calculates the result value for each coordinate of the grid so that the result value decreases, arranges the result value at a position on the matrix corresponding to each coordinate of the grid, and uses the result value arranged in the matrix. It may include a mapping module 125 that generates a target map in the form of an image.

Referring to FIG. 3 together, the grid generation module 121 can generate a grid including grid points at regular intervals for a specific area where one or more sound sources exist, and display the location of each sound source on the grid. You can.

Here, the narrower the spacing (K) between grid points forming the grid is, the more accurate the acquisition of the sound source map is, but the longer the computation time is, and the wider the spacing (K) is, the shorter the computation time required to acquire the sound source map becomes. The accuracy becomes somewhat lower. Accordingly, according to an embodiment of the present invention, the desired spacing (K) between grid points of the grid provided by the grid generation module 121 can be set to an appropriate size through repeated experiments.

The result calculation module 123 uses the grid points {(1,1), (1,2) ~ (N,M; N,M are natural numbers)} of the grid provided by the grid generation module 121 and the grid The result can be calculated from the distance (R) to one or more sound sources (A, B, C) located in .

Here, the result value refers to the specific value each pixel has when imaging the acquired sound signal, and can be calculated using the distance (R) between the sound source (A, B, C) and each coordinate of the grid. there is.

Additionally, in calculating the result value, the rate of change of the result value according to the distance (R) can be adjusted by adjusting the multiplier of R. In other words, by setting the multiplier of the square root of R to a relatively large value, the rate of change of the resulting value according to the distance (R) can be increased, making it possible to more clearly indicate the difference in the location or intensity of the sound source in the sound source map.

The mapping module 125 can arrange the result values calculated using the distance (R) described above in positions on a matrix corresponding to the grid. In particular, the mapping module 125 can generate a matrix of the same size as the grid generated by the grid generation module 121 by repeating the operation until the result values for all coordinates are calculated, and the resulting matrix can be added to the generated matrix. You can arrange the values and use them to create a goal map in the form of an image.

Accordingly, the target map generated through the target map generator 120 can be input to the neural network learning unit 130, which will be described later, and used as learning data for the corresponding beamforming image.

The neural network learning unit 130 receives the target map from the target map generator 120 and repeatedly performs machine learning to construct feature maps for various sound sources, that is, beamforming images, and responds to sound signals in various frequency bands. By receiving analysis data in the form of a beamforming image and extracting features, data with spatial aliasing removed can be extracted and a sound source map can be generated and provided.

As the spatial aliasing phenomenon appears spread throughout the sound source map, the structure of the deep learning model needs to be different at frequencies lower than the spatial Nyquist frequency. Here, the spatial Nyquist frequency refers to the frequency at which the spacing between neighboring microphones on the microphone array is equal to half a wavelength.

To this end, according to an embodiment of the present invention, the deep learning model applied to the system may be a 2 encoder-1 decoder structure based on a convolutional encoder-decoder network.

In detail, the neural network learning unit 130 according to an embodiment of the present invention is composed of a plurality of layers and includes a feature map including one or more main features extracted from the input data in the first frequency band through a convolution operation. A first encoder 131 that generates, a second encoder 132 that is composed of a plurality of layers and generates a feature map including one or more main features extracted from the input data in the second frequency band through a convolution operation. , a switch 135 that selects the output of one of the first and second encoders 131 and 132 according to the type of input data, and, depending on the selection of the switch 135, the first or second encoder 131 , 132) may include a decoder 137 that receives and restores the feature map output from 132) to generate an output image.

The first encoder 131 and the second encoder 132 are encoders composed of a plurality of convolutional layers, and can extract feature maps based on different frequency standards for input data. As an example, the first encoder 131 is set to be suitable for input data having a general frequency wavelength in which spatial aliasing does not occur, and the second encoder 132 is set to be suitable for input data having a frequency wavelength in which spatial aliasing occurs. It can be set to do so.

Here, the above-described frequency standard is set based on the spatial Nyquist frequency, but may be set differently depending on the size of the microphone array and the distance between microphones.

Accordingly, the first encoder 131 may be designed to generate a feature map from input data having a general frequency wavelength, and the second encoder 132 may be designed to generate a feature map from input data having a higher frequency wavelength. It can be.

A detailed description of the specific network structure and characteristics of the decoder 137, including the first and second encoders 131 and 132, will be described later.

The switch 135 serves to connect one selected among the first and second encoders 131 and 132 and a decoder 137, which will be described later. In particular, according to an embodiment of the present invention, when analyzing analysis data, one encoder with different setting values is selectively operated according to the frequency wavelength, and the switch 135 operates at a certain frequency wavelength through the wavelength for the analysis data. , and the feature map generated by one encoder can be transmitted to the decoder 137.

The decoder 137 may generate output data by restoring the feature map extracted by the encoder through decoding. The decoder 137 may be configured in a structure that is symmetrical to the encoder, and may derive an image with maximized features for a specific frequency component through a deconvolution process for the feature map. Accordingly, when the segmented analysis data is a beamforming image for one or more sound sources, a sound source map that maximizes the location and intensity of the sound source is provided.

According to the above-described structure, the improved sound source map system 100 according to an embodiment of the present invention can provide a sound source map that can intuitively check the location and intensity of the sound source from the beamforming image through deep learning technology, In particular, by constructing the neural network learning unit with at least two encoders set to extract different feature maps according to different frequency wavelengths, a sound source map with minimized spatial aliasing phenomenon can be provided.

Hereinafter, with reference to the drawings, the technical idea of the present invention will be described in detail through the network structure of the neural network learning unit mounted on the improved sound source map providing system according to the embodiment of the present invention.

Figure 4 is a diagram showing the network structure of the neural network learning unit included in the improved sound source map providing system according to an embodiment of the present invention.

Referring to FIG. 4, the neural network learning unit according to an embodiment of the present invention may be composed of two convolutional encoders and one convolutional decoder. Each encoder and decoder have a mutually symmetrical structure, and the convolutional encoder-decoder reduces the size of the feature map extracted from the input image and then makes it larger to the size of the input image to classify each pixel of the input image. As a result, classes can be classified.

Each of the two convolutional encoders may be composed of multiple layers. The plurality of layers may include at least one convolutional layer, a pooling layer, a batch normalization layer, and a non-linear activation layer. And in the convolutional encoder, each of the above-described layers can be sequentially arranged into a convolutional layer, a batch normalization layer, and a non-linearization layer.

Additionally, one layer may be repeated as many times as the number of channels (ch). That is, one layer may have the structure of ‘(convolutional layer, batch normalization layer, and non-linearization layer) × number of channels (ch) + pooling layer’.

This convolutional layer can output a feature map through convolutional operations on the input image. At this time, the filter that performs the convolutional operation is called the kernel. Additionally, the operation parameters that make up the kernel are called kernel parameters or weights. In a convolutional layer, different types of kernels can be used for one input.

Additionally, the convolutional layer performs a convolution operation targeting a specific area of the input image, and the operation area is called a window. These windows can be moved one space from the top left to the bottom right of the image, and the size of the movement at one time can be adjusted. At this time, the movement size is called stride.

The convolutional layer moves the window in the input image and performs a convolution operation on all areas of the input image.

As an example, when performing a convolution operation on an input value, the output value of the convolution layer for the input size ( N, C _in , H, W ) and output (N, C _out , H, W) is the following math. It can be expressed as Equation 1 (where ' N ' is the batch size, ' C ' is the number of channels, ' H ' is the height of the kernel, and ' W ' is the width of the kernel).

Here, 'out' represents output data, 'bias' represents bias value, 'weight' represents weight, '*' represents convolution, and 'input' represents input data.

Accordingly, according to an embodiment of the present invention, the first and second convolutional encoders each perform a convolution process according to different kernel sizes { (H, W) = k }, that is, the value of 'k'.

In particular, according to an embodiment of the present invention, the two convolutional encoders are characterized in that they generate different feature maps as the kernel size (k) is set differently corresponding to the frequency band of the input data.

The convolutional encoder is determined by the feature map extracted according to the kernel size (k) during the convolution process. In this convolution process, a relatively small-sized kernel is suitable for low-band frequency images, that is, images in which spatial aliasing does not occur, and a relatively large-sized kernel is suitable for high-band frequency images, i.e., images in which spatial aliasing occurs. Suitable for images that need to be minimized.

Therefore, the first convolution encoder of the present invention has a relatively small kernel size (k) based on the spatial Nyquist frequency (ex. k = 5), and the second convolution encoder has a relatively small kernel size (k). The size (k) can be set to a large value (ex. k=7).

The pooling layer subsamples the feature map obtained as a result of the convolutional layer. Pooling operations include max pooling and average pooling. Max pooling takes the largest sample value within the window. Average pooling samples the average value of the values contained in the window. In general, pooling is used to ensure that the stride and window sizes are the same.

The non-linearization layer is a layer that determines the output value from the neuron, and can use a predetermined transfer function. Known transfer functions include Relu, sigmoid, etc., and the Relu function can be used in the system according to the embodiment of the present invention.

According to the above-described hierarchical structure, two convolutional encoders can generate a feature map for the input image.

The switch can transfer the output of either of the two convolutional encoders to the convolutional decoder described later. These switches are controlled according to the frequency band of the input data, and transmit the output of one convolutional encoder to the convolutional decoder.

The convolutional decoder can generate an output image using the feature maps generated by the two convolutional encoders. In contrast to the two convolutional encoders, one convolutional decoder is installed and can be composed of multiple layers.

One layer may include an upsampling layer and a deconvolutional layer. The deconvolutional layer can perform the reverse operation of the convolutional layer of the convolutional encoder. Additionally, the deconvolutional layer may repeat the structures of the convolutional layer, batch normalization layer, and non-linearization layer.

Here, the convolutional decoder may have a symmetrical structure with the convolutional encoder and thus has a similar number of convolutional layers and the same convolutional kernel size (k) and number as the convolutional encoder.

The upsampling layer can perform the reverse operation of the pooling layer. The upsampling layer can perform upsampling, and the pooling layer conversely serves to expand the dimension.

The deconvolutional layer can perform the reverse operation of the convolutional layer. The deconvolutional layer performs a convolution operation in the opposite direction to the convolutional layer. The deconvolutional layer receives a feature map as input and can generate an output image for visualization through a convolution operation using a kernel.

Here, if the stride is set to 1, the deconvolutional layer can output an image in which the horizontal and vertical sizes of the feature map are the same as the horizontal and vertical sizes of the output, and if the stride is set to 2, the deconvolutional layer can output an image with the horizontal and vertical sizes of the feature map. It is possible to output images half the size.

Accordingly, the output image that has completed all of the deconvolutional procedures has high accuracy above a certain level as it is restored by a feature map with minimized spatial aliasing from the selected encoder.

Hereinafter, a method for generating a sound source map by a system according to an embodiment of the present invention will be described in detail with reference to the drawings.

Figure 5 is a diagram showing a machine learning method of an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention, and Figure 6 is a diagram showing an improved sound source map providing system including a neural network learning unit according to an embodiment of the present invention. This diagram shows how the system provides sound source maps.

In the following description, unless otherwise stated, the executor of each step is the neural network learning unit, the system, and each component thereof according to an embodiment of the present invention.

Referring to FIG. 5, according to the machine learning method using the improved sound source map providing system according to an embodiment of the present invention, when a microphone array in which one or more microphones are arranged for data collection receives an acoustic signal from a sound source (S100), The system receives acoustic signals from the microphone array in the form of electrical signals.

In addition, the above-described step S100 can be applied not only by collecting sound in real time to create a sound source map, but also by receiving sound that has been previously received and recorded through a separate microphone array.

Next, the system uses the input sound signal to create a beamforming image through beamforming technology (S110). Beam forming techniques such as delay-and-sum beamforming may be used in step S110.

Additionally, the system can perform a procedure to generate a target map to secure learning data for beamforming learning.

Next, the system generates a grid including grid points at regular intervals for a specific area where one or more sound sources exist from the sound signal (S120). At this time, the location of each sound source is displayed on a grid, and the desired spacing between grid points is set to an appropriate size through repeated experiments.

Next, the system calculates a result value, which is a specific value of the pixel, when imaging the sound signal according to the positions and distances of the grid points and one or more sound sources located on the grid (S130).

Next, the system arranges the results calculated using the distance from the sound source on the coordinates to positions on the matrix corresponding to the grid and uses them to generate a target map in the form of an image (S140).

Afterwards, the system inputs the target map into the mounted neural network learning unit and performs learning as learning data for the corresponding beamforming image (S150).

Next, referring to FIG. 6, according to the method of providing a sound source map using a system according to an embodiment of the present invention, the acoustic signal obtained from the microphone array is converted into a beamforming image form, and the learned neural network learning unit By inputting and extracting features through convolution, a sound source map can be created where the location and intensity of the sound source are clearly displayed on the image.

To this end, when the neural network learning unit receives analysis data, it extracts key features of low-band frequencies from the analysis data through a first encoder with a kernel of a size below or below the standard and generates a feature map including these (S200) . In addition, the neural network learning unit extracts key features of high-band frequencies from the analysis data through a second encoder in which a kernel of a size exceeding or larger than the standard is set for the analysis data and generates a feature map including these (S210).

Next, the neural network learning unit selects the second encoder if the type of analysis data, that is, the frequency wavelength of the analysis data is a frequency at which the above-mentioned inter-microphone interval is larger than a half-wavelength, and if not, selects the first encoder (S220) .

Next, the feature map output from the encoder selected in step S220 is sent to the decoder, and the decoder restores the feature map through a deconvolution procedure so that the intensity and location of the sound source are accurately displayed, and the space that occurs in the case of high-band frequency is displayed. Generate a sound source map with minimal aliasing (S230).

Although many details are described in detail in the above description, this should be interpreted as an example of a preferred embodiment rather than limiting the scope of the invention. Therefore, the invention should not be determined by the described embodiments, but by the scope of the patent claims and their equivalents.

10: Microphone array 100: Improved sound source guidance system
110: image generation unit 120: target map generation unit
121: grid generation module 123: result calculation module
125: Mapping module 130: Neural network learning unit
131: first encoder 132: second encoder
135: switch 137: decoder

Claims (9)

a first encoder that is composed of a plurality of layers and generates a feature map including one or more main features extracted from input data in a first frequency band through a convolution operation;
a second encoder composed of a plurality of layers and generating a feature map including one or more main features extracted from input data in a second frequency band through a convolution operation;
a switch for selecting an output of one of the first and second encoders according to the type of the input data; and
A decoder that receives the feature map output from the first or second encoder and restores it to generate an output image according to the selection of the switch.
Neural network learning unit including. According to claim 1,
The first and second frequency bands are,
A neural network learning unit that is divided into high and low bands, respectively, based on the spatial Nyquist frequency wavelength for the input data. According to claim 2,
The plurality of layers are,
One or more convolutional layers, a pooling layer, a batch normalization layer, and a non-linearization layer each.
Neural network learning unit including. According to claim 3,
The first and second convolutional encoders are:
A neural network learning unit in which the kernel sizes of the convolutional layers are set differently. According to claim 2,
The switch is
During learning, the neural network learning unit selects the first encoder when spatial aliasing does not exist in the input data, and selects the second encoder when spatial aliasing exists in the input data. A sound source map providing system that generates a sound source map from sound signals received from a microphone array in which a plurality of microphones are arranged at regular intervals, comprising:
an image generator that generates a beamforming image through a beamforming process from the acoustic signal received from the microphone array;
a target map generator that generates a target map using the beamforming image; and
A neural network learning unit that receives the target map, performs learning, and generates a sound source map that images the location and intensity of the sound source included in the analysis image according to one or more key features included in the image based on the set deep learning model.
An improved sound source map providing system including. According to claim 6,
The neural network learning unit,
a first encoder that is composed of a plurality of layers and generates a feature map including one or more main features extracted in a first frequency band from the beamforming image through a convolution operation;
a second encoder that is composed of a plurality of layers and generates a feature map including one or more main features extracted in a second frequency band from the beamforming image through a convolution operation;
a switch for selecting an output of one of the first and second encoders according to the shape of the beamforming image; and
A decoder that generates the sound source map by restoring the feature map output from the first or second encoder according to selection of the switch.
An improved sound source map providing system including. According to claim 7,
The first and second frequency bands are,
An improved sound source map providing system that is divided into high and low bands based on the spatial Nyquist frequency wavelength for the beamforming image, respectively. According to claim 7,
The switch is
An improved sound source map providing system that selects the first encoder when spatial aliasing does not exist in the beamforming image, and selects the second encoder when spatial aliasing exists in the beamforming image.

Images