KR20130065022A - Sound field displaying method using image correction of image distortion - Google Patents

Abstract

PURPOSE: A method for displaying sound field through image distortion correction is provided to display the position of a sound source on a corrected image accurately through beam forming technique, by correcting a measured image with an error value calculated between a subject and a measured image pixel. CONSTITUTION: Omnidirectional images of a vehicle room space, where a noise source is generated, are acquired by using a number of fisheye cameras arranged in the vehicle room space. The acquired image is corrected by using an error correction equation. The corrected image is overlaid on a sound source prediction plane so that the corrected image coincides with the sound source prediction plane on a beam forming technique. [Reference numerals] (AA) Start a sound visualization algorithm; (BB) Microphone signal; (CC) Beam forming technique; (DD) Sound visualization result; (EE) Create a beam power map on a sound source prediction plane(square shape); (FF) Image overlay; (GG) Overlay the sound visualization result corresponding to each image of omnidirectional images (1+2 each); (HH) Start an image correction algorithm; (II) Obtain image data; (JJ) Use an expanded image lattice form using a fisheye lens for the obtained image; (KK) Apply an error correction formula; (LL) Image correction; (MM) Spread the obtained image using the error correction formula; (NN) Adjust the spread image in line with the lattice of the sound source prediction plane; (OO) Image synthesis; (PP) Generate an omnidirectional image using the five corrected images; (QQ) Operate the n-th camera (n = 1 - 5); (RR) Repeat the operation for five image sensors

Description

Sound field displaying method using image correction of image distortion}

The present invention relates to a sound field display method through image distortion correction, and more particularly, to a sound field display method through image distortion correction that can accurately display the location of the noise source generated in the vehicle interior space.

As the noise characteristics of various products such as automobiles and home appliances are recognized as the main performance indicators, efforts have been made to make products with lower noise from the development stage.

In order to reduce the noise of the product, first, it is used to find out where the noise occurs and what is the cause of the noise, and to reduce the noise by changing the design. Needed.

Currently, a method of measuring sound pressure intensity using an intensity probe and a noise source location visualization method using a microphone array beamforming method are generally used.

A relatively recent method for measuring the location of noise sources using a microphone-spherical array uses several microphones, noise measuring sensors, and signal sources using the phase difference of the measured signal according to the distance difference between the noise source and the measuring sensor. Calculate the distribution and use techniques to estimate the location of the noise source based on its strength and weakness.

As described above, the beamforming technique is used to measure the location of the noise source (sound field visualization) in the vehicle interior space.

FIG. 1 is a block diagram illustrating a process for a beamforming technique according to the prior art. Converting the signal into a cross spectral matrix, calculating a signal in the transformed frequency domain, configuring an A matrix defining a signal characteristic relationship between the measured signals, and a microphone location Constructing a C vector, which is an electronic steering vector, through a relationship between the position of a virtual grid location and a position of a virtual grid location, and calculating a beam power map matrix by performing operations between the C vector and the A matrix. And configuring the beam power map by displaying the calculated beam power map matrix on the coordinate.

As shown in FIG. 2, the sound pressure distribution of the noise source may be displayed by matching the indoor image image photographing the indoor image of the vehicle by using the beam power map configured by the beamforming technique and the plurality of image sensors.

However, in the case of sound field visualization of the noise source generated in the vehicle interior space, the image distortion does not coincide with the sound source prediction plane (square lattice) of the beam forming technique due to image distortion at the boundary between each image obtained from the image sensor. There is a problem that can not accurately display the position of, the intuitive determination of the sound pressure distribution of the noise source is difficult.

The present invention has been invented to solve the above problems, by expanding the image acquired through the fisheye camera using the error correction equation and then adjusting the size to match the grid of the sound source prediction plane, the sound source prediction on the beam forming technique It is an object of the present invention to provide a sound field display method through image distortion correction, which can accurately display the position of a sound source in accordance with a plane, and can intuitively and easily determine the sound pressure distribution of a noise source.

In order to achieve the above object, the sound field display method through the image distortion correction according to the present invention comprises the steps of acquiring an omnidirectional image of the vehicle interior space using a plurality of fisheye camera disposed in the vehicle interior space; Correcting a video image obtained from the fisheye camera using an error correction equation; And overlaying the corrected image on the sound source prediction plane according to the beamforming technique. The position of the sound source can be accurately displayed.

을 이용하여 어안 카메라로부터 취득한 이미지를 펼치는 단계; 상기 펼친 이미지를 음원예측평면의 격자에 맞게 크기조절하는 단계; 및 상기 보정된 이미지를 합성하는 단계;를 포함하여 이루어지고, 식에서 e(i,j)은 i번째 j번째 픽셀의 오차값, x는 이미지의 픽셀 x 방향 위치값, x_true 은 격자점 실제 x 방향 위치값, y는 이미지 픽셀 y 방향 위치값, y_true 은 격자점 실제 y 방향 위치값 인 것을 특징으로 한다.Here, the step of correcting the video image is an equation

Unfolding the image acquired from the fisheye camera using; Resizing the unfolded image to a grid of a sound source prediction plane; And synthesizing the corrected image, wherein e (i, j) is an error value of the i-th j-th pixel, x is a pixel x-direction position value of the image, x _true is a grid point actual x The direction position value, y is an image pixel y direction position value, and y _true is a lattice point actual y direction position value.

The fisheye camera 11 is arranged in a spherical array at regular intervals along the circumferential direction, so as to secure an omnidirectional image of the vehicle interior.

The advantages of the sound field display method through the image distortion correction according to the present invention are as follows.

First, after calculating the error value between the subject and the measured image pixels, the measured image can be corrected by using the calculated error value to accurately display the position of the sound source through the beamforming technique on the corrected image, and the existing measurement It is possible to improve the distortion problem of the image.

Second, the fisheye lens can be used to display the location of the noise source in a wider area than the conventional method using an image sensor.

For example, in the case of a competitor, 12 image sensors are used, but in the present invention, by using less than five image sensors, it is possible to reduce the cost by enabling omnidirectional sound field visibility.

Third, it is possible to visualize the omni-directional sound field with a small number of image sensors, which has the advantage of developing smaller-sized spherical array equipment than competitors.

Fourth, the sound field display method according to the prior art is incorrect display of the location of the noise source due to the distortion of the image edge portion, it is possible to accurately display the location of the noise through the corrected image and the overlay can greatly improve the intuitiveness.

1 is a block diagram illustrating a process for a beamforming technique according to the prior art.
Figure 2 is an image showing a problem of poor matching at the image boundary portion in the case of sound field visualization according to the prior art
3 is a front view and a plan view of a spherical array equipped with a fisheye camera according to an embodiment of the present invention.
4 is a flowchart illustrating a sound field display method through image distortion correction according to the present invention.
Fig. 5A shows lattice points arranged in a lattice form in the sound source prediction plane, and Fig. 5B is a schematic diagram showing lattice points obtained from an image sensor equipped with a fisheye lens.
6 is a schematic diagram for explaining a method of indicating a location of a sound source using a general beamforming technique;
Figure 7 is an image showing the interior space of the vehicle with sound field visualization according to the prior art
Figure 8 is an image showing the interior space of the vehicle with sound field visualization according to the present invention

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

FIG. 3 is a front view and a plan view of a spherical array equipped with a fisheye camera according to an embodiment of the present invention, and FIG. 4 is a flowchart showing a sound field display method through image distortion correction according to the present invention.

The present invention relates to a sound field display method through the image distortion correction that can accurately display the position of the sound source by correcting the distortion of the image acquired from the image sensor 11 when the sound field is visualized using the beamforming technique.

In order to capture and display a wider area of the room by using the image sensor as shown in FIG. 3, the fisheye cameras 11 are installed in the spherical array at intervals in the circumferential direction, and the fisheye cameras 11 are installed. Through this, the indoor interior of the vehicle may be simultaneously captured at an omnidirectional angle (360 degrees), thereby acquiring indoor indoor images of a wide area.

In this case, an angle that may be photographed by one fisheye camera 11 may be 120 degrees.

The present invention can be applied to the sound field visualization of the noise source generated in the interior space of the vehicle.

Sound field display method through the image distortion correction according to the present invention is a sound field visualization algorithm for receiving a signal from the microphone as shown in Figure 4 to configure the beam power map on the sound source prediction plane using a beamforming technique, and from the image sensor And an image correction algorithm for correcting the acquired image.

The sound field visualization algorithm measures the noise source through the microphone 12 mounted on the spherical array 10, receives the signal measured by the microphone 12, and receives the signal in the time domain through the FFT. The cross spectral matrix (A matrix), which is a signal in the frequency domain, is formed by converting to.

The beamforming technique adjusts a time delay between sound pressure signals received by each microphone 12 as shown in Equation 1, and estimates the position of the noise source by a delay-sum beam forming method.

는 빔파워, m은 마이크로폰의 인덱스, M은 총 마이크로폰의 개수, j는 허수, ω는 각주파수(rad/s), Δ_m(

)=

·

/c은 시간 지연,

은 음파의 진행방향,

은 기준점에서 m번째 마이크로폰까지의 거리, c는 공기 중에서의 음파 전파 속도를 의미한다.only,

Is the beam power, m is the microphone index, M is the total number of microphones, j is the imaginary number, ω is the angular frequency (rad / s), Δ _m (

) =

·

/ c is the time delay,

Is the direction of sound waves,

Is the distance from the reference point to the m-th microphone, and c is the speed of sound propagation in air.

FIG. 4 is a conceptual diagram illustrating a method of estimating the position of a sound source using a beamforming technique for a position of an actual noise extending in all directions.

(t)은 i번째 후보 음원과 j번째 마이크로폰(12) 사이의 위치 벡터,

(t)는 실제 음원의 위치 벡터,

(t)는 j번째 마이크로폰(12)의 위치 벡터이다.In this case, in FIG. 4, S (t) is monopole amplitude extending in all directions, Zi (t) is beam power at the i-th candidate sound source position, and Pj (t) is sound pressure measured at the j-th microphone 12. ,

(t) is the position vector between the i-th candidate sound source and the j-th microphone 12,

(t) is the position vector of the actual sound source,

(t) is the position vector of the j-th microphone 12.

As shown in FIG. 4, a steering vector (C vector) is formed through the relationship between the position of the microphone 12 and the position of the sound source prediction plane, and the A matrix and the steering vector C defining the signal characteristic relationship between the measured signals. The beam power map matrix is calculated by calculating the beam power map matrix and displayed on the sound source prediction plane to construct the beam power map representing the sound pressure distribution of the noise source.

Thereafter, the beam power map constructed by the sound field visualization algorithm may be overlaid on the indoor video image obtained from the image sensor to visually display the location of the noise source and the sound pressure distribution on the indoor video image.

However, when capturing an image of an indoor space through the fisheye camera 11, although it is possible to capture an omnidirectional area of the indoor image with a smaller number of cameras than the conventional technology, the five images obtained from the fisheye camera 11 are placed in one place. When placed side by side, distortion still occurs at the corners of each image.

For example, as shown in FIG. 5A, the grid points of the sound source prediction plane are points that have been previously determined when composing the beam power map using the beamforming technique, and the spacing of these grid points is a known value. It is constant.

In the image acquired through the fisheye camera 11 mounted on the spherical array 10 as shown in FIG. 5B, the lattice points are arranged to be curved in a circular direction toward both edges.

For the same reason as above, the image image of the fisheye camera 11 does not coincide with the sound source prediction plane on the sound field visualization algorithm (beam forming technique), so that the exact sound source position cannot be displayed.

In order to solve the above problems, in one embodiment of the present invention by sequentially correcting the image obtained from the image sensor 11 to match the sound source prediction plane in the beam forming technique, the matching between the interface between the acquired image You can display the correct sound source up.

The image acquired from the fisheye camera 11 is corrected five times by an error correction equation.

At this time, the acquired image takes the form of a grid of the expanded image image using the fisheye camera 11.

After calculating an error value between the subject and the measured image pixel according to Equation 2, the image obtained from the fisheye camera 11 is unfolded using the error value.

The forward size of the unfolded image is corrected in a manner that is adjusted to the lattice of the sound source prediction plane according to the beamforming technique.

Where e (i, j) is the error value of the i-th j-th pixel, x is the pixel x-direction position value of the image, x _true is the grid point actual x-direction position value, y is the image pixel y-direction position value, y _true Is the actual position of the grid point in the y direction.

The 360-degree image is composed of five images that are corrected sequentially.

Then, by overlaying the sound field visualization results on the corrected omnidirectional image image, the position of the sound source can be accurately displayed by matching the image image acquired from the fisheye camera 11 with the sound source prediction plane according to the beamforming technique.

Therefore, according to the present invention, the image acquired through the fisheye camera 11 is corrected by using an error correction equation to coincide with the sound source prediction plane according to the beam forming technique, so that the position of the sound source can be accurately displayed, and the existing measured image Can improve the distortion problem.

In addition, it is possible to display the location of the noise source in a wider area than the method using the image sensor 11 according to the prior art using the fisheye lens.

For example, in the case of a competitor, 12 image sensors are used, but in the present invention, by using five image sensors 11 smaller than that, it is possible to reduce the cost by omnidirectional sound field visibility.

In addition, since the omnidirectional sound field can be visualized with a small number of image sensors 11, there is an advantage in that a small-sized spherical array 10 equipment can be developed compared to competitors.

In addition, the sound field display method according to the prior art is an incorrect display of the location of the noise source due to the distortion of the image edge portion, but the accurate display of the location of the sound through the overlay of the corrected image and the sound source prediction plane enables sound pressure distribution and Intuition about position can be greatly improved.

5 is an image showing the sound field visualization of the vehicle interior space according to the prior art, Figure 6 is an image showing the sound field visualization of the vehicle interior space according to the present invention.

As shown, the sound field visualization image according to the present invention has better matching at the edge of the image than the conventional sound field visualization image, and the distortion of the image is improved.

10: spherical array 11: fisheye camera
12: microphone

Claims (3)

In the sound field display method for displaying the location of the noise source using a beam forming technique,
Acquiring an omnidirectional image image of the vehicle interior space in which the noise source is generated by using the plurality of fisheye cameras 11 arranged in the vehicle interior space;
Correcting the video image acquired from the fisheye camera (11) using an error correction equation;
Overlaying the corrected video image with a sound source prediction plane according to a beamforming technique;
Sound field display method through the image distortion correction, characterized in that comprises a.
The method according to claim 1,
Compensating the video image is formula

Unfolding the image acquired from the fisheye camera 11 using;
Resizing the unfolded image to a grid of a sound source prediction plane; And
Synthesizing the corrected image;
Where e (i, j) is the error value of the i-th j-th pixel, x is the pixel x-direction position value of the image, x _true is the grid point actual x-direction position value, y is the image pixel y-direction Position value, y _true is the sound field display method through the image distortion correction, characterized in that the lattice point actual y-direction position value.
The method according to claim 1,
The fisheye camera (11) is arranged in a spherical array at regular intervals along the circumferential direction, so as to secure the omnidirectional image of the vehicle interior, sound field display method through image distortion correction.

Images