CN114463179A - Closed space multi-camera image splicing method for sound source positioning judgment - Google Patents
Closed space multi-camera image splicing method for sound source positioning judgment Download PDFInfo
- Publication number
- CN114463179A CN114463179A CN202210125158.6A CN202210125158A CN114463179A CN 114463179 A CN114463179 A CN 114463179A CN 202210125158 A CN202210125158 A CN 202210125158A CN 114463179 A CN114463179 A CN 114463179A
- Authority
- CN
- China
- Prior art keywords
- camera
- coordinate system
- sound source
- microphone array
- closed space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims abstract description 22
- 230000009466 transformation Effects 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 3
- 238000011835 investigation Methods 0.000 description 3
- 230000004807 localization Effects 0.000 description 2
- 206010067484 Adverse reaction Diseases 0.000 description 1
- 208000017667 Chronic Disease Diseases 0.000 description 1
- 230000006838 adverse reaction Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
- Studio Devices (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Abstract
The invention relates to the technical field of acoustic sound source positioning, in particular to a method for splicing images of multiple cameras in a closed space for sound source positioning judgment, which acquires noise space information through a microphone array, calibrates the space information between a microphone and the cameras, acquires the images on the upper, lower, left and right sides of the cameras, deduces pixel coordinates of the noise sound source position on the images according to the noise sound source space position information, a rotation matrix and a translation vector between a microphone coordinate system and a camera coordinate system by combining a calibration result, and displays the pixel coordinates on the image splicing result of the multiple cameras. And transmitting the position to the nearest camera, and displaying the noise position in the fused image.
Description
Technical Field
The invention relates to the technical field of acoustic sound source positioning, in particular to a method for splicing images of multiple cameras in a closed space for sound source positioning judgment.
Background
The noise not only damages the hearing of human ears, but also causes adverse reaction to the physiological function of human body, and the long-term exposure in the strong noise environment can reduce the health level of the human body and induce various chronic diseases.
In the prior art, a method for judging the position of a noise source in a closed space is insufficient, and the noise in the closed space cannot be accurately judged and visually checked, so that certain trouble is caused to the noise investigation.
Therefore, a method for splicing multiple cameras in a closed space for sound source positioning judgment needs to be designed, a coordinate transformation relation between a sound source position and an image is calculated, and the sound source position is mapped to a spliced multiple-camera image plane, so that a basis can be provided for the noise sound source investigation in the closed space, and the method has an important actual reference value.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method for splicing images of multiple cameras in a closed space for sound source positioning judgment.
In order to achieve the above object, the present invention provides a method for stitching closed-space multi-camera images for sound source localization determination, comprising the following steps:
s1: acquiring noise sound source space information of a closed space through a microphone array;
the sound source space information is obtained by calculating the distance d between two adjacent microphones and the time difference delta t of signals received by the two adjacent microphones;
calculating L-v-delta t according to the propagation velocity v of sound in the air; l is the sound path difference;
thus, the angle theta of the microphone receiving the noise signal is calculated to be arcsin (L/d), and the position of the noise source is determined;
s2: calibrating spatial information between the microphone array and the camera system;
the world coordinate system coordinate of a certain point P in the closed space is PwThe coordinate of P in the camera coordinate system is PcAnd the coordinate of P in the coordinate system of the microphone array is Pm(ii) a Then P isw、Pc、PmThe coordinate transformation relation of (2) is as follows:
Pc＝R1Pw+t1
Pm＝R2Pw+t2
(R1，t1) As rotation matrix and translation vector between world coordinate system and camera coordinate system, (R)2，t2) A rotation matrix and a translation vector between a world coordinate system and a microphone array coordinate system are obtained; according to the above two formulas, the transformation matrix relation of a certain point P in the closed space between the camera coordinate system and the microphone array coordinate system is calculated as follows:
s3: an image splicing method based on multiple cameras in a closed space;
the images acquired by the camera are acquired at the upper, lower, left and right sides of the position of the camera;
forming a display interface of the spliced image at the periphery between the camera and the image;
the upper, lower, left and right points of the camera and the 4 connecting lines corresponding to the upper, lower, left and right points of the display interface are the joints of the image splicing area;
a triangular area formed by the display interface, the 4 connecting lines and the camera is represented by a scene from a long scene to a short scene under different scale factors, and the long scene or the short-field scene from the camera is transformed into a unified coordinate system by applying different scale factors;
s4: and deducing pixel coordinates of the noise sound source position on the image according to the spatial position information of the noise sound source acquired in S1 and S2, the rotation matrix and the translation vector between the microphone array coordinate system and the camera coordinate system and the calibration result of the camera, and displaying the pixel coordinates on the image splicing result of the multi-camera image according to S3.
Compared with the prior art, the invention takes images in a closed space through a plurality of cameras and carries out image fusion, and obtains the spatial position information of noise according to a sound source positioning method and the information on the images obtained by the corresponding cameras to calculate a corresponding transformation matrix, thereby providing a corresponding image mark for the determined sound source positioning, transmitting the position to the nearest camera and displaying the noise position in the fused image.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention.
FIG. 2 is a schematic diagram of the system of the present invention for locating a sound source.
FIG. 3 is a schematic diagram of image stitching with multiple cameras according to the present invention.
Detailed Description
The invention will now be further described with reference to the accompanying drawings.
Referring to fig. 1 to 3, the invention provides a method for splicing images of multiple cameras in a closed space for sound source localization judgment, which comprises the following steps:
s1: acquiring noise sound source space information of a closed space through a microphone array;
the sound source space information is obtained by calculating the distance d between two adjacent microphones and the time difference delta t of signals received by the two adjacent microphones;
calculating L-v-delta t according to the propagation velocity v of sound in the air; l is the sound path difference;
thus, the angle theta of the microphone receiving the noise signal is calculated to be arcsin (L/d), and the position of the noise source is determined;
s2: calibrating spatial information between the microphone array and the camera system;
the world coordinate system coordinate of a certain point P in the closed space is PwThe coordinate of P in the camera coordinate system is PcAnd the coordinate of P in the coordinate system of the microphone array is Pm(ii) a Then P isw、Pc、PmThe coordinate transformation relation of (2) is as follows:
Pc＝R1Pw+t1
Pm＝R2Pw+t2
(R1，t1) As rotation matrix and translation vector between world coordinate system and camera coordinate system, (R)2，t2) A rotation matrix and a translation vector between a world coordinate system and a microphone array coordinate system are obtained; according to the above two formulas, the transformation matrix relation of a certain point P in the closed space between the camera coordinate system and the microphone array coordinate system is calculated as follows:
s3: an image splicing method based on multiple cameras in a closed space;
the images acquired by the camera are acquired at the upper, lower, left and right sides of the position of the camera;
forming a display interface of the spliced image at the periphery between the camera and the image;
the upper, lower, left and right points of the camera and the 4 connecting lines corresponding to the upper, lower, left and right points of the display interface are the joints of the image splicing area;
a triangular area formed by the display interface, the 4 connecting lines and the camera is represented by a scene from a long scene to a short scene under different scale factors, and the long scene or the short-field scene from the camera is transformed into a unified coordinate system by applying different scale factors;
s4: and deducing pixel coordinates of the noise sound source position on the image according to the spatial position information of the noise sound source acquired in S1 and S2, the rotation matrix and the translation vector between the microphone array coordinate system and the camera coordinate system and the calibration result of the camera, and displaying the pixel coordinates on the image splicing result of the multi-camera image according to S3.
Example (b):
step 1: as shown in fig. 2, noise source space information of a closed space is acquired by a microphone array, wherein the noise source space information includes a distance d between two adjacent microphones; the time difference Δ t of the signals received by two adjacent microphones is calculated according to the propagation speed v of sound in the air, so that L is v · Δ t; l is the difference of the sound path, so that the angle θ of the noise signal received by the microphone is calculated as arcsin (L/d), thereby determining the position of the noise source.
Step 2: calibrating spatial information between the microphone array and the camera system, if the coordinate of the world coordinate system of a certain point P in the closed space is PwIts coordinate in the camera coordinate system is PcAnd the coordinate of the point in the coordinate system of the microphone array is Pm. Then P isw、Pc、PmThe coordinate transformation relation of (2) is as follows:
Pc＝R1Pw+t1
Pm＝R2Pw+t2
wherein (R)1，t1) As rotation matrix and translation vector between world coordinate system and camera coordinate system, (R)2，t2) For a rotation matrix and a translation between a world coordinate system and a microphone array coordinate systemAmount of the compound (A). According to the above two formulas, the transformation matrix relation of a certain point P in the closed space between the camera coordinate system and the microphone array coordinate system can be estimated as follows:
wherein
Representing the rotation matrix and translation vector between the microphone array coordinate system and the camera coordinate system.
And step 3: as shown in fig. 3, in the image stitching method based on multiple cameras in an enclosed space, a black square marked as 0123 in the figure represents a device for installing the cameras in the enclosed space, and four ellipses represent images acquired by the cameras and respectively correspond to images acquired by front, rear, left and right cameras of the device. And region 4567 represents the display interface of the entire stitched image, and the four diagonal lines 04, 15, 26, 37 represent the image stitching region junctions. In the four areas 0154, 1265, 2376 and 0374, the corresponding images are represented by scenes from a far scene to a near scene under different scale factors, and the aim is to apply different scale factors to transform the far scene or the near scene of the distance camera into a unified coordinate system.
And 4, step 4: and (3) according to the spatial position information of the noise sound source obtained in the steps (1) and (2), the rotation matrix and the translation vector between the microphone array coordinate system and the camera coordinate system, and the calibration result of the camera, the pixel coordinates of the noise sound source position on the image can be deduced, and then according to the step (3), the pixel coordinates are displayed on the multi-camera image splicing result.
The above are only preferred embodiments of the present invention, and are only used to help understanding the method and the core idea of the present application, the scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.
The invention integrally solves the technical problems that the method for judging the position of the noise sound source in the closed space in the prior art is insufficient, the noise in the closed space cannot be accurately judged and visually checked, and certain trouble is caused to the noise investigation.
Claims (1)
1. A closed space multi-camera image splicing method for sound source positioning judgment is characterized by comprising the following steps:
s1: acquiring noise sound source space information of a closed space through a microphone array;
the sound source space information is obtained by calculating the distance d between two adjacent microphones and the time difference delta t of signals received by the two adjacent microphones;
calculating L-v-delta t according to the propagation velocity v of sound in the air; l is the sound path difference;
thus, the angle theta of the microphone receiving the noise signal is calculated to be arcsin (L/d), and the position of the noise source is determined;
s2: calibrating spatial information between the microphone array and the camera system;
the world coordinate system coordinate of a certain point P in the closed space is PwThe coordinate of the P in the camera coordinate system is PcThe coordinate of the P in the coordinate system of the microphone array is Pm(ii) a Then P isw，Pc、PmThe coordinate transformation relation of (2) is as follows:
Pc＝R1Pw+t1
Pm＝R2Pw+t2
said (R)1，t1) Is a rotation matrix and a translation vector between a world coordinate system and a camera coordinate system, the (R)2，t2) A rotation matrix and a translation vector between a world coordinate system and a microphone array coordinate system are obtained; according to the above two formulas, the relation of a transformation matrix of a certain point P in the closed space between a camera coordinate system and a microphone array coordinate system is calculated as follows:
the above-mentioned
The rotation matrix and the translation vector between the microphone array coordinate system and the camera coordinate system are obtained;
s3: an image splicing method based on multiple cameras in a closed space;
the images acquired by the camera are acquired at the upper, lower, left and right sides of the position of the camera;
forming a display interface of a spliced image at the periphery between the camera and the image;
the upper, lower, left and right points of the camera and the 4 connecting lines corresponding to the upper, lower, left and right points of the display interface are the joints of the image splicing area;
a triangular area formed by the display interface, the 4 connecting lines and the camera is represented by a scene from a long scene to a short scene under different scale factors of the image, and the long scene or the short-field scene from the camera is transformed into a unified coordinate system by applying different scale factors;
s4: and deducing pixel coordinates of the noise sound source position on the image according to the spatial position information of the noise sound source acquired in the steps S1 and S2, and the rotation matrix and the translation vector between the microphone array coordinate system and the camera coordinate system and by combining a camera calibration result, and displaying the pixel coordinates on a multi-camera image splicing result according to the step S3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210125158.6A CN114463179A (en) | 2022-02-10 | 2022-02-10 | Closed space multi-camera image splicing method for sound source positioning judgment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210125158.6A CN114463179A (en) | 2022-02-10 | 2022-02-10 | Closed space multi-camera image splicing method for sound source positioning judgment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114463179A true CN114463179A (en) | 2022-05-10 |
Family
ID=81412689
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210125158.6A Pending CN114463179A (en) | 2022-02-10 | 2022-02-10 | Closed space multi-camera image splicing method for sound source positioning judgment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463179A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102543099A (en) * | 2010-12-24 | 2012-07-04 | 索尼公司 | Sound information display device, sound information display method, and program |
| CN102629372A (en) * | 2012-02-22 | 2012-08-08 | 北京工业大学 | 360 degree panoramic aerial view generation method used for assisting vehicle driving |
| CN103871071A (en) * | 2014-04-08 | 2014-06-18 | 北京经纬恒润科技有限公司 | Method for camera external reference calibration for panoramic parking system |
| US20170099460A1 (en) * | 2015-10-05 | 2017-04-06 | Polycom, Inc. | Optimizing panoramic image composition |
| CN109982038A (en) * | 2019-03-15 | 2019-07-05 | 深圳市沃特沃德股份有限公司 | Show the method, apparatus and computer equipment of sound source position |
-
2022
- 2022-02-10 CN CN202210125158.6A patent/CN114463179A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102543099A (en) * | 2010-12-24 | 2012-07-04 | 索尼公司 | Sound information display device, sound information display method, and program |
| CN102629372A (en) * | 2012-02-22 | 2012-08-08 | 北京工业大学 | 360 degree panoramic aerial view generation method used for assisting vehicle driving |
| CN103871071A (en) * | 2014-04-08 | 2014-06-18 | 北京经纬恒润科技有限公司 | Method for camera external reference calibration for panoramic parking system |
| US20170099460A1 (en) * | 2015-10-05 | 2017-04-06 | Polycom, Inc. | Optimizing panoramic image composition |
| CN109982038A (en) * | 2019-03-15 | 2019-07-05 | 深圳市沃特沃德股份有限公司 | Show the method, apparatus and computer equipment of sound source position |
Non-Patent Citations (1)
| Title |
|---|
| 温俊杰 等: "声场可视化系统中声像阵列空间关系标定研究", 《仪器仪表学报》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11386572B2 (en) | Calibration system and method to align a 3D virtual scene and a 3D real world for a stereoscopic head-mounted display | |
| US20210058608A1 (en) | Method and apparatus for generating three-dimensional (3d) road model | |
| CN106679651B (en) | Sound localization method, device and electronic equipment | |
| JP4757142B2 (en) | Imaging environment calibration method and information processing apparatus | |
| EP1596329B1 (en) | Marker placement information estimating method and information processing device | |
| US7397481B2 (en) | Image display method and image display system | |
| US7529387B2 (en) | Placement information estimating method and information processing device | |
| EP3465391B1 (en) | Digital camera with audio, visual and motion analysis | |
| Inoue et al. | Visualization system for sound field using see-through head-mounted display | |
| JP2004127239A (en) | Method and system for calibrating multiple cameras using calibration object | |
| JP2017106749A (en) | Point group data acquisition system and method thereof | |
| CN112655024A (en) | Image calibration method and device | |
| CN111461963B (en) | Fisheye image stitching method and device | |
| JP2010117211A (en) | Laser radar installation position verification apparatus, laser radar installation position verification method, and program for laser radar installation position verification apparatus | |
| CN111243034A (en) | Panoramic auxiliary parking calibration method, device, equipment and storage medium | |
| Kunz et al. | Stereo self-calibration for seafloor mapping using AUVs | |
| KR101816068B1 (en) | Detection System for Vehicle Surroundings and Detection Method for Vehicle Surroundings Using thereof | |
| CN113920191B (en) | 6D data set construction method based on depth camera | |
| CN114463179A (en) | Closed space multi-camera image splicing method for sound source positioning judgment | |
| JP2005141655A (en) | Three-dimensional modeling apparatus and three-dimensional modeling method | |
| CN115327480A (en) | Sound source positioning method and system | |
| CN112422848A (en) | Video splicing method based on depth map and color map | |
| CN116309071B (en) | Panoramic image seamless splicing method | |
| CN111080718A (en) | Camera module calibration method and device for 720-degree environment detection | |
| JP2008140047A (en) | Information processing method and information processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |