CN110133663A - A kind of distributed acoustic is as joint calibration localization method - Google Patents

A kind of distributed acoustic is as joint calibration localization method Download PDF

Info

Publication number
CN110133663A
CN110133663A CN201910378075.6A CN201910378075A CN110133663A CN 110133663 A CN110133663 A CN 110133663A CN 201910378075 A CN201910378075 A CN 201910378075A CN 110133663 A CN110133663 A CN 110133663A
Authority
CN
China
Prior art keywords
camera
target
coordinate system
distance
air sonar
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910378075.6A
Other languages
Chinese (zh)
Other versions
CN110133663B (en
Inventor
项彬
马石磊
陈建峰
蔺贝
李晓强
温洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xi'an Lianfeng Xunsheng Information Technology Co Ltd
Original Assignee
Xi'an Lianfeng Xunsheng Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xi'an Lianfeng Xunsheng Information Technology Co Ltd filed Critical Xi'an Lianfeng Xunsheng Information Technology Co Ltd
Priority to CN201910378075.6A priority Critical patent/CN110133663B/en
Publication of CN110133663A publication Critical patent/CN110133663A/en
Application granted granted Critical
Publication of CN110133663B publication Critical patent/CN110133663B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/02Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems using reflection of acoustic waves
    • G01S15/06Systems determining the position data of a target
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/02Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems using reflection of acoustic waves
    • G01S15/06Systems determining the position data of a target
    • G01S15/08Systems for measuring distance only
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)

Abstract

The present invention provides a kind of distributed acoustics as joint calibration localization method, the contour degree of video camera and air sonar is mounted on above road, determine that air sonar at a distance from camera center, is constructed using air sonar array center and camera video center as the three-dimensional cartesian coordinate system of origin;Position coordinates of the target in air sonar coordinate system are determined by air sonar, and by two origins, transversely and horizontally distance and antitrigonometric function formula calculate the angle of target and horizontal plane and plummet face under camera coordinate system;Target corresponding pixel coordinate in camera views is calculated according to camera parameters;Acoustic target position is reflected in camera views by pixel coordinate, realizes sound image combined calibration positioning.The present invention can be realized the combination to city vision and the sense of hearing, provide enough technical supports for intelligent transportation and the development of intelligent city.

Description

A kind of distributed acoustic is as joint calibration localization method
Technical field
The invention belongs to field of signal processing, are related to sonar Wave beam forming, acoustical signal processing, sound image combined, target positioning Scheduling theory.
Background technique
With the progress of social science and technology, video camera and air sonar are gradually sent out in social fields such as traffic, security protection, environmental protection Wave increasingly important role.Worked together by video camera and air sonar, combination to video image and real-time audio and Joint calibration realizes the accurate positionin to audible target in video image.
Air sonar is usually the microphone array being made of multiple high-precision microphone modules, has acoustics positioning, sound The functions such as sound identification, sound enhancing.Since the eighties in last century, array signal process technique is widely used in sound source The research of Passive Positioning, but sound locating effect can not intuitively show existing city sensory devices due to being more abstracted In, it is therefore desirable to the specific location that could more intuitively reflect sound source is used in combination with camera chain.
Sonar positioning at present and video there is no preferable joint calibration method, air sonar and video camera point in practical application Cloth installation, since air sonar is not overlapped with camera center origin, causes air sonar locating effect to be difficult to accurately map To camera views, need artificially to carry out conversion label in video image, and precision is not accurate enough, cannot achieve automatic precision True alignment by union.
Summary of the invention
For overcome the deficiencies in the prior art, the present invention provides a kind of distributed acoustic as joint calibration method, realizes to city The combination of city's vision and the sense of hearing provides enough technical supports for intelligent transportation and the development of intelligent city, for vehicle in city The development that target is accurately positioned is of great importance with practical engineering application.
The technical solution adopted by the present invention to solve the technical problems is the following steps are included: by video camera and air sonar etc. Height is mounted on above road, determine air sonar at a distance from camera center, building with air sonar array center and Camera video center is the three-dimensional cartesian coordinate system of origin;Determine target in air sonar coordinate system by air sonar Position coordinates, by two origins transversely and horizontally distance and antitrigonometric function formula calculate under camera coordinate system target with The angle of horizontal plane and plummet face;Target corresponding pixel coordinate in camera views is calculated according to camera parameters;Pass through Acoustic target position is reflected in camera views by pixel coordinate, realizes sound image combined calibration positioning.
Step of the invention is unfolded, the following contents is specifically included:
Video camera is installed on transverse center position above road by the first step, and the height such as air sonar and video camera is installed, Three-dimensional cartesian coordinate system o is established using air sonar array center and camera video center as origin respectively1And o2, two origins Transversely and horizontally distance be d;
Second step determines target s in air sonar coordinate system o by air sonar Passive Positioning1In folder with horizontal plane Angle α, the angle β with mounting rod vertical plane, and position of the target in air sonar coordinate system is calculated by trigonometric function and is sat It marks (a, b, h), whereinH is the height of mounting rod;
Third step, find out target in camera coordinate system with the angle of horizontal planeAnd and mounting rod The angle of vertical plane
4th step determines that the picture pixel size of video camera is m × n, and the field angle for adjusting video camera is θ, measurement camera shooting The distance of bottom edge distance detection rod piece in machine visual field is d1
5th step calculates target Y-axis pixel coordinate in camera viewsWith in X-axis Pixel coordinateXl is centre distance left side road boundary in X-axis in formula Pixel distance, centered on xr on the right side of distance road boundary pixel distance, location of pixels of the road boundary in picture can table It is shown as straight line, kl is the slope of left boundary in this coordinate system, and kr is the slope of right side boundary in this coordinate system;
6th step is determined the reflection of acoustic target position on the image by pixel coordinate, realizes that sound image combined calibration is fixed Position.
The beneficial effects of the present invention are: can precise positioning acoustic target in video camera at the position in image, method Simple and practical, all kinds of acoustical signal targets on road can be reflected in the image of video camera by this method in real time, can be used for disobeying The fields such as method whistle vehicle location, suspicious object positioning, environment noise monitoring, for establishing intelligent transportation, improving city peace Anti-, construction smart city etc. has far reaching significance, has pushed Passive Positioning technology in the application in the fields such as traffic, security protection, monitoring With development.
Detailed description of the invention
Fig. 1 is method flow schematic block diagram of the invention;
Fig. 2 is air sonar and camera installation locations schematic diagram;
Fig. 3 is video camera scheme of installation;
Fig. 4 is monitored picture display schematic diagram.
Specific embodiment
Present invention will be further explained below with reference to the attached drawings and examples, and the present invention includes but are not limited to following implementations Example.
For in practical application when air sonar and video camera distribution are installed, air sonar and camera center origin It is not overlapped, air sonar locating effect is caused to be difficult to the problem of being mapped exactly to camera views, the present invention proposes a kind of distribution Formula Sound image localization joint calibration method, item of this method in known air sonar center origin position and camera installation locations Under part, calibrated by the calibration with acoustic target physical location to camera views, it can be quickly by air sonar orientation angle It is mapped on pixel corresponding to camera views, which can really reflect the tool of sound source in camera supervised region Body position is realized to the acoustic targets automatic and accurate alignment by union such as vehicle on road.
As shown in Figure 1, a kind of distributed acoustic proposed by the present invention is as joint calibration method, key step is as follows:
Step 1: determining air sonar at a distance from camera center
As shown in Fig. 2, video camera and air sonar are using distributed installation, video camera is installed on road transverse center position It sets, air sonar is located at beside video camera, establishes three using air sonar array center and camera video center as origin respectively Tieing up rectangular coordinate system is respectively o1And o2, the transversely and horizontally distance of two origins is d.
Step 2: air sonar positions
Determine target s in air sonar coordinate system o by air sonar Passive Positioning1In with the angle α of horizontal plane, with peace The angle β of bar vertical plane is filled, and position coordinates (a, b, h) of the target in air sonar coordinate system are calculated by trigonometric function
Wherein h is the height of mounting rod, that is, the z-axis coordinate of target.
Step 3: calculating the angle of the target under camera coordinate system
Video camera and air sonar are mounted on sustained height h, and by two origins, transversely and horizontally distance and antitrigonometric function are public Formula, find out target in camera coordinate system with the angle α of horizontal plane1With the angle β with mounting rod vertical plane1:
D is the transversely and horizontally distance of two coordinate origins in formula.
Step 4: determining the parameter of video camera
Camera views pixel size is m × n, and the field angle for adjusting video camera is θ, measures the bottom edge in camera field of view The distance of distance detection rod piece is d1, specific as shown in Figure 3.
Step 5: calculating target corresponding location of pixels in camera views
As shown in figure 4, corresponding location of pixels of the target in picture in Y-axis:
Target may be expressed as: in the pixel coordinate of X-direction
Xl is the pixel distance on road boundary on the left of centre distance in X-axis in formula, road boundary on the right side of distance centered on xr Pixel distance, location of pixels of the road boundary in picture be represented by straight line, and kl is left boundary in the coordinate Slope under system, kr are the slope of right side boundary in this coordinate system.
Step 6: sound image combined positioning
It is determined the reflection of acoustic target position on the image by pixel coordinate, realizes sound image combined calibration positioning.
The embodiment of the present invention the following steps are included:
Step 1: determining air sonar at a distance from camera center
Using distributed installation, video camera is installed on road transverse center position, air sonar for video camera and air sonar Beside video camera, three-dimensional rectangular coordinate is established using air sonar array center and camera video center as origin respectively System is respectively o1And o2, the transversely and horizontally distance of two origins is d.
In this coordinate system for any point s on road, all have with air sonar with camera coordinate system unique corresponding Point.Under three-dimensional system of coordinate, road plane take up an official post meaning one point s and origin o line vector beThen vectorIn plane Y0O0Z0Be projected asThen vectorIn plane Y0O0Z0Be projected asDefinition,Angle with y-axis is α,With x The angle of axis is β, then can uniquely confirm a point s by α, β.
Step 2: air sonar positions
Determine target s in air sonar coordinate system o by air sonar Passive Positioning1In with the angle α of horizontal plane and with The angle β of mounting rod vertical plane.And by trigonometric function calculate target in air sonar coordinate system position coordinates (a, b, h)
Wherein h is the height of mounting rod, that is, the z-axis coordinate of target.
Step 3: calculating the angle of the target under camera coordinate system
Video camera and air sonar are mounted on sustained height h, and by two origins, transversely and horizontally distance and antitrigonometric function are public Formula, find out target in camera coordinate system with the angle α of horizontal plane1With the angle β with mounting rod vertical plane1:
D is the transversely and horizontally distance of two coordinate origins in formula.
Step 4: determining the parameter of video camera
Camera views pixel size is m × n, and the field angle for adjusting video camera is θ, measures the bottom edge in camera field of view The distance of distance detection rod piece is d1, it is specific as shown in Figure 3:
Step 5: calculating target corresponding location of pixels in camera views:
Due to being correspondingly, for what is given under camera coordinate system between the pixel and field angle of camera views Arbitrary target position, it may be determined that corresponding location of pixels of the target in picture in Y-axis:
In X-direction, angle and pixel are not one-to-one, but distance is one-to-one with angle, it is assumed that sl table Show that actual range of the road left border apart from Y-axis, sr indicate actual range of the road right side boundary apart from Y-axis, then imaging Road left border is represented by the straight line of a vertical X axis under camera coordinate system in machine picture:
X=sl
Road right side boundary can also be expressed as the straight line of a vertical X axis
X=-sr
Coordinate system as shown in Figure 4 is started against in video camera road monitored picture, at this time target Y in picture The corresponding location of pixels conversion of axis are as follows:
n11=n-n1
Assuming that the center of camera views is O, the pixel distance on road boundary is on the left of centre distance in X-axis in picture The pixel distance of xl, centre distance right side road boundary are xr, it can be seen that location of pixels of the road boundary in picture can table It is shown as straight line, the slope of left boundary in this coordinate system is kl, and the slope of right side boundary in this coordinate system is kr.
Pixel boundary may be expressed as: in the X-axis of left side
Y=kl × x+b b=-kl × xl
Pixel boundary may be expressed as: in the X-axis of right side
Y=kr × x+b b=-kr × xr
Pixel distance xl, the pixel distance xr apart from right side road boundary on road boundary on the left of X-axis picture centre distance, Left boundary slope k l, the right boundary slope k r can demarcate acquisition in camera views.
In known Y-axis location of pixels n1Under conditions of, then it can be in the hope of the actual pixels distance on road boundary at this time.
Left border distance XleftIt indicates are as follows:
Right side boundary distance XrightIt indicates are as follows:
When coordinate of the target s under camera coordinate system be (a-d, b, h), may be expressed as: in the pixel coordinate of X-direction
That is:
Step 6: sound image combined positioning
It is determined the reflection of acoustic target position on the image by pixel coordinate, realizes sound image combined calibration positioning.

Claims (2)

1. a kind of distributed acoustic is as joint calibration localization method, it is characterised in that the following steps are included: by video camera and air-borne sound Equal height is mounted on above road, determines that air sonar at a distance from camera center, is constructed with air sonar array center And camera video center is the three-dimensional cartesian coordinate system of origin;Determine target in air sonar coordinate system by air sonar In position coordinates, by two origins transversely and horizontally distance and antitrigonometric function formula calculate under camera coordinate system target with The angle of horizontal plane and plummet face;Target corresponding pixel coordinate in camera views is calculated according to camera parameters;Pass through Acoustic target position is reflected in camera views by pixel coordinate, realizes sound image combined calibration positioning.
2. distributed acoustic according to claim 1 is as joint calibration localization method, it is characterised in that include the following steps:
Video camera is installed on transverse center position above road by the first step, and the height such as air sonar and video camera is installed, respectively Three-dimensional cartesian coordinate system o is established using air sonar array center and camera video center as origin1And o2, the cross of two origins It is d to horizontal distance;
Second step determines target s in air sonar coordinate system o by air sonar Passive Positioning1In with the angle α of horizontal plane, with The angle β of mounting rod vertical plane, and by trigonometric function calculate target in air sonar coordinate system position coordinates (a, b, H), whereinH is the height of mounting rod;
Third step, find out target in camera coordinate system with the angle of horizontal planeAnd with mounting rod vertical plane Angle
4th step determines that the picture pixel size of video camera is m × n, and the field angle for adjusting video camera is θ, measurement video camera view The distance of bottom edge distance detection rod piece in is d1
5th step calculates target Y-axis pixel coordinate in camera viewsWith in X-axis pixel CoordinateXl is the pixel on road boundary on the left of centre distance in X-axis in formula Distance, centered on xr on the right side of distance road boundary pixel distance, location of pixels of the road boundary in picture be represented by one Straight line, kl are the slope of left boundary in this coordinate system, and kr is the slope of right side boundary in this coordinate system;
6th step is determined the reflection of acoustic target position on the image by pixel coordinate, realizes sound image combined calibration positioning.
CN201910378075.6A 2019-05-08 2019-05-08 Distributed acoustic image joint calibration positioning method Active CN110133663B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910378075.6A CN110133663B (en) 2019-05-08 2019-05-08 Distributed acoustic image joint calibration positioning method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910378075.6A CN110133663B (en) 2019-05-08 2019-05-08 Distributed acoustic image joint calibration positioning method

Publications (2)

Publication Number Publication Date
CN110133663A true CN110133663A (en) 2019-08-16
CN110133663B CN110133663B (en) 2023-03-10

Family

ID=67576474

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910378075.6A Active CN110133663B (en) 2019-05-08 2019-05-08 Distributed acoustic image joint calibration positioning method

Country Status (1)

Country Link
CN (1) CN110133663B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110441737A (en) * 2019-08-26 2019-11-12 上海声茵科技有限公司 It is a kind of to use fish-eye sound localization method and its equipment
CN110672313A (en) * 2019-10-22 2020-01-10 上海声茵科技有限公司 Fault diagnosis method and equipment based on sound signals
CN110807901A (en) * 2019-11-08 2020-02-18 西安联丰迅声信息科技有限责任公司 Non-contact industrial abnormal sound detection method
CN111915918A (en) * 2020-06-19 2020-11-10 中国计量大学 System and method for calibrating automobile whistling snapshot device on site based on dynamic characteristics

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101101333A (en) * 2006-07-06 2008-01-09 三星电子株式会社 Apparatus and method for producing assistant information of driving vehicle for driver
CN101359043A (en) * 2008-09-05 2009-02-04 清华大学 Determining method for sound field rebuilding plane in acoustics video camera system
CN106886017A (en) * 2017-01-11 2017-06-23 浙江大学 Submarine target locus computational methods based on double frequency identification sonar
CN108051007A (en) * 2017-10-30 2018-05-18 上海神添实业有限公司 AGV navigation locating methods based on ultrasonic wave networking and stereoscopic vision
JP2018189463A (en) * 2017-05-01 2018-11-29 株式会社Soken Vehicle position estimating device and program
CN109479088A (en) * 2017-06-02 2019-03-15 深圳市大疆创新科技有限公司 The system and method for carrying out multiple target tracking based on depth machine learning and laser radar and focusing automatically

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101101333A (en) * 2006-07-06 2008-01-09 三星电子株式会社 Apparatus and method for producing assistant information of driving vehicle for driver
CN101359043A (en) * 2008-09-05 2009-02-04 清华大学 Determining method for sound field rebuilding plane in acoustics video camera system
CN106886017A (en) * 2017-01-11 2017-06-23 浙江大学 Submarine target locus computational methods based on double frequency identification sonar
JP2018189463A (en) * 2017-05-01 2018-11-29 株式会社Soken Vehicle position estimating device and program
CN109479088A (en) * 2017-06-02 2019-03-15 深圳市大疆创新科技有限公司 The system and method for carrying out multiple target tracking based on depth machine learning and laser radar and focusing automatically
CN108051007A (en) * 2017-10-30 2018-05-18 上海神添实业有限公司 AGV navigation locating methods based on ultrasonic wave networking and stereoscopic vision

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
贺静: "基于计算机视觉的目标二维坐标定位实现", 《现代计算机(专业版)》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110441737A (en) * 2019-08-26 2019-11-12 上海声茵科技有限公司 It is a kind of to use fish-eye sound localization method and its equipment
CN110672313A (en) * 2019-10-22 2020-01-10 上海声茵科技有限公司 Fault diagnosis method and equipment based on sound signals
CN110807901A (en) * 2019-11-08 2020-02-18 西安联丰迅声信息科技有限责任公司 Non-contact industrial abnormal sound detection method
CN110807901B (en) * 2019-11-08 2021-08-03 西安联丰迅声信息科技有限责任公司 Non-contact industrial abnormal sound detection method
CN111915918A (en) * 2020-06-19 2020-11-10 中国计量大学 System and method for calibrating automobile whistling snapshot device on site based on dynamic characteristics

Also Published As

Publication number Publication date
CN110133663B (en) 2023-03-10

Similar Documents

Publication Publication Date Title
CN110133663A (en) A kind of distributed acoustic is as joint calibration localization method
US12033388B2 (en) Positioning method, apparatus, device, and computer-readable storage medium
CN106651990B (en) Indoor map construction method and indoor positioning method based on indoor map
US20180003498A1 (en) Visual positioning system and method based on high reflective infrared identification
CN106383596B (en) Virtual reality anti-dizzy system and method based on space positioning
CN107884767A (en) A kind of method of binocular vision system measurement ship distance and height
CN104064055B (en) A kind of inland river navigation boats and ships superelevation detects early warning system and method for work thereof
CN105798917B (en) A kind of Community Safety alarm method and patrol robot
CN110009682B (en) Target identification and positioning method based on monocular vision
JP2014098683A (en) Method for remotely measuring crack
CN103292710A (en) Distance measuring method applying binocular visual parallax error distance-measuring principle
CN106443650A (en) Monocular vision range finding method based on geometric relation
CN109212540A (en) Distance measuring method, device and readable storage medium storing program for executing based on laser radar system
CN101701814A (en) Method for judging spatial position of target by linkage of multi-cameras and system thereof
CN208953962U (en) A kind of robot tracking control and robot
CN111435081A (en) Sea surface measuring system, sea surface measuring method and storage medium
CN104391273B (en) A kind of visible ray localization method based on circular projection and system
TW201329426A (en) Camera testing device and test method thereof
CN105894511A (en) Calibration target setting method and device and parking auxiliary system
CN106033614A (en) Moving object detection method of mobile camera under high parallax
CN112068567A (en) Positioning method and positioning system based on ultra-wideband and visual image
JP2021117087A (en) Information processing apparatus, detector, road side unit, and calibration method
CN110261825A (en) Bistatic Long baselines hydrolocation method and system
CN103884332B (en) A kind of barrier decision method, device and mobile electronic device
CN107132525A (en) Submarine target locus computational methods based on two vertically arranged identification sonars

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A Distributed Audio Image Joint Calibration and Localization Method

Granted publication date: 20230310

Pledgee: Xi'an Caijin Financing Guarantee Co.,Ltd.

Pledgor: LIANFENG ACOUSTIC TECHNOLOGIES Co.,Ltd.

Registration number: Y2024980011038

PE01 Entry into force of the registration of the contract for pledge of patent right