CN101699219B - Method and sensor for binocular vision 3D measurement - Google Patents
Method and sensor for binocular vision 3D measurement Download PDFInfo
- Publication number
- CN101699219B CN101699219B CN2009102369912A CN200910236991A CN101699219B CN 101699219 B CN101699219 B CN 101699219B CN 2009102369912 A CN2009102369912 A CN 2009102369912A CN 200910236991 A CN200910236991 A CN 200910236991A CN 101699219 B CN101699219 B CN 101699219B
- Authority
- CN
- China
- Prior art keywords
- signal processing
- video camera
- processing unit
- digital signal
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses a method and a sensor for binocular vision 3D measurement, and belongs to the technical field of computer vision. The sensor for the binocular vision 3D measurement comprises a binocular camera and a control circuit board, wherein the binocular camera comprises a left camera and a right camera; and in the control circuit board, two analog-to-digital converting units are connected with the two cameras respectively, a field programmable gate array unit is connected with the two analog-to-digital converting units and a digital signal processing unit respectively, and the digital signal processing unit is connected with a memory unit. A 3D measurement algorithm solidified on the digital signal processing unit can realize the 3D coordinate calculation of a vision target. The method and the sensor can improve the speed of binocular vision 3D measurement and avoid the defects of large volume, high power consumption and expensive price caused by using a PC and an industrial computer for running the binocular vision 3D measurement algorithm.
Description
Technical field
The present invention relates to technical field of computer vision, particularly the method for binocular vision 3 D measurement and sensor.
Background technology
One of research direction of computer vision field is obtained the three-dimensional information of interested target or environment exactly the image information of the two dimension of taking from video camera.By operation binocular vision 3 D measurement algorithm two dimensional image is carried out three-dimensional measurement, can from two dimensional image, obtain the three-dimensional information of target or environment.Utilize the binocular vision 3 D measurement algorithm to obtain three-dimensional information and be widely used in fields such as security monitoring, traffic monitoring, virtual reality, robot autonomous navigation.Wherein, all be to move the binocular vision 3 D measurement algorithm at present by PC or industrial controlled machine, the two dimensional image that video camera is taken carries out three-dimensional measurement, obtains the three-dimensional information of target or environment.
In realizing process of the present invention, the inventor finds that there is following problem at least in prior art:
PC needs the operating system support, therefore, and when operation binocular vision 3 D measurement algorithm, operating system need take a certain amount of resource, makes the speed of three-dimensional measurement be difficult to satisfy the demands, in addition, PC or industrial computer have again that volume is big, power consumption is big, the shortcoming that price is more expensive relatively.
Summary of the invention
For the speed that can improve three-dimensional measurement and avoid the shortcoming that volume is big, power consumption is big and price is expensive that adopts PC and industrial computer operation binocular vision 3 D measurement algorithm to bring, the embodiment of the invention provides a kind of method and sensor of binocular vision 3 D measurement.Described technical scheme is as follows:
A kind of sensor of binocular vision 3 D measurement, described sensor comprises:
Binocular camera and control circuit board, two video cameras about described binocular camera comprises, in described control circuit board, embed and comprise two-way AD conversion unit, field programmable gate array unit, digital signal processing unit and storage unit, arbitrary road in the described two-way AD conversion unit links to each other with left video camera, another road links to each other with right video camera, described field programmable gate array unit links to each other with described digital signal processing unit with described two-way AD conversion unit respectively, and described digital signal processing unit links to each other with described storage unit;
Described left and right cameras is used for same object is gathered simultaneously the analog picture signal of being gathered separately;
Described two-way AD conversion unit is used for obtaining described analog picture signal from described left and right cameras, and converts described analog picture signal to two width of cloth digital pictures;
Described field programmable gate array unit is used for obtaining described two width of cloth digital pictures from described two-way AD conversion unit, and described two width of cloth digital pictures is carried out filtering, gives described digital signal processing unit with two width of cloth Digital Image Transmission of filtering;
RAM between program and data random-access memory when program and data flash memory space FLASH when described storage unit comprises inoperative, operation;
Described FLASH is used to store binocular vision 3 D measurement algorithm and binocular calibrating procedure;
Described RAM is used to store the program that described digital signal processing unit duplicates out from described FLASH, and the data of described program generation when operation;
Described digital signal processing unit, be used for obtaining described binocular calibrating procedure from described FLASH, and described binocular camera is demarcated by moving described binocular calibrating procedure, obtain left video camera matrix and the described right video camera matrix of described binocular camera under world coordinate system, obtain two width of cloth digital pictures of filtering from described field programmable gate array unit, and obtain the binocular vision 3 D measurement algorithm from described storage unit, according to left video camera matrix and right video camera matrix, the target in described two width of cloth images is carried out three-dimensional measurement by moving described binocular vision 3 D measurement algorithm.
Described sensor also comprises: digital I/O unit, link to each other with described digital signal processing unit, and be used to import outside switching value.
Described sensor also comprises:
Network communication unit links to each other with described digital signal processing unit, is used to realize that described digital signal processing unit communicates by letter with outer net.
Described sensor also comprises:
Display unit links to each other with described digital signal processing unit, is used to show the result of three-dimensional measurement.
Described sensor also comprises:
Serial communication unit links to each other with described digital signal processing unit, is used to realize that described digital signal processing unit and host computer communicate.
A kind of method of binocular vision 3 D measurement, described method comprises:
Left side video camera and right video camera carry out image acquisition at synchronization respectively, obtain analog image information;
Two-way analog-converted unit converts described analog picture signal to digital picture;
The field programmable gate array unit is divided into square with described digital picture respectively, and described square is carried out filtering, filters out the part noise in the square, more described filtered square is formed two filtered digital pictures;
Digital signal processing unit is according to left video camera matrix and right video camera matrix, and operation binocular vision 3 D measurement algorithm carries out three-dimensional measurement to the target in the described filtered digital picture.
Described digital signal processing unit is according to left video camera matrix and right video camera matrix, and operation binocular vision 3 D measurement algorithm carries out the target in the described filtered digital picture also comprising before the three-dimensional measurement:
Described digital signal processing unit operation calibrating procedure carries out the binocular demarcation to described left video camera and described right video camera, obtains described left video camera matrix and described right video camera matrix.
By two-way AD conversion unit, field programmable gate array unit, digital signal processing unit and storage unit all are embedded in the control circuit board, make the volume of binocular vision sensor become very little, reduced power consumption and price; The digital signal processing unit computing power is powerful, does not need the support of operating system, can directly move the binocular vision 3 D measurement algorithm, has improved the speed of three-dimensional measurement.
Description of drawings
Fig. 1 is a kind of binocular vision sensor structural representation that the embodiment of the invention 1 provides;
Fig. 2 is a kind of two-way AD conversion unit that provides of the embodiment of the invention 1 and the connection diagram of binocular camera;
Fig. 3 is a kind of field programmable gate array cell schematics that the embodiment of the invention 1 provides;
Fig. 4 is the connection diagram between the embodiment of the invention 1 digital signal processing unit, network communication unit and the display unit that provide;
Fig. 5 is a kind of FLASH space synoptic diagram that the embodiment of the invention 1 provides;
Fig. 6 is a kind of ram space synoptic diagram that the embodiment of the invention 1 provides;
Fig. 7 is a kind of serial communication unit synoptic diagram that the embodiment of the invention 1 provides;
Fig. 8 is the method flow diagram of a kind of binocular vision 3 D measurement of providing of the embodiment of the invention 2;
Fig. 9 is the synoptic diagram of the square objects A that provides of the embodiment of the invention 2;
Figure 10 be the collection that provides of the embodiment of the invention 2 about two width of cloth image synoptic diagram;
Figure 11 is the method flow diagram of a kind of demarcation of providing of the embodiment of the invention 3;
Figure 12 is a kind of scaling board synoptic diagram that the embodiment of the invention 3 provides.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, embodiment of the present invention is described further in detail below in conjunction with accompanying drawing.
As shown in Figure 1, the embodiment of the invention provides a kind of binocular vision sensor, comprises binocular camera 1 and control circuit board 2 two parts:
Wherein, binocular camera 1 comprises left video camera 11 and right video camera 12, and left video camera 11 and right video camera 12 take same object simultaneously, and wherein, left video camera 11 and right video camera 12 photograph analog picture signal separately.
Wherein, be embedded in AD conversion unit 21, AD conversion unit 22, field programmable gate array unit 23, digital signal processing unit 24 and storage unit 25 at control circuit board 2.AD conversion unit 21 links to each other with the left video camera 11 that binocular camera 1 comprises, AD conversion unit 22 links to each other with the right video camera 12 that binocular camera 1 comprises, and AD conversion unit 21 all links to each other with field programmable gate array unit 23 with 22, field programmable gate array unit 23 links to each other with digital signal processing unit 24, and digital signal processing unit 24 links to each other with storage unit 25 again.
AD conversion unit 21 and AD conversion unit 22 are obtained analog picture signal respectively from left video camera 11 and right video camera 12, convert the analog picture signal that obtains separately to two width of cloth digital pictures.
Field programmable gate array unit 23 obtains two width of cloth digital pictures from AD conversion unit 21 and AD conversion unit 22, move the image preprocessor of self two width of cloth digital pictures are carried out filtering, filter out the part noise in two width of cloth digital pictures, give digital signal processing unit 24 two width of cloth Digital Image Transmission of filtering.
Two width of cloth digital pictures of digital signal processing unit 24 receiving filtrations, from storage unit 25, obtain the binocular vision 3 D measurement algorithm, according to left video camera matrix and right shooting matrix, the binocular vision 3 D measurement algorithm that operation is obtained carries out three-dimensional measurement to the target in two width of cloth images.
The data that storage unit 25 is used to store binocular vision 3 D measurement algorithm, system configuration information, generate during by digital signal processing unit 24 working procedures.
Further, control circuit board 2 also comprises digital I/O unit 26, and wherein, digital I/O unit 26 links to each other with digital signal processing unit 24, is used to import outside switching value.Can carry out the data communication of switching value with the outside by digital I/O unit.
Further, control circuit board 2 also comprises network communication unit 27, and wherein, network communication unit 27 links to each other with digital signal processing unit 24, is used for communicating by letter with outer net, can make binocular sensor and outer net carry out data interaction.
Further, control circuit board 2 also comprises display unit 28, and wherein, display unit 28 links to each other with digital signal processing unit 24, is used to show image, duty, the result of demarcation and the result of three-dimensional measurement of collection.
Further, control circuit board 2 also comprises serial communication unit 29, and wherein, serial communication unit 29 links to each other with digital signal processing unit, is used for communicating by letter with host computer, makes binocular vision sensor and host computer carry out data interaction.
Further, storage unit 25 also is used to store calibrating procedure;
Correspondingly, digital signal processing unit 24 also is used for obtaining calibrating procedure from storage unit 25, by the operation calibrating procedure binocular camera 1 is demarcated, and obtains left video camera matrix and right video camera matrix.
Be illustrated in figure 2 as the annexation figure of binocular camera 1 and AD conversion unit 21 and AD conversion unit 22.AD conversion unit 21 links to each other with right video camera 12 with the left video camera 11 of binocular camera 1 respectively with AD conversion unit 22.
AD conversion unit 21 is obtained PAL/NTSC (Phase Alternating Line/National Television System Committee from left video camera 11, inverted image quadrature amplitude balance system/quadrature amplitude balance system line by line) analog picture signal of standard, and convert this analog picture signal to digital picture.AD conversion unit 21 is at video control clock line F_VP0C[0:3] in the effect of clock signal under, by video data line F_VP0D[0:9] give field programmable gate array unit 23 with Digital Image Transmission, wherein, video control clock line F_VP0C[0:3] clock signal provide by field programmable gate array unit 23.
AD conversion unit 22 is obtained the analog picture signal of PAL/NTSC standard from right video camera 12, and converts this analog picture signal to digital picture.AD conversion unit 22 is at video control clock line F_VP1C[0:3] in the effect of clock signal under, by video data line F_VP1D[0:9] give field programmable gate array unit 23 with Digital Image Transmission, wherein, video control clock line F_VP1C[0:3] clock signal provide by field programmable gate array unit 23.
Wherein, AD conversion unit comprises modulus conversion chip or picture decoding chip etc., and AD conversion unit 21 and 22 all needs power supply to power and crystal oscillator provides clock, and power supply can provide 1.8v and two kinds of voltages of 3.3v.
Be illustrated in figure 3 as field programmable gate array unit 23 synoptic diagram in the control circuit board 2 that present embodiment provides.Field programmable gate array unit 23 comprises online download jtag2 interface, in-circuit emulation jtag1 interface, gate array configuration flash memory E2PROM, high speed signal processing unit configuration flash memory E2PROM, four way word output interface FPGA_OUT[1:4] and four way word input interface FPGA_IN[1:4].
Wherein, online download jtag2 interface is used for the download images preprocessor, and program burn writing is disposed flash memory E2PROM to gate array, and wherein, this gate array configuration flash memory E2PROM can preserve the image preprocessor under the situation of outage.
In-circuit emulation jtag1 interface is used for the image preprocessor is carried out on-line debugging.
Field programmable gate array configuration flash memory E2PROM is used to store the image preprocessor that online download jiag2 interface is downloaded.
High speed signal processing unit configuration flash memory E2PROM is used for storage system configuration information.
Field programmable gate array unit 23 is by video control clock line F_VP0C[0:3] and video data line F_VP0D[0:9] from AD conversion unit 21, obtain digital picture, by video control clock line F_VP1C[0:3] and video data line F_VP1D[0:9] from AD conversion unit 22, obtain digital picture, the reading images preprocessor is carried out filtering to two width of cloth digital pictures of obtaining from gate array configuration flash memory E2PROM, filter out the part noise, again by video control line VP0C[0:4] and VP0C[0:4] and video data line VP0D[0:9] and VP0D[0:9] digital signal processing unit 24 given with filtered two width of cloth Digital Image Transmission.
The detailed process that field programmable gate array unit 23 carries out filtering by the operation image preprocessor to two width of cloth digital pictures is: because the processing procedure that two width of cloth digital pictures are handled is identical, for convenience of explanation, only the processing procedure at wherein a width of cloth digital picture describes, its processing procedure is specially: from this digital picture is divided into several squares, by FPGA (Field Programmable Gate Array, field programmable gate array) hardware programming produces CPU (the Central Processing Unit of respective amount, central authorities handle junior unit), concurrently each square is carried out filtering, filter out the part noise in the square, the square that filters is assembled into filtered image, wherein, another width of cloth digital picture is also carried out filtering by above-mentioned same method, and field programmable gate array unit 23 is simultaneously two width of cloth digital pictures to be carried out filtering.
Wherein, field programmable gate array unit 23 filters out the part noise in two width of cloth digital pictures, can share the part work of high-speed digital signal processing unit 24, improves the speed that the binocular vision 3 D measurement sensor carries out three-dimensional measurement.
Data in the field programmable gate array unit 23 also can be passed through data bus AED[0:31], address bus AEA[3:22], byte gate bus ABE[0:3] visited by digital signal processing unit 24.
Field programmable gate array unit 23 has disposed four way word output interface FPGA_OUT[1:4] and four way word input interface FPGA_IN[1:4], be used to realize that field programmable gate array unit 23 is mutual with the switching value data of external unit, then through data bus AED[0:31], address bus AEA[3:22] and byte gate bus ABE[0:3] visited by digital signal processing unit 24.
Field programmable gate array unit 23 needs power supply power supply and crystal oscillator that clock is provided.
The connection diagram of the digital signal processing unit 24 in the control circuit board 2 that Fig. 4 provides for the embodiment of the invention, network communication unit 27, display unit 28.
Digital signal processing unit 24 comprises that three video ports are respectively 0 video port, 1 video port and 2 video ports, online download and emulation interface jtag, universal I/O port GP0[4:15].
Wherein, online download and emulation interface jtag are used to download calibrating procedure and binocular vision 3 D measurement algorithm, and the program of downloading is carried out on-line debugging.
0 video port is by video control signal line VP0C[0:4] and video data line VP0D[0:9] link to each other with field programmable gate array unit 23, be used to transmit a filtered digital picture.
1 video port is by video control signal line VP1C[0:4] and video data line VP1D[0:9] link to each other with field programmable gate array unit 23, be used to transmit another filtered digital picture.
2 video ports are by video control signal line VP2C[0:4] and video data line VP2D[0:9] link to each other with display unit 28, be used for the result that digital signal processing unit 24 produces is shown.
Universal I/O port GP0[4:15], in FPGA, decipher the generation chip selection signal.Among Fig. 4, provide clock at crystal oscillator, power supply provides under required 3.3V of work and the effect of 1.4V voltage, digital signal processing unit 24 is by address bus AEA[3:22], data bus AED[0:63], byte enable bus ABE[0:7] and universal I/O port GP0[4:15] the outer peripheral hardware resource of visit sheet, for example the field programmable gate array unit 24, storage unit 28 and serial communication unit 29.Wherein, supply voltage is by watch dog monitoring, in case power supply occurs unusual or when powering on, the house dog reset pin that will set out makes digital signal processing unit 24 enter reset mode, and is normal up to power up.
Digital signal processing unit 24 links to each other with network communication unit 27 by the network data control line, thereby can realize and outside network service.
Network communication unit 27 comprises network physical interface chip and RJ45 joint, be used to realize high-speed digital signal processing unit 24 and outside network service, the network layer communication agreement is TCP/IP (Transmission Control Protocol/Internet Protocol, a transmission control protocol/the Internet interconnection protocol).
Display unit 28 comprises video coding chip and VGA interface.Video coding chip is by video control line VP2C[0:4] and video data line VP2D[0:9] link to each other with 2 video ports of digital signal processing unit 24, be used for converting analog picture signal to, and pass through the demonstration of VGA interface handling an image result.
In addition, digital signal processing unit also needs the outer storage unit 25 of sheet.Storage unit 25 comprises FLASH (flash memory space of program and data during inoperative), RAM (the random-access memory space of program and data during operation).
The structural representation of the FLASH that provides for present embodiment as shown in Figure 5.Flash capacity among Fig. 5 is the 32Mb capacity, by 8 position datawire AED[0:7], address bus AEA[3:22], chip selection signal CE1 links to each other with digital signal processing unit 24.Wherein, FLASH is used to store binocular vision 3 D measurement algorithm and calibrating procedure.
The structural representation of the RAM that provides for present embodiment as shown in Figure 6.RAM capacity among Fig. 6 is formed 32MB by RAM1 and the RAM2 of two 16MB, by 64 position datawire AED[0:63], address bus AEA[3:22], chip selection signal CE0, and byte enable signal ABE[0:7] link to each other with digital signal processing unit 24.The RAM type is SDRAM, and required clock is provided by digital signal processing unit 24.
In addition, digital signal processing unit 24 also links to each other with serial communication unit 29.The structural representation of the serial communication unit 29 that provides for present embodiment as Fig. 7.Serial communication unit 29 is by address bus AEA[3:5] with address sum AEA[0:7] communicate by letter with digital signal processing unit.Serial communication unit 29 adopts and changes serial connection mouthful chip realization and string conversion, and the serial signal of the Transistor-Transistor Logic level after the conversion converts RS485/RS232 form level to by two 485/322 physical interface chip, realizes serial communication.The serial ports chip selection signal of field programmable gate array unit 23 links to each other with the choosing of the serial ports sheet of serial communication unit 29, the serial ports that is used for gating serial communication unit 29. serial communication unit 29 interrupts linking to each other with the serial ports look-at-me of field programmable gate array unit 23, sends the look-at-me of serial communication to field programmable gate array unit 23.
In embodiments of the present invention, AD conversion unit 21 and 22, field programmable gate array unit 23, digital signal processing unit 24 and storage unit 25 all are embedded in the control circuit board 2, make the volume of binocular vision sensor become very little, reduced power consumption and price; Digital signal processing unit 24 computing powers are powerful, do not need the support of operating system, can directly move the binocular vision 3 D measurement algorithm, improved the speed of three-dimensional measurement; Adopt the 23 pairs of images in field programmable gate array unit to carry out parallel filtering, further improve the speed of three-dimensional measurement.
Embodiment 2
As shown in Figure 8, the sensor of the binocular vision 3 D measurement that provides based on embodiment 1, the embodiment of the invention provides a kind of method of binocular vision 3 D measurement, comprising:
Step 101: collect the two-way analog picture signal about 1 pair of same object of binocular camera carries out;
Particularly, left video camera 11 and right video camera 12 that binocular camera 1 comprises are gathered same object simultaneously, obtain analog picture signal separately.
For example, utilize left video camera 11 that binocular camera 1 comprises and right video camera 12 simultaneously as shown in Figure 9 square objects A to be gathered, obtain two-way analog picture signal S1 and S2.
Step 102: the AD conversion unit 21 and 22 in the control circuit board 2 is obtained the two-way analog picture signal respectively from left video camera 11 and right video camera 12, and the analog picture signal that will obtain separately respectively converts two width of cloth digital pictures to, is transferred to field programmable gate array unit 23;
Wherein, AD conversion unit 21 is at video control clock line F_VP0C[0:3] in the effect of clock signal under, by video data line F_VP0D[0:9] give field programmable gate array unit 23 with Digital Image Transmission, wherein, video control clock line F_VP0C[0:3] clock signal provide by field programmable gate array unit 23; AD conversion unit 22 is at video control clock line F_VP1C[0:3] in the effect of clock signal under, by video data line F_VP1D[0:9] give field programmable gate array unit 23 with Digital Image Transmission, wherein, video control clock line F_VP1C[0:3] clock signal provide by field programmable gate array unit 23.
For example, AD conversion unit 21 and 22 in the control circuit board 2 is obtained analog picture signal S1 and S2 respectively from left video camera 11 and right video camera 12, and convert analog picture signal S1 and S2 to digital picture D1 and D2 respectively, simultaneously digital picture D1 and D2 are transferred out field programmable gate array unit 23.
Step 103: field programmable gate array unit 23 receives two width of cloth digital pictures, and digital picture is carried out filtering, gives digital signal processing unit 24 with filtered two width of cloth Digital Image Transmission;
Particularly, field programmable gate array unit 23 receives two width of cloth digital pictures from AD conversion unit 21 and 22 transmission, at a width of cloth digital picture wherein, the square that this digital picture is divided into some, field programmable gate array unit 23 generates the CPU identical with square quantity by hardware programming, parallel carries out filtering to each square, filters out the part noise in the square, and the square with filtering is reassembled into filtered digital picture again.Field programmable gate array unit 23 also carries out filtering in the same way to another width of cloth digital picture, and simultaneously two width of cloth digital pictures is carried out filtering.
For example, field programmable gate array unit 23 receives digital picture A1 and the A2 from AD conversion unit 21 and 22 transmission, shown in image 10, respectively A1 and A2 are divided into a plurality of squares, each square is carried out filtering, filter out the part noise in the square, again the square of filtering is reformulated filtered digital picture A1 and A2, and filtered digital picture A1 and A2 are transferred to digital signal processing unit 24.
In addition, at this moment, the digital signal processing unit 24 of control circuit board 2 obtains the binocular vision 3 D measurement algorithm from storage unit 25, begins to carry out following steps this two width of cloth image is carried out three-dimensional measurement.
Step 104: digital signal processing unit 24 receives two width of cloth digital pictures;
For example, digital signal processing unit 24 receptions are from two width of cloth digital picture A1 and A2 of field programmable gate array unit 23.
Step 105: digital signal processing unit 24 obtains angle point in all that the target in every width of cloth digital picture comprises respectively from two width of cloth digital pictures;
Wherein, interior angle point is the intersection point on many limits in the digital picture, and particularly, the scanning digital image is determined in the target intersection point on all two or more limit, with the intersection point determined as interior angle point.Wherein, a real intersection point in all corresponding object of angle point in each.
For example, hypothetical target is square objects A, scanning two width of cloth digital picture A1 and A2 as shown in figure 10, all interior angle points that obtain the target in the A1 image are respectively the E1 of B1, C1, D1, and all interior angle points that obtain the target in the A2 image are respectively the E2 of B2, C2, D2.Wherein, the corresponding real intersection points B of interior angle point B1 and B2, the corresponding real intersection point C of interior angle point C1 and C2, the corresponding real intersection point D of interior angle point D1 and D2, the corresponding real intersection point E of interior angle point E1 and E2.
Step 106: digital signal processing unit 24 adopts existing matching algorithm, to about interior angle point in two width of cloth digital pictures mate, corners Matching is right in obtaining;
Wherein, matching algorithm is according to the gradient feature around the interior angle point, to about interior angle point in two width of cloth images mate, coupling is to the same intersection point in two interior angle point homologue bodies that comprise.
For example, adopt matching algorithm that two width of cloth images shown in 10 figure are mated, the interior corners Matching that obtains is to being respectively＜B1 B2 〉,＜C1, C2 〉,＜D1, D2〉and＜E1, E2 〉, and＜B1, B2 〉,＜C1, C2 〉,＜D1, D2〉and＜E1, E2〉intersection points B, C, D and the E among counterparty's shaped object A respectively.
Step 107: to the image coordinate that comprises angle point in each, left video camera matrix and right video camera matrix, obtain real intersection point the world coordinates in world coordinate system of this coupling to correspondence according to coupling.
Particularly, with the image coordinate m that mates the two interior angle points that comprise
1And m
r, and left and right cameras matrix P
1And P
rBe brought into formula (1), calculate the world coordinates M of coupling the real intersection point of correspondence.
Wherein, in step 107, obtain each coupling the world coordinates of the real intersection point of correspondence is the three-dimensional information that finally obtains.In addition, video camera is just set up an image coordinate system for the image of each video camera when images acquired, and therefore, each pixel in the image all has an image coordinate in image coordinate system.
For example, for the right＜B1 of coupling, B2 〉, according to the image coordinate of interior angle point B1 and B2, left video camera matrix and right video camera matrix are by formula (1) acquisitions＜B1, B2〉world coordinates of true intersection points B of correspondence.
In embodiments of the present invention, digital signal processing unit 24 does not need the support of operating system, can directly move the binocular vision 3 D measurement algorithm, has improved the speed of three-dimensional measurement; Adopt the 23 pairs of images in field programmable gate array unit to carry out filtering, can improve the speed of three-dimensional measurement.
As shown in figure 11, the embodiment of the invention provides a kind of method of demarcation, and wherein, left video camera matrix among the embodiment 2 and right video camera matrix are that the scaling method that provides by present embodiment is in advance demarcated binocular camera 1 and obtained, and this method comprises:
Step 201: digital signal processing unit 24 is determined the world coordinates in the alive boundary of the angle point coordinate system in each in the scaling board according to the scaling board parameter;
Scaling board as shown in figure 12 is made up of the black and white grid, and the scaling board parameter comprises the position of each grid on the directions X, the position of each grid on the Y direction and the grid length of side.The summit of each grid is interior angle point.According to the length of side of grid, and the position of this grid on position on the directions X and Y direction, determine the world coordinates of four interior angle points that this grid comprises.
Step 202: utilize left video camera 11 to gather several scaling board images from different perspectives, digital signal processing unit 24 scans every width of cloth images and read the image coordinate of angle point in all from every width of cloth image;
Wherein, left video camera has the coordinate system of self, and this coordinate system is based upon in the image of collection, and making all has an image coordinate system in the image of gathering, and makes that so each point in the image all has image coordinate.So far, obtain a world coordinates and the image coordinate of angle point in each.
Step 203: digital signal processing unit 24 by the Levenberg-Marquardt iterative algorithm, obtains the intrinsic parameter matrix A of left video camera according to the world coordinates and the image coordinate of angle point in each
1
Wherein, directly obtain the intrinsic parameter of left video camera, comprise the focal distance f of left video camera at directions X by the Levenberg-Marquardt iterative algorithm
X1, the focal distance f of left video camera on the Y direction
Y1, left video camera the coordinate (cx of photocentre in image coordinate system
1, cy
1), again these intrinsic parameters are formed the intrinsic parameter matrix A of left video camera
1, as shown in Equation (2).
So far, finished the demarcation of machine that a left side is taken the photograph.
Step 204: utilize right video camera to gather several scaling board images from different perspectives, digital signal processing unit 24 every width of cloth images of scanning and the image coordinate of angle point in all in the reading images from image;
Wherein, right video camera has the coordinate system of self, and this coordinate system is based upon in the image of collection, and making all has an image coordinate system in the image of gathering, and makes that so each point in the image all has image coordinate.So far, obtain a world coordinates and the image coordinate of angle point in each.
Step 205: digital signal processing unit 24 according to obtain from step 204 each in the world coordinates and the image coordinate of angle point, by the Levenberg-Marquardt iterative algorithm, obtain the intrinsic parameter matrix A of right video camera
r
Wherein, directly obtain the intrinsic parameter of right video camera, comprise the focal distance f of right video camera at directions X by the Levenberg-Marquardt iterative algorithm
Xr, the focal distance f of right video camera on the Y direction
Yr, the coordinate (cx in the coordinate system of the photocentre of right video camera in himself
r, cy
r), again these intrinsic parameters are formed right video camera matrix A
r, as shown in Equation (3).
So far, finished right Camera calibration.Wherein, can also demarcate right video camera earlier in the present embodiment, more left video camera be demarcated.
Step 206: two width of cloth scaling board images about utilizing left video camera 11 and right video camera 12 to gather respectively simultaneously, two width of cloth images about digital signal processing unit 24 scanning obtain the image coordinate of angle points in all that left image comprises and the image coordinate of right image comprises in all angle points;
Step 207: digital signal processing unit 24 is according to the world coordinates and the image coordinate of angle point in the scaling board all, by the Levenberg-Marquardt iterative algorithm, the outer parameter rotation matrix R and the translation matrix t of two video cameras about acquisition, as shown in Equation (4);
Step 208: digital signal processing unit 24 is according to the intrinsic parameter matrix A 1 of left video camera, the intrinsic parameter matrix A r of right video camera, and the outer parameter rotation matrix R and the translation matrix t of left and right cameras set up left video camera matrix P
1With right video camera matrix P
r
Particularly, the coordinate of regulation left side video camera is a world coordinate system, the left video camera matrix P of foundation
1With right video camera matrix P
r, as shown in Equation (5):
Or the coordinate of the right video camera of regulation is a world coordinate system, the left video camera matrix P of foundation
1With right video camera matrix P
r, as shown in Equation (6):
Wherein,
All or part of content in the technical scheme that above embodiment provides can realize that its software program is stored in the storage medium that can read by software programming, storage medium for example: the hard disk in the computing machine, CD or floppy disk.
The above only is preferred embodiment of the present invention, and is in order to restriction the present invention, within the spirit and principles in the present invention not all, any modification of being done, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (7)
1. the sensor of a binocular vision 3 D measurement is characterized in that, described sensor comprises:
Binocular camera and control circuit board, two video cameras about described binocular camera comprises, in described control circuit board, embed and comprise two-way AD conversion unit, field programmable gate array unit, digital signal processing unit and storage unit, arbitrary road in the described two-way AD conversion unit links to each other with left video camera, another road links to each other with right video camera, described field programmable gate array unit links to each other with described digital signal processing unit with described two-way AD conversion unit respectively, and described digital signal processing unit links to each other with described storage unit;
Described left and right cameras is used for same object is gathered simultaneously the analog picture signal of being gathered separately;
Described two-way AD conversion unit is used for obtaining described analog picture signal from described left and right cameras, and converts described analog picture signal to two width of cloth digital pictures;
Described field programmable gate array unit is used for obtaining described two width of cloth digital pictures from described two-way AD conversion unit, and described two width of cloth digital pictures is carried out filtering, gives described digital signal processing unit with two width of cloth Digital Image Transmission of filtering;
Program and data random-access memory space RAM when program and data flash memory space FLASH when described storage unit comprises inoperative, operation;
Described FLASH is used to store binocular vision 3 D measurement algorithm and binocular calibrating procedure;
Described RAM is used to store the program that described digital signal processing unit duplicates out from described FLASH, and the data of described program generation when operation;
Described digital signal processing unit, be used for obtaining described binocular calibrating procedure from described FLASH, and described binocular camera is demarcated by moving described binocular calibrating procedure, obtain left video camera matrix and the described right video camera matrix of described binocular camera under world coordinate system, obtain two width of cloth digital pictures of filtering from described field programmable gate array unit, and obtain the binocular vision 3 D measurement algorithm from described storage unit, according to left video camera matrix and right video camera matrix, the target in described two width of cloth images is carried out three-dimensional measurement by moving described binocular vision 3 D measurement algorithm.
2. sensor as claimed in claim 1 is characterized in that, described sensor also comprises: digital I/O unit, link to each other with described digital signal processing unit, and be used to import outside switching value.
3. sensor as claimed in claim 1 is characterized in that, described sensor also comprises:
Network communication unit links to each other with described digital signal processing unit, is used to realize that described digital signal processing unit communicates by letter with outer net.
4. sensor as claimed in claim 1 is characterized in that, described sensor also comprises:
Display unit links to each other with described digital signal processing unit, is used to show the result of three-dimensional measurement.
5. sensor as claimed in claim 1 is characterized in that, described sensor also comprises:
Serial communication unit links to each other with described digital signal processing unit, is used to realize that described digital signal processing unit and host computer communicate.
6. the method for a binocular vision 3 D measurement is characterized in that, described method comprises:
Left side video camera and right video camera carry out image acquisition at synchronization respectively, obtain analog image information;
Two-way analog-converted unit converts described analog picture signal to digital picture;
The field programmable gate array unit is divided into square with described digital picture respectively, and described square is carried out filtering, filters out the part noise in the square, more described filtered square is formed two filtered digital pictures;
Digital signal processing unit is according to left video camera matrix and right video camera matrix, and operation binocular vision 3 D measurement algorithm carries out three-dimensional measurement to the target in the described filtered digital picture.
7. as method as described in the claim 6, it is characterized in that, described digital signal processing unit is according to left video camera matrix and right video camera matrix, and operation binocular vision 3 D measurement algorithm carries out the target in the described filtered digital picture also comprising before the three-dimensional measurement:
Described digital signal processing unit operation calibrating procedure carries out the binocular demarcation to described left video camera and described right video camera, obtains described left video camera matrix and described right video camera matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102369912A CN101699219B (en) | 2009-10-30 | 2009-10-30 | Method and sensor for binocular vision 3D measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102369912A CN101699219B (en) | 2009-10-30 | 2009-10-30 | Method and sensor for binocular vision 3D measurement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101699219A CN101699219A (en) | 2010-04-28 |
| CN101699219B true CN101699219B (en) | 2011-01-26 |
Family
ID=42147681
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102369912A Expired - Fee Related CN101699219B (en) | 2009-10-30 | 2009-10-30 | Method and sensor for binocular vision 3D measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101699219B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102095413B (en) * | 2009-12-11 | 2012-08-29 | 中国科学院沈阳自动化研究所 | Double-path cursor real-time measurement device |
| CN102175251A (en) * | 2011-03-25 | 2011-09-07 | 江南大学 | Binocular intelligent navigation system |
| CN102445323B (en) * | 2011-09-27 | 2014-07-30 | 浙江中控太阳能技术有限公司 | Image processing-based heliostat fault diagnosis method and system |
| CN103134444B (en) * | 2013-02-01 | 2015-07-29 | 同济大学 | The varifocal three-dimension measuring system of double-view field |
| CN107205145A (en) * | 2016-03-17 | 2017-09-26 | 中航华东光电（上海）有限公司 | Terminal guidance video image three dimensional data collection system |
| CN108225502B (en) * | 2017-12-06 | 2019-08-23 | 中南大学 | A kind of truck loads ore quality estimation method and system |
| CN110361865A (en) * | 2018-04-10 | 2019-10-22 | 幻视互动（北京）科技有限公司 | A kind of retina light field display MR mixed reality glasses accelerating digital pupil signal transmission |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1566900A (en) * | 2003-06-11 | 2005-01-19 | 北京航空航天大学 | Vision measuring method for spaced round geometrical parameters |
| CN1616920A (en) * | 2003-11-14 | 2005-05-18 | 北京理工大学 | Active real-time three-dimensional positioning system based on binocular vision and laser distance detection |
| CN1971206A (en) * | 2006-12-20 | 2007-05-30 | 北京航空航天大学 | Calibration method for binocular vision sensor based on one-dimension target |
| CN101030318A (en) * | 2007-03-16 | 2007-09-05 | 廊坊智通机器人系统有限公司 | Signal processing system in measurement technology |
| CN101561270A (en) * | 2009-05-27 | 2009-10-21 | 天津理工大学 | Embedded omnidirectional ball vision object detection and mobile monitoring system and embedded omnidirectional ball vision object detection and mobile monitoring method |
-
2009
- 2009-10-30 CN CN2009102369912A patent/CN101699219B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1566900A (en) * | 2003-06-11 | 2005-01-19 | 北京航空航天大学 | Vision measuring method for spaced round geometrical parameters |
| CN1616920A (en) * | 2003-11-14 | 2005-05-18 | 北京理工大学 | Active real-time three-dimensional positioning system based on binocular vision and laser distance detection |
| CN1971206A (en) * | 2006-12-20 | 2007-05-30 | 北京航空航天大学 | Calibration method for binocular vision sensor based on one-dimension target |
| CN101030318A (en) * | 2007-03-16 | 2007-09-05 | 廊坊智通机器人系统有限公司 | Signal processing system in measurement technology |
| CN101561270A (en) * | 2009-05-27 | 2009-10-21 | 天津理工大学 | Embedded omnidirectional ball vision object detection and mobile monitoring system and embedded omnidirectional ball vision object detection and mobile monitoring method |
Non-Patent Citations (2)
| Title |
|---|
| JP特开平5-164517A 1993.06.29 |
| 周文晖等.《基于FPGA的双目立体视觉系统》.《中国图象图形学报》.2005,第10卷(第9期),1166-1170. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101699219A (en) | 2010-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101699219B (en) | Method and sensor for binocular vision 3D measurement | |
| CN110677599A (en) | System and method for reconstructing 360-degree panoramic video image | |
| CN104820418B (en) | A kind of embedded vision system and its application method for mechanical arm | |
| CN106023302A (en) | Mobile communication terminal, three-dimensional reconstruction method thereof and server | |
| CN110264563A (en) | A kind of Octree based on ORBSLAM2 builds drawing method | |
| CN101916429A (en) | Geometric correction and disparity extraction device of binocular camera | |
| CN102507592A (en) | Fly-simulation visual online detection device and method for surface defects | |
| CN103161133A (en) | Intelligent road-cleaning vehicle based on machine vision and controlling method thereof | |
| CN111337037B (en) | Mobile laser radar slam drawing device and data processing method | |
| CN104794713A (en) | Greenhouse crop digital-imaging method based on ARM and binocular vision | |
| CN108352060A (en) | Method and apparatus for generating the data for indicating pixel beam | |
| TWM565860U (en) | Smart civil engineering information system | |
| CN113284233A (en) | Visual monitoring method, device, system, electronic device and storage medium | |
| CN101567078B (en) | Dual-bus visual processing chip architecture | |
| Feng et al. | Semi-automatic 3d reconstruction of piecewise planar building models from single image | |
| CN117437357A (en) | Model construction method and device, nonvolatile storage medium and electronic equipment | |
| CN115294277B (en) | Three-dimensional reconstruction method and device of object, electronic equipment and storage medium | |
| CN109343713B (en) | Human body action mapping method based on inertial measurement unit | |
| CN111292411A (en) | Real-time dynamic human body three-dimensional reconstruction method based on inward looking-around multiple RGBD cameras | |
| CN102801998A (en) | Portable polarimetric image collecting and processing system | |
| CN212515897U (en) | Active and passive three-dimensional imaging real-time processing system | |
| CN107274449B (en) | Space positioning system and method for object by optical photo | |
| CN201256433Y (en) | Tridimensional mechanical vision device based on bionic design | |
| CN107678329A (en) | A kind of panorama interconnected control systems and control method | |
| Drofova et al. | Use of scanning devices for object 3D reconstruction by photogrammetry and visualization in virtual reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20171018 Address after: 100081, Beijing Haidian District Zhongguancun South Street 5, two, 204, a layer 114 Patentee after: BEIJING HARIBIT INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 100081 intelligent robot research institute, Institute of Aerospace Science, Beijing Institute of Technology, 5 South Avenue, Haidian District, Beijing, Zhongguancun Patentee before: BEIJING INSTITUTE OF TECHNOLOGY |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110126 Termination date: 20211030 |