CN114359358A - Image registration method for area-array camera and structured light camera - Google Patents
Image registration method for area-array camera and structured light camera Download PDFInfo
- Publication number
- CN114359358A CN114359358A CN202111654719.3A CN202111654719A CN114359358A CN 114359358 A CN114359358 A CN 114359358A CN 202111654719 A CN202111654719 A CN 202111654719A CN 114359358 A CN114359358 A CN 114359358A
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- area
- array
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000003287 optical effect Effects 0.000 claims abstract description 14
- 230000004927 fusion Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 9
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 abstract description 8
- 238000012937 correction Methods 0.000 abstract description 2
- 238000009877 rendering Methods 0.000 abstract description 2
- 230000001360 synchronised effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Abstract
The invention discloses a registration method for collected images of an area-array camera and an optical structure camera, which is characterized in that an area-array image and a line-scanning image are synchronously collected, the characteristic that the horizontal and longitudinal pixel representation distance of the area-array image is stable is applied, the first line and the last line of the area-array camera are subjected to pixel fusion with the specific line of the line-scanning image, and then the image is zoomed until the agreed pixels represent the distance image, so that the one-to-one correspondence of the area-array image and the line-scanning image is realized. The one-to-one correspondence engineering value is realized: 1) when the area-array camera adopts color imaging and the line scanning camera adopts structured light depth imaging, color rendering can be carried out on each pixel of a depth image to obtain a true color three-dimensional model; 2) for the registered planar array image and line scanning image pair, the homing correction can be carried out by utilizing the imaging non-dislocation and distortion characteristics of the planar array camera and combining the line scanning image with methods such as feature matching and the like, so that the target distortion phenomenon caused by the uneven travelling path of the line scanning camera is eliminated.
Description
Technical Field
The invention relates to the field of visual sensors, in particular to a registration method for images acquired by an area-array camera and an optical structure camera.
Background
At present, a conventional visual sensor mainly comprises two types of surface scanning and line scanning, wherein a surface scanning type camera can perform integral imaging once, and the pixels of an image represent stable parameters such as distance, distortion and the like and can accurately perform measurement such as size and the like. The original image data of the simultaneously captured surface scanning type camera and the structured light camera have no one-to-one corresponding relation due to the difference of imaging mechanisms, and the multi-element data fusion needs two types of images to be in one-to-one correspondence at the pixel level, so that a simple, reliable and high-precision registration method is needed.
Disclosure of Invention
Aiming at the problems, the invention provides an image registration method for an area-array camera and a structured light camera, which has the advantages of accurately corresponding the pixels of an area-array image and a structured light image and improving the reliability of registration.
The technical scheme of the invention is as follows:
an image registration method for an area-array camera and a structured light camera comprises the following steps:
s1, simultaneously using an area-array camera and a structured light camera to collect images at the same position;
s2, defining the transverse and longitudinal resolution of the image collected by the area-array camera as S1Picture size of h1×w1Defining the image of the structured light camera as each J line, and each horizontal pixel as w2Transverse resolution of s2；
S3, obtaining a structural light camera image corresponding to the view field area of the ith area-array camera image;
s4, carrying out pixel fusion on the area-array camera image and the optical structure camera image which correspond to each other, wherein the pixel fusion comprises structured light image zooming processing and image cutting processing, and the registration is completed after the pixel fusion;
the scaling parameters are as follows:
the vertical scaling parameters are:
the lateral scaling parameters are:
the registered structured light image size becomes:
the image cutting processing formula is as follows:
1) when in use
When the structure light registration image is cut into the width w of the image by bilateral symmetry1；
2) When in use
When the surface array image is cut into the width of the image by two-side symmetry
3) When in use
In the process, the area array image and the structural light registration image are not processed;
the area-array camera and the corresponding structured light image realize pixel-to-pixel correspondence with the center of the image;
and N represents the distance by a single pulse of the encoder, and represents the actual advancing distance of the vehicle by outputting 1 pulse by the encoder.
In S3, the first row image of the ith area-array camera corresponds to the image sequence number of the structured light camera
The row number is
For rounding up the symbol, rem () is the remainder symbol, and the serial number of the last row image of the ith area-array camera corresponding to the structural optical camera image is
The row number is
Generating images according to the increasing sequence of data of all lines between the image lines of the structured light camera corresponding to the first line and the last line of the area array camera image, and obtaining the structured light camera image corresponding to the ith area array camera image;
wherein J is the image line number of a single light structure camera, and D is the lag distance of the shooting area of the structure light camera relative to the visual field center of the area array camera.
After the images are generated in the increasing order, the pixel size of the image of the area-array camera is h1×w1The horizontal and vertical resolutions are all s1Registering structured light image pixel size of
Transverse resolution of s2Longitudinal resolution of Nxk2。
The invention has the beneficial effects that:
by synchronously acquiring an area array image and a line scanning image, and applying the characteristic that the horizontal and longitudinal pixel representation distance of the area array image is stable, the first line and the last line of an area array camera are registered with the specific line of a structural light camera image, and then the first line and the last line of the area array camera are zoomed to a uniform pixel representation distance image, and then the image with a larger view field is cut into the size consistent with the image with a smaller view field, so that the one-to-one correspondence of the area array image and the structural light image pixel is realized. The one-to-one correspondence engineering value is realized:
1) when the area-array camera adopts color imaging and the line scanning camera adopts structured light depth imaging, color rendering can be carried out on each pixel of the depth image obtained by the structured light camera according to the color of the corresponding pixel of the color area-array image to obtain a true color three-dimensional model;
2) for the registered planar array image and line scanning image pair, the homing correction can be carried out by utilizing the imaging non-dislocation and distortion characteristics of the planar array camera and combining the line scanning image with methods such as feature matching and the like, so that the target distortion phenomenon caused by the uneven travelling path of the line scanning camera is eliminated.
Drawings
Fig. 1 is a schematic diagram corresponding to an image of a registration method of an area-array camera and an image acquired by an optical structure camera according to an embodiment of the present invention;
fig. 2 is a schematic image cutting diagram of a registration method of an area-array camera and an image acquired by an optical structure camera according to an embodiment of the present invention;
fig. 3 is a schematic view of camera installation of a method for registering collected images of an area-array camera and an optical structure camera according to an embodiment of the present invention;
fig. 4 is a control structure diagram of a registration method of an area-array camera and an optical structure camera according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be further described with reference to the accompanying drawings.
Example (b):
as shown in fig. 1-4, a method for registering images collected by an area-array camera and an optical structure camera structurally comprises an area-array camera unit and an optical structure camera unit, wherein the area-array camera of the area-array camera unit is vertically installed downwards, an area-array light source irradiation center coincides with the center of a shooting area of the area-array camera, the optical structure camera unit comprises a laser and a structure light depth camera, the laser is linear laser and is vertically installed downwards, the structure light depth camera is obliquely installed, the center line of the shooting area coincides with a laser line, the laser line is transversely parallel to a rectangular area shot by the area-array camera, and the distance Dmm between the laser line and the center of the shooting area of the area-array camera.
When the synchronous trigger module is used, a pulse counter of the synchronous trigger module receives an encoder pulse signal, a single pulse of an encoder represents a distance mm, and trigger frequency division numbers of an area array camera and a structured light camera stored in STM32 are respectively represented as follows; after detecting an original encoder pulse, the pulse counter outputs 1 trigger pulse of the area array camera at a pulse output port 1 of the synchronous trigger module, and the area array camera collects 1 image of the area array and names the image in a natural number increasing sequence; the pulse counter outputs 1 structured light camera trigger pulse at a pulse output port 2 of the synchronous trigger module after detecting an original encoder pulse, the structured light camera collects 1 line of structured light data, and 1 structured light image is generated after collecting J lines of structured light data and named in a natural number increasing sequence.
And the industrial personal computer is responsible for acquisition control and data registration of the area-array camera and the structured light camera. The method comprises the steps of collecting control parameter setting and work control, wherein the parameter setting comprises camera exposure time and an interested area, and the work control comprises camera starting, trigger signal zero clearing, collection starting and collection stopping.
The method comprises the following steps:
s1, simultaneously using an area-array camera and a light structure camera to collect images at the same position;
s2, defining the transverse and longitudinal resolution of the image collected by the area-array camera as S1Picture size of h1×w1Defining the image of the structured light camera as each J line, and each horizontal pixel as w2Transverse resolution of s2；
S3, obtaining a structural light camera image corresponding to the view field area of the ith area-array camera image;
s4, carrying out pixel fusion on the area-array camera image and the optical structure camera image which correspond to each other, wherein the pixel fusion comprises structured light image zooming processing and image cutting processing, and the registration is completed after the pixel fusion;
the scaling parameters are as follows:
the vertical scaling parameters are:
the lateral scaling parameters are:
the registered structured light image size becomes:
the image cutting processing formula is as follows:
1) when in use
When the structure light registration image is cut into the width w of the image by bilateral symmetry1；
2) When in use
When the surface array image is cut into the width of the image by two-side symmetry
3) When in use
In the process, the area array image and the structural light registration image are not processed;
the area-array camera and the corresponding structured light image realize pixel-to-pixel correspondence with the center of the image;
and N represents the distance by a single pulse of the encoder, and represents the actual advancing distance of the vehicle by outputting 1 pulse by the encoder.
The image sequence number of the first line image of the ith area array camera corresponding to the structured light camera is
The row number is
In order to round up the symbol,rem () is a remainder symbol, and the image number of the structured light camera corresponding to the last line image of the ith area-array camera is
The row number is
Generating images by data of all rows between image rows of the structured light camera corresponding to the first row and the last row of the area array camera according to an increasing order, and obtaining a structured light camera image corresponding to the ith area array camera image;
wherein J is the image line number of a single structured light camera, and D is the lag distance of the shooting area of the structured light camera relative to the visual field center of the area array camera.
After the image is generated in an increasing order, the pixel size of the image of the area-array camera is h1×w1The horizontal and vertical resolutions are all s1Registering structured light image pixel size of
Transverse resolution of s2Longitudinal resolution of Nxk2。
And after the target area acquisition is completed, stopping the system.
The above-mentioned embodiments only express the specific embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (3)
1. An image registration method for an area-array camera and a structured light camera is characterized by comprising the following steps:
s1, simultaneously using an area-array camera and a light structure camera to collect images at the same position;
s2, defining the transverse and longitudinal resolution of the image collected by the area-array camera as S1Picture size of h1×w1Definition of a knotThe image of the light-forming camera is J lines, and the transverse pixel of each image is w2Transverse resolution of s2；
S3, obtaining a structural light camera image corresponding to the view field area of the ith area-array camera image;
s4, carrying out pixel fusion on the area-array camera image and the optical structure camera image which correspond to each other, wherein the pixel fusion comprises structured light image zooming processing and image cutting processing, and image registration is completed after the pixel fusion;
the scaling parameters are as follows:
the vertical scaling parameters are:
the lateral scaling parameters are:
the registered structured light image size becomes:
the image cutting processing formula is as follows:
1) when in use
When the structure light registration image is cut into the width w of the image by bilateral symmetry1；
2) When in use
When the surface array image is cut into the width of the image by two-side symmetry
3) When in use
In the process, the area array image and the structural light registration image are not processed;
the area-array camera and the corresponding structured light image realize pixel-to-pixel correspondence with the center of the image;
and N represents the distance by a single pulse of the encoder, and represents the actual advancing distance of the vehicle by outputting 1 pulse by the encoder.
2. The method as claimed in claim 1, wherein in S3, the serial number of the image of the first row of the i-th area-array camera corresponding to the image of the structured light camera is
The row number is
For rounding up the symbol, rem () is the remainder symbol, and the serial number of the last row image of the ith area-array camera corresponding to the structural optical camera image is
The row number is
Generating images according to the increasing sequence of data of all lines between the image lines of the structured light camera corresponding to the first line and the last line of the area array camera image, and obtaining the structured light camera image corresponding to the ith area array camera image;
wherein J is the image line number of a single light structure camera, and D is the lag distance of the shooting area of the structure light camera relative to the visual field center of the area array camera.
3. The method as claimed in claim 2, wherein the pixel size of the image of the area-array camera is h after the image is generated in the increasing order1×w1The horizontal and vertical resolutions are all s1Registering structured light imageHas a unit size of
Transverse resolution of s2Longitudinal resolution of Nxk2。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111654719.3A CN114359358A (en) | 2021-12-30 | 2021-12-30 | Image registration method for area-array camera and structured light camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111654719.3A CN114359358A (en) | 2021-12-30 | 2021-12-30 | Image registration method for area-array camera and structured light camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114359358A true CN114359358A (en) | 2022-04-15 |
Family
ID=81103906
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111654719.3A Pending CN114359358A (en) | 2021-12-30 | 2021-12-30 | Image registration method for area-array camera and structured light camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114359358A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106289106A (en) * | 2016-08-04 | 2017-01-04 | 北京航空航天大学 | Stereo vision sensor that a kind of line-scan digital camera and area array cameras combine and scaling method |
| US20190142524A1 (en) * | 2016-04-28 | 2019-05-16 | Intellijoint Surgical Inc. | Systems, methods and devices to scan 3d surfaces for intra-operative localization |
| CN113074660A (en) * | 2021-03-26 | 2021-07-06 | 深度光学科技(天津)有限公司 | Surface shape measuring method for large-size transparent object |
| US20210364288A1 (en) * | 2020-05-22 | 2021-11-25 | Harbin Institute Of Technology | Optical measurement and calibration method for pose based on three linear array charge coupled devices (ccd) assisted by two area array ccds |
-
2021
- 2021-12-30 CN CN202111654719.3A patent/CN114359358A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190142524A1 (en) * | 2016-04-28 | 2019-05-16 | Intellijoint Surgical Inc. | Systems, methods and devices to scan 3d surfaces for intra-operative localization |
| CN106289106A (en) * | 2016-08-04 | 2017-01-04 | 北京航空航天大学 | Stereo vision sensor that a kind of line-scan digital camera and area array cameras combine and scaling method |
| US20210364288A1 (en) * | 2020-05-22 | 2021-11-25 | Harbin Institute Of Technology | Optical measurement and calibration method for pose based on three linear array charge coupled devices (ccd) assisted by two area array ccds |
| CN113074660A (en) * | 2021-03-26 | 2021-07-06 | 深度光学科技(天津)有限公司 | Surface shape measuring method for large-size transparent object |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3983573B2 (en) | Stereo image characteristic inspection system | |
| CN1260544C (en) | Compatible and accurate calibration method for double eye line structure photo-sensor and implementing apparatus | |
| EP2106527A2 (en) | A method, device and system for imaging | |
| CN116156342B (en) | Multi-linear array image sensor splicing method, linear array image acquisition system and device | |
| CN110672037A (en) | Linear light source grating projection three-dimensional measurement system and method based on phase shift method | |
| CN103175851A (en) | Calibrating tool and calibrating method for multi-camera scanning system | |
| CN112055195B (en) | Method for measuring distortion of surveying and mapping camera | |
| CN106012778B (en) | Digital image acquisition analysis method for express highway pavement strain measurement | |
| CN101635782A (en) | Method and device for obtaining image based on dynamic time delay integral | |
| CN105139393A (en) | Method for calibrating intrinsic parameters of linear array camera | |
| CN104156977A (en) | Point target movement velocity detection method based on multiple linear moveout scanning, extending and sampling | |
| CN114359358A (en) | Image registration method for area-array camera and structured light camera | |
| CN111272101A (en) | Four-dimensional hyperspectral depth imaging system | |
| CN115388891A (en) | Space positioning method and system for large-view-field moving target | |
| CN114877806A (en) | Double-camera point cloud measuring method and device with color real-time mapping function | |
| CN112184578A (en) | Method for calibrating central coordinates of each microlens imaging in light field imaging | |
| KR20000060731A (en) | Calibraion method of high resolution photographing equipment using multiple imaging devices. | |
| CN112710662A (en) | Generation method and device, generation system and storage medium | |
| US20110096209A1 (en) | Shading correction method, shading-correction-value measuring apparatus, image capturing apparatus, and beam-profile measuring apparatus | |
| KR20180103657A (en) | Calibration method and apparatus of stereo camera module, computer readable storage medium | |
| JP2007033039A (en) | Method and device for calibrating optical head part in three-dimensional shape measuring instrument by optical cutting method | |
| CN211205210U (en) | Four-dimensional hyperspectral depth imaging system | |
| CN117291995A (en) | Dual-camera calibration method based on three-dimensional line scanning laser detection system | |
| CN117689734A (en) | Cereal quality real-time acquisition and recognition method and system | |
| CN116309063A (en) | Correction information generation method, image stitching method and device and image acquisition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |