CN112729246B - Black surface object depth image measuring method based on binocular structured light - Google Patents
Black surface object depth image measuring method based on binocular structured light Download PDFInfo
- Publication number
- CN112729246B CN112729246B CN202011422503.XA CN202011422503A CN112729246B CN 112729246 B CN112729246 B CN 112729246B CN 202011422503 A CN202011422503 A CN 202011422503A CN 112729246 B CN112729246 B CN 112729246B
- Authority
- CN
- China
- Prior art keywords
- black
- depth image
- surface object
- image
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/30—Interpretation of pictures by triangulation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a binocular structured light-based black surface object depth image measuring method. The scheme includes adjusting a black surface object image acquisition device and a black surface object to an initial state; adjusting the black-surface object image capturing apparatus and the black-surface object from the initial state to an exposure state; shooting to obtain a color image; acquiring a gray level mean value and a standard deviation of a black object according to the color picture, and acquiring a first depth image and a second depth image by using a depth fusion method and a multi-depth image fusion algorithm; and obtaining a target depth image of the black surface object by an image fusion algorithm. According to the scheme, multiple exposure, aperture and power adjustment of the multi-eye structured light are performed, and the problems that the size of an object on the black surface cannot be accurately measured and point cloud data is easily lost in measurement in the existing 3D measurement technology are solved by utilizing multi-image data fusion, so that the method has the advantages of being simple in operation, high in efficiency and high in universality.
Description
Technical Field
The invention relates to the technical field of 3D structured light measurement, in particular to a black surface object depth image measuring method based on binocular structured light.
Background
The rapidly developing computer vision technology increasingly affects or even changes people's lives and is widely used in the industrial and manufacturing fields. Three-dimensional visual imaging and inspection is an important branch of the field of computer vision, and has been applied in many fields in recent years because of its high efficiency, high precision and capability of satisfying various application requirements of an automated production line. The 3D structured light measurement technology is an important branch of the three-dimensional vision technology, and the principle of the technology is that a projection device is adopted to project a structured light pattern with coded information on a measured object, and a camera records the information of the structured light pattern to decode, so that three-dimensional information of an object is restored. The 3D structured light measurement technology has the advantages of high precision, non-contact, high detection speed, simplicity in operation and the like, and is applied more and more widely in industrial scenes.
However, when a black object is imaged, the conventional 3D structured light measurement technology is prone to cause that the size of the black surface object cannot be accurately measured due to the problems of black absorption light and low reflectivity, and the situation that measurement data is lost and a measurement blind area is generated during final measurement may occur.
Disclosure of Invention
In order to overcome the problems in the related art, the invention provides a binocular-structured-light-based black surface object depth image measuring method, so that the defect that measurement blind areas occur due to the fact that black absorbs light and measurement data are likely to be lost during measurement is overcome.
According to the embodiment of the invention, the black surface object depth image measuring method based on the binocular structure light is provided. The method comprises the following steps:
adjusting a black surface object image acquisition device and a black surface object to an initial state, wherein the black surface object image acquisition device comprises a first color camera, a second color camera, an infrared projection device, an infrared camera and a base platform;
adjusting the black-surface object image acquisition device and the black-surface object from the initial state to an exposure state;
shooting through the black surface object image acquisition equipment in the exposure state and the black surface object to obtain a color image;
acquiring a gray mean value and a standard deviation of a black object according to the color picture;
obtaining a first depth image by utilizing a first depth fusion method and a multi-depth image fusion algorithm according to the gray level mean value and the standard deviation of the black object;
obtaining a second depth image by utilizing a second depth fusion method and a multi-depth image fusion algorithm according to the gray level mean value and the standard deviation of the black object;
and fusing the first depth image and the second depth image through an image fusion algorithm to obtain a black surface object target depth image.
In one or more embodiments, preferably, the adjusting the black-surface object image acquiring device and the black-surface object to the initial state specifically includes:
mounting the infrared projection device, the first color camera, the second color camera and the infrared camera on the base platform, wherein the power of the infrared projection device is variable, and the first color camera, the second color camera and the infrared camera are lenses provided with electric diaphragms;
mounting the infrared projection device at the middle position of the base platform;
mounting the infrared camera at a position beside the infrared projection device;
mounting the first color camera and the second color camera on both sides of the base platform;
the lens provided with the electric diaphragm is specifically adjusted by controlling the size of the diaphragm of the lens through a microcomputer.
In one or more embodiments, preferably, the adjusting the black-surface object image acquiring apparatus and the black-surface object from the initial state to the exposure state specifically includes:
setting an exposure time of the first color camera, the second color camera, and the infrared camera to half of a maximum exposure time;
setting the motorized aperture of the first color camera, the second color camera, and the infrared camera to be half of a maximum motorized aperture.
In one or more embodiments, preferably, the obtaining a mean gray scale value and a standard deviation of a black object according to the color picture specifically includes:
converting the color picture into a gray scale image;
extracting the image edge of the gray scale image by adopting a canny operator;
calculating edge perimeter information according to the image edge, and screening out the edge outline of the black surface object;
calculating a mean and a standard deviation of the gray scale of the black object using the gray scale map within the edge profile of the black object.
In one or more embodiments, preferably, the obtaining the first depth image by using the first depth fusion method and the multiple-depth image fusion algorithm according to the gray level mean and the standard deviation of the black object specifically includes:
adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device according to the gray level mean value and the standard deviation of the black object by utilizing a first exposure time, the electric aperture and the emission power adjustment rule;
after the exposure time, the electric aperture and the emission power are adjusted every time, shooting a first structured light pattern projected on the black surface object by the infrared projection device by using the infrared camera;
decoding the structured light pattern according to camera calibration parameters of the first color camera, the second color camera and the infrared camera by using the first structured light pattern, and calculating to obtain a plurality of first black surface object depth images;
and fusing all the first black surface object depth images by using the multi-depth map fusion algorithm to obtain the first depth image.
In one or more embodiments, preferably, the first exposure time, the motorized aperture, and the transmission power adjustment rule include:
adjusting the exposure time by using a first calculation formula;
adjusting the electric aperture by using a second calculation formula;
adjusting the transmitting power by using a third calculation formula;
the first calculation formula is:
wherein exp i For the adjusted exposure time, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object，exp max Is the maximum exposure time of the infrared camera;
the second calculation formula is:
wherein, F i For the adjusted electric aperture, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, F max An electric iris of the infrared camera;
the third calculation formula is:
wherein, TX i For the adjusted transmit power, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, TX max Is the maximum emission power of the infrared projection device.
In one or more embodiments, preferably, the obtaining a second depth image by using a second depth fusion method and a multi-depth image fusion algorithm according to the gray level mean and the standard deviation of the black object specifically includes:
adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device by utilizing a second exposure time, the electric aperture and the emission power adjustment rule according to the gray level mean value and the standard deviation of the black object;
after the exposure time, the electric aperture and the emission power are adjusted every time, shooting a second structured light pattern projected on the black surface object by the infrared projection device by using the infrared camera;
decoding the structured light pattern according to camera calibration parameters of the first color camera, the second color camera and the infrared camera by using the second structured light pattern, and calculating to obtain a plurality of black surface object depth images;
and fusing all the second black surface object depth images by using the multi-depth-map fusion algorithm to obtain the second depth image.
In one or more embodiments, preferably, the second exposure time and the electric aperture adjustment rule include:
adjusting the exposure time by using a fourth calculation formula;
adjusting the electric aperture by using a fifth calculation formula;
the fourth calculation formula is:
wherein exp i For the adjusted exposure time, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, exp max Is the maximum exposure time of the infrared camera;
the fifth calculation formula is:
wherein, F i For the adjusted electric aperture, i represents the adjustment times, mean is the mean value of the gray scale of the black object, std is the standard deviation of the black object, and F max Is an electric aperture of the infrared camera.
In one or more embodiments, preferably, the multiple depth image fusion algorithm specifically uses a sixth calculation formula to obtain a new picture, where the sixth calculation formula is:
wherein Im g is a fused depth image and Im g i For the ith depth image, i represents the number of adjustments.
In one or more embodiments, preferably, the image fusion algorithm specifically obtains the new picture by using a seventh calculation formula, where the seventh calculation formula is:
wherein Mix is the target depth image of the black surface object, mix 1 For the first depth image, mix 2 For the second depth image, std is the standard deviation of the black object.
The technical scheme provided by the embodiment of the invention can have the following beneficial effects:
1. this scheme shoots through the color camera of binocular structure to obtain a plurality of images through multiple exposure time, electronic light ring and transmitting power, and then obtain the degree of depth image through image depth processing, overcome the shortcoming that present 3D structured light can't accurate measurement black surface object size information, realize the clarity, the accurate measurement to black surface object.
2. In the scheme, only 2 color cameras and 1 infrared camera are used for image acquisition, camera adjustment and picture depth fusion are carried out according to preset rules, a universal acquisition mode for various black surface objects can be realized, and the scheme has the characteristics of economy and reliability in acquiring the depth image of the workpiece due to less repetitive operation and smaller operation amount.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a black surface object depth image measuring method based on binocular structured light according to an embodiment of the present invention.
Fig. 2 is a flowchart of adjusting the black-surface object image obtaining apparatus and the black-surface object to initial states in the black-surface object depth image measuring method based on the binocular structured light according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a black-surface object image obtaining apparatus and a black-surface object in a black-surface object depth image measuring method based on binocular structured light according to an embodiment of the present invention.
Fig. 4 is a flowchart of adjusting from the initial state to the exposure state in the binocular structured light-based black surface object depth image measuring method according to an embodiment of the present invention.
Fig. 5 is a flowchart of acquiring a mean gray scale value and a standard deviation of a black object in a binocular-structured-light-based black-surface object depth image measuring method according to an embodiment of the present invention.
Fig. 6 is a flowchart of obtaining a first depth image by using a first depth fusion method and a multi-depth image fusion algorithm in a black surface object depth image measuring method based on binocular structured light according to an embodiment of the present invention.
Fig. 7 is a flowchart of obtaining a second depth image by using a second depth fusion method and a multi-depth image fusion algorithm in the black surface object depth image measuring method based on the binocular structured light according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.
In some flows described in the present specification and claims and above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, with the order of the operations being given as 101, 102, etc. merely to distinguish between various operations, and the order of the operations itself does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Although 2D vision is the mainstream at present, as the requirement for measurement accuracy is higher and higher, the conditions of the measured object are more and more complex, the defects of the 2D system are more and more prominent, and the 3D vision technology is continuously broken through, is incomparable with 2D in terms of accuracy and flexibility, and has gradually exhibited its advantages in various fields at present.
The 3D structured light measurement technology is an important branch of the three-dimensional vision technology, the principle of the technology is that a projection device is adopted to project a structured light pattern with coded information on a measured object, and a camera records the information of the structured light pattern to decode, so that three-dimensional information of an object is restored. The 3D structured light measurement technology has the advantages of high precision, non-contact, high detection speed, simplicity in operation and the like, and is applied more and more widely in industrial scenes. At present, the 3D structured light measurement technology is applied more in the field of industrial workpiece dimension measurement.
However, the existing 3D structured light measurement technology has the following disadvantages: the existing 3D structured light measurement technology is easy to cause the problem that the size of a black surface object cannot be accurately measured due to the low black absorption light and reflectivity when a black object is imaged, and the situation that measurement data is lost and a measurement blind area is generated during final measurement is possible.
The technical scheme of the invention provides a binocular-structured-light-based black surface object depth image measuring method, which enables a plurality of groups of image data to be obtained through binocular structured light, the image data are mutually supplemented, and a target depth image is finally obtained through an image depth fusion method, so that the measurement precision of the black object measured by the 3D structured light cannot be influenced due to image information loss caused by black light absorption and low reflectivity.
Fig. 1 is a flowchart of a black surface object depth image measuring method based on binocular structured light according to an embodiment of the present invention.
As shown in fig. 1, in one or more embodiments, preferably, the binocular structured light-based black surface object depth image measuring method specifically includes:
s101, adjusting a black surface object image acquisition device and a black surface object to an initial state, wherein the black surface object image acquisition device comprises a first color camera, a second color camera, an infrared projection device, an infrared camera and a base platform;
s102, adjusting the black surface object image acquisition equipment and the black surface object from the initial state to an exposure state;
s103, shooting through the black surface object image acquisition equipment in the exposure state and the black surface object to obtain a color image;
s104, acquiring a gray average value and a standard deviation of a black object according to the color picture;
s105, obtaining a first depth image by utilizing a first depth fusion method and a multi-depth image fusion algorithm according to the gray level mean value and the standard deviation of the black object;
s106, obtaining a second depth image by using a second depth fusion method and a multi-depth image fusion algorithm according to the gray level mean value and the standard deviation of the black object;
s107, fusing the first depth image and the second depth image through an image fusion algorithm to obtain a black surface object target depth image.
In the embodiment of the invention, through a series of steps of black surface object image acquisition equipment, initialization, exposure, shooting, image depth processing, readjustment to obtain a second group of images, fusion of a plurality of groups of image data and the like, the full shooting of the black surface object is completed, a binocular structured light target depth image without information loss is further obtained, and the size measurement of the black surface object is accurately completed.
Further, in fig. 2 to 7, the steps of adjusting the initial state to the exposure state, obtaining the first depth image and the second depth image by using the depth fusion method and the multi-depth image fusion algorithm, and the like are described in detail.
Fig. 2 is a flowchart of adjusting the black-surface object image obtaining apparatus and the black-surface object to initial states in the black-surface object depth image measuring method based on the binocular structured light according to an embodiment of the present invention.
As shown in fig. 2, in one or more embodiments, preferably, the adjusting the black-surface object image obtaining device and the black-surface object to the initial state specifically includes:
s201, mounting the infrared projection device, the first color camera, the second color camera and the infrared camera on the base platform, wherein the power of the infrared projection device is variable, and the first color camera, the second color camera and the infrared camera are all lenses provided with electric diaphragms;
s202, mounting the infrared projection device in the middle of the base platform;
s203, installing the infrared camera at a position beside the infrared projection device;
s204, mounting the first color camera and the second color camera on two sides of the base platform;
the lens provided with the electric diaphragm is specifically adjusted by controlling the size of the diaphragm of the lens by using a microcomputer.
In the embodiment of the present invention, the infrared projection apparatus, the first color camera, the second color camera, and the infrared camera are configured at corresponding positions to initialize the device state.
Fig. 3 is a schematic diagram of a black-surface object image obtaining apparatus and a black-surface object in a black-surface object depth image measuring method based on binocular structured light according to an embodiment of the present invention.
In one or more embodiments, the black surface object image acquisition device comprises a first color camera 301, a second color camera 302, an infrared projection device 303, an infrared camera 304, a base platform 305; the projection range of the infrared projection device 303 is indicated by dashed lines below the black surface object 306. The first color camera 301 and the second color camera 302 are two color cameras with 200 ten thousand pixels, the infrared projection device 303 is an infrared projection device with a resolution of 100 ten thousand pixels, and the infrared camera 304 is an infrared camera with 200 ten thousand pixels. With the configuration in this embodiment, it is possible to perform shooting of a black surface object, but it is also possible to perform shooting with a camera device that exceeds the pixels in this embodiment.
Fig. 4 is a flowchart of adjusting from the initial state to the exposure state in the binocular structured light-based black surface object depth image measuring method according to an embodiment of the present invention.
As shown in fig. 4, in one or more embodiments, preferably, the adjusting the black-surface object image obtaining device and the black-surface object from the initial state to the exposure state specifically includes:
s401, setting the exposure time of the first color camera, the second color camera and the infrared camera to be half of the maximum exposure time;
s402, setting electric diaphragms of the first color camera, the second color camera and the infrared camera to be half of a maximum electric diaphragm.
In the embodiment of the invention, the exposure time and the electric aperture are set before exposure, and the exposure time and the electric aperture are set by half of the maximum value, so that the reliability in the whole exposure process is ensured, specifically, the adjustment increase and decrease ranges of the exposure time and the electric aperture can be maximized, and the adjustment decrease and increase degrees can be ensured to be the same.
Fig. 5 is a flowchart of acquiring a mean gray level and a standard deviation of a black object in a binocular structured light-based black surface object depth image measuring method according to an embodiment of the present invention.
As shown in fig. 5, in one or more embodiments, preferably, the acquiring a mean value and a standard deviation of a gray scale of a black object according to the color picture specifically includes:
s501, converting the color picture into a gray-scale image;
s502, extracting the image edge of the gray scale image by adopting a canny operator;
the canny operator is an edge detection computing theory (computational theory of edge detection), and the goal of the canny operator is to find an optimal edge detection algorithm.
S503, calculating edge perimeter information according to the image edge, and screening out the edge outline of the black surface object;
s504, calculating the gray level mean value and the standard deviation of the black object by utilizing the gray level image in the edge outline of the black object.
In the embodiment of the invention, a color image of the surface of a workpiece is shot by a color camera, so that the color image is used for carrying out gray scale division by utilizing color change, the edge of the image is extracted by a canny operator, the edge of the black object to be tested is obtained, the image only with the black object is obtained after final cutting, and then the average value calculation and the standard deviation calculation of the gray scale are carried out on the part of the image.
Fig. 6 is a flowchart of obtaining a first depth image by using a first depth fusion method and a multi-depth image fusion algorithm in a black surface object depth image measuring method based on binocular structured light according to an embodiment of the present invention.
As shown in fig. 6, in one or more embodiments, preferably, the obtaining a first depth image by using a first depth fusion method and a multi-depth image fusion algorithm according to the mean and the standard deviation of the gray scale of the black object specifically includes:
s601, adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device according to the gray level mean value and the standard deviation of the black object by utilizing a first exposure time, electric aperture and emission power adjustment rule;
s602, after the exposure time, the electric aperture and the emission power of each time are adjusted, shooting a first structured light pattern projected on the black surface object by the infrared projection device by using the infrared camera;
s603, decoding the structured light pattern according to camera calibration parameters of the first color camera, the second color camera and the infrared camera by using the first structured light pattern, and calculating to obtain a plurality of first black surface object depth images;
s604, fusing all the first black surface object depth images by using the multi-depth map fusion algorithm to obtain the first depth image.
In the embodiment of the invention, a new depth image can be obtained by adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device, and the new depth image has different light absorption degrees for a black object due to different exposures, electric apertures and transmission powers, so that new features can be generated, and when the number of pictures is large, the light absorption of the surface of the black object is averaged, and the measurement result cannot be influenced finally.
In one or more embodiments, preferably, the first exposure time, the motorized aperture, and the transmission power adjustment rule include:
adjusting the exposure time by using a first calculation formula;
adjusting the electric aperture by using a second calculation formula;
adjusting the transmitting power by using a third calculation formula;
the first calculation formula is:
wherein exp i For the adjusted exposure time, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, exp max Is the maximum exposure time of the infrared camera;
the second calculation formula is:
wherein, F i For the adjusted electric aperture, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, F max An electric iris of the infrared camera;
the third calculation formula is:
wherein, TX i For the adjusted transmit power, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, TX max Is the maximum emission power of the infrared projection device.
According to the embodiment of the invention, the exposure time of the infrared camera, the electric aperture of the lens and the infrared projection emission power are adjusted according to the second exposure time and the electric aperture adjustment rule according to the gray average value and the standard deviation of the black surface object.
Fig. 7 is a flowchart of obtaining a second depth image by using a second depth fusion method and a multi-depth image fusion algorithm in the black surface object depth image measuring method based on the binocular structured light according to an embodiment of the present invention.
As shown in fig. 7, in one or more embodiments, preferably, the obtaining a second depth image by using a second depth fusion method and a multi-depth image fusion algorithm according to the mean and the standard deviation of the gray scale of the black object specifically includes:
s701, adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device according to the gray level mean value and the standard deviation of the black object by utilizing a second exposure time, the electric aperture and the emission power adjustment rule;
s702, after the exposure time, the electric aperture and the emission power are adjusted every time, shooting a second structured light pattern projected on the black surface object by the infrared projection device by using the infrared camera;
s703, decoding the structured light pattern by using the second structured light pattern according to camera calibration parameters of the first color camera, the second color camera and the infrared camera, and calculating to obtain a plurality of black surface object depth images;
s704, fusing all the second black surface object depth images by using the multi-depth-map fusion algorithm to obtain the second depth image.
In one or more embodiments, preferably, the second exposure time and the motorized aperture adjustment rule include:
adjusting the exposure time by using a fourth calculation formula;
adjusting the electric aperture by using a fifth calculation formula;
the fourth calculation formula is:
wherein exp i For the adjusted exposure time, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, exp max Is the maximum exposure time of the infrared camera;
the fifth calculation formula is:
wherein, F i For the adjusted electric aperture, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, F max Is an electric aperture of the infrared camera.
According to the gray level mean value and the standard deviation of the black surface object, the exposure time of the color camera and the electric aperture of the lens are adjusted according to the second exposure time and the electric aperture adjustment rule.
In one or more embodiments, preferably, the multiple depth image fusion algorithm specifically uses a sixth calculation formula to obtain a new picture, where the sixth calculation formula is:
wherein Im g is a fused depth image and Im g i For the ith depth image, i represents the number of adjustments.
In one or more embodiments, preferably, the image fusion algorithm specifically obtains the new picture by using a seventh calculation formula, where the seventh calculation formula is:
wherein Mix is the target depth image of the black surface object, mix 1 For the first depth image, mix 2 For the second depth image, std is a standard deviation of the black object.
In the embodiment of the invention, the first depth image and the second depth image are finally processed into a single target depth image through an image fusion algorithm. The target depth image can display the measured size of the workpiece, and meanwhile, the influence of light absorbed by the surface of a black object under different exposure times and different electric apertures on the depth image is reduced through multiple times of image fusion, so that the depth image of the surface of the black object is acquired.
In the embodiment of the invention, a binocular structure light-based black surface object depth image measuring method is provided, so that color camera data of a binocular structure are subjected to multiple adjustment and data fusion. The following effects can be produced:
1. this scheme shoots through the color camera of binocular structure to obtain a plurality of images through multiple exposure time, electronic light ring and transmitting power, and then obtain the degree of depth image through image depth processing, overcome the shortcoming that present 3D structured light can't accurate measurement black surface object size information, realize the clarity, the accurate measurement to black surface object.
2. In the scheme, only 2 color cameras and 1 infrared camera are used for image acquisition, camera adjustment and picture depth fusion are carried out according to preset rules, a universal acquisition mode for various black surface objects can be realized, and the scheme has the characteristics of economy and reliability in acquiring the depth image of the workpiece due to less repetitive operation and smaller operation amount.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (8)
1. A black surface object depth image measuring method based on binocular structure light is characterized by comprising the following steps:
adjusting a black surface object image acquisition device and a black surface object to an initial state, wherein the black surface object image acquisition device comprises a first color camera, a second color camera, an infrared projection device, an infrared camera and a base platform;
adjusting the black-surface object image capturing apparatus and the black-surface object from the initial state to an exposure state;
shooting through the black surface object image acquisition equipment in the exposure state and the black surface object to obtain a color picture;
acquiring a gray average value and a standard deviation of a black object according to the color picture;
according to the gray level mean value and the standard deviation of the black object, a first depth image is obtained by utilizing a first depth fusion method and a multi-depth image fusion algorithm;
obtaining a second depth image by utilizing a second depth fusion method and a multi-depth image fusion algorithm according to the gray level mean value and the standard deviation of the black object;
fusing the first depth image and the second depth image through an image fusion algorithm to obtain a black surface object target depth image;
the obtaining of the first depth image by using the first depth fusion method and the multiple depth image fusion algorithm according to the gray level mean value and the standard deviation of the black object specifically includes:
adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device according to the gray level mean value and the standard deviation of the black object by utilizing a first exposure time, the electric aperture and the emission power adjustment rule;
after the exposure time, the electric aperture and the emission power are adjusted every time, shooting a first structured light pattern projected on the black surface object by the infrared projection device by using the infrared camera;
decoding the structured light pattern according to camera calibration parameters of the first color camera, the second color camera and the infrared camera by using the first structured light pattern, and calculating to obtain a plurality of first black surface object depth images;
fusing all the first black surface object depth images by using the multi-depth map fusion algorithm to obtain the first depth image;
wherein the first exposure time, motorized aperture and emission power adjustment rules comprise:
adjusting the exposure time by using a first calculation formula;
adjusting the electric aperture by using a second calculation formula;
adjusting the transmitting power by using a third calculation formula;
the first calculation formula is:
wherein exp i For the adjusted exposure time, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, exp max Is the maximum exposure time of the infrared camera;
the second calculation formula is:
wherein, F i For the adjusted electric aperture, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, F max An electric iris of the infrared camera;
the third calculation formula is:
wherein, TX i For the adjusted transmit power, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, TX max Is the maximum emission power of the infrared projection device.
2. The binocular structured light-based black surface object depth image measuring method according to claim 1, wherein the adjusting the black surface object image acquiring device and the black surface object to an initial state specifically comprises:
mounting the infrared projection device, the first color camera, the second color camera, and the infrared camera on the base platform, wherein the infrared projection device has a variable power, and the first color camera, the second color camera, and the infrared camera are lenses configured with electric apertures;
mounting the infrared projection device at the middle position of the base platform;
mounting the infrared camera at a position beside the infrared projection device;
mounting the first color camera and the second color camera on both sides of the base platform;
the lens provided with the electric diaphragm is specifically adjusted by controlling the size of the diaphragm of the lens by using a microcomputer.
3. The binocular structured light-based black surface object depth image measuring method according to claim 1, wherein the adjusting the black surface object image acquiring apparatus and the black surface object from the initial state to an exposure state specifically comprises:
setting exposure times of the first color camera, the second color camera, and the infrared camera to half of a maximum exposure time;
setting the motorized aperture of the first color camera, the second color camera, and the infrared camera to be half of a maximum motorized aperture.
4. The binocular structured light-based black surface object depth image measuring method according to claim 1, wherein the obtaining of the gray level mean and the standard deviation of the black object according to the color picture specifically comprises:
converting the color picture into a gray scale image;
extracting the image edge of the gray scale image by adopting a canny operator;
calculating edge perimeter information according to the image edge, and screening out the edge outline of the black surface object;
calculating the mean and standard deviation of the gray scale of the black object by using the gray scale map in the edge contour of the black object.
5. The binocular-structured-light-based black surface object depth image measuring method according to claim 1, wherein the obtaining of the second depth image by using a second depth fusion method and a multi-depth image fusion algorithm according to the gray mean and standard deviation of the black object specifically comprises:
adjusting the exposure time and the electric aperture of the infrared camera and adjusting the emission power of the infrared projection device by utilizing a second exposure time, the electric aperture and the emission power adjustment rule according to the gray level mean value and the standard deviation of the black object;
after the exposure time, the electric aperture and the emission power are adjusted every time, shooting a second structured light pattern projected on the black surface object by the infrared projection device by using the infrared camera;
decoding the structured light pattern according to camera calibration parameters of the first color camera, the second color camera and the infrared camera by using the second structured light pattern, and calculating to obtain a plurality of black surface object depth images;
and fusing all the black surface object depth images by utilizing the multi-depth-map fusion algorithm to obtain the second depth image.
6. The binocular structured light-based black surface object depth image measuring method according to claim 5, wherein the second exposure time and the electric aperture adjustment rule include:
adjusting the exposure time by using a fourth calculation formula;
adjusting the electric aperture by using a fifth calculation formula;
the fourth calculation formula is:
wherein exp i For the adjusted exposure time, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, exp max Is the maximum exposure time of the infrared camera;
the fifth calculation formula is:
wherein, F i For the adjusted electric aperture, i represents the adjustment times, mean is the mean of the gray levels of the black object, std is the standard deviation of the black object, F max Is an electric aperture of the infrared camera.
7. The binocular structured light-based black surface object depth image measuring method according to claim 1, wherein the multi-depth image fusion algorithm is specifically to obtain a new picture by using a sixth calculation formula, and the sixth calculation formula is as follows:
wherein Im g is the fused depth image, im g i For the ith depth image, i represents the number of adjustment times.
8. The binocular structured light-based black surface object depth image measuring method according to claim 1, wherein the image fusion algorithm is specifically to obtain a new picture by using a seventh calculation formula, and the seventh calculation formula is:
wherein Mix is the target depth image of the black surface object, mix 1 For the first depth image, mix 2 For the second depth image, std is the standard deviation of the black object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011422503.XA CN112729246B (en) | 2020-12-08 | 2020-12-08 | Black surface object depth image measuring method based on binocular structured light |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011422503.XA CN112729246B (en) | 2020-12-08 | 2020-12-08 | Black surface object depth image measuring method based on binocular structured light |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112729246A CN112729246A (en) | 2021-04-30 |
| CN112729246B true CN112729246B (en) | 2022-12-16 |
Family
ID=75599225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011422503.XA Active CN112729246B (en) | 2020-12-08 | 2020-12-08 | Black surface object depth image measuring method based on binocular structured light |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112729246B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0921620A (en) * | 1995-07-05 | 1997-01-21 | Fuji Facom Corp | Method for measuring three-dimensional shape of object |
| CN109741405A (en) * | 2019-01-21 | 2019-05-10 | 同济大学 | A kind of depth information acquisition system based on dual structure light RGB-D camera |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5959001B2 (en) * | 2012-07-20 | 2016-08-02 | 株式会社小森コーポレーション | Sheet inspection equipment |
| US10666928B2 (en) * | 2015-02-06 | 2020-05-26 | The University Of Akron | Optical imaging system and methods thereof |
| CN104964672B (en) * | 2015-06-29 | 2017-05-31 | 济南大学 | A kind of long-distance barrier detecting sensor based on line-structured light |
| CN106550228B (en) * | 2015-09-16 | 2019-10-15 | 上海图檬信息科技有限公司 | The equipment for obtaining the depth map of three-dimensional scenic |
| KR101737518B1 (en) * | 2015-10-27 | 2017-05-18 | 성균관대학교산학협력단 | Method and system for determining optimal exposure time and frequency of structured light based 3d camera |
| EP3501984B1 (en) * | 2017-12-21 | 2020-08-12 | Everdrone AB | A method for validating sensor units in a uav, and a uav |
| DE102018208417A1 (en) * | 2018-05-28 | 2019-11-28 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Projection device and projection method |
| CN110245618B (en) * | 2019-06-17 | 2021-11-09 | 深圳市汇顶科技股份有限公司 | 3D recognition device and method |
| CN110942060B (en) * | 2019-10-22 | 2023-05-23 | 清华大学 | Material identification method and device based on laser speckle and modal fusion |
| CN110686599B (en) * | 2019-10-31 | 2020-07-03 | 中国科学院自动化研究所 | Three-dimensional measurement method, system and device based on colored Gray code structured light |
-
2020
- 2020-12-08 CN CN202011422503.XA patent/CN112729246B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0921620A (en) * | 1995-07-05 | 1997-01-21 | Fuji Facom Corp | Method for measuring three-dimensional shape of object |
| CN109741405A (en) * | 2019-01-21 | 2019-05-10 | 同济大学 | A kind of depth information acquisition system based on dual structure light RGB-D camera |
Non-Patent Citations (2)
| Title |
|---|
| 基于稀疏激光点云数据和单帧图像融合的三维重构算法;贺秉安等;《计测技术》;20170628(第03期);全文 * |
| 基于结构光技术的高光表面三维测量方法;吴丽娟等;《工具技术》;20110320(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112729246A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106412426B (en) | Total focus camera and method | |
| CN112150528A (en) | Depth image acquisition method, terminal and computer readable storage medium | |
| US20160212411A1 (en) | Method and apparatus for multiple technology depth map acquisition and fusion | |
| EP3291004A1 (en) | Ranging method, automatic focusing method and device | |
| CN110689581A (en) | Structured light module calibration method, electronic device and computer readable storage medium | |
| US20110222734A1 (en) | Methods for evaluating distances in a scene and apparatus and machine readable medium using the same | |
| CN105466334B (en) | The lens location calibration method of more camera lens vision collecting devices | |
| US10721462B2 (en) | Method and apparatus for calibrating parameter of three-dimensional (3D) display apparatus | |
| CN111325803B (en) | Calibration method for evaluating internal and external participation time synchronization of binocular camera | |
| CN110087049A (en) | Automatic focusing system, method and projector | |
| US20210254968A1 (en) | Method and System for Automatic Focusing for High-Resolution Structured Light 3D Imaging | |
| CN107493427A (en) | Focusing method, device and the mobile terminal of mobile terminal | |
| CN116433780B (en) | Automatic calibration method for laser structured light based on machine vision | |
| CN112648935A (en) | Image processing method and device and three-dimensional scanning system | |
| CN112595262B (en) | Binocular structured light-based high-light-reflection surface workpiece depth image acquisition method | |
| CN117061868A (en) | Automatic photographing device based on image recognition | |
| CN110827360A (en) | Photometric stereo measurement system and method for calibrating light source direction thereof | |
| CN111199536B (en) | Focusing evaluation method and device | |
| CN111970500A (en) | Automatic distance step calibration method and system for projection equipment | |
| CN116625258A (en) | Chain spacing measuring system and chain spacing measuring method | |
| CN112950727B (en) | Large-view-field multi-target simultaneous ranging method based on bionic curved compound eye | |
| CN105335959A (en) | Quick focusing method and device for imaging apparatus | |
| CN112729246B (en) | Black surface object depth image measuring method based on binocular structured light | |
| KR102080506B1 (en) | 3D optical scanner | |
| CN115393555A (en) | Three-dimensional image acquisition method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |