CN108955562B - Digital extension method and system for microscopic depth of field of microscopic vision system - Google Patents
Digital extension method and system for microscopic depth of field of microscopic vision system Download PDFInfo
- Publication number
- CN108955562B CN108955562B CN201810622582.5A CN201810622582A CN108955562B CN 108955562 B CN108955562 B CN 108955562B CN 201810622582 A CN201810622582 A CN 201810622582A CN 108955562 B CN108955562 B CN 108955562B
- Authority
- CN
- China
- Prior art keywords
- microscopic
- field
- depth
- dimensional
- tomography
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
Abstract
The invention discloses a depth of field digital extension method and system of a microscopic vision system based on a computer microscopic vision tomography technology, which comprises the steps of firstly obtaining a tomography image through the computer microscopic vision tomography technology, and obtaining a three-dimensional tomography view field space corresponding to the tomography image through the combination of the tomography image and the step length of a precision positioning system; secondly, reconstructing three-dimensional fault space digital information through technologies such as rasterization, digitalization and the like; and finally, acquiring the depth-of-field expanded digital microscopic field space according to all the acquired digital information of the fault space and the fault space position information, and realizing depth-of-field expansion. The invention overcomes the contradiction between high resolution and large depth of field, breaks through the limitation that the small depth of field of the microscopic vision system can not accurately obtain the spatial information of the ultra-depth of field, and expresses the spatial information of the microscopic field of the micro-assembly system with the ultra-large depth of field by adopting digital information, thereby vividly and visually representing the three-dimensional object information of the microscopic field of the ultra-depth of field.
Description
Technical Field
The invention belongs to the fields of intelligent manufacturing and scientific research, particularly serves the fields of micro-assembly and micro-operation, particularly relates to a micro-vision technology, and particularly relates to observation and operation of objects with cross-scale in a micro-visual field space.
Background
The microscopic vision system is a key device for observing tiny parts, tiny objects and cells. In order to clearly observe the global morphology information of objects with different sizes, the depth of field of the micro-vision system needs to be adjusted according to the sizes of the objects, which increases the operation difficulty and reduces the observation precision of the micro-vision system. Microscopic vision systems can see tiny objects clearly under high resolution and high magnification, but the inverse relation between the resolution and the depth of field is the inherent characteristic of the microscopic vision systems, so that the microscopic vision systems cannot observe information in a larger depth range. Thus, the high resolution and large depth of field conflict limit the performance of the micro-vision system. In the field of micro-assembly, aiming at the assembly of objects or parts with different dimensions, the micro-vision system cannot obtain the full view of all the objects to be assembled in depth, when one part of the parts or the objects are observed, the other part of the parts exceeds the depth of field of the micro-vision system and cannot be observed simultaneously, so that an effective control signal cannot be provided for executing the parts when the objects are assembled or operated, and the assembly or the operation cannot be carried out smoothly. Based on the above problems, there is a need to develop a solution for the microscopic depth of field extension of a microscopic vision system.
Disclosure of Invention
The invention aims to provide a method and a system for digitally expanding the depth of field of a microscopic vision system based on a computer microscopic vision tomography technology, aiming at the problems of large technical difficulty, low precision, incapability of observation and the like of an observation task caused by the contradiction problem that a microscopic vision system has high resolution and large depth of field and is difficult to meet simultaneously in the observation of multi-scale parts.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a digital extension method of depth of field of microscopic visual system based on computer microscopic visual tomography, it mainly uses computer microscopic visual tomography technology to scan the microscopic visual field space of the micro-assembly system by using the microscopic visual system, to obtain the tomography image; the microscopic depth of field extension method based on the computer microscopic visual tomography technology utilizes a tomography image sequence acquired by the computer microscopic visual tomography technology to be combined with step length reconstruction of a precision positioning system to acquire a corresponding three-dimensional tomography visual field space, and calculates the digital information of the three-dimensional tomography visual field space to further acquire the digital microscopic visual field space with extended depth of field, thereby realizing the digital extension of the microscopic depth of field. The method comprises the following specific steps:
2.1, acquiring a corresponding three-dimensional fault view field space sequence vector by utilizing a fault scanning image sequence and a precision positioning system step length sequence;
2.2 removing information outside the three-dimensional fault view field space;
2.3, rasterizing and grid numerating the three-dimensional fault view field space sequence to obtain the digital information of the three-dimensional fault view field space;
2.4 according to three-dimensional faultsDigital information of visual field space, digital three-dimensional microscopic visual field space S for calculating depth of field extensioneAnd the microscopic depth of field of the microscopic field space is expanded.
Further, in step 1, acquiring a tomographic image of the microscopic field space, and performing tomographic scanning on the microscopic visual space along the optical axis direction of the microscopic visual space by using a computer microscopic visual tomographic scanning technology and controlling a microscopic visual system through a precision positioning system, so as to acquire a two-dimensional tomographic image of a local tomographic space of the microscopic visual space, wherein the specific contents are as follows:
(1.1) determining the step length, the moving direction, the moving mode, the moving speed, the initial position and the vertical distance D between the objective lens of the micro-vision system and the origin of a defined coordinate system at the initial position of the precision positioning system for computer micro-vision tomographycThe position of the principal point of the image of the optical axis of the microscopic vision system passing through the focal plane is (x)0，y0) (ii) a Determining the field resolution, the depth of field, the pixel size and the magnification of the corresponding microscopic vision system, and setting the appropriate light intensity of the light source.
(1.2) the precision positioning system (1) controls the microscopic vision system (2) to carry out image tomography along a coordinate axis of a defined coordinate system (7) with a certain movement step length to obtain a two-dimensional tomography image sequence in the Z-axis direction, and the step length of the precision positioning system is delta. The sequence of tomographic images is recorded and the displacement of the precision positioning system is as follows:
Imgz＝[Img1Img2… Imgk… ImgN]
Dz＝[D1D2… Dk… DN]
wherein ImgzVector constructed by tomographic image sequence obtained by tomographic scanning of a precision positioning system control microscopic vision system in a micro-assembly system, N is the scanning frequency of the precision positioning system control microscopic vision system along the Z-axis direction, DzAnd controlling the displacement vector of the microscopic vision system for the precise positioning system during tomography. Displacement D after k-th movement of precision positioning systemkThe relation with its step size Δ is as follows:
Dk＝(k-1)Δ，k＝1，2，…，N
further, in the step 2, (2.1) reconstructing a three-dimensional fault view field space sequence of each fault scanning image based on a fault scanning image sequence obtained by scanning of a computer micro-vision fault scanning technology and realized by combining the movement step length of a precision positioning system; setting the Height (Height) of a visual field of a microscopic visual system (2) as H, the Width (Width) of the visual field as W, the step length of a precise positioning system as delta, and a tomographic image sequence Img scanned along the Z axiszThe corresponding three-dimensional fault space sizes are all H multiplied by W multiplied by delta, and the obtained three-dimensional fault view field space sequence vector is as follows:
Sz＝[S1S2… Sk… SN]
in the formula SkComprises the following steps:
in the formula xk、yk、zkD is the range in X-axis, Y-axis and Z-axis directions of the coordinate system (7)2Is the object distance of the microscopic visual system (2).
(2.2) removing information outside the three-dimensional fault view field space sequence; three-dimensional tomographic field space S in Z-axis directionkThe displacement of the corresponding precision positioning system along the Z-axis direction is Dk(ii) a Object distance d corresponding to the image distance of the microscopic vision system (2)2In the X-axis direction interval of
In the Y-axis direction interval of
In the Z-axis direction interval of
Signals within the range are all three-dimensional tomography space SkRemoving signals not within the range;
(2.3) three-dimensional tomographic field of view skyRasterizing and grid numeralization of the inter-sequence to obtain digitized information of a three-dimensional fault view field space; for three-dimensional fault view field space SkSetting a grid cube of n x n pixel points, using
Grid cube pair three-dimensional fault view field space SkDiscretizing, and constructing a three-dimensional digital matrix according to the position of the grid cube and the function value of the grid cube
And (4) showing. The number N of the pixel points in each grid cube is set to be 1pixSetting the assignment threshold value of the grid cube as TH, if N ispixIf TH is greater, the grid cube is assigned a value of 1, otherwise the grid cube is assigned a value of 0. Three-dimensional tomographic field of view space SkIn (p)i，qi，ri) The location grid cube has an assignment function of
Namely:
wherein
pi∈[1 2 … p]，qi∈[1 2 … q]，ri∈[1 2 … r]，Npix(pi，qi，ri) For three-dimensional fault view field space SkThe median position is (p)i，qi，ri) The number of the pixel points in the grid cube is 1.
(2.4) calculating the digital three-dimensional microscopic field of view with extended depth of field according to the three-dimensional fault field of view space digital information along the Z-axis directionAnd the space realizes the microscopic depth of field extension of the microscopic field space. Digital three-dimensional microscopic field space S for calculating depth of field extensioneThe specific process is as follows:
(2.4.1) calculating two adjacent three-dimensional tomographic spaces SkAnd Sk+1Can splice the digital matrix G of the calculationk、Gk+1. Defining a precise positioning system (1) to control a microscopic vision system (2) to carry out tomography (Flag) along the positive direction of a Z axisz1) from two adjacent three-dimensional fault spaces SkAnd Sk+1The digitized matrix is
Then:
when the precise positioning system (1) controls the micro-vision system (2) to carry out tomography (Flag) along the Z-axis negative directionzWhen-1):
(2.4.2) calculating an extended microscopic field space S of the microscopic Vision System (2)eDigital information G ofeComprises the following steps:
wherein]' denotes a matrix transpose, FlagzFor recording the direction of scanning along the Z-axis of the defined coordinate system. Digital information GeExtended depth of field microscopic field space S along Z-axiseSize H × W × DeDepth of field D of the extended microscopic field spaceeComprises the following steps:
De＝N×Δ
where N is the number of tomographic images acquired.
The invention further provides a digital extension system of the microscopic depth of field of the microscopic vision system based on the computer microscopic vision tomography technology, which comprises a precision positioning system (1), a microscopic vision system (2) and a host computer (25).
The precise positioning system (1) is used for driving the micro vision system (2) to move along the optical axis direction of the micro vision system and carrying out precise positioning; the device comprises a motion device for realizing one-dimensional precise motion, a high-precision positioning motion driving actuator for realizing the matching of positioning precision and the depth of field of a microscopic vision system, and a controller;
the microscopic vision system (2) is used for carrying out image tomography to obtain a tomography image sequence; the device comprises a microscopic amplification unit, an imaging unit and a control unit, wherein the microscopic amplification unit is used for amplifying an imaging object in a microscopic field space through an optical microscope or an electron microscope, and the imaging unit is used for imaging the object in the microscopic field space through a CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) camera;
the host computer (25) is used for controlling and calculating the precision positioning system (1) and the microscopic vision system (2) and displaying a digital microscopic view field space result.
The invention can be extended to realize the field depth extension of the microscopic vision system by utilizing a binocular and multi-eye microscopic vision system, and provides more detailed information such as space object distribution, size, shape and the like for the super-field depth and cross-scale micro-assembly system.
The invention has the following advantages:
(1) aiming at the contradiction between the high resolution and the depth of field range of the microscopic vision system, the invention enlarges the clear imaging range of the microscopic vision system under the condition of keeping the high resolution of the microscopic vision system;
(2) compared with the existing method for improving the depth of field by adjusting the magnification factor in the microscopic vision system, the method avoids the problem of calculation error caused by the fact that the internal and external parameters of the camera model are changed without repeatedly calibrating the camera due to the adjustment of the magnification factor;
(3) the three-dimensional microscopic field space after the field depth expansion is obtained by utilizing the digitization technology, so that the defect that the definition of the whole image is reduced due to the fact that the field depth expansion is obtained through image fusion in the prior art is avoided;
(4) the space occupation condition of the part is obtained through the three-dimensional digital microscopic field space after the field depth is expanded, the three-dimensional information of the part is obtained, and conditions are provided for high-precision reconstruction of the part.
Because the computer microscopic vision tomography technology acquires microscopic views of different fault spaces, each fault space is digitally reconstructed, and the digital global information of the whole microscopic field space is acquired according to the digital information of all the acquired fault spaces, the unified description of the information of the super depth of field and the information of the space in the depth of field range is realized, the limitation that the information of the super depth of field cannot be accurately acquired due to the small depth of field of the microscopic vision system is broken through, and the depth of field extension is realized. The method not only overcomes the contradiction between high resolution and large depth of field, but also expresses the microscopic field space information of the micro-assembly system with ultra-large depth of field by adopting digital information, and vividly and visually represents the three-dimensional object information of the microscopic field space with ultra-large depth of field.
Drawings
FIG. 1 is a schematic diagram of a computer microscopy tomography technique;
FIG. 2 is a schematic view of scanning of a fine positioning system with different size relationships between the motion step and the depth of field;
FIG. 3 is a block diagram of a computer micro-vision tomography implementation of a precision positioning system with a displacement sensor;
FIG. 4 is a block diagram of a computer micro-vision tomography implementation system of a conventional precision positioning system;
FIG. 5 is a schematic view of the digital extension of the microscopic depth of field of the micro-vision system of the micro-assembly system of the binocular orthogonal micro-vision system;
FIG. 6 is a computer microscopic vision tomography implementation system structure diagram of a binocular orthogonal microscopic vision system of a precision positioning system with a displacement sensor;
FIG. 7 is a structural diagram of a computer microscopic vision tomography implementation system of a binocular orthogonal microscopic vision system of a conventional precision positioning system;
FIG. 8 is a schematic diagram of a computed microscopy tomography scanning technique for a multi-view microscopy system;
FIG. 9 is a block diagram of a computer micro-vision tomography implementation system of a multi-view orthogonal micro-vision system of a conventional precision positioning system;
fig. 10 is a computer microscopic vision tomography implementation system structure diagram of a multi-view orthogonal microscopic vision system of a precision positioning system with a displacement sensor.
In the figure: 1. the precise positioning system I, 2, the microscopic vision system I, 3, the field of view width, 4, the motion step length of the precise positioning system I, 5, the range of the microscopic field of view spatial depth extension, 6, the workbench, 7, the defined coordinate system, 8, the field of view height, 9, the three-dimensional spatial depth range of the 1 st acquired tomographic image reconstruction, 10, the three-dimensional spatial depth range of the 2 nd acquired tomographic image reconstruction, 11, the field depth of the 1 st acquired tomographic image, 12, the field depth of the 2 nd acquired tomographic image, 13, the field depth of the 3 rd acquired tomographic image, 14, the step length of which is equal to the field depth, 15, the step length of which is greater than the field depth, 16, the step length of which is less than the field depth of the kth scanning step length, 17, the step length of which is less than the field depth of the kth +1 th scanning step length, 18, the step length of which is less than the field depth of the kth +2, 20. the step length is larger than the depth of field of the depth of field, 21, the step length is larger than the non-scanning area of the depth of field, 22, the kth scanning step length is smaller than the depth of field of the depth of field, 23, the kth +1 scanning step length is smaller than the depth of field of the depth of field, 24, the kth +2 scanning step length is smaller than the depth of field of the depth of field, 25, a host computer, 26, an image acquisition card, 27, a light source controller, 28, a precision positioning system controller, 29, a displacement sensor controller, 30, a micro clamp controller, 31, a displacement sensor of the precision positioning system I, 32, a coaxial light source of the micro vision system I, 33, a micro clamp system, 34, parts, 35, a lifting rotating workbench, 36, a three-dimensional space depth range reconstructed by a tomographic image acquired at the 1 st time of the micro vision system II, 37, a three-dimensional space depth range reconstructed by a tomographic image acquired at the 2 nd time of the micro vision system, 39 microscopic vision system II, 40, precise positioning system II, 41, step length of the precise positioning system II, 42, depth of field of the 1 st acquired tomographic image of the microscopic vision system II, 43, depth of field of the 2 nd acquired tomographic image of the microscopic vision system II, 44, depth of field of the 3 rd acquired tomographic image of the microscopic vision system II, 45, displacement sensor of the precise positioning system II, 46, coaxial light source of the microscopic vision system II, 47, precise positioning system III, 48, microscopic vision system III, 49, step length of the precise positioning system III, 50, coaxial light source of the microscopic vision system III, 51 and displacement sensor of the precise positioning system III. ,
Detailed Description
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The preferred embodiments are merely illustrative of the present invention and are not intended to limit the scope of the present invention.
Example 1:
fig. 1 shows a microscopic depth of field extension system based on computer microscopic vision tomography for a monocular microscopic vision system, which has a precision positioning system I (1) controlling a microscopic vision system I (2) to perform tomography on a microscopic field space along a Z-axis direction defining a coordinate system (7), and a step length constraint for performing tomography by the precision positioning system is shown in fig. 2. According to the schematic diagram of the relationship between the step length of the precision positioning system and the depth of field of the micro-vision system shown in fig. 2, when Δ ═ DOF is shown in reference numeral (14), the depth of the three-dimensional tomographic space of the micro-vision system performing tomographic scanning along the Z axis is DOF, and when Δ ═ DOF is shown in reference numeral (14), the three-dimensional space is H × W × DOF; if Δ < DOF as indicated by reference numerals (16, 17, 18), the three-dimensional tomographic field space size is: h × W × Δ; if Δ > DOF, as shown by reference numeral (15), the three-dimensional tomographic field space size at this time is: h × W × Δ; however, the area is larger than the maximum spatial area that can be clearly imaged, i.e., a depth of field area, and there is an unclear imaging space as indicated by reference numeral 21 in the tomographic field space at this time, so that the tomographic scanning cannot completely acquire microscopic field spatial information, thereby causing data loss. The hardware system structure of the microscopic depth of field extension system based on the computer microscopic visual tomography technology is shown in fig. 3 and 4; the microscopic vision system I (2) is controlled to perform tomography according to the precise positioning system I (1) shown in FIG. 4. Wherein the microscopic vision system I (2) is used for imaging an object in a microscopic field space by a CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) camera; the precision positioning system I (1) is composed of a motion device for one-dimensional precision motion, a high-precision positioning motion driving device for realizing the matching of the positioning precision and the depth of field of the microscopic vision system and a controller. In order to realize high-precision displacement monitoring of a precision positioning system, a displacement sensor (31) is additionally arranged to realize displacement detection, as shown in fig. 3, wherein the displacement sensor (31) is used for recording high-precision displacement. For this reason, in order to realize the hardware system of the computer microscopic vision tomography technology, a displacement sensor controller (29) is required to be added to match with the displacement sensor.
Example 2:
fig. 5 shows a depth of field extension system for a binocular orthogonal micro vision system, the depth of field extension hardware architecture of which is shown in fig. 6 and 7; in fig. 7, a precision positioning system II (40) is added to control the microscopic vision system II (39) to perform tomography scanning on the microscopic field space in the horizontal direction, and a precision positioning system I (1) controls the microscopic vision system I (2) to perform tomography scanning on the microscopic field space in the vertical direction. In order to realize high-precision displacement monitoring of the precision positioning systems (1, 40), displacement sensors are respectively additionally arranged to realize displacement detection as shown in fig. 6, wherein the displacement sensors (31, 45) are respectively used for recording high-precision displacement of the precision positioning systems (1, 40). In order to realize a hardware system of the computer microscopic vision tomography technology, a displacement sensor controller (29) is required to be added to match with a displacement sensor device.
Example 3:
fig. 8 shows a depth of field extension system for a trinocular orthogonal micro-vision system, in which a precision positioning system III (47) and a control micro-vision system III (48) are added, and the precision positioning systems (1, 40, 47) respectively control the micro-vision systems (2, 39, 48) to perform tomography along the Z-axis, the X-axis and the Y-axis of a defined coordinate system (7) to respectively acquire a sequence of tomographic images. The hardware system structure of the trinocular orthogonal microscopic vision system based on the microscopic field depth expansion of the computer microscopic vision tomography technology is shown in fig. 9 and 10. In fig. 9, a precise positioning system I (1) controls a microscopic vision system I (2) to perform tomography scanning on a microscopic field space in the vertical direction, a precise positioning system II (40) controls a microscopic vision system II (39) to perform tomography scanning on the microscopic field space in the horizontal direction on the left side, and a precise positioning system III (47) controls a microscopic vision system III (48) to perform tomography scanning on the microscopic field space in the horizontal direction on the back side, thereby respectively acquiring a tomography image sequence and displacement amounts of three precise positioning systems. In order to realize accurate measurement of displacement of the precision positioning system and improve the precision of a digital reconstruction three-dimensional tomography space, displacement sensors are respectively additionally arranged on the precision positioning systems (1, 40 and 47) as shown in fig. 10. In fig. 10, the displacement sensors (31, 45, 51) record the displacement amounts of the precision positioning systems (1, 40, 47), respectively. In order to realize a hardware system of the high-precision displacement measurement computer micro-vision tomography technology, a displacement sensor controller (29) is required to be added to be matched with a displacement sensor device.
Example 4:
by the systems shown in the above embodiments 1 to 3, the system can obtain information such as a tomographic image, and further perform the following processing on the information by the host computer (25), and finally obtain the effect of the microscopic depth of field extension, the process is as follows:
step 1.1 determining step length, moving direction, moving mode, moving speed, initial position of precision positioning system for computer micro-vision tomography and vertical distance D between objective lens of micro-vision system and origin of defined coordinate system at initial positioncThe position of the principal point of the image of the optical axis of the microscopic vision system passing through the focal plane is (x)0，y0) (ii) a Determining the field resolution, the depth of field, the pixel size and the magnification of the corresponding microscopic vision system, and setting the appropriate light intensity of the light source.
Step 1.2, the precision positioning system (1) controls the micro vision system (2) to carry out image tomography along a coordinate axis of a defined coordinate system (7) with a certain movement step length to obtain a two-dimensional tomography image sequence in the Z-axis direction, and the step length of the precision positioning system is delta. The sequence of tomographic images is recorded and the displacement of the precision positioning system is as follows:
Imgz＝[Img1Img2… Imgk… ImgN]
Dz＝[D1D2… Dk… DN]
wherein ImgzVector constructed by tomographic image sequence obtained by tomographic scanning of a precision positioning system control microscopic vision system in a micro-assembly system, N is the scanning frequency of the precision positioning system control microscopic vision system along the Z-axis direction, DzAnd controlling the displacement vector of the microscopic vision system for the precise positioning system during tomography. Displacement D after k-th movement of precision positioning systemkThe relation with its step size Δ is as follows:
Dk＝(k-1)Δ，k＝1，2，…，N
and 2.1, acquiring a corresponding three-dimensional fault view field space sequence vector by utilizing a fault scanning image sequence and a precision positioning system step length sequence. Specifically, a three-dimensional tomographic view field space sequence of each tomographic image is reconstructed based on a tomographic image sequence obtained by scanning through a computer microscopic vision tomographic scanning technology and realized by combining the motion step length of a precision positioning system; setting the Height (Height) of a visual field of a microscopic visual system (2) as H, the Width (Width) of the visual field as W, the step length of a precise positioning system as delta, and a tomographic image sequence Img scanned along the Z axiszThe corresponding three-dimensional fault space sizes are all H multiplied by W multiplied by delta, and the obtained three-dimensional fault view field space sequence vector is as follows:
Sz＝[S1S2… Sk… SN]
in the formula SkComprises the following steps:
step 2.2, removing information outside the three-dimensional fault view field space;
three-dimensional tomographic field space S in Z-axis directionkThe displacement of the corresponding precision positioning system along the Z-axis direction is Dk(ii) a Object distance d corresponding to the image distance of the microscopic vision system (2)2In the X-axis direction interval of
In the Y-axis direction interval of
In the Z-axis direction interval of
Signals within the range are all three-dimensional tomography space SkRemoving signals not within the range.
Step 2.3, rasterizing and grid numerating the three-dimensional fault view field space sequence to obtain the digital information of the three-dimensional fault view field space:
for three-dimensional fault view field space SkSetting a grid cube of n x n pixel points, using
Grid cube pair three-dimensional fault view field space SkDiscretizing, and constructing a three-dimensional digital matrix according to the position of the grid cube and the function value of the grid cube
And (4) showing. The number N of the pixel points in each grid cube is set to be 1pixSetting a grid standThe cube assignment threshold is TH if
Then the grid cube is assigned a value of 1 and otherwise it is assigned a value of 0. Three-dimensional tomographic field of view space SkIn (p)i，qi，ri) The location grid cube has an assignment function of
Namely:
wherein
pi∈[1 2 … p]，qi∈[1 2 … q]，ri∈[1 2 … r]，Npix(pi，qi，ri) For three-dimensional fault view field space SkThe median position is (p)i，qi，ri) The number of the pixel points in the grid cube is 1.
And 2.4, calculating a digital three-dimensional microscopic field space with extended depth of field according to the three-dimensional fault field space digital information along the Z-axis direction, and realizing the microscopic depth of field extension of the microscopic field space. Digital three-dimensional microscopic field space S for calculating depth of field extensioneThe specific process is as follows:
step 2.4.1 calculating two adjacent three-dimensional fault spaces SkAnd Sk+1Can splice the digital matrix G of the calculationk、Gk+1. Defining a precise positioning system (1) to control a microscopic vision system (2) to carry out tomography (Flag) along the positive direction of a Z axisz1) from two adjacent three-dimensional fault spaces SkAnd Sk+1The digitized matrix is
Then:
when the precise positioning system (1) controls the micro-vision system (2) to carry out tomography (Flag) along the Z-axis negative directionzWhen-1):
step 2.4.2 calculating the extended microscopic field space S of the microscopic Vision System (2)eDigital information G ofeComprises the following steps:
wherein]' denotes matrix transposition, FlagzFor recording the direction of scanning along the Z-axis of the defined coordinate system. Digital information GeExtended depth of field microscopic field space S along Z-axiseSize H × W × DeDepth of field D of the extended microscopic field spaceeComprises the following steps:
De＝N×Δ
where N is the number of tomographic images acquired.
For the depth of field extension of binocular and trinocular orthogonal microscopic vision systems, the increased X direction and Y direction are processed in the same way.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and it is apparent that those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A digital extension method for microscopic depth of field of a microscopic vision system is characterized by comprising the following steps:
(1) adopting a computer microscopic vision tomography scanning technology to obtain a tomography image sequence and a displacement sequence of a precision positioning system;
(2) reconstructing a three-dimensional fault view field space according to the obtained fault scanning image sequence and the step length of the precision positioning system, and calculating digital information of the three-dimensional fault space to obtain a digital microscopic view field space with extended depth of field, so as to realize digital extension of the microscopic depth of field; the step (2) specifically comprises the following steps:
(2.1) acquiring a corresponding three-dimensional fault view field space sequence vector by utilizing a fault scanning image sequence and a precision positioning system step length sequence;
(2.2) removing information outside the three-dimensional fault view field space;
(2.3) rasterizing and grid numerating the three-dimensional fault view field space sequence to obtain the digital information of the three-dimensional fault view field space;
(2.4) calculating the digital three-dimensional microscopic field space S with extended depth of field according to the digital information of the three-dimensional fault field spaceeAnd the digital extension of the microscopic depth of field of the microscopic field space is realized.
2. The method of claim 1, wherein: the computer microscopic visual tomography scanning technology of the step (1) comprises the following steps:
(1.1) determining the step length, the motion direction, the motion mode, the motion speed, the initial position and the vertical distance D between the objective lens of the micro-vision system and the origin of a defined coordinate system when the micro-vision system carries out the micro-vision tomographycThe position of the principal point of the image of the optical axis of the microscopic vision system passing through the focal plane is (x)0，y0) (ii) a Determining the field resolution, the depth of field, the pixel size and the magnification of a corresponding microscopic vision system, and setting proper light intensity of a light source;
(1.2) the precision positioning system (1) controls the microscopic vision system (2) to carry out image tomography with a certain movement step length along the coordinate axis of the defined coordinate system (7) to obtain a Z-axis directionThe step length of a precision positioning system is delta, and the recording tomography image sequence is ImgzAnd the displacement sequence of the precision positioning system is Dz：
Imgz＝[Img1Img2… Imgk… ImgN]
Dz＝[D1D2… Dk… DN]
Wherein ImgzVector constructed by tomographic image sequence obtained by tomographic scanning of a precision positioning system control microscopic vision system in a micro-assembly system, N is the scanning frequency of the precision positioning system control microscopic vision system along the Z-axis direction, DzControlling a displacement vector of a microscopic vision system for a precise positioning system during tomography;
displacement D after k-th movement of precision positioning systemkThe relation with its step size Δ is as follows:
Dk＝(k-1)Δ，k＝1，2，…，N。
3. the method according to claim 2, characterized in that the precision positioning system motion step size Δ of the tomography needs to satisfy: delta is less than or equal to DOF
Where DOF is the depth of field of the microscopic vision system performing the tomography.
4. A method according to any one of claims 1-3, characterized in that: the step (2.1) acquires the corresponding three-dimensional fault view field space by combining the acquired fault scanning image sequence with the step length sequence of the precision positioning system as follows: setting the height of a visual field of the microscopic vision system (2) as H, the width of the visual field as W, the step length of the precise positioning system as delta, and scanning a tomographic image sequence Img along the Z axiskThe corresponding three-dimensional fault space sizes are all H multiplied by W multiplied by delta, and the obtained three-dimensional fault view field space sequence vector is as follows:
Sz＝[S1S2… Sk… SN]
in the formula SkComprises the following steps:
in the formula xk、yk、zkD is the range in X-axis, Y-axis and Z-axis directions of the coordinate system (7)2Is the object distance of the microscopic visual system (2).
5. A method according to any one of claims 1-3, characterized in that: the method for removing the information out of the three-dimensional fault view field space in the step (2.2) is as follows: three-dimensional tomographic field space S in Z-axis directionkThe displacement of the corresponding precision positioning system along the Z-axis direction is DkThe vertical distance between the objective lens of the micro-vision system and the origin of the defined coordinate system at the initial position of the precision positioning system is DcAccording to the corresponding object distance d of the microscopic vision system (2)2In the X-axis direction interval of
In the Y-axis direction interval of
In the Z-axis direction interval of
All signals within the range of (A) are three-dimensional tomographic space SkRemoving signals not within the range.
6. A method according to any one of claims 1-3, characterized in that: the step (2.3) of rasterizing and grid digitizing the three-dimensional fault view field space sequence to obtain the digitized information of the three-dimensional fault view field space comprises the following steps: for three-dimensional fault view field space SkSetting a grid cube of n x n pixel points, using
A grid cube pair of threeDimension fault field space SkDiscretizing, and constructing a three-dimensional digital matrix according to the position of the grid cube and the function value of the grid cube
Representing, setting the number N of pixel points in each grid cube to be 1pixSetting the assignment threshold value of the grid cube as TH, if N ispixIf the grid cube is more than or equal to TH, the value of the grid cube is 1, otherwise, the value of the grid cube is 0, and the three-dimensional fault view field space S is obtainedkIn (p)i，qi，ri) The location grid cube has an assignment function of
Namely:
wherein
pi∈[1 2 … p]，qi∈[1 2 … q]，ri∈[1 2 … r]，Npix(pi，qi，ri) For three-dimensional fault view field space SkThe median position is (p)i，qi，ri) The number of the pixel points in the grid cube is 1.
7. A method according to any one of claims 1-3, characterized in that: the step (2.4) of calculating the digital three-dimensional microscopic field space S with the extended depth of fieldeThe specific process is as follows:
(2.4.1) calculating two adjacent three-dimensional tomographic spaces SkAnd Sk+1Can splice the digital matrix G of the calculationk、Gk+1: when precisely fixedThe position system (1) controls the microscopic vision system (2) to carry out tomography Flag along the positive direction of the Z axiszAccording to two adjacent three-dimensional fault spaces S ═ 1kAnd Sk+1The digitized matrix is
Then:
when the precise positioning system (1) controls the micro-vision system (2) to carry out tomography Flag along the Z-axis negative directionzWhen the ratio is-1:
(2.4.2) calculating an extended microscopic field space S of the microscopic Vision System (2)eDigital information G ofeComprises the following steps:
wherein]' denotes matrix transposition, FlagzFor recording the direction of scanning along the Z-axis of a defined coordinate system, digitising information GeExtended depth of field microscopic field space S along Z-axiseSize H × W × DeDepth of field D of the extended microscopic field spaceeComprises the following steps:
De＝N×Δ
where N is the number of tomographic images acquired.
8. The method according to any one of claims 1 to 3, wherein the method is applied to microscopic vision systems being monocular, binocular, trinocular and multiocular, microscopic depth of field extensions of microscopic vision systems applied to microfabricated, micromanipulation or cell manipulation systems.
9. Digital extension system of microscopic depth of field of a microscopic vision system implementing the method according to any of claims 1 to 3, comprising a precision positioning system (1), a microscopic vision system (2) and a host computer (25),
the precise positioning system (1) is used for driving the micro vision system (2) to move along the optical axis direction of the micro vision system and carrying out precise positioning; the device comprises a motion device for realizing one-dimensional precise motion, a high-precision positioning motion driving actuator for realizing the matching of positioning precision and the depth of field of a microscopic vision system, and a controller;
the microscopic vision system (2) is used for carrying out image tomography to obtain a tomography image sequence; the device comprises a microscopic amplification unit, an imaging unit and a control unit, wherein the microscopic amplification unit is used for amplifying an imaging object in a microscopic field space through an optical microscope or an electron microscope, and the imaging unit is used for imaging the object in the microscopic field space through a CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) camera;
the host computer (25) is used for controlling and calculating the precision positioning system (1) and the microscopic vision system (2) and displaying a digital microscopic view field space result.
10. The system of claim 9, further configured with a displacement metric system, controlling slice position and recording position information of the obtained slices; the device comprises a displacement sensor which is arranged on a motion mechanism of a precision positioning system to realize displacement sensing, and a precision positioning system controller and a displacement sensor controller which are used for carrying out guide rail control motion feedback control.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810622582.5A CN108955562B (en) | 2018-06-15 | 2018-06-15 | Digital extension method and system for microscopic depth of field of microscopic vision system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810622582.5A CN108955562B (en) | 2018-06-15 | 2018-06-15 | Digital extension method and system for microscopic depth of field of microscopic vision system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108955562A CN108955562A (en) | 2018-12-07 |
| CN108955562B true CN108955562B (en) | 2020-06-16 |
Family
ID=64489251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810622582.5A Active CN108955562B (en) | 2018-06-15 | 2018-06-15 | Digital extension method and system for microscopic depth of field of microscopic vision system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108955562B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111121675B (en) * | 2019-12-11 | 2021-09-03 | 南京理工大学 | Visual field expansion method for microsphere surface microscopic interferometry |
| CN112903236A (en) * | 2021-01-29 | 2021-06-04 | 上海交通大学 | Focal plane scanning-based aerodynamic thermal parameter optical test device and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104111242A (en) * | 2014-06-17 | 2014-10-22 | 费鹏 | Three dimensional pixel super-resolution microscopic imaging method |
| CN105180806A (en) * | 2015-08-25 | 2015-12-23 | 大连理工大学 | Trans-scale geometrical parameter measurement method based on microscopic visual sense |
| CN107144241A (en) * | 2017-06-09 | 2017-09-08 | 大连理工大学 | A kind of binocular vision high-precision measuring method compensated based on the depth of field |
-
2018
- 2018-06-15 CN CN201810622582.5A patent/CN108955562B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104111242A (en) * | 2014-06-17 | 2014-10-22 | 费鹏 | Three dimensional pixel super-resolution microscopic imaging method |
| CN105180806A (en) * | 2015-08-25 | 2015-12-23 | 大连理工大学 | Trans-scale geometrical parameter measurement method based on microscopic visual sense |
| CN107144241A (en) * | 2017-06-09 | 2017-09-08 | 大连理工大学 | A kind of binocular vision high-precision measuring method compensated based on the depth of field |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108955562A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7646938B2 (en) | Method and system for super-resolution of confocal images acquired through an image guide, and device used for implementing such a method | |
| US10809515B2 (en) | Observation method and specimen observation apparatus | |
| CN109559355B (en) | Multi-camera global calibration device and method without public view field based on camera set | |
| JP4806630B2 (en) | A method for acquiring optical image data of three-dimensional objects using multi-axis integration | |
| US20090295963A1 (en) | Method and apparatus and computer program product for collecting digital image data from microscope media-based specimens | |
| JP2013117848A (en) | Image processing apparatus and image processing method | |
| JP6061619B2 (en) | Microscope system | |
| CN108955562B (en) | Digital extension method and system for microscopic depth of field of microscopic vision system | |
| JP2013068617A (en) | Machine vision inspection system and method for determining position measurement result of the system | |
| CN109272575B (en) | Method for improving modeling speed of digital slice scanner | |
| JP2015108837A (en) | Image processing apparatus and image processing method | |
| Sahay et al. | Dealing with parallax in shape-from-focus | |
| CN113884519A (en) | Self-navigation X-ray imaging system and imaging method | |
| CN108983702B (en) | Computer microscopic visual slice scanning technology-based microscopic visual field digital extension method and system for microscopic visual system | |
| CN107590787B (en) | Image distortion correction method of scanning electron microscope | |
| US11438570B2 (en) | Data processing apparatus and data processing method for generation of calibration data for performing image processing | |
| JP2016099318A (en) | Stereo matching device, stereo matching program, and stereo matching method | |
| CN108961419B (en) | Microscopic visual field space digitizing method and system for microscopic visual system of micro assembly system | |
| CN108876838B (en) | Method and system for digitizing micro-operation space of micro-operation system | |
| JP5925109B2 (en) | Image processing apparatus, control method thereof, and control program | |
| US8914255B2 (en) | Method, apparatus, and computer program product for focus prediction | |
| CN108897279B (en) | Digital extension method and system for field and depth of field of microscopic vision system of micro-assembly system | |
| CN108734763B (en) | Method and system for digitizing micro-assembly space of micro-assembly system | |
| Sahay et al. | Shape extraction of low‐textured objects in video microscopy | |
| Wang et al. | Depth measurement for the objects with a small height using depth-focus-based microscopic vision system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |