CN117670784A - Height measurement method based on visual imaging detection equipment and related device - Google Patents
Height measurement method based on visual imaging detection equipment and related device Download PDFInfo
- Publication number
- CN117670784A CN117670784A CN202211058029.6A CN202211058029A CN117670784A CN 117670784 A CN117670784 A CN 117670784A CN 202211058029 A CN202211058029 A CN 202211058029A CN 117670784 A CN117670784 A CN 117670784A
- Authority
- CN
- China
- Prior art keywords
- detected
- area
- image
- point cloud
- height
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 47
- 230000000007 visual effect Effects 0.000 title claims abstract description 47
- 238000001514 detection method Methods 0.000 title claims abstract description 46
- 238000000691 measurement method Methods 0.000 title claims abstract description 18
- 230000000875 corresponding effect Effects 0.000 claims abstract description 62
- 230000002596 correlated effect Effects 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims description 25
- 238000005259 measurement Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000003709 image segmentation Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 229910000679 solder Inorganic materials 0.000 description 26
- 235000012431 wafers Nutrition 0.000 description 17
- 238000010586 diagram Methods 0.000 description 15
- 238000004364 calculation method Methods 0.000 description 5
- 238000005530 etching Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000005284 excitation Effects 0.000 description 3
- 230000000877 morphologic effect Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 241001270131 Agaricus moelleri Species 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003628 erosive effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application discloses a height measurement method based on visual imaging detection equipment and a related device. The height measurement method based on the visual imaging detection equipment comprises the following steps: acquiring point cloud data of an object to be detected, wherein the object to be detected is provided with a plurality of areas to be detected, and the surface of each area to be detected is provided with a target to be detected; projecting the point cloud data into a first two-dimensional gray level image, wherein the gray level value of each point cloud pixel point in the first two-dimensional gray level image is positively correlated with the height value of the point cloud in the corresponding point cloud data; dividing an image area of each area to be detected on the object to be detected from the first two-dimensional gray level image; acquiring a height value and a reference height value of a target to be detected in each image area; and calculating the height dimension of each object to be measured according to the height value of each object to be measured and the reference height value. By the aid of the scheme, accuracy of detecting the height of the target to be detected can be improved.
Description
Technical Field
The application relates to the technical field of visual imaging detection, in particular to a height measurement method based on visual imaging detection equipment and a related device.
Background
At present, in the field of semiconductor measurement such as wafers, three-dimensional morphology of the semiconductor such as the wafers can be detected by adopting 3D visual imaging equipment, and further measurement of corresponding dimensional parameters is carried out according to the three-dimensional morphology. The existing 3D visual imaging apparatus is usually an optical measurement apparatus, which can emit detection light through a light source, and then send a formed modulated light beam to the surface of an object to be measured after the detection light modulation, so that the surface of the object to be measured emits corresponding excitation light (reflection light corresponding to the modulated light beam or fluorescence generated by excitation of the modulated light beam), and further obtains an optical signal corresponding to the excitation light by using a sensor, converts the optical signal into a corresponding electrical signal, and sends the electrical signal to a corresponding processor for processing, thereby obtaining a three-dimensional shape corresponding to semiconductor measurement such as a wafer.
In the prior art, a plurality of chips can be arranged on a wafer, and each chip is provided with a plurality of solder balls, when the solder ball height on the chip is detected and detected through a 3D visual imaging device, the whole wafer is generally adopted as a reference, however, when the whole wafer has the problems of warping and the like, or the chip on the wafer has the problem of protruding or sinking in position, the technical problem of inaccurate measurement of the solder ball height on the chip can be caused.
Disclosure of Invention
The main purpose of the present application is to provide a height measurement method based on a visual imaging detection device and a related device, which aims to solve the above technical problems.
To achieve the above object, the present application proposes a height measurement method based on a visual imaging detection apparatus, the measurement data compensation method comprising:
acquiring point cloud data of an object to be detected, wherein the object to be detected is provided with a plurality of areas to be detected, and the middle surface of each area to be detected is provided with a target to be detected;
projecting the point cloud data into a first two-dimensional gray level image, wherein gray level values of pixel points of each point cloud in the first two-dimensional gray level image are positively correlated with height values of point clouds corresponding to the point cloud data;
dividing an image area of each area to be detected on the object to be detected from the first two-dimensional gray level image;
acquiring a height value and a reference height value of the target to be detected in each image area;
and calculating the height dimension of each object to be measured according to the height value of each object to be measured and the reference height value.
Optionally, the step of dividing the image area of each area to be measured on the object to be measured from the first two-dimensional gray scale image includes:
establishing a minimum surrounding area surrounding all the targets to be detected in the area to be detected for each area to be detected, wherein the minimum surrounding area is positioned in the area to be detected, and the minimum surrounding area is not connected with the edge of the area to be detected;
dividing a sub-point cloud corresponding to the minimum surrounding area from the point cloud data according to the coordinate range of the minimum surrounding area;
projecting the sub-point cloud to a corresponding second two-dimensional gray scale image, and setting the second two-dimensional gray scale image as an image area of the area to be detected.
Optionally, the establishing a minimum surrounding area surrounding the at least one target to be measured for each region to be measured includes:
acquiring the minimum external annular shape of all the targets to be detected in the region to be detected;
expanding the minimum external ring by a set distance relative to the target to be detected so as to form a new external ring;
and determining the area surrounded by the new circumscribing ring as the minimum surrounding area.
Optionally, calculating a reference height value from the second two-dimensional gray scale image includes:
dividing initial image data of a reference plane from the second two-dimensional gray level image by adopting an adaptive threshold segmentation method;
dividing missing image data of the target to be detected from the second two-dimensional gray level image according to a set threshold dividing method;
obtaining the image data of the reference plane according to the difference set of the initial image data of the reference plane and the missing image data of the target to be detected;
and obtaining the reference height value according to the average value of the point cloud gray scale in the image data of the reference plane.
Optionally, calculating the reference height value of the target to be measured according to the second two-dimensional gray scale image includes:
and obtaining the reference height value according to the average value of gray values of a plurality of points in the point cloud middle area of the target to be detected.
Optionally, the obtaining the height value according to an average value of gray values of points in a point cloud middle area of the target to be measured includes:
selecting an area of a point cloud pixel point of a preset target to be detected from the second two-dimensional gray level image;
and selecting an average value of gray values of a plurality of point cloud pixel points in the central area of the point cloud pixel points to obtain the height value.
Optionally, the selecting an average value of gray values of a plurality of point cloud pixels in a central area of the point cloud pixel to obtain the height dimension value includes:
taking the central coordinate of the area of the point cloud pixel point as a circle center and taking a first set radius as a radius to make a circle;
and selecting an average value of gray values of a plurality of point cloud pixel points in the circle to obtain the height value.
In order to achieve the above purpose, the present application further provides a height measurement device based on a visual imaging detection device, which is characterized by comprising a data receiving module, an image conversion module, an image segmentation module and an image data processing module which are coupled in sequence;
the data receiving module is used for receiving the acquired point cloud data of the object to be detected, the object to be detected is provided with a plurality of areas to be detected, and the surface in each area to be detected is provided with a target to be detected;
the image conversion module is used for projecting the point cloud data into a first two-dimensional gray level image, wherein gray level values of point cloud pixel points in the first two-dimensional gray level image are positively correlated with height values of point cloud corresponding to the point cloud data;
the image segmentation module is used for segmenting an image area of each area to be detected on the object to be detected from the first two-dimensional gray level image;
the image data processing module is used for acquiring the height value and the reference height value of the target to be detected in each image area; and calculating the height dimension of each object to be measured according to the height value of each object to be measured and the reference height value.
To achieve the above object, the present application further proposes a visual imaging detection apparatus comprising a memory, a processor and a visual imaging detection apparatus-based height measurement program stored on the memory and executable on the processor, the visual imaging detection apparatus-based height measurement program, when executed by the processor, implementing the steps of the visual imaging detection apparatus-based height measurement method as described above.
To achieve the above object, the present application further proposes a computer-readable storage medium having stored thereon a height measurement program based on a visual imaging detection apparatus, which when executed by a processor, implements the steps of the height measurement method based on a visual imaging detection apparatus as described above.
In the technical scheme of the application, therefore, in the embodiment, the image area corresponding to each region to be measured on the object to be measured is segmented from the first two-dimensional gray level image, and then the height value and the reference height value of each object to be measured are obtained through the image area corresponding to each region to be measured, and then the height value of each object to be measured is calculated, so that the reference plane of each object to be measured can be calculated respectively, and the accuracy of detecting the height of each object to be measured can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from the structures shown in these drawings without inventive effort to a person of ordinary skill in the art.
Fig. 1 is a schematic flow chart of an embodiment of a height measurement method based on a visual imaging detection apparatus provided in the present application;
FIG. 2a is a schematic view of an image area of one of the test areas of the test object of the present application;
FIG. 2b is a schematic diagram of an embodiment of a minimum bounding region established in the image region of FIG. 2 a;
FIG. 2c is a schematic diagram of another embodiment of a minimum bounding region established in the image region of FIG. 2 a;
FIG. 2d is a schematic view of an image of the first image data corresponding to the minimum bounding region shown in FIG. 2 c;
FIG. 2e is a schematic diagram of an image of the second image data corresponding to the minimum bounding region shown in FIG. 2 c;
FIG. 2f is a schematic illustration of an image corresponding to a difference set of a set formed by first image data after the etching process and second image data after the etching process;
FIG. 3 is a schematic view of a height measuring device based on a visual imaging detection apparatus according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a visual imaging detection apparatus according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a framework of one embodiment of a computer readable storage medium provided herein.
The realization, functional characteristics and advantages of the present application will be further described with reference to the embodiments, referring to the attached drawings.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
In the present embodiment, if a directional instruction is referred to, the directional instruction is merely used to explain the relative positional relationship, movement condition, etc. between the components in a specific posture, and if the specific posture is changed, the directional instruction is changed accordingly.
In addition, if there is a description of "first", "second", etc. in the embodiments of the present application, the description of "first", "second", etc. is for descriptive purposes only and is not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.
Fig. 1 is a schematic flow chart of an embodiment of a height measurement method based on a visual imaging detection apparatus provided in the present application.
The height measurement method based on the visual imaging detection equipment specifically comprises the following steps:
s110: the method comprises the steps of obtaining point cloud data of an object to be detected, wherein the object to be detected is provided with a plurality of areas to be detected, and each area to be detected is provided with at least one target to be detected.
In this step, a visual imaging detection device (e.g., a 3D camera) may be used to scan the measurement surface side of the object to be measured, so that point cloud data on the measurement surface side of the object to be measured may be obtained. The point cloud data is 3d point cloud data and can represent the three-dimensional morphological characteristics of one side of the measuring surface of the object to be measured. In this embodiment, the object to be measured has a plurality of areas to be measured, and each area to be measured has at least one target to be measured.
For example, the object to be tested may be a wafer, and each wafer may be generally divided into a plurality of chip areas, and each chip area may correspond to the above-mentioned area to be tested; one chip may be generally formed in each chip area of the wafer, and each chip surface is provided with a plurality of solder balls. When the height measurement is carried out on the solder balls arranged on the surface of the chip, the solder balls can form the object to be measured. The surface of the solder ball on the wafer may be a reference plane, and the height of the solder ball relative to the reference plane is the height of the object to be tested.
S120: and projecting the point cloud data into a first two-dimensional gray level image, wherein the gray level value of each point cloud pixel point in the first two-dimensional gray level image is positively correlated with the height value of the point cloud in the corresponding point cloud data.
After the point cloud data of the object to be detected is obtained, the point cloud data can be converted into a first two-dimensional gray level image.
When detecting an object to be detected, a space coordinate system is generally established with the object to be detected (or a carrying platform for carrying the object to be detected) as a reference. The object to be measured can be horizontally arranged, and the vertical direction is used for representing the height direction of the object to be measured. When the visual imaging detection equipment is adopted to acquire the point cloud data of the object to be detected, the spatial coordinates of each point in the point cloud data can be correspondingly acquired.
Converting the point cloud data into a first two-dimensional gray scale image may be represented as projecting the point cloud data into the first two-dimensional gray scale image. Specifically, the coordinates of each point in the point cloud data may be projected onto a horizontal plane, and the gray value of each point is positively correlated with the vertical coordinate, so that a first two-dimensional gray image may be formed.
Specifically, taking a point (X1, Y1, Z1) in the point cloud data as an example, a maximum coordinate value Zmax in the vertical most direction and a minimum coordinate value Zmin in the vertical most direction in the point cloud data may be selected. The coordinates of the point on the first two-dimensional Gray scale image after projection may be represented as (X1, Y1), and the Gray value g1= (Z1-Zmin)/(Zmax-Zmin) ×gray of the point, where Gray may be a set Gray value. For example, gray may be set to 255 or 65535 or the like.
Through the above operation, each point in the point cloud can be projected into the first two-dimensional gray scale image, and each point has a gray scale value corresponding to the vertical coordinate value at the projection point of the first two-dimensional gray scale image. And the larger the vertical coordinate value of the point in the point cloud data is, the higher the height of the point on the surface is, and the larger the gray value corresponding to the point is.
Further, for the wafer, the solder balls are all disposed on the surface of the measurement surface, that is, the plurality of solder balls have the same reference plane (i.e., measurement surface), and the point cloud data obtained by the visual imaging detection device includes the point cloud of the reference plane and the point cloud of the solder balls, and the point cloud of the solder balls is generally higher than the point cloud of the reference plane. Therefore, the classical ransac algorithm can be used for extracting the reference plane, plane correction is carried out on the point cloud of the solder ball according to the extracted reference plane parameters, and then the corrected point cloud is converted into a first two-dimensional gray level image.
S130: and dividing an image area corresponding to each area to be detected on the object to be detected from the first two-dimensional gray level image.
In this step, an image area corresponding to each area to be detected may be further divided from the first two-dimensional gray scale image. Specifically, a plurality of regions to be measured on the object to be measured can be segmented from the first two-dimensional gray scale image. For the wafer, the setting position of each chip on the wafer can be determined according to the chip distribution diagram on the wafer, and then the coordinate area of each chip area can be obtained according to the setting position, and then the image area corresponding to each area to be detected can be divided on the first two-dimensional gray level image according to the coordinate area of each chip area.
S140: and acquiring the height value and the reference height value of the target to be detected in each image area.
In this step, after the image area corresponding to each to-be-detected area is divided on the first two-dimensional gray scale image, the image area corresponding to each to-be-detected area can be extracted, and the height value and the reference height value of the target to be detected in each to-be-detected area can be confirmed based on the image area corresponding to each to-be-detected area.
One of the areas to be tested is taken as an example for illustration. Firstly, the image area of the area to be detected can be divided, the area corresponding to the object to be detected in the area to be detected can be removed, and then the reference height value can be calculated through the residual image part. The height value of the object to be measured can be calculated through the divided area corresponding to the object to be measured.
The reference height value can be obtained by calculating the average value of the gray scales of other areas after the areas corresponding to the objects to be detected in the areas are removed; and calculating the average value of the gray scales of the central area in the to-be-detected target to be segmented, so that the height value of the to-be-detected target can be calculated.
Or in other embodiments, the reference plane corresponding to the reference height value may be calculated by removing the point cloud corresponding to the other region corresponding to the region to be measured, and further, the coordinates of the point cloud corresponding to the central region of the to-be-measured target and the reference plane are adopted to calculate, so that the height dimension of the to-be-measured target can be obtained subsequently.
S150: and calculating the height dimension of the object to be measured according to the height value of each object to be measured and each reference height value.
In this step, the height dimension of the object to be measured may be calculated using the difference between the height value of each object to be measured and the reference height value.
Therefore, in this embodiment, the image area corresponding to each region to be measured on the object is segmented from the first two-dimensional gray level image, and then the height value and the reference height value of each object to be measured are obtained through the image area corresponding to each region to be measured, and then the height value of each object to be measured is calculated, so that the reference plane of each object to be measured can be calculated, and the accuracy of detecting the height of each object to be measured can be improved.
Further, in the present embodiment, taking the object to be measured as an example of a wafer, referring to fig. 2a, fig. 2a is a schematic diagram of an image area of one of the areas to be measured of the object to be measured in the present application. The white part O1 in the image area is the area corresponding to the point cloud of the solder ball; a black area O2 surrounding each white part O1 corresponds to an area corresponding to the missing point cloud of the solder ball; the rest of the gray portion O3 is the area corresponding to the point cloud of the reference plane.
The step of dividing the image area of each area to be measured on the object to be measured from the first two-dimensional gray level image may specifically use the complete image of each divided area to be measured as the image area.
Alternatively, in other embodiments, a portion may be segmented from the complete image of each region to be measured as the image region.
Specifically, referring to fig. 2b, fig. 2b is a schematic diagram illustrating an embodiment of a minimum bounding region established in the image region shown in fig. 2 a. The method comprises the steps that a minimum enclosing area L for enclosing all objects to be tested in a to-be-tested area can be built for each to-be-tested area, the minimum enclosing area L is located in the to-be-tested area, and the minimum enclosing area is not connected with the edge of the to-be-tested area; dividing a sub-point cloud corresponding to the minimum surrounding area from the point cloud data according to the coordinate range of the minimum surrounding area; projecting the sub-point cloud to a corresponding second two-dimensional gray scale image, and setting the second two-dimensional gray scale image as an image area of the area to be detected. The method has the advantages that partial image data of the complete image edge area of the area to be measured can be removed, so that the number of point clouds contained in subsequent data calculation can be reduced, the calculated amount in subsequent height dimension measurement is reduced, and the height dimension measurement rate can be further accelerated.
Taking a region to be tested on a wafer as an example, in a specific embodiment, the region to be tested has a corresponding complete image, the region to be tested corresponds to a single chip, a plurality of solder balls are disposed on the surface of the region to be tested, and each solder ball corresponds to a target to be tested.
When the minimum surrounding area of the area to be measured is acquired, the minimum circumscribing ring surrounding all solder balls can be acquired in the complete image of the area to be measured. Wherein, the minimum external ring shape can be matched with the arrangement shape of all solder balls. The minimum bounding rectangle may then correspond to the minimum bounding region corresponding to the region to be tested. The solder balls are arranged in an array, and the minimum circumscribing ring can be the minimum circumscribing rectangle surrounding all the solder balls.
Alternatively, referring to fig. 2c, fig. 2c is a schematic diagram illustrating another embodiment of a minimum bounding region established in the image region shown in fig. 2 a. After the minimum external ring surrounding all the solder balls is obtained, the minimum external ring can be expanded to a preset size (for example, the minimum external ring is amplified in an equal proportion under the condition that the position of the central point is unchanged), a new external ring is formed, the new external ring surrounds the minimum external ring and does not contact the edge of the region to be detected, and the new external ring can correspond to the minimum surrounding region L corresponding to the region to be detected.
By selecting the minimum surrounding area and selecting the point cloud corresponding to the pixel points in the minimum surrounding area for subsequent calculation of the reference plane of the target to be measured, the surrounding noise point interference can be reduced, the number of the point clouds is reduced, the calculated amount is reduced, and the calculation rate is improved.
In the above embodiment, the plurality of solder balls are arranged in an array, and the formed minimum surrounding area may be a rectangle surrounding all the solder balls; in other embodiments, if the plurality of solder balls are arranged in a circular shape, the minimum surrounding area may be a circular shape surrounding all the solder balls.
Further, in this embodiment, the reference height value corresponding to the reference plane and the height value corresponding to the target to be measured may be obtained through the second two-dimensional gray scale image.
The reference height value corresponding to the reference plane can be obtained by the following steps:
s210: and (3) segmenting the initial image data of the reference plane from the second two-dimensional gray level image by adopting an adaptive threshold segmentation method.
Each pixel in the second two-dimensional gray scale image may then correspond to a point in the point cloud parameter. And the image data of the second two-dimensional gray scale image may include coordinates and gray scale values of each pixel point in the second two-dimensional gray scale image.
In the step, the image data of the second two-dimensional gray scale image is processed by adopting an adaptive threshold segmentation method, so that the first image data is obtained. The first image data corresponds to initial image data of a reference plane.
Referring to fig. 2d, fig. 2d is a schematic diagram of an image of the first image data corresponding to the minimum bounding region shown in fig. 2 c. Wherein, the image is obtained by adopting the image corresponding to the minimum surrounding area to perform further self-adaptive threshold segmentation processing. In fig. xx, a gray area O3 surrounding the black portion O2 and the white portion O1 corresponds to a point cloud of the reference plane.
In this step, morphological erosion processing may be further performed on the first image data, so that the first image data after the erosion processing may be obtained.
S220: and dividing the missing image data of the target to be detected from the second two-dimensional gray level image according to a set threshold dividing method.
In this step, the missing image data of the object to be measured may be segmented from the second two-dimensional gray-scale image by using a set threshold segmentation method.
For example, when the visual imaging detection device is a 3D camera, when the 3D camera performs three-dimensional point cloud acquisition, light rays irradiated on the surface to be detected have a certain inclination angle, so that when the point cloud of the object to be detected is acquired, partial point cloud loss is caused.
In this step, the image data corresponding to the missing point cloud, that is, the missing image data of the object to be measured, may be segmented. For the missing part of the point cloud, the missing part is usually black, that is, the gray value is 0, so that the image data of the second two-dimensional gray image can be processed by adopting a 0-threshold segmentation method, thereby obtaining second image data, and the second image data corresponds to the missing image data of the target to be detected.
Referring to fig. 2e, fig. 2e is a schematic diagram of an image of the second image data corresponding to the minimum bounding region shown in fig. 2 c. The image is obtained by further thresholding the image corresponding to the minimum bounding region as described above. The black portion O2 in the drawing is a region corresponding to the point cloud missing portion of the reference plane.
In this step, morphological expansion processing (expansion coefficient is set) may be further performed on the second image data, so that the expanded second image data can be obtained.
S230: and obtaining the image data of the reference plane according to the difference set of the initial image data of the reference plane and the missing image data of the target to be detected.
In this step, the image data of the reference plane may be further obtained according to a difference set between the initial image data of the reference plane and the missing image data of the target to be measured. Specifically, the image data of the reference plane may be obtained by forming a difference set between the set formed by the data in the first image data and the set formed by the second image data.
Referring to fig. 2f, fig. 2f is a schematic diagram of an image corresponding to a difference set of a set formed by using first image data after etching and second image data after etching in the present application. By adopting the scheme, the contour data and the reference plane data of each object to be measured can be obtained.
S240: and obtaining a reference height value according to the average value of the point cloud gray scale in the image data of the reference plane.
In this step, the reference height value may be further obtained from an average value of gray values of each pixel point in the image data of the reference plane.
Specifically, the image data of each pixel point in the reference plane can be arranged according to the gray value order, so that a certain amount of image data with minimum gray values and maximum gray values can be removed, and the interference of extreme values is reduced. For example, if n pieces of image data are arranged in order from large to small to obtain A1, A2, a3. Wherein, the certain proportion can be 10% -20% of the total amount of the image data. That is, image data accounting for 10% -20% of the total amount of image data from the start data A1 (maximum data) may be removed, and image data accounting for 10% -20% of the total amount of image data from the end data An (minimum data) may be removed, and image data near the intermediate value of the plurality of image data and accounting for 60% -80% of the total amount of image data may remain. Wherein the number of image data removed from both the start data A1 and the end data An is the same.
For example, from the start data A1, and from the end data An, both of which are removed image data accounting for 10% of the total amount of image data, image data accounting for 80% of the total amount of image data may remain for average calculation. The method has the advantages that by removing the largest part of image data and the smallest part of image data, abrupt changes of the image data can be reduced, subsequent mean value calculation is interfered, and accuracy and stability of the finally formed reference height value can be improved.
Further, in this embodiment, the step of obtaining the height dimension value according to the average value of the gray values of the points in the point cloud middle area of the target to be measured specifically includes: selecting a central area of a point cloud pixel point of a preset target to be detected from the second two-dimensional gray level image; and selecting an average value of gray values of a plurality of point cloud pixel points in the central area of the point cloud pixel points to obtain a height value. Furthermore, according to the difference between the height value and the reference height value obtained by the previous technology, the height size of the target to be measured can be obtained.
Specifically, a midpoint of the target to be measured can be determined from the target to be measured corresponding to the second two-dimensional gray level image, then a circle is made by using the center of the midpoint and a set radius, then image data corresponding to a plurality of pixel points are selected from the circle, and then a height value is obtained according to the average value of gray levels of the pixel points. In this embodiment, the set radius may be set based on the point cloud data of the target to be measured. The set radius may be 3-5 times the pixel length in the second two-dimensional gray scale image. And, the area of the circle does not exceed the area corresponding to the cloud of the target point to be measured (i.e. does not exceed the white area described above).
Firstly, a circle can be drawn by taking the pixel length of 3 times as the radius, and the pixel points in the circle are judged, so as to judge whether the Gray value of each pixel point in the circle is 0 or Gray; if not, further judging whether the Gray value of each pixel point in the circle is 0 or Gray; if yes, removing each pixel point with the Gray value of 0 or Gray in the circle, and obtaining the height value of the target to be detected by adopting the average value of the Gray values of the rest other pixel points; if not, the Gray value of each point in the circle is neither 0 nor Gray, and the average value of the Gray values of all the pixel points in the circle can be used to obtain the height value of the object to be measured.
Further, if the Gray value of each pixel in the circle is 0 or Gray, the radius may be enlarged (for example, the radius is enlarged to 3 times the pixel length), so as to form a new circle, and it is determined whether the Gray value of each pixel in the new circle is 0 or Gray. If not, further judging whether the Gray value of each pixel point in the new circle is partially 0 or Gray; if so, removing each pixel point with the Gray value of 0 or Gray in the new circle, and obtaining the height value of the target to be detected by adopting the average value of the Gray values of the rest other pixel points; if not, the Gray value of each point in the new circle is neither 0 nor Gray, and the average value of the Gray values of all the pixel points in the new circle can be used to obtain the height value of the object to be measured.
Further, based on the same inventive concept, the present application further provides a height measuring device based on a visual imaging detection apparatus, and referring to fig. 3, fig. 3 is a schematic structural diagram of an embodiment of the height measuring device based on the visual imaging detection apparatus provided in the present application.
The height measuring device 30 includes a data receiving module 310, an image converting module 320, an image dividing module 330, and an image data processing module 340, which are coupled in sequence. The height measuring device 30 may be used to implement the height measuring method as described previously.
The data receiving module 310 is configured to receive the obtained point cloud data of the object to be measured, where the object to be measured has a plurality of areas to be measured, and each of the areas to be measured has a surface provided with a target to be measured.
The image conversion module 320 is configured to project the point cloud data into a first two-dimensional gray scale image, where a gray scale value of each point cloud pixel point in the first two-dimensional gray scale image is positively correlated with a height value of the point cloud corresponding to the point cloud data.
The image segmentation module 330 is configured to segment an image area of each of the areas to be detected on the object to be detected from the first two-dimensional gray scale image.
The image data processing module 340 is configured to obtain a height value and a reference height value of the target to be measured in each image area; and calculating the height dimension of each object to be measured according to the height value of each object to be measured and the reference height value.
Further, based on the same inventive concept, the application also provides a visual imaging detection device. Referring to fig. 4, fig. 4 is a schematic diagram of a frame of an embodiment of a visual imaging detection apparatus provided in the present application. The visual imaging detection apparatus 40 comprises a memory 410, a processor 420 and a visual imaging detection apparatus based height measurement program 411 stored on the memory 410 and executable on the processor 420, which when executed by the processor implements the steps of the height measurement method embodiment of any of the objects described above.
Referring to fig. 5, fig. 5 is a schematic diagram of a frame of an embodiment of a computer readable storage medium provided in the present application. The computer readable storage medium 50 stores program instructions 510 executable by the processor, the program instructions 510 may be a height measurement program based on a visual imaging detection apparatus for implementing the steps of the height measurement method embodiment of any of the objects described above.
In the several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.
The elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, or a part or all of the technical solution contributing to the prior art, may be embodied in the form of a software product stored in a computer-readable storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned computer-readable storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a random access Memory (RAM, randomAccess Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the claims, and all equivalent structural changes made in the present application and the accompanying drawings or direct/indirect application in other related technical fields are included in the scope of the present application.
Claims (10)
1. A height measurement method based on a visual imaging detection apparatus, characterized in that the measurement data compensation method comprises:
acquiring point cloud data of an object to be detected, wherein the object to be detected is provided with a plurality of areas to be detected, and the middle surface of each area to be detected is provided with a target to be detected;
projecting the point cloud data into a first two-dimensional gray level image, wherein gray level values of pixel points of each point cloud in the first two-dimensional gray level image are positively correlated with height values of point clouds corresponding to the point cloud data;
dividing an image area of each area to be detected on the object to be detected from the first two-dimensional gray level image;
acquiring a height value and a reference height value of the target to be detected in each image area;
and calculating the height dimension of each object to be measured according to the height value of each object to be measured and the reference height value.
2. The height measurement method based on a visual imaging detection apparatus according to claim 1, wherein the step of dividing the image area of each of the areas to be detected on the object to be detected from the first two-dimensional gray scale image comprises:
establishing a minimum surrounding area surrounding all the targets to be detected in the area to be detected for each area to be detected, wherein the minimum surrounding area is positioned in the area to be detected, and the minimum surrounding area is not connected with the edge of the area to be detected;
dividing a sub-point cloud corresponding to the minimum surrounding area from the point cloud data according to the coordinate range of the minimum surrounding area;
projecting the sub-point cloud to a corresponding second two-dimensional gray scale image, and setting the second two-dimensional gray scale image as an image area of the area to be detected.
3. The method for measuring height based on a visual imaging detection apparatus according to claim 2, wherein,
the establishing a minimum surrounding area surrounding the at least one target to be measured for each region to be measured comprises the following steps:
acquiring the minimum external annular shape of all the targets to be detected in the region to be detected;
expanding the minimum external ring by a set distance relative to the target to be detected so as to form a new external ring;
and determining the area surrounded by the new circumscribing ring as the minimum surrounding area.
4. A height measuring method based on a visual imaging detection apparatus according to claim 3,
calculating a reference height value from the second two-dimensional gray scale image, comprising:
dividing initial image data of a reference plane from the second two-dimensional gray level image by adopting an adaptive threshold segmentation method;
dividing missing image data of the target to be detected from the second two-dimensional gray level image according to a set threshold dividing method;
obtaining the image data of the reference plane according to the difference set of the initial image data of the reference plane and the missing image data of the target to be detected;
and obtaining the reference height value according to the average value of the point cloud gray scale in the image data of the reference plane.
5. A height measuring method based on a visual imaging detection apparatus according to claim 3,
calculating a reference height value of the target to be measured according to the second two-dimensional gray scale image, including:
and obtaining the reference height value according to the average value of gray values of a plurality of points in the point cloud middle area of the target to be detected.
6. The method for measuring height based on a visual imaging detection apparatus according to claim 5, wherein,
the step of obtaining the height value according to the average value of the gray values of the points in the point cloud middle area of the target to be detected comprises the following steps:
selecting an area of a point cloud pixel point of a preset target to be detected from the second two-dimensional gray level image;
and selecting an average value of gray values of a plurality of point cloud pixel points in the central area of the point cloud pixel points to obtain the height value.
7. The method for measuring height based on a visual imaging detection apparatus according to claim 6, wherein,
the step of obtaining the height dimension value by selecting an average value of gray values of a plurality of point cloud pixel points in a central area of the point cloud pixel points includes:
taking the central coordinate of the area of the point cloud pixel point as a circle center and taking a first set radius as a radius to make a circle;
and selecting an average value of gray values of a plurality of point cloud pixel points in the circle to obtain the height value.
8. The height measuring device based on the visual imaging detection equipment is characterized by comprising a data receiving module, an image conversion module, an image segmentation module and an image data processing module which are sequentially coupled;
the data receiving module is used for receiving the acquired point cloud data of the object to be detected, the object to be detected is provided with a plurality of areas to be detected, and the surface in each area to be detected is provided with a target to be detected;
the image conversion module is used for projecting the point cloud data into a first two-dimensional gray level image, wherein gray level values of point cloud pixel points in the first two-dimensional gray level image are positively correlated with height values of point cloud corresponding to the point cloud data;
the image segmentation module is used for segmenting an image area of each area to be detected on the object to be detected from the first two-dimensional gray level image;
the image data processing module is used for acquiring the height value and the reference height value of the target to be detected in each image area; and calculating the height dimension of each object to be measured according to the height value of each object to be measured and the reference height value.
9. A visual imaging detection apparatus, characterized in that it comprises a memory, a processor and a visual imaging detection apparatus-based height measurement program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the visual imaging detection apparatus-based height measurement method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a height measurement program based on a visual imaging detection apparatus, which when executed by a processor, implements the steps of the height measurement method based on a visual imaging detection apparatus according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211058029.6A CN117670784A (en) | 2022-08-31 | 2022-08-31 | Height measurement method based on visual imaging detection equipment and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211058029.6A CN117670784A (en) | 2022-08-31 | 2022-08-31 | Height measurement method based on visual imaging detection equipment and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117670784A true CN117670784A (en) | 2024-03-08 |
Family
ID=90068726
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211058029.6A Pending CN117670784A (en) | 2022-08-31 | 2022-08-31 | Height measurement method based on visual imaging detection equipment and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117670784A (en) |
-
2022
- 2022-08-31 CN CN202211058029.6A patent/CN117670784A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107272019B (en) | Road edge detection method based on laser radar scanning | |
| US10288418B2 (en) | Information processing apparatus, information processing method, and storage medium | |
| US8964189B2 (en) | Three-dimensional measurement apparatus, method for three-dimensional measurement, and computer program | |
| KR100422370B1 (en) | An Apparatus and Method to Measuring Dimensions of 3D Object on a Moving Conveyor | |
| EP2293244A1 (en) | System and method for providing 2-dimensional computerized-tomography image corresponding to 2-dimensional ultrasound image | |
| JP6115214B2 (en) | Pattern processing apparatus, pattern processing method, and pattern processing program | |
| US20180150969A1 (en) | Information processing device, measuring apparatus, system, calculating method, storage medium, and article manufacturing method | |
| CN111144213B (en) | Object detection method and related equipment | |
| US11841421B2 (en) | Synthetic aperture radar image analysis system, synthetic aperture radar image analysis method, and synthetic aperture radar image analysis program | |
| CN111768450A (en) | Automatic detection method and device for line deviation of structured light camera based on speckle pattern | |
| US20230267593A1 (en) | Workpiece measurement method, workpiece measurement system, and program | |
| CN111507919B (en) | Denoising processing method for three-dimensional point cloud data | |
| CN111680574A (en) | Face detection method and device, electronic equipment and storage medium | |
| CN113112490A (en) | Three-dimensional medical image mark point extraction method and system | |
| CN105283750A (en) | Method for processing a digital image of the surface of a tire in order to detect an anomaly | |
| CN112001962B (en) | Ship height measuring and calculating method and system | |
| CN114332014A (en) | Projector quality evaluation method, device, equipment and storage medium | |
| CN117670784A (en) | Height measurement method based on visual imaging detection equipment and related device | |
| CN110602486B (en) | Detection method, detection device, depth camera and computer readable storage medium | |
| CN117670976A (en) | Diameter measurement method based on visual imaging detection equipment and related device | |
| CN113723406A (en) | Processing method and device for positioning bracket of coronary angiography image | |
| CN114066992A (en) | Camera calibration method, system, device and storage medium | |
| CN109118489B (en) | Coronary artery position detection method and system | |
| CN113391298A (en) | Parameter calibration method and device for laser radar | |
| EP4078221A1 (en) | Time-of-flight imaging circuitry, time-of-flight imaging system, time-of-flight imaging method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |