CN117437149A - Image processing method, point cloud generating method, electronic device and storage medium - Google Patents
Image processing method, point cloud generating method, electronic device and storage medium Download PDFInfo
- Publication number
- CN117437149A CN117437149A CN202311570794.0A CN202311570794A CN117437149A CN 117437149 A CN117437149 A CN 117437149A CN 202311570794 A CN202311570794 A CN 202311570794A CN 117437149 A CN117437149 A CN 117437149A
- Authority
- CN
- China
- Prior art keywords
- pixel
- phase
- phase value
- image processing
- missing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000010586 diagram Methods 0.000 claims abstract description 43
- 238000010587 phase diagram Methods 0.000 claims abstract description 39
- 238000001914 filtration Methods 0.000 claims abstract description 26
- 230000010363 phase shift Effects 0.000 claims abstract description 21
- 230000008859 change Effects 0.000 claims abstract description 8
- 238000012217 deletion Methods 0.000 claims description 9
- 230000037430 deletion Effects 0.000 claims description 9
- 230000000877 morphologic effect Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 description 22
- 230000007812 deficiency Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000013507 mapping Methods 0.000 description 4
- 230000002829 reductive effect Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4023—Scaling of whole images or parts thereof, e.g. expanding or contracting based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The disclosure provides an image processing method, a point cloud generating method, electronic equipment and a storage medium. The image processing method of the present disclosure includes: generating a two-dimensional unfolded phase diagram based on a phase shift code diagram and a Gray code diagram obtained from the surface of the measured object, wherein pixel points in the two-dimensional unfolded phase diagram are distributed in two dimensions, and the phase values of the pixel points arranged along a first direction change in the first direction; dividing the phase values of the pixel points which are arranged along the first direction and have the same second direction coordinates into sections so as to obtain at least one phase value subsection with monotonically increasing phase values, wherein the first direction is perpendicular to the second direction; judging the pixel points corresponding to the phase values outside the phase value subinterval as pixel noise points; and removing the pixel noise points to obtain a two-dimensional unwrapped phase map after phase filtering.
Description
Technical Field
The disclosure relates to the technical field of image processing and point cloud generation, and in particular relates to an image processing method, a point cloud generation method, electronic equipment and a storage medium.
Background
In the three-dimensional reconstruction scheme based on the fringe projection (phase shift coding diagram and Gray code diagram), the resolving precision of the unfolding phase directly influences the reconstruction effect of the point cloud, so that the follow-up working procedures such as matching identification grabbing based on the point cloud are influenced.
Due to the influence of ambient light of a scene where a measured object (various workpieces and the like) is located and different complexity degrees of the surface morphology of the measured object, errors with different degrees exist in the phase calculation of the unfolded phase diagram, cloud noise is caused, and it is very important to correct or filter the phase errors.
Disclosure of Invention
The disclosure provides an image processing method, a point cloud generating method, electronic equipment and a storage medium.
According to an aspect of the present disclosure, there is provided an image processing method including: generating a two-dimensional unfolded phase diagram based on a phase shift code diagram and a Gray code diagram obtained from the surface of the measured object, wherein pixel points in the two-dimensional unfolded phase diagram are distributed in two dimensions, and the phase values of the pixel points arranged along a first direction change in the first direction;
dividing the phase values of pixel points which are arranged along a first direction and have the same second direction coordinates into intervals so as to obtain at least one phase value subinterval in which the phase values monotonically increase, wherein the first direction is perpendicular to the second direction; judging the pixel points corresponding to the phase values outside the phase value subinterval as pixel noise points; and removing the pixel noise points to obtain a two-dimensional unfolded phase diagram after phase filtering.
The image processing method according to at least one embodiment of the present disclosure further includes: judging whether the phase value number of the phase value subinterval is larger than or equal to a phase value number threshold value or not; if so, not judging the pixel point corresponding to the phase value in the phase value subinterval as a pixel noise point; if not, the pixel point corresponding to the phase value in the phase value subinterval is judged as the pixel noise point.
The image processing method according to at least one embodiment of the present disclosure further includes: and carrying out pixel missing judgment on the two-dimensional unwrapped phase map after phase filtering, and carrying out pixel compensation based on the neighborhood pixel point phase value on the missing pixels to obtain the two-dimensional unwrapped phase map after pixel compensation.
According to an image processing method of at least one embodiment of the present disclosure, performing pixel missing judgment on the two-dimensional unwrapped phase map after phase filtering, and performing pixel compensation based on a neighborhood pixel point phase value on a missing pixel, including: judging pixel missing along the first direction, and judging the domains with the number of missing pixel points being greater than or equal to the threshold number as pixel missing domains in the first direction; acquiring all first-direction pixel missing domains of the two-dimensional unwrapped phase map after phase filtering; and performing the pixel compensation on the first direction pixel missing domain based on the neighborhood pixel point phase value of the first direction pixel missing domain.
According to an image processing method of at least one embodiment of the present disclosure, performing the pixel compensation on a first direction pixel missing domain based on a neighborhood pixel point phase value of the first direction pixel missing domain includes: and sequentially carrying out pixel compensation based on the neighborhood pixel point phase values on the pixel missing domains in the first direction along the first direction.
According to an image processing method of at least one embodiment of the present disclosure, pixel compensation based on a neighborhood pixel point phase value is sequentially performed on a first direction pixel missing domain along the first direction, including: and performing interpolation compensation for ensuring that the phase value monotonically increases along the first direction on the first-direction pixel missing domain based on the phase value of the pixel point, in which the phase value difference in the first-direction neighborhood of the first-direction pixel missing domain is in a preset difference range.
According to an image processing method of at least one embodiment of the present disclosure, the performing the pixel compensation on the first direction pixel missing domain based on the neighborhood pixel point phase value of the first direction pixel missing domain further includes: and when the phase value difference in the first direction adjacent region of the first direction pixel missing domain is out of a preset difference range, performing interpolation compensation for ensuring that the phase value monotonically increases along the first direction on the basis of the phase value of the pixel point of which the phase value difference in the second direction adjacent region of the first direction pixel missing domain is in the preset difference range.
The image processing method according to at least one embodiment of the present disclosure further includes: and performing secondary noise point removal based on image morphology on the two-dimensional expansion phase diagram after pixel compensation.
According to an image processing method of at least one embodiment of the present disclosure, performing secondary noise point removal based on an image morphology on the pixel-compensated two-dimensional unfolded-phase map, includes: generating a matching mask image for the two-dimensional unfolded phase image after pixel compensation; mapping the position with the phase value in the two-dimensional unfolded phase map after pixel compensation to the corresponding position of the matched mask map and marking the position as 1, and mapping the position without the phase value to the corresponding position of the matched mask map and marking the position as 0 so as to obtain a matched binary phase map; and carrying out morphological open operation on the matched binary phase diagram so as to compare the matched binary phase diagram with the matched mask diagram, positioning pixel noise points in the two-dimensional unfolded phase diagram and removing the pixel noise points.
According to another aspect of the present disclosure, there is provided a point cloud generating method, including: acquiring a phase shift coding diagram and a Gray code diagram of the surface of a measured object; acquiring point cloud data representing the surface morphological characteristics of the measured object based on the two-dimensional unfolded phase diagram; the two-dimensional expansion phase map is generated and processed by the image processing method according to any embodiment of the disclosure.
According to still another aspect of the present disclosure, there is provided an image processing apparatus including: the two-dimensional unfolding phase map generating module generates a two-dimensional unfolding phase map based on a phase shift coding map and a Gray code map obtained from the surface of a measured object, wherein pixel points in the two-dimensional unfolding phase map are distributed in two dimensions, and the phase values of the pixel points arranged along a first direction change in the first direction; the phase value subinterval generation module is used for dividing the phase values of the pixel points which are arranged along a first direction and have the same second direction coordinates into intervals so as to obtain at least one phase value subinterval with monotonically increasing phase values, and the first direction is perpendicular to the second direction; the noise point judging module judges the pixel points corresponding to the phase values outside the phase value subinterval as pixel noise points; and the phase filtering module removes the pixel noise points to obtain a two-dimensional unwrapped phase map after phase filtering.
An image processing apparatus according to at least one embodiment of the present disclosure further includes: and the pixel missing judging and compensating module is used for judging the pixel missing of the two-dimensional unfolded phase diagram after the phase filtering and compensating the pixel missing based on the phase value of the neighborhood pixel point to obtain the two-dimensional unfolded phase diagram after the pixel compensation.
An image processing apparatus according to at least one embodiment of the present disclosure further includes: and the secondary filtering module is used for removing secondary noise points based on image morphology from the two-dimensional expansion phase diagram after pixel compensation.
According to still another aspect of the present disclosure, there is provided a point cloud generating apparatus including: the image acquisition module acquires a phase shift coding diagram and a Gray code diagram of the surface of the object to be measured; the point cloud data generation module acquires point cloud data representing the surface appearance characteristics of the measured object based on the two-dimensional unfolding phase diagram; wherein the two-dimensional expansion phase map is a two-dimensional expansion phase map generated and processed by the image processing apparatus according to any one of the embodiments of the present disclosure.
According to still another aspect of the present disclosure, there is provided an electronic apparatus including: a memory storing execution instructions; and a processor executing the execution instructions stored by the memory, causing the processor to perform the image processing method of any one of the embodiments of the present disclosure and/or to perform the point cloud generation method of any one of the embodiments of the present disclosure.
According to still another aspect of the present disclosure, there is provided a readable storage medium having stored therein execution instructions which, when executed by a processor, are to implement the image processing method of any one embodiment of the present disclosure and/or to perform the point cloud generating method of any one embodiment of the present disclosure.
According to the image processing method of some embodiments of the present disclosure, by filtering the unfolded phase diagram, phase noise is reduced, and integrity of phase data is retained to the greatest extent, so that point cloud noise is reduced, and signal to noise ratio of the point cloud data can be improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.
Fig. 1 is a flow diagram of an image processing method in some embodiments of the present disclosure.
Fig. 2 is an ideal two-dimensional unwrapped phase diagram.
Fig. 3 is phase value data of a certain row (first direction) in an ideal two-dimensional unwrapped phase map.
Fig. 4 is a line (first direction) of phase value data in a two-dimensional unfolded phase map of the surface of an object to be measured obtained from an exemplary actual scene.
Fig. 5 shows a schematic diagram of a certain row of phase values divided into monotonically increasing phase value subintervals according to one embodiment of the present disclosure.
Fig. 6 shows a flow diagram of an image processing method of other embodiments of the present disclosure.
Fig. 7 is a partially enlarged schematic view of a two-dimensional unwrapped phase map obtained by pixel noise point determination and pixel noise point removal after phase filtering.
Fig. 8 is a flow chart of a method for determining and compensating for phase value loss according to some embodiments of the present disclosure.
Fig. 9 shows a flow diagram of an image processing method of other embodiments of the present disclosure.
Fig. 10 shows a schematic flow chart of secondary noise point removal.
Fig. 11 is a schematic diagram of the position of the calibration noise in the mask map (matching binary phase map) according to one embodiment of the present disclosure.
Fig. 12 is a mask diagram (matching binary phase diagram) after morphological open operation processing according to an embodiment of the present disclosure.
Fig. 13 is a schematic block diagram of an image processing apparatus employing a hardware implementation of a processor according to an embodiment of the present disclosure.
Fig. 14 is a schematic block diagram of a point cloud generating apparatus employing a hardware implementation of a processor according to an embodiment of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the drawings and the embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant content and not limiting of the present disclosure. It should be further noted that, for convenience of description, only a portion relevant to the present disclosure is shown in the drawings.
In addition, embodiments of the present disclosure and features of the embodiments may be combined with each other without conflict. The technical aspects of the present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Unless otherwise indicated, the exemplary implementations/embodiments shown are to be understood as providing exemplary features of various details of some ways in which the technical concepts of the present disclosure may be practiced. Thus, unless otherwise indicated, features of the various implementations/embodiments may be additionally combined, separated, interchanged, and/or rearranged without departing from the technical concepts of the present disclosure.
The use of cross-hatching and/or shading in the drawings is typically used to clarify the boundaries between adjacent components. As such, the presence or absence of cross-hatching or shading does not convey or represent any preference or requirement for a particular material, material property, dimension, proportion, commonality between illustrated components, and/or any other characteristic, attribute, property, etc. of a component, unless indicated. In addition, in the drawings, the size and relative sizes of elements may be exaggerated for clarity and/or descriptive purposes. While the exemplary embodiments may be variously implemented, the specific process sequences may be performed in a different order than that described. For example, two consecutively described processes may be performed substantially simultaneously or in reverse order from that described. Moreover, like reference numerals designate like parts.
When an element is referred to as being "on" or "over", "connected to" or "coupled to" another element, it can be directly on, connected or coupled to the other element or intervening elements may be present. However, when an element is referred to as being "directly on," "directly connected to," or "directly coupled to" another element, there are no intervening elements present. For this reason, the term "connected" may refer to physical connections, electrical connections, and the like, with or without intermediate components.
The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, when the terms "comprises" and/or "comprising," and variations thereof, are used in the present specification, the presence of stated features, integers, steps, operations, elements, components, and/or groups thereof is described, but the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof is not precluded. It is also noted that, as used herein, the terms "substantially," "about," and other similar terms are used as approximation terms and not as degree terms, and as such, are used to explain the inherent deviations of measured, calculated, and/or provided values that would be recognized by one of ordinary skill in the art.
The image processing method, the point cloud generating method, the image processing apparatus, the point cloud generating apparatus, and the like of the present disclosure may be configured on a terminal device in the form of computer software. The terminal device may be a hardware device such as a computer device having various operating systems.
The image processing method, the point cloud generating method, the image processing apparatus, the point cloud generating apparatus, and the like of the present disclosure are described in detail below with reference to fig. 1 to 14.
Fig. 1 is a flow diagram of an image processing method in some embodiments of the present disclosure.
Referring to fig. 1, in some embodiments of the present disclosure, an image processing method S100 of the present disclosure includes:
s102, generating a two-dimensional unfolded phase diagram based on a phase shift code diagram and a Gray code diagram obtained from the surface of the measured object, wherein pixel points in the two-dimensional unfolded phase diagram are distributed in two dimensions, and the phase values of the pixel points arranged along the first direction change in the first direction.
S104, dividing the phase values of the pixel points which are arranged along the first direction (the column increasing direction) and have the same second direction coordinates into intervals so as to obtain at least one phase value subinterval with monotonically increasing phase values, wherein the first direction is perpendicular to the second direction.
S106, judging the pixel points corresponding to the phase values outside the phase value subinterval as pixel noise points.
S108, removing pixel noise points to obtain a two-dimensional unfolded phase diagram after phase filtering.
The image processing method is a filtering method based on the monotonically increasing characteristic of the phase, and the error phase value is reduced at the phase level, so that the difficulty of subsequent stereo matching is reduced, and the point cloud with higher quality is obtained.
Fig. 2 is an ideal two-dimensional unwrapped-phase diagram, and fig. 3 is phase value data of a certain row (first direction) in the ideal two-dimensional unwrapped-phase diagram.
Referring to fig. 2 and 3, it can be seen that in the unwrapped phase map, the phase values corresponding thereto are also monotonically increasing along the column increasing direction (first direction).
In an actual scene, the phase values of a plurality of pixel points (i.e., phase value points) in the obtained two-dimensional unfolded phase diagram are influenced by the environment light, the type of the measured object, the surface morphology and the like, and the phase values do not have the monotonically increasing characteristic, but have the characteristic of random and random arrangement, and the points are all noise.
Fig. 4 is a diagram of phase value data of a certain line (first direction) in a two-dimensional unfolded phase map of the surface of the measured object obtained from an exemplary actual scene, and the position where the square frame is in fig. 4 is the wrong phase value.
In order to filter out the erroneous phase values (i.e., pixel noise points) in the two-dimensional unfolded phase map, the image processing method of the present disclosure first obtains the two-dimensional unfolded phase map (step S102).
Both the phase shift Code pattern and the Gray Code pattern (i.e., binary Gray Code pattern) are images used in structured light imaging.
The phase shift coding diagram refers to changing the phase of a projection grating pattern when the structured light projection is carried out, obtaining different projection patterns, and obtaining the three-dimensional shape information of the object surface by collecting and processing the patterns.
In structured light imaging, a plurality of grating patterns with different phases are used, and projected one by one onto the surface of an object to be measured, and reflected light signals are photographed by using a camera. In each image, the distance between the object surface and the projection grating can cause the phase difference of the reflected light to change, and the obtained image can show different brightness distribution. By converting the phase difference between these images into a distance difference using a phase shift algorithm, three-dimensional shape information of the object surface can be obtained.
The unfolded phase diagram method is a method for acquiring three-dimensional shape information of the object surface from the phase shift code diagram and the gray code diagram. The phase shift code image and the Gray code image respectively provide phase difference and code information of the object surface, and the precise distance between each pixel point and the projection light source can be obtained by combining the information of the phase shift code image and the Gray code image.
The basic idea of the unfolded phase diagram method is to pair a phase shift code diagram and a Gray code diagram, and calculate the distance between each pixel point and a projection light source according to the code information in the Gray code diagram and the phase difference information in the phase shift code diagram.
In the image processing method of the present disclosure, the method for acquiring the unfolded phase map may be obtained by calculating a plurality of phase shift code maps and gray code maps acquired from the surface of the object to be measured by using an existing method.
In the image processing method of the present disclosure, the pixel points (i.e., the phase value points) in the generated two-dimensional unfolded phase map are distributed in two dimensions, and referring to fig. 2, the phase values of the pixel points arranged along the first direction change in the first direction.
Referring to fig. 2 and 3, the correct unfolded phase diagram has one characteristic: along the column increasing direction (first direction in the drawing), the corresponding phase value is also monotonically increasing. Based on this premise, the present disclosure next performs the following processing for each line in the two-dimensional expanded phase map (i.e., each line extending in the first direction):
all phase value intervals (i.e., phase value subintervals) satisfying the monotonically increasing phase value in one row are found, that is, the phase values of the pixel points arranged along the first direction (column increasing direction) and having the same second direction coordinates are divided into intervals, so as to obtain at least one phase value subinterval in which the phase value monotonically increases. In the present disclosure, the first direction is perpendicular to the second direction.
Fig. 5 shows a schematic diagram of a certain row of phase values (consistent with the phase value data in fig. 4) divided into monotonically increasing phase value subintervals, with 5 monotonically increasing phase value subintervals being exemplarily shown in fig. 5. When there is only one pixel point (phase value point) in the subinterval, the start point and the end point of the subinterval coincide, referring to fig. 5.
In some embodiments of the present disclosure, further, pixel points corresponding to phase values outside the phase value subinterval are determined as pixel noise points, and removing the pixel noise points obtains a two-dimensional unwrapped phase map after phase filtering, thereby removing erroneous phase values at the phase level.
In other embodiments of the present disclosure, based on the above embodiments, step S106 of the present disclosure (i.e., the pixel noise point determining step) further includes:
judging whether the number of the phase values in the phase value subinterval is larger than or equal to a threshold value of the number of the phase values, wherein the threshold value is preset, such as 2, 3 and the like; if so, not judging the pixel point corresponding to the phase value in the phase value subinterval as a pixel noise point; if not, the pixel point corresponding to the phase value in the phase value subinterval is judged as the pixel noise point.
That is, on the basis of the above embodiment, the present disclosure further determines the number of phase value points in each monotonically increasing phase value subinterval, and when the number is greater than (or equal to) a certain threshold value, the subinterval is reserved, otherwise, the subinterval is considered as noise, and the phase value corresponding to the subinterval can be deleted, i.e., deleted.
For example, if a subinterval of which the number is equal to or smaller than a threshold value of the number of phase value points is set to 2, two discrete subintervals (subintervals whose start points overlap) are further deleted in fig. 5.
Through the steps, the image processing method disclosed by the invention can further reduce the wrong phase value at the phase level.
Fig. 6 shows a flow diagram of an image processing method of other embodiments of the present disclosure. Referring to fig. 6, on the basis of the steps of the image processing method shown in fig. 1, the image processing method S100 of the present disclosure further includes:
s110, performing pixel missing judgment on the two-dimensional unwrapped phase map after phase filtering, and performing pixel compensation on missing pixels based on the neighborhood pixel point phase values to obtain the two-dimensional unwrapped phase map after pixel compensation.
Based on the pixel noise point determination and pixel noise point removal process described above, a stripe-like deletion of the phase value point in the second direction may be caused. For example, due to the difference in the severity of the threshold settings of the number of phase value points in the interval described above (the threshold is greatly different), a stripe defect of the phase value points along the second direction may occur, fig. 7 is a partial enlarged schematic diagram of a two-dimensional unwrapped phase map obtained by performing pixel noise point judgment and pixel noise point removal after phase filtering, showing the stripe defect of the phase value points, and it can be seen that some black small points or small stripes exist in fig. 7, and these positions are all phase value defect points.
In order to obtain a more accurate unfolded phase map, the present disclosure proposes a method for determining and compensating for phase value deficiency (pixel deficiency) based on the above-described image processing method, and fig. 8 is a flow chart illustrating a method for determining and compensating for phase value deficiency (pixel deficiency) according to some embodiments of the present disclosure.
Referring to fig. 8, a method of determining and compensating for a phase value deficiency (pixel deficiency) of the present disclosure includes:
s1102, performing pixel missing determination along the first direction (the column increment direction), and determining a domain with a number of missing pixels (i.e., missing phase value points) greater than or equal to a threshold number (e.g., 3) as a first direction pixel missing domain.
S1104, obtaining all first direction pixel missing domains of the two-dimensional unwrapped phase map after phase filtering.
S1106, performing pixel compensation on the first-direction pixel missing domain based on the neighborhood pixel point phase value of the first-direction pixel missing domain.
Preferably, the present disclosure performs pixel compensation based on a neighborhood pixel point phase value for a missing pixel based on the following method, including:
and sequentially carrying out pixel compensation based on the neighborhood pixel point phase values on the pixel missing domains in the first direction along the first direction.
In some embodiments of the present disclosure, sequentially performing pixel compensation based on a neighborhood pixel point phase value for a first direction pixel deletion domain along the first direction includes:
interpolation compensation is performed on the first-direction pixel missing domain, wherein the interpolation compensation is performed on the first-direction pixel missing domain, and the interpolation compensation is performed on the first-direction pixel missing domain, based on the phase values of pixel points, wherein the phase value differences (such as difference values) in the first-direction neighborhood of the first-direction pixel missing domain are in a preset difference range (such as a preset difference range).
In other words, the present disclosure first judges a first direction neighborhood of a missing pixel domain (point), and when the phase values of the pixel points in the neighborhood are similar (i.e., the phase value difference is within a preset difference/threshold range), performs interpolation compensation for ensuring that the phase value monotonically increases along the first direction for the missing position by using the phase value of the neighborhood.
In some embodiments of the present disclosure, the pixel compensation for the first direction pixel deletion domain based on the neighborhood pixel point phase values of the first direction pixel deletion domain further includes:
when the phase value difference (e.g., difference value) in the first direction adjacent region of the first direction pixel missing domain is out of a preset difference range (e.g., preset difference range/threshold value), interpolation compensation is performed on the first direction pixel missing domain, wherein the interpolation compensation is performed on the first direction pixel missing domain, and the phase value of the interpolation compensation is guaranteed to monotonically increase along the first direction, based on the phase value of the pixel point, in which the phase value difference in the second direction adjacent region of the first direction pixel missing domain is in the preset difference range, otherwise, the interpolation compensation is not performed.
After the above-described process of judging and compensating for the phase value deficiency, the phase noise of the unwrapped phase map has been filtered to some extent, and the repair of the unwrapped phase map to some extent has been completed.
The unfolded phase map may have some burrs at the boundary of the measured object or some small holes in the scene, and in the preferred embodiment of the present disclosure, these problems are further processed based on image morphology, that is, on the basis of the above steps, the steps are performed:
and step S112, performing secondary noise point removal based on the image morphology on the two-dimensional expansion phase diagram after pixel compensation.
Fig. 9 shows a flow diagram of an image processing method of other embodiments of the present disclosure including step S112. Fig. 10 shows a schematic flow chart of secondary noise point removal.
In a preferred embodiment of the present disclosure, in the image processing method of the present disclosure, S112, performing secondary noise point removal based on an image morphology on the two-dimensional unwrapped phase map after pixel compensation, includes:
s1122, a matching mask map (equal-sized mask map) is generated for the pixel-compensated two-dimensional expansion phase map.
S1124, mapping the position with the phase value in the two-dimensional unfolded phase map after pixel compensation to the corresponding position of the matched mask map and marking the position as 1, and mapping the position without the phase value to the corresponding position of the matched mask map and marking the position as 0, so as to obtain a matched binary phase map.
S1124, performing morphological open operation on the matched binary phase diagram to compare with the matched mask diagram, positioning out pixel noise points in the two-dimensional unfolded phase diagram and removing the pixel noise points.
Through the secondary noise point removal process described above, isolated point blocks in the unfolded phase diagram can be further removed, and the boundary of the object to be measured is smoothed without changing its shape and area.
In the present disclosure, the morphological open operation is to erode the image and then expand it, which can be used to locate some fine locations in the phase map, which is usually noise, as indicated by the rectangular box circled locations in fig. 11. In addition, the above method of the present disclosure can better maintain the boundary while removing noise, as shown in fig. 12. According to the method and the device, the two-dimensional expansion phase diagram is further processed according to the result of the open operation, and the phase points filtered by the open operation can be directly deleted.
On the basis of the image processing method, the present disclosure also provides a point cloud generating method, which includes: acquiring a phase shift coding diagram and a Gray code diagram of the surface of a measured object; acquiring point cloud data representing the surface morphological characteristics of the measured object based on the two-dimensional unfolded phase diagram; the two-dimensional expansion phase map is a two-dimensional expansion phase map generated and processed by the image processing method S100 according to any one of the embodiments of the present disclosure.
The present disclosure also provides an image processing apparatus.
Fig. 13 is a schematic block diagram of an image processing apparatus employing a hardware implementation of a processor according to an embodiment of the present disclosure.
Referring to fig. 13, in some embodiments of the present disclosure, an image processing apparatus 1000 of the present disclosure includes:
the two-dimensional unfolded phase map generating module 1002, the two-dimensional unfolded phase map generating module 1002 generates a two-dimensional unfolded phase map based on the phase shift code map and the gray code map obtained from the surface of the object to be measured, the pixels in the two-dimensional unfolded phase map are distributed in two dimensions, and the phase values of the pixels arranged along the first direction change in the first direction.
The phase value subinterval generating module 1004 divides the phase values of the pixel points arranged along the first direction and having the same coordinates in the second direction to obtain at least one phase value subinterval in which the phase values monotonically increase, wherein the first direction is perpendicular to the second direction.
The noise determination module 1006 determines the pixel point corresponding to the phase value outside the phase value subinterval as the pixel noise point.
Phase filtering module 1008, phase filtering module 1008 removes pixel noise points to obtain a phase filtered two-dimensional unwrapped phase map.
In some embodiments of the present disclosure, the image processing apparatus 1000 of the present disclosure further includes:
the pixel missing judging and compensating module 1010 judges the pixel missing of the two-dimensional unfolded phase diagram after phase filtering, and performs pixel compensation based on the phase value of the neighborhood pixel point on the missing pixels to obtain the two-dimensional unfolded phase diagram after pixel compensation.
In some embodiments of the present disclosure, the image processing apparatus 1000 of the present disclosure further includes:
the secondary filtering module 1012, the secondary filtering module 1012 performs secondary noise point removal based on the image morphology on the two-dimensional unfolded phase map after pixel compensation.
The present disclosure also provides a point cloud generating apparatus 2000, including:
the image acquisition module 2002, the image acquisition module 2002 acquires a phase shift code image and a Gray code image of the surface of the measured object.
The point cloud data generating module 2004, the point cloud data generating module 2004 obtains point cloud data characterizing the surface topography of the object under test based on the two-dimensional unfolded phase map.
The two-dimensional expansion phase map is a two-dimensional expansion phase map generated and processed by the image processing apparatus 1000 according to any of the embodiments of the present disclosure.
Fig. 14 is a schematic block diagram of a point cloud generating apparatus employing a hardware implementation of a processor according to an embodiment of the present disclosure.
The image processing apparatus of the present disclosure, the image processing apparatus may include respective modules that perform each or several steps in the flowcharts described above. Thus, each step or several steps in the flowcharts described above may be performed by respective modules, and the apparatus may include one or more of these modules. A module may be one or more hardware modules specifically configured to perform the respective steps, or be implemented by a processor configured to perform the respective steps, or be stored within a computer-readable medium for implementation by a processor, or be implemented by some combination.
The image processing apparatus, the hardware structure of the image processing apparatus of the present disclosure may be realized using a bus architecture. The bus architecture may include any number of interconnecting buses and bridges depending on the specific application of the hardware and the overall design constraints. Bus 1100 connects together various circuits including one or more processors 1200, memory 1300, and/or hardware modules. Bus 1100 may also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, and the like.
Bus 1100 may be an industry standard architecture (ISA, industry Standard Architecture) bus, a peripheral component interconnect (PCI, peripheral Component) bus, or an extended industry standard architecture (EISA, extended Industry Standard Component) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one connection line is shown in the figure, but not only one bus or one type of bus.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present disclosure in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present disclosure. The processor performs the various methods and processes described above. For example, method embodiments in the present disclosure may be implemented as a software program tangibly embodied on a machine-readable medium, such as a memory. In some embodiments, part or all of the software program may be loaded and/or installed via memory and/or a communication interface. One or more of the steps of the methods described above may be performed when a software program is loaded into memory and executed by a processor. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above in any other suitable manner (e.g., by means of firmware).
Logic and/or steps represented in the flowcharts or otherwise described herein may be embodied in any readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
For the purposes of this description, a "readable storage medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner if necessary, and then stored in a memory.
It should be understood that portions of the present disclosure may be implemented in hardware, software, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or part of the steps implementing the method of the above embodiments may be implemented by a program to instruct related hardware, and the program may be stored in a readable storage medium, where the program when executed includes one or a combination of the steps of the method embodiments.
Furthermore, each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product. The storage medium may be a read-only memory, a magnetic disk or optical disk, etc.
The present disclosure also provides an electronic device, including: a memory storing execution instructions; and a processor or other hardware module that executes the execution instructions stored in the memory, so that the processor or other hardware module executes the image processing method and/or the point cloud generating method described above.
The disclosure also provides a readable storage medium having stored therein execution instructions which, when executed by a processor, are configured to implement the image processing method and/or the point cloud generating method described above.
It will be appreciated by those skilled in the art that the above-described embodiments are merely for clarity of illustration of the disclosure, and are not intended to limit the scope of the disclosure. Other variations or modifications will be apparent to persons skilled in the art from the foregoing disclosure, and such variations or modifications are intended to be within the scope of the present disclosure.
Claims (10)
1. An image processing method, comprising:
generating a two-dimensional unfolded phase diagram based on a phase shift code diagram and a Gray code diagram obtained from the surface of the measured object, wherein pixel points in the two-dimensional unfolded phase diagram are distributed in two dimensions, and the phase values of the pixel points arranged along a first direction change in the first direction;
dividing the phase values of pixel points which are arranged along a first direction and have the same second direction coordinates into intervals so as to obtain at least one phase value subinterval in which the phase values monotonically increase, wherein the first direction is perpendicular to the second direction;
judging the pixel points corresponding to the phase values outside the phase value subinterval as pixel noise points; and
and removing the pixel noise points to obtain a two-dimensional unfolded phase diagram after phase filtering.
2. The image processing method according to claim 1, characterized by further comprising:
judging whether the phase value number of the phase value subinterval is larger than or equal to a phase value number threshold value or not;
if so, not judging the pixel point corresponding to the phase value in the phase value subinterval as a pixel noise point;
if not, the pixel point corresponding to the phase value in the phase value subinterval is judged as the pixel noise point.
3. The image processing method according to claim 1 or 2, characterized by further comprising:
and carrying out pixel missing judgment on the two-dimensional unwrapped phase map after phase filtering, and carrying out pixel compensation based on the neighborhood pixel point phase value on the missing pixels to obtain the two-dimensional unwrapped phase map after pixel compensation.
4. The image processing method according to claim 3, wherein performing pixel missing judgment on the two-dimensional unwrapped phase map after phase filtering, performing pixel compensation based on a neighborhood pixel point phase value on missing pixels, comprises:
judging pixel missing along the first direction, and judging the domains with the number of missing pixel points being greater than or equal to the threshold number as pixel missing domains in the first direction;
acquiring all first-direction pixel missing domains of the two-dimensional unwrapped phase map after phase filtering; and
and carrying out pixel compensation on the first-direction pixel missing domain based on the neighborhood pixel point phase value of the first-direction pixel missing domain.
5. The image processing method according to claim 4, wherein performing the pixel compensation for the first-direction pixel deletion domain based on a neighborhood pixel point phase value of the first-direction pixel deletion domain includes:
and sequentially carrying out pixel compensation based on the neighborhood pixel point phase values on the pixel missing domains in the first direction along the first direction.
6. The image processing method according to claim 5, wherein pixel compensation based on a neighborhood pixel point phase value is performed sequentially on a first-direction pixel deletion domain along the first direction, comprising:
and performing interpolation compensation for ensuring that the phase value monotonically increases along the first direction on the first-direction pixel missing domain based on the phase value of the pixel point, in which the phase value difference in the first-direction neighborhood of the first-direction pixel missing domain is in a preset difference range.
7. The image processing method according to any one of claims 4 to 6, wherein the pixel compensation for the first-direction pixel deletion domain based on a neighborhood pixel point phase value of the first-direction pixel deletion domain further includes:
and when the phase value difference in the first direction adjacent region of the first direction pixel missing domain is out of a preset difference range, performing interpolation compensation for ensuring that the phase value monotonically increases along the first direction on the basis of the phase value of the pixel point of which the phase value difference in the second direction adjacent region of the first direction pixel missing domain is in the preset difference range.
8. A method of generating a point cloud, comprising:
acquiring a phase shift coding diagram and a Gray code diagram of the surface of a measured object;
acquiring point cloud data representing the surface morphological characteristics of the measured object based on the two-dimensional unfolded phase diagram;
wherein the two-dimensional expansion phase map is a two-dimensional expansion phase map generated and processed based on the image processing method according to any one of claims 1 to 7.
9. An electronic device, comprising:
a memory storing execution instructions; and
a processor executing the execution instructions stored by the memory, causing the processor to perform the image processing method of any one of claims 1 to 7 and/or to perform the point cloud generation method of claim 8.
10. A readable storage medium, characterized in that the readable storage medium has stored therein execution instructions, which when executed by a processor are for implementing the image processing method of any one of claims 1 to 7 and/or for executing the point cloud generating method of claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311570794.0A CN117437149B (en) | 2023-11-23 | Image processing method, point cloud generating method, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311570794.0A CN117437149B (en) | 2023-11-23 | Image processing method, point cloud generating method, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117437149A true CN117437149A (en) | 2024-01-23 |
| CN117437149B CN117437149B (en) | 2024-07-09 |
Family
ID=
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109040885A (en) * | 2017-06-09 | 2018-12-18 | 中国科学院声学研究所 | A kind of personalized loudness compensation method and system for headphone equalization based on volume |
| CN116109772A (en) * | 2023-02-06 | 2023-05-12 | 北京理工大学 | Point cloud reconstruction method, device, equipment and readable storage medium |
| CN116152119A (en) * | 2023-04-19 | 2023-05-23 | 北京现龙科技有限公司 | Phase denoising method, device, equipment and medium for stripe structure light |
| CN116188329A (en) * | 2023-04-27 | 2023-05-30 | 北京迁移科技有限公司 | Three-dimensional point cloud filtering method, reconstruction method, electronic device and readable storage medium |
| CN116385657A (en) * | 2023-04-17 | 2023-07-04 | 北京迁移科技有限公司 | Image processing method, point cloud generating method, electronic device and readable storage medium |
| CN116793247A (en) * | 2023-03-27 | 2023-09-22 | 西安交通大学 | Stripe projection profilometry stripe series correction method based on region statistics |
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109040885A (en) * | 2017-06-09 | 2018-12-18 | 中国科学院声学研究所 | A kind of personalized loudness compensation method and system for headphone equalization based on volume |
| CN116109772A (en) * | 2023-02-06 | 2023-05-12 | 北京理工大学 | Point cloud reconstruction method, device, equipment and readable storage medium |
| CN116793247A (en) * | 2023-03-27 | 2023-09-22 | 西安交通大学 | Stripe projection profilometry stripe series correction method based on region statistics |
| CN116385657A (en) * | 2023-04-17 | 2023-07-04 | 北京迁移科技有限公司 | Image processing method, point cloud generating method, electronic device and readable storage medium |
| CN116152119A (en) * | 2023-04-19 | 2023-05-23 | 北京现龙科技有限公司 | Phase denoising method, device, equipment and medium for stripe structure light |
| CN116188329A (en) * | 2023-04-27 | 2023-05-30 | 北京迁移科技有限公司 | Three-dimensional point cloud filtering method, reconstruction method, electronic device and readable storage medium |
| CN116721036A (en) * | 2023-04-27 | 2023-09-08 | 北京迁移科技有限公司 | Noise point judging method, device, electronic equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 施宇清: "莫尔相移测偏技术中的计算机图像处理", 振动、测试与诊断, no. 03, 30 October 2002 (2002-10-30) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6040911A (en) | Reference image forming method and pattern inspection apparatus | |
| CN111047615B (en) | Image-based straight line detection method and device and electronic equipment | |
| CN110211185B (en) | Method for identifying characteristic points of calibration pattern in group of candidate points | |
| CN109427046B (en) | Distortion correction method and device for three-dimensional measurement and computer readable storage medium | |
| JP2021520008A (en) | Vehicle inspection system and its method | |
| US20090226111A1 (en) | Image processing apparatus and method thereof | |
| CN112927306B (en) | Calibration method and device of shooting device and terminal equipment | |
| CN110866869B (en) | Image mosaic method and device combining Thiessen polygon and minimum spanning tree image segmentation | |
| CN110660072A (en) | Method and device for identifying straight line edge, storage medium and electronic equipment | |
| CN112308934B (en) | Calibration detection method and device, storage medium and computing equipment | |
| CN110197096B (en) | Two-dimensional code correction method, device and equipment | |
| CN117437149B (en) | Image processing method, point cloud generating method, electronic device and storage medium | |
| CN117058411B (en) | Method, device, medium and equipment for identifying edge appearance flaws of battery | |
| CN114187363A (en) | Method and device for obtaining radial distortion parameter value and mobile terminal | |
| CN117218148B (en) | Channel boundary determining method and system based on satellite image and readable storage medium | |
| CN117437149A (en) | Image processing method, point cloud generating method, electronic device and storage medium | |
| CN112784633B (en) | Lane boundary processing method and device, electronic equipment and storage medium | |
| CN111257346B (en) | PCB positioning device and method based on projection filtering | |
| CN116228535A (en) | Image processing method and device, electronic equipment and vehicle | |
| CN113781508B (en) | She Xingkong image contour line extraction and segmentation method, system, equipment and storage medium | |
| CN111986245A (en) | Depth information evaluation method and device, electronic equipment and storage medium | |
| CN116012322A (en) | Camera dirt detection method, device, equipment and medium | |
| CN115185078A (en) | Incoherent aperture synthetic aberration correction method and device | |
| JP2961140B2 (en) | Image processing method | |
| CN116188329B (en) | Three-dimensional point cloud filtering method, reconstruction method, electronic device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |