CN110800020B - Image information acquisition method, image processing equipment and computer storage medium - Google Patents
Image information acquisition method, image processing equipment and computer storage medium Download PDFInfo
- Publication number
- CN110800020B CN110800020B CN201780092646.9A CN201780092646A CN110800020B CN 110800020 B CN110800020 B CN 110800020B CN 201780092646 A CN201780092646 A CN 201780092646A CN 110800020 B CN110800020 B CN 110800020B
- Authority
- CN
- China
- Prior art keywords
- point
- image
- depth
- detected
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
Abstract
The embodiment of the invention discloses an image information acquisition method, an image processing device and a computer storage medium, which are used for reducing unnecessary hardware expenditure during multi-view stereoscopic vision detection. The method provided by the embodiment of the invention comprises the following steps: acquiring an actual image characteristic value of a point to be detected, wherein the point to be detected is a pixel point in a first image or a second image, the point to be detected is not contained in a matching region, and the matching region is a region included in both the first image and the second image; acquiring a characteristic value set corresponding to the first image and the second image, wherein the characteristic value set comprises actual image characteristic values of all pixel points in the matching region; finding out a target pixel point in the matching region according to the actual image characteristic value and the characteristic value set of the point to be detected, wherein the difference rate between the actual image characteristic value of the target pixel point and the actual image characteristic value of the point to be detected is smaller than a first preset difference value; and taking the depth of the target pixel point as the depth of the point to be detected.
Description
Technical Field
The invention belongs to the technical field of information analysis, and particularly relates to an image information acquisition method, image processing equipment and a computer storage medium.
Background
The binocular stereo vision is an important branch of computer vision, the binocular stereo vision is a principle of simulating human vision, a method for passively sensing distance by using a computer is used, namely two identical cameras are used for imaging the same object from different positions to obtain a stereo image pair of the object, the offset between pixels is calculated by a triangulation principle according to the pixel matching relation between the images to obtain the three-dimensional information of the object, the depth information of the object is obtained, and the actual distance between the object and the cameras, the three-dimensional size of the object and the actual distance between the two points can be calculated.
However, in practical application, when an object is blocked, the sight lines of the two cameras are blocked, so that the depth cannot be effectively calculated, and the visual error is large. In the prior art, cameras are generally added to perform imaging shooting from multiple angles, and then depth information of a blocked pixel is recovered according to multiple pictures shot by the cameras from multiple angles, so that the depth of a blocked part of an object is obtained to eliminate a dead angle.
However, in the prior art, a plurality of cameras are added to eliminate dead angles, and when a plurality of shielded directions of an object exist, a plurality of cameras are also correspondingly added to the shielded direction of the object, which increases hardware cost.
Disclosure of Invention
Embodiments of the present invention provide an image information acquisition method, an image processing apparatus, and a computer storage medium, which are used to reduce unnecessary hardware costs during multi-view stereoscopic vision detection.
A first aspect of an embodiment of the present invention provides an image information obtaining method, including:
acquiring an actual image characteristic value of a point to be detected, wherein the point to be detected is a pixel point in a first image or a second image, the point to be detected is not contained in a matching area, the matching area is an area included in both the first image and the second image, the first image is shot by a first camera, the second image is shot by a second camera, and the first image and the second image are images obtained by shooting the same target at different angles;
acquiring a feature value set corresponding to the first image and the second image, wherein the feature value set comprises actual image feature values of all pixel points in the matching region;
finding out a target pixel point in the matching region according to the actual image characteristic value of the point to be detected and the characteristic value set, wherein the difference rate between the actual image characteristic value of the target pixel point and the actual image characteristic value of the point to be detected is smaller than a first preset difference value;
and taking the depth of the target pixel point as the depth of the point to be detected.
A second aspect of embodiments of the present invention provides an image processing apparatus, including:
a memory, a processor, and a sensor;
the memory for storing a computer program;
the processor is configured to obtain an actual image feature value of a point to be detected, where the point to be detected is a pixel point in a first image or a second image, the point to be detected is not included in a matching region, the matching region is a region included in both the first image and the second image, the first image is captured by a first camera, the second image is captured by a second camera, and the first image and the second image are images obtained by capturing the same target at different angles; the characteristic value set is used for acquiring the characteristic value set corresponding to the first image and the second image, and the characteristic value set comprises the actual image characteristic value of each pixel point in the matching region; the matching area is used for finding out a target pixel point in the matching area according to the actual image characteristic value of the point to be detected and the characteristic value set, and the difference rate between the actual image characteristic value of the target pixel point and the actual image characteristic value of the point to be detected is smaller than a first preset difference value; the depth of the target pixel point is used as the depth of the point to be detected;
the sensor is used for acquiring the first image and the second image.
A third aspect of embodiments of the present invention provides a computer storage medium including a computer program which, when run on a computer, causes the computer to perform the method of the above aspects.
In the technical solution provided in the embodiment of the present invention, a first camera and a second camera respectively shoot a same target at different angles to obtain a first image and a second image, a pixel point only existing in the first image or the second image is referred to as a point to be detected, regions both included in the first image and the second image are referred to as matching regions, an actual image feature value of the point to be detected is obtained by detecting the image where the point to be detected is located, an actual image feature value of each pixel point in the matching region is obtained by detecting the first image and the second image to obtain a feature value set, a target pixel point is found according to the actual image feature value and the feature value set of the point to be detected, a difference rate between the actual image feature value of the target pixel point and the actual image feature value of the point to be detected is smaller than a first preset value, a depth of the target pixel point is obtained and is used as a depth of the point to, in the embodiment, the shielded part, namely the part only existing in the first image or the second image, is eliminated without additionally adding a camera, and the target pixel point is found out in the unshielded part, namely the matching area, so that the point to be detected in the shielded part is restored, and unnecessary hardware expenditure is reduced.
Drawings
FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present invention;
FIG. 2A is a flowchart of an embodiment of an image information obtaining method according to an embodiment of the present invention;
FIG. 2B is a schematic diagram of a depth calculation technique according to an embodiment of the invention;
FIG. 3A is a flowchart of another embodiment of an image information obtaining method according to an embodiment of the present invention;
FIG. 3B is a schematic diagram of a technique for determining epipolar lines in an embodiment of the present invention;
fig. 4 is a device diagram of an embodiment of an image processing apparatus in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention is suitable for the application scenario shown in fig. 1. Points a and b on the object a are projected onto the sensor 1 through the lens 1, while the points a and b are blocked from the view of the lens 2, so that the effective depths of the points a and b cannot be calculated. In the prior art, a plurality of cameras are added to shoot from multiple angles to eliminate dead angles, however, when the sheltered direction of an object is multiple, a plurality of cameras also need to be added correspondingly, and the hardware cost is increased.
In view of this, in the embodiment of the present invention, it is not necessary to add an additional camera to eliminate the blocked part, that is, the part existing only in the first image or the second image, where the first image and the second image are images obtained by shooting the same target at different angles, and the target pixel point is found in the matching area, which is the part that is not to be blocked, to restore the point to be detected in the blocked part, so as to reduce unnecessary hardware cost.
For convenience of understanding, a specific flow in the embodiment of the present invention is described below, and referring to fig. 2A, an embodiment of the image information acquiring method in the embodiment of the present invention includes:
201. and acquiring the actual image characteristic value of the point to be detected.
When the distance needs to be measured and the 3D scene needs to be recovered, one of the most common methods for binocular stereo vision is to use the parallax of two images of the same target obtained by two cameras to calculate the depth. In the embodiment of the present invention, the first camera takes the first image, the second camera takes the second image, and the first image and the second image are taken at different angles with respect to the same object, so that the first image and the second image have the same region and different regions. The device may detect an image where the point to be detected is located by using an image analysis method to obtain an actual image feature value of the point to be detected, where the point to be detected is a pixel point in the first image or the second image and is not included in the matching region.
It should be noted that, in practical applications, there are various ways to obtain the actual image feature value, for example, a gray variance algorithm, which may specifically adopt the following formula to calculate:
wherein the content of the first and second substances,
representing the mean of the grey values of the image, f (x, y) the grey value of the image at the point (x, y), Nx, Ny the width and height of the image, respectively, and s the actual image feature value of the point (x, y).
It should be noted that the embodiment of the present invention can be applied to the multi-view stereoscopic technology, that is, includes at least two cameras, and for convenience of description, the embodiment of the present invention is described by taking two cameras as an example.
202. And acquiring a characteristic value set corresponding to the first image and the second image.
Similar to the manner of obtaining the actual image feature values of the points to be detected in step 201, in step 202, the first image and the second image are detected through image analysis to obtain a feature value set corresponding to the first image and the second image, where the feature value set includes the actual image feature value of each pixel point in a matching region, that is, a region included in both the first image and the second image.
It should be noted that, the device obtains the actual image feature value of the point to be detected through step 201, and obtains the feature value set according to step 202, and the two processes do not have a precedence relationship, and may execute step 201 first, or execute step 202 first, or execute them simultaneously, and the specific details are not limited herein.
203. And finding out a target pixel point in the matching area according to the actual image characteristic value and the characteristic value set of the point to be detected.
The device obtains the characteristic value set and the actual image characteristic value of the point to be detected, and then finds out a target pixel point in the matching area according to the characteristic value set and the actual image characteristic value of the point to be detected, wherein the difference rate between the actual image characteristic value of the target pixel point and the actual image characteristic value of the point to be detected is smaller than a first preset difference value.
204. And taking the depth of the target pixel point as the depth of the point to be detected.
After the device determines the target pixel point in the matching region, because the target pixel point is a pixel point included in the first image and the second image, the device can determine the depth of the target pixel point according to a preset algorithm, and the depth of the target pixel point is used as the depth of a point to be detected. Where depth refers to the distance of a point in the scene from the XY plane in which the center of the camera is located. In practical applications, the depth information of each point in the scene can be represented by a depth map, that is, each pixel in the depth map records the distance from a certain point in the scene to the XY plane where the center of the camera is located. In addition, there are various ways to determine the depth of the pixel points, for example, special hardware devices may be used to actively acquire the depth information of each pixel point in the image, for example, an infrared pulse light source is used to transmit a signal to a scene, and then an infrared sensor is used to detect infrared light reflected by an object in the scene, so as to determine the distance from each pixel point in the image to the camera; alternatively, based on the conventional computer stereo vision method, the depth information of an object is restored by stereo matching using two images or a plurality of view images of the same scene obtained at two different views, including: (1) carrying out stereo matching on the image pair to obtain a parallax image of a corresponding point; (2) and calculating the depth according to the relation between the parallax and the depth of the corresponding point, so that the parallax image is converted into a depth image. Therefore, in the embodiment of the present invention, referring to fig. 2B, the following formula can be applied to calculate the depth of the pixel point: z ═ B × f/(x-x '), where O and O' respectively represent the first camera and the second camera, Z is used to represent the depth of the pixel, B is used to represent the distance between the optical center of the first camera and the optical center of the second camera, f is used to represent the focal length of the first camera or the second camera, x and x 'correspond to the distance between the pixel and the projection point of the camera center on the image plane, and the difference between the two, i.e., (x-x'), is used to represent the parallax of the pixel.
According to the technical scheme, the device finds the target pixel point in the matching area through the characteristic value set corresponding to the first image and the second image and the actual image characteristic value of the point to be detected, calculates the depth of the point to be detected by taking the depth of the target pixel point as the depth of the point to be detected, does not need to additionally increase a camera, and reduces unnecessary hardware expenditure.
For ease of understanding, the image information acquisition method of the embodiment of the present invention will be described in detail below. Referring to fig. 3A, fig. 3A is a flowchart illustrating an image information obtaining method according to another embodiment of the present invention.
301. And acquiring the actual image characteristic value of the point to be detected.
302. And acquiring a characteristic value set corresponding to the first image and the second image.
In the embodiment of the present invention, steps 301 to 302 in fig. 3A are similar to steps 201 to 202 in fig. 2A, and are not repeated here.
303. Judging whether a target actual image characteristic value exists in the characteristic value set or not; if yes, go to step 304; if not, go to step 306.
After obtaining the actual image characteristic value and the characteristic value set of the image to be detected, judging whether a target actual image characteristic value exists in the characteristic value set, wherein the difference rate between the target actual image characteristic value and the actual image characteristic value of the point to be detected is smaller than a first preset difference value, namely, the equipment judges that the target actual image characteristic value is the same as the actual image characteristic value to be detected or the error range is within an acceptable range, if the target actual image characteristic value exists, executing a step 304; if not, go to step 306.
304. And the equipment selects one pixel point from the pixel points corresponding to the characteristic value of the target actual image as a target pixel point.
If the device determines that the target actual image feature value exists in the feature value set, it can be understood that the target actual image feature value may be one or more, and the corresponding pixel point may also be one or more, so that the device may randomly select one pixel point from the corresponding pixel points as the target pixel point.
It should be noted that, in practical applications, there are various ways to select the target pixel, for example, a pixel with the minimum difference rate from the actual image feature value of the point to be detected may also be selected as the target pixel, so the specific way to select the target pixel is not limited here.
305. And taking the depth of the target pixel point as the depth of the point to be detected.
In the embodiment of the present invention, step 305 in fig. 3A is similar to step 204 in fig. 2A, and is not repeated here.
306. A reference value of the first reference point is obtained.
When it is determined that the target actual image feature value does not exist in the feature value set, the device selects a pixel point from the matching region, which may be referred to as a first reference point in the embodiment of the present invention, and obtains a reference value of the first reference point, where the reference value at least includes a reference actual image feature value and a reference theoretical image feature value, the reference actual image feature value of the first reference point may be obtained by using an image detection technology, and a manner of obtaining the reference actual image feature value of the first reference point is similar to that of obtaining the actual image feature value of the point to be detected in step 201 in the embodiment shown in fig. 2A, and details are not repeated here. In addition, in practical applications, the reference theoretical image feature value of the first reference point may be obtained by a preset calculation formula, for example, the preset calculation formula may be as follows: c ═ d ═ F2/(2nU2M), where n represents the aperture value of the camera; c represents a theoretical image characteristic value of the first reference point, and in the formula, the theoretical image characteristic value is a theoretical ambiguity value; u represents the depth of a point to be detected; f denotes a lens focal length of the camera; d is a fixed value when the camera system is fixed; m is the depth of field, where depth of field is understood to be the range of subject fore-aft distances measured at the front edge of the camera lens or other imager to enable sharp imaging.
307. And calculating a theoretical image characteristic value of the point to be detected according to the reference value of the first reference point and the actual image characteristic value of the point to be detected.
After obtaining the reference value of the first reference point, the device calculates the theoretical image feature value of the point to be detected according to the detected actual image feature value of the point to be detected, and the calculation process may include the following steps: the reference actual image characteristic value of the first reference point is set as R1, the reference theoretical image characteristic value of the first reference point is set as M1, the actual image characteristic value of the point to be detected is set as R2, and the theoretical image characteristic value of the point to be detected is set as M2.
308. And calculating the depth of the point to be detected according to the theoretical image characteristic value of the point to be detected.
After obtaining the theoretical image feature value of the point to be detected, the device calculates the depth of the point to be detected according to a preset formula, for example, the depth of the point to be detected may be calculated as follows:
2ncU2/F2d/m, where n represents the aperture value of the camera; c represents the theoretical ambiguity value of the point to be detected; u represents the depth of a point to be detected; f denotes a lens focal length of the camera; d is a fixed value when the camera system is fixed; m is the depth of field, where depth of field is understood to be the range of subject fore-aft distances measured at the front edge of the camera lens or other imager to enable sharp imaging. Therefore, n, c, F, d and m are known, so that the depth U of the point to be detected can be calculated.
309. And correspondingly finding a first closed edge and a second closed edge in the first area and the second area respectively through a contour extraction method.
After the device obtains the depth of the point to be detected through calculation, the device verifies the depth of the point to be detected in order to ensure that the depth of the point to be detected is credible. In the embodiment of the present invention, an area of a matching area in a first image is referred to as a first area, an area of the matching area in a second image is referred to as a second area, and an apparatus may find a first closed edge in the first area and a second closed edge in the second area by using a contour extraction method in the prior art, where the purpose of the contour extraction method is to obtain peripheral contour features of the image, and the steps of the contour extraction method may include first finding any point on a contour of the extracted image as a starting point, starting from the starting point, searching in a field of the starting point along one direction, constantly finding a next contour boundary point of the detected image, finally obtaining a complete contour area, and obtaining a closed edge of the contour area.
310. And when the first closed edge and the second closed edge are matched, finding out a first world point and a second world point according to the first closed edge or the second closed edge and the polar plane.
311. And determining a target intersection point according to the first world point and the second world point.
312. The depth of the target intersection is taken as the target value.
After the device finds the first closed edge in the first area and the second closed edge in the second area, the number of the first closed edge and the second closed edge may be one or more, so that the second closed edge matched with each first closed edge needs to be determined. In practical application, the matching between the first closed edge and the second closed edge may be determined through a preset matching algorithm, which specifically includes performing correlation calculation on a point on the first closed edge and a point on the second closed edge, for example, accumulating correlation values of each point on the first closed edge and each point on each second closed edge to obtain an accumulated value, and finding out the second closed edge corresponding to the largest accumulated value among the second closed edges, which is considered to be matched with the first closed edge.
Suppose that there is a point P on the object captured by the first camera and the second camera1Its projection point on the first camera imaging plane is P1The projection point on the imaging plane of the second camera is P2As shown in FIG. 3B, wherein C1And C2The optical centers of the first camera and the second camera, respectively, i.e. the origins of the camera coordinate systems. In epipolar geometry, called C1And C2The connecting line of (1) is a baseline. The intersection of the base line and the imaging plane of the first camera is called the intersection e1The intersection point e1For the poles of the first camera, the intersection of the base line and the imaging plane of the second camera is, likewise, called the intersection e2The intersection point e2The poles of the second camera are the optical centers C of the two cameras1And C2Projection coordinates on the corresponding camera imaging plane. P, C1And C2The resulting triangular plane is called the polar plane pi. Pi and the intersection l of the two camera imaging planes1And l2Called polar line, which can be called l1Is a point P1Corresponding polar line,/2Is a point P2The corresponding polar line.
And after the first closed edge is matched with the second closed edge, arbitrarily selecting a point M from the first closed edge, determining an polar plane formed by the point M, the optical center of the first camera and the optical center of the second camera, wherein the intersection line of the polar plane and the imaging plane of the first camera is an epipolar line, on the two-dimensional plane, the epipolar line and the first closed edge have at least two intersection points, for convenience of description, the at least two intersection points are called a first world point and a second world point, and in a quadrilateral area formed by the four points of the first world point, the second world point, the optical center of the first camera and the optical center of the second camera, a point where diagonals in the quadrilateral intersect is found out to be a target intersection point, and the depth of the target intersection point is determined to be a target value so as to verify the depth of the point to be detected. It is understood that, in practical applications, a point may be taken from the second closed edge to form a polar plane with the optical center of the first camera and the optical center of the second camera, and the description is not limited herein.
313. Verifying whether the depth of the point to be detected is greater than the target value, if yes, executing step 314; if not, go to step 315.
And after the equipment acquires the target value, verifying whether the depth of the point to be detected is credible according to the target value. Comparing the target value with the depth of the point to be detected, and if the depth of the point to be detected is greater than the target value, executing step 314; when the depth of the point to be detected is not greater than the target value, step 315 is performed.
314. And confirming that the depth of the point to be detected passes the verification.
And when the depth of the point to be detected is greater than the target value, the equipment confirms that the depth of the point to be detected is credible, namely confirms that the depth of the point to be detected passes verification.
315. And taking the average value of the depths of the pixel points adjacent to the point to be detected as the depth of the point to be detected.
When the depth of the point to be detected is not greater than the target value, the device uses the average of the depths of the pixel points adjacent to the point to be detected as the depth of the point to be detected, the adjacent pixel points include a plurality of pixel points, the pixel points in the matching region can be included, the depths of the pixel points in the matching region are known, the pixel points not in the matching region can also include pixel points not in the matching region, it needs to be stated that the pixel points not in the matching region are points whose depths have been estimated and whose depths have passed verification, that is, if the pixel points adjacent to the point to be detected are not in the matching region and the estimated depths do not pass verification, the depth of the point is not considered when the average of the depths of the pixel points adjacent to the point to be detected is calculated.
In addition, it can be understood that there are various ways to select the pixel point adjacent to the point to be detected, for example, select the pixel point directly adjacent to the point to be detected, and the like, and the specific details are not limited herein.
316. And synthesizing the first image and the second image into a depth image.
In the embodiment of the invention, a first area of a first image is matched with a second area of a second image, and then the first area of the first image is overlapped with the second area of the second image to obtain an overlapped image, namely the depth image, wherein the depth image comprises the matching area and a shielding area, the shielding area comprises an area of the first image excluding the first area, and an area of the second image excluding the second area.
317. A target occlusion point is selected from the sub-occlusion region.
In the embodiment of the invention, for convenience of expression, the area is called a sub-shielding area, namely the depth distribution of the sub-shielding area does not correspond to the gray value distribution of the sub-shielding area, for example, the depth distribution of the sub-shielding area is progressively increased distribution, and the gray value distribution of the sub-shielding area is firstly decreased and then increased distribution, so that the equipment needs to perform smooth preprocessing on the sub-shielding area and select a target shielding point in the sub-shielding area, wherein the depth difference between the target shielding point and an adjacent point of the target shielding point is greater than a preset difference.
318. And taking the depth average value of the adjacent points of the target occlusion point as the depth of the target occlusion point.
After the device determines the target shielding point, the depth of the adjacent point of the target shielding point is obtained, the average value of the depths of the adjacent points is calculated to obtain the average value of the depths of the adjacent points, and the device can use the average value of the depths as the depth of the target shielding point.
It should be noted that, in practical application, the process is an optional step, and is not limited here specifically.
The present invention further provides an apparatus, please refer to fig. 4, which is a device diagram of an apparatus in an embodiment of the present invention, wherein the apparatus 40 includes: memory 410, processor 420, sensor 430.
Wherein the memory 410 is used for storing a computer program;
the processor 420 is adapted to perform the following steps by calling the computer program stored in the memory 410:
acquiring an actual image characteristic value of a point to be detected, wherein the point to be detected is a pixel point in a first image or a second image, the point to be detected is not contained in a matching area, the matching area is an area included in both the first image and the second image, the first image is shot by a first camera, the second image is shot by a second camera, and the first image and the second image are images obtained by shooting the same target at different angles; the characteristic value set is used for acquiring the characteristic value set corresponding to the first image and the second image, and the characteristic value set comprises the actual image characteristic value of each pixel point in the matching region; the matching area is used for finding out a target pixel point in the matching area according to the actual image characteristic value of the point to be detected and the characteristic value set, and the difference rate between the actual image characteristic value of the target pixel point and the actual image characteristic value of the point to be detected is smaller than a first preset difference value; the depth of the target pixel point is used as the depth of the point to be detected;
a sensor 430 for acquiring the first image and the second image.
It should be noted that, in this embodiment, the processor 420 may also be referred to as a Central Processing Unit (CPU).
The memory 410 is used for storing computer programs and data so that the processor 420 can call the computer programs to realize corresponding operations, and may include a read-only memory and a random access memory. A portion of Memory 410 may also include Non-Volatile Random Access Memory (NVRAM).
The device 40 further comprises a bus system 440, the bus system 440 coupling together the various components of the device 40, including the sensor 410, the memory 420, the processor 430, wherein the bus system 440 may include a power bus, a control bus, a status signal bus, etc., in addition to a data bus. For clarity of illustration, however, the various buses are labeled in the figure as bus system 440.
In this embodiment, it should be further noted that the method disclosed in the above embodiment of the present invention may be applied to the processor 420, or implemented by the processor 420. Processor 420 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by a computer program in the form of hardware integrated logic circuits or software in the processor 420. The processor 420 may be a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware component. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may reside in ram, flash memory, rom, prom, or eprom, registers, among other computer storage media that are well known in the art. The computer storage medium is located in the memory 410, and the processor 420 reads the information in the memory 410 and performs the steps of the above method in combination with the hardware thereof.
The specific implementation of the processor 420 in the foregoing embodiment to find the target pixel point in the matching region according to the first actual image feature value and the feature value set may be:
judging whether a target actual image characteristic value exists in the characteristic value set or not, wherein the difference rate between the target actual image characteristic value and the actual image characteristic value is smaller than the first preset difference value; and if so, selecting one pixel point from the pixel points corresponding to the characteristic value of the target actual image as the target pixel point.
In another possible embodiment, the processor 420 may also call the computer program in the memory 410 to perform the following steps:
when the characteristic value set does not have the characteristic value of the target actual image, acquiring a reference theoretical image characteristic value of a first reference point according to a preset calculation formula, wherein the first reference point is a pixel point in the matching region; calculating theoretical image characteristic values of the points to be detected according to the reference values of the first reference points and the actual image characteristic values of the points to be detected, wherein the reference values of the first reference points comprise reference actual image characteristic values and reference theoretical image characteristic values of the first reference points; and calculating the depth of the point to be detected according to the theoretical image characteristic value of the point to be detected.
In another possible embodiment, the processor 420 may also call the computer program in the memory 410 to perform the following steps:
verifying whether the depth of the point to be detected is greater than a target value;
if yes, confirming that the depth of the point to be detected passes verification;
and if not, taking the average value of the depths of the pixel points adjacent to the point to be detected as the depth of the point to be detected.
In another possible embodiment, the processor 420 may also call the computer program in the memory 410 to perform the following steps:
correspondingly finding a first closed edge and a second closed edge in a first area and a second area respectively through a contour extraction method, wherein the first area is an area of a matching area in a first image, and the second area is an area of the matching area in a second image;
when the first closed edge is matched with the second closed edge, finding out a first world point and a second world point according to the first closed edge or the second closed edge and the polar plane;
determining a target intersection point according to the first world point and the second world point;
the depth of the target intersection is taken as the target value.
In another possible embodiment, the processor 420 may also call the computer program in the memory 410 to perform the following steps:
synthesizing the first image and the second image into a depth image, wherein the depth image comprises a matching area and an occlusion area, and the occlusion area comprises an area of the first image except the first area and an area of the second image except the second area;
and when the depth distribution of the sub-shielding region does not correspond to the gray value distribution of the corresponding image of the sub-shielding region, preprocessing the shielding region, wherein the sub-shielding region is contained in the shielding region.
The specific implementation of the processor 420 for preprocessing the sub-occlusion region in the above embodiment may be:
selecting a target shielding point from the sub-shielding area, wherein the depth difference between the target shielding point and an adjacent point of the target shielding point is greater than a preset difference value;
and taking the depth average value of the adjacent points of the target occlusion point as the depth of the target occlusion point.
In the above embodiment, the first camera and the second camera respectively shoot the same target at different angles to obtain a first image and a second image, the pixel point only existing in the first image or the second image is called a point to be detected, the regions included in both the first image and the second image are called matching regions, the image where the point to be detected is located is detected to obtain the actual image feature value of the point to be detected, the actual image feature value of each pixel point in the matching region is obtained by detecting the first image and the second image to obtain the feature value set, the target pixel point is found according to the actual image feature value and the feature value set of the point to be detected, the difference between the actual image feature value of the target pixel point and the actual image feature value of the point to be detected is smaller than the first preset value, and the depth of the target pixel point is obtained and is used as the depth of the point to be detected, the shielded part, namely the part only existing in the first image or the second image, is eliminated without additionally adding a camera, and the target pixel points are found out in the unshielded part, namely the matching area, so that the points to be detected in the shielded part are restored, and unnecessary hardware expenditure is reduced.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several computer programs to enable a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. An image information acquisition method, applied to a multi-view stereoscopic technique, comprising:
acquiring an actual image characteristic value of a point to be detected, wherein the point to be detected is a pixel point in a first image or a second image, the point to be detected is not contained in a matching area, the matching area is an area included in both the first image and the second image, the first image is shot by a first camera, the second image is shot by a second camera, and the first image and the second image are images obtained by shooting the same target at different angles;
acquiring a feature value set corresponding to the first image and the second image, wherein the feature value set comprises actual image feature values of all pixel points in the matching region;
judging whether a target actual image characteristic value exists in the characteristic value set or not, wherein the difference rate between the target actual image characteristic value and the actual image characteristic value is smaller than the first preset difference value;
if yes, selecting one pixel point from pixel points corresponding to the target actual image characteristic value as the target pixel point; taking the depth of the target pixel point as the depth of the point to be detected;
if the reference theoretical image characteristic value does not exist, acquiring a reference theoretical image characteristic value of a first reference point according to a preset calculation formula, wherein the first reference point is a pixel point in the matching area; calculating theoretical image characteristic values of the points to be detected according to the reference values of the first reference points and the actual image characteristic values of the points to be detected, wherein the reference values of the first reference points comprise the reference actual image characteristic values of the first reference points and the reference theoretical image characteristic values; and calculating the depth of the point to be detected according to the theoretical image characteristic value of the point to be detected.
2. The image information acquisition method according to claim 1, wherein after the depth of the target pixel point is taken as the depth of the point to be detected, the method further comprises:
correspondingly finding a first closed edge and a second closed edge in a first area and a second area respectively through a contour extraction method, wherein the first area is an area of the matching area in the first image, and the second area is an area of the matching area in the second image;
when the first closed edge is matched with the second closed edge, finding out a first world point and a second world point according to the first closed edge or the second closed edge and the polar plane;
determining a target intersection point according to the first world point and the second world point;
taking the depth of the target intersection point as the target value;
verifying whether the depth of the point to be detected is greater than a target value;
if so, confirming that the depth of the point to be detected passes verification;
and if not, taking the average value of the depths of the pixel points adjacent to the point to be detected as the depth of the point to be detected.
3. The image information obtaining method according to claim 2, wherein after the depth of the target pixel point is taken as the depth of the point to be detected, the method further includes:
synthesizing the first image and the second image into a depth image, wherein the depth image comprises the matching area and an occlusion area, and the occlusion area comprises an area of the first image except the first area and an area of the second image except the second area;
and when the depth distribution of the sub-shielding region does not correspond to the gray value distribution of the corresponding image of the sub-shielding region, preprocessing the shielding region, wherein the sub-shielding region is contained in the shielding region.
4. The image information acquisition method according to claim 3, wherein the preprocessing the occlusion region includes:
selecting a target shielding point from the sub-shielding area, wherein the depth difference between the target shielding point and an adjacent point of the target shielding point is greater than a preset difference value;
and taking the depth average value of the adjacent points of the target occlusion point as the depth of the target occlusion point.
5. An image processing apparatus characterized by comprising:
a memory, a processor, and a sensor;
the memory for storing a computer program;
the processor is configured to obtain an actual image feature value of a point to be detected, where the point to be detected is a pixel point in a first image or a second image, the point to be detected is not included in a matching region, the matching region is a region included in both the first image and the second image, the first image is captured by a first camera, the second image is captured by a second camera, and the first image and the second image are images obtained by capturing the same target at different angles; acquiring a feature value set corresponding to the first image and the second image, wherein the feature value set comprises actual image feature values of all pixel points in the matching region; judging whether a target actual image characteristic value exists in the characteristic value set or not, wherein the difference rate between the target actual image characteristic value and the actual image characteristic value is smaller than the first preset difference value; if yes, selecting one pixel point from pixel points corresponding to the target actual image characteristic value as the target pixel point; taking the depth of the target pixel point as the depth of the point to be detected; if the reference theoretical image characteristic value does not exist, acquiring a reference theoretical image characteristic value of a first reference point according to a preset calculation formula, wherein the first reference point is a pixel point in the matching area; calculating theoretical image characteristic values of the points to be detected according to the reference values of the first reference points and the actual image characteristic values of the points to be detected, wherein the reference values of the first reference points comprise the reference actual image characteristic values of the first reference points and the reference theoretical image characteristic values; calculating the depth of the point to be detected according to the theoretical image characteristic value of the point to be detected;
the sensor is used for acquiring the first image and the second image.
6. The image processing device of claim 5, wherein the processor is further configured to:
after the depth of the target pixel point is taken as the depth of the point to be detected, a first closed edge and a second closed edge are respectively and correspondingly found in a first area and a second area through a contour extraction method, wherein the first area is an area of the matching area in the first image, and the second area is an area of the matching area in the second image; when the first closed edge is matched with the second closed edge, finding out a first world point and a second world point according to the first closed edge or the second closed edge and the polar plane; determining a target intersection point according to the first world point and the second world point; taking the depth of the target intersection point as the target value; verifying whether the depth of the point to be detected is greater than a target value; if so, confirming that the depth of the point to be detected passes verification; and if not, taking the average value of the depths of the pixel points adjacent to the point to be detected as the depth of the point to be detected.
7. The image processing device of claim 6, wherein the processor is further configured to:
after the depth of the target pixel point is taken as the depth of the point to be detected, synthesizing the first image and the second image into a depth image, wherein the depth image comprises the matching area and a shielding area, and the shielding area comprises an area of the first image except the first area and an area of the second image except the second area; and when the depth distribution of the sub-shielding region does not correspond to the gray value distribution of the corresponding image of the sub-shielding region, preprocessing the shielding region, wherein the sub-shielding region is contained in the shielding region.
8. The image processing device of claim 7, wherein the processor is further configured to:
selecting a target shielding point from the sub-shielding area, wherein the depth difference between the target shielding point and an adjacent point of the target shielding point is greater than a preset difference value; and taking the depth average value of the adjacent points of the target occlusion point as the depth of the target occlusion point.
9. A computer storage medium comprising a computer program which, when run on a computer, causes the computer to execute the image information acquisition method according to any one of claims 1 to 4.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/094932 WO2019019160A1 (en) | 2017-07-28 | 2017-07-28 | Method for acquiring image information, image processing device, and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110800020A CN110800020A (en) | 2020-02-14 |
| CN110800020B true CN110800020B (en) | 2021-07-09 |
Family
ID=65039334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780092646.9A Active CN110800020B (en) | 2017-07-28 | 2017-07-28 | Image information acquisition method, image processing equipment and computer storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110800020B (en) |
| WO (1) | WO2019019160A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111627061B (en) * | 2020-06-03 | 2023-07-11 | 如你所视(北京)科技有限公司 | Pose detection method and device, electronic equipment and storage medium |
| CN113484852B (en) * | 2021-07-07 | 2023-11-07 | 烟台艾睿光电科技有限公司 | Distance measurement method and system |
| CN117575886B (en) * | 2024-01-15 | 2024-04-05 | 之江实验室 | Image edge detector, detection method, electronic equipment and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102204262A (en) * | 2008-10-28 | 2011-09-28 | 皇家飞利浦电子股份有限公司 | Generation of occlusion data for image properties |
| CN103679739A (en) * | 2013-12-26 | 2014-03-26 | 清华大学 | Virtual view generating method based on shielding region detection |
| CN104063702A (en) * | 2014-07-16 | 2014-09-24 | 中南大学 | Three-dimensional gait recognition based on shielding recovery and partial similarity matching |
| CN104574331A (en) * | 2013-10-22 | 2015-04-29 | 中兴通讯股份有限公司 | Data processing method, device, computer storage medium and user terminal |
| CN105184780A (en) * | 2015-08-26 | 2015-12-23 | 京东方科技集团股份有限公司 | Prediction method and system for stereoscopic vision depth |
| CN105279786A (en) * | 2014-07-03 | 2016-01-27 | 顾海松 | Method and system for obtaining object three-dimensional model |
| CN106355570A (en) * | 2016-10-21 | 2017-01-25 | 昆明理工大学 | Binocular stereoscopic vision matching method combining depth characteristics |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5768684B2 (en) * | 2011-11-29 | 2015-08-26 | 富士通株式会社 | Stereo image generation apparatus, stereo image generation method, and computer program for stereo image generation |
-
2017
- 2017-07-28 CN CN201780092646.9A patent/CN110800020B/en active Active
- 2017-07-28 WO PCT/CN2017/094932 patent/WO2019019160A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102204262A (en) * | 2008-10-28 | 2011-09-28 | 皇家飞利浦电子股份有限公司 | Generation of occlusion data for image properties |
| CN104574331A (en) * | 2013-10-22 | 2015-04-29 | 中兴通讯股份有限公司 | Data processing method, device, computer storage medium and user terminal |
| CN103679739A (en) * | 2013-12-26 | 2014-03-26 | 清华大学 | Virtual view generating method based on shielding region detection |
| CN105279786A (en) * | 2014-07-03 | 2016-01-27 | 顾海松 | Method and system for obtaining object three-dimensional model |
| CN104063702A (en) * | 2014-07-16 | 2014-09-24 | 中南大学 | Three-dimensional gait recognition based on shielding recovery and partial similarity matching |
| CN105184780A (en) * | 2015-08-26 | 2015-12-23 | 京东方科技集团股份有限公司 | Prediction method and system for stereoscopic vision depth |
| CN106355570A (en) * | 2016-10-21 | 2017-01-25 | 昆明理工大学 | Binocular stereoscopic vision matching method combining depth characteristics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110800020A (en) | 2020-02-14 |
| WO2019019160A1 (en) | 2019-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| AU2016355215B2 (en) | Methods and systems for large-scale determination of RGBD camera poses | |
| CN107392958B (en) | Method and device for determining object volume based on binocular stereo camera | |
| KR101706216B1 (en) | Apparatus and method for reconstructing dense three dimension image | |
| CN107798702B (en) | Real-time image superposition method and device for augmented reality | |
| EP3016071B1 (en) | Estimating device and estimation method | |
| CN108510540B (en) | Stereoscopic vision camera and height acquisition method thereof | |
| JP2012070389A5 (en) | ||
| CN111160232B (en) | Front face reconstruction method, device and system | |
| CN107590444B (en) | Method and device for detecting static obstacle and storage medium | |
| CN110800020B (en) | Image information acquisition method, image processing equipment and computer storage medium | |
| CN110505398B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN111757084A (en) | Acquisition method and acquisition device for three-dimensional image and readable storage medium | |
| WO2014092550A2 (en) | Method for camera motion estimation with presence of moving object | |
| CN109902675B (en) | Object pose acquisition method and scene reconstruction method and device | |
| CN111882655A (en) | Method, apparatus, system, computer device and storage medium for three-dimensional reconstruction | |
| CN111160233B (en) | Human face in-vivo detection method, medium and system based on three-dimensional imaging assistance | |
| CN111383255A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| US7158665B2 (en) | Image processing device for stereo image processing | |
| CN104463958A (en) | Three-dimensional super-resolution method based on disparity map fusing | |
| Ahmed et al. | Distance alert system using Stereo vision and feature extraction | |
| CN111489384A (en) | Occlusion assessment method, device, equipment, system and medium based on mutual view | |
| JP2013190938A (en) | Stereo image processing device | |
| CN111080689B (en) | Method and device for determining face depth map | |
| KR101844004B1 (en) | Image generating apparatus including distance information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |