WO2022218161A1 - Procédé et appareil d'appariement de cible, dispositif, et support de stockage - Google Patents

Procédé et appareil d'appariement de cible, dispositif, et support de stockage Download PDF

Info

Publication number
WO2022218161A1
WO2022218161A1 PCT/CN2022/084323 CN2022084323W WO2022218161A1 WO 2022218161 A1 WO2022218161 A1 WO 2022218161A1 CN 2022084323 W CN2022084323 W CN 2022084323W WO 2022218161 A1 WO2022218161 A1 WO 2022218161A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target
distance
feature point
matching
Prior art date
Application number
PCT/CN2022/084323
Other languages
English (en)
Chinese (zh)
Inventor
李泉录
李若岱
马堃
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2022218161A1 publication Critical patent/WO2022218161A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures

Definitions

  • the present application relates to the technical field of image processing, and in particular, to a method, apparatus, device and storage medium for target matching.
  • Object matching in images is a fundamental problem in computer vision, and the accuracy of object matching will affect the operations after object matching.
  • the common target matching method in images mainly uses the two closest feature points in the two images to be matched as the matching point pair. Taking face matching as an example, it is mainly to judge the offset of the faces in the two images. If the coordinates of the faces in the vertical direction in the two images are the same, and the difference in the horizontal direction is the smallest, it is considered to be a mutual match. face.
  • the problem with this method is that in the case of multiple faces in the image, the face with the smallest position difference in the two images may not be the face that actually matches each other.
  • the present application provides at least one method, apparatus, device and storage medium for target matching.
  • the present application provides a target matching method, including: acquiring a first image and a second image, wherein the first image and the second image are obtained by shooting a target to be matched by different shooting components; based on the relationship between the target and at least one shooting component The first distance between the first image and the second image is obtained to obtain the parallax of the target between the first image and the second image; according to the parallax, the matching feature point pair about the target in the first image and the second image is determined.
  • the first image and the second image can be effectively realized.
  • Object matching between two images compared with the method of directly using the feature point with the closest position in the two images as the matching feature point pair, the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then uses the parallax to obtain Matching point pairs improves the accuracy of target matching between different images.
  • determining the pair of matching feature points about the target in the first image and the second image according to the parallax includes: determining, based on the parallax, a reference position corresponding to the first feature point in the second image, wherein the first feature point is a pair of The first image is obtained by performing target feature point detection; based on the reference position, a second feature point matching the first feature point is selected in the second image, wherein the second feature point is the target feature point detection performed on the second image. owned.
  • the second feature point is determined according to the reference position, the area in the second image that matches the first feature point is reduced, and the match can be quickly found Point to improve the matching accuracy.
  • selecting a second feature point matching the first feature point in the second image includes: determining a candidate region including the reference position in the second image, and selecting a candidate region that matches the first feature point in the candidate region The second feature point whose positional relationship of the epipolar line corresponding to the point in the second image satisfies the preset requirement is taken as the second feature point matching the first feature point.
  • the accuracy of target matching is improved compared to using a single dimension.
  • the epipolar line corresponding to the first feature point in the second image is determined by the following operations: the epipolar line is obtained by using the coordinates of the first feature point and the fundamental matrix.
  • the preset requirement is that the distance between the second feature point and the epipolar line in the candidate region is the smallest.
  • the second feature point corresponding to the first feature point should be on the epipolar line, but errors will inevitably occur during the target matching process, resulting in the second feature point not being on the epipolar line, so the candidate is selected.
  • the feature point with the smallest distance from the epipolar line in the region is used as the second feature point corresponding to the first feature point, which can improve the success rate of target matching.
  • the first image or the second image is the reference image
  • the photographing component corresponding to the reference image is the reference photographing component
  • the first distance is the distance between the target and the reference photographing component
  • the first distance is obtained through the following steps: determining that the target is in the reference The first size of the corresponding target area in the image; the first distance is obtained by using the ratio between the first size and the preset size.
  • the technical effect that the distance between the target and the shooting component can be determined by using a single image is achieved, which breaks the
  • the traditional perception of the distance between the target and the photographed component can be obtained by combining at least two or more images.
  • the first size includes the first width and the first height of the target area
  • the preset size includes the preset width and the preset height
  • using the ratio between the first size and the preset size to obtain the first distance including: obtaining a first ratio between the first width and the preset width, and a second ratio between the first height and the preset height; the second distance between the target and the reference shooting component is obtained based on the first ratio, and based on the second
  • the ratio obtains the third distance between the target and the reference photographing component; the first distance is obtained based on the second distance and the third distance.
  • the first distance is obtained by the weighted summation of the second distance and the third distance, which can comprehensively consider the results in the width and height directions, rather than using the second or third distance obtained in a single direction as the first distance In other words, the accuracy of the first distance can be improved.
  • obtaining the second distance between the target and the reference photographing component based on the first ratio includes: multiplying the first ratio by the first focal length of the reference photographing component in the width direction to obtain the second distance; obtaining the target based on the second ratio
  • the third distance from the reference photographing component includes: multiplying the second ratio by the second focal length of the reference photographing component in the height direction to obtain the third distance; and obtaining the first distance based on the second distance and the third distance
  • the method includes: weighted summation of the second distance and the third distance to obtain the first distance.
  • the resolution of the reference capture assembly may be different in the width or height direction, by multiplying the first ratio by the first focal length of the reference capture assembly in the width direction, and multiplying the second ratio by the reference capture assembly in the height direction Multiplying the second focal length of , the second and third distances are more accurate.
  • determining the first size of the target corresponding to the target area in the reference image includes any one of the following: obtaining the first size based on the coordinates of at least two first feature points obtained by detecting the target feature points on the reference image; or, based on The size of the area obtained by performing the target area detection on the reference image is obtained to obtain the first size.
  • the process is simple and the computing power required by the device is low.
  • the first distance is the distance between the target and the reference photographing component
  • the reference photographing component is the photographing component corresponding to the first image or the second image
  • the parallax between an image and the second image includes: obtaining the parallax based on the first distance, the focal length of the reference shooting component, and the baseline between the shooting components corresponding to the first image and the second image.
  • the whole process is simple and requires The computing power is low, so that the processing resources of the execution device can be saved.
  • the present application provides a target matching device, comprising: an image acquisition module for acquiring a first image and a second image, wherein the first image and the second image are obtained by shooting a target to be matched by different shooting components; parallax acquisition a module for obtaining the parallax of the target between the first image and the second image based on the first distance between the target and the at least one photographing component; the matching module for determining the difference between the first image and the second image according to the parallax Matching feature point pairs about the target.
  • the present application provides an electronic device, including a memory and a processor, where the processor is configured to execute program instructions stored in the memory, so as to implement the target matching method in the first aspect.
  • a fourth aspect of the present application provides a computer-readable storage medium on which program instructions are stored, and when the program instructions are executed by a processor, the target matching method in the above-mentioned first aspect is implemented.
  • the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then uses the parallax to obtain Matching point pairs improves the accuracy of target matching between different images.
  • Fig. 1 is a schematic flow chart 1 of an embodiment of a target matching method of the present application
  • FIG. 2 is a second schematic flowchart of an embodiment of a target matching method of the present application
  • FIG. 3 is a schematic structural diagram of an embodiment of a target matching device of the present application.
  • FIG. 4 is a schematic structural diagram of an embodiment of an electronic device of the present application.
  • FIG. 5 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application.
  • Target matching refers to matching corresponding feature points belonging to the same target in the first image and the second image, so as to find out the same target in the first image and the second image.
  • the target can be any object, such as a face, a building, a vehicle, and so on.
  • FIG. 1 is a first schematic flowchart of an embodiment of a target matching method of the present application. Specifically, the target matching method may include the following steps:
  • Step S11 Acquire a first image and a second image, wherein the first image and the second image are obtained by photographing the target to be matched by different photographing components, respectively.
  • the shooting components corresponding to the first image and the second image can be placed arbitrarily, as long as the imaging planes of the first image and the second image are parallel, for example, the two shooting components can be placed side by side in the horizontal direction, It can also be placed side by side in the vertical direction.
  • Two shooting components can form a binocular camera system.
  • two photographing components are placed side by side in the horizontal direction as an example.
  • the device that performs target matching and the photographing component may be an integrated device, or may be independent devices. Integrated means that the device that performs target matching and the photographing component can be controlled by the same processor, and independent means that the execution Target matching equipment and shooting components are controlled by different processors.
  • the first image and the second image may be images that have not been image-processed, or may be images that have been image-processed. Image processing can be adjusting brightness, resolution, etc. Further, the modality of the first image and the second image may be the same or different.
  • the first image and the second image may be both visible light images or infrared light images, or one of them may be a visible light image and the other may be an infrared light image. image.
  • the form of the first image and the second image is not specified here.
  • Both the first image and the second image include at least one target, and one of the targets is used as the target to be matched, so as to match the feature points of the target in the first image and the second image, and then compare the first image and the second image. Find the target in the image to achieve target matching.
  • the target is a human face as an example.
  • Step S12 Obtain the parallax of the target between the first image and the second image based on the first distance between the target and the at least one photographing component.
  • the first distance between the target and at least one photographing component may be the distance between the target and one of the photographing components, or the distance from the position between the two photographing components.
  • the distance from the position between the two photographing components may specifically be the distance from the midpoint of the baselines of the two photographing components. Because the first distance from the photographing components is different, the parallax between the images obtained by photographing is also different. Specifically, the longer the first distance between the target and the photographing component, the smaller the parallax. Similarly, the closer the first distance between the target and the photographing component is, the greater the parallax. Therefore, the magnitude of the parallax can be determined according to the first distance between the target and the photographing component.
  • Step S13 According to the parallax, determine the matching feature point pair about the target in the first image and the second image.
  • the method of determining the matching feature point pair about the target in the first image and the second image according to the parallax may be to first determine a certain feature point in one of the images, and then determine the corresponding feature point in the other image according to the parallax Matching points to obtain the matching feature point pair of the target. If the target is a human face, the feature points related to the target in the first image and the second image may be obtained by performing feature detection on a region containing a human face in the images.
  • the first step can be effectively achieved.
  • Object matching between an image and a second image compared to the method of directly using the closest feature point in the two images as the matching feature point pair, the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then utilizes the parallax
  • the matching feature point pairs are obtained, which improves the accuracy of target matching between different images.
  • the first image or the second image is a reference image
  • the photographing component corresponding to the reference image is the reference photographing component
  • the first distance is the distance between the target and the reference photographing component.
  • the first size includes a first width and a first height of the target area.
  • the target area may be an area corresponding to a specific part of the target in the reference image.
  • the target region may be a corresponding region in the reference image of a face region composed of eyebrows, eyes, nose and mouth.
  • the first method is to obtain the first size based on the coordinates of the first feature point obtained by performing the target feature point detection on the reference image.
  • the embodiment of the present disclosure takes the reference image as the first image as an example.
  • the feature point coordinate information (X11, Y11), . . . (Xln, Yln) of the target to be detected in the first image and the second image are obtained
  • the first width may be the difference between the maximum value and the minimum value in all the width coordinates of the reference image, expressed as the first width by a formula
  • the first height can be the difference between the maximum value and the minimum value among all the height coordinates of the reference image, expressed as the first height by a formula
  • the feature point coordinate information of another image is simultaneously acquired, which simplifies the target matching process.
  • the second is to obtain the first size based on the size of the region obtained by performing target region detection on the reference image.
  • the area obtained by performing the target area detection on the reference image may be the area included in the target detection frame, and the area included in the target detection frame may correspond to a specific part of the target.
  • the width of the target detection frame is used as the first width
  • the height of the target detection frame is used as the first height.
  • the target detection frame may be a face detection frame, wherein the area included in the face detection frame is an area jointly formed by eyebrows, eyes, nose and mouth.
  • the reference image is input into the second target detection deep neural network, and a face detection frame is output, wherein the width of the face detection frame can be used as the first width, and the height of the face detection frame can be used as the first height.
  • the first distance is obtained by using the ratio between the first size and the preset size.
  • the preset size may include a preset width and a preset height.
  • the ratio between the first size and the preset size may be the ratio between the widths of the two or the ratio between the heights, or a combination of the ratio between the widths and the height.
  • the first distance may be obtained by performing perspective transformation using the first size.
  • the first ratio between the first width and the preset width, and the second ratio between the first height and the preset height may be acquired.
  • the second distance between the target and the reference photographing component is obtained based on the first ratio
  • the third distance between the target and the reference photographing component is obtained based on the second ratio.
  • the preset size here is the real size of the preset target with the same attribute as the target.
  • the preset size is the preset real size of other faces.
  • both the first size and the preset size may be sizes corresponding to the target preset part.
  • the preset size corresponds to the real size of an area composed of eyebrows, eyes, nose and mouth of the preset target
  • the first size corresponds to a target composed of eyebrows, eyes, nose and mouth in the reference image size of the area.
  • the second distance may be obtained by multiplying the first ratio by the first focal length of the reference photographing component in the width direction.
  • the formula for obtaining the second distance D2 is:
  • W1 is the preset width
  • w is the first width
  • fx is the first focal length of the reference photographing component in the width direction
  • different photographing components have different focal lengths in the width direction.
  • f x can be obtained by calibration of two reference shooting components, mainly by converting the real focal length of the reference shooting components to obtain the first focal length in the width direction and the second focal length in the height direction.
  • the definition of the width and height here is consistent with the definition of the width and height of the reference image.
  • the third distance may be obtained by multiplying the second ratio by the second focal length of the reference photographing component in the height direction.
  • the formula for obtaining the third distance D3 may be
  • H 1 refers to the preset height
  • f y refers to the second focal length of the reference shooting component in the height direction, and different shooting components have different focal lengths in the height direction
  • h refers to the first height
  • the first distance is obtained.
  • the second distance and the third distance may be weighted and summed to obtain the first distance.
  • the formula for the weighted summation can be
  • D 1 refers to the first distance
  • D 2 refers to the second distance
  • D 3 refers to the third distance
  • ⁇ is a preset parameter, and 0 ⁇ 1, for example, the value of ⁇ is 0.5.
  • ⁇ is a fixed parameter
  • the value of ⁇ is mainly related to the resolution of the reference photographing component. If the resolution of the reference photographing component is higher, the value of ⁇ is closer to 0.5.
  • the value of ⁇ can also be dynamically adjusted.
  • ⁇ is set to 0.5; if the absolute value of the difference between the acquired second distance and the third distance is greater than the first difference, the value of ⁇ is greater than 0.5 or less than 0.5.
  • the value of ⁇ is greater than 0.5, and if the second distance is less than the third distance, the value of ⁇ is less than 0.5.
  • the first distance is obtained by the weighted summation of the second distance and the third distance, which can comprehensively consider the results in the width and height directions, rather than using the second or third distance obtained in a single direction as the first distance In other words, the accuracy of the first distance can be improved.
  • the resolution of the reference photographing component in the width or height direction may be different, by multiplying the first ratio by the first focal length of the reference photographing component in the width direction, and multiplying the second ratio by the reference photographing component in the The second focal length in the height direction is multiplied, and the obtained second and third distances are more accurate.
  • the first distance is the distance between the target and the reference photographing component
  • the reference photographing component may be the photographing component corresponding to the first image or the second image.
  • the parallax is obtained based on the first distance, the focal length of the reference photographing component, and the baseline between the photographing components corresponding to the first image and the second image.
  • the formula for obtaining the parallax ⁇ d is as follows:
  • ⁇ d is the parallax
  • f x is the focal length of the reference photographing component in the width direction
  • D 1 is the first distance
  • B is the length of the baseline.
  • the length of the baseline can be obtained by calibrating the two shooting components.
  • the step of determining the matching feature point pair about the target in the first image and the second image includes: based on the parallax, determining a reference position corresponding to the first feature point in the second image.
  • the first feature point is obtained by detecting target feature points on the first image. Assuming that the coordinates of the first feature point in the first image are (x ln , y ln ) and the parallax is ⁇ d, the corresponding reference position of the first feature point in the second image should be (x ln + ⁇ d,y ln + ⁇ d).
  • the second feature point corresponding to the first feature point in the second image should be near the reference position (x ln + ⁇ d, y ln + ⁇ d). Then, based on the reference position, a second feature point that matches the first feature point is selected in the second image.
  • the second feature point is obtained by performing target feature point detection on the second image.
  • the feature point closest to the reference position in the second image may be selected as the second feature point matching the first feature point.
  • the distance here can be the Euclidean distance.
  • the second feature point is determined according to the reference position, the area in the second image that matches the first feature point is reduced, and the match can be quickly found Point to improve the matching accuracy.
  • the matching speed is also greatly improved, and the power consumption of the apparatus for performing the target matching method is reduced.
  • the step of determining the matching feature point pair about the target in the first image and the second image further includes determining the epipolar line corresponding to the first feature point in the second image.
  • the coordinates of the first feature point and the basic matrix can be used to obtain the epipolar line corresponding to the first feature point in the second image.
  • the basic matrix is obtained by calibrating two shooting components. Specifically, the process of calibrating the two photographing components can be performed before step S11.
  • the formula to obtain the epipolar line l can be:
  • (x ln , y ln , 1) is the homogeneous coordinate of the first feature point
  • F is the fundamental matrix
  • a candidate area including the reference position may be determined in the second image, and a second feature point whose positional relationship with the epipolar line meets a preset requirement may be selected in the candidate area.
  • the candidate area may be an area of a preset candidate size centered on the reference position.
  • the determination of the preset candidate size can be set according to experience, and no specific provisions are made here.
  • the feature point whose positional relationship with the epipolar line meets the preset requirements as the second feature point matching the first feature point it is equivalent to reducing the feature matching area from two-dimensional to one-dimensional, reducing two The effect of placement between photographic components on target matching.
  • target matching based on the two dimensions of disparity and epipolar geometric constraints, even when there are multiple faces in the first image and the second image, false matching is less likely to occur.
  • the preset requirement may be that the distance between the second feature point and the epipolar line in the candidate region is the smallest. That is, in the candidate region, the feature point with the smallest distance from the epipolar line is selected as the second feature point matching the first feature point.
  • the distance here can be the Euclidean distance.
  • the second feature point corresponding to the first feature point should be on the epipolar line, but errors will inevitably occur during the target matching process, resulting in the second feature point not being on the epipolar line, so the candidate is selected.
  • the feature point with the smallest distance from the epipolar line in the region is used as the second feature point corresponding to the first feature point, which can improve the success rate of target matching.
  • FIG. 2 is a second schematic flowchart of an embodiment of a target matching method of the present application.
  • the target matching method proposed by the embodiment of the present disclosure can be subdivided into the following steps:
  • Step S11 Acquire a first image and a second image, wherein the first image and the second image are obtained by photographing the target by different photographing components respectively.
  • Step S121 Obtain the first distance between the target and the reference photographing component.
  • the reference photographing component may be a photographing component corresponding to the first image, or may be a photographing component corresponding to the second image.
  • the reference photographing component may be made to the above, and details are not described herein again.
  • Step S122 Based on the first distance, obtain the parallax of the target between the first image and the second image.
  • the method of obtaining the parallax can refer to the above, and details are not repeated here.
  • Step S131 Determine a reference position corresponding to the first feature point in the second image based on the parallax, wherein the first feature point is obtained by performing target feature point detection on the first image.
  • Step S132 Determine the epipolar line corresponding to the first feature point in the second image.
  • step S131 and step S132 may be performed synchronously. Of course, in other embodiments, one may be performed first, and then the other may be performed. The execution order of step S131 and step S132 is not specified here.
  • Step S133 Determine a candidate region containing the reference position in the second image, and in the candidate region, select a second feature point whose positional relationship with the epipolar line meets the preset requirements, as the second feature point matching the first feature point. Feature points.
  • the preset requirements may be as described above; in the candidate area, the process of selecting the second feature point matching the first feature point is as described above, which will not be repeated here.
  • the second feature point matching the first feature point is determined by integrating the two dimensions of disparity and epipolar geometric constraint, which improves the accuracy of target matching compared to using a single dimension. Moreover, by selecting the feature point whose positional relationship with the epipolar line meets the preset requirements as the second feature point matching the first feature point, it is equivalent to reducing the feature matching area from two-dimensional to one-dimensional, reducing two The effect of placement between photographic components on target matching. And by performing target matching based on the two dimensions of disparity and epipolar geometric constraints, even when there are multiple faces in the first image and the second image, false matching is less likely to occur.
  • the extracted matching feature point pair about the target can be further utilized, for example, depth acquisition, living body detection, body temperature measurement, etc.
  • depth acquisition For example, in a multi-person temperature measurement scenario, it is necessary to locate the same face captured by different cameras in the binocular camera system, and then obtain the position of the face in the thermal imager according to the face coordinates, and finally perform temperature extraction. Therefore, face matching is a pre-task for rapid body temperature measurement in a multi-person scenario.
  • a face matching result with high precision and high robustness can be obtained to assist the body temperature in a multi-person scenario. Measurement.
  • the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then uses the parallax to obtain Matching point pairs improves the accuracy of target matching between different images.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • the execution subject of the target matching method may be a target matching apparatus.
  • the target matching method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment with requirements such as visual positioning, three-dimensional reconstruction, and image registration.
  • UE User Equipment
  • PDA Personal Digital Assistant
  • handheld devices computing devices, in-vehicle devices, wearable devices and autonomous vehicles, there are Robots with positioning and mapping requirements, medical imaging systems with registration requirements, glasses, helmets and other products for augmented reality or virtual reality.
  • the target matching method may be implemented by the processor invoking computer-readable instructions stored in the memory.
  • the target matching device 30 includes an image acquisition module 31 , a parallax acquisition module 32 , and a matching module 33 .
  • the image acquisition module 31 is used to acquire a first image and a second image, wherein the first image and the second image are obtained by shooting the target to be matched by different shooting components;
  • the parallax acquisition module 32 is used for shooting based on the target and at least one The first distance between the components is used to obtain the parallax of the target between the first image and the second image;
  • the matching module 33 is configured to determine the matching feature point pair about the target in the first image and the second image according to the parallax.
  • the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then utilizes the parallax The matching feature point pairs are obtained, which improves the accuracy of target matching between different images.
  • the matching module 33 determines, according to the parallax, a pair of matching feature points about the target in the first image and the second image, including: determining a reference position corresponding to the first feature point in the second image based on the parallax, wherein , the first feature point is obtained by performing target feature point detection on the first image; based on the reference position, a second feature point matching the first feature point is selected in the second image, wherein the second feature point is the first feature point.
  • the two images are obtained by detecting the target feature points.
  • the second feature point is determined according to the reference position, and the area in the second image that is matched with the first feature point is reduced, which can quickly Find matching point pairs to improve the matching accuracy.
  • the matching module 33 selects a second feature point in the second image that matches the first feature point based on the reference position, including: determining a candidate region including the reference position in the second image, and selecting a candidate region in the candidate region. , a second feature point whose positional relationship with the epipolar line corresponding to the first feature point in the second image satisfies a preset requirement is selected as the second feature point matching the first feature point.
  • the second feature point matching the first feature point is determined by integrating the two dimensions of disparity and epipolar geometric constraint, which improves the accuracy of target matching compared to using a single dimension.
  • the matching module 33 determines the epipolar line corresponding to the first feature point in the second image, including: obtaining the epipolar line by using the coordinates of the first feature point and the fundamental matrix.
  • the preset requirement is that the distance between the second feature point and the epipolar line in the candidate region is the smallest.
  • the second feature point corresponding to the first feature point should be on the epipolar line, but errors inevitably occur during the target matching process, resulting in the second feature point not being on the epipolar line, Therefore, selecting the feature point with the smallest distance from the epipolar line in the candidate region as the second feature point corresponding to the first feature point can improve the success rate of target matching.
  • the first image or the second image is a reference image
  • the photographing component corresponding to the reference image is the reference photographing component
  • the first distance is the distance between the target and the reference photographing component
  • the parallax obtaining module 32 is further configured to execute The first distance is obtained by the following steps: determining the first size of the target corresponding to the target area in the reference image; and obtaining the first distance by using the ratio between the first size and the preset size.
  • the technical effect that the distance between the target and the shooting component can be determined by using a single image is achieved. , which breaks the traditional cognition that it is necessary to combine at least two images to obtain the distance between the target and the shooting component.
  • the first size includes a first width and a first height of the target area
  • the preset size includes a preset width and a preset height
  • the parallax obtaining module 32 obtains by using the ratio between the first size and the preset size.
  • the first distance includes: obtaining a first ratio between the first width and a preset width, and a second ratio between the first height and the preset height; and obtaining the first ratio between the target and the reference shooting component based on the first ratio.
  • two distances, and the third distance between the target and the reference photographing component is obtained based on the second ratio; the first distance is obtained based on the second distance and the third distance.
  • the first distance is obtained by weighting the second distance and the third distance, and the results in the width and height directions can be comprehensively considered. Compared with the second distance or the third distance obtained by using a single direction as For the first distance, the accuracy of the first distance can be improved.
  • the parallax obtaining module 32 obtains the second distance between the target and the reference photographing component based on the first ratio, including: multiplying the first ratio by the first focal length of the reference photographing component in the width direction to obtain the second distance. ; Obtain the third distance between the target and the reference shooting component based on the second ratio, comprising: multiplying the second ratio by the second focal length of the reference shooting component in the height direction to obtain the third distance; Based on the second distance and the third distance Obtaining the first distance includes: weighted summation of the second distance and the third distance to obtain the first distance.
  • the resolution of the reference photographing component in the width or height direction may be different, by multiplying the first ratio by the first focal length of the reference photographing component in the width direction, and multiplying the second ratio by the reference photographing component in the width direction.
  • the second focal length in the height direction is multiplied, and the obtained second and third distances are more accurate.
  • the parallax obtaining module 32 determines the first size of the target corresponding to the target area in the reference image, including: obtaining the first size based on the coordinates of at least two first feature points obtained by performing target feature point detection on the reference image. or, obtaining the first size based on the size of the region obtained by performing target region detection on the reference image.
  • the first size of the target area is obtained according to the coordinates of the first feature point or the size of the area obtained by detecting the target area on the first image, the process is simple, and the computing power required by the device is low.
  • the first distance is the distance between the target and the reference photographing component, and the reference photographing component is the photographing component corresponding to the first image or the second image;
  • the parallax obtaining module 32 is based on the distance between the target and at least one photographing component.
  • the first distance is to obtain the parallax of the target between the first image and the second image, including: based on the first distance, the focal length of the reference shooting component, and the baseline between the shooting components corresponding to the first image and the second image, obtaining Parallax.
  • the above scheme obtains the parallax of the target between the first image and the second image based on the first distance, the focal length of the reference shooting component, and the baseline between the shooting components corresponding to the first image and the second image, and the whole process is simple , the required computing power is low, so that the processing resources of the execution device can be saved.
  • the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then utilizes the parallax The matching feature point pairs are obtained, which improves the accuracy of target matching between different images.
  • FIG. 4 is a schematic structural diagram of an embodiment of an electronic device of the present application.
  • the electronic device 40 includes a memory 41 and a processor 42, and the processor 42 is configured to execute program instructions stored in the memory 41, so as to implement the steps in the above-mentioned embodiments of the target matching method.
  • the electronic device 40 may include, but is not limited to, a microcomputer and a server.
  • the electronic device 40 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
  • the processor 42 is used to control itself and the memory 41 to implement the steps in the above-mentioned embodiment of the target matching method.
  • the processor 42 may also be referred to as a CPU (Central Processing Unit, central processing unit).
  • the processor 42 may be an integrated circuit chip with signal processing capability.
  • the processor 42 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the processor 42 may be jointly implemented by an integrated circuit chip.
  • the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then uses the parallax to obtain Matching point pairs improves the accuracy of target matching between different images.
  • FIG. 5 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application.
  • the computer-readable storage medium 50 stores program instructions 501 that can be executed by a processor, and the program instructions 501 are used to implement the steps in the above embodiments of the target matching method.
  • the present application obtains a more accurate parallax by referring to the distance between the target and the shooting component, and then uses the parallax to obtain Matching point pairs improves the accuracy of target matching between different images.
  • the functions or modules included in the apparatuses provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments.
  • the disclosed method and apparatus may be implemented in other manners.
  • the device implementations described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other divisions.
  • modules or units may be combined or integrated. to another system, or some features can be ignored or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

La présente demande concerne un procédé et un appareil d'appariement de cible, un dispositif, et un support de stockage. Le procédé d'appariement de cible comprend les étapes consistant à : acquérir une première image et une seconde image, la première image et la seconde image étant obtenues par photographie par différents composants de photographie sur une cible à apparier ; obtenir une parallaxe de la cible entre la première image et la seconde image sur la base d'une première distance entre la cible et au moins un composant de photographie ; et en fonction de la parallaxe, déterminer une paire de points caractéristiques d'appariement relativement à la cible dans la première image et la seconde image. Selon la solution ci-dessus, la précision d'appariement de cible entre différentes images peut être améliorée.
PCT/CN2022/084323 2021-04-16 2022-03-31 Procédé et appareil d'appariement de cible, dispositif, et support de stockage WO2022218161A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110413815.2A CN113159161A (zh) 2021-04-16 2021-04-16 目标匹配方法和装置、设备及存储介质
CN202110413815.2 2021-04-16

Publications (1)

Publication Number Publication Date
WO2022218161A1 true WO2022218161A1 (fr) 2022-10-20

Family

ID=76868521

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/084323 WO2022218161A1 (fr) 2021-04-16 2022-03-31 Procédé et appareil d'appariement de cible, dispositif, et support de stockage

Country Status (3)

Country Link
CN (1) CN113159161A (fr)
TW (1) TW202242716A (fr)
WO (1) WO2022218161A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113159161A (zh) * 2021-04-16 2021-07-23 深圳市商汤科技有限公司 目标匹配方法和装置、设备及存储介质
CN113837246B (zh) * 2021-09-06 2022-12-27 广州极飞科技股份有限公司 图像的匹配方法及其匹配装置、无人设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150009295A1 (en) * 2013-07-03 2015-01-08 Electronics And Telecommunications Research Institute Three-dimensional image acquisition apparatus and image processing method using the same
CN108876835A (zh) * 2018-03-28 2018-11-23 北京旷视科技有限公司 深度信息检测方法、装置和系统及存储介质
CN111160178A (zh) * 2019-12-19 2020-05-15 深圳市商汤科技有限公司 图像处理方法及装置、处理器、电子设备及存储介质
CN111739081A (zh) * 2020-08-06 2020-10-02 成都极米科技股份有限公司 特征点匹配方法、拼接方法、装置、电子设备及存储介质
CN113159161A (zh) * 2021-04-16 2021-07-23 深圳市商汤科技有限公司 目标匹配方法和装置、设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6202367B2 (ja) * 2013-05-14 2017-09-27 株式会社リコー 画像処理装置、距離測定装置、移動体機器制御システム、移動体及び画像処理用プログラム
CN110335315B (zh) * 2019-06-27 2021-11-02 Oppo广东移动通信有限公司 一种图像处理方法及装置、计算机可读存储介质
CN111457886B (zh) * 2020-04-01 2022-06-21 北京迈格威科技有限公司 距离确定方法、装置及系统
CN111901499B (zh) * 2020-07-17 2022-04-01 青岛聚好联科技有限公司 一种计算视频图像中像素实际距离的方法及设备
CN112560592A (zh) * 2020-11-30 2021-03-26 深圳市商汤科技有限公司 图像处理方法及装置、终端控制方法及装置
CN112507992B (zh) * 2021-02-04 2021-05-07 腾讯科技（深圳）有限公司 道路图像之间的拍摄距离确定方法、装置、设备及介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150009295A1 (en) * 2013-07-03 2015-01-08 Electronics And Telecommunications Research Institute Three-dimensional image acquisition apparatus and image processing method using the same
CN108876835A (zh) * 2018-03-28 2018-11-23 北京旷视科技有限公司 深度信息检测方法、装置和系统及存储介质
CN111160178A (zh) * 2019-12-19 2020-05-15 深圳市商汤科技有限公司 图像处理方法及装置、处理器、电子设备及存储介质
CN111739081A (zh) * 2020-08-06 2020-10-02 成都极米科技股份有限公司 特征点匹配方法、拼接方法、装置、电子设备及存储介质
CN113159161A (zh) * 2021-04-16 2021-07-23 深圳市商汤科技有限公司 目标匹配方法和装置、设备及存储介质

Also Published As

Publication number Publication date
TW202242716A (zh) 2022-11-01
CN113159161A (zh) 2021-07-23

Similar Documents

Publication Publication Date Title
US9111347B2 (en) Method for determining the pose of a camera with respect to at least one real object
WO2022218161A1 (fr) Procédé et appareil d'appariement de cible, dispositif, et support de stockage
US8600192B2 (en) System and method for finding correspondence between cameras in a three-dimensional vision system
WO2017076106A1 (fr) Procédé et dispositif d'assemblage d'images
US9615081B2 (en) Method and multi-camera portable device for producing stereo images
CN107084680B (zh) 一种基于机器单目视觉的目标深度测量方法
WO2014084181A1 (fr) Dispositif de mesure d'image
CN112184811B (zh) 单目空间结构光系统结构校准方法及装置
CN112802114A (zh) 多视觉传感器融合装置及其方法和电子设备
WO2021193672A1 (fr) Procédé de génération de modèle tridimensionnel et dispositif de génération de modèle tridimensionnel
JP2014060611A (ja) 画像処理装置、映像投影システムおよびプログラム
CN115049738A (zh) 人与相机之间距离的估计方法及系统
CN116929290A (zh) 一种双目视角差立体深度测量方法、系统及存储介质
CN109902695B (zh) 一种面向像对直线特征匹配的线特征矫正与提纯方法
WO2015079054A1 (fr) Estimation de regard à partir de points de mesure oculaire non étalonnés
EP2866446B1 (fr) Procédé et dispositif portable multicaméra pour produire des images stéréo
CN112330726B (zh) 一种图像处理方法及装置
TWI731430B (zh) 資訊顯示方法與資訊顯示系統
JP5727969B2 (ja) 位置推定装置、方法、及びプログラム
CN113034615B (zh) 一种用于多源数据融合的设备标定方法及相关装置
WO2021259287A1 (fr) Procédé de génération de carte de profondeur, dispositif, et support de stockage
CN114663519A (zh) 一种多相机标定方法、装置及相关设备
CN111489384B (zh) 基于互视角的遮挡评估方法及装置、设备、系统和介质
TWI637353B (zh) 測量裝置及測量方法
WO2023179390A1 (fr) Procédé et appareil de reconnaissance d'objet, dispositif électronique, support d'enregistrement lisible par ordinateur et produit programme d'ordinateur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22787388

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE