JP6453908B2 - Method for matching feature points of planar array of 4 cameras and measurement method based thereon - Google Patents

Description

  The present invention relates to the technical field of optoelectronic measurement, and more specifically, a method for matching feature points of a four-camera set planar array, and a measurement method based on a method for matching feature points of a four-camera set planar array It is about.

  At present, three-dimensional stereoscopic vision measurement generally uses a light section measurement technique using a line laser or a combination of binocular measurement and structured light illumination. The main reason why line-type laser light or structured light is widely used in 3D measurement is to clarify the matching point corresponding to the imaging through the indication of line-type laser or structured light, and reduce the ambiguity of matching. This is because deterministic and unique matching is realized. However, if the line laser or structured light indication is canceled, binocular matching will not be able to avoid the ambiguity of matching multiple points and will not be able to meet the measurement requirements. Can be used to measure only the line laser or structured light imaging site, which limits the scope of the technology, and the use of a line laser or structured light can adversely affect the object being measured, e.g. a person. give.

  At present, binocular matching often uses a method of assigning an identification point to the surface of a subject, and the use of such a method is also for improving the accuracy of matching. However, the method of assigning identification points to the surface of the subject has a demerit that artificial measurement and intervention are required for the object to be measured in advance.

  In view of these points, the embodiment of the present invention reduces the complexity of the matching method and the measurement method, simplifies the calculation process of the space size, and reduces the measurement error of the system. An object of the present invention is to provide a feature point matching method and a measurement method based on a feature point matching method of a planar array of four cameras.

In a first embodiment, an embodiment of the present invention provides a method for matching feature points of a planar array of four camera sets, the method comprising:
One of the four image planes corresponding to the four cameras in one four-camera set is set as a base image plane, and a lateral direction with respect to one feature point of the base image plane In step a1, searching for all matching points that match the feature point in an image plane adjacent to the base image plane;
Searching for all matching points that match the feature point in the image plane adjacent to the base image plane in the vertical direction with respect to the feature point of the base image plane in step a1;
Rematching all matching points found in step a1 and all matching points found in step a2 to find all corresponding sub matching point groups (Sub Matching Point Groups);
Based on the feature points of the base image plane and all the sub-matching point sets found in step a3, the base image in the diagonal position image plane that is the diagonal position of the base image plane. A step a4 for finding matching points corresponding to the feature points on the plane and all sub-matching points found in the step a3;
Uniqueness corresponding to the same subject in the four image planes based on the feature points of the base image plane, all sub-matching point sets found in step a3, and matching points found in step a4 And a step a5 for determining a matching point set.

  In combination with the first embodiment, the embodiments of the present invention provide a first possible implementation of the first embodiment. Here, in the step a1, for all feature points of the base image plane, all matching with the feature point in the image plane adjacent to the base image plane in the lateral direction is performed based on the matching condition 1). Searching for a matching point, the matching condition 1) is that an imaging point (Imaging Point) in two image planes adjacent to each other in the lateral direction of one subject is connected to the image plane on the left side of the subject. The image point on the right image plane corresponding to the subject and the image point on the left image plane with respect to the coordinate origin of the left image plane is located on the same straight line parallel to the horizontal coordinate axis. Is satisfied to be larger than the horizontal offset of the imaging point in the right image plane with respect to the coordinate origin of the right image plane.

  In combination with the first embodiment, the embodiments of the present invention provide a second possible implementation of the first embodiment. Here, in the step a2, the feature points on the base image plane in the step a1 are matched with the feature points in the image plane adjacent to the base image plane in the vertical direction based on the matching condition 2). In this case, the matching condition 2) is that an image point on two image planes adjacent to each other in the vertical direction of one object is imaged on an image plane above the object. The imaging point on the upper image plane with respect to the coordinate origin of the upper image plane is located on the same straight line parallel to the longitudinal coordinate axis. Is satisfied to be larger than the vertical offset of the imaging point in the lower image plane with respect to the coordinate origin of the lower image plane.

  In combination with the first embodiment, the embodiments of the present invention provide a third possible implementation of the first embodiment. Here, in the step a3, all matching points found in the step a1 and all matching points found in the step a2 are rematched based on the matching condition 3), and the matching condition 3) Matching points that satisfy the matching condition 3) are paired according to the magnitude relationship of the horizontal offset value or the vertical offset value to form a sub-matching point set, The matching condition 3) is that the matching point set corresponding to the four image planes of one set of four cameras of one subject has a ratio between the horizontal offset value and the vertical offset value that is the length and width of the reference square. Satisfying the condition that the ratio of the horizontal offset value is equal to The imaging point in one left image plane is a horizontal offset with respect to the coordinate origin of the left image plane, and the imaging point in the right image plane corresponding to the subject is a horizontal offset with respect to the coordinate origin of the right image plane. The vertical offset value is the difference between the vertical offset with respect to the coordinate origin of the upper image plane and the corresponding lower image plane of the subject. The imaging point is the difference in vertical offset with respect to the coordinate origin of the lower image plane, where the reference square is a square formed from the four focal points of one 4-camera set.

  In combination with the first embodiment, the embodiment of the present invention provides a fourth possible implementation of the first embodiment. Here, in the step a4, based on the feature points on the base image plane and all the submatching point sets found in the step a3, the matching condition 1) for any submatching point set. The matching point in the diagonal position image plane is searched for on the basis of the matching condition 2) and the coordinates of the matching point located on the diagonal position image plane based on the coordinates of the sub-matching point set found in step a3. The abscissa of the matching point is the same as the abscissa of the matching point in the image plane adjacent to the diagonal position image plane in the vertical direction, and the ordinate is the diagonal position image plane in the horizontal direction. It is the same as the ordinate of the matching point in the adjacent image plane, and after the coordinates are determined, the matching point in the diagonal position image plane, and another 3 If image feature similarity matching is performed on matching points in the image plane, and matching is successful, the four image points in the four image planes form one matching point set, making matching impossible. If it is, the matching point in the diagonal position image plane and the sub-matching point set found in step a3 corresponding to the matching point are excluded.

  In combination with the first embodiment, the embodiment of the present invention provides a fifth possible implementation of the first embodiment. Here, in the step a5, the uniqueness matching point group corresponding to the same subject in the four image planes satisfies the matching condition 4), and the matching condition 4) The imaging points in the four image planes corresponding to the four camera sets form one rectangular shape, and the ratio of the horizontal length and the vertical length of the rectangular shape is the ratio of the length and width of the reference rectangular shape. The same is true, and the imaging points in the two image planes at the two diagonal positions are located on two straight lines parallel to the two diagonal lines of the reference square.

  In combination with the first embodiment, the embodiment of the present invention provides a sixth possible implementation of the first embodiment. Here, in step a5, if there is only one matching point set that satisfies the matching condition 4), the matching result is unique.

In combination with the first embodiment, the embodiment of the present invention provides a seventh possible implementation of the first embodiment. Here, when there are a plurality of matching point sets satisfying the matching condition 4) in the step a5, the base image plane satisfying the following condition on the base image plane, that is, the feature point as one end point, An extension of a diagonal line having the feature point of the base square as one end point passes through the diagonal position image plane, and the length of the diagonal line of any two matching point sets corresponding to the feature point It is the same as the distance between two matching points in the diagonal position image plane, the base square is similar to the reference square, and another end point of the base square has an image feature of the feature point Whether or not there is a base square that satisfies the condition that the feature point is a matching point in the base image plane of the feature point,
If there is such a square in the base image plane, for any base square, whether there is a single corresponding square in the diagonal position image plane that is exactly the same as the base square Confirm
If present, first, a unique matching point corresponding to the feature point is determined in the diagonal position image plane, and then another 2 of the feature point is determined based on the matching condition 1) and the matching condition 2). A method for determining a unique matching point in one image plane and eliminating an unclear set of matching points, wherein a unique matching point corresponding to the feature point in the diagonal position image plane is defined as the base When the feature point of the image plane is one end point of one diagonal of the base square, the only matching point in the diagonal position image plane extends in the only corresponding square in the diagonal position image plane When the line is one of two end points on a diagonal line passing through the feature point, and the feature point is located at the upper end of the base square, the diagonal position image The only matching point corresponding in the plane is the top end point of the only corresponding square, otherwise it is the bottom end point, relative to the non-unique matching point in the diagonal position image plane , It can be determined that the only matching point of the point in the base image plane is another endpoint on the diagonal where the feature point of the base image plane is located,
If not, the two matching point sets corresponding to the feature points correspond to two different views, and the two different views are pixel projections relative to the base image plane of the feature points in space. Located on the extension of the line,
When such a rectangular shape does not exist in the base image plane, the plurality of matching point sets correspond to a plurality of different subjects, and the plurality of different subjects include the base image of the feature point in space. It is located on the extension line of the pixel projection line relative to the plane.

  In combination with the first embodiment, the embodiments of the present invention provide an eighth possible implementation of the first embodiment. Here, the feature point is an imaging point corresponding to one or a plurality of matching points, and the imaging point has an image feature that is distinguished from another imaging point.

In a second embodiment, the present invention provides a measurement method based on a method for matching feature points of a planar array of four camera sets, the method comprising:
After image collection is complete, using the matching method to find a unique matching point set corresponding to all feature points in the base image plane; b1;
A step b2 of calculating a spatial position coordinate of the subject based on the image coordinates of the uniqueness matching point set acquired in the step b1;
A step b3 of forming 3D point cloud data, establishing a 3D point cloud graphic, and reproducing a three-dimensional stereoscopic image based on the spatial position coordinates of the subject obtained in step b2.

In combination with the second embodiment, the embodiment of the present invention provides a first possible implementation of the second embodiment. Here, in the step b2, the focal points of the upper left a camera, the upper right b camera, the lower left c camera, and the lower right d camera of one 4-camera set are respectively O _a , O _b , O _c and O _d , the four focal points are located in the same plane, and form one square, the length O _a O _{b of the} square is m, and the width O _a O _c is n And the center point of the square is set as O, and one three-dimensional rectangular coordinate system is established with O as the origin, where the X-axis is a square O _a O _b and O _c O _d sides The Y axis is parallel to the rectangular O _a O _c and O _b O _d sides, the Z axis is perpendicular to the plane on which the focal point is located, and the optical axis directions of the four cameras The arrangement of the four cameras is exactly the same, and the sky of one subject P of the subject is empty. Position coordinate P (P _{x, P} y, _{P z)} is, a camera point P, is b camera, c cameras, and d the camera image coordinates of the image point in the image plane corresponding to these four cameras The spatial position of the point P when P _a (P _ax , P _ay ), P _b (P _bx , P _by ), P _c (P _cx , P _cy ), and P _d (P _dx , P _dy ), respectively. The expression of coordinates is as follows.

  In the above equation, f is the focal length of the four cameras, u is the length of the target surface of the image sensor, v is the width of the target surface of the image sensor, where Δx is At the time of direction matching, the lateral offset value of the imaging point in the b image plane with respect to the imaging point in the a image plane and the lateral offset of the imaging point in the d image plane with respect to the imaging point in the c image plane are defined, and Δy is defined as At the time of vertical matching, the vertical offset of the imaging point in the c image plane with respect to the imaging point in the a image plane and the vertical offset of the imaging point in the d image plane with respect to the imaging point in the b image plane are defined.

  The present invention has at least the following beneficial effects.

  1.4 The matching method of feature points of a planar array of camera sets is based on the positions of imaging points in four image planes of one 4-camera set of one subject of a subject. The unique imaging point set corresponding to the subject can be quickly matched, and the versatility and uniqueness matching for the subject that can be imaged by any of the four cameras can be realized.

  2.4 The measurement method based on the matching method of the feature points of the planar array of the camera set is that if the collected image is sufficiently clear under any light condition, It is possible to realize a three-dimensional measurement of the subject with exactly the same measurement method, and this measurement method does not require any calibration with respect to the field of view. The resolution is related only to the measurement system, regardless of the subject, and automatic measurement can be realized completely.

  3. Due to the versatility and reliability of the above matching and measurement methods, program optimization is convenient, and it is possible to realize embedded level (chip level) and chip level calculations, and to realize 3D sensing and measurement quickly. .

It is a schematic diagram of the spatial coordinate system established based on one 4 camera group. It is a projection schematic diagram of the image formation point in the image plane of two cameras in the horizontal direction of a subject with a subject and one 4-camera set of the subject. FIG. 3 is a schematic projection view of the subject of FIG. 2 and a corresponding image formation point on an OXZ coordinate plane. It is a three-dimensional schematic diagram in which an indefinite point appears when performing horizontal direction matching on two image planes adjacent to each other in the horizontal direction of one 4-camera set. It is a three-dimensional schematic diagram in which one ideal straight line forms an image on four image planes of one four-camera set. FIG. 6 is a schematic plan view of image forming points in the four image planes of FIG. 5. It is a projection schematic diagram which proves matching uniqueness. It is an exemplary three-dimensional schematic diagram demonstrating matching uniqueness. FIG. 9 is a schematic plan view of image forming points in the four image planes of FIG. 8. FIG. 6 is another exemplary three-dimensional schematic diagram demonstrating matching uniqueness. It is a plane schematic diagram of the image formation point in four image planes of FIG. FIG. 9 is yet another exemplary plan schematic diagram demonstrating matching uniqueness. It is a flowchart of the matching method of the feature point of the planar array of 4 camera sets of the Example of this invention. It is a flowchart of the measuring method based on the matching method of the feature point of the planar array of 4 camera sets of the Example of this invention.

  The above drawings are only illustrative of some embodiments of the present invention and should not be regarded as a limitation on the scope, and those skilled in the art will be able to refer to these drawings or embodiments without creative efforts. Based on this, another related drawing or example can be obtained.

  In order to clarify the purpose, technology and advantages of the embodiments of the present invention, the technology of the embodiments of the present invention will be described clearly and completely in combination with the drawings of the embodiments of the present invention. These are some embodiments of the present invention, but not all embodiments. Accordingly, the detailed description of the embodiments of the invention provided in the following drawings is not intended to limit the scope of the invention to be protected, but merely to illustrate selected embodiments of the invention. is there. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention without creative labor belong to the protection scope of the present invention.

  In the description of the present invention, it should be understood that the terms “origin”, “center”, “vertical direction”, “lateral direction”, “length”, “width”, “depth”, “top”, Orientation indicated by “down”, “front”, “back”, “left side”, “right side”, “upper left”, “lower left”, “upper right”, “lower right”, “vertical”, “horizontal”, etc. The positional relationship is based on the orientation or positional relationship shown in the drawings, and is merely for convenience of explanation and simplicity of explanation for the present invention, and the indicated device or element always has a specific orientation. It should not be understood as a limitation to the present invention, as it is configured in a particular orientation and does not indicate or imply that it should be operated.

  In the present invention, the image formation point is an image formation in which the object point of the object corresponds to the pixel position on the image plane (or image), and each object point of the object is four images of one set of four cameras. There is one corresponding image point in the image plane. A matching point satisfies any of the matching conditions with the image point found on the image plane or another image plane with respect to the image point on one image plane, and the image feature ( For example, a pattern, color, or gray level) is an approximate image point. There may be one or more matching points corresponding to one imaging point.

  In the present invention, the feature points are the following imaging points corresponding to one or more matching points. The image point has an image feature that is distinguished from another image point, such as a different pattern, color, or gray level, but is not specifically limited in the embodiments of the present invention, and depends on the actual situation. Different image features can be selected as the basis for determination. In general, any imaging point corresponding to a viewpoint at a position such as a peripheral edge of the object to be measured or a transient region of a pattern has a clear image feature.

  In the present invention, feature point matching, matching operation and calculation rules compare and analyze imaging points at two or more positions to give a similarity measure of image features at two or more imaging points. If the comparison result reaches the planned value of the index, matching is successful, and a similarity index of different image features can be selected according to the actual situation. The matching conditions 1), 2), 3), 4), and the uniqueness matching method submitted in the present invention are based on the current image processing method, in addition to matching on the similarity of image features. Based on the geometric positional relationship between the imaging points in the set of four image planes, matching according to the matching condition submitted in the present invention, the image features are similar, but the positional relationship is incorrect. Verify and eliminate matching ambiguities and ensure uniqueness of matching results.

1.4 Establishing a three-dimensional coordinate system of a before image model of a planar array of camera sets and proof of matching conditions
(1) Establishment of a three-dimensional measurement system and a three-dimensional coordinate system for a four-camera plane array For convenience of explanation of a matching method and a measurement method for feature points of a four-camera plane array according to an embodiment of the present invention. It is necessary to establish a three-dimensional measurement system for a planar array of one four-camera set that satisfies the requirements, and a three-dimensional coordinate system corresponding to the three-dimensional measurement system for the four-array set of four cameras. In order to more conveniently apply the method of the example, a forward projection model is used, a three-dimensional measurement system of a four-camera planar array established in the following manner, and a corresponding three-dimensional coordinate system. .

FIG. 1 is a schematic diagram of a spatial coordinate system established based on one 4-camera set, where one 4-camera set is arranged in the form of a 2 × 2 array, a camera located at the upper left, and located at the upper right The four arrangements of the camera b, the camera c located in the lower left, and the camera d located in the lower right include the same camera. The same arrangement means that the parameters of the four cameras such as the lens, image sensor, and focal length are completely different. The same arrangement. The focal points O _a , O _b , O _c , and O _d of the four cameras are located in the same plane, and the optical axes of the four cameras are all perpendicular to the plane, and O _a , O _b , and O _c. , O _d form a rectangular shape having a length m and a width n, and the central point of the rectangular shape is set to O, and one three-dimensional rectangular coordinate system is established with O as the origin. Here, the X axis is parallel to the rectangular O _a O _b and O _c O _d sides, and the direction of the X axis is referred to as a length direction, a horizontal direction, or a horizontal direction, and the X axis is a horizontal direction. Extension (the direction indicated by the arrow on the X axis faces the right side and is defined as a positive direction here) is possible. The Y axis is parallel to the rectangular O _a O _c and O _b O _d sides, and the direction of the Y axis is referred to as the width direction, the vertical direction, or the vertical direction, and the Y axis extends in the vertical direction (Y axis The direction pointed to by the arrow points upward and is defined as a positive direction here). The Z axis is perpendicular to the plane on which the focal point is located and is parallel to the optical axis direction of the four cameras, and the direction of the Z axis is referred to as the depth direction (the direction indicated by the arrow on the Z axis is the depth direction). Defined as positive direction). In the positive direction of the Z-axis, a plane that is parallel to the plane on which the focal points O _a , O _b , O _c , and O _d are located and whose distance from the plane is the focal length f is set. , B, c, and d are set, and according to the imaging principle, the center of each image plane is the passing point of the optical axis of the corresponding camera, and the origin of the plane coordinates of each image plane is Set to the upper left corner of the image plane, O _a ′, O _b ′, O _c ′, and O _d ′, respectively. The two-dimensional rectangular coordinate axis of the a image plane is set as O _a 'X _a Y _a , the two-dimensional rectangular coordinate axis of the b image plane is set as O _b ' X _b Y _b, and the two-dimensional rectangular coordinate axis of the c image plane is set as O 'is set to X _c Y _c, the 2-dimensional perpendicular coordinate axes of d image plane O _{d' c} is set to X _d Y _d, corresponding to the length of each length of the image plane (the target surface of the image sensor of the camera ) Is set to u, and the width (corresponding to the width of the target surface of the image sensor of each camera) is set to v (not shown).

  One four-camera set (ie, a four-camera planar array) using the above arrangement proposed in the present invention is one smallest basic three-dimensional stereo measurement unit, with similar rules and methods. Accordingly, it is possible to build a three-dimensional measurement system for a planar array of multi-camera sets consisting of more cameras (2N, where N is a positive integer greater than or equal to 2), with similar rules and methods Based on this, a three-dimensional coordinate system corresponding to each set of four cameras can be constructed, or a unified three-dimensional coordinate system can be constructed. The matching method and the measuring method of the embodiment of the present invention are used for matching and measuring the three-dimensional coordinate position of the viewpoint of the object to be measured that can be imaged by four cameras of one four-camera set arranged as described above. Applied.

  For convenience of explanation, a rectangular shape formed from four focal points of one four-camera set is referred to as a reference rectangular shape. At the same time, the a and d image planes are mutually diagonal position image planes, and the b and c image planes are mutually diagonal position image planes.

(2) Lateral matching and vertical matching of feature points of a four-camera plane array, and calculation of spatial position coordinates of feature points FIG. 2 shows one view point P of a subject and an a image of the point. FIG. 2 is a schematic diagram of projection of image formation points on the plane and the b image plane. Referring to FIG. 2, the image formation points of the P point a and b image planes are P _a and P _b , respectively, according to the image formation principle. point P, _{P a} point, and two focus _O a, _{O b} is the position of _{P b} point, and projection in a plane parallel to OXZ coordinate plane, respectively P 'point, _{P a'} point, the _{P b} 'point is there.

The O _a O _b straight line is parallel to the image plane consisting of two image planes a and _b , and a triangle formed by three points P, O _a and O _b is the two image planes a and b. Intersects the plane on which is located and the intersecting line is a straight line P _a P _b , the P _a P _b straight line and the O _a O _b straight line are parallel to each other.

3, P point of Fig. 2 is a projection schematic view in OXZ coordinate plane _{P a} point, and _{P b} points.

Referring to FIG. 3, m is the length of O _a O _b , u is the length of the target surface of each image sensor, and the points P ′, P _a ′, and P _b ′ are P points respectively, and the projection points in OXZ coordinate plane _{P a} point, and _{P b point P ax} and _{P bx} are, X-axis in a image plane and b image plane _{P a} 'point and _{P b'} point, respectively The coordinate value of the direction.

Apparently, P _ax is greater than P _bx . That is, the horizontal offset of the image point in the a image plane of the point P with respect to the coordinate origin of the a image plane is larger than the horizontal offset of the image point in the b image plane of the point with respect to the coordinate origin of the b image plane.

  Therefore, the following matching conditions are obtained.

  Matching condition 1): When one subject forms an image on four image planes of a set of four cameras, the image point on two image planes adjacent to each other in the lateral direction is the image plane on the left side of the subject And the image point on the right image plane corresponding to the subject is located on the same straight line parallel to the coordinate axis in the horizontal direction, and on the left image plane relative to the coordinate origin of the left image plane. The condition that the horizontal offset of the imaging point is greater than the horizontal offset of the imaging point in the right image plane relative to the coordinate origin of the right image plane is satisfied.

  Matching condition 1) applies to lateral matching of the a and b image planes, and similarly applies to lateral matching of the c and d image planes. Two image planes that are adjacent to each other in the horizontal direction in one set of four cameras are a corresponding left-side plane and right-side plane. That is, the right image plane corresponding to the left image plane a is b, and the right image plane corresponding to the left image plane c is d.

  Similarly, the following matching conditions are obtained.

  Matching condition 2): When one subject forms an image on four image planes of one four-camera set, an image point on two image planes adjacent to each other in the vertical direction is an image plane above the subject And the image point on the lower image plane corresponding to the subject is located on the same straight line parallel to the longitudinal coordinate axis, and on the upper image plane relative to the coordinate origin of the upper image plane. The condition that the vertical offset of the imaging point is greater than the vertical offset of the imaging point in the lower image plane relative to the coordinate origin of the lower image plane is satisfied.

  Matching condition 2) applies to vertical matching of the a and c image planes, and similarly applies to vertical matching of the b and d image planes. Two image planes adjacent to each other in the vertical direction in one set of four cameras are the corresponding upper and lower planes. That is, the lower image plane corresponding to the upper image plane a is c, and the lower image plane corresponding to the upper image plane b is d.

  Based on the above and similar principles of triangles, the following equations are obtained for FIG.

(P _ax -P _bx ) is defined as a lateral offset value of the imaging point on the b image plane with respect to the imaging point on the a image plane at the time of the horizontal matching, and is defined as Δx. Then, the following formula is obtained.

  Similarly, the coordinate calculation formula at the time of the horizontal matching of the c and d image planes can be estimated as follows.

That is, (P _ax −P _bx ) = (P _cx −P _dx ) = Δx, and (P _cx −P _dx ) is an image in the d image plane with respect to an image forming point in the c image plane at the time of horizontal matching. The horizontal offset value of the point.

As in the case of the horizontal matching, a coordinate calculation formula at the time of the vertical matching can be inferred, and (P _ay −P _cy ) is imaged on the c image plane with respect to the image forming point on the a image plane at the time of vertical matching. It is defined as the vertical offset value of the point and is defined as Δy.

That is, (P _ay -P _cy ) = (P _by -P _dy ) = Δy, and (P _by -P _dy ) is an image formed on the d image plane with respect to an image forming point on the b image plane during vertical matching. This is the vertical offset value of the point.

(3) Further derivation of the horizontal matching and vertical matching of the feature points of the four-camera plane array Based on the calculation formula 2, the calculation formula 4, the calculation formula 6, and the calculation formula 8 for the same viewpoint Thus, the following equation can be estimated.

  Furthermore, the following calculation formula (calculation formula 9) can be obtained.

From this, the following matching conditions can be obtained.
Matching condition 3): Matching point pairs corresponding to four image planes of one four-camera set of one subject are such that the ratio of the horizontal offset value and the vertical offset value is the ratio of the length and width of the reference square. Where the horizontal offset value is the horizontal offset relative to the coordinate origin of the left image plane relative to the coordinate origin of the left image plane. The imaging point on the right image plane corresponding to the subject is the difference in horizontal offset from the coordinate origin of the right image plane, and the vertical offset value is the imaging point on one upper image plane of the subject Is the vertical offset with respect to the coordinate origin of the upper image plane and the imaging point on the lower image plane corresponding to the subject is the vertical offset with respect to the coordinate origin of the lower image plane Here, the matching point set is four imaging points that satisfy the matching condition 3) in the four image planes and have similar image characteristics, and the sub-matching point set is particularly based on This is an imaging point that satisfies the matching condition 3) in two image planes adjacent to the image plane.

2.4 Uniqueness Proof of Matching Method for Feature Point of Plane Array of 4 Cameras Set Uniqueness matching for feature point can be realized by using matching method of planar array of 4 camera set, and the matching method is good It has the universality and applicability and proves as follows.

First, FIG. 4 shows a three-dimensional schematic diagram in which an indefinite point appears when performing lateral matching on the a and b image planes. Referring to FIG. 4, point P represents one object to be measured in space. The viewpoint is a coordinate, P (P _x , P _y , P _z ), and imaging points (Imaging Points) on the a and b image planes of the subject are P _a and P _b , respectively. Time matching calculation, the reference point P _a point (feature point), look for imaging point matching to the point in b image plane. After P _a point is determined, based on the matching condition 1), the imaging point matching the P _a point in b image plane passes through the P _a point, and in one straight line parallel to the X axis position, and horizontal coordinate values of b image plane of the imaging points of the matching is smaller than the horizontal coordinate value of a picture plane P _a point. For P _a point, referred to it as soon as the position in a picture plane is once determined, the viewpoint corresponding to P _{a point,} and _O a P _a is located ray (or pixel projection line, i.e., one camera The matching between the a and b image planes belongs to binocular matching, and currently known binocular matching calculation rules , calculation methods, and according to the constraint condition, P when the image features of _a point is given, the imaging point of matching in b image plane of the points to be determined on the basis of this, generally, one In addition, since there is a possibility that there are a plurality of them, a problem occurs in the uniqueness of binocular matching. Currently, such to be solved problem, among the plurality of matching points that satisfy the matching condition, eliminating one or more points ambiguity exists, imaging of only matching and P _a point in b image plane it is to determine the point P _b.

For this, assuming that the different image points P b _'point is matched with the P _a point in b image plane, the matching condition 1), the point and P _a P _b are same straight line in set position, and rays and O _a P _a to be determined by the connecting line of the point and O _b is located rays are positioned in the same plane, and the two rays intersect each other, an intersection point between P 'point To do. Therefore, an imaging point P b _'point P' in the b image plane of the point can be the point also considered to correspond to the P _a point in a picture plane.

FIG. 5 is a three-dimensional schematic diagram in which one ideal straight line PP ′ forms an image on four image planes, and FIG. 6 is a schematic plan view of image forming points on the four image planes in FIG. Referring to FIGS. 5 and 6, the image points on the c image plane and the d image plane at the points P and P ′ can be seen. Here, the imaging points Pa and Pa ′ in the a image plane of the point P and the point P ′ overlap each other. P _a point, P _b point, P _c point, and P _d point (P _a , P _b , P _c , and P _d ) are one matching point set corresponding to P point and correspond to P point Uniqueness matching point set. The points P _a ′, P _b ′, P _c ′, and P _d ′ (P _a ′, P _b ′, P _c ′, and P _d ′) are one matching point corresponding to the point P ′. It is a uniqueness matching point set corresponding to the P ′ point.

The uniqueness of the matching algorithm for the four-camera plane array is to search for an imaging point that uniquely matches the b, c, d image plane with Pa as _a reference point according to the matching condition. First, looking imaging point matching in b image plane P _b, and the other arbitrarily assumed matched to imaging point P _{b ',} if the two image forming point is the matching point of both P _a, the image By the imaging principle, the imaging points shown in FIG. 5 are obtained in the a, b, c, and d image planes. Further, P _a (P _a ′), P _b P _b ′, P _c P _c ′, and P _d P _d ′ in FIG. 6 are projected images on four image planes of the PP ′ straight line, where P _It can also be inferred that _a and P _a ′ overlap each other.

If there is one ideal straight line that is the same as or similar to PP ′ in FIG. 5, the imaging points in the four image planes of one four-camera set of the straight line are the same as in FIG. 5 and FIG. 6. As shown in PP 'is only one example, actually, PP' all imaging point in a picture plane of the viewpoint during each a P _a point. In other words, for such a situation, if the base image plane a image plane, P _a point, but correspond to a plurality of pairs of matching points, each set of matching points, corresponding to only the viewpoint To do.

  Based on FIG. 6, the following calculation formula 9:

  In combination, the following matching conditions can be obtained.

  Matching condition 4): The imaging points on the four image planes of one four-camera set of one subject form one square, and the ratio of the length in the horizontal direction to the length in the vertical direction of the square Is the same as the ratio of the length and width of the reference square, and the imaging points on the two image planes at the two diagonal positions are on two straight lines parallel to the two diagonals of the reference square, respectively. To position.

  Based on the above contents, if a straight line similar to PP 'as shown above appears on the object, one point matches multiple points when binocular matching is performed according to the a and b image planes. If this occurs, the uniqueness matching requirement cannot be satisfied, and matching uncertainty will occur. However, if the c and d image planes are used as a reference for matching the a and b image planes, and the PP ′ straight line is a single inclined line in the d image plane, each image plane of P ′ in the four image planes. Uniqueness matching points in, and uniqueness matching point pairs corresponding to each point on the PP ′ line, and further calculating the spatial position coordinates of the points on the PP ′ line based on the above coordinate arithmetic expression Can do. This is similar to the need for three views in mechanical drawing, and in the case of two perspectives, the ambiguity of understanding of the object appears, but in the case of three perspective views, Such a situation does not occur.

  To further prove the uniqueness of matching, consider the following situation.

First situation: P _b ′ point is another point where the image feature in the b image plane matches the image feature of point P _a with each other, and P _b ′ point is Assuming that it is not a projected imaging point in the b image plane, but any other one that can generate a P _b 'projected imaging point in the b image plane in space, it is set as the P _n point. FIG. 7 is a schematic diagram of projection that proves matching uniqueness. Referring to FIG. 7, the image forming points on the b image plane of the points P ′, P _n and P ″ overlap each other. The solid model projected on the four image planes of the point P _n by the image forming principle. The figure is as shown in Fig. 8. If the projection points of the P _n points on the a, b, c, and d image planes are set as P _na , P _nb , P _nc , and P _nd , respectively, Indicates image forming points in the a, b, c, and d image planes of the P point and the P _n point.

The matching method of feature points 4 camera set of planar arrays, as can be seen from Figure 9, P _nc point and P _a point, because it does not lie on a single longitudinal straight, does not satisfy the matching condition 2) , P _nd and P _a points do not satisfy the matching condition 4) because they are not positioned on one straight line parallel to the diagonal of the rectangular quadrilateral formed from the focus of one 4-camera set. Therefore, in binocular matching, imaging points P _b ′ and P _nb in which matching is unclear appear, but another one arbitrarily assumed matching is performed by the matching method of the feature points of the four-camera plane array. It can be determined that the point P _nb is not an imaging point that matches the imaging point on the a image plane of the P point. In such a case, the ambiguity that often appears in binocular matching can be efficiently eliminated by using a method of matching feature points of a planar array of four cameras.

Second situation: the point P _b ′ is not a projection of the P ′ point on the b image plane, but specifically a projection of any point P ″ on the b image plane on the straight line PP ″ parallel to the O _a O _b line. Is set as P _b ″. FIG. 10 shows a three-dimensional schematic diagram in which the P ″ point is projected onto four image planes, and FIG. 11 shows a, b, Each image point in the c and d image planes is shown, where the P _b ″ point and the P _n point overlap each other.

The 4 camera sets the matching method of the feature points of the planar array, as can be seen from FIG. 11, P c _"point and P _a point, because it does not lie on a single longitudinal straight, not satisfy the matching condition 2) not, P d _"point and P _a point, because not located on one straight line parallel to the diagonal of the rectangular shape which is formed from one of the four camera sets the focus, not satisfied matching condition 4). Therefore, the imaging point in the b image plane of any one other point P ″ on the straight line PP ″ parallel to the O _a O _b straight line matches the imaging point in the a image plane of the P point. It can be determined that this is not the case.

  However, if there is one plane parallel to a rectangular shape formed from the focus of one 4-camera set, and there are four viewpoints P, P1, P2, P3 whose image features are approximated to the plane. As shown in FIG. 12, assuming that the four viewpoints are arranged like a square that approximates the size of the square, if a matching point corresponding to the point P is found, P3 The points also satisfy the matching condition of the feature points of the planar array of 4 camera sets, which is a further special case considered in the second situation.

That is, the matching point sets corresponding to the viewpoints P, P1, P2, and P3 are actually (P _a , P _b , P _c , and P _d ), (P _a1 , P _b1 , P _c1 , and P _c , respectively). _d1 ), (P _a2 , P _b2 , P _c2 , and P _d2 ), and (P _a3 , P _b3 , P _c3 , and P _d3 ), but matching to the P point due to the presence of the P3 point In addition to excluding (P _a , P _b , P _c , and P _d ) from among a plurality of matching point groups found by the method, one unclear matching point group (P _a , P _b1 , P _c2 and P _d3 ) are further present.

If such a situation appears, four points P _a , P _a1 , P _a2 , and P _a3 corresponding to the viewpoints P, P1, P2, and P3 in the a image plane can be searched, where , (P _a , P _a1 , P _a2 , and P _a3 ), (P _b , P _b1 , P _b2 , and P _b3 ), (P _c , P _c1 , P _c2 , and P _c3 ), and (P _d , P _d1 , P _d2 , and P _d3 ) each form a rectangular shape that is similar to the rectangular shape formed from the focus of one 4-camera set.

One unclear matching point set can be searched for by the following method. That is, for the situation described above, based on the matching condition with respect to the imaging point _{P a} at the base image plane, a plurality of matching point pairs _{(P a,} P b, _{P c,} and _{P d)} and (P _{a, P b1, P c2,} and the _{P d3)} Find the _{P a} point as the upper left end point in a image plane, and reference exists rectangular shape rectangular shape similar to the _P a _P a1 _P a2 _{P a3} Furthermore, if it is determined that there exists a square quadrilateral P _a3 P _b3 P _c3 P _d3 that is exactly the same as the square quadrilateral P _a P _a1 P _a2 P _a3 in the d image plane, the only matching point corresponding to P _a Can be determined to be P _d and the only matching point corresponding to P _a3 is P _d3 , and unclear matching point sets can be eliminated.

  Similarly, for situations where the b, c, or d image plane is the base image plane, a similar reasoning scheme can be used to eliminate one or more unambiguous matching point sets that have been found. .

  In an embodiment of the present invention, a spatial position coordinate of a subject is obtained by matching a subject that can be imaged on four image planes of one set of four cameras according to matching conditions, and a 3D point cloud is obtained. Data can be formed, 3D point cloud graphics can be established, and 3D solid reproduction can be performed on the object. Due to the uniqueness characteristics of the four-camera group matching, the spatial position coordinates of the subject are unique, and there is no one-to-many or many-to-one ambiguity.

3.4 Method for Matching and Measuring Feature Point of Planar Array of Camera Set Based on the above matching principle, matching condition, matching calculation formula, and matching method, in one embodiment, referring to FIG. The method for matching feature points of a set of planar arrays includes the following steps.

  a1.1 Any one of four image planes corresponding to four cameras of a set of four cameras is set as a base image plane, and the feature image is laterally applied to one feature point of the base image plane. Find all matching points that match the feature point in the image plane adjacent to the base image plane.

  In step a1, with respect to one feature point on the base image plane, all matching points that match the feature point in the image plane adjacent to the base image plane in the lateral direction are searched based on the matching condition 1). .

  a2. With respect to the feature points on the base image plane in step a1, all matching points that match the feature points in the image plane adjacent to the base image plane in the vertical direction are searched.

  In step a2, all matchings that match the feature points in the image plane adjacent to the base image plane in the vertical direction based on the matching condition 2) with respect to the feature points of the base image plane in step a1. Find a point.

  a3. All matching points found in step a1 are rematched with all matching points found in step a2, and all corresponding sub-matching point sets are searched.

  In step a3, all matching points found in step a1 and all matching points found in step a2 are rematched based on the matching condition 3), and the matching does not satisfy the matching condition 3) The matching points satisfying the matching condition 3) are paired according to the magnitude relationship of the horizontal direction offset value or the vertical direction offset value to eliminate the points and form a sub-matching point set.

  a4. Based on the feature points of the base image plane and all the sub-matching point sets found in step a3, the base image in the diagonal position image plane, which is the image plane at the diagonal position of the base image plane, Search is made for the matching points corresponding to the feature points on the plane and all the sub-matching point sets found in step a3.

  In step a4, based on the feature points on the base image plane and all the submatching point sets found in step a3, the matching condition 1) and the matching condition are set for any submatching point set. 2), the matching point in the diagonal position image plane is searched for, the coordinates of the matching point in the diagonal position image plane are obtained based on the coordinates of the sub-matching point set found in step a3, and the matching point Is the same as the abscissa of the matching point in the image plane adjacent to the diagonal position image plane in the vertical direction, and the ordinate is the matching in the image plane adjacent to the diagonal position image plane in the horizontal direction. Is the same as the ordinate of the point, and after the coordinates are established, the matching point in the diagonal position image plane and the other three image planes When image feature similarity matching is performed on matching points and matching is successful, the four image points on the four image planes form one matching point set, and matching is impossible The matching points in the diagonal position image plane and the matching point set found in step a3 corresponding to the matching points are excluded.

  a5. Uniqueness corresponding to the same subject in the four image planes based on the feature points of the base image plane, all sub-matching point sets found in step a3, and matching points found in step a4 Determine the matching point set.

  In step a5, the unique matching point set corresponding to the same subject in the four image planes satisfies the matching condition 4), and the matching condition 4) is applied to one four camera set of one subject. The imaging points in the corresponding four image planes form one rectangular shape, and the ratio of the horizontal length and the vertical length of the rectangular shape is the same as the ratio of the length and width of the reference rectangular shape, That is, the image forming points in the two image planes at the two diagonal positions are located on two straight lines parallel to the two diagonal lines of the reference square.

  In step a5, if there is only one matching point set that satisfies the matching condition 4), the matching result is unique.

  When there are a plurality of matching point sets satisfying the matching condition 4) in step a5, the base rectangular shape satisfying the following condition on the base image plane, that is, the feature point as one end point, and the base rectangular shape The diagonal line of the two matching point pairs in which the extension line of the diagonal line having the feature point of one of the end points passes through the diagonal position image plane and the length of the diagonal line corresponds to the feature point It is the same as the distance between two matching points in the image plane, the base square is approximated to the reference square, and another end point of the base square is approximated by the image feature of the feature point. Whether or not there is a base square that satisfies the condition that the feature point is a matching point on the base image plane.

  If there is such a square in the base image plane, whether or not there is a single corresponding square in the diagonal position image plane exactly the same as the base square for any of the base squares Confirm.

  In an embodiment of the present invention, a square corresponding to the only square exactly the same as any of the base squares in the diagonal position image plane means that the square is exactly the same as the base square and the four squares of the square The four end points corresponding to the end points and the base rectangular shape satisfy the matching conditions and are the only corresponding rectangular shapes having similar image characteristics. In combination with various conventional matching calculation methods or rules, It is possible to determine whether or not there is a single corresponding square in the angular position image plane corresponding to any of the base squares, and the related matching calculation method or rule does not belong to the content of the invention of the present invention. . Thus, without being described in detail here, one skilled in the art could combine various different schemes to find the only corresponding square.

  If present, first, a unique matching point corresponding to the feature point in the diagonal position image plane is determined, and then another 2 of the feature point is determined based on the matching condition 1) and the matching condition 2). Determine the only matching points in one image plane and eliminate the unclear set of matching points. Here, a method of determining a unique matching point corresponding to the feature point on the diagonal position image plane is as follows. When the feature point of the base image plane is one end point of one diagonal line of the base square, the only matching point in the diagonal position image plane is the only corresponding square in the diagonal position image plane Is one of the two end points on the diagonal passing through the feature point, and the feature point is located at the upper end of the base square, the only corresponding point in the diagonal position image plane The matching point is the end point at the upper end of the only corresponding quadrilateral, otherwise it is the end point at the lower end. For a non-unique matching point in the diagonal position image plane, the only matching point of the point in the base image plane is another end point on the diagonal where the feature point in the base image plane is located. Can be determined.

  If not, the two matching point pairs corresponding to the feature points correspond to two different subjects, and the two different subjects are pixel projections where the feature points are relative to the base image plane in space. Located on the line extension.

  When such a rectangular shape does not exist in the base image plane, the plurality of matching point sets correspond to a plurality of different subjects, and the feature points have the feature points in the base image plane in space. It is located on the extension line of the pixel projection line opposite to.

  One example will be described in detail below.

The following is a brief description with reference to FIG. Assuming that the a image plane is the base image plane, for one feature point Pa in the _a image plane, find all matching points, for example, P _b and P _b1 , that match the point in the b image plane, the feature point _{P a,} all of the matching points to match to the point in the c image plane, for example _{P c,} locate the _{P c2,} further _{P b, P b1} and _{P c,} and re-matching the _{P c2,} sub Matching point sets (P _b , P _c ) and (P _b1 , P _c2 ) are determined. _{Then, (P b, P c)} P corresponding to _d, and the coordinates of _{P d, (P b1, P} c2) corresponding to the _{P d3,} and the coordinates of _{P d3} respectively determined through determination, _{(P a} , P _b , P _c , P _d ) are similar in image characteristics, and the four imaging points of (P _a , P _b1 , P _c2 , P _d3 ) When the image feature is also determined to be similar, _{P a} point, two matching point pairs _{(P a, P b, P} c, P d) and _{(P a, P b1, P} c2, P d3) Corresponding to Subsequently, to confirm the number a image plane based rectangular shape _P a _P a1 _P a2 _{P a3} is present, the rectangular shape is a _{P a} as one end point, an extension of a diagonal line _P a _{P a3} is a d image plane Passes and the length of P _a P _a3 is the same as the distance of P _d P _d3 and is similar to the reference square, and the image features of P _a , P _a1 , P _a2 , and P _a3 are similar If the condition is satisfied, it is further determined that there is a rectangular quadrilateral P _d P _d1 P _d2 P _{d3 that} is the same and corresponding to the rectangular quadrangle exactly in the d image plane. Here, P _a and P _d , P _a1 and P _d1 , P _a2 and P _d2 , P _a3 and P _d3 are matching points corresponding to each other, and the image features are similar, the d image plane The only matching point corresponding to P _a is P _d , and the non-unique matching point P _d3 in the d image plane can be determined to correspond to the feature point P _a3 in the a image plane, and matching is performed based on this. The point set (P _a , P _b1 , P _c2 , P _d3 ) is excluded, and (P _a , P _b , P _c , P _d ) is determined to be a unique matching point set corresponding to the same subject. .

  In another embodiment, referring to FIG. 14, a measurement method based on a method for matching feature points of a planar array of four camera sets includes the following steps.

  b1. After the image collection is completed, the unique matching point set corresponding to all the feature points in the base image plane is searched using the matching method.

  b2. Based on the image coordinates of the uniqueness matching point set obtained in step b1, the spatial position coordinates of the subject are calculated.

In step b2, the focus of the upper left a camera, the upper right b camera, the lower left c camera, and the lower right d camera of one 4-camera set is respectively O _a , O _b , O _c , and O _d , the four focal points are located on the same plane and form one rectangular shape, the length O _a O _{b of the} rectangular shape is m, the width O _a O _c is n, The center point of the rectangle is set as O, and one three-dimensional rectangular coordinate system is established with O as the origin. Here, the X axis is parallel to the rectangular O _a O _b and O _c O _d sides, the Y axis is parallel to the rectangular O _a O _c and O _b O _d sides, and Z The axis is perpendicular to the plane on which the focal point is located, and is parallel to the optical axis direction of the four cameras. The arrangement of the four cameras is exactly the same, and the space of one subject P of the subject The position coordinates are P (P _x , P _y , P _z ), and the image coordinates of the corresponding image forming points on the image planes of the four cameras a camera, b camera, c camera, and d camera. When P _a (P _ax , P _ay ), P _b (P _bx , P _by ), P _c (P _cx , P _cy ), P _d (P _dx , P _dy ), respectively, the spatial position coordinates of the P point The expression of is as follows.

  In the above equation, f is the focal length of the four cameras, u is the length of the target surface of the image sensor, v is the width of the target surface of the image sensor, where Δx is At the time of direction matching, the lateral offset value of the imaging point in the b image plane with respect to the imaging point in the a image plane and the lateral offset of the imaging point in the d image plane with respect to the imaging point in the c image plane are defined, and Δy is defined as At the time of vertical matching, the vertical offset of the imaging point in the c image plane with respect to the imaging point in the a image plane and the vertical offset of the imaging point in the d image plane with respect to the imaging point in the b image plane are defined.

  b3. Based on the spatial position coordinates of the subject obtained in step b2, 3D point cloud data is formed, a 3D point cloud figure is established, and a three-dimensional stereoscopic image is reproduced.

  The matching method of the present invention and the three-dimensional measurement system constructed based on the matching method belong to a voluntary matching method and a measurement system. The matching condition and method do not change due to a change in the external environment, and the parameters of the measurement system are temporarily changed. Once confirmed, the field of view range measured by the system, measurement accuracy, spatial resolution, adaptability to the environment, etc. can be determined, for example, long distance objects can be measured by changing the focal length When the camera is changed, for example, when an infrared camera is used, nighttime measurement can be realized, and when a microscope lens is used, microscopic three-dimensional measurement and the like can be realized.

  The above method and steps can realize the calculation and measurement of the three-dimensional coordinate position of the feature point of the object, but the image plane is affected by the field of view range, resolution, environment, and light conditions. , Adjustment of shooting parameters such as shutter, exposure time, geometrical features of the subject, peripheral features, surface reflection and pattern features affect the matching of imaging points, and there is always a situation where perfect matching is not possible, Since the image features include other constraints such as continuity, any of the known perimeters after the resolution of the image perimeter, and another feature clear observation point with the method according to embodiments of the present invention By performing image analysis and processing on another part that cannot be completely matched using image processing methods and means, the imaging problem of most three-dimensional objects can be solved. The processing techniques as group, provides the basic matching method and measurement method to the three-dimensional vision measuring technique.

  It should be understood that in some embodiments according to the present invention, the provided method can be implemented in other ways. It should also be noted that in some replacement implementation schemes, the method steps or flows may occur out of the order shown in the claims or drawings. For example, two consecutive steps or flows can actually be performed almost in parallel, or can be performed in the opposite order, as determined by the specific circumstances.

  When the above method or function of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can be recorded on a computer-readable recording medium. Based on this understanding, the essence of the technical method of the present invention, the part that contributed to the prior art, or the part of the technical method can be embodied in the form of a software product, and the computer software product is stored in one recording medium. And including several commands, one computer device (such as a personal computer, server, or network device) is allowed to execute all or some of the steps of the above method of each embodiment of the present invention. The recording medium includes various media capable of recording program codes such as a USB memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic tape disk, or a compact disk.

  The above is a preferred embodiment of the present invention, and is not intended to limit the present invention. Various modifications and changes can be made by those skilled in the art. Any modifications, equivalent replacements, changes, etc. made in accordance with the spirit and principle of the present invention belong to the protection scope of the present invention.

Claims (11)

One of the four image planes corresponding to the four cameras in one set of four cameras is set as a base image plane, and the base in the lateral direction with respect to one feature point of the base image plane. Searching for all matching points that match the feature point in an image plane adjacent to the image plane; and
Searching for all matching points that match the feature point in the image plane adjacent to the base image plane in the vertical direction with respect to the feature point of the base image plane in step a1;
Rematching all matching points found in step a1 with all matching points found in step a2, and searching for all corresponding sub-matching point sets;
Based on the feature points of the base image plane and all the sub-matching point sets found in step a3, the base image in the diagonal position image plane that is the diagonal position of the base image plane. A step a4 for finding matching points corresponding to the feature points on the plane and all sub-matching points found in the step a3;
Uniqueness corresponding to the same subject in the four image planes based on the feature points of the base image plane, all sub-matching point sets found in step a3, and matching points found in step a4 Step a5 for determining a matching point set;
A method for matching feature points of a planar array of four camera sets. In the step a1, all matching points that match the feature point in the image plane adjacent to the base image plane in the lateral direction based on the matching condition 1) for one feature point on the base image plane Looking for
The matching condition 1) is that an imaging point in two image planes adjacent to each other in the lateral direction of one subject is an image on the left image plane of the subject and the right image corresponding to the subject. The imaging point in the plane is located on the same straight line parallel to the horizontal coordinate axis, and the horizontal offset (offset) of the imaging point in the left image plane relative to the coordinate origin of the left image plane is the right image plane. 2. The method for matching feature points of a four-camera planar array according to claim 1, wherein a condition that the horizontal offset of the imaging point in the right image plane with respect to the coordinate origin is larger is satisfied. In the step a2, all of the feature points on the base image plane in the step a1 that match the feature points in the image plane adjacent to the base image plane in the vertical direction based on the matching condition 2) Find matching points for
The matching condition 2) is that the image formation point in two image planes adjacent to each other in the vertical direction of one subject is an image formation point in the upper image plane of the subject and a lower side corresponding to the subject. The image point on the image plane is located on the same straight line parallel to the longitudinal coordinate axis, and the vertical offset of the image point on the upper image plane relative to the coordinate origin of the upper image plane is the coordinates of the lower image plane. 3. The method for matching feature points of a four-camera set planar array according to claim 2 , wherein the condition is greater than the vertical offset of the imaging point in the lower image plane relative to the origin. In step a3, all matching points found in step a1 and all matching points found in step a2 are rematched based on matching condition 3), and the matching condition 3) is not satisfied. A matching point is excluded, and matching points satisfying the matching condition 3) are paired according to the magnitude relationship of the horizontal direction offset value or the vertical direction offset value to form a sub-matching point set,
The matching condition 3) is that matching point sets corresponding to four image planes of one four-camera set of one subject have a ratio of the horizontal offset value and the vertical offset value of the length and width of the reference square. To satisfy the condition that the ratio is the same,
The horizontal offset value is the horizontal offset of the image point on the left image plane of the subject with respect to the coordinate origin of the left image plane and the image point on the right image plane corresponding to the subject on the right side. The horizontal offset difference with respect to the coordinate origin of the image plane of the image, and the vertical offset value is the vertical offset with respect to the coordinate origin of the image plane on the upper side relative to the coordinate origin of the image plane on the upper side. The imaging point in the corresponding lower image plane is the difference in vertical offset with respect to the coordinate origin of the lower image plane,
4. The method for matching feature points of a four-camera set planar array according to claim 3 , wherein the reference square is a square formed from four focal points of one four-camera set. Based on the feature point on the base image plane in step a4 and the submatching point set found in step a3, the matching condition 1) and the matching condition are set for any submatching point set. 2), a matching point on the diagonal position image plane is searched for, and the coordinates of the matching points located on the diagonal position image plane are obtained based on the coordinates of all the sub-matching point sets found in step a3. The abscissa of the matching point is the same as the abscissa of the matching point in the image plane adjacent to the diagonal position image plane in the vertical direction, and the ordinate of the matching point is the diagonal position image in the horizontal direction. Same as the ordinate of the matching point in the image plane adjacent to the plane, and after the coordinates are determined, the matching point in the diagonal position image plane When the image feature similarity matching is performed on the matching points in the other three image planes, and the matching is successful, the four imaging points in the four image planes constitute one matching point set, 5. The method according to claim 4 , wherein when matching is impossible, the matching point on the diagonal position image plane and the sub-matching point set found in step a <b> 3 corresponding to the matching point are excluded. A method for matching feature points of a planar array of camera sets. In step a5, the uniqueness matching point set corresponding to the same subject in the four image planes satisfies the matching condition 4), and the matching condition 4) is a set of four cameras for one subject. The imaging points in the corresponding four image planes form one rectangular shape, and the ratio of the horizontal length and the vertical length of the rectangular shape is the same as the ratio of the length and width of the reference rectangular shape. , according to claim 5, wherein the imaging point in the two image planes located at diagonal positions of the two pairs is to be located on two straight lines parallel to the two diagonals of each reference rectangular shape A method for matching feature points of a planar array of four cameras. In step a5, the case where the matching point set to satisfy the matching condition 4) there is only one matching result, feature points 4 camera sets of planar array according to claim 6, characterized in that the sole Matching method. In step a5, if the matching point set satisfying the matching condition 4) there are a plurality of base rectangular shape which satisfies the following conditions in the base image plane, that is, the feature point and one end point, said base An extension line of a diagonal line having the feature point of the square as one end point passes through the diagonal position image plane, and the length of the diagonal line corresponds to the pair of any two matching point sets corresponding to the feature point. It is the same as the distance between two matching points in the angular position image plane, the base square is similar to the reference square, and another end point of the base square has an image feature image of the feature point Determining whether there is a base square that satisfies the condition that it is similar to a feature and is a matching point of the feature point in the base image plane;
When such a square is present in the base image plane, for any base square, whether or not there is a square corresponding to the diagonal position image plane that is the same as the base square. Confirm,
If present, first, a unique matching point corresponding to the feature point is determined in the diagonal position image plane, and then another 2 of the feature point is determined based on the matching condition 1) and the matching condition 2). Determine the only matching points in one image plane, eliminate the unclear set of matching points,
A method of determining a unique matching point corresponding to the feature point in the diagonal position image plane is that the feature point of the base image plane is one end point of one diagonal of the base square. The only matching point in the angular position image plane is one of the two end points on the diagonal line through which the extension line passes through the feature point in the only corresponding square in the diagonal position image plane, and the feature point Is located at the upper end of the base square, the only matching point in the diagonal position image plane is the upper end of the only corresponding square, otherwise it is the lower end , For a non-unique matching point in the diagonal position image plane, the unique matching point of the point in the base image plane is the feature point of the base image plane. It can be determined to be another endpoint on the diagonal,
If not, the two matching point pairs corresponding to the feature points correspond to two different viewpoints, and the two different viewpoints are pixel projections where the feature points are relative to the base image plane in space. Located on the extension of the line,
When such a rectangular shape does not exist in the base image plane, the plurality of matching point sets correspond to a plurality of different subjects, and the feature points of the plurality of different subjects include the base image in space. 7. The method for matching feature points of a planar array of four camera sets according to claim 6 , wherein the matching method is located on an extension line of a pixel projection line facing the plane.   The feature point is an image point corresponding to one or a plurality of matching points, and the image point has an image feature that is distinguished from another image point. 9. A method for matching feature points of a planar array of four camera sets according to any one of 1 to 8. Step b1 of searching for a unique matching point set corresponding to all feature points in the base image plane by using the matching method according to any one of claims 1 to 9 after image acquisition is completed. When,
A step b2 of calculating a spatial position coordinate of the subject based on the image coordinates of the uniqueness matching point set acquired in the step b1;
A step b3 of forming 3D point cloud data, establishing a 3D point cloud graphic, and reproducing a three-dimensional stereoscopic image based on the spatial position coordinates of the subject obtained in step b2;
A measurement method based on a matching method of feature points of a planar array of four camera sets, characterized by comprising: In step b2, the focal points of the upper left a camera, upper right position b camera, lower left position c camera, and lower right position d camera of one 4-camera set are respectively O _a , O _b , O _c , and O _d , the four focal points are located in the same plane, and form one square, the length O _a O _{b of the} square is m, the width O _a O _c is n, The center point of the square is set to O, and one three-dimensional rectangular coordinate system is established with O as the origin, where the X axis is parallel to the O _a O _b and O _c O _d sides of the square Yes, the Y axis is parallel to the square O _a O _c and O _b O _d sides, the Z axis is perpendicular to the plane in which the focal point is located, and parallel to the optical axis direction of the four cameras Yes, the arrangement of the four cameras is the same, and the spatial position coordinate of one subject P of the subject is P ( _{x, P} y, a _{P z),} a camera point P, b camera, c cameras, and d the camera image coordinates of the image point in the image plane corresponding to these four cameras each _P a _{(P ax} , P _ay ), P _b (P _bx , P _by ), P _c (P _cx , P _cy ), and P _d (P _dx , P _dy ), the expression of the spatial position coordinates of the P point is It is as follows,
In the above equation, f is the focal length of the four cameras, u is the length of the target surface of the image sensor, v is the width of the target surface of the image sensor, where Δx At the time of lateral matching, the lateral offset value of the imaging point in the b image plane with respect to the imaging point in the a image plane and the lateral offset of the imaging point in the d image plane with respect to the imaging point in the c image plane are defined as Δ Define y as the vertical offset of the imaging point in the c image plane relative to the imaging point in the a image plane and the vertical offset of the imaging point in the d image plane relative to the imaging point in the b image plane during vertical matching. The measuring method based on the matching method of the feature point of the planar array of 4 camera sets of Claim 10 characterized by these.