CN115239801B

CN115239801B - Object positioning method and device

Info

Publication number: CN115239801B
Application number: CN202211161935.9A
Authority: CN
Inventors: 李凯文; 刘若阳; 殷琪
Original assignee: Nanjing Boshi Medical Technology Co ltd
Current assignee: Nanjing Boshi Medical Technology Co ltd
Priority date: 2022-09-23
Filing date: 2022-09-23
Publication date: 2023-03-24
Anticipated expiration: 2042-09-23
Also published as: CN115239801A

Abstract

The invention provides an object positioning method and device, which are used for acquiring object detection point coordinates of a first object image and a second object image obtained by shooting a target object by a first camera and a second camera, and determining three-dimensional coordinates of object detection points in an initial three-dimensional world coordinate system.

Description

Object positioning method and device

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an object positioning method and apparatus.

Background

In the detection process of object detection, such as face detection, eye detection, pupil detection, etc., in order to improve the detection accuracy of the detection instrument, a doctor or an operator needs to manually or automatically align the detection instrument with a detection point in an object, and therefore, the detection point in the object needs to be located regardless of whether the manual or automatic method is used.

At present, a plurality of positioning methods exist, but most of the methods are to process an object image to detect the position of a detection point in the image, tell an operator to manually move an instrument to align the detection point, and cannot obtain the three-dimensional coordinates of the detection point to assist the instrument to automatically position the instrument to an instrument center for subsequent inspection. The existing automatic positioning method mainly relies on an additionally added light path design to position the distance of a detection point on the space, or adds a distance sensor to detect the front and back distance of the detection point from an instrument, and the like. This not only increases the complexity of the optical path, but also the cost of the instrument.

Disclosure of Invention

In view of this, the present invention provides an object positioning method and device, which can realize automatic positioning of an object detection point without additionally adding an optical path design and hardware equipment.

In order to achieve the above purpose, the invention provides the following specific technical scheme:

in a first aspect, an embodiment of the present invention provides an object positioning method, including:

acquiring a first object image shot by a first camera on a target object and a second object image shot by a second camera on the target object, and acquiring object detection point coordinates in the first object image and the second object image;

determining three-dimensional coordinates of object detection points in an initial three-dimensional world coordinate system according to object detection point coordinates in the first object image and the second object image, wherein the initial three-dimensional world coordinate system is constructed by taking the optical center of the first camera or the second camera as an origin;

converting the three-dimensional coordinates of the object detection points in the initial three-dimensional world coordinate system into three-dimensional coordinates in a target three-dimensional world coordinate system, wherein the target three-dimensional world coordinate system is constructed by taking a target point as an origin;

and determining the relative position of the object detection point and the target point according to the three-dimensional coordinates of the object detection point in the target three-dimensional world coordinate system.

In some embodiments, acquiring object detection point coordinates in the first object image and the second object image comprises:

respectively preprocessing the first object image and the second object image to obtain a target object area in the first object image and the second object image and an edge point coordinate of the target object area;

respectively carrying out contour fitting on the edge point coordinates of the target object area in the first object image and the second object image and obtaining contour center point coordinates;

and determining the coordinate of the contour center point corresponding to the first object image as the coordinate of the object detection point in the first object image, and determining the coordinate of the contour center point corresponding to the second object image as the coordinate of the object detection point in the second object image.

In some embodiments, the pre-processing the first object image and the second object image respectively to obtain the target object area in the first object image and the second object image and the edge point coordinates of the target object area includes:

carrying out graying processing and Gaussian filtering processing on the target object image in sequence to obtain a fuzzy image, wherein the target object image is the first object image or the second object image;

calculating a global threshold value of the blurred image;

creating a first blank image, a second blank image and a third blank image which are consistent with the size of the blurred image for the blurred image;

respectively calculating a line threshold value of each line in the blurred image, performing binarization processing on the line by taking the minimum value of the product of the line threshold value and a first coefficient and the product of the global threshold value and a second coefficient as a final threshold value, and putting the binarization processing result of each line in the blurred image into a corresponding line of the first blank image;

respectively calculating a column threshold value of each column in the blurred image, performing binarization processing on the column by taking the minimum value of the product of the column threshold value and a first coefficient and the product of the global threshold value and a second coefficient as a final threshold value, and putting the binarization processing result of each column in the blurred image into a corresponding column of the second blank image;

putting the binarization processing results of which the positions are the same in the first blank image and the second blank image and the binarization processing results are 255 into the corresponding position of the third blank image, and setting the pixel value of the rest position of the third blank image to be 0 to obtain a segmentation image corresponding to the blurred image;

screening candidate regions of the segmentation image;

and determining the target object area in the target object image and the edge point coordinates of the target object area according to the candidate area screening result of the segmented image.

In some embodiments, candidate region screening of the segmented image comprises:

sequentially carrying out candidate region detection and connected domain detection on the segmentation image to obtain contour parameters of all candidate regions in the segmentation image, a connected domain of each candidate region and circumscribed rectangle parameters of the connected domain;

and screening candidate regions of which the areas of the connected domains are not in a preset area interval, the lengths or the widths of the external rectangles of the connected domains are not in a preset length interval, the aspect ratios of the connected domains are not in a preset aspect ratio interval, the ratio of the areas of the connected domains to the area of the external rectangles is not in a preset area ratio interval or the pixel mean value of the candidate regions is not in a preset pixel value interval, so as to obtain a candidate region screening result of the segmented image.

In some embodiments, the determining the target object region in the target object image and the edge point coordinates of the target object region according to the candidate region screening result of the segmented image includes:

under the condition that the number of candidate areas in the candidate area screening result of the segmented image is not 0, acquiring corresponding area image blocks in the target object image according to circumscribed rectangle parameters of the candidate areas in the segmented image;

calculating normalized cross-correlation coefficients of image blocks corresponding to circumscribed rectangles of the candidate regions in the segmented image and image blocks corresponding to the regions in the target object image to obtain normalized cross-correlation coefficients corresponding to each candidate region;

determining a candidate region corresponding to a maximum value in the normalized cross-correlation coefficient larger than a threshold value in candidate region screening results of the segmented images as the target object region of the target object image;

and carrying out edge point detection on the target object area of the target object image to obtain edge point coordinates of the target object area.

In some embodiments, the method for constructing the initial three-dimensional world coordinate system includes:

calibrating the first camera and the second camera to obtain internal and external parameters of the first camera and the second camera, and determining a rotation parameter and a translation parameter of the first camera relative to the second camera;

establishing a mapping relation between a pixel coordinate system of the second camera and the initial three-dimensional world coordinate system, wherein the initial three-dimensional world coordinate system takes the optical center of the second camera as an origin;

and constructing an initial three-dimensional reconstruction model based on the mapping relation between the second camera from a pixel coordinate system to the initial three-dimensional world coordinate system, the internal and external parameters of each camera and the rotation parameter and the translation parameter of the first camera relative to the second camera, wherein the initial three-dimensional reconstruction model is used for calculating the three-dimensional coordinates of the corresponding points in the first object image and the second object image in the initial three-dimensional world coordinate system.

In some embodiments, the target three-dimensional world coordinate system is constructed by:

acquiring direction vectors of three coordinate axes in the target three-dimensional world coordinate system;

determining a linear equation of a first coordinate axis and a second coordinate axis of the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system according to the position relationship between the optical centers of the first camera and the second camera and the target point and direction vectors of three coordinate axes in the target three-dimensional world coordinate system;

and calculating the coordinates of the target point in the initial three-dimensional world coordinate system, and determining a linear equation of a third coordinate axis of the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system.

In some embodiments, converting the three-dimensional coordinates of the object detection points in the initial three-dimensional world coordinate system to three-dimensional coordinates in a target three-dimensional world coordinate system comprises:

determining plane equations of three coordinate planes in the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system according to coordinates of the target point in the initial three-dimensional world coordinate system and linear equations of a first coordinate axis, a second coordinate axis and a third coordinate axis in the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system;

and in the initial three-dimensional world coordinate system, the distances from the object detection points to three coordinate planes are respectively calculated as the coordinates of the object detection points in the target three-dimensional world coordinate system.

In some embodiments, obtaining direction vectors of three coordinate axes in the target three-dimensional world coordinate system comprises:

selecting a point A in the real world, selecting a point B in the horizontal direction of the passing point A, and selecting a point C in the vertical direction of the passing point A;

detecting pixel coordinates of A, B and C in the first object image and pixel coordinates of A, B and C in the second object image;

calculating three-dimensional coordinates of A, B and C in the initial three-dimensional world coordinate system according to the pixel coordinates of A, B and C in the first object image and the pixel coordinates of A, B and C in the second object image;

taking the direction of the vector AB as the forward direction of a first coordinate axis of the target three-dimensional world coordinate system, and determining a direction vector of the first coordinate axis according to the three-dimensional coordinates of A and B;

taking a vector CA as the forward direction of a second coordinate axis of the target three-dimensional world coordinate system, and determining a direction vector of the second coordinate axis according to the three-dimensional coordinates of C and A;

and taking the cross product of the vector CA and the vector AB as the positive direction of a third coordinate axis of the target three-dimensional world coordinate system, and determining a direction vector of the third coordinate axis.

In a second aspect, an embodiment of the present invention provides a pupil diameter measurement method, including:

when the target object is a pupil, acquiring an ellipse edge and an ellipse center point coordinate obtained by performing ellipse fitting on an edge point coordinate of a pupil area;

determining a horizontal line passing through the center point of the ellipse and determining coordinates of two intersection points of the horizontal line and the edge of the ellipse;

and respectively acquiring three-dimensional coordinates of the two intersection points in the initial three-dimensional world coordinate system or the target three-dimensional world coordinate system by using the object positioning method, and further calculating the diameter of the pupil.

In a third aspect, an embodiment of the present invention provides an object positioning apparatus, including:

an object detection point two-dimensional coordinate acquisition unit, configured to acquire a first object image captured by a first camera on a target object and a second object image captured by a second camera on the target object, and acquire object detection point coordinates in the first object image and the second object image;

an object detection point three-dimensional coordinate determination unit, configured to determine, according to object detection point coordinates in the first object image and the second object image, three-dimensional coordinates of an object detection point in an initial three-dimensional world coordinate system, where the initial three-dimensional world coordinate system is constructed with an optical center of the first camera or the second camera as an origin;

a three-dimensional coordinate conversion unit, configured to convert three-dimensional coordinates of the object detection point in the initial three-dimensional world coordinate system into three-dimensional coordinates in a target three-dimensional world coordinate system, where the target three-dimensional world coordinate system is constructed with a target point as an origin;

and the relative position determining unit is used for determining the relative position of the object detection point and the target point according to the three-dimensional coordinates of the object detection point in the target three-dimensional world coordinate system.

In some embodiments, the object detection point two-dimensional coordinate acquisition unit includes:

the image preprocessing subunit is configured to respectively preprocess the first object image and the second object image to obtain a target object area in the first object image and the second object image and an edge point coordinate of the target object area;

a contour fitting subunit, configured to perform contour fitting on the edge point coordinates of the target object region in the first object image and the second object image, respectively, and obtain a contour center point coordinate;

and the object detection point coordinate determining subunit is configured to determine the contour center point coordinate corresponding to the first object image as the object detection point coordinate in the first object image, and determine the contour center point coordinate corresponding to the second object image as the object detection point coordinate in the second object image.

In some embodiments, the image pre-processing subunit comprises:

a blurred image obtaining subunit, configured to perform graying processing and gaussian filtering processing on the target object image in sequence to obtain a blurred image, where the target object image is the first object image or the second object image;

the overall threshold value operator unit is used for calculating an overall threshold value of the blurred image;

a blank image creating subunit configured to create, for the blurred image, a first blank image, a second blank image, and a third blank image that are in accordance with the size thereof;

the first binarization processing subunit is used for respectively calculating a line threshold value of each line in the blurred image, performing binarization processing on the line by taking a minimum value of a product of the line threshold value and a first coefficient and a product of the global threshold value and a second coefficient as a final threshold value, and putting a binarization processing result of each line in the blurred image into a corresponding line of the first blank image;

the second binarization processing subunit is used for respectively calculating a column threshold value of each column in the blurred image, performing binarization processing on the column by taking the minimum value of a product of the column threshold value and the first coefficient and a product of the global threshold value and the second coefficient as a final threshold value, and putting a binarization processing result of each column in the blurred image into a corresponding column of the second blank image;

a segmented image obtaining subunit, configured to put binarization processing results that are in the same position in the first blank image and the second blank image and have binarization processing results of 255 into a corresponding position in the third blank image, and set a pixel value of a remaining position in the third blank image to 0, so as to obtain a segmented image corresponding to the blurred image;

a candidate region screening subunit, configured to perform candidate region screening on the segmented image;

and the object region determining subunit is used for determining the target object region in the target object image and the edge point coordinates of the target object region according to the candidate region screening result of the segmented image.

In some embodiments, the candidate region screening subunit is specifically configured to:

In some embodiments, the object region determining subunit is specifically configured to:

determining a candidate region corresponding to a maximum value in the normalized cross-correlation coefficient larger than a threshold value in the candidate region screening results of the segmented images as the target object region of the target object image;

In some embodiments, the apparatus further comprises an initial three-dimensional world coordinate system construction unit for:

and constructing an initial three-dimensional reconstruction model based on the mapping relation between the second camera and the initial three-dimensional world coordinate system from the pixel coordinate system, the internal and external parameters of each camera and the rotation parameters and the translation parameters of the first camera relative to the second camera, wherein the initial three-dimensional reconstruction model is used for calculating the three-dimensional coordinates of the corresponding points in the first object image and the second object image in the initial three-dimensional world coordinate system.

In some embodiments, the apparatus further comprises a target three-dimensional world coordinate system construction unit for:

In some embodiments, the three-dimensional coordinate transformation unit is specifically configured to:

In some embodiments, the relative position determining unit is specifically configured to:

Compared with the prior art, the invention has the following beneficial effects:

the invention discloses an object positioning method, which comprises the steps of obtaining object detection point coordinates of a first object image and a second object image obtained by shooting a target object by a first camera and a second camera, and determining three-dimensional coordinates of object detection points in an initial three-dimensional world coordinate system.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic flowchart of an object positioning method according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a portion of a method for object location according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating a portion of a method for object location according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a candidate region according to an embodiment of the disclosure;

FIG. 5 is a schematic diagram illustrating a comparison between a candidate area and a corresponding original image block according to an embodiment of the disclosure;

FIG. 6 is a flowchart illustrating a portion of a method for object location according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a checkerboard image according to an embodiment of the present invention;

fig. 8 is a schematic view of a binocular camera disclosed in an embodiment of the present invention;

fig. 9 is a schematic diagram of a checkerboard image pair captured by a binocular camera according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of an initial three-dimensional world coordinate system disclosed in an embodiment of the present invention;

FIG. 11 is a schematic view of a target three-dimensional world coordinate system disclosed in an embodiment of the present invention;

FIG. 12 is a flowchart illustrating a portion of a method for object location in accordance with an embodiment of the present invention;

FIG. 13 is a schematic view of a checkerboard image labeled with directional vectors of three coordinate axes in a target three-dimensional world coordinate system, as disclosed in an embodiment of the present invention;

FIG. 14 is a schematic diagram of a pupil diameter disclosed in an embodiment of the present invention;

fig. 15 is a schematic structural diagram of an object positioning apparatus according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention provides an object positioning method and device, which can be applied to any existing positioning equipment, such as a face detection instrument, an eye detection instrument, a visual light measuring instrument and the like, wherein the positioning equipment is provided with two cameras.

Specifically, referring to fig. 1, an object positioning method provided by the embodiment of the present invention includes the following steps:

s101: acquiring a first object image shot by a first camera on a target object and a second object image shot by a second camera on the target object, and acquiring object detection point coordinates in the first object image and the second object image;

it is understood that the first object image and the second object image are two images of the same target object captured by two cameras, respectively. The first camera and the second camera may be two cameras of a binocular camera, or may not be cameras of the binocular camera, and the present invention is not particularly limited.

The object detection points are different for different application scenarios, for example: the object detection point in the face detection may be a nose tip center point, the object detection point in the eye detection may be an inner eye corner point, and the object detection point in the pupil detection may be a pupil center point.

Further, referring to fig. 2, this embodiment provides an optional implementation manner of S101, which specifically includes the following steps:

s201: respectively preprocessing the first object image and the second object image to obtain a target object area in the first object image and the second object image and edge point coordinates of the target object area;

referring to fig. 3, this embodiment provides an optional implementation manner of S201, which specifically includes the following steps:

s301: carrying out graying processing and Gaussian filtering processing on a target object image in sequence to obtain a fuzzy image, wherein the target object image is a first object image or a second object image;

s302: calculating a global threshold of the blurred image;

specifically, the universe method may be used to calculate the global threshold of the blurred image, and of course, other existing methods may also be used to calculate the global threshold of the blurred image, which is not limited in the present invention.

S303: creating a first blank image, a second blank image and a third blank image which are consistent with the size of the blurred image for the blurred image;

s304: respectively calculating a line threshold value of each line in the blurred image, performing binarization processing on the line by taking the minimum value of the product of the line threshold value and the first coefficient and the product of the global threshold value and the second coefficient as a final threshold value, and putting the binarization processing result of each line in the blurred image into the corresponding line of the first blank image;

specifically, the population method may be used to calculate the line threshold of each line in the blurred image, and of course, other existing methods may also be used to calculate the line threshold of each line in the blurred image, which is not limited in the present invention.

S305: respectively calculating a column threshold value of each column in the blurred image, performing binarization processing on the column by taking the minimum value of the product of the column threshold value and the first coefficient and the product of the global threshold value and the second coefficient as a final threshold value, and putting the binarization processing result of each column in the blurred image into a corresponding column of the second blank image;

specifically, the column threshold of each column in the blurred image may be calculated by using the tsui method, and of course, other existing methods may also be used to calculate the column threshold of each column in the blurred image, which is not limited in the present invention.

S306: putting the binarization processing results of which the positions are the same in the first blank image and the second blank image and the binarization processing results are 255 into the corresponding position of a third blank image, and setting the pixel value of the rest position of the third blank image to be 0 to obtain a segmentation image corresponding to the blurred image;

s307: screening candidate regions of the segmented image;

s308: and determining a target object area in the target object image and the edge point coordinates of the target object area according to the candidate area screening result of the segmented image.

It should be noted that, for convenience of description, the first object image or the second object image is represented by the target object image, that is, both the first object image and the second object image perform the above-mentioned processing of S301 to S308. For convenience of description, the first object image is represented by image1, and the second object image is represented by image 2.

The image1 and the image2 are subjected to graying processing, and then gaussian filtering processing is performed to obtain a first blurred image imageBlur1 and a second blurred image imageBlur2. The global thresholds threshAll1 and threshAll2 of imageBlur1 and imageBlur2, respectively, are obtained using the Otsu method. Taking the first blurred image imageBlur1 as an example, three blank images, namely a first blank image imageX1, a second blank image imageY1 and a third blank image imageBW1, which are consistent with the first blurred image imageBlur1 in size are defined.

Each line of the first blurred image imageBlur1 is processed, a threshold value of each line is calculated by using the atsu method, and thresh1i represents a threshold value of the ith line in the imageBlur 1. And comparing the sizes of thresh1i r1 and threshAll1 r2, selecting the smaller one as a final threshold value to carry out binarization processing on the row, and putting the result into a corresponding position in the first blank image imageX 1. Each column of the first blurred image imageBlur1 is the same, and a filled second blank image imageY1 is obtained, wherein r1 and r2 are two coefficients, and the value can be 0.5. And processing the first blank image imageX1 and the second blank image imageY1, if the positions of the first blank image imageX1 and the second blank image imageY1 are the same and the binarization processing results are 255, setting the corresponding position of the third blank image imageBW1 to be 255, otherwise, setting the corresponding position of the third blank image imageBW1 to be 0, and obtaining a first segmentation image.

Then, candidate region screening is performed on the first segmented image, and in order to improve the screening accuracy of the candidate region, the candidate region screening result of the first segmented image may be obtained through the following five rounds of screening:

and acquiring contour parameters of all connected domains in the third blank image imageBW1 by using a findContours function, and acquiring circumscribed rectangle parameters of all connected domains in the third blank image imageBW1 by using a boundingRec algorithm.

Because the target object region is not very large or very small in an image, candidate regions with connected components having areas not within a preset area interval are filtered out, for example, candidate regions with connected component areas larger than the maximum threshold 15000 or smaller than the minimum threshold 100 are filtered out.

The length or width of the circumscribed rectangle of the connected domain of the target object is not very long or very short, so that candidate regions with the length or width of the connected domain not within the preset length interval are screened out, for example, candidate regions with the length or width of the circumscribed rectangle larger than the maximum threshold 200 or smaller than the minimum threshold 20 are screened out.

For a bounding rectangle of the connected domain of the target object, the aspect ratio is within a certain range. Especially, in the case where the target object is a pupil or an iris, taking the target object as the pupil as an example, since the pupil itself is circular, even if the pupil is deformed and becomes an ellipse at different angles, such as viewing the pupil from bottom to top, viewing the pupil from a side, and the like, the aspect ratio or the aspect ratio thereof is not greater than the maximum value ratio max and is not less than the minimum value ratio min. Therefore, candidate regions with the aspect ratios of the circumscribed rectangles of the connected domain not within the preset aspect ratio interval are screened out, for example, candidate regions with the aspect ratios of the circumscribed rectangles larger than the maximum threshold 2 or smaller than the minimum threshold 0.5 are screened out.

For a target object connected domain, the ratio of the area of the connected domain to the area of the external rectangle of the connected domain is larger than a certain value, and if the area ratio is smaller, the outline has a large hollow area and is not a target object area. Taking the target object as the pupil as an example, as shown in fig. 4, the first is a pupil region connected region, and the last three are non-pupil region connected regions, so that even if the pupil has interference of bright spots, the area ratio of the connected regions is still larger than that of the last three. Therefore, candidate regions in which the ratio of the area of the connected domain to the area of the circumscribed rectangle is not within the preset area ratio range are screened out, for example, candidate regions in which the ratio of the area of the connected domain is less than the threshold value of 0.55 are screened out.

Since the pixel value of the target object region is smaller than that of the peripheral region, the mean value thereof is not close to 0. And screening candidate areas with the pixel mean value not in the preset pixel value interval, for example, according to circumscribed rectangle parameters of connected domains of the candidate areas, intercepting the image blocks blockBlur and blockBW at corresponding positions in imagBlur1 and imageBW1, and calculating the pixel mean value gradmean according to the following formula. And screening out candidate areas with the grayMean larger than the maximum threshold grayMax or smaller than the minimum threshold grayMin. Such as screening out candidate regions having a grayMean of less than 0.075 or greater than 0.25.

Wherein, the blockBW is an image block at a corresponding position intercepted by the imageBW1 according to the circumscribed rectangle parameter;

the blockBlur is an image block at a corresponding position in the imageBlur1, which is intercepted according to the circumscribed rectangle parameter;

the representation count is used for counting the number of pixels belonging to a connected domain in the image block BW;

a pixel value with (m, n) in the image block bw is represented;

a pixel value representing a position (m, n) in the block Blur of the image block

The sum of pixel values that meet the conditions in the above formula is represented.

After the five rounds of screening, if the number of remaining candidate regions is 0, it indicates that the target object does not exist in the image. If not, the remaining candidate region is the candidate region screening result.

Further, the remaining candidate regions are further screened by using NCC (Normalized cross correlation) to obtain a target object region.

The target object region blockBlur intercepted by the circumscribed rectangle is determined to have a pixel value of the middle region smaller than the pixel values of the surrounding, and the intercepted candidate region blockBW is inverted to have a middle black and a surrounding white, so that the target object region has a certain similarity with the contour region, the NCC value is higher, and the similarity of the non-target object candidate regions with the corresponding contour regions is lower, namely the NCC value is lower. Taking the target object as the pupil as an example, as shown in fig. 5, three groups of images are respectively a schematic comparison diagram between a candidate region and a corresponding original image block, where a left image in each group of images is an image block including the candidate region, a right image is an original image block, an image group (1) represents the pupil region, and image groups (2) and (3) represent non-pupil regions, and it can be seen that the similarity between a connected region image block of the pupil region and a corresponding original image block is greater than the latter two cases, that is, the NCC value thereof is the largest among the three cases.

Therefore, in the present embodiment, the candidate region having the largest NCC value is used as the final target region. And after the block BW negation operation, calculating the image blocks corresponding to the circumscribed rectangles of the candidate areas in the segmented image and the NCC of the image blocks corresponding to the corresponding areas in the target object image, wherein the image blocks corresponding to the corresponding areas in the target object image can be obtained according to parameters of the circumscribed rectangles of the candidate areas in the segmented image, and for example, the image blocks corresponding to the areas in the target object image can be obtained according to the top left corner vertexes of the circumscribed rectangles of the candidate areas in the segmented image and the length and width of the circumscribed rectangles.

The formula for the calculation of the NCC value is as follows:

is the pixel mean of the blockBlur,

pixel mean value as blockBW。

And selecting the candidate area with the NCC value larger than the threshold value and the largest NCC value as the target object area.

And acquiring edge point coordinates of the target object region, which are obtained by detecting the connected domain through a findContours algorithm.

S202: respectively carrying out contour fitting on the edge point coordinates of the target object area in the first object image and the second object image, and acquiring the coordinates of the center point of the contour;

the contour of the target object region may be a rectangle, a triangle, another polygon, an ellipse, a circle, etc., and is determined according to the shape of the target object, and taking the target object as a pupil as an example, the contour fitting of the edge point coordinates of the target object region is specifically ellipse fitting.

S203: and determining the coordinate of the contour center point corresponding to the first object image as the coordinate of the object detection point in the first object image, and determining the coordinate of the contour center point corresponding to the second object image as the coordinate of the object detection point in the second object image.

For convenience of explanation, the object detection point coordinates of the imageBlur1 image are represented as (x 1, y 1), and the object detection point coordinates (x 2, y 2) are obtained by the same process as described above for the imageBlur2.

Taking the target object as the pupil as an example, the object detection point is the pupil center point.

S102: determining three-dimensional coordinates of object detection points in an initial three-dimensional world coordinate system according to object detection point coordinates in the first object image and the second object image, wherein the initial three-dimensional world coordinate system is constructed by taking the optical center of the first camera or the second camera as an origin;

before the positioning apparatus is put into use, it is necessary to construct an initial three-dimensional world coordinate system with an optical center of one of the first camera and the second camera as an origin, please refer to fig. 6, where constructing the initial three-dimensional world coordinate system includes the following steps:

s401: calibrating the first camera and the second camera to obtain internal and external parameters of the first camera and the second camera, and determining rotation parameters and translation parameters of the first camera relative to the second camera;

s402: establishing a mapping relation between a pixel coordinate system and an initial three-dimensional world coordinate system of the second camera, wherein the initial three-dimensional world coordinate system takes the optical center of the second camera as an origin;

s403: and constructing an initial three-dimensional reconstruction model based on the mapping relation between the second camera from the pixel coordinate system to the initial three-dimensional world coordinate system, the internal and external parameters of each camera and the rotation parameters and translation parameters of the first camera relative to the second camera, wherein the initial three-dimensional reconstruction model is used for calculating the three-dimensional coordinates of corresponding points in the object image pair shot by the first camera and the second camera in the initial three-dimensional world coordinate system.

In this embodiment, a Zhangyingyou calibration method is used to calibrate the first camera and the second camera, and the following is a specific calibration process:

in the preparation stage, a checkerboard image, an alternative checkerboard image is shown in fig. 7, preferably with different numbers of black and white intersections in the horizontal and vertical directions. The prepared checkerboard image is pasted on a steel plate or a hardboard, so that the checkerboard image is conveniently placed in front of the assembled first camera and the second camera. The first camera and the second camera may constitute a binocular camera, and an example of the binocular camera constituted by the first camera and the second camera is shown in fig. 8, in order to ensure that the two cameras can simultaneously capture all checkerboard images. Each time a pair of images is captured, the checkerboard is adjusted in position or posture, so that N pairs of checkerboard images are captured, where N is a positive integer, and an example of the captured pair of checkerboard images is shown in fig. 9.

For each pair of checkerboard images, a corner detection algorithm, a checkerboard corner searching algorithm (findchessboardcorrers) or other algorithms are used to respectively detect the corner coordinates of the black and white intersection points in each checkerboard image along a preset sequence from the upper left corner of the checkerboard image, so as to obtain two sets of corner coordinates, and the two sets of corner coordinates are in one-to-one correspondence. Because a pair of checkerboard images corresponds to a pose of the checkerboard in a real world coordinate system, three-dimensional coordinates (x, y, z) corresponding to angular points can be obtained simultaneously, the three-dimensional coordinates of the angular points at the upper left corner are set to be (0, 0) from the angular point at the upper left corner, and the three-dimensional coordinate amplification of the adjacent angular points is side, namely the side length side of each square block in the checkerboard, and the unit is mm. Taking two intersection points adjacent to each other at the left and right as an example, the coordinates are (x, y, z) and (x + side, y, z), respectively. For convenience of subsequent calibration processing, z is set to 0.

After the one-to-one correspondence relationship between the checkerboard image and the corresponding points on the real checkerboard is established, the point pairs with the established correspondence relationship can be transmitted into a calibration function stereocalibration, and the internal reference matrixes camera matrix1 and camera matrix2, the distortion parameter matrixes distCoeffs1 and distCoeffs2, and the external reference matrixes R1, R2, T1 and T2 of the two cameras are calculated. Wherein the internal reference matrices, the camera matrix1, the camera matrix2, and the external reference matrices R1, R2 are all matrices of 3 × 3 size, and T1 and T2 are matrices of 1 × 3 size.

The above is only to obtain the internal and external parameters of each camera, and since the two cameras are used in this embodiment, the external parameters between the two cameras need to be calculated, and specifically, the rotation matrix parameters R and the translation matrix parameters T between the two cameras can be obtained by introducing all the parameters into the function stereorectification. The calculated R and T are the rotation parameter R and the translation parameter T of the first camera with respect to the second camera. Thus, the calibration process of the first camera and the second camera is completed.

Because the coordinates of the pupil center point in the pupil image pair shot by the first camera and the second camera are two-dimensional coordinates, an initial three-dimensional world coordinate system model needs to be constructed according to the calibrated parameters of the first camera and the second camera, and the initial three-dimensional world coordinate system model is used for calculating the three-dimensional coordinates of the point in the pupil image pair shot by the first camera and the second camera in the initial three-dimensional world coordinate system. The method specifically comprises the following steps:

in the Zhangyingyou calibration method, the established calculation formula of the mapping between the monocular camera from the pixel coordinate system to the world coordinate system is as follows:

（1）

wherein (u, v) is a cameraThe pixel coordinates of the point in the captured image, (X, Y, Z) are the three-dimensional coordinates of the corresponding point in a three-dimensional coordinate system, Z _c Indicating the vertical distance of the point from the camera, the camera matrix being the camera's internal reference matrix.

Is a mathematical parameter representation obtained by expanding the left part of the above equation (1).

Order to

And

respectively represent the parameters corresponding to the two cameras,

and

the coordinates of corresponding points shot by the two cameras are respectively expressed, and the formula (1) is developed to obtain:

（2）

eliminating Z based on formula (1) and formula (2) _c Obtaining an equation set:

（3）

it should be noted that R and T defined by the first camera and the second camera are a rotation matrix and a translation matrix of the first camera relative to the second camera. Using R and T as rotation matrix parameters of the first camera

And translation matrix parameters

. Rotation matrix of second camera

And the translation matrix may be arranged as

：

（4）

And substituting the rotation matrixes and the translation matrixes of the two cameras into the formulas (1) and (3) to calculate the three-dimensional coordinates (X, Y and Z) corresponding to any point P in the pair of pupil images shot by the first camera and the second camera.

In this way, the construction of the initial three-dimensional world coordinate system model is completed, and the optical center of the second camera is taken as the origin of the initial three-dimensional world coordinate system for the above description, it is understood that the optical center of the first camera may also be taken as the origin of the initial three-dimensional world coordinate system, and the present invention is not limited in particular.

Based on the three-dimensional coordinates of the object detection point in the initial three-dimensional world coordinate system, the three-dimensional coordinates of the object detection point in the first object image and the second object image can be determined.

S103: converting the three-dimensional coordinates of the object detection points in the initial three-dimensional world coordinate system into three-dimensional coordinates in a target three-dimensional world coordinate system, wherein the target three-dimensional world coordinate system is constructed by taking the target point as an origin;

it should be noted that the initial three-dimensional world coordinate system is a model created by using the optical center of one of the first camera and the second camera as the origin and the optical center axis as the Z axis, as shown in fig. 10. The origin of the initial three-dimensional world coordinate model is the optical center of the second camera, i.e., the point O in fig. 10, and XYZ axes are shown in fig. 10. Therefore, the calculated three-dimensional coordinates cannot visually and accurately represent the point P of the object detected by the first camera and the second cameraThe distance between the front and back of the plane and the distance between the upper, lower, left and right of the plane. In such a case, the calculated three-dimensional coordinates of the object detection points cannot represent the distance from the object detection points to the front and back of the plane where the first camera and the second camera are located, i.e., the distance measurement cannot be performed, and the three-dimensional coordinates of the object detection points cannot represent the target points in the distance map of the object detection points

The upper, lower, left and right distances of (c).

Therefore, it is necessary to reconstruct the three-dimensional world coordinate system model again to construct the desired target three-dimensional world coordinate system model, as shown in fig. 11, i.e. moving the origin of the initial three-dimensional world coordinate system in fig. 10 to the target point

The target three-dimensional world coordinate system is established by taking the plane where the first camera and the second camera are located as an XY plane and taking the direction vertical to the XY plane as a Z axis. Referring to fig. 12, the specific operation of constructing the target three-dimensional world coordinate system is as follows:

s501: acquiring direction vectors of three coordinate axes in a target three-dimensional world coordinate system;

s502: determining a linear equation of a first coordinate axis and a second coordinate axis of a target three-dimensional world coordinate system in an initial three-dimensional world coordinate system according to the position relationship between the optical centers of the first camera and the second camera and the target point and direction vectors of three coordinate axes in the target three-dimensional world coordinate system;

s503: and calculating the coordinates of the target point in the initial three-dimensional world coordinate system, and determining a linear equation of a third coordinate axis of the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system.

Selecting a point A in the real world, selecting a point B in the horizontal direction of the passing point A, and selecting a point C in the vertical direction of the passing point A; a person skilled in the art may also select more points, for example, select points D and E in the real world to represent the horizontal direction, and points F and G to represent the vertical direction, and the calculation principle is the same as the following method, in this embodiment, for convenience of calculation, only the optimal solution is used as an example;

Specifically, it is ensured that the vertical direction of the checkerboard image is the vertical downward direction of the real world, the checkerboard image plane is completely parallel to the plane where the first camera and the second camera are located, and a pair of images is captured, as shown in fig. 13, where a straight line AB is the horizontal direction in the real world, a straight line AC is the vertical direction in the real world, and a perpendicular line of the plane where the straight line AB and the straight line AC are located is the front-back direction of the real world.

Detecting the pixel coordinates of the points A, B and C in the two images by using a corner detection algorithm, a chessboard angle searching algorithm (findChessboardCoranders) or other algorithms:

，

，

substituting two of them into formula (3) to obtain three-dimensional coordinates of point A, point B and point C

，

，

。

Starting to construct the origin and three axes of the target three-dimensional world coordinate system model, for convenience of description, the coordinates in the initial three-dimensional world coordinate system model constructed for the first time are all represented by capital letters, and the coordinates in the target three-dimensional world coordinate system model constructed for the second time are represented by lowercase letters.

The direction of the vector AB is taken as the positive direction of the X axis, the vector CA is taken as the positive direction of the Y axis, and the cross product of the vector CA and the vector AB

Forward direction as Z axis:

（5）

wherein the content of the first and second substances,

，

，

；

，

，

；

，

，

。

after the direction vectors of the three coordinate axes are obtained, the three coordinate axes and the coordinate origin of the target three-dimensional world coordinate system are required to be constructed by the coordinates of the two cameras in the initial three-dimensional world coordinate system. In the calibration process, an initial three-dimensional world coordinate system with the optical center of the second camera as the origin is established, and the camera is set as C2, where C1 is the right camera and C2 is the left camera. Since the first model constructed is with C2 as the origin O, the coordinates of the optical center of C2 are (0, 0). The coordinates of the C1 optical center can be obtained according to equation (6):

（6）

where R and T are the rotation matrix parameter and translation matrix parameter, respectively, of the first camera relative to the second camera, (X) _C2 ,Y _C2 ,Z _C2 ) Coordinates representing the C2 optical center, (X) _C1 ,Y _C1 ,Z _C1 ) Coordinates representing the optical center of C1, let:

（7）

wherein the content of the first and second substances,

for the post-R-expansion mathematical parametric representation,

for T exhibitionThe mathematical parameters after the opening represent the form.

Unfolding to obtain:

（8）

substituting the correlation value to solve to obtain the coordinate (X) of the C1 optical center _C1 ,Y _C1 ,Z _C1 )。

The origin and the three coordinate axes in the target three-dimensional world coordinate system can be constructed according to the optical center coordinates of the two cameras and the direction vectors of the three coordinate axes. There are two cases, one is that the symmetry axes of the optical centers of the two cameras pass through the target point

Point, another case is that the symmetry axes of the optical centers of the two cameras do not pass through the target point

And (4) point. Target point

The point is where the pupil needs to be aligned, i.e. the point is where the pupil needs to be aligned

Line connecting point and pupil center point and plane

And is vertical. Let C1C2 midpoint be C3 (X) _C3 ,Y _C3 ,Z _C3 ) Then its coordinate is (X) _C1 /2,Y _C1 /2,Z _C1 /2) let the target point

The coordinates of the points are:

then, the solving process is as follows:

1) Two camera lightsThe axis of symmetry of the center passing through the center

Point:

(1-1): if it is

C1, C2 are collinear, then

Point C3;

(1-2): if straight line

And a straight line

And is vertical. Taking fig. 11 as an example, the X-axis of the target three-dimensional world coordinate system model must pass through the optical center point C1 of the first camera, i.e. the side camera, and the Y-axis must pass through the optical center point C2 of the second camera, i.e. the lower camera.

The equations for the X-axis line and the Y-axis line are:

（9）

wherein (X, Y, Z) in the equation 1 of the straight line where the X axis is located is the three-dimensional coordinate of the corresponding point on the straight line where the X axis is located in the initial three-dimensional coordinate system, and (X, Y, Z) in the equation 2 of the straight line where the Y axis is located is the three-dimensional coordinate of the corresponding point on the straight line where the Y axis is located in the initial three-dimensional coordinate system.

Then solving the origin of the target three-dimensional world coordinate system model by solving the intersection point of the straight line and the plane

From the formula (9), a plane equation of the passing point C1 (T1, T2, T3) perpendicular to the straight line 2 can be obtained, that is, the equation is obtained

A plane equation; the equation of the plane where the passing point C2 (0, 0) is perpendicular to the straight line 1 can also be obtained, i.e.

The plane equation:

（10）

in that

In the plane equation (X, Y, Z) is

Three-dimensional coordinates of the corresponding point on the plane in the initial three-dimensional coordinate system

In the plane equation (X, Y, Z) is

Three-dimensional coordinates of the corresponding point on the plane in the initial three-dimensional coordinate system.

Then solving the origin

：

Solving for the X-axis straight line and

plane or Y-axis straight line with

The point of intersection of the planes can obtain the origin

On the Y axis with

Taking the intersection point of the planes as an example, the equation set is constructed as follows:

（11）

(X, Y, Z) represents a Y-axis line and

and (3) three-dimensional coordinates of the intersection point of the planes in the initial three-dimensional coordinate system, and t is an intermediate variable of the calculation process.

Unfolding can obtain:

（12）

the value of t is obtained as:

（13）

substituting into formula (12) to obtain the original point of the new model

Coordinates of the object

。

Therefore, the equation of the line where the Z axis of the new model is located is as follows:

（14）

(X, Y, Z) represents the three-dimensional coordinates of the corresponding point on the straight line of the Z axis in the initial three-dimensional coordinate system.

(1-3): other situations. Obtaining one of the camera-to-point by physical measurement

Distance D of ₀ The following can be obtained:

（15）

representing camera C1 to point

The distance of (a) to (b),

representing camera C2 to point

The distance of (c).

（16）

（17）

（18）

If point

At a line C1C2 and in the plane

To the left or upper side of the vertical plane, take

Is greater than0 corresponding solution as a point

Coordinates;

if point

At a line C1C2 and in the plane

The right or lower edge of the vertical plane is taken

Solutions corresponding to less than 0 are taken as points

And (4) coordinates.

2) The symmetry axes of the two cameras do not pass through the center

In this case, it is necessary to physically measure the two cameras to the center

Distances D1, D2 of points. The distance between two cameras can be calculated by using a distance formula between two points, and can also be measured physically, and the distance is set as D3. Using the cosine theorem to find out the point

The distances D4 and D5 between the projection point C4 and the distances C1 and C2 on the line segment C1C2 are solved according to the distances D4 and D5 and the linear equation C1C2 to obtain the coordinate (X) of C4 _C4 ,Y _C4 ,Z _C4 ). The subsequent solution is the same as that in (1-3), except that the coordinate of C3 is changed to C4.

Thus obtaining the center

Coordinates of points

And then, combining the three direction vectors in the formula (5) to determine a linear equation of three axes, namely completing the construction of the target three-dimensional world coordinate system.

After the linear equations of the three axes are obtained, a point D in the space can be calculated (the three-dimensional coordinate obtained by calculating the point D in the initial three-dimensional world coordinate system model reconstructed for the first time is set as (X) _D ,Y _D ,Z _D ) Coordinates in a target three-dimensional world coordinate system model

The main principle is that the solution mode of the vertical distance from the point to the plane is utilized to respectively solve the point D to a new model

A plane surface,

A plane surface,

Distance of the plane:

the first step is as follows: respectively solving the over point D and

straight line perpendicular to plane

Intersection, passing point D of the planes and

straight line perpendicular to plane

Intersection, passing point D of the planes and

straight line with vertical planeAnd

intersection of planes:

（19）

the second step is that: calculating point D coordinates

Absolute value of (a):

（20）

the third step: calculating Point D coordinates

Positive and negative values of (c):

（21）

according to the principle, the three-dimensional coordinates of the object detection points in the target three-dimensional world coordinate system can be calculated according to the three-dimensional coordinates of the object detection points in the initial three-dimensional world coordinate system and the linear equations of the X axis, the Y axis and the Z axis in the target three-dimensional world coordinate system.

S104: and determining the relative positions of the object detection point and the target point according to the three-dimensional coordinates of the object detection point in the target three-dimensional world coordinate system.

Specifically, according to the coordinates of the target point in the initial three-dimensional world coordinate system and the linear equations of the first coordinate axis, the second coordinate axis and the third coordinate axis in the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system, the plane equations of the three coordinate planes in the target three-dimensional world coordinate system in the initial three-dimensional world coordinate system are determined. And then, respectively calculating the distances from the center point of the object to the three coordinate planes in the initial three-dimensional world coordinate system, and taking the distances as the coordinates of the center point of the object in the target three-dimensional world coordinate system to obtain the relative positions of the detection point and the target point of the object.

It can be seen that, in the object locating method disclosed in this embodiment, after acquiring coordinates of an object detection point in a pupil image pair obtained by shooting a target object with a first camera and a second camera, and determining three-dimensional coordinates of the object detection point in an initial three-dimensional world coordinate system, since the initial three-dimensional world coordinate system uses an optical center of one of the first camera and the second camera as an origin, a relative position between the object detection point and a target point cannot be determined intuitively.

Further, in a case where the target object is a pupil and the object detection point is a pupil center point, the embodiment further discloses a pupil diameter measurement method, which includes the following steps:

1. a first pupil image of a target object shot by a first camera and a second pupil image of the target object shot by a second camera are obtained.

2. And respectively preprocessing the first pupil image and the second pupil image to obtain a pupil area and edge point coordinates of the pupil area in the first pupil image and the second pupil image.

3. Respectively performing ellipse fitting on the edge point coordinates of the pupil areas in the first pupil image and the second pupil image to obtain the coordinates of the ellipse edge and the ellipse center point, i.e., the point P in fig. 14.

4. A horizontal line is determined that passes through the center point of the ellipse and the coordinates of the two intersections of the horizontal line with the edges of the ellipse are determined.

Specifically, the coordinates of two intersection points of the horizontal line and the ellipse edge can be determined according to the linear equation of the horizontal line passing through the ellipse center point and the edge point coordinates of the pupil area.

Referring to fig. 14, a horizontal line passing through the center point P of the ellipse, two intersections x1 and x2 of the horizontal line and the edge of the ellipse.

5. And respectively acquiring three-dimensional coordinates of the two intersection points in the initial three-dimensional world coordinate system or the target three-dimensional world coordinate system by using the object positioning method, and further calculating the diameter of the pupil.

Specifically, the actual distance between two intersection points is solved according to the three-dimensional coordinates, and the pupil diameter is obtained. Note that the pupil diameter here is not the pixel diameter, but an actual true diameter.

Based on the object positioning method disclosed in the foregoing embodiment, the present embodiment correspondingly discloses an object positioning apparatus, please refer to fig. 15, which includes:

an object detection point two-dimensional coordinate acquisition unit 101, configured to acquire a first object image captured by a first camera on a target object and a second object image captured by a second camera on the target object, and acquire object detection point coordinates in the first object image and the second object image;

an object detection point three-dimensional coordinate determination unit 102, configured to determine, according to object detection point coordinates in the first object image and the second object image, three-dimensional coordinates of an object detection point in an initial three-dimensional world coordinate system, where the initial three-dimensional world coordinate system is constructed with an optical center of the first camera or the second camera as an origin;

a three-dimensional coordinate conversion unit 103 configured to convert three-dimensional coordinates of the object detection points in the initial three-dimensional world coordinate system into three-dimensional coordinates in a target three-dimensional world coordinate system constructed with a target point as an origin;

a relative position determining unit 104, configured to determine a relative position between the object detection point and the target point according to the three-dimensional coordinates of the object detection point in the target three-dimensional world coordinate system.

In some embodiments, the object detection point two-dimensional coordinate acquisition unit 101 includes:

In some embodiments, the image pre-processing subunit comprises:

a blank image creating subunit, configured to create, for the blurred image, a first blank image, a second blank image, and a third blank image that are consistent with the size of the blurred image;

In some embodiments, the three-dimensional coordinate transformation unit 103 is specifically configured to:

and respectively calculating the distances from the object detection points to three coordinate planes in the initial three-dimensional world coordinate system as the coordinates of the object detection points in the target three-dimensional world coordinate system.

In some embodiments, the relative position determining unit 104 is specifically configured to: selecting a point A in the real world, selecting a point B in the horizontal direction of the passing point A, and selecting a point C in the vertical direction of the passing point A; detecting pixel coordinates of A, B and C in the first object image and pixel coordinates of A, B and C in the second object image; calculating three-dimensional coordinates of A, B and C in the initial three-dimensional world coordinate system according to the pixel coordinates of A, B and C in the first object image and the pixel coordinates of A, B and C in the second object image; taking the direction of the vector AB as the forward direction of a first coordinate axis of the target three-dimensional world coordinate system, and determining a direction vector of the first coordinate axis according to the three-dimensional coordinates of A and B; taking a vector CA as the forward direction of a second coordinate axis of the target three-dimensional world coordinate system, and determining a direction vector of the second coordinate axis according to the three-dimensional coordinates of C and A; and taking the cross product of the vector CA and the vector AB as the positive direction of a third coordinate axis of the target three-dimensional world coordinate system, and determining a direction vector of the third coordinate axis.

In the embodiment, after acquiring the coordinates of the object detection points of the first object image and the second object image obtained by shooting the target object by the first camera and the second camera, and determining the three-dimensional coordinates of the object detection points in the initial three-dimensional world coordinate system, because the initial three-dimensional world coordinate system uses the optical center of one of the first camera and the second camera as the origin, the relative positions of the object detection points and the target points cannot be determined intuitively.

Claims

1. An object positioning method, comprising:

determining the relative positions of the object detection point and the target point according to the three-dimensional coordinates of the object detection point in the target three-dimensional world coordinate system;

acquiring the coordinates of the object detection points in the first object image and the second object image, including:

respectively preprocessing the first object image and the second object image to obtain a target object area in the first object image and the second object image and edge point coordinates of the target object area;

respectively carrying out contour fitting on the edge point coordinates of the target object area in the first object image and the second object image and acquiring contour central point coordinates;

determining the contour central point coordinate corresponding to the first object image as an object detection point coordinate in the first object image, and determining the contour central point coordinate corresponding to the second object image as an object detection point coordinate in the second object image;

the preprocessing the first object image and the second object image respectively to obtain the target object area in the first object image and the second object image and the edge point coordinates of the target object area includes:

carrying out graying processing and Gaussian filtering processing on a target object image in sequence to obtain a fuzzy image, wherein the target object image is the first object image or the second object image;

calculating a global threshold value of the blurred image;

respectively calculating a line threshold value of each line in the blurred image, performing binarization processing on the line by taking the minimum value of the product of the line threshold value and a first coefficient and the product of the global threshold value and a second coefficient as a final threshold value, and putting the binarization processing result of each line in the blurred image into the corresponding line of the first blank image;

screening candidate regions of the segmented image;

2. The method of claim 1, wherein candidate region screening of the segmented image comprises:

3. The method of claim 1, wherein the determining the target object region in the target object image and the edge point coordinates of the target object region according to the candidate region screening result of the segmented image comprises:

4. The method of claim 1, wherein the method of constructing the initial three-dimensional world coordinate system comprises:

5. The method of claim 1, wherein the target three-dimensional world coordinate system is constructed by:

6. The method of claim 5, wherein converting the three-dimensional coordinates of the object detection point in the initial three-dimensional world coordinate system to three-dimensional coordinates in a target three-dimensional world coordinate system comprises:

7. The method of claim 5, wherein obtaining direction vectors of three coordinate axes in the target three-dimensional world coordinate system comprises:

8. An object positioning device, comprising:

an object detection point three-dimensional coordinate determination unit, configured to determine three-dimensional coordinates of object detection points in an initial three-dimensional world coordinate system according to object detection point coordinates in the first object image and the second object image, where the initial three-dimensional world coordinate system is constructed using an optical center of the first camera or the second camera as an origin;

a relative position determining unit, configured to determine a relative position between the object detection point and the target point according to a three-dimensional coordinate of the object detection point in the target three-dimensional world coordinate system;

the object detection point two-dimensional coordinate acquisition unit acquires object detection point coordinates in the first object image and the second object image, and includes:

calculating a global threshold value of the blurred image;

screening candidate regions of the segmentation image;