WO2023000528A1 - Map positioning method and apparatus, computer-readable storage medium and terminal device - Google Patents

Abstract

The present application belongs to the field of positioning technology, and in particular relates to a map positioning method and apparatus, a computer-readable storage medium and a terminal device. The method comprises: by means of a camera apparatus, acquiring a global image of a target moving apparatus traveling on a map; detecting an identification code of the map in the global image, and determining the height of the camera apparatus from the map according to the detection result of the identification code of the map; determining the positions of corner points of the contour of the map in the global image, and determining the scale of the global image according to the positions of the corner points and a preset actual map size; detecting an identification code of the target moving apparatus in the global image, and determining the center point of the identification code of the target moving apparatus according to the detection result of the identification code of the target moving apparatus; and determining the actual position of the target moving apparatus in the map according to the center point of the identification code of the target moving apparatus, the positions of the corner points of the contour of the map, and the height and scale of the camera apparatus from the map.

Description

Map positioning method, device, computer readable storage medium and terminal equipment

This application claims priority to a Chinese patent application with application number 202110837305.8 filed at the China Patent Office on July 23, 2021, the entire contents of which are incorporated herein by reference.

technical field

The present application belongs to the technical field of positioning, and in particular relates to a map positioning method, device, computer-readable storage medium and terminal equipment.

Background technique

In the prior art, moving devices such as cars can be controlled to walk along the black lines on the white ground through the technical scheme of infrared line inspection. When the car is moving, it continuously emits infrared light to the ground. When the infrared light meets the white ground, diffuse reflection occurs, and the reflected light is received by the receiving tube on the car; if it encounters a black line, the infrared light will be absorbed. The receiving tube cannot receive infrared light. The single-chip microcomputer system on the trolley determines the position of the black line and the walking route of the trolley based on whether it receives the reflected infrared light or not. Although this method is relatively simple, it cannot effectively locate the position of the trolley, and the trolley can only travel along a preset fixed route, with extremely poor flexibility.

technical problem

In view of this, the embodiment of the present application provides a map positioning method, device, computer-readable storage medium, and terminal equipment to solve the problem that the existing technology cannot effectively locate the position of the trolley, and the trolley can only be positioned along a pre-set The fixed route travels, the problem of extremely poor flexibility.

technical solution

The first aspect of the embodiments of the present application provides a map positioning method, which may include:

Obtain a global image of the target moving device traveling on the map through a preset camera; the camera is located at the top of the map; the target moving device and the map are provided with corresponding identification codes;

Detecting the identification code of the map in the global image, and determining the height of the camera device from the map according to the detection result of the identification code of the map;

determining the corner position of the outline of the map in the global image, and determining the scale of the global image according to the corner position and a preset actual map size;

Detecting the identification code of the target moving device in the global image, and determining the center point of the identification code of the target moving device according to the detection result of the identification code of the target moving device;

Determine the actual position of the target mobile device in the map according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height of the camera device from the map, and the scale .

In a specific implementation of the first aspect, the determining the corner position of the outline of the map in the global image may include:

performing grayscale processing on the global image to obtain a grayscale image;

Performing Gaussian blur processing on the grayscale image to obtain a Gaussian blurred image;

performing expansion processing on the Gaussian blurred image to obtain an expanded image;

performing edge detection on the expanded image to obtain the edge of the map;

performing contour detection on the edge of the map to obtain the contour of the map;

Perform corner location on the outline of the map to obtain the corner position of the outline of the map.

In a specific implementation of the first aspect, the determining the scale of the global image according to the corner position and the preset actual map size may include:

determining the pixel size of the map in the global image according to the corner position;

calculating the width ratio of the actual map size to the pixel size;

calculating the height ratio of the actual map size to the pixel size;

The mean value of the width ratio and the height ratio is determined as the scale of the global image.

In a specific implementation of the first aspect, the determination is based on the center point of the identification code of the target moving device, the corner position of the outline of the map, the height of the camera device from the map, and the scale The actual location of the target mobile device in the map may include:

determining a first position of the target mobile device in the map according to the center point of the identification code of the target mobile device and the corner position of the outline of the map;

correcting the first position according to the height of the camera from the map and the preset height of the target moving device to obtain a second position of the target moving device in the map;

An actual position of the target mobile device in the map is determined based on a second position of the target mobile device in the map and the scale.

In a specific implementation of the first aspect, the first position is corrected according to the height of the camera device from the map and the preset height of the target mobile device to obtain the target mobile device at The second location in the map may include:

The first position is corrected according to the following formula:

Wherein, (x ^* , y ^* ) is the coordinate of the first position, h _m is the height of the camera device from the map, h _c is the preset height of the target moving device, (x, y) is the coordinates of the second location.

In a specific implementation of the first aspect, the determining the actual position of the target mobile device in the map according to the second position of the target mobile device in the map and the scale may include:

The actual position of the target mobile device in the map is calculated according to the following formula:

Wherein, (x, y) is the coordinate of the second position, s is the scale, W is the width of the actual map size, H is the height of the actual map size, (x ^m , y ^m ) is The coordinates of the actual location.

In a specific implementation of the first aspect, the acquisition of the global image of the target moving device traveling on the map through the preset camera device may include:

Carrying out camera calibration on the camera device to obtain camera internal parameters and distortion coefficients of the camera device;

Obtaining a global original image of the target moving device traveling on the map through the camera device;

De-distorting the global original image according to the internal camera parameters and the distortion coefficient to obtain the global image.

The second aspect of the embodiments of the present application provides a map positioning device, which may include:

The image acquisition module is used to obtain the global image of the target moving device traveling on the map through the preset camera device; the camera device is located at the top of the map; the target moving device and the map are provided with corresponding identification code;

A map identification code detection module, configured to detect the identification code of the map in the global image, and determine the height of the camera device from the map according to the detection result of the identification code of the map;

a corner position determining module, configured to determine the corner position of the outline of the map in the global image;

A scale determination module, configured to determine the scale of the global image according to the corner position and the preset actual map size;

A device identification code detection module, configured to detect the identification code of the target moving device in the global image, and determine the center point of the identification code of the target moving device according to the detection result of the identification code of the target moving device;

A position determination module, configured to determine the position of the target mobile device at the target mobile device according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height of the camera device from the map, and the scale actual location on the map.

In a specific implementation of the second aspect, the corner position determining module may include:

a grayscale unit, configured to perform grayscale processing on the global image to obtain a grayscale image;

a Gaussian blur unit, configured to perform Gaussian blur processing on the grayscale image to obtain a Gaussian blurred image;

an expansion unit, configured to perform expansion processing on the Gaussian blurred image to obtain an expanded image;

an edge detection unit, configured to perform edge detection on the dilated image to obtain an edge of the map;

a contour detection unit, configured to perform contour detection on the edge of the map to obtain the contour of the map;

The corner positioning unit is configured to perform corner positioning on the outline of the map to obtain corner positions of the outline of the map.

In a specific implementation of the second aspect, the scale determination module may include:

a pixel size determining unit, configured to determine the pixel size of the map in the global image according to the corner position;

a width ratio calculation unit, configured to calculate a width ratio between the actual map size and the pixel size;

a height ratio calculation unit, configured to calculate the height ratio between the actual map size and the pixel size;

A scale determining unit, configured to determine an average value of the width ratio and the height ratio as the scale of the global image.

In a specific implementation of the second aspect, the position determination module may include:

A first position determining unit, configured to determine a first position of the target mobile device in the map according to the center point of the identification code of the target mobile device and the corner position of the outline of the map;

The second position determination unit is configured to correct the first position according to the height of the camera device from the map and the preset height of the target mobile device, to obtain the position of the target mobile device in the map second position;

An actual position determining unit, configured to determine the actual position of the target mobile device in the map according to the second position of the target mobile device in the map and the scale.

In a specific implementation of the second aspect, the second position determining unit is specifically configured to correct the first position according to the following formula:

Wherein, (x ^* , y ^* ) is the coordinate of the first position, h _m is the height of the camera device from the map, h _c is the preset height of the target moving device, (x, y) is the coordinates of the second location.

In a specific implementation of the second aspect, the actual position determination unit is specifically configured to calculate the actual position of the target mobile device in the map according to the following formula:

Wherein, (x, y) is the coordinate of the second position, s is the scale, W is the width of the actual map size, H is the height of the actual map size, (x ^m , y ^m ) is The coordinates of the actual location.

In a specific implementation of the second aspect, the image acquisition module may include:

A camera calibration unit, configured to perform camera calibration on the camera device to obtain camera internal parameters and distortion coefficients of the camera device;

an original image acquisition unit, configured to acquire a global original image of the target moving device traveling on the map through the camera device;

A de-distortion unit, configured to perform de-distortion processing on the global original image according to the internal camera parameters and the distortion coefficients, to obtain the global image.

A third aspect of the embodiments of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any one of the above-mentioned map positioning methods are implemented.

The fourth aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the computer program Steps for implementing any one of the above-mentioned map positioning methods.

A fifth aspect of the embodiments of the present application provides a computer program product, which, when the computer program product is run on a terminal device, causes the terminal device to execute the steps of any one of the above map positioning methods.

Beneficial effect

Compared with the prior art, the embodiment of the present application has the beneficial effect that the embodiment of the present application pre-sets the camera device on the top of the map on which the target moving device travels, and sets corresponding identification codes on both the target moving device and the map Obtain the global image of the target moving device on the map through the camera; detect the identification code of the map in the global image, and determine the height of the camera from the map according to the detection result of the identification code of the map; determine the map in the global image The corner position of the outline, and determine the scale of the global image according to the corner position and the preset actual map size; detect the identification code of the target mobile device in the global image, and determine it according to the detection result of the identification code of the target mobile device The center point of the identification code of the target mobile device; determine the actual position of the target mobile device in the map according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height and the scale of the camera from the map, so as to realize the Precise positioning of target motion devices. On this basis, the target motion device can be controlled to complete various complex tasks, instead of simply performing simple line inspection tasks along a preset fixed route, which has extremely high flexibility.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only for the present application For some embodiments, those skilled in the art can also obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flow chart of an embodiment of a map positioning method in the embodiment of the present application;

Fig. 2 is the schematic flowchart of obtaining the global image of the target moving device moving on the map through the preset camera device;

Fig. 3 is an example diagram of the identification code of the map;

Fig. 4 is a schematic flowchart of determining the corner position of the outline of the map in the global image;

Fig. 5 is a schematic flowchart of determining the scale of the global image according to the corner position and the preset actual map size;

Fig. 6 is a schematic flowchart of determining the actual position of the target mobile device in the map;

Fig. 7 is a schematic diagram of the relationship between trigonometric functions for position correction;

FIG. 8 is a structural diagram of an embodiment of a map positioning device in the embodiment of the present application;

FIG. 9 is a schematic block diagram of a terminal device in an embodiment of the present application.

Embodiments of the present invention

In order to make the purpose, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the following The described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude one or more other features. , whole, step, operation, element, component and/or the presence or addition of a collection thereof.

It should also be understood that the terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural referents unless the context clearly dictates otherwise.

It should also be further understood that the term "and/or" used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .

As used in this specification and the appended claims, the term "if" may be construed as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context . Similarly, the phrases "if determined" or "if [the described condition or event is detected]" may be construed, depending on the context, to mean "once determined" or "in response to the determination" or "once detected [the described condition or event] ]” or “in response to detection of [described condition or event]”.

In addition, in the description of the present application, terms such as "first", "second", and "third" are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

Referring to Fig. 1, an embodiment of a map positioning method in the embodiment of the present application may include:

Step S101 , acquiring a global image of the target moving device traveling on the map through a preset camera device.

In the embodiment of the present application, the target motion device may be various motion devices such as a car, a drone, or a robot. The map refers to the ground area where the target mobile device travels. For example, a square sand table can be constructed in advance, and the target mobile device travels on the sand table and performs various specified tasks, then the sand table is a map. The camera device is located on the top of the map. For example, the camera device can be installed on the ceiling of the room where the map is located, so that the camera device can overlook the entire map from top to bottom and collect a complete image of the map. Corresponding identification codes are provided on the target moving device and the map to facilitate its identification and positioning. The identification code can be any code pattern that can be identified and located in the prior art. Preferably, a two-dimensional code as an identification code.

In a specific implementation of the embodiment of the present application, step S101 may specifically include the process shown in Figure 2:

Step S1011 , perform camera calibration on the camera device, and obtain camera intrinsic parameters and distortion coefficients of the camera device.

The specific camera calibration method used can be set according to the actual situation. In the embodiment of the present application, the checkerboard calibration method of Zhang Zhengyou can be preferably used to obtain the internal camera reference (denoted as mtx) and distortion coefficient (denoted as dist) of the camera device.

Step S1012, acquire the global original image of the target moving device traveling on the map through the camera device.

Wherein, the global original image is the original image directly collected by the camera device without past distortion processing. When the camera device is collecting real-time video streams, each frame of image in the video stream is a global original image.

Step S1013, performing de-distortion processing on the global original image according to the internal camera parameters and distortion coefficients to obtain a global image.

In order to prevent image distortion from affecting the effect of subsequent two-dimensional code detection and map detection, the embodiment of the present application may perform de-distortion processing on each frame of the global original image.

It should be noted that the main functions used for distortion correction of distorted images in OpenCV are the UndistortImage function, and the initUndistortRectifyMap combined with the remap function. And UndistortImage is a simple combination of initUndistortRectifyMap and remap, the effect is the same. However, since the distortion coordinate mapping matrices mapx and mapy only need to be calculated once, repeated calls to UndistortImage will repeatedly calculate mapx and mapy, which seriously affects the processing efficiency. Therefore, for the real-time video stream input by the camera device, initUndistortRectifyMap can be used only once to obtain the distortion coordinate mapping matrices mapx and mapy as the input of the remap function, and then repeatedly call the remap function for each frame of image in the video stream to perform distortion correction. Can.

Step S102: Detect the identification code of the map in the global image, and determine the height of the camera device from the map according to the detection result of the identification code of the map.

In this embodiment of the application, an identification code may be pasted on the upper left corner of the map in advance. Taking the two-dimensional code as an example, its various parameters such as the number of grids, the side length of the two-dimensional code and the label (marked as id) can be set according to the actual situation. Figure 3 shows the 4×4 grid, the side length of the two-dimensional code A QR code with a size of 100mm and an id of 1.

In the embodiment of this application, the aruco library of OpenCV can be used to detect the QR code of the map in the global image. Specifically, through the aruco.detectMarkers function of OpenCV, the coordinates and id of the four corners of the two-dimensional code can be obtained, which represents the current two-dimensional code, and then the coordinates of the four corners are input to OpenCV's aruco. The estimatePoseSingleMarkers function solves the equation by PnP (Perspective-n-Point) to obtain the rotation matrix (denoted as rvec) and translation matrix (denoted as tvec) of the two-dimensional code. The third parameter of tvec is the translation amount of the z-axis. Represents the height of the camera from the map (marked as map_h).

Step S103, determine the corner position of the outline of the map in the global image, and determine the scale of the global image according to the corner position and the preset actual map size.

In a specific implementation of the embodiment of the present application, the corner position of the outline of the map can be determined through the process shown in Figure 4:

Step S1031 , performing grayscale processing on the global image to obtain a grayscale image.

Compared with the original RGB image, grayscale images are processed faster. After grayscale, the matrix dimension decreases, the operation speed is greatly improved, and important gradient information is still retained. In the embodiment of this application, the cvtColor function of OpenCV can be used to perform color space conversion (BGR, HSV, GRAY and other color spaces) on the global image, and the parameters are converted to grayscale using COLOR_BGR2GRAY.

Step S1032 , performing Gaussian blur processing on the grayscale image to obtain a Gaussian blurred image.

Gaussian blur is a low-pass filter for images, which can make the image blurred and smooth, and is usually used to reduce image noise and reduce the level of detail. In the embodiment of the present application, the Gaussian Blur method of OpenCV can be used to perform Gaussian blur processing on the image.

Step S1033 , performing dilation processing on the Gaussian blurred image to obtain the dilated image.

In the embodiment of the present application, the dilate function of OpenCV can be used to dilate the image. Because the map is a square, the edge may lose pixels after de-distortion, resulting in blurring. Therefore, the edge of the map is expanded by expansion, and the edge of the map target or the smaller holes inside are filled in, so that the edge of the map is slightly changed. Large (to prevent disconnection) effect, which is beneficial to subsequent edge detection.

Step S1034, performing edge detection on the dilated image to obtain the edge of the map.

In the embodiment of this application, the Canny function of OpenCV can be used to perform edge detection on the image, and finally an output binary image is obtained, which contains the edges existing in the image, from which the map edges can be obtained.

Step S1035, performing contour detection on the edge of the map to obtain the contour of the map.

In the embodiment of the present application, the findContours function of OpenCV can be used to find the contour, and the parameter cv2.RETR_EXTERNAL indicates that only the outer contour is detected, that is, the outer circle contour of the map required by the embodiment of the present application. The contour (denoted as contour) is represented by a series of points, but not all points on the contour are stored, only the number of points that can describe the contour with a straight line is stored. For a map, only 4 points are needed to describe its outline, and these 4 points are the corner points of the outline of the map.

After obtaining the outline, because there may be more than one outline of the map in the image, there may be other graphics, but it is certain that the outline of the map is the largest among them, so by sorting the outlines, the largest outline is found to be the outline of the map.

After finding the outline of the map, the outline of the map can be approximated by the arcLength function and approxPolyDP function of OpenCV, and the outline drawing operation can be performed using drawCountors. The arcLength function is used to calculate the perimeter of the closed contour or the length of the curve, and aprroxPolyDP(cnt, epsilon, True) is used to obtain the approximate value of the contour, where cnt is the input contour value; epsilon is the threshold T, usually using the circumference of the contour Long as the threshold; True means the contour is closed.

Step S1036, perform corner location on the outline of the map to obtain the corner position of the outline of the map.

The purpose of corner positioning is to arrange the positions of the four corner points, generally in the clockwise direction of upper left, upper right, lower right and lower left.

Specifically, the sum of the abscissa and ordinate of the four corner points can be calculated separately, with the smallest sum being the upper left corner, and the largest sum being the lower right corner, as shown in the following formula:

S _i =x _i +y _i

Among them, i represents the calculation number of the corner point, x _i is the abscissa of the i-th corner point, y _i is the ordinate of the i-th corner point, S _i is the sum of the abscissa and ordinate of the i-th corner point , argmax represents the maximum value, argmin represents the minimum value, id is the order of the corner points sorted in order, id=0 represents the upper left corner point, id=1 represents the upper right corner point, id=2 represents the lower right corner point The corner point of , id=3 represents the bottom left corner point.

Next, continue to sort the remaining two corner points, respectively calculate the absolute value of the difference between the abscissa and ordinate of the two corner points, the smaller value is the upper right corner, and the larger value is the lower left corner , as shown in the following formula:

D _i ＝| _xi -y _i |

Among them, D _i is the absolute value of the difference between the abscissa and ordinate of the i-th corner point.

After the above calculations are completed, the corner positions of the outlines of the maps sorted clockwise (ids are sorted from 0 to 3) can be obtained.

After determining the corner position of the outline of the map in the global image, the scale of the global image can be determined according to the corner position and the actual map size, so that the pixel coordinates calculated in the subsequent image can be converted into the distance on the actual map through the scale. For example, a scale of s=2 indicates that 1 pixel in the image represents an actual distance of 2 mm in the map.

As shown in Figure 5, determining the scale of the global image according to the corner position and the actual map size may specifically include the following steps:

Step S1037. Determine the pixel size of the map in the global image according to the corner position.

In the embodiment of this application, the length of the uppermost side of the map in the global image (denoted as w1, the unit is pixel) can be obtained through the upper left and upper right corner points, and the maximum length of the map can be obtained through the lower left and right corner points. The length of the following side in the global image (denoted as w2, the unit is pixel), take the mean value w of both w1 and w2 as the width of the map in the global image. In the same way, the length of the leftmost side of the map in the global image can be obtained through the upper left and lower left corners (denoted as h1, and the unit is pixel), and the rightmost side of the map can be found in the global image through the upper right and lower right corners. The length in the image (denoted as h2, the unit is pixel), take the mean value h of both h1 and h2 as the height of the map in the global image. When the map is square, w and h are relatively close.

Step S1038, calculate the width ratio between the actual map size and the pixel size, and calculate the height ratio between the actual map size and the pixel size, as shown in the following formula:

Among them, _W and _H are the width and height of the actual map, respectively, sw is the width ratio of the actual map size to the pixel size, and sh is the height ratio of the actual map size to the pixel size.

Step S1039, determining the mean value of the width ratio and the height ratio as the scale of the global image.

When the map is square, s _w and s _h are relatively close, and the mean value of the two can be determined as the scale of the global image (denoted as s).

So far, the step of map detection is completed. At this time, the four corner points of the outline of the map are obtained, and the actual distance of the map and its scale in the global image are obtained.

Step S104: Detect the identification code of the target mobile device in the global image, and determine the center point of the identification code of the target mobile device according to the detection result of the identification code of the target mobile device.

In the embodiment of the present application, the identification code of the target mobile device may be pasted on the upper surface of the target mobile device to facilitate the camera device to capture, and the identification code may be used to represent the position of the target mobile device.

The process of detecting the identification code of the target moving device is similar to the process of detecting the identification code of the map in step S102. The coordinates and id of the four corner points are mainly obtained through the aruco.detectMarkers function of OpenCV, but OpenCV is not required at this time The aruco.estimatePoseSingleMarkers function solves PnP to obtain the rotation matrix rvec and the translation matrix tvec, but directly calculates the coordinates of the center point of the identification code of the target moving device through the coordinates of the four corner points. For example, the upper left corner point and Calculate the mean value of the coordinates of the corner point on the lower right, and the mean value is the coordinate of the center point of the identification code of the target moving device.

Step S105: Determine the actual position of the target mobile device on the map according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height of the camera device from the map, and the scale.

In a specific implementation of the embodiment of the present application, step S105 may specifically include the process shown in Figure 6:

Step S1051. Determine the first position of the target mobile device in the map according to the center point of the identification code of the target mobile device and the corner position of the outline of the map.

In this embodiment of the application, the intersection point of the two sides of the map can be used as the origin of the map coordinate system. For example, the intersection point of the two sides on the upper side and the left side of the map can be used as the origin of the map coordinate system. The positive direction of the x-axis is from left to right, and the positive direction of the y-axis is from top to bottom along the left side. By calculating the distances between the target mobile device and the two sides, the position of the target mobile device in the global image can be converted to a position relative to the map, that is, a position in the map coordinate system.

First, it is necessary to determine whether the target mobile device is located on the left side or the right side of the left side of the map, and whether the target mobile device is located on the upper side or the lower side of the upper side side of the map. The calculation formula is as follows:

T _l ＝(y ₀ -y ₃ )*x _c +(x ₃ -x ₀ )*y _c +(x ₀ *y ₃ )-(x ₃ *y ₀ )

T _t ＝(y ₀ -y ₃ )*x _c +(x ₃ -x ₀ )*y _c +(x ₀ *y ₃ )-(x ₃ *y ₀ )

Wherein, x _c is the abscissa of the central point of the identification code of the target mobile device, and y _c is the vertical coordinate of the central point of the identification code of the target mobile device. If T _l is greater than 0, the target mobile device is located on the left side of the left side of the map, otherwise, the target mobile device is located on the right side of the left side of the map; if T _t is greater than 0, then the target mobile device is located on the upper side of the map The upper side of the edge, otherwise, the target mobile device is located on the lower side of the edge on the upper side of the map.

Then, the distances of the target motion device from the two edges can be calculated separately. Taking the side on the left as an example, the distance can be calculated from the center point of the identification code of the target moving device and two points on the side, and the distance from a point to a straight line can be converted into the height of a triangle, and the height of a triangle can be calculated by vector cross product The specific calculation of the area is shown in the following formula:

vec ₁ =P ₁ -P _m

vec ₂ =P ₂ -P _m

d=np.abs(np.cross(vec ₁ ,vec ₂ ))/np.linalg.norm(P ₁ -P ₂ )

Among them, P ₁ represents the first point of the side, P ₂ represents the second point of the side, P _m represents the center point of the identification code of the target motion device, vec ₁ represents the vector from P _m to P ₁ , vec ₂ Represents the vector from P _m to P ₂ , np.abs is the function to find the absolute value, np.cross is the function to find the vector cross product, np.linalg.norm is the function to find the norm, d is the target movement device and the left The distance from the edge of the side.

The calculation process of the distance between the target moving device and the upper side is similar, and reference may be made to the above content, which will not be repeated here.

In determining whether the target mobile device is to the left or right of the edge on the left side of the map, whether the target mobile device is above or below the edge on the upper side of the map, the distance of the target mobile device from the left edge, and the target motion After determining the distance between the device and the upper side, the first position of the target mobile device in the map can be determined.

Step S1052 , correcting the first position according to the height of the camera device from the map and the preset height of the target mobile device to obtain a second position of the target mobile device on the map.

As shown in Figure 7, since the target moving device has a certain height difference from the ground, there is a deviation in the light projection, where O ₁ is the optical center position of the camera device, O ₃ is the intersection point of the vertical line from O ₁ to the map, p ₁ is the second position, that is, the actual position of the target moving device in the map, but due to the height difference and light propagation, the imaging in the image is p ₃ , which is the first position, and p ₂ is from p ₁ to the map Make the intersection point of the vertical line, the horizontal and vertical coordinates of p ₁ and p ₂ are the same, O ₂ is the intersection point of the vertical line from p ₁ to the line segment O ₁ O ₃ , the horizontal and vertical coordinates of O ₁ , O ₂ and O ₃ are the same.

In the embodiment of the present application, the position correction can be performed according to the trigonometric function relationship in Figure 7, and the specific calculation formula is as follows:

Among them, (x ^* , y ^* ) is the coordinate of the first position, h _m is the height of the camera device from the map, h _c is the preset height of the target mobile device, that is, the distance from the center point of the identification code of the target mobile device to the map Height, (x,y) is the coordinate of the second position.

Step S1053. Determine the actual position of the target mobile device on the map according to the second position of the target mobile device on the map and the scale.

Specifically, the actual position of the target mobile device in the map can be calculated according to the following formula:

Wherein, (x ^m , y ^m ) are the coordinates of the actual position of the target moving device in the map coordinate system. So far, the map detection, the construction of the map coordinate system, and the calculation of the actual position of the target moving device on the map coordinate system have all been completed. For each frame of image, the actual position of the target moving device in the map coordinate system can be obtained in real time. On this basis, the target motion device can be controlled to complete various complex tasks, instead of simply performing simple line inspection tasks along a preset fixed route, which has extremely high flexibility.

To sum up, in the embodiment of the present application, a camera device is set in advance on the top of the map where the target mobile device travels, and corresponding identification codes are set on both the target mobile device and the map; The global image of progress; the identification code of the map is detected in the global image, and the height of the camera device from the map is determined according to the detection result of the identification code of the map; the corner position of the contour of the map is determined in the global image, and according to the corner point The position and the preset actual map size determine the scale of the global image; detect the identification code of the target mobile device in the global image, and determine the center point of the identification code of the target mobile device according to the detection result of the target mobile device; according to the target The center point of the identification code of the moving device, the corner position of the outline of the map, the height and the scale of the camera device from the map determine the actual position of the target moving device in the map, so that the precise positioning of the target moving device can be realized. On this basis, the target motion device can be controlled to complete various complex tasks, instead of simply performing simple line inspection tasks along a preset fixed route, which has extremely high flexibility.

It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

Corresponding to the map positioning method described in the above embodiments, FIG. 8 shows a structural diagram of an embodiment of a map positioning device provided in the embodiment of the present application.

In this embodiment, a map positioning device may include:

The image acquisition module 801 is used to obtain a global image of the target moving device traveling on the map through a preset camera device; the camera device is located at the top of the map; the target moving device and the map are provided with corresponding the identification code;

A map identification code detection module 802, configured to detect the identification code of the map in the global image, and determine the height of the camera device from the map according to the detection result of the identification code of the map;

A corner position determination module 803, configured to determine the corner position of the outline of the map in the global image;

A scale determination module 804, configured to determine the scale of the global image according to the corner position and the preset actual map size;

A device identification code detection module 805, configured to detect the identification code of the target moving device in the global image, and determine the center point of the identification code of the target moving device according to the detection result of the identification code of the target moving device ;

A position determination module 806, configured to determine the location of the target mobile device according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height of the camera device from the map, and the scale. The actual location on the map.

In a specific implementation of the embodiment of the present application, the corner position determination module may include:

a grayscale unit, configured to perform grayscale processing on the global image to obtain a grayscale image;

a Gaussian blur unit, configured to perform Gaussian blur processing on the grayscale image to obtain a Gaussian blurred image;

an expansion unit, configured to perform expansion processing on the Gaussian blurred image to obtain an expanded image;

an edge detection unit, configured to perform edge detection on the dilated image to obtain an edge of the map;

a contour detection unit, configured to perform contour detection on the edge of the map to obtain the contour of the map;

The corner positioning unit is configured to perform corner positioning on the outline of the map to obtain corner positions of the outline of the map.

In a specific implementation of the embodiment of the present application, the scale determination module may include:

a pixel size determining unit, configured to determine the pixel size of the map in the global image according to the corner position;

a width ratio calculation unit, configured to calculate a width ratio between the actual map size and the pixel size;

a height ratio calculation unit, configured to calculate the height ratio between the actual map size and the pixel size;

A scale determining unit, configured to determine an average value of the width ratio and the height ratio as the scale of the global image.

In a specific implementation of the embodiment of the present application, the location determination module may include:

A first position determining unit, configured to determine a first position of the target mobile device in the map according to the center point of the identification code of the target mobile device and the corner position of the outline of the map;

The second position determination unit is configured to correct the first position according to the height of the camera device from the map and the preset height of the target mobile device, to obtain the position of the target mobile device in the map second position;

An actual position determining unit, configured to determine the actual position of the target mobile device in the map according to the second position of the target mobile device in the map and the scale.

In a specific implementation of the embodiment of the present application, the second position determining unit is specifically configured to correct the first position according to the following formula:

Wherein, (x ^* , y ^* ) is the coordinate of the first position, h _m is the height of the camera device from the map, h _c is the preset height of the target moving device, (x, y) is the coordinates of the second location.

In a specific implementation of the embodiment of the present application, the actual position determining unit is specifically configured to calculate the actual position of the target moving device in the map according to the following formula:

Wherein, (x, y) is the coordinate of the second position, s is the scale, W is the width of the actual map size, H is the height of the actual map size, (x ^m , y ^m ) is The coordinates of the actual location.

In a specific implementation of the embodiment of the present application, the image acquisition module may include:

A camera calibration unit, configured to perform camera calibration on the camera device to obtain camera internal parameters and distortion coefficients of the camera device;

an original image acquisition unit, configured to acquire a global original image of the target moving device traveling on the map through the camera device;

A de-distortion unit, configured to perform de-distortion processing on the global original image according to the internal camera parameters and the distortion coefficients, to obtain the global image.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described devices, modules and units can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

FIG. 9 shows a schematic block diagram of a terminal device provided by an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.

As shown in FIG. 9 , the terminal device 9 of this embodiment includes: a processor 90 , a memory 91 , and a computer program 92 stored in the memory 91 and operable on the processor 90 . When the processor 90 executes the computer program 92, it realizes the steps in the above embodiments of the map positioning method, for example, steps S101 to S105 shown in FIG. 1 . Alternatively, when the processor 90 executes the computer program 92, the functions of the modules/units in the above-mentioned device embodiments are realized, for example, the functions of the modules 801 to 806 shown in FIG. 8 .

Exemplarily, the computer program 92 can be divided into one or more modules/units, and the one or more modules/units are stored in the memory 91 and executed by the processor 90 to complete this application. The one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 92 in the terminal device 9 .

The terminal device 9 may be a computing device such as a mobile phone, a tablet computer, a desktop computer, a notebook, a palmtop computer, and a robot. Those skilled in the art can understand that FIG. 9 is only an example of the terminal device 9, and does not constitute a limitation to the terminal device 9. It may include more or less components than those shown in the figure, or combine certain components, or different components. For example, the terminal device 9 may also include an input and output device, a network access device, a bus, and the like.

The processor 90 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The storage 91 may be an internal storage unit of the terminal device 9 , such as a hard disk or memory of the terminal device 9 . The memory 91 can also be an external storage device of the terminal device 9, such as a plug-in hard disk equipped on the terminal device 9, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Further, the memory 91 may also include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store the computer program and other programs and data required by the terminal device 9 . The memory 91 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units, Completion of modules means that the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit, and the above-mentioned integrated units may adopt hardware It can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the above system, reference may be made to the corresponding process in the foregoing method embodiments, and details will not be repeated here.

In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated module/unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments in the present application can also be completed by instructing related hardware through computer programs. The computer programs can be stored in a computer-readable storage medium, and the computer When the program is executed by the processor, the steps in the above-mentioned various method embodiments can be realized. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) ), Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable Storage media excludes electrical carrier signals and telecommunication signals.

The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still implement the foregoing embodiments Modifications to the technical solutions described in the examples, or equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application, and should be included in the Within the protection scope of this application.

Claims (10)

1. A map positioning method, characterized in that, comprising:

   Obtain a global image of the target moving device traveling on the map through a preset camera; the camera is located at the top of the map; the target moving device and the map are provided with corresponding identification codes;

   Detecting the identification code of the map in the global image, and determining the height of the camera device from the map according to the detection result of the identification code of the map;

   determining the corner position of the outline of the map in the global image, and determining the scale of the global image according to the corner position and a preset actual map size;

   Detecting the identification code of the target moving device in the global image, and determining the center point of the identification code of the target moving device according to the detection result of the identification code of the target moving device;

   Determine the actual position of the target mobile device in the map according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height of the camera device from the map, and the scale .
2. The map positioning method according to claim 1, wherein the determining the corner position of the outline of the map in the global image comprises:

   performing grayscale processing on the global image to obtain a grayscale image;

   Performing Gaussian blur processing on the grayscale image to obtain a Gaussian blurred image;

   performing expansion processing on the Gaussian blurred image to obtain an expanded image;

   performing edge detection on the expanded image to obtain the edge of the map;

   performing contour detection on the edge of the map to obtain the contour of the map;

   Perform corner location on the outline of the map to obtain the corner position of the outline of the map.
3. The map positioning method according to claim 1, wherein the determining the scale of the global image according to the corner position and the preset actual map size includes:

   determining the pixel size of the map in the global image according to the corner position;

   calculating the width ratio of the actual map size to the pixel size;

   calculating the height ratio of the actual map size to the pixel size;

   The mean value of the width ratio and the height ratio is determined as the scale of the global image.
4. The map positioning method according to claim 1, wherein the center point of the identification code according to the target moving device, the corner position of the outline of the map, the height of the camera device from the map, and The scale determines the actual location of the target mobile device on the map, including:

   determining a first position of the target mobile device in the map according to the center point of the identification code of the target mobile device and the corner position of the outline of the map;

   correcting the first position according to the height of the camera from the map and the preset height of the target moving device to obtain a second position of the target moving device in the map;

   An actual position of the target mobile device in the map is determined based on a second position of the target mobile device in the map and the scale.
5. The map positioning method according to claim 4, wherein the first position is corrected according to the height of the camera device from the map and the preset height of the target moving device to obtain the A second location of the target mobile device in said map, comprising:

   The first position is corrected according to the following formula:

   

   

   Wherein, (x ^* , y ^* ) is the coordinate of the first position, h _m is the height of the camera device from the map, h _c is the preset height of the target moving device, (x, y) is the coordinates of the second location.
6. The map positioning method according to claim 4, wherein the actual position of the target moving device in the map is determined according to the second position of the target moving device in the map and the scale ,include:

   The actual position of the target mobile device in the map is calculated according to the following formula:

   

   

   Wherein, (x, y) is the coordinate of the second position, s is the scale, W is the width of the actual map size, H is the height of the actual map size, (x ^m , y ^m ) is The coordinates of the actual location.
7. The map positioning method according to any one of claims 1 to 6, wherein the acquisition of a global image of the target moving device traveling on the map through a preset camera device includes:

   Carrying out camera calibration on the camera device to obtain camera internal parameters and distortion coefficients of the camera device;

   Obtaining a global original image of the target moving device traveling on the map through the camera device;

   De-distorting the global original image according to the internal camera parameters and the distortion coefficient to obtain the global image.
8. A map positioning device is characterized in that it comprises:

   The image acquisition module is used to obtain the global image of the target moving device traveling on the map through the preset camera device; the camera device is located at the top of the map; the target moving device and the map are provided with corresponding identification code;

   A map identification code detection module, configured to detect the identification code of the map in the global image, and determine the height of the camera device from the map according to the detection result of the identification code of the map;

   a corner position determining module, configured to determine the corner position of the outline of the map in the global image;

   A scale determination module, configured to determine the scale of the global image according to the corner position and the preset actual map size;

   A device identification code detection module, configured to detect the identification code of the target moving device in the global image, and determine the center point of the identification code of the target moving device according to the detection result of the identification code of the target moving device;

   A position determination module, configured to determine the position of the target mobile device at the target mobile device according to the center point of the identification code of the target mobile device, the corner position of the outline of the map, the height of the camera device from the map, and the scale actual location on the map.
9. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that, when the computer program is executed by a processor, the map positioning method according to any one of claims 1 to 7 is implemented A step of.
10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, characterized in that, when the processor executes the computer program, the following claims 1 to 1 are implemented. 7. The steps of any one of the map positioning methods.

Images