CN118135554A - Pose recognition method based on reflective markers and movable equipment - Google Patents

Abstract

The application discloses a pose recognition method and movable equipment based on a reflective marker, wherein the pose recognition method based on the reflective marker comprises the following steps: constructing a sliding window to traverse points in the point cloud frame; calculating a relative intensity threshold based on the reflected intensity at the midpoint of the sliding window; responding to the fact that the reflection intensity of the existing points in the sliding window is larger than a relative intensity threshold value, starting Gao Liangdian cloud selection, and obtaining a highlight point cloud set corresponding to the reflective marker; if the reflection intensity of the continuous preset points in the subsequent traversing path is not greater than the relative intensity threshold, finishing the selection of the highlight point cloud, reconstructing the rest points in the point cloud frame of the sliding window, and continuing traversing; and responding to the completion of the traversal of the point cloud frame, and carrying out pose recognition on the movable equipment based on the selected highlight point cloud set. The accuracy and the efficiency of the highlight point cloud selection are improved, and the situation that the point cloud is selected incorrectly due to the fact that the reflection intensity of laser on the reflective marker is suddenly reduced in a short distance is avoided.

Description

Pose recognition method based on reflective markers and movable equipment

Technical Field

The application relates to the technical field of laser identification, in particular to a pose identification method based on a reflective marker and movable equipment.

Background

When a mobile robot performs task operations, such as object handling in a warehouse, the mobile robot needs to precisely know its position and posture in the current environment in the task environment, so that the mobile robot can accurately perform tasks. At present, a pose recognition is generally performed on a mobile robot by adopting a laser recognition mode, and the pose of the mobile robot is determined by recognizing the point cloud data of the reflective marker in the collected point cloud data.

However, in the laser short-distance recognition process, the reflection intensity of the laser on the reflective marker is reduced, so that the reflective marker point cloud and the non-reflective marker point cloud cannot be distinguished by directly utilizing the point cloud reflection intensity, and the recognized pose is low in accuracy.

Disclosure of Invention

In order to solve the problems, the application at least provides a pose recognition method based on reflective markers and movable equipment.

The first aspect of the application provides a pose recognition method based on a reflective marker, which is applied to movable equipment, wherein a point cloud acquisition device is deployed on the movable equipment; the method comprises the following steps: acquiring a point cloud frame obtained by carrying out point cloud acquisition on a scene to be positioned by a point cloud acquisition device, constructing a sliding window and traversing points in the point cloud frame; wherein, a reflective marker is preset in the scene to be positioned; calculating a relative intensity threshold based on the reflected intensity at the midpoint of the sliding window; responding to the fact that the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold, starting highlight point cloud selection, sequentially selecting the points, corresponding to the sliding window, in the subsequent traversal path, of which the reflection intensity is larger than the relative intensity threshold, and obtaining a highlight point cloud set corresponding to the reflective marker; if the reflection intensity of the continuous preset points in the subsequent traversing path is not greater than the relative intensity threshold, finishing the selection of the highlight point cloud, reconstructing the rest points in the point cloud frame of the sliding window, and continuing traversing; and responding to the completion of the traversal of the point cloud frame, and carrying out pose recognition on the movable equipment based on the selected highlight point cloud set.

In one embodiment, the calculating the relative intensity threshold based on the reflected intensity at the midpoint of the sliding window includes: acquiring a minimum reflection intensity value in a sliding window; and calculating a relative intensity threshold value by using the minimum reflection intensity value.

In one embodiment, before initiating Gao Liangdian the cloud selection in response to the reflected intensity of the point of presence in the sliding window being greater than the relative intensity threshold, further comprising: taking edge points in the traversing movement direction in the sliding window as reference points; comparing the reflected intensity of the fiducial point with a relative intensity threshold; responsive to the reflected intensity of the point of presence in the sliding window being greater than the relative intensity threshold, enabling Gao Liangdian cloud selection, including: responsive to the reflected intensity of the fiducial point being greater than the relative intensity threshold, gao Liangdian cloud selection is initiated.

In an embodiment, responsive to completion of the point cloud frame traversal, pose recognition is performed on the mobile device based on the selected highlight point cloud set, including: converting Gao Liangdian cloud sets into a map coordinate system corresponding to a scene to be positioned, and obtaining a point cloud detection position of the reflective marker aiming at the map coordinate system; acquiring a reference map position of the reflective marker aiming at a map coordinate system, and taking the highlight point cloud set as a target point cloud set if the distance between the point cloud detection position and the reference map position is smaller than a preset distance threshold value; and calculating pose information of the movable equipment by using the target point cloud set.

In an embodiment, before converting the highlight point cloud set to the map coordinate system corresponding to the scene to be positioned, to obtain the point cloud detection position of the reflective marker for the map coordinate system, the method further includes: based on the coordinates of each point in the Gao Liangdian cloud set, calculating average coordinates corresponding to the highlight point cloud set respectively; calculating the distance difference between the average coordinates corresponding to the highlight point cloud sets among a plurality of continuous point cloud frames; converting Gao Liangdian the cloud set to a map coordinate system corresponding to a scene to be positioned to obtain a point cloud detection position of the reflective marker aiming at the map coordinate system, wherein the method comprises the following steps: and converting the highlight point cloud set with the distance difference meeting the preset condition into a map coordinate system corresponding to the scene to be positioned, and obtaining the point cloud detection position of the reflective marker aiming at the map coordinate system.

In an embodiment, the mobile device is further deployed with an image acquisition apparatus; the method further comprises the steps of: if the distance between the movable equipment and the reflective marker is detected to be smaller than a preset distance threshold value, acquiring a scene image of the scene to be positioned by using an image acquisition device; and combining the scene image and Gao Liangdian cloud sets, and carrying out pose recognition on the movable equipment.

In one embodiment, the relatively retroreflective markers are deployed with image markers; combining the scene image and Gao Liangdian cloud sets, performing pose recognition on the movable equipment, including: dividing a scene image into a concerned area containing an image marker by utilizing the highlight cloud set; and carrying out pose recognition on the movable equipment based on the image characteristics of the region of interest.

In an embodiment, using Gao Liangdian cloud sets, partitioning a region of interest containing image markers from a scene image includes: calculating average coordinates corresponding to the highlight point cloud set based on the coordinates of each point in the Gao Liangdian cloud set; converting the average coordinates of the Gao Liangdian cloud set into a pixel coordinate system corresponding to the scene image to obtain image coordinates of the reflective marker in the scene image; acquiring a deployment position relationship between the reflective marker and the image marker; and dividing the scene image into a region of interest containing the image marker based on the image coordinates and the deployment position relationship.

In an embodiment, combining the scene image and Gao Liangdian cloud sets, pose recognition is performed on the mobile device, including: performing pose recognition on the movable equipment by using the scene image to obtain an image pose recognition result; and performing pose recognition on the movable equipment by using the selected highlight point cloud set to obtain a point cloud pose recognition result; and fusing the image pose recognition result and the point cloud pose recognition result to obtain a fused pose result of the movable equipment.

The second aspect of the application provides a pose recognition device based on reflective markers, which is deployed on movable equipment, wherein the movable equipment is provided with a point cloud acquisition device; the pose recognition device based on the reflective marker comprises: the window traversing module is used for acquiring a point cloud frame obtained by the point cloud acquisition device for carrying out point cloud acquisition on a scene to be positioned, constructing a sliding window and traversing points in the point cloud frame; wherein, a reflective marker is preset in the scene to be positioned; the threshold calculating module is used for calculating a relative intensity threshold based on the reflection intensity of the midpoint of the sliding window; the starting judgment module is used for starting highlight point cloud selection in response to the fact that the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold value, and sequentially selecting the points, corresponding to the sliding window, of which the reflection intensity is larger than the relative intensity threshold value in the subsequent traversal path to obtain a highlight point cloud set corresponding to the reflective marker; the ending judgment module is used for ending the selection of the highlight point cloud if the reflection intensity of the continuous preset points in the subsequent traversal path is not greater than the relative intensity threshold value, reconstructing the rest points in the point cloud frame of the sliding window and continuing traversal; and the pose recognition module is used for responding to completion of the traversal of the point cloud frame and carrying out pose recognition on the movable equipment based on the selected highlight point cloud set.

The third aspect of the application provides a mobile device, comprising a memory and a processor, wherein the processor is used for executing program instructions stored in the memory to realize the pose recognition method based on the reflective marker.

A fourth aspect of the present application provides a computer readable storage medium having stored thereon program instructions which, when executed by a processor, implement the above-described method for identifying pose based on reflective markers.

According to the scheme, the point cloud frame obtained by carrying out point cloud acquisition on the scene to be positioned by the point cloud acquisition device is obtained, and points in the point cloud frame of the sliding window are constructed and traversed; calculating a relative intensity threshold based on the reflected intensity at the midpoint of the sliding window; responding to the fact that the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold, starting highlight point cloud selection, sequentially selecting the points, corresponding to the sliding window, in the subsequent traversal path, of which the reflection intensity is larger than the relative intensity threshold, and obtaining a highlight point cloud set corresponding to the reflective marker; if the reflection intensity of the continuous preset points in the subsequent traversing path is not greater than the relative intensity threshold, finishing the selection of the highlight point cloud, reconstructing the rest points in the point cloud frame of the sliding window, and continuing traversing; responding to the completion of the point cloud frame traversal, carrying out pose recognition on the movable equipment based on the selected highlight point cloud set, accurately starting or ending the point cloud selection step according to the change condition of the reflection intensity of the point, improving the accuracy and efficiency of highlight point cloud selection, and avoiding the condition of point cloud selection errors caused by the sudden reduction of the reflection intensity of laser on the reflective marker in a short distance.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 is a schematic diagram of a scene to be located provided by an embodiment of the present application;

FIG. 2 is a flow chart of a method for identifying pose based on reflective markers, according to an exemplary embodiment of the application;

FIG. 3 is a flow chart illustrating highlight cloud selection in accordance with an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram illustrating highlight cloud selection in accordance with an exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of an artificial feature shown in an exemplary embodiment of the application;

FIG. 6 is a flow chart illustrating pose recognition according to an exemplary embodiment of the present application;

fig. 7 is a block diagram of a reflective marker-based pose recognition device according to an exemplary embodiment of the present application;

FIG. 8 is a schematic diagram of an electronic device shown in an exemplary embodiment of the application;

Fig. 9 is a schematic diagram of a structure of a computer-readable storage medium according to an exemplary embodiment of the present application.

Detailed Description

The following describes embodiments of the present application in detail with reference to the drawings.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.

The term "and/or" is herein merely an association information describing an associated object, meaning that three relationships may exist, e.g., a and/or B may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship. Further, "a plurality" herein means two or more than two. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, may mean including any one or more elements selected from the group consisting of A, B and C.

The pose recognition method based on the reflective marker provided by the embodiment of the application is explained below.

Referring to fig. 1, a schematic diagram of a scene to be located according to an embodiment of the present application is shown, and as shown in fig. 1, the scene to be located may include a movable device 110 and a reflective marker 120.

The mobile device 110 may be a sweeping robot, a cargo robot, etc., and the present application is not limited in the type of mobile device 110.

Illustratively, the movable apparatus 110 is deployed with a moving device 111, a point cloud acquisition device 112 and an image acquisition device 113, the point cloud acquisition device 112 is used for acquiring a point cloud frame, the image acquisition device 113 is used for acquiring a scene image, and a movement strategy of the movable apparatus 110 is determined according to the point Yun Zhen and/or the scene image by driving the moving device 111 to move the movable apparatus 110 in the scene to be positioned.

Wherein the mobile device 111 contains a motion controller, an odometer, a battery, a motor, an embedded computer, etc.; the point cloud acquisition device 112 may be a lidar; the image capturing device 113 may be a camera sensor.

The point cloud reflection intensity of the reflective marker 120 is larger than that of other objects in the environment, and the reflective marker 120 assists the movable equipment 110 in identifying the pose in the scene to be positioned.

An example of the positioning of the mobile device 110 is illustrated: the mobile device 110 stores in advance a scene map of a scene to be positioned, such as a two-dimensional grid map, in which positions of obstacles, scene building structures of the scene to be positioned, and the like are marked, such as positions of a shelf, a workbench, a house support frame, goods to be carried, and the like. The movable equipment 110 uses the odometer to estimate the motion variable in the moving process, and simultaneously uses the point cloud acquisition device 112 to scan the scene to be positioned to obtain a point cloud frame, and identifies the position of the reflective marker in the point cloud frame, so that the position and the gesture of the movable equipment 110 in the scene map are determined by fusing the position of the reflective marker and the motion variable estimated by the odometer, and the positioning of the movable equipment 110 is realized.

According to the gesture recognition method based on the reflective marker provided by the embodiment of the application, the execution subject of each step can be the movable equipment 110, such as a target application program installed and operated in the movable equipment 110, or a server in communication connection with the movable equipment 110, or the movable equipment 110 and the server are in interaction coordination, namely, one part of the steps of the method are executed by the movable equipment 110, and the other part of the steps are executed by the server in communication connection with the movable equipment 110.

Referring to fig. 2, fig. 2 is a flowchart illustrating a pose recognition method based on reflective markers according to an exemplary embodiment of the present application. The pose recognition method based on the reflective markers can be applied to the implementation environment shown in fig. 1 and is specifically executed by movable equipment in the implementation environment. It should be understood that the method may be adapted to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and the implementation environments to which the method is adapted are not limited by the present embodiment.

As shown in fig. 2, the pose recognition method based on the reflective marker at least includes steps S210 to S250, which are described in detail as follows:

Step S210: acquiring a point cloud frame obtained by carrying out point cloud acquisition on a scene to be positioned by a point cloud acquisition device, constructing a sliding window and traversing points in the point cloud frame; wherein, the scene to be positioned is preset with a reflective marker.

The reflective markers are used for assisting the movable equipment in identifying the pose.

Illustratively, an artificial feature is affixed at a predetermined location in the scene to be located, the artificial feature comprising a flat panel affixed with retroreflective markers, which may include a plurality of retroreflective strips.

It can be appreciated that in the actual application process, the specific implementation manner of the artificial feature can be flexibly determined according to the specific application scenario, which is not limited by the present application.

The point cloud acquisition device scans a scene to be positioned to acquire point clouds, a point cloud frame is obtained, coordinates and reflection intensity of each point are contained in the point cloud frame, and the reflection intensity of a return signal is recorded through the reflectivity of the surface of an object.

Then, a sliding window is constructed to traverse the points in the point cloud frame.

The size of the sliding window can be preset, and the size of the sliding window can be flexibly calculated.

For example, a window setting influencing factor of the mobile device in the current scene to be positioned is determined (the window setting influencing factor corresponding to the query may be based on the device type of the mobile device, the service currently executed by the mobile device, the scene type of the scene to be positioned, etc.), and the size of the sliding window is calculated based on the window setting influencing factor.

For example, the window setting influencing factors comprise the distance between the movable device and the reflective marker, and the size of the sliding window is calculated according to the distance, if the larger the distance is, the smaller the sliding window is, and the smaller the distance is, the larger the sliding window is; the window setting influencing factors comprise positioning accuracy corresponding to the service currently executed by the mobile equipment, and the size of the sliding window is calculated according to the positioning accuracy, if the positioning accuracy is higher, the sliding window is smaller, and the positioning accuracy is lower, the sliding window is larger.

According to the traversing of the points in the point cloud frame by the sliding window, it should be noted that the traversing of the points in the point cloud frame may be performed by the sliding window, the moving step length of the sliding window may be preset or may be flexibly calculated, and the calculating manner of the moving step length may refer to the calculating manner of the size of the sliding window, which is not described herein.

Step S220: based on the reflected intensity at the midpoint of the sliding window, a relative intensity threshold is calculated.

And obtaining the reflection intensity of each point in the sliding window, and calculating the relative intensity threshold according to the reflection intensity of each point in the sliding window.

Illustratively, the reflection intensities of the points in the sliding window are sorted in ascending order, a reflection intensity minimum value, a reflection intensity intermediate value and the like in the sorting result are obtained, and the relative intensity threshold is calculated based on the reflection intensity minimum value and/or the reflection intensity intermediate value.

Illustratively, the reflected intensities of the points in the sliding window are obtained, an average value of the reflected intensities of the points is calculated, and a relative intensity threshold is obtained based on the average value.

It can be appreciated that the calculation mode of the relative intensity threshold can be flexibly selected according to the specific application scenario, which is not limited by the present application.

Step S230: and in response to the fact that the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold, starting highlight point cloud selection, sequentially selecting the points, corresponding to the sliding window, in the subsequent traversal path, of which the reflection intensity is larger than the relative intensity threshold, and obtaining a highlight point cloud set corresponding to the reflective marker.

After the relative intensity threshold is obtained, whether the reflection intensity of the point in the sliding window is larger than the relative intensity threshold is judged.

If the reflection intensity of the point which does not exist in the sliding window is larger than the relative intensity threshold, continuing to move the sliding window according to the moving step length to traverse the point in the point cloud frame, and continuing to calculate the relative reflection intensity threshold after the moving, and judging whether the reflection intensity of the point which exists in the sliding window is larger than the relative intensity threshold or not until the reflection intensity of the point which exists in the sliding window is larger than the relative intensity threshold.

If the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold, starting highlight point cloud selection, and sequentially selecting the points with the reflection intensity larger than the relative intensity threshold in the subsequent traversal path corresponding to the sliding window through Gao Liangdian cloud selection operation to obtain a highlight point cloud set corresponding to the reflective marker.

The sliding window traverses points in the point cloud frame according to a designated search route, and the subsequent traversing path refers to a path taking the current position of the sliding window as a starting point in the traversing search route corresponding to the sliding window. The method comprises the steps of taking the current position of a sliding window as a starting point, sequentially judging whether the reflection intensity of points is larger than a relative intensity threshold according to a traversing search route, selecting the points with the reflection intensity larger than the relative intensity threshold, and adding the selected points into Gao Liangdian cloud sets.

Whether the reflection intensity of the point exists in the sliding window is larger than the relative intensity threshold value or not is detected, whether the point exists in the sliding window is judged to be at the edge of the point cloud data corresponding to the reflective marker or not at present, if the reflection intensity of the point exists in the moving window is larger than the relative intensity threshold value, the point which belongs to the reflective marker exists in the point which is to be traversed subsequently, and at the moment, the highlight point cloud selection operation is executed, so that the accuracy and the efficiency of point cloud selection are improved.

Step S240: if the reflection intensity of the continuous preset points in the subsequent traversing path is not greater than the relative intensity threshold, finishing the selection of the highlight point cloud, reconstructing the rest points in the point cloud frame of the sliding window, and continuing traversing.

And in the process of executing the highlight point cloud selection operation, judging whether the reflection intensity of the continuous preset points is not larger than a relative intensity threshold value.

And if the reflection intensity of the points which are not continuously preset is not greater than the relative intensity threshold value, continuously executing the highlight point cloud selection.

If the reflection intensities of the continuous preset number of points are not greater than the relative intensity threshold, the highlight point cloud selection operation in the current period is considered to be completed, the highlight point cloud selection is ended, the remaining points in the point cloud frame are continuously traversed by reconstructing the sliding window, and the traversing mode is referred to the above steps S220 to S240, and is not repeated here.

By repeatedly executing the traversing operation and Gao Liangdian cloud selection operation, the highlight point cloud belonging to the reflective marker in the point cloud frame can be accurately selected, and the accuracy of the subsequent pose recognition of the movable equipment is improved.

Optionally, the preset number may be a preset value, and the preset number may also be a value that is flexibly calculated, and the calculation manner of the preset number may refer to the calculation manner of the size of the sliding window, which is not described herein.

Step S250: and responding to the completion of the traversal of the point cloud frame, and carrying out pose recognition on the movable equipment based on the selected highlight point cloud set.

After the point cloud frame is traversed, the pose relation between the reflective marker and the movable equipment can be determined according to the selected highlight point cloud set, so that the pose of the movable equipment in the scene to be positioned is obtained according to the deployment position of the reflective marker in the scene to be positioned.

Next, some embodiments of the present application will be described in detail.

In some embodiments, the calculating the relative intensity threshold in step S220 based on the reflected intensity at the midpoint of the sliding window includes:

step S221: the minimum reflected intensity value in the sliding window is obtained.

Step S222: and calculating a relative intensity threshold value by using the minimum reflection intensity value.

For example, a multiple parameter is obtained, a multiplication calculation is performed on a preset multiple parameter and a minimum reflection intensity value, and a calculation result is used as a relative intensity threshold.

The multiple parameter may be a preset value or a value that is flexibly calculated. For example, the distance between the movable device and the reflective marker is obtained, and the multiple parameter is calculated according to the distance, and the distance and the multiple parameter can be in direct proportion, that is, the larger the distance is, the larger the multiple parameter is, the smaller the distance is, and the multiple parameter is smaller. Of course, for accuracy of point cloud selection, a maximum value and a minimum value may be set for the multiple parameters.

In some implementations, referring to fig. 3, fig. 3 is a flowchart illustrating a highlight cloud selection according to an exemplary embodiment of the present application, and as shown in fig. 3, the method includes:

step S310: and constructing a sliding window, and traversing the points in the point cloud frame based on the sliding window.

Step S320: and taking the edge point in the traversing movement direction in the sliding window as a reference point.

Step S330: whether the reflection intensity of the reference point is greater than the relative intensity threshold is determined, if so, step S340 is performed, otherwise, step S310 is performed.

Step S340: and starting highlight point cloud selection, and sequentially selecting points with reflection intensity larger than a relative intensity threshold value in the subsequent traversal path corresponding to the sliding window to obtain a highlight point cloud set corresponding to the reflective marker.

Step S350: judging whether the reflection intensity of the continuous preset points is not larger than the relative intensity threshold, if so, ending the selection of the highlight point cloud, executing the step S310, otherwise, continuing to execute the step S340.

For example, referring to fig. 4, fig. 4 is a schematic diagram illustrating selection of a highlight cloud according to an exemplary embodiment of the present application, as shown in fig. 4, a sliding window traverses points in a point cloud frame, uses an edge point Pi in a traversing motion direction in the sliding window as a reference point, obtains a minimum reflection intensity value in the sliding window as Imin, and calculates a relative intensity threshold as s×imin, where S represents a multiple parameter, such as s=2.

If the reflection intensity Ii of the point Pi is greater than s×imin, the point Pi is considered to be a point cloud on the reflective marker, the Gao Liangdian cloud selection is started, and the point with the reflection intensity greater than the relative intensity threshold value in the points after the point Pi is selected is added to the Gao Liangdian cloud set. And in the process of selecting the highlight point cloud, counting the number of continuous points with the reflection intensity not larger than the relative intensity threshold, and ending the highlight point cloud selection in the current period if the number of continuous points with the reflection intensity not larger than the relative intensity threshold is larger than the preset number.

For example, in fig. 4, if the preset number is w, and the reflection intensities from pi+q points to the points between pi+q+w are not greater than the relative intensity threshold, the selection of the highlight point cloud in the current period is considered to be finished, and the highlight point cloud selected in the current period is from the point Pi to the point pi+q. Then, reconstructing a sliding window from Pi+q+w to perform point cloud traversal, and repeatedly performing selection judgment of the highlight point cloud.

Optionally, the reflective marker includes a plurality of reflective strips, and may be a cluster of the highlight clouds in the highlight cloud set, where different clusters correspond to different reflective strips, for example, K reflective strips, and the highlight cloud cluster in the highlight cloud set may be represented as (M1, M2, MK).

Optionally, when selecting the highlight cloud, it may further be determined whether the number of points of the highlight cloud selected in the present period is greater than a minimum number threshold, if so, the selected highlight cloud is considered to be trusted, if not, the selected highlight cloud is considered to be untrusted, and the trusted highlight cloud is added to the highlight cloud set, so that accuracy of selecting the highlight cloud is improved. The minimum number threshold may be preset, or may be flexibly calculated according to practical situations, for example, according to a size parameter of the reflective marker, a distance between the movable device and the reflective marker, and the like.

After the point cloud frame is traversed, the pose of the movable equipment in the scene to be positioned can be directly calculated according to the selected highlight point cloud set; and the quality of the highlight cloud set can be judged, and if the quality of the Gao Liangdian cloud set meets the preset quality condition, the pose of the movable equipment in the scene to be positioned is calculated according to the highlight cloud set.

A quality judgment method of Gao Liangdian cloud sets is exemplified.

In some embodiments, in response to the completion of the traversal of the point cloud frame in step S250, performing pose recognition on the mobile device based on the selected highlight point cloud set, including:

step S251: and converting the Gao Liangdian cloud set into a map coordinate system corresponding to the scene to be positioned, and obtaining the point cloud detection position of the reflective marker aiming at the map coordinate system.

The pose of the movable device when the point cloud frame corresponding to the highlight point cloud set is acquired, the external parameters of the point cloud acquisition device, the pose of the center of the reflective marker corresponding to the selected highlight point cloud set relative to the laser coordinate system are acquired, and the Gao Liangdian cloud set is converted to the map coordinate system corresponding to the scene to be positioned based on the parameters, so that the point cloud detection position of the reflective marker relative to the map coordinate system is obtained. The specific conversion formula can be expressed as formula 1:

Tmd＝Tmb*TbL*TLd

In the formula 1, tmd is the point cloud detection position of the reflective marker aiming at the map coordinate system; tmb is the pose of the movable equipment when the point cloud frames corresponding to the highlight point cloud sets are acquired; tbL is an external parameter of the point cloud acquisition device relative to the equipment center of the movable equipment; TLd is the pose of the center d of the reflective marker corresponding to the selected highlight point cloud set relative to the laser coordinate system.

Step S252: and acquiring a reference map position of the reflective marker aiming at a map coordinate system, and taking the highlight point cloud set as the target point cloud set if the distance between the point cloud detection position and the reference map position is smaller than a preset distance threshold value.

The reference map position may be a position of a reflective marker identified by a historical point cloud frame before the current point cloud frame, or a preset position of the reflective marker in a map coordinate system, which is not limited in the present application.

For example, taking the central coordinate Mc-1 of the reflective marker identified by the historical point cloud frame aiming at the map coordinate system as a reference map position, determining the reflective marker center of the reflective marker according to the high-bright point cloud set selected by the current point cloud frame, obtaining the central coordinate Mc-2 of the reflective marker center aiming at the map coordinate system according to the formula 1, taking the central coordinate Mc-2 as a point cloud detection position, calculating the distance between Mc-1 and Mc-2, and if the distance is smaller than a preset distance threshold, considering the high-bright point cloud set currently selected to be in accordance with the condition, and taking the high-bright point cloud set as a target point cloud set.

Step S253: and calculating pose information of the movable equipment by using the target point cloud set.

In some embodiments, before converting the highlight point cloud set to the map coordinate system corresponding to the scene to be positioned, to obtain the point cloud detection position of the reflective marker for the map coordinate system, the method further includes: based on the coordinates of each point in the Gao Liangdian cloud set, calculating average coordinates corresponding to the highlight point cloud set respectively; calculating the distance difference between the average coordinates corresponding to the highlight point cloud sets among a plurality of continuous point cloud frames; converting Gao Liangdian the cloud set to a map coordinate system corresponding to a scene to be positioned to obtain a point cloud detection position of the reflective marker aiming at the map coordinate system, wherein the method comprises the following steps: and converting the highlight point cloud set with the distance difference meeting the preset condition into a map coordinate system corresponding to the scene to be positioned, and obtaining the point cloud detection position of the reflective marker aiming at the map coordinate system.

Alternatively, the satisfaction of the preset condition by the distance difference may be: the distance difference between the average coordinates corresponding to the highlight point cloud set among the plurality of continuous point cloud frames is smaller than the preset distance difference. The distance difference meeting the preset condition may be: and counting the accumulated number of frames, between a plurality of continuous point cloud frames, of which the distance difference between average coordinates corresponding to the highlight point cloud set is smaller than the preset distance difference value, wherein the accumulated number is larger than a preset number threshold.

Taking a plurality of highlight cloud clusters in the highlight cloud set as an example, each highlight cloud cluster corresponds to one reflective strip in the reflective marker for illustration: and (3) carrying out highlight point cloud selection on the t-th frame point cloud frame, and setting the average coordinate of each highlight point in the k-th highlight point cloud cluster in the selected highlight point cloud set as MCk (t). Acquiring a t-1 frame point cloud frame acquired before, acquiring tracking count Tcnt corresponding to a kth highlight point cloud cluster if the average coordinate of each highlight point in the kth highlight point cloud cluster in the highlight point cloud set acquired by selecting the t-1 frame point cloud frame is MCk (t-1), and if the distance difference between MCk (t) and MCk (t-1) is smaller than a preset distance difference value, adding 1 to the tracking count Tcnt; otherwise, updating the value of the tracking count Tcnt corresponding to the kth highlight cloud cluster to be 0. Then, judging whether the tracking count Tcnt is larger than a count threshold ths (for example, 4 times), if so, considering that the distance difference meets the preset condition, and converting the high-brightness point cloud set obtained by selecting the point cloud frame of the t frame into a map coordinate system corresponding to the scene to be positioned to obtain the point cloud detection position of the reflective marker aiming at the map coordinate system so as to carry out subsequent judgment.

By the method, the highlight point cloud set with the quality meeting the preset quality condition is selected to conduct pose recognition on the movable equipment, and accuracy of pose recognition is improved.

Of course, other ways of quality scoring the highlight point cloud collection may be used in addition to the embodiments described above, as the application is not limited in this regard.

In some embodiments, the mobile device is further deployed with an image acquisition apparatus; the method further comprises the steps of: if the distance between the movable equipment and the reflective marker is detected to be smaller than a preset distance threshold value, acquiring a scene image of the scene to be positioned by using an image acquisition device; and combining the scene image and Gao Liangdian cloud sets, and carrying out pose recognition on the movable equipment.

When the distance between the movable equipment and the reflective marker is smaller than a preset distance threshold, the reflection intensity of laser on the reflective marker is reduced, so that the positioning accuracy of the movable equipment is further improved, an image acquisition device is started to acquire a scene image of a scene to be positioned, and the movable equipment is subjected to pose recognition by combining the scene image and Gao Liangdian cloud sets, so that the vision and the laser dimension are fused to perform pose recognition on the movable equipment, and the positioning accuracy is improved.

The image markers are used for assisting the movable equipment in identifying the pose in the scene image.

For the scene image, the position relation of the movable equipment relative to the image marker can be determined by calculating the image content characteristics of the scene image, for example, the image content belonging to the image marker in the scene image is subjected to geometric analysis, so that the position relation of the movable equipment relative to the image marker is obtained.

Optionally, in order to improve the computing speed and efficiency, the region of interest containing the image marker may be obtained by dividing the scene image, and the pose recognition is performed on the movable device according to the image characteristics of the region of interest.

Illustratively, the relatively retroreflective markers are deployed with image markers; combining the scene image and Gao Liangdian cloud sets, performing pose recognition on the movable equipment, including: dividing a scene image into a concerned area containing an image marker by utilizing the highlight cloud set; and carrying out pose recognition on the movable equipment based on the image characteristics of the region of interest.

Specifically, using Gao Liangdian cloud sets, a region of interest containing image markers is partitioned from a scene image, including: calculating average coordinates corresponding to the highlight point cloud set based on the coordinates of each point in the Gao Liangdian cloud set; converting the average coordinates of the Gao Liangdian cloud set into a pixel coordinate system corresponding to the scene image to obtain image coordinates of the reflective marker in the scene image; acquiring a deployment position relationship between the reflective marker and the image marker; and dividing the scene image into a region of interest containing the image marker based on the image coordinates and the deployment position relationship.

For example, referring to fig. 5, fig. 5 is a schematic diagram of an artificial feature shown in an exemplary embodiment of the present application, as shown in fig. 5, a reflective marker includes two reflective strips 1 and 2 that are oppositely disposed, and an image marker is disposed between the two reflective strips that are oppositely disposed, where the image marker may be composed of a preset number of circular images, and by using the image markers of the inner and outer circles of the circular rings, the circular center recognition of the circular ring with an angle is more robust and accurate, the calculated pose is more accurate and has full-image universality, and the fluctuation of the small-range pose recognition is avoided.

Of course, the reflective markers and the image markers may be arranged in other patterns, which is not limited in the present application.

According to external parameters of a point cloud acquisition device of the movable equipment, the average coordinates of the highlight point cloud clusters corresponding to the light reflecting strips 1 and 2 are converted into a vehicle body coordinate system corresponding to the movable equipment, so that a first average coordinate is obtained; then, according to external parameters of an image acquisition device of the movable equipment, converting a first average coordinate under a vehicle body coordinate system into a coordinate system corresponding to the image acquisition device to obtain a second average coordinate; and then according to the internal parameters of the image acquisition device, projecting the three-dimensional second average coordinates to the image coordinates in the scene image to obtain the image coordinates of the reflective strips 1 and 2 in the scene image.

Since the reflective strips 1 and 2 are fixed artificial features, the central coordinates of the upper and lower edges of the reflective strips 1 and 2 can also be obtained by the above coordinate system transformation according to the actual reflective strip size, and since the deployment position relationship between the image markers relative to the reflective markers is determined, the approximate position of the image markers in the scene image can be obtained according to the obtained image coordinates and deployment position relationship, and the region of interest of the image markers can be represented by a quadrangle enclosed by a red dotted line frame as shown in fig. 5.

After the attention area of the image marker in the scene image is obtained, the pose recognition of the movable equipment is carried out according to the image characteristics of the attention area, and an image pose recognition result is obtained.

For example, still taking fig. 5 as an example, the region of interest in the scene image is snapped, and the pixel center coordinates of each of the four ring images can be identified and obtained without an angle between the default image acquisition device and the image markers. When the image acquisition device has a certain angle relative to the image marker, the circular ring image is approximately elliptical, and the center coordinate of the elliptical ring is acquired as the center coordinate of the pixel. And then, calculating to obtain the relative pose of the movable equipment relative to the image marker by using the pixel center coordinates corresponding to the four circular images and the actual position relationship of the four circular images preset by the artificial features.

The method for identifying the pose of the movable equipment by using the high-point cloud set is similar to the method for identifying the pose of the movable equipment according to the image identifier, and is not described herein.

Of course, other ways may be used to perform pose recognition on the mobile device, for example, a neural network model with pose analysis function is trained in advance, and a scene image and/or a highlight cloud set may be input into the neural network model to obtain a pose recognition result output by the neural network model.

In some implementations, combining the scene image and Gao Liangdian cloud sets, pose recognition for the mobile device includes: performing pose recognition on the movable equipment by using the scene image to obtain an image pose recognition result; and performing pose recognition on the movable equipment by using the selected highlight point cloud set to obtain a point cloud pose recognition result; and fusing the image pose recognition result and the point cloud pose recognition result to obtain a fused pose result of the movable equipment.

The pose includes position and angle.

Optionally, fusing the position in the point cloud pose recognition result and the position in the image pose recognition result according to the position fusion weight parameter; and fusing the angles in the point cloud pose recognition result and the angles in the image pose recognition result according to the angle fusion weight parameters.

The adopted position fusion weight parameters and angle fusion weight parameters can be preset and can be flexibly calculated according to actual conditions.

For example, according to the distance between the movable device and the reflective marker, the position fusion weight parameter and the angle fusion weight parameter are determined, for example, the calculation accuracy of the lateral deviation is reduced due to the laser at a close distance, but the angle accuracy is still reliable due to the length factor of the reflective strip, so that when the distance between the movable device and the reflective marker is reduced, the weight parameter of the position in the image pose recognition result can be improved, but the weight parameter of the angle in the point cloud pose recognition result is kept unchanged.

Optionally, the position in the image pose recognition result can be selected as the position of the fusion pose result, the angle in the point cloud pose recognition result is selected as the angle of the fusion pose result, and the pose of the movable equipment relative to the artificial feature under the final close condition is combined by using the angle information obtained by the point cloud acquisition device and the position information obtained by the image acquisition device, so that the final fusion pose result is obtained.

It can be understood that the specific fusion mode of the image pose recognition result and the point cloud pose recognition result can be flexibly selected according to actual conditions, and the application is not limited to this.

Referring to fig. 6 for an exemplary embodiment of the present application, fig. 6 is a flowchart illustrating pose recognition, as shown in fig. 6, including:

Step S610: according to Gao Liangdian cloud sets, carrying out pose recognition on the movable equipment to obtain a point cloud pose recognition result;

Step S620: judging whether the distance between the movable equipment and the reflective marker is smaller than a preset distance threshold value, if so, executing step S630 to step S640; otherwise, step S650 is performed.

Step S630: according to the scene image, carrying out pose recognition on the movable equipment to obtain an image pose recognition result;

Step S640: fusion point cloud pose recognition results and image pose recognition results are obtained and output;

Step S650: and outputting a point cloud pose recognition result.

According to the pose recognition method based on the reflective marker, the point cloud frame obtained by carrying out point cloud acquisition on the scene to be positioned by the point cloud acquisition device is obtained, and a sliding window is constructed to traverse points in the point cloud frame; calculating a relative intensity threshold based on the reflected intensity at the midpoint of the sliding window; responding to the fact that the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold, starting highlight point cloud selection, sequentially selecting the points, corresponding to the sliding window, in the subsequent traversal path, of which the reflection intensity is larger than the relative intensity threshold, and obtaining a highlight point cloud set corresponding to the reflective marker; if the reflection intensity of the continuous preset points in the subsequent traversing path is not greater than the relative intensity threshold, finishing the selection of the highlight point cloud, reconstructing the rest points in the point cloud frame of the sliding window, and continuing traversing; responding to the completion of the point cloud frame traversal, carrying out pose recognition on the movable equipment based on the selected highlight point cloud set, accurately starting or ending the point cloud selection step according to the change condition of the reflection intensity of the point, improving the accuracy and efficiency of highlight point cloud selection, and avoiding the condition of point cloud selection errors caused by the sudden reduction of the reflection intensity of laser on the reflective marker in a short distance.

FIG. 7 is a block diagram of a reflective marker-based pose recognition device deployed on a mobile device with a point cloud acquisition device, according to an exemplary embodiment of the present application; as shown in fig. 7, the exemplary reflective marker-based pose recognition apparatus 700 includes: a window traversal module 710, a threshold calculation module 720, a start determination module 730, an end determination module 740, and a pose recognition module 750. Specifically:

The window traversing module 710 is configured to obtain a point cloud frame obtained by performing point cloud acquisition on a scene to be positioned by using the point cloud acquisition device, and construct a sliding window to traverse points in the point cloud frame; wherein, a reflective marker is preset in the scene to be positioned;

A threshold calculation module 720, configured to calculate a relative intensity threshold based on the reflection intensity at the midpoint of the sliding window;

The starting judging module 730 is configured to start highlight point cloud selection in response to the reflection intensity of the existing points in the sliding window being greater than the relative intensity threshold, and sequentially select the points in the subsequent traversal path corresponding to the sliding window, where the reflection intensity is greater than the relative intensity threshold, to obtain a highlight point cloud set corresponding to the reflective marker;

the ending judgment module 740 is configured to end the selection of the highlight point cloud if the reflection intensities of the continuous preset points in the subsequent traversal path are not greater than the relative intensity threshold, reconstruct the remaining points in the point cloud frame of the sliding window, and continue traversal;

The pose recognition module 750 is configured to perform pose recognition on the mobile device based on the selected highlight point cloud set in response to completion of the traversal of the point cloud frame.

It should be noted that, the pose recognition device based on the reflective marker provided in the above embodiment and the pose recognition method based on the reflective marker provided in the above embodiment belong to the same concept, where the specific manner in which each module and unit perform the operation has been described in detail in the method embodiment, and is not described herein again. In practical application, the pose recognition device based on the reflective marker provided in the above embodiment may allocate the functions to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above, which is not limited herein.

Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the application. The electronic device 800 comprises a memory 801 and a processor 802, the processor 802 being configured to execute program instructions stored in the memory 801 to implement the steps of any of the above-described embodiments of the reflective marker-based pose recognition method. In one particular implementation scenario, electronic device 800 may include, but is not limited to: the electronic device 800 may also include mobile devices such as a notebook computer and a tablet computer, and is not limited herein.

Specifically, the processor 802 is configured to control itself and the memory 801 to implement the steps of any of the embodiments of the reflective marker-based pose recognition method described above. The processor 802 may also be referred to as a central processing unit (Central Processing Unit, CPU). The processor 802 may be an integrated circuit chip with signal processing capabilities. The Processor 802 may also be a general-purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an Application SPECIFIC INTEGRATED Circuit (ASIC), a Field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 802 may be commonly implemented by an integrated circuit chip.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an embodiment of a computer readable storage medium according to the present application. The computer readable storage medium 900 stores program instructions 910 executable by the processor, where the program instructions 910 are configured to implement the steps in any of the above-described embodiments of the method for identifying a pose based on reflective markers.

In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.

The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims (10)

1. The pose recognition method based on the reflective markers is characterized by being applied to movable equipment, wherein a point cloud acquisition device is deployed on the movable equipment; the method comprises the following steps:

Acquiring a point cloud frame obtained by carrying out point cloud acquisition on a scene to be positioned by the point cloud acquisition device, constructing a sliding window and traversing points in the point cloud frame; wherein, the scene to be positioned is preset with a reflective marker;

calculating a relative intensity threshold based on the reflection intensity of the midpoint of the sliding window;

Responding to the fact that the reflection intensity of the existing points in the sliding window is larger than the relative intensity threshold, starting highlight point cloud selection, and sequentially selecting points, corresponding to the sliding window, in the subsequent traversal path, of which the reflection intensity is larger than the relative intensity threshold, so that a highlight point cloud set corresponding to the reflective marker is obtained;

if the reflection intensity of the continuous preset points in the subsequent traversing path is not greater than the relative intensity threshold, ending Gao Liangdian cloud selection, reconstructing a sliding window, and continuing traversing the rest points in the point cloud frame;

And responding to the completion of the point cloud frame traversal, and carrying out pose recognition on the movable equipment based on the selected highlight point cloud set.

2. The method of claim 1, wherein calculating the relative intensity threshold based on the reflected intensity at the midpoint of the sliding window comprises:

Acquiring a minimum reflection intensity value in the sliding window;

and calculating to obtain a relative intensity threshold value by using the minimum reflection intensity value.

3. The method of claim 1, further comprising, prior to said initiating Gao Liangdian cloud selection in response to the reflected intensity of a point of presence in the sliding window being greater than the relative intensity threshold:

taking edge points in the traversing movement direction in the sliding window as reference points;

comparing the reflected intensity of the fiducial point to the relative intensity threshold;

And in response to the reflected intensity of the point of presence in the sliding window being greater than the relative intensity threshold, initiating Gao Liangdian cloud selection, including:

Responsive to the reflected intensity of the fiducial being greater than the relative intensity threshold, gao Liangdian cloud selection is initiated.

4. The method of claim 1, wherein the pose recognition of the mobile device based on the selected set of high-intensity clouds in response to the point cloud frame traversal being completed comprises:

Converting the Gao Liangdian cloud set into a map coordinate system corresponding to the scene to be positioned, and obtaining a point cloud detection position of the reflective marker aiming at the map coordinate system;

Acquiring a reference map position of the reflective marker aiming at the map coordinate system, and if the distance between the point cloud detection position and the reference map position is smaller than a preset distance threshold value, taking the highlight point cloud set as a target point cloud set;

And calculating pose information of the movable equipment by using the target point cloud set.

5. The method of claim 4, wherein before the converting the Gao Liangdian cloud set to the map coordinate system corresponding to the scene to be located, obtaining the point cloud detection position of the reflective marker with respect to the map coordinate system, the method further comprises:

based on the coordinates of each point in the Gao Liangdian cloud set, respectively calculating average coordinates corresponding to the highlight point cloud set;

Calculating the distance difference between the average coordinates corresponding to the highlight point cloud sets among a plurality of continuous point cloud frames;

converting the Gao Liangdian cloud set to a map coordinate system corresponding to the scene to be positioned to obtain a point cloud detection position of the reflective marker aiming at the map coordinate system, wherein the method comprises the following steps:

and converting the highlight point cloud set with the distance difference meeting the preset condition into a map coordinate system corresponding to the scene to be positioned, and obtaining the point cloud detection position of the reflective marker aiming at the map coordinate system.

6. The method of claim 1, wherein the mobile device is further deployed with an image acquisition apparatus; the method further comprises the steps of:

if the distance between the movable equipment and the reflective marker is detected to be smaller than a preset distance threshold value, acquiring a scene image of the scene to be positioned by using the image acquisition device;

and combining the scene image and the Gao Liangdian cloud set, and carrying out pose recognition on the movable equipment.

7. The method of claim 6, wherein an image marker is disposed relative to the retroreflective marker; the combining the scene image and the Gao Liangdian cloud set, performing pose recognition on the movable device, includes:

dividing the scene image into a concerned area containing the image marker by utilizing the Gao Liangdian cloud set;

and carrying out pose recognition on the movable equipment based on the image characteristics of the region of interest.

8. The method of claim 7, wherein the partitioning the scene image into the region of interest containing the image markers using the Gao Liangdian cloud set comprises:

calculating average coordinates corresponding to the highlight point cloud set based on the coordinates of each point in the Gao Liangdian cloud set;

converting the average coordinates of the Gao Liangdian cloud set into a pixel coordinate system corresponding to the scene image to obtain image coordinates of the reflective marker in the scene image; acquiring a deployment position relationship between the reflective marker and the image marker;

And dividing the scene image into a region of interest containing the image marker based on the image coordinates and the deployment position relationship.

9. The method of claim 6, wherein the combining the scene image and the Gao Liangdian cloud set, pose recognition for the mobile device, comprises:

Performing pose recognition on the movable equipment by using the scene image to obtain an image pose recognition result; and carrying out pose recognition on the movable equipment by using the selected highlight point cloud set to obtain a point cloud pose recognition result;

And fusing the image pose recognition result and the point cloud pose recognition result to obtain a fused pose result of the movable equipment.

10. A removable device comprising a memory and a processor for executing program instructions stored in the memory to implement the steps of the method according to any of claims 1-9.