CN114882343A - Slope detection method, robot and storage medium - Google Patents

Abstract

The present application is applicable to the technical field of data processing, and provides a slope detection method, a robot and a storage medium. The method includes: collecting a depth image through a depth camera, converting the depth image into point cloud data, and obtaining a corresponding depth image according to the point cloud data. The first grayscale image of the first grayscale image is obtained by using the grayscale value of the first pixel in the first grayscale image to obtain the gradient value of the first pixel. Flood fill is done to determine the sloped area of the ground. The present application uses the depth image collected by the depth camera to determine the slope area. The area collected by the depth camera is large, and the resolution of the depth camera is high. Compared with the use of laser radar to determine the slope area on the ground in the prior art, the present application can be faster and more efficient. More accurate determination of sloped areas of the ground.

Description

A slope detection method, robot and storage medium

technical field

The present application belongs to the technical field of data processing, and in particular, relates to a slope detection method, a robot and a storage medium.

Background technique

With the development of robots, robots have been able to move and navigate autonomously. The key to autonomous navigation of robots lies in the detection of the ground. Through the detection of the ground, infeasible areas on the ground, such as obstacles, cliffs, etc., are identified, and the movement route of the robot is planned according to the infeasible areas.

At present, the detection of the ground is mostly stopped to determine whether there are obstacles or cliffs on the ground, and the detection of the slope area on the ground is lacking, and the robot's ability to perceive the external environment is insufficient.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a slope detection method, a robot, and a storage medium, which can solve the problem that the robot cannot perceive the external environment due to the inability to detect the slope area on the ground.

In a first aspect, an embodiment of the present application provides a slope detection method, including:

acquiring a first grayscale image of a depth image collected by a depth camera, where the depth image includes the ground;

Calculate the gradient value of the first pixel point based on the grayscale value of the first pixel point in the first grayscale image, and the number of the first pixel point is multiple;

determining a key pixel point from the first pixel point based on the gradient value of the first pixel point;

Determine the slope area in the first grayscale image based on the grayscale value of the key pixel point and the grayscale value of the second pixel point, wherein the second pixel point is in the first grayscale image For the first pixel points other than the key pixel points, the slope area in the first grayscale image is used to determine the slope area of the ground.

In a second aspect, an embodiment of the present application provides a slope detection device, including:

an image acquisition module, configured to acquire a first grayscale image of a depth image collected by a depth camera, where the depth image includes the ground;

a gradient calculation module, configured to calculate the gradient value of the first pixel point based on the grayscale value of the first pixel point in the first grayscale image, and the number of the first pixel point is multiple;

a key point determination module, configured to determine a key pixel point from the first pixel point based on the gradient value of the first pixel point;

a slope detection module, configured to determine a slope area in the first grayscale image based on the grayscale value of the key pixel point and the grayscale value of the second pixel point, wherein the second pixel point is the The first pixel points other than the key pixel points in the first grayscale image, and the slope area in the first grayscale image is used to determine the slope area of the ground.

In a third aspect, an embodiment of the present application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program When implementing the slope detection method according to any one of the first aspect above.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements any one of the above-mentioned first aspect the slope detection method.

In a fifth aspect, an embodiment of the present application provides a computer program product that, when the computer program product runs on a terminal device, enables the terminal device to execute the slope detection method described in any one of the first aspects above.

Compared with the prior art, the embodiment of the first aspect of the present application has the following beneficial effects: the present application uses the gray value of the first pixel in the first gray image of the depth image collected by the depth camera to obtain the gray value of the first pixel. Gradient value, and then use the gradient value of the first pixel point to obtain the key pixel point, use the gray value of the key pixel point and the gray value of other pixel points to determine the slope area in the first grayscale image, and then determine the slope area of the ground . Compared with the prior art that cannot detect the slope area on the ground, the present application uses the depth image collected by the depth camera to determine the slope area. The area collected by the depth camera is large, and the resolution of the depth camera is high. Determine the slope area of the ground to improve the robot's ability to perceive the external environment.

It can be understood that, for the beneficial effects of the second aspect to the fifth aspect, reference may be made to the relevant description in the first aspect, which is not repeated here.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present application. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic diagram of an application scenario of a slope detection method provided by an embodiment of the present application;

2 is a schematic flowchart of a slope detection method provided by an embodiment of the present application;

3 is a schematic flowchart of a method for determining a first grayscale image provided by an embodiment of the present application;

4 is a schematic flowchart of a method for determining a key pixel point provided by an embodiment of the present application;

5 is a schematic flowchart of a method for determining a slope angle of a pixel point provided by an embodiment of the present application;

6 is a schematic structural diagram of a slope detection device provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.

Detailed ways

It is to be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described feature, integer, step, operation, element and/or component, but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or sets thereof.

It will also be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and should not be construed as indicating or implying relative importance.

References in this specification to "one embodiment" or "some embodiments" and the like mean that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically emphasized otherwise.

At present, when using lidar or RGB camera for ground detection, the effect of ground detection can be achieved. The slope detection method provided by the present application uses the depth image collected by the depth camera to determine the slope area on the ground. Since the depth image includes distance and spatial information, when using the depth camera to detect the slope on the ground, not only can the slope area on the ground be determined , you can also determine the angle of the slope, the type of slope, etc. In addition, depth cameras are cheaper than lidars. Therefore, using depth cameras for road detection can save costs compared to using lidars for road detection.

FIG. 1 is a schematic diagram of an application scenario of a slope detection method provided by an embodiment of the present application, and the above slope detection method can be used to detect a slope area on the ground. Among them, the depth camera 10 is used to collect depth images of the ground. The data processing device 20 is used for acquiring the depth image collected by the depth camera 10, and outputting the slope area in the depth image according to the depth image.

FIG. 2 shows a schematic flowchart of the slope detection method provided by the present application. Referring to FIG. 2 , the method is described in detail as follows:

S101: Acquire a first grayscale image of a depth image collected by a depth camera, where the depth image includes the ground.

In this embodiment, the depth camera is a camera that can perform depth measurement. The depth camera may be a depth camera installed on the robot; the depth camera may also be a depth camera installed in a fixed area. The installation angle of the depth camera is based on the images collected on the ground. For example, the depth camera on the robot can collect information on the ground in front of the robot, and the front of the robot is the moving direction of the robot.

A depth image, also known as a range image, refers to an image that uses the distance (depth) values of each point in the scene captured by the depth camera as the pixel value, which directly reflects the geometry of the visible surface of the scene.

Grayscale image, also known as grayscale image, divides white and black into several levels according to the logarithmic relationship, which is called grayscale.

In this embodiment, the grayscale value of the pixel point in the first grayscale image is determined based on the pixel value (depth) of the pixel point in the depth image.

S102: Calculate the gradient value of the first pixel point based on the grayscale value of the first pixel point in the first grayscale image, where the number of the first pixel point is multiple.

In this embodiment, each pixel in the first grayscale image may correspond to one grayscale value. The first pixel point may include each pixel point in the first grayscale image, and a pixel point in an area in the first grayscale image may be recorded as the first pixel point, and the area includes the ground. For example, if the first grayscale image The figure includes a chair, and the pixels in the area other than the chair area are recorded as the first pixel.

In this embodiment, the gradient value of the first pixel point may include a first-order gradient value and a second-order gradient value.

Calculate the first-order derivative of the first grayscale image to obtain the first-order gradient value of the first pixel. Calculate the derivative of the first-order gradient value to obtain the second-order gradient value of the first pixel point.

Specifically, the coordinate system where the first grayscale image is located is established, for example, the coordinate system is established with the lower left corner of the first grayscale image as the origin and the two sides of the first grayscale image passing through the origin as the x-axis and the y-axis. Calculate the first-order gradient of the first pixel in the x-axis direction and the first-order gradient on the y-axis to obtain the first-order gradient value of the first pixel, and then obtain the first-order gradient map of the first grayscale image. The gradient is calculated for the first-order gradient map to obtain the second-order gradient map of the first grayscale image.

Specifically, the value of the first pixel in the first-order gradient graph that is greater than or equal to the first threshold is set to the first value (for example, 255), and the values of other first pixels are set to the second value (for example, 0). , to get the first binary image. The first threshold can be set as required.

The value of the first pixel point in the second-order gradient graph that is less than or equal to the second threshold is set as the first value, and the values of other first pixel points are set as the second value to obtain a second binary image. The second threshold can be set as required.

In the first grayscale image, the first pixel that has the first value in the first binary image and is also the first value in the second binary image is denoted as the third pixel, and the third pixel is denoted as the third pixel. The value of is set to the first value, and the values of the other first pixels are set to the second value to obtain the third binary image.

S103: Determine a key pixel point from the first pixel point based on the gradient value of the first pixel point.

In this embodiment, the image marked with the gradient value of the first pixel is input into the key point extraction model to obtain the key pixel in the first pixel. The key point extraction model can be a convolutional neural network model, a fully connected neural network, or the like.

The number of key pixel points is one or more.

S104: Determine a slope area in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of a second pixel, where the second pixel is the first grayscale The first pixel in the image other than the key pixel.

In this embodiment, the slope area in the first grayscale image is used to determine the slope area of the ground.

In this embodiment, each key pixel point is a seed point, and the slope area corresponding to each key pixel point is calculated by using the flood filling method.

In the embodiment of the present application, the gray value of the first pixel in the first gray image of the depth image collected by the depth camera is used to obtain the gradient value of the first pixel, and then the gradient value of the first pixel is used to obtain the key pixel point, and use the gray value of the key pixel point and the gray value of other pixel points to determine the slope area in the first grayscale image, and then determine the slope area of the ground. In this application, the depth image collected by the depth camera is used to determine the slope area. The area collected by the depth camera is large, for example, the depth camera can reach 480 lines, and the resolution of the depth camera is high. ) to determine the slope area of the ground, the present application can determine the slope area of the ground more quickly and accurately, and the application uses a depth camera to detect the slope area of the ground, which reduces the use cost compared to the use of lidar to detect the slope area. In addition, using the depth camera to detect the ground does not require high hardware. For example, the central processing unit (CPU) and memory usage are not high, and hardware with low computing power can be selected to reduce resource consumption. The depth camera has no strict restrictions on the installation angle, position and height, and the installation is simple. Using the depth camera on the robot to determine the slope area of the ground can improve the robot's ability to perceive the environment and ensure the safety of the robot's movement.

As shown in FIG. 3, in a possible implementation manner, the implementation process of step S101 may include:

S1011. Convert the depth image into first point cloud data in a camera coordinate system, where the camera coordinate system is a coordinate system determined based on the pose of the depth camera.

In this embodiment, dimensionality reduction processing is performed on the data in the depth image to obtain the first point cloud data in the camera coordinate system. The camera coordinate system includes an x-axis, a y-axis and a z-axis. The x-axis and the y-axis represent the coordinates of the pixel point, and the z-axis is the pixel value (depth) of the pixel point. Converting the depth image to point cloud data in the camera coordinate system is to prepare for converting the depth image to point cloud data in the standard coordinate system.

S1012: Convert the first point cloud data into second point cloud data in a standard coordinate system, where the standard coordinate system is a world coordinate system determined based on a horizontal plane.

In this embodiment, coordinate transformation is performed on the first point cloud data to obtain the second point cloud data. The x-axis and the y-axis of the standard coordinate system are parallel to the horizontal plane and perpendicular to each other; the z-axis in the standard coordinate system is perpendicular to the x-axis and the y-axis. The second point cloud data includes x-axis data, y-axis data, and z-axis data.

S1013, normalize the height value in the second point cloud data to obtain a normalized height value, where the height value is used to indicate that the ground is higher than the origin of the standard coordinate system degree.

In this embodiment, the height value (z-axis data) in the second point cloud data is normalized, for example, normalized to the interval [-0.42m, 0.22m], and the normalized value is obtained. The height value to make data processing simple and fast.

S1014, obtaining a first grayscale image of the depth image based on the normalized height value.

In this embodiment, the normalized height value is converted into grayscale image data (0-255) to obtain a second grayscale image. Perform median filter processing and dimension reduction processing on the second grayscale image to obtain a first grayscale image.

Optionally, perform reverse normalization on the first grayscale image to obtain filtered z-axis data, and the x-axis data, y-axis data, and filtered z-axis data in the second point cloud data to form a filter. post point cloud data.

In this embodiment of the present application, the depth image is converted into the first grayscale image, so as to determine the slope area according to the first grayscale image. Converting the point cloud data into an image, the processing speed of the image can reach the millisecond level, so it is easier and faster to determine the slope area using the first grayscale image.

As shown in FIG. 4 , in a possible implementation manner, the implementation process of step S103 may include:

S1031: Determine a third pixel point in the first pixel point that meets a preset requirement, where the preset requirement includes that the absolute value of the first-order gradient value is greater than or equal to the first threshold, and the absolute value of the second-order gradient value is less than or equal to the second threshold.

In this embodiment, the first pixel with the first value in the third binary image is the third pixel.

S1032: Determine a first candidate region in the first grayscale image based on the position of the third pixel in the first grayscale image.

In this embodiment, the first candidate area is determined in the third binary image, and the area corresponding to the third binary image in the first grayscale image is the first candidate area.

Specifically, the adjacent third pixels are grouped into one area, and the adjacent third pixels are located adjacent to each other to obtain the first candidate area. Adjacent may include being adjacent in the x-axis direction, or adjacent in the y-axis direction. For example, the third pixel point A is the pixel point in the ith row and the jth column in the first grayscale image, the third pixel point B is the pixel point in the ith row and the j+1th column in the first grayscale image, and the third pixel point The pixel point C is the pixel point in the i+1th row and the jth column in the first grayscale image, the third pixel point D is the pixel point in the i+1th row and j+1th column in the first grayscale image, and the third pixel point D is the pixel point in the i+1th row and j+1th column in the first grayscale image. The pixel point A, the third pixel point B, the third pixel point C, and the third pixel point D form a first candidate area.

S1033: Determine a second candidate region based on the area of the first candidate region, where the second candidate region is a first candidate region whose area is greater than or equal to a preset area in the first candidate region.

In this embodiment, the first candidate regions are eroded and expanded to determine the contour of each first candidate region, and then the area of each first candidate region is calculated according to the contour of each first candidate region.

S1034: Determine the center of gravity of the second candidate region.

In this embodiment, the center of gravity of the second candidate area may be within the second candidate area, or may be outside the second area. Each second candidate region corresponds to a centroid point.

S1035: Determine a key pixel point from the first pixel points included in the second candidate area based on the positional relationship between the second candidate area and the center of gravity of the second candidate area.

In this embodiment, different strategies are selected to determine the key pixel points according to whether the center of gravity is inside or outside the second candidate region. A centroid point can determine a key pixel point, therefore, a second candidate region corresponds to a key pixel point.

In the embodiment of the present application, the first candidate area is first determined according to the gradient value of the first pixel point, and then the area of the first candidate area is screened to obtain the second candidate area. The second candidate area is more accurate than the first candidate area. Finally, the key pixel points of the second candidate area are determined according to the center of gravity of the second candidate area, so that when the slope area is determined according to the key pixel points, the obtained slope area is more accurate.

In a possible implementation manner, the implementation process of step S1035 may include:

S10351. The first centroid point in the second candidate region, and the key pixel point includes a first pixel point corresponding to the first centroid point.

In this embodiment, if the center of gravity of the second candidate region is within the second candidate region, the center of gravity is recorded as the first center of gravity. The first pixel point corresponding to the first centroid point is the key pixel point of the second candidate area. The key pixel points of the first grayscale image include the first centroid point.

S10352: For a second centroid point that is not in the second candidate region, determine a fourth pixel point based on the position of the second centroid point in the first grayscale image.

In this embodiment, the fourth pixel is a first pixel in the second candidate region that is on the same row as the second center of gravity and corresponding to the second center of gravity, and is the same as the first pixel in the second candidate region. The double centroid points are in the same column and are the first pixel points in the second candidate region corresponding to the second centroid point.

In this embodiment, if the centroid point of the second candidate area is not within the second candidate area, the centroid point is recorded as the second centroid point. Find the first pixel in the same line and column as the second center of gravity in the first grayscale image, and record the first pixel in the second candidate area and in the same row and column as the second center of gravity as the first pixel Four pixels.

S10353: Determine a fifth pixel with the smallest distance from the second center of gravity among the fourth pixels, where the key pixel includes the fifth pixel.

In this embodiment, the distance between each fourth pixel point and the second center of gravity point is calculated to obtain each distance value. The fourth pixel point corresponding to the minimum value of all the distance values is recorded as the fifth pixel point, and the key pixel point of the second candidate area is the fifth pixel point. The key pixel points of the first grayscale image include the fifth pixel point.

In a possible implementation manner, the implementation process of step S104 may include:

Calculate the first difference between the gray value of the key pixel point and the gray value of the second pixel point.

If the first difference is within a preset interval, the key is obtained based on the position of the second pixel corresponding to the first difference within the preset interval in the first grayscale image The slope area corresponding to the pixel point, the slope area in the first grayscale image includes the slope area corresponding to all key pixel points.

In this embodiment, the slope area is determined by the flood filling method according to the key pixel points.

Specifically, it is calculated and diffused gradually outward with each key pixel as the center. Specifically, with the key pixel as the center, in the direction from near to far away from the key pixel, calculate the grayscale difference between the key pixel and the second pixel in turn, and record the grayscale difference as the first difference. When the first difference value is within the preset interval, the diffusion is continued, and the second pixel point corresponding to the first difference value within the preset interval and the corresponding key pixel point form a slope area. Until the first difference is not within the preset interval, the diffusion is stopped to obtain the final slope area.

The preset interval can be set as required. A key pixel corresponds to a slope area, and there may be intersections between different slope areas. The point cloud data of the first pixel point of the slope area is determined according to the second point cloud data.

If the first difference value is not within the preset interval, the second pixel point corresponding to the first difference value not within the preset interval and the corresponding key pixel point cannot form a slope area.

As shown in FIG. 5, in a possible implementation manner, after step S104, the above method may further include:

S201 , performing a plane fitting process on the slope area to obtain a fitting plane of the slope area.

In this embodiment, a plane fitting process is performed on each slope area to obtain the fitting plane of each slope area.

S202: Calculate the slope angle between the fitting plane and the horizontal plane, where the slope angle between the fitting plane and the horizontal plane is the slope angle of the slope area.

S203, for the sixth pixel point, the sixth pixel point is a first pixel point existing in a slope area among the first pixel points, and the slope angle of the slope area where the sixth pixel point is located is the first pixel point in the slope area. The slope angle of the six-pixel point.

In this embodiment, if the first pixel exists in a slope area, the first pixel is recorded as the sixth pixel. The slope angle of the sixth pixel is the slope angle of the slope region where the sixth pixel is located.

S204, for a seventh pixel point, the seventh pixel point is a first pixel point existing in at least two slope areas in the first pixel point, and the slope angle in the slope area of the seventh pixel point is to be included The maximum value is taken as the slope angle of the seventh pixel point.

In this embodiment, if the first pixel exists in at least two slope regions, the first pixel is recorded as the seventh pixel. The maximum value of the slope angles of the at least two slope regions is determined, and the maximum value of the slope angles of the at least two slope regions is used as the slope angle of the seventh pixel point.

As an example, if the seventh pixel is in the slope area A and the slope area B, the slope angle of the slope area A is 20 degrees, the slope angle of the slope area B is 24 degrees, and the slope angle of the seventh pixel point is 24 degrees.

In this embodiment, the slope angle of the slope area can be determined by the plane fitting method, and then the slope angle of each first pixel point in the slope area is determined according to the slope angle, so as to output the slope angle of each first pixel point in the slope area , to get richer slope data.

In a possible implementation manner, after step S103, the above method may further include:

Calculate the second difference value of the gray value of the first pixel point in the i-1th row and the jth column of the first grayscale image minus the gray value of the first pixel point in the i-th row and the jth column, so The first pixel in the i-th row and the j-th column is the first pixel in the slope area.

If the second difference value is greater than a preset value, it is determined that the slope type of the first pixel point in the i-th row and the j-th column relative to the first pixel point in the i-1-th row and the j-th column is a downward slope. The slope type of the first pixel point in the i-1th row and the jth column relative to the first pixel point in the i-th row and the jth column is an uphill slope. The default value can be 0.

If the second difference is smaller than the preset value, determine that the slope type of the first pixel in the i-th row and the j-th column relative to the first pixel in the i-1-th row and the j-th column is up. slope. The slope type of the first pixel point in the i-1th row and the jth column relative to the first pixel point in the i-th row and the jth column is downslope.

If the second difference is equal to the preset value, determine that the slope type of the first pixel in the i-th row and the j-th column relative to the first pixel in the i-1-th row and the j-th column is parallel. .

In this embodiment, the gray value of the first pixel in the i-th row and the j-th column minus the gray value of the first pixel in the i-th row and the j-1 column can also be calculated. If If the third difference is greater than the preset value, the slope type of the first pixel in the i-th row and the j-th column relative to the first pixel in the i-th row and the j-1 column is an uphill slope. If the third difference is smaller than the preset value, the slope type of the first pixel in the i-th row and the j-th column relative to the first pixel in the i-th row and the j-1 column is a downslope. If the third difference is equal to the preset value, the slope type of the first pixel in the i-th row and the j-th column relative to the first pixel in the i-th row and the j-1-th column is parallel.

In this embodiment, if the depth camera is installed on the robot, the depth image is an image of the ground ahead captured when the robot moves forward or needs to move forward. The first pixel in row i is closer to the depth camera than the first pixel in row i-1. If the first pixel at the i-th row and the j-th column is a point in the slope area, calculate the gray value of the first pixel at the i-1-th row and the j-th column minus the first pixel at the i-th row and the j-th column. The gray value of , obtains the fourth difference value. If the fourth difference is greater than the preset value, the slope type of the first pixel point in the i-th row and the j-th column is uphill, that is, when the robot moves to the first pixel point in the i-th row and the j-th column, if it continues to move Need to go uphill. If the fourth difference is less than the preset value, the slope type of the first pixel point in the i-th row and the j-th column is downhill, that is, when the robot moves to the first pixel point in the i-th row and the j-th column, if it continues to move Need to go downhill.

In the embodiment of the present application, the slope type of each first pixel in the slope area can be determined according to the gray value of the pixel point, and then the slope type of the first pixel in the slope area is output to obtain more abundant slope data.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Corresponding to the slope detection method described in the above embodiment, FIG. 6 shows a structural block diagram of the slope detection apparatus provided by the embodiment of the present application. For convenience of description, only the part related to the embodiment of the present application is shown.

Referring to FIG. 6 , the apparatus 300 may include: an image acquisition module 310 , a gradient calculation module 320 , a key point determination module 330 and a slope detection module 340 .

Wherein, the image acquisition module 310 is configured to acquire the first grayscale image of the depth image collected by the depth camera, and the depth image includes the ground;

a gradient calculation module 320, configured to calculate the gradient value of the first pixel point based on the grayscale value of the first pixel point in the first grayscale image, and the number of the first pixel point is multiple;

a key point determination module 330, configured to determine a key pixel point from the first pixel point based on the gradient value of the first pixel point;

The slope detection module 340 is configured to determine the slope area in the first grayscale image based on the grayscale value of the key pixel point and the grayscale value of the second pixel point, wherein the second pixel point is the The first pixel points other than the key pixel points in the first grayscale image, and the slope area in the first grayscale image is used to determine the slope area of the ground.

In a possible implementation manner, the gradient value of the first pixel point includes a first-order gradient value and a second-order gradient value;

The key point determination module 330 can be specifically used for:

Determine the third pixel point in the first pixel point that meets the preset requirements, the preset requirements include that the absolute value of the first-order gradient value is greater than or equal to the first threshold value, and the absolute value of the second-order gradient value is less than or equal to the first threshold value. two thresholds;

determining a first candidate region in the first grayscale image based on the position of the third pixel in the first grayscale image;

determining a second candidate area based on the area of the first candidate area, wherein the second candidate area is a first candidate area in the first candidate area whose area is greater than or equal to a preset area;

determining the center of gravity of the second candidate region;

Based on the positional relationship between the second candidate area and the center of gravity of the second candidate area, key pixels are determined from the first pixels included in the second candidate area.

In a possible implementation manner, the key point determination module 330 may specifically be used for:

The first centroid point in the second candidate region, the key pixel point includes the first pixel point corresponding to the first centroid point;

For the second centroid point that is not in the second candidate area, a fourth pixel is determined based on the position of the second centroid in the first grayscale image, wherein the fourth pixel is the same as the The second centroid point is in the same row and the first pixel point in the second candidate region corresponding to the second centroid point, and the second centroid point is in the same column and in the second the first pixel in the second candidate region corresponding to the center of gravity;

A fifth pixel point with the smallest distance from the second center of gravity point among the fourth pixel points is determined, and the key pixel point includes the fifth pixel point.

In a possible implementation manner, the slope detection module 340 may be specifically used for:

Calculate the first difference between the gray value of the key pixel point and the gray value of the second pixel point;

If the first difference is within a preset interval, the key is obtained based on the position of the second pixel corresponding to the first difference within the preset interval in the first grayscale image The slope area corresponding to the pixel point, the slope area in the first grayscale image includes the slope area corresponding to all key pixel points.

In a possible implementation manner, the image acquisition module 310 may be specifically used for:

converting the depth image into first point cloud data in a camera coordinate system, where the camera coordinate system is a coordinate system determined based on the pose of the depth camera;

Converting the first point cloud data into second point cloud data under a standard coordinate system, where the standard coordinate system is a world coordinate system determined based on a horizontal plane;

Normalizing the height value in the second point cloud data to obtain a normalized height value, where the height value is used to represent the degree to which the ground is higher than the origin of the standard coordinate system;

Based on the normalized height value, a first grayscale image of the depth image is obtained.

In a possible implementation manner, connected to the slope detection module 340 further includes:

a plane fitting module, configured to perform a plane fitting process on the slope area to obtain a fitting plane of the slope area;

The angle calculation module is configured to calculate the slope angle between the fitting plane and the horizontal plane, and the slope angle between the fitting plane and the horizontal plane is the slope angle of the slope area.

In a possible implementation manner, connected to the angle calculation module further includes:

The first angle determination module is used for the sixth pixel point, the sixth pixel point is the first pixel point in the first pixel point that exists in a slope area, and the sixth pixel point is located in the slope area. The slope angle is the slope angle of the sixth pixel point;

The second angle determination module is configured to target the seventh pixel point, the seventh pixel point is the first pixel point existing in at least two slope regions in the first pixel point, and will include the seventh pixel point The maximum value of the slope angle in the slope region of , is taken as the slope angle of the seventh pixel point.

In a possible implementation manner, connected to the slope detection module 340 further includes:

A difference value calculation module, configured to calculate the gray value of the first pixel in the i-1th row and the jth column of the first grayscale image minus the gray value of the first pixel in the i-th row and the jth column. The first difference value of , the first pixel point of the i-th row and the j-th column is the pixel point in the slope area;

A first judgment module, configured to determine the first pixel point of the i-th row and the j-th column relative to the first pixel point of the i-1-th row and the j-th column if the first difference value is greater than a preset value The slope type is downhill;

The second judgment module is configured to determine, if the first difference is smaller than the preset value, the first pixel in the i-th row and the j-th column relative to the first pixel in the i-1-th row and the j-th column. The slope type of the pixel point is uphill;

A third judging module, configured to determine, if the first difference is equal to the preset value, the first pixel in the i-th row and the j-th column relative to the first pixel in the i-1-th row and the j-th column. The slope type of the pixel is parallel.

It should be noted that the information exchange, execution process and other contents between the above-mentioned devices/units are based on the same concept as the method embodiments of the present application. For specific functions and technical effects, please refer to the method embodiments section. It is not repeated here.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

An embodiment of the present application further provides a terminal device. Referring to FIG. 7 , the terminal device 400 may include: at least one processor 410 , a memory 420 , and a terminal device stored in the memory 420 and available on the at least one processor 410 The running computer program, when the processor 410 executes the computer program, implements the steps in any of the foregoing method embodiments, for example, steps S101 to S104 in the embodiment shown in FIG. 2 . Alternatively, when the processor 410 executes the computer program, the functions of the modules/units in the above device embodiments, such as the functions of the modules 310 to 340 shown in FIG. 6 , are implemented.

Exemplarily, the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 420 and executed by the processor 410 to complete the present application. The one or more modules/units may be a series of computer program segments capable of accomplishing specific functions, and the program segments are used to describe the execution process of the computer program in the terminal device 400 .

Those skilled in the art can understand that FIG. 7 is only an example of a terminal device, and does not constitute a limitation on the terminal device. It may include more or less components than the one shown in the figure, or combine some components, or different components, such as Input and output devices, network access devices, buses, etc.

The processor 410 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf processors Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 420 may be an internal storage unit of the terminal device, or may be an external storage device of the terminal device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card (Flash Card) etc. The memory 420 is used to store the computer program and other programs and data required by the terminal device. The memory 420 may also be used to temporarily store data that has been output or will be output.

The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, a Peripheral Component (Peripheral Component, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, or the like. The bus can be divided into address bus, data bus, control bus and so on. For convenience of representation, the buses in the drawings of the present application are not limited to only one bus or one type of bus.

The slope detection method provided by the embodiments of the present application can be applied to terminal devices such as computers, tablet computers, notebook computers, netbooks, and personal digital assistants (PDAs), and the embodiments of the present application do not impose any restrictions on the specific types of the terminal devices. .

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed terminal device, apparatus and method may be implemented in other manners. For example, the terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components. May be combined or may be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer When the program is executed by one or more processors, the steps of the foregoing method embodiments can be implemented.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer When the program is executed by one or more processors, the steps of the foregoing method embodiments can be implemented.

Likewise, as a computer program product, when the computer program product runs on a terminal device, the steps in the foregoing method embodiments can be implemented when the terminal device is executed.

Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Excluded are electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still be used for the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims (10)

1. A method of slope detection, comprising:

acquiring a first gray image of a depth image acquired by a depth camera, wherein the depth image comprises the ground;

calculating gradient values of first pixel points in the first gray image based on gray values of the first pixel points, wherein the number of the first pixel points is multiple;

determining key pixel points from the first pixel points based on the gradient values of the first pixel points;

determining a slope region in the first gray image based on the gray value of the key pixel point and the gray value of a second pixel point, wherein the second pixel point is a first pixel point in the first gray image except the key pixel point, and the slope region in the first gray image is used for determining the slope region of the ground.

2. The slope detection method according to claim 1, wherein the gradient values of the first pixel point include a gradient value and a second gradient value;

determining a key pixel point from the first pixel points based on the gradient value of the first pixel point, including:

determining a third pixel point which meets preset requirements in the first pixel points, wherein the preset requirements comprise that the absolute value of a gradient value is greater than or equal to a first threshold value, and the absolute value of a second gradient value is less than or equal to a second threshold value;

determining a first candidate region in the first gray image based on the position of the third pixel point in the first gray image;

determining a second candidate region based on the area of the first candidate region, wherein the second candidate region is a first candidate region of the first candidate region, and the area of the second candidate region is larger than or equal to a preset area;

determining a center of gravity point of the second candidate region;

and determining key pixel points from first pixel points included in the second candidate region based on the position relation between the second candidate region and the gravity center points of the second candidate region.

3. The slope detection method according to claim 2, wherein the determining a key pixel point from first pixel points included in the second candidate region based on the second candidate region and a positional relationship of the gravity center points of the second candidate region includes:

a first center of gravity point in the second candidate region, wherein the key pixel points comprise first pixel points corresponding to the first center of gravity point;

determining a fourth pixel point based on the position of the second gravity center point in the first gray-scale image, wherein the fourth pixel point is a first pixel point which is in the same row with the second gravity center point and is in the second candidate area corresponding to the second gravity center point, and a first pixel point which is in the same column with the second gravity center point and is in the second candidate area corresponding to the second gravity center point;

and determining a fifth pixel point with the minimum distance from the second center of gravity point in the fourth pixel points, wherein the key pixel points comprise the fifth pixel point.

4. The slope detection method of claim 1, wherein the determining the slope region in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of the second pixel comprises:

calculating a first difference value between the gray value of the key pixel point and the gray value of the second pixel point;

if the first difference value is within a preset interval, obtaining a slope area corresponding to the key pixel point based on the position of the second pixel point corresponding to the first difference value within the preset interval in the first gray-scale image, wherein the slope area in the first gray-scale image comprises slope areas corresponding to all key pixel points.

5. The slope detection method according to any one of claims 1 to 4, wherein the acquiring a first grayscale image of the depth image acquired by the depth camera comprises:

converting the depth image into first point cloud data in a camera coordinate system, wherein the camera coordinate system is a coordinate system determined based on the pose of the depth camera;

converting the first point cloud data into second point cloud data under a standard coordinate system, wherein the standard coordinate system is a world coordinate system determined based on a horizontal plane;

normalizing the height value in the second point cloud data to obtain a normalized height value, wherein the height value is used for representing the degree that the ground is higher than the origin of the standard coordinate system;

and obtaining a first gray image of the depth image based on the height value after the normalization processing.

6. The slope detection method according to any one of claims 1 to 4, wherein after determining the slope region in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of the second pixel, the method further comprises:

performing plane fitting processing on the slope area to obtain a fitting plane of the slope area;

and calculating the slope angle between the fitting plane and the horizontal plane, wherein the slope angle between the fitting plane and the horizontal plane is the slope angle of the slope area.

7. The slope detection method of claim 6, wherein after the calculating a slope angle between the fitted plane and a horizontal plane, the method further comprises:

aiming at a sixth pixel point, the sixth pixel point is a first pixel point existing in a slope region in the first pixel points, and the slope angle of the slope region where the sixth pixel point is located is the slope angle of the sixth pixel point;

and aiming at a seventh pixel point, the seventh pixel point is a first pixel point existing in at least two slope regions in the first pixel point, and the maximum value of the slope angle in the slope region comprising the seventh pixel point is used as the slope angle of the seventh pixel point.

8. The slope detection method according to any one of claims 1 to 4, wherein after determining the slope region in the first grayscale image based on the grayscale value of the key pixel and the grayscale value of the second pixel, the method further comprises:

calculating a second difference value of the gray value of the first pixel point in the ith-1 th row and the jth column in the first gray image minus the gray value of the first pixel point in the ith row and the jth column, wherein the first pixel point in the ith row and the jth column is the first pixel point in the slope region;

if the second difference value is larger than a preset value, determining that the slope type of the first pixel point in the ith row and the jth column relative to the first pixel point in the ith-1 row and the jth column is downhill;

if the second difference is smaller than the preset value, determining that the slope type of the first pixel point of the ith row and the jth column relative to the first pixel point of the ith-1 row and the jth column is an ascending slope;

and if the second difference is equal to the preset value, determining that the slope type of the first pixel point of the ith row and the jth column is parallel to that of the first pixel point of the ith-1 row and the jth column.

9. A robot comprising a depth camera, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements a hill detection method as recited in any one of claims 1 to 8.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out a hill detection method according to any one of claims 1 to 8.

Images