CN112773272A - Moving direction determining method and device, sweeping robot and storage medium - Google Patents

Moving direction determining method and device, sweeping robot and storage medium Download PDF

Info

Publication number
CN112773272A
CN112773272A CN202011602527.3A CN202011602527A CN112773272A CN 112773272 A CN112773272 A CN 112773272A CN 202011602527 A CN202011602527 A CN 202011602527A CN 112773272 A CN112773272 A CN 112773272A
Authority
CN
China
Prior art keywords
image
line segment
ceiling
determining
ceiling image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011602527.3A
Other languages
Chinese (zh)
Other versions
CN112773272B (en
Inventor
杨勇
吴泽晓
曾浩宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen 3irobotix Co Ltd
Original Assignee
Shenzhen 3irobotix Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen 3irobotix Co Ltd filed Critical Shenzhen 3irobotix Co Ltd
Priority to CN202011602527.3A priority Critical patent/CN112773272B/en
Publication of CN112773272A publication Critical patent/CN112773272A/en
Priority to PCT/CN2021/139946 priority patent/WO2022143284A1/en
Application granted granted Critical
Publication of CN112773272B publication Critical patent/CN112773272B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/24—Floor-sweeping machines, motor-driven
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4002—Installations of electric equipment
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4011—Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/04—Automatic control of the travelling movement; Automatic obstacle detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing

Abstract

The application relates to a moving direction determining method, a moving direction determining device, a sweeping robot and a storage medium. The method comprises the following steps: acquiring a ceiling image shot by the sweeping robot; performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image; determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment; and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image. By adopting the method, the covering efficiency and the cleaning effect of the sweeping robot can be improved.

Description

Moving direction determining method and device, sweeping robot and storage medium
Technical Field
The application relates to the technical field of cleaning, in particular to a moving direction determining method and device, a sweeping robot and a storage medium.
Background
With the development of artificial intelligence technology, a sweeping robot appears. The sweeping robot can automatically finish cleaning work in a certain space by means of certain artificial intelligence. The sweeping robot is provided with a movable device with an automatic technology and a vacuum dust collection device with a dust collection box, so that the sweeping robot can repeatedly walk indoors according to a set path and can perform sweeping work in the walking process, and a humanoid cleaning effect is achieved.
In the conventional art, the sweeping robot records an initial direction and moves linearly in the initial direction. If an obstacle is encountered, a straight-line motion is made along the edge of the obstacle. If the distance of the obstacle moving along the edge of the obstacle is larger than or equal to the threshold value, the obstacle is determined to be a wall body, the obstacle moves in the direction parallel to or perpendicular to the edge of the obstacle, and cleaning work is carried out in the moving process. If the distance of the obstacle moving along the edge of the obstacle is smaller than the threshold value, the obstacle is determined to be not a wall body, and the obstacle moves linearly in the initial direction and performs cleaning work in the moving process.
However, if the edge of the wall is not parallel or perpendicular to the edge of the entire moving space, the moving track of the sweeping robot may not cover the entire moving space, which reduces the cleaning effect of the sweeping robot.
Disclosure of Invention
In view of the above, it is necessary to provide a moving direction determining method, a moving direction determining apparatus, a sweeping robot, and a storage medium, which can improve coverage efficiency.
A method of movement direction determination, the method comprising:
acquiring a ceiling image shot by the sweeping robot;
performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image;
determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment;
and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
In one embodiment, the performing morphological image processing on the ceiling image to determine the distribution position of the line segment in the ceiling image includes:
converting the ceiling image to a grayscale image;
converting the gray level image into a binary image;
performing morphological operation on the binary image by adopting the structural elements to obtain a line segment image;
and determining the distribution position of the line segments in the ceiling image based on the line segment image.
In one embodiment, the determining the distribution position of the line segments in the ceiling image based on the line segment image includes:
determining a distribution area of an object in the line segment image according to the value of each pixel in the line segment image;
determining a maximum distance between two pixels in the object based on the distribution area of the object;
and determining the distribution area of the object with the maximum distance reaching the length threshold value as the distribution position of the line segment in the ceiling image.
In one embodiment, the performing morphological image processing on the ceiling image to determine the distribution position of the line segment in the ceiling image further includes:
searching pixels with brightness values reaching a brightness threshold value in the ceiling image to obtain the distribution positions of the light sources;
selecting the distribution position of the line segment in the ceiling image based on the distribution position of the light source, wherein the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value.
In one embodiment, the determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment includes:
determining two pixels with the farthest distance in each line segment based on the distribution positions of the line segments in the ceiling image;
determining the inclination angle of a connecting line between two pixels which are farthest away in each line segment as the inclination angle of the line segment;
and determining the inclination angle with the largest quantity in the inclination angles of the line segments as the inclination angle of the line segment in the ceiling image.
In one embodiment, the acquiring a ceiling image taken by the sweeping robot includes:
in the moving process of the sweeping robot, shooting a ceiling image at set time intervals and storing the ceiling image;
and acquiring a plurality of ceiling images with the shooting time closest to the current time.
In one embodiment, the determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image includes:
arranging the obtained plurality of ceiling images according to the size of the inclination angle;
selecting the ceiling images arranged at the middlemost;
and determining the moving direction of the sweeping robot according to the selected inclination angle of the line segment in the ceiling image.
A movement direction determination apparatus, the apparatus comprising:
the acquisition module is used for acquiring a ceiling image shot by the sweeping robot;
the processing module is used for carrying out morphological image processing on the ceiling image and determining the distribution position of the line segment in the ceiling image;
the angle determining module is used for determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment;
and the direction determining module is used for determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
A sweeping robot comprising a memory and a processor, the memory storing a computer program which when executed by the processor effects the steps of:
acquiring a ceiling image shot by the sweeping robot;
performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image;
determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment;
and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a ceiling image shot by the sweeping robot;
performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image;
determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment;
and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
According to the moving direction determining method and device, the sweeping robot and the storage medium, the ceiling image shot by the sweeping robot is obtained, the morphological image processing can be carried out on the ceiling image, the distribution position of the line segment in the ceiling image is determined, the inclination angle of the line segment in the ceiling image can be determined according to the distribution position of the line segment, and finally the moving direction of the sweeping robot is determined according to the inclination angle of the line segment in the ceiling image. Since the line segment in the ceiling image is generally the edge line of the wall, the line segment inclination angle obtained by the sweeping robot based on the captured ceiling image can indicate the inclination direction of the wall edge line. The direction of movement, determined by the angle of inclination of the line segments in the ceiling image, is substantially parallel or perpendicular to the direction of the wall. At the moment, the sweeping robot can move according to the Chinese character 'gong' type track, the whole area to be cleaned is covered comprehensively, and the cleaning effect is effectively improved.
Drawings
FIG. 1 is a diagram of an application environment of a method for determining a moving direction in one embodiment;
fig. 2 is a schematic structural diagram of the sweeping robot in one embodiment;
FIG. 3 is a flow diagram illustrating a method for determining a direction of movement according to one embodiment;
FIG. 4 is a schematic illustration of a ceiling image in one embodiment;
FIG. 5 is a diagram of morphologically processing an image in one embodiment;
FIG. 6 is a schematic diagram of line segment tilt angles in one embodiment;
FIG. 7 is a schematic illustration of determining a direction of movement in one embodiment;
FIG. 8 is a block diagram showing the structure of a moving direction determining apparatus according to an embodiment;
fig. 9 is an internal structure view of the sweeping robot in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The moving direction determining method provided by the application can be applied to the application environment shown in fig. 1. The sweeping robot 102 moves in an area to be cleaned 106 defined by the wall 104, and performs sweeping operation during the moving process, so as to achieve a personified cleaning effect. In this embodiment, the sweeping robot 106 can move along the zigzag track 108 to cover the entire cleaning area 104, thereby achieving the best cleaning effect.
In order to move along the zigzag track 108, the sweeping robot 102 needs to control the moving direction to be parallel or perpendicular to the wall 104. In this embodiment, the sweeping robot 102 captures images of the ceiling during movement. And performing morphological image processing on the ceiling image to determine the distribution position of the line segment in the ceiling image. And determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment. And determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image. The sweeping robot 102 can thus move in the direction of the inclination of the line segment in the ceiling. Since the line segment in the ceiling image is generally the edge of the wall 104, the sweeping robot 102 can be kept parallel to the wall 104, and can move along the zigzag track 108 to cover the whole area 104 to be cleaned, thereby achieving the best cleaning effect.
As shown in fig. 2, the sweeping robot may include a base 202, a traveling mechanism 204, a cleaning mechanism 206, and a shooting device 208. The traveling mechanism 204 and the cleaning mechanism 206 are disposed at the bottom of the housing 202, and the photographing apparatus 208 is disposed at the top of the housing 202. The traveling mechanism 204 can drive the base 202, and the cleaning mechanism 206 and the shooting device 208 on the base 202 to move in the area to be cleaned. The cleaning mechanism 206 may perform sweeping operations on the area to be cleaned during the movement. The shooting device 208 can acquire a shot ceiling image in the moving process, so that the sweeping robot can move according to a Chinese character 'gong' type track to completely cover the whole area to be cleaned, and the best cleaning effect is achieved.
In one embodiment, as shown in fig. 3, a moving direction determining method is provided, which is described by taking the method as an example for the sweeping robot in fig. 1 or fig. 2, and includes the following steps:
and step S302, acquiring a ceiling image shot by the sweeping robot.
Wherein the ceiling image includes a ceiling image. The ceiling is a place of the indoor top surface of a building. The edge of the ceiling is overlapped with the edge of the wall, and the ceiling image comprises the edge line of the wall. For example, in the ceiling image shown in fig. 4, the edge of the ceiling 402 coincides with the edge of the wall 404, and the ceiling image includes an edge line 403 common to the ceiling 402 and the wall 404.
In particular, the sweeping robot moves and cleans on the floor opposite the ceiling. The top of the sweeping robot is provided with a shooting device, and a camera of the shooting device faces the ceiling, so that the ceiling image can be shot. In addition, a storage device is arranged in the sweeping robot, and the photographed ceiling image can be stored. The ceiling image may be acquired from a photographing apparatus or may be acquired from a memory.
In practical application, a camera of the shooting equipment can face to the right upper side, namely, the camera faces to a ceiling; it is also possible to direct the image obliquely upwards, i.e. obliquely towards the ceiling, e.g. obliquely forwards, which is advantageous in ensuring that the wall edge lines are included in the ceiling image in the case of a large area to be cleaned. The camera of the shooting device can also rotate on a vertical plane to ensure that the ceiling image contains the edge line of the wall.
In this embodiment, through obtaining the ceiling image that the robot of sweeping the floor shot, include the edge line of wall body in the ceiling image, the robot of sweeping the floor can be based on the edge line of wall body, moves along the direction of parallel or perpendicular to wall body, realizes carrying out the bow style of calligraphy and removes in whole clean area, covers whole the area of waiting to clean comprehensively, improves the cleaning effect of robot of sweeping the floor.
Illustratively, the ceiling images are shot at intervals of set time length or at each time of movement of the set length, so that the moving direction of the floor sweeping robot can be changed in time according to the change of the wall, the situation that the floor sweeping robot deviates from a Chinese character 'gong' track due to the situations of deviation, errors and the like in the moving process of the floor sweeping robot is avoided, and the coverage efficiency of the floor sweeping robot is ensured. For example, a ceiling image is taken once at time a, once at time a + B, once at time a +2 × B, once at time a +3 × B, … …, and so on. For another example, the ceiling image is captured once when the movement length reaches C, once when the movement distance reaches C + D, once when the movement distance reaches C +2 × D, once when the movement distance reaches C +3 × D, … …, and so on.
And step S304, performing morphological image processing on the ceiling image, and determining the distribution position of the line segment in the ceiling image.
Among them, Morphology, i.e., mathematical Morphology (mathematical Morphology), is one of the most widely used techniques in image processing. The morphological image processing is mainly used for extracting image components which are meaningful for expressing and describing the shape of the region from the image, so that the most essential shape features of the target object, such as the most distinguishing shape features (most distinguishing shape) of a boundary, a connected region and the like, can be caught by the subsequent recognition work.
The morphologically processed image is based on shape. Morphological processing images use a structured parameter in the input image to obtain an output image of the same size. The value of each pixel in the output image is based on the contrast between the corresponding pixel in the input image and its neighborhood. By selecting the size and shape of the neighborhood, a morphological operation can be constructed that is sensitive to a particular shape in the input image.
In particular, by appropriate selection of the size and shape of the neighborhood, a morphological operation can be constructed that is sensitive to line segments in the ceiling image. This morphological operation is applied to the ceiling image to obtain an output image of the same size. The value of the pixels corresponding to the line segments in the output image is greatly different from the value of the pixels corresponding to the non-line segments, so that the line segment characteristics can be highlighted. Meanwhile, according to the numerical value of the pixel, whether the pixel belongs to the line segment or not can be obviously distinguished, pixels corresponding to all the line segments are found, and then the distribution position of the line segments in the ceiling image is determined. For example, the image shown in fig. 5 can be obtained by performing morphological image processing on the ceiling image shown in fig. 4. In the image shown in fig. 5, the pixels 502 corresponding to the line segments are black, and the pixels 504 corresponding to the non-line segments are white, and according to the values of the pixels, all the pixels in black are determined as the pixels corresponding to the line segments, and the distribution positions of all the pixels in black are determined as the distribution positions of the line segments.
And the distribution position of the line segment is the distribution position of the line segment in the ceiling image.
In this embodiment, the distribution position of the line segment in the ceiling image can be determined by performing morphological image processing on the ceiling image, and the inclination angle of the line segment in the ceiling image can be further determined. The line segments in the ceiling image are generally arranged on the edge line of the wall body, the sweeping robot moves according to the inclination direction of the line segments based on the inclination angle of the line segments in the ceiling image, the moving direction is parallel to or perpendicular to the wall body, the arched movement in the whole cleaning area can be realized, the whole cleaning area to be cleaned is completely covered, and the cleaning effect of the sweeping robot is improved.
And S306, determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment.
The inclination angle of the line segment is the angle between the positive direction of the X axis and the upward direction of the line segment in a rectangular coordinate system with the bottom of the ceiling image as the X axis and the left side of the ceiling image as the Y axis. The bottom and left sides of the ceiling image are both oriented as determined when a person is facing the ceiling image.
For example, fig. 6 is obtained by creating a rectangular coordinate system in the image shown in fig. 5, and as shown in fig. 6, the X-axis of the rectangular coordinate system is the bottom side of the image, and the Y-axis of the rectangular coordinate system is the left side of the image. The line segment 602 is inclined at an angle α and the line segment 604 is inclined at an angle β.
Specifically, according to the distribution position of the line segments, an equation of a straight line where the line segments are located may be established based on a rectangular coordinate system with the bottom side of the ceiling image as the X axis and the left side of the ceiling image as the Y axis. And then according to the slope in the linear equation, obtaining the inclination angle of the line segment by utilizing an inverse trigonometric function.
In practical application, a rectangular coordinate system with the bottom side of the ceiling image as an X axis and the left side of the ceiling image as a Y axis is established, and coordinates (X, Y) of each pixel where a line segment is located can be obtained. Based on the coordinates (x, y) of each pixel, a straight line is fitted by the least square method, and a straight line equation y ═ k × x + b is obtained. And substituting the slope k in the linear equation into the arctan function arctan to obtain the result, namely the inclination angle of the line segment.
In this embodiment, the moving direction of the sweeping robot can be determined according to the inclination angle of the line segment in the ceiling image by determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment. The line segments in the ceiling image are generally arranged on the edge line of the wall body, the sweeping robot moves according to the inclination direction of the line segments based on the inclination angle of the line segments in the ceiling image, the moving direction is parallel to or perpendicular to the wall body, the arched movement in the whole cleaning area can be realized, the whole cleaning area to be cleaned is completely covered, and the cleaning effect of the sweeping robot is improved.
And step S308, determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
The moving direction of the sweeping robot refers to the advancing direction of the sweeping robot.
Specifically, a three-dimensional coordinate system is established by taking the floor sweeping robot as a reference, the forward direction of an X axis is the advancing direction of the floor sweeping robot, a Y axis is parallel to the moving plane of the floor sweeping robot and is vertical to the X axis, and a Z axis is vertical to the moving plane of the floor sweeping robot. And taking the inclination angle of the line segment in the ceiling image as the positive anticlockwise deflection angle of the X axis, wherein the positive deflected X axis is the moving direction of the sweeping robot.
For example, as shown in FIG. 7, the line segment 702 in the ceiling image is tilted at an angle γ, deflecting the X-axis forward counterclockwise from the first direction 704 by the angle γ to the second direction 706. The first direction 704 is perpendicular to the bottom edge of the ceiling image, so that the sum of the deflection angle γ of the positive X-axis and the angle θ between the second direction 706 and the bottom edge of the ceiling image is 90 °, the sum of the inclination angle γ of the line segment 702 in the ceiling image and the angle θ between the second direction 706 and the bottom edge of the ceiling image is also 90 °, and the second direction 706 is perpendicular to the line segment 702 in the ceiling image, facilitating movement in a zigzag trajectory.
In this embodiment, the moving direction of the sweeping robot is determined according to the inclination angle of the line segment in the ceiling image. The line segments in the ceiling image are generally arranged on the edge line of the wall body, the sweeping robot moves according to the inclination direction of the line segments based on the inclination angle of the line segments in the ceiling image, the moving direction is parallel to or perpendicular to the wall body, the arched movement in the whole cleaning area can be realized, the whole cleaning area to be cleaned is completely covered, and the cleaning effect of the sweeping robot is improved.
In the moving direction determining method, the ceiling image shot by the sweeping robot is obtained, the morphological image processing can be performed on the ceiling image, the distribution position of the line segment in the ceiling image is determined, the inclination angle of the line segment in the ceiling image can be determined according to the distribution position of the line segment, and finally the moving direction of the sweeping robot is determined according to the inclination angle of the line segment in the ceiling image. Since the line segment in the ceiling image is generally the edge line of the wall, the line segment inclination angle obtained by the sweeping robot based on the captured ceiling image can indicate the inclination direction of the wall edge line. The direction of movement, determined by the angle of inclination of the line segments in the ceiling image, is substantially parallel or perpendicular to the direction of the wall. At the moment, the sweeping robot can move according to the Chinese character 'gong' type track, the whole area to be cleaned is covered comprehensively, and the cleaning effect is effectively improved.
In one embodiment, the morphological image processing is performed on the ceiling image, and the determining the distribution position of the line segment in the ceiling image comprises: converting the ceiling image into a grayscale image; converting the gray level image into a binary image; performing morphological operation on the binary image by adopting the structural elements to obtain a line segment image; based on the line segment image, the distribution position of the line segment in the ceiling image is determined.
Where a grayscale image is an image with only one sample color per pixel. Such images are typically displayed in gray scale from darkest black to brightest white. Grayscale images are different from black and white images, which have only two colors, and grayscale images have many levels of color depth between black and white.
The binary image is an image with each pixel point having only two values. The gray value of any pixel in such an image is 0 or 255, which represents black and white, respectively.
The structural elements are the basic transformation structures for iterative operations in mathematical morphology. The most basic shape of the structural element is square, rectangle or diamond, and can be used for carrying out operations such as corrosion, expansion and the like.
The most basic morphological operations are dilation and erosion. Dilation increases pixels at the edge of the object in one image and erosion removes pixels from the edge of the object. Adding or removing pixels in an image is based on the size and shape of the structuring parameters used to process the image. In morphological dilation and erosion operations, the properties of any given pixel in the output image are determined by using a criterion on the corresponding pixel in the input image and its neighborhood. The criteria for processing the pixels define the operation as dilation or erosion.
The value of the dilated output pixel is the maximum value of all pixels in the input pixel neighborhood. The expansion enables the object boundaries to be enlarged, the specific expansion result being related to the shape of the image itself and the structural elements. Dilation is often used to bridge the same object in the image that was originally broken apart. After the image is binarized, a communicated object is easily broken into two parts, which causes trouble for subsequent image analysis, and at the moment, the broken gap can be bridged by means of expansion.
The value of the eroded output pixel is the minimum of all pixels in the neighborhood of the input pixel. Erosion can ablate the boundaries of an object, with the specific erosion result being related to the shape of the image itself and the structural elements. If the object is larger than the structural elements on the whole, the corroded structure makes the object become thin for one circle, and the size of the circle is determined by the structural elements; if the object itself is smaller than the structural element, the object will disappear completely in the image after erosion.
The opening operation is to corrode and expand firstly, the opening operation can enable the outline of the image to become smooth, narrow connection can be disconnected, and fine burrs can be eliminated. If the structural elements are adopted to carry out the opening operation on the binary image, the obtained line segment image does not have obvious influence on the shape and the outline of an object while filtering noise, and the subsequent accurate determination of the distribution position of the line segment is facilitated.
The morphological gradient is the expansion map minus the erosion map, which results in the profile of the object. If the structural elements are adopted to carry out morphological gradient operation on the binary image, the accurate line segment image is favorably obtained.
The line segment image is an image with only line segments distributed. Based on the difference between the pixels corresponding to the line segments and other pixels, the distribution positions of the line segments on the image can be obtained, and therefore the distribution positions of the line segments in the ceiling image are determined.
Specifically, the grayscale image may be formed by first extracting the grayscale value of each pixel from the ceiling image. And comparing the gray value of each pixel with a classification threshold value to determine whether the pixel belongs to white or black to form a binary image. For example, if the gray value of the pixel is less than the classification threshold, the pixel is determined to be black; and if the gray value of the pixel is greater than or equal to the classification threshold, determining that the pixel belongs to white. And then, carrying out morphological operation on the binary image by adopting the structural elements, and filtering out non-line segment characteristics in the binary image to obtain a line segment image. And finally, taking the distribution position of the pixels corresponding to the line segment in the line segment image as the distribution position of the line segment in the ceiling image.
In this embodiment, the interference of the color size of the pixel on the judgment line segment can be filtered out by converting the ceiling image into the gray image. By converting the gray level image into a binary image, the interference of the color depth of the pixel on the judgment line segment can be filtered. By adopting the structural elements to perform the psychology operation on the binary image, the interference of non-line segment characteristics can be filtered out, and the line segment image only with the line segment characteristics is obtained. And finally, determining the distribution position of the line segments in the ceiling image based on the line segment image.
In one embodiment, determining the distribution position of the line segments in the ceiling image based on the line segment image comprises: determining a distribution area of an object in the line segment image according to the value of each pixel in the line segment image; determining a maximum distance between two pixels in the object based on the distribution area of the object; and determining the distribution area of the object with the maximum distance reaching the length threshold value as the distribution position of the line segment in the ceiling image.
Wherein, the value of the pixel is one of two values in the binary image. For example, if the value of a pixel is 0, the pixel appears black; if the value of the pixel is 1, the pixel is white.
The line segment image includes an object and a background. The value of the pixel corresponding to the object and the value of the pixel corresponding to the background are usually different. For example, the value of the pixel corresponding to the object is 0, and the value of the pixel corresponding to the background is 1.
The objects in the line-segment image are typically line segments, and may also have other interferents.
Specifically, the value of the pixel is one of two values in the binary image, and the value of the pixel corresponding to the object is different from the value of the pixel corresponding to the background. If the value of the pixel corresponding to the object is 0 and the value of the pixel corresponding to the background is 1, the distribution position of each pixel with the value of 0 in the line segment image can be used as the distribution area of the object in the line segment image. And calculating the distance between every two pixels in the continuous area with the value of 0 in the line segment image, and taking the maximum value as the maximum distance between two pixels in the object. If the maximum distance reaches the length threshold value, the object belongs to the wall body edge line, and the distribution area is determined as the distribution position of the line segment in the ceiling image. If the maximum distance is smaller than the length threshold value, the surface object is not the wall body edge line, and the distribution area cannot be determined as the distribution position of the line segment in the ceiling image, so that the sweeping robot is prevented from being guided to the wrong direction.
In practical application, a disturbing object may exist on the ceiling, and the accurate identification of the line segment is influenced. In this embodiment, the distribution area of the object in the line segment image is determined according to the value of each pixel in the line segment image, and the maximum distance between two pixels in the object is determined based on the distribution area of the object, so that the length of the object can be obtained. The length of the object is compared with a set length threshold value, if the length of the object reaches the length threshold value, the object can be determined to be a line segment, and at the moment, the distribution position of the line segment in the ceiling image can be further determined, so that the moving direction of the sweeping robot can be determined according to the inclination angle of the line segment in the ceiling image, the sweeping robot can move according to the Chinese character 'gong' shaped track, the whole area to be cleaned is covered comprehensively, and the best cleaning effect is achieved. If the length of the object is smaller than the length threshold value, the object can be determined not to be a line segment, and at the moment, the next step is stopped, so that the situation that the sweeping robot is guided to the wrong direction by the interferent can be avoided.
Illustratively, the length threshold may be 20 pixels.
In one embodiment, the morphological image processing is performed on the ceiling image, and the distribution position of the line segment in the ceiling image is determined, further comprising: searching pixels with brightness values reaching a brightness threshold value in the ceiling image to obtain the distribution positions of the light sources; based on the distribution position of the light source, the distribution position of the line segment in the ceiling image is selected, and the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value.
In particular, the luminance values of the individual pixels in the ceiling image may be compared with a luminance threshold value, respectively. If the brightness value of the pixel reaches the brightness threshold value, determining that the pixel belongs to the light source; if the brightness value of the pixel is smaller than the brightness threshold value, the pixel is determined not to belong to the light source. The distribution position of each pixel belonging to the light source is taken as the distribution position of the light source. If the distance between the distribution position of the line segment in the ceiling image and the distribution position of the light source is smaller than the distance threshold value, the line segment may be the light source or the shadow of the light source, and at this time, the line segment needs to be filtered. If the distance between the distribution position of the line segment in the ceiling image and the distribution position of the light source reaches the distance threshold value, the line segment can be excluded from being the shadow of the light source or the light source, the distribution position of the line segment is selected for subsequent processing, the moving direction of the sweeping robot can be parallel to or perpendicular to a wall body, the sweeping robot can move in a bow shape in the whole cleaning area, the whole area to be cleaned is completely covered, and the cleaning effect of the sweeping robot is improved.
In practical applications, if a strip light source is provided on the ceiling, accurate identification of the line segment may be affected. In this embodiment, the distribution position of the light source can be obtained by searching for a pixel whose luminance value reaches the luminance threshold value in the ceiling image. And selecting the distribution position of the line segment in the ceiling image based on the distribution position of the light source, wherein the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value. Therefore, the line segment distribution positions with the distance between the line segment distribution positions and the light source distribution positions smaller than the distance threshold value can be filtered. If the distance between the distribution position of the line segment in the ceiling image and the distribution position of the light source is smaller than the distance threshold value, the line segment can be the light source or the shadow of the light source, the distribution position of the line segment is filtered, and the situation that the sweeping robot is guided to the wrong direction by the light source can be avoided.
In one embodiment, determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment comprises: determining two pixels with the farthest distance in each line segment based on the distribution positions of the line segments in the ceiling image; determining the inclination angle of a connecting line between two pixels which are farthest away in each line segment as the inclination angle of the line segment; and determining the inclination angle with the largest number in the inclination angles of the line segments as the inclination angle of the line segment in the ceiling image.
Specifically, first, of pixels in which line segments are distributed in the ceiling image, two adjacent pixels are determined to belong to the same line segment. And then, calculating the distance between every two pixels in the pixels distributed on the same line segment. And then connecting the two pixels corresponding to the maximum value. And then taking the inclination angle of the connecting line as the inclination angle of the line segment. And finally, selecting the inclination angle with the maximum number of line segments as the inclination angle of the line segments in the ceiling image.
In practical applications, the edge of the ceiling may be decorated with a plurality of plaster lines, in which case the number of line segments in the ceiling image may be more than one. In this embodiment, for each line segment, two pixels farthest away from the line segment are determined based on the distribution position of the line segment, and the inclination angle of the connection line between the two pixels farthest away from the line segment is used as the inclination angle of the line segment. Then, the inclination angle with the largest quantity in the inclination angles of all the line segments is used as the inclination angle of the line segment in the ceiling image. Since most of the plaster lines are parallel to the edge lines of the wall, most of the line segments are inclined at angles parallel to the edge lines of the wall. The inclination angle of the line segment with the largest number is selected, the moving direction of the sweeping robot can be parallel to or perpendicular to the direction of the wall body, and then the sweeping robot moves according to the Chinese character 'gong' type track, so that the whole area to be cleaned is comprehensively covered, and the cleaning effect is effectively improved.
In one embodiment, acquiring a ceiling image taken by a sweeping robot includes: in the moving process of the sweeping robot, a ceiling image is shot and stored every set time length; and acquiring a plurality of ceiling images with the shooting time closest to the current time.
In practical applications, a single ceiling image has the possibility of being affected by a disturbing object. In the embodiment, the ceiling images at different positions can be obtained by shooting one ceiling image every set time length in the moving process of the sweeping robot, so that the influence of interference in a single ceiling image is eliminated.
Specifically, the sweeping robot can move according to a set track in an initial state, for example, the sweeping robot leaves a charging seat and keeps linear motion, and a ceiling image is shot and stored every set time length in the moving process until the ceiling image with the set number is obtained, and the moving direction is determined and then adjusted. For example, the set number of the ceiling images is five, the sweeping robot moves according to the set track between the time a and the time a + 4B, the first ceiling image is captured and stored at the time a, the second ceiling image is captured and stored at the time a + B, the third ceiling image is captured and stored at the time a + 2B, the fourth ceiling image is captured and stored at the time a + 3B, and the fifth ceiling image is captured and stored at the time a + 4B. And then acquiring the five ceiling images for processing, and determining the moving direction of the sweeping robot so that the sweeping robot continues to move according to the determined moving direction.
The sweeping robot can move according to the last determined moving direction in the moving process, and a ceiling image is shot and stored every set time length in the moving process. And the moving direction of the sweeping robot is determined again by the ceiling image shot each time and the ceiling image stored before, and the sweeping robot is enabled to move continuously according to the determined moving direction.
In one embodiment, determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image comprises: arranging the acquired multiple ceiling images according to the size of the inclination angle; selecting the ceiling images arranged at the middlemost; and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the selected ceiling image.
Specifically, any one of the ceiling images may be arranged in the queue, and then the remaining ceiling images may be compared with the ceiling images in the queue in order by the magnitude of the inclination angle, and the ceiling images may be arranged in the queue in accordance with the magnitude relationship.
For example, the sweeping robot takes a first image of the ceiling with an inclination angle of 0 °, a second image of the ceiling with an inclination angle of 5 °, a third image of the ceiling with an inclination angle of 10 °, a fourth image of the ceiling with an inclination angle of 8 °, and a fifth image of the ceiling with an inclination angle of 5 °.
The first ceiling image is arranged in the queue, and then the second ceiling image is compared with the first ceiling image in the queue for the magnitude of the inclination angle. Since the inclination angle of the line segment in the second ceiling image is larger than that of the line segment in the first ceiling image, the second ceiling image is arranged behind the first ceiling image in the queue.
The third ceiling image is then compared to the first and second ceiling images in the queue in order for the tilt angle to be greater or lesser. Since the inclination angle of the line segment in the third ceiling image is larger than both the inclination angle of the line segment in the second ceiling image and the inclination angle of the line segment in the first ceiling image, the third ceiling image is arranged behind the second ceiling image in the line.
And then comparing the fourth ceiling image with the first ceiling image to the third ceiling image in the queue in sequence. Since the inclination angle of the line segment in the fourth ceiling image is smaller than the comparative inclination angle of the third ceiling image and larger than both the inclination angle of the line segment in the second ceiling image and the inclination angle of the line segment in the first ceiling image, the fourth ceiling image is arranged in the queue behind the second ceiling image and in front of the third ceiling image.
And finally, comparing the inclination angle of the fifth ceiling image with the first ceiling image to the fourth ceiling image in the queue in sequence. Since the inclination angle of the line segment in the fifth ceiling image is equal to the inclination angle of the line segment in the second ceiling image, it is sufficient to arrange the fifth ceiling image one before or one after the second ceiling image in the queue.
Therefore, the finally obtained queues are a first ceiling image, a second ceiling image, a fifth ceiling image, a fourth ceiling image and a third ceiling image; alternatively, the first ceiling image, the fifth ceiling image, the second ceiling image, the fourth ceiling image, and the third ceiling image.
And selecting the ceiling images arranged at the middlemost from the queue, and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the selected ceiling images. Also in the above example, the selected ceiling image is the fifth ceiling image or the second ceiling image, and the inclination angles of the line segments in the selected ceiling image are all 5 °, and the moving direction of the sweeping robot is determined based on this 5 °.
In practical application, a plurality of ceiling images at different positions are acquired to eliminate the influence of interference in a single ceiling image. In this embodiment, arrange a plurality of ceiling images through the size according to inclination with acquireing to arrange the line segment inclination in the middle of the ceiling image, as the moving direction of robot of sweeping the floor, can adjust the direction of robot of sweeping the floor gradually, avoid the direction mistake that single adjustment range too big appears. And the inclination angle of the line segment in the single ceiling image belonging to the interference should be the foremost or the rearmost, and the ceiling image arranged at the middle is selected, so that the interference of the single ceiling image can be effectively eliminated.
In one embodiment, the method further comprises: acquiring a current moving direction; and if the included angle between the current moving direction and the determined moving direction reaches the included angle threshold value, moving according to the determined moving direction.
In this embodiment, the current moving direction is obtained to determine whether an included angle between the current moving direction and the determined moving direction reaches an included angle threshold. If the included angle between the current moving direction and the determined moving direction reaches the included angle threshold value, the deviation of the moving direction is large, and the robot moves according to the determined moving direction, so that the moving direction of the sweeping robot can be corrected in time, and the bow-shaped movement is realized. If the included angle between the current moving direction and the determined moving direction is smaller than the included angle threshold value, the deviation of the moving direction is small, and errors possibly caused by the errors are possible.
Illustratively, the angle threshold may be 1 °.
In one embodiment, the method further comprises: if collision is detected, the moving direction of the sweeping robot is rotated by 90 degrees; if the moving distance of the sweeping robot reaches the steering threshold, the moving direction of the sweeping robot is rotated by 90 degrees, and the rotating directions of the two moving directions are the same.
In this embodiment, through detecting the collision, can turn to in time when the robot that sweeps floor removes and reaches the wall body position and adjust the removal route to can continue to carry out the work of cleaning in other regions. When collision is detected, the moving direction of the sweeping robot is rotated by 90 degrees, and the collision can be avoided to the greatest extent. When the moving distance of the sweeping robot reaches the steering threshold value, the moving direction of the sweeping robot is rotated by 90 degrees continuously, the sweeping work in the opposite direction can be carried out, and the sweeping robot can move according to the zigzag path.
Specifically, the sweeping robot can be provided with collision detection equipment, such as a laser detector, for detecting surrounding objects. If the distance between the robot and the surrounding object is detected to be reduced to be below the collision threshold value, the collision is determined to occur, at the moment, the moving direction of the sweeping robot is rotated by 90 degrees, and the robot is prevented from continuing to move linearly and colliding with the object. The sweeping robot can be further provided with at least one of a distance sensor, a speed sensor and a timer so as to determine whether the moving distance of the sweeping robot reaches a steering threshold value. If the moving distance of the sweeping robot reaches the steering threshold value, the moving direction of the sweeping robot is rotated by 90 degrees continuously, the moving direction of the sweeping robot is changed by 180 degrees, the current moving direction is opposite to the original moving direction, and the bow-shaped movement is realized.
It should be understood that, although the steps in the flowchart of fig. 3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 3 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
In one embodiment, as shown in fig. 8, there is provided a movement direction determination apparatus including: an obtaining module 801, a processing module 802, an angle determining module 803, and a direction determining module 804, wherein:
an obtaining module 801, configured to obtain a ceiling image captured by the sweeping robot.
The processing module 802 is configured to perform morphological image processing on the ceiling image, and determine a distribution position of a line segment in the ceiling image.
And the angle determining module 803 is configured to determine the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment.
The direction determining module 804 is configured to determine a moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
According to the movement direction determining device, the ceiling image shot by the sweeping robot is obtained, the morphological image processing can be carried out on the ceiling image, the distribution position of the line segment in the ceiling image is determined, the inclination angle of the line segment in the ceiling image can be further determined according to the distribution position of the line segment, and finally the movement direction of the sweeping robot is determined according to the inclination angle of the line segment in the ceiling image. Since the line segment in the ceiling image is generally the edge line of the wall, the line segment inclination angle obtained by the sweeping robot based on the captured ceiling image can indicate the inclination direction of the wall edge line. The direction of movement, determined by the angle of inclination of the line segments in the ceiling image, is substantially parallel or perpendicular to the direction of the wall. At the moment, the sweeping robot can move according to the Chinese character 'gong' type track, the whole area to be cleaned is covered comprehensively, and the cleaning effect is effectively improved.
In one embodiment, the processing module 802 includes: a first conversion unit, a second conversion unit, an operation unit, and a determination unit, wherein: a first conversion unit for converting the ceiling image into a grayscale image; a second conversion unit for converting the grayscale image into a binary image; the operation unit is used for performing morphological operation on the binary image by adopting the structural elements to obtain a line segment image; and the determining unit is used for determining the distribution position of the line segments in the ceiling image based on the line segment image.
In one embodiment, the determining unit includes: a first determining subunit, a second determining subunit, and a third determining subunit, wherein: the first determining subunit is used for determining the distribution area of the object in the line segment image according to the value of each pixel in the line segment image; a second determining subunit, configured to determine a maximum distance between two pixels in the object based on the distribution area of the object; and the third determining subunit is used for determining the distribution area of the object with the maximum distance reaching the length threshold as the distribution position of the line segment in the ceiling image.
In one embodiment, the processing module 802 further comprises: a search unit and a selection unit, wherein: the searching unit is used for searching pixels of which the brightness values reach a brightness threshold value in the ceiling image to obtain the distribution positions of the light sources; and the selection unit is used for selecting the distribution position of the line segment in the ceiling image based on the distribution position of the light source, and the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value.
In one embodiment, the angle determination module 803 includes: pixel determination unit, angle determination unit and angle selection unit, wherein: the pixel determining unit is used for determining two pixels with the farthest distance in each line segment based on the distribution position of the line segment in the ceiling image; the angle determining unit is used for determining the inclination angle of a connecting line between two pixels with the farthest distance in each line segment as the inclination angle of the line segment; and the angle selection unit is used for determining the inclination angle with the largest number in the inclination angles of the line segments as the inclination angle of the line segment in the ceiling image.
In one embodiment, the obtaining module 801 includes: a photographing unit and an acquisition unit, wherein: the shooting unit is used for shooting and storing a ceiling image every set time length in the moving process of the sweeping robot; and the acquisition unit is used for acquiring a plurality of ceiling images with the shooting time closest to the current time.
In one embodiment, the direction determination module 804 includes: an arrangement unit, an image selection unit, and a direction determination unit, wherein: the arrangement unit is used for arranging the acquired multiple ceiling images according to the size of the inclination angle; an image selecting unit for selecting the ceiling images arranged at the center; and the direction determining unit is used for determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the selected ceiling image.
For the specific definition of the moving direction determining device, reference may be made to the above definition of the moving direction determining method, which is not described herein again. The respective modules in the above-described moving direction determining device may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a sweeping robot is provided, the internal structure of which can be shown in fig. 9. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a movement direction determination method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a sweeping robot is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor realizes the following steps when executing the computer program: acquiring a ceiling image shot by the sweeping robot; performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image; determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment; and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
In one embodiment, the processor, when executing the computer program, further performs the steps of: converting the ceiling image into a grayscale image; converting the gray level image into a binary image; performing morphological operation on the binary image by adopting the structural elements to obtain a line segment image; based on the line segment image, the distribution position of the line segment in the ceiling image is determined.
In one embodiment, the processor, when executing the computer program, further performs the steps of: determining a distribution area of an object in the line segment image according to the value of each pixel in the line segment image; determining a maximum distance between two pixels in the object based on the distribution area of the object; and determining the distribution area of the object with the maximum distance reaching the length threshold value as the distribution position of the line segment in the ceiling image.
In one embodiment, the processor, when executing the computer program, further performs the steps of: searching pixels with brightness values reaching a brightness threshold value in the ceiling image to obtain the distribution positions of the light sources; based on the distribution position of the light source, the distribution position of the line segment in the ceiling image is selected, and the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value.
In one embodiment, the processor, when executing the computer program, further performs the steps of: determining two pixels with the farthest distance in each line segment based on the distribution positions of the line segments in the ceiling image; determining the inclination angle of a connecting line between two pixels which are farthest away in each line segment as the inclination angle of the line segment; and determining the inclination angle with the largest number in the inclination angles of the line segments as the inclination angle of the line segment in the ceiling image.
In one embodiment, the processor, when executing the computer program, further performs the steps of: in the moving process of the sweeping robot, a ceiling image is shot and stored every set time length; and acquiring a plurality of ceiling images with the shooting time closest to the current time.
In one embodiment, the processor, when executing the computer program, further performs the steps of: arranging the acquired multiple ceiling images according to the size of the inclination angle; selecting the ceiling images arranged at the middlemost; and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the selected ceiling image.
According to the sweeping robot, the ceiling image shot by the sweeping robot is obtained, the morphological image processing can be carried out on the ceiling image, the distribution position of the line segment in the ceiling image is determined, the inclination angle of the line segment in the ceiling image can be further determined according to the distribution position of the line segment, and finally the moving direction of the sweeping robot is determined according to the inclination angle of the line segment in the ceiling image. Since the line segment in the ceiling image is generally the edge line of the wall, the line segment inclination angle obtained by the sweeping robot based on the captured ceiling image can indicate the inclination direction of the wall edge line. The direction of movement, determined by the angle of inclination of the line segments in the ceiling image, is substantially parallel or perpendicular to the direction of the wall. At the moment, the sweeping robot can move according to the Chinese character 'gong' type track, the whole area to be cleaned is covered comprehensively, and the cleaning effect is effectively improved.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: acquiring a ceiling image shot by the sweeping robot; performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image; determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment; and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
In one embodiment, the computer program when executed by the processor further performs the steps of: converting the ceiling image into a grayscale image; converting the gray level image into a binary image; performing morphological operation on the binary image by adopting the structural elements to obtain a line segment image; based on the line segment image, the distribution position of the line segment in the ceiling image is determined.
In one embodiment, the computer program when executed by the processor further performs the steps of: determining a distribution area of an object in the line segment image according to the value of each pixel in the line segment image; determining a maximum distance between two pixels in the object based on the distribution area of the object; and determining the distribution area of the object with the maximum distance reaching the length threshold value as the distribution position of the line segment in the ceiling image.
In one embodiment, the computer program when executed by the processor further performs the steps of: searching pixels with brightness values reaching a brightness threshold value in the ceiling image to obtain the distribution positions of the light sources; based on the distribution position of the light source, the distribution position of the line segment in the ceiling image is selected, and the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value.
In one embodiment, the computer program when executed by the processor further performs the steps of: determining two pixels with the farthest distance in each line segment based on the distribution positions of the line segments in the ceiling image; determining the inclination angle of a connecting line between two pixels which are farthest away in each line segment as the inclination angle of the line segment; and determining the inclination angle with the largest number in the inclination angles of the line segments as the inclination angle of the line segment in the ceiling image.
In one embodiment, the computer program when executed by the processor further performs the steps of: in the moving process of the sweeping robot, a ceiling image is shot and stored every set time length; and acquiring a plurality of ceiling images with the shooting time closest to the current time.
In one embodiment, the computer program when executed by the processor further performs the steps of: arranging the acquired multiple ceiling images according to the size of the inclination angle; selecting the ceiling images arranged at the middlemost; and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the selected ceiling image.
The storage medium can perform morphological image processing on the ceiling image by acquiring the ceiling image shot by the sweeping robot, determine the distribution position of the line segment in the ceiling image, further determine the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment, and finally determine the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image. Since the line segment in the ceiling image is generally the edge line of the wall, the line segment inclination angle obtained by the sweeping robot based on the captured ceiling image can indicate the inclination direction of the wall edge line. The direction of movement, determined by the angle of inclination of the line segments in the ceiling image, is substantially parallel or perpendicular to the direction of the wall. At the moment, the sweeping robot can move according to the Chinese character 'gong' type track, the whole area to be cleaned is covered comprehensively, and the cleaning effect is effectively improved.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A method for determining a direction of movement, the method comprising:
acquiring a ceiling image shot by the sweeping robot;
performing morphological image processing on the ceiling image, and determining the distribution position of line segments in the ceiling image;
determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment;
and determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
2. The method according to claim 1, wherein the performing morphological image processing on the ceiling image to determine the distribution position of the line segment in the ceiling image comprises:
converting the ceiling image to a grayscale image;
converting the gray level image into a binary image;
performing morphological operation on the binary image by adopting the structural elements to obtain a line segment image;
and determining the distribution position of the line segments in the ceiling image based on the line segment image.
3. The method of claim 2, wherein determining the distribution position of the line segments in the ceiling image based on the line segment image comprises:
determining a distribution area of an object in the line segment image according to the value of each pixel in the line segment image;
determining a maximum distance between two pixels in the object based on the distribution area of the object;
and determining the distribution area of the object with the maximum distance reaching the length threshold value as the distribution position of the line segment in the ceiling image.
4. The method according to any one of claims 1 to 3, wherein the performing morphological image processing on the ceiling image to determine the distribution position of the line segment in the ceiling image further comprises:
searching pixels with brightness values reaching a brightness threshold value in the ceiling image to obtain the distribution positions of the light sources;
selecting the distribution position of the line segment in the ceiling image based on the distribution position of the light source, wherein the distance between the distribution position of the line segment and the distribution position of the light source reaches a distance threshold value.
5. The method according to any one of claims 1 to 3, wherein the determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment comprises:
determining two pixels with the farthest distance in each line segment based on the distribution positions of the line segments in the ceiling image;
determining the inclination angle of a connecting line between two pixels which are farthest away in each line segment as the inclination angle of the line segment;
and determining the inclination angle with the largest quantity in the inclination angles of the line segments as the inclination angle of the line segment in the ceiling image.
6. The method of any one of claims 1 to 3, wherein the acquiring the ceiling image taken by the sweeping robot comprises:
in the moving process of the sweeping robot, shooting a ceiling image at set time intervals and storing the ceiling image;
and acquiring a plurality of ceiling images with the shooting time closest to the current time.
7. The method of claim 6, wherein determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image comprises:
arranging the obtained plurality of ceiling images according to the size of the inclination angle;
selecting the ceiling images arranged at the middlemost;
and determining the moving direction of the sweeping robot according to the selected inclination angle of the line segment in the ceiling image.
8. A movement direction determination apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring a ceiling image shot by the sweeping robot;
the processing module is used for carrying out morphological image processing on the ceiling image and determining the distribution position of the line segment in the ceiling image;
the angle determining module is used for determining the inclination angle of the line segment in the ceiling image according to the distribution position of the line segment;
and the direction determining module is used for determining the moving direction of the sweeping robot according to the inclination angle of the line segment in the ceiling image.
9. A sweeping robot comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202011602527.3A 2020-12-29 2020-12-29 Moving direction determining method and device, sweeping robot and storage medium Active CN112773272B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202011602527.3A CN112773272B (en) 2020-12-29 2020-12-29 Moving direction determining method and device, sweeping robot and storage medium
PCT/CN2021/139946 WO2022143284A1 (en) 2020-12-29 2021-12-21 Method and apparatus for determining movement direction, and sweeping robot and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011602527.3A CN112773272B (en) 2020-12-29 2020-12-29 Moving direction determining method and device, sweeping robot and storage medium

Publications (2)

Publication Number Publication Date
CN112773272A true CN112773272A (en) 2021-05-11
CN112773272B CN112773272B (en) 2022-10-18

Family

ID=75751736

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011602527.3A Active CN112773272B (en) 2020-12-29 2020-12-29 Moving direction determining method and device, sweeping robot and storage medium

Country Status (2)

Country Link
CN (1) CN112773272B (en)
WO (1) WO2022143284A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114569004A (en) * 2022-02-22 2022-06-03 杭州萤石软件有限公司 Traveling direction adjustment method, mobile robot system, and electronic device
WO2022143284A1 (en) * 2020-12-29 2022-07-07 深圳市杉川机器人有限公司 Method and apparatus for determining movement direction, and sweeping robot and storage medium
CN115359114A (en) * 2022-08-16 2022-11-18 中建一局集团第五建筑有限公司 Positioning method, positioning device, electronic equipment and computer readable storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0662618A1 (en) * 1994-01-10 1995-07-12 Charlatte Method and apparatus for localising a mobile object in an environment
CN102914303A (en) * 2012-10-11 2013-02-06 江苏科技大学 Navigation information acquisition method and intelligent space system with multiple mobile robots
CN103020632A (en) * 2012-11-20 2013-04-03 北京航空航天大学 Fast recognition method for positioning mark point of mobile robot in indoor environment
CN104864889A (en) * 2015-05-29 2015-08-26 山东鲁能智能技术有限公司 System and method for correcting robot speedometer based on visual sense
CN108036786A (en) * 2017-12-01 2018-05-15 安徽优思天成智能科技有限公司 Position and posture detection method, device and computer-readable recording medium based on auxiliary line
CN108387234A (en) * 2018-02-06 2018-08-10 广州科语机器人有限公司 The map creating method of mobile robot based on laser range sensor
CN110362099A (en) * 2018-03-26 2019-10-22 科沃斯机器人股份有限公司 Robot cleaning method, device, robot and storage medium
CN110801180A (en) * 2018-08-03 2020-02-18 速感科技（北京）有限公司 Operation method and device of cleaning robot

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6761786B2 (en) * 2017-08-10 2020-09-30 本田技研工業株式会社 Ceiling map creation method, ceiling map creation device and ceiling map creation program
CN108888188B (en) * 2018-06-14 2020-09-01 深圳市无限动力发展有限公司 Sweeping robot position calibration method and system
CN112773272B (en) * 2020-12-29 2022-10-18 深圳市杉川机器人有限公司 Moving direction determining method and device, sweeping robot and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0662618A1 (en) * 1994-01-10 1995-07-12 Charlatte Method and apparatus for localising a mobile object in an environment
CN102914303A (en) * 2012-10-11 2013-02-06 江苏科技大学 Navigation information acquisition method and intelligent space system with multiple mobile robots
CN103020632A (en) * 2012-11-20 2013-04-03 北京航空航天大学 Fast recognition method for positioning mark point of mobile robot in indoor environment
CN104864889A (en) * 2015-05-29 2015-08-26 山东鲁能智能技术有限公司 System and method for correcting robot speedometer based on visual sense
CN108036786A (en) * 2017-12-01 2018-05-15 安徽优思天成智能科技有限公司 Position and posture detection method, device and computer-readable recording medium based on auxiliary line
CN108387234A (en) * 2018-02-06 2018-08-10 广州科语机器人有限公司 The map creating method of mobile robot based on laser range sensor
CN110362099A (en) * 2018-03-26 2019-10-22 科沃斯机器人股份有限公司 Robot cleaning method, device, robot and storage medium
CN110801180A (en) * 2018-08-03 2020-02-18 速感科技（北京）有限公司 Operation method and device of cleaning robot

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022143284A1 (en) * 2020-12-29 2022-07-07 深圳市杉川机器人有限公司 Method and apparatus for determining movement direction, and sweeping robot and storage medium
CN114569004A (en) * 2022-02-22 2022-06-03 杭州萤石软件有限公司 Traveling direction adjustment method, mobile robot system, and electronic device
WO2023160305A1 (en) * 2022-02-22 2023-08-31 杭州萤石软件有限公司 Travelling direction adjusting method, mobile robot system and electronic device
CN114569004B (en) * 2022-02-22 2023-12-01 杭州萤石软件有限公司 Travel direction adjustment method, mobile robot system and electronic device
CN115359114A (en) * 2022-08-16 2022-11-18 中建一局集团第五建筑有限公司 Positioning method, positioning device, electronic equipment and computer readable storage medium

Also Published As

Publication number Publication date
CN112773272B (en) 2022-10-18
WO2022143284A1 (en) 2022-07-07

Similar Documents

Publication Publication Date Title
CN112773272B (en) Moving direction determining method and device, sweeping robot and storage medium
CN110807350B (en) System and method for scan-matching oriented visual SLAM
KR101643672B1 (en) Optical flow tracking method and apparatus
US9519968B2 (en) Calibrating visual sensors using homography operators
CN109001757B (en) Parking space intelligent detection method based on 2D laser radar
JP4725490B2 (en) Automatic tracking method
RU2627914C1 (en) Device for calculation of own position and method of calculation of own position
JP7209115B2 (en) Detection, 3D reconstruction and tracking of multiple rigid objects moving in relatively close proximity
CN107678551B (en) Gesture recognition method and device and electronic equipment
US9747507B2 (en) Ground plane detection
JP6924932B2 (en) Mobile tracking methods, mobile tracking devices, and programs
KR102525570B1 (en) Method of removing outliers in lidar data for lidar-camera image fusion and computing device performing the same method
CN111142514B (en) Robot and obstacle avoidance method and device thereof
JP6399485B2 (en) Travel path recognition device
CN109143167B (en) Obstacle information acquisition device and method
JP2018022247A (en) Information processing apparatus and control method thereof
CN110264397B (en) Method and device for extracting effective region of fisheye image
JP5177030B2 (en) Target tracking device
EP3098682B1 (en) Moving object controller, program, and integrated circuit
US6650362B1 (en) Movement detecting apparatus with feature point extractor based on luminance gradient in current frame
KR101677171B1 (en) Moving object segmentation method by the pixel-based background estimation
CN112154479A (en) Method for extracting feature points, movable platform and storage medium
JP2021050931A (en) Attached matter detection device and attached matter detection method
US11227166B2 (en) Method and device for evaluating images, operating assistance method, and operating device
US20200167005A1 (en) Recognition device and recognition method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant