CN114569004A - Traveling direction adjustment method, mobile robot system, and electronic device - Google Patents

Traveling direction adjustment method, mobile robot system, and electronic device Download PDF

Info

Publication number
CN114569004A
CN114569004A CN202210161527.7A CN202210161527A CN114569004A CN 114569004 A CN114569004 A CN 114569004A CN 202210161527 A CN202210161527 A CN 202210161527A CN 114569004 A CN114569004 A CN 114569004A
Authority
CN
China
Prior art keywords
line segment
indoor
top image
target
indoor top
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210161527.7A
Other languages
Chinese (zh)
Other versions
CN114569004B (en
Inventor
陈豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Ezviz Software Co Ltd
Original Assignee
Hangzhou Ezviz Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Ezviz Software Co Ltd filed Critical Hangzhou Ezviz Software Co Ltd
Priority to CN202210161527.7A priority Critical patent/CN114569004B/en
Publication of CN114569004A publication Critical patent/CN114569004A/en
Priority to PCT/CN2023/072374 priority patent/WO2023160305A1/en
Application granted granted Critical
Publication of CN114569004B publication Critical patent/CN114569004B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/24—Floor-sweeping machines, motor-driven
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4002—Installations of electric equipment
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4011—Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4061—Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated

Landscapes

  • Image Analysis (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

The embodiment of the invention provides a traveling direction adjusting method, a mobile robot system and electronic equipment, which are applied to the technical field of mobile robots. The method comprises the following steps: acquiring an indoor top image obtained by upward acquisition aiming at the indoor position of the mobile robot; determining a plurality of line segments contained in the indoor top image; identifying at least one target line segment from the line segments; each target line segment is a line segment with the line segment direction parallel to the ground; screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment; and adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized. Through the scheme, the moving direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface.

Description

Traveling direction adjustment method, mobile robot system, and electronic device
Technical Field
The present invention relates to the field of mobile robot technology, and in particular, to a method for adjusting a traveling direction, a mobile robot system, and an electronic device.
Background
In order for the mobile robot to perform tasks efficiently on the floor in a room, the moving path of the mobile robot generally needs to be parallel or perpendicular to the wall surface in the room.
Therefore, when the mobile robot starts to work, the moving direction of the mobile robot needs to be adjusted so that the moving direction of the mobile robot is parallel to or perpendicular to the wall surface. Therefore, how to adjust the traveling direction of the mobile robot to be parallel or perpendicular to the wall surface is an urgent technical problem to be solved.
Disclosure of Invention
An object of an embodiment of the present invention is to provide a method for adjusting a traveling direction, a mobile robot system, and an electronic device, so as to adjust the traveling direction of a mobile robot to be parallel to or perpendicular to a wall surface. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a method for adjusting a traveling direction, where the method includes:
acquiring an indoor top image obtained by upward acquisition aiming at the indoor position of the mobile robot;
determining a plurality of line segments contained in the indoor top image;
identifying at least one target line segment from the line segments; each target line segment is a line segment with the line segment direction parallel to the ground;
screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment;
and adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized.
Optionally, the screening, based on the segment length of each target segment, a to-be-utilized segment direction parallel to or perpendicular to the indoor wall surface from the segment direction of each target segment includes:
summing the line segment lengths of the target line segments belonging to the line segment direction aiming at each line segment direction in the line segment directions of all the target line segments to obtain the length corresponding to the line segment direction;
and screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the direction of each line segment based on the length corresponding to the direction of each line segment.
Optionally, the screening, from each line direction, of the line directions to be utilized that are parallel to or perpendicular to the indoor wall surface based on the length corresponding to each line direction includes:
and screening the line segment direction with the longest corresponding length from all the line segment directions to be used as the line segment direction to be utilized which is parallel to or vertical to the indoor wall surface.
Optionally, after summing the line segment lengths of the target line segments belonging to the line segment direction in each of the line segment directions of the target line segments to obtain a length corresponding to the line segment direction, and before screening, from each line segment direction based on the length corresponding to each line segment direction, a line segment direction parallel to or perpendicular to the indoor wall surface as a line segment direction to be utilized, the method further includes:
and for each line segment direction, if a target direction perpendicular to the line segment direction exists in each line segment direction, length compensation is carried out on the length corresponding to the line segment direction based on the length corresponding to the target direction perpendicular to the line segment direction.
Optionally, the length compensation of the length corresponding to the line segment direction based on the length corresponding to the target direction perpendicular to the line segment direction includes:
and length compensation is carried out on the length corresponding to the line segment direction according to the following formula:
Ls′＝Ls+α*Lv
wherein L issIs the length corresponding to the line segment direction, Ls' is the length corresponding to the line direction after length compensation, alpha is a preset compensation coefficient, LvThe length corresponding to the target direction.
Optionally, the identifying at least one target line segment from the line segments includes:
at least one target line segment is identified from the line segments based on the relative position between the line segments and the center point of the indoor top image.
Optionally, the identifying at least one target line segment from the line segments based on the relative position between the line segments and the central point of the indoor top image includes:
determining the minimum internal angle of the triangle corresponding to each line segment as the minimum internal angle corresponding to the line segment; wherein, three endpoints of the triangle corresponding to each line segment are respectively: two end points of the line segment and the center point of the indoor top image;
and selecting at least one line segment with the corresponding minimum internal angle larger than a preset angle threshold value from all the line segments to obtain at least one target line segment.
Optionally, the determining a plurality of line segments included in the indoor top image includes:
performing edge detection on the indoor top image to obtain the indoor top image after edge detection;
and identifying a plurality of line segments contained in the indoor top image after edge detection.
Optionally, the performing edge detection on the indoor top image to obtain the indoor top image after edge detection includes:
performing edge detection on the indoor top image to obtain the indoor top image after initial detection;
performing morphological closed operation processing on the indoor top image after the initial detection to obtain a processed indoor top image;
and carrying out edge detection on the processed indoor top image to obtain the indoor top image after edge detection.
Optionally, the indoor top image is an image acquired by using a fisheye camera;
prior to the determining a plurality of line segments included in the indoor top image, further comprising:
and carrying out distortion correction on the indoor top image.
Optionally, the adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized includes:
adjusting the traveling direction of the mobile robot to be the direction of the line segment to be utilized; alternatively, the first and second electrodes may be,
and adjusting the traveling direction of the mobile robot to be perpendicular to the direction of the line segment to be utilized.
In a second aspect, an embodiment of the present invention provides a mobile robot system, including:
the image acquisition module is used for acquiring indoor top images of the indoor where the mobile robot is located;
the processor is used for determining a plurality of line segments contained in the indoor top image according to the indoor top image and identifying at least one target line segment from the line segments, wherein each target line segment is a line segment of which the line segment direction is parallel to the ground; screening a line segment direction to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment;
and the power module is used for adjusting the advancing direction of the mobile robot based on the direction of the line segment to be utilized.
In a third aspect, an embodiment of the present invention provides a travel direction adjustment apparatus, including:
the image acquisition module is used for acquiring an indoor top image acquired by upward acquisition in the room where the mobile robot is;
the line segment determining module is used for determining a plurality of line segments contained in the indoor top image;
the line segment identification module is used for identifying at least one target line segment from all line segments; each target line segment is a line segment with the line segment direction parallel to the ground;
the direction screening module is used for screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment;
and the direction adjusting module is used for adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of the first aspect when executing a program stored in the memory.
In a fifth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method steps of any one of the first aspect.
The embodiment of the invention has the following beneficial effects:
in the method for adjusting the traveling direction provided by the embodiment of the invention, an indoor top image acquired by upward acquisition in a room where the mobile robot is located can be acquired, a plurality of line segments contained in the indoor top image are further determined, at least one target line segment parallel to the ground is identified from each line segment, and a line segment direction to be utilized, which is parallel to or perpendicular to an indoor wall surface, is screened from the line segment direction of each target line segment based on the line segment length of each target line segment, so that the traveling direction of the mobile robot can be adjusted based on the line segment direction to be utilized. Because for the indoor, the contour lines of common indoor objects such as wall seams at the indoor top, top outlines of a wardrobe and window edges are generally parallel or perpendicular to the wall surface, the indoor top images acquired by upward acquisition contain the outlines of the objects, so that the determined multiple line segments contain the line segments corresponding to the outlines of the objects, and further, the line segments corresponding to the vertical outlines contained in the indoor top images are eliminated by identifying the target line segments parallel to the ground, the interference of the indoor vertical outlines can be avoided, and further, in the line segment direction of each target line segment, the line segment direction parallel or perpendicular to the indoor wall surface to be utilized is screened out according to the length, and the traveling direction of the mobile robot is adjusted based on the line segment direction to be utilized. Therefore, by the scheme, the traveling direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface.
Of course, it is not necessary for any product or method to achieve all of the above-described advantages at the same time for practicing the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 is a flowchart of a method for adjusting a traveling direction according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an image capture scenario according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an indoor ceiling image according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another indoor top image provided by an embodiment of the invention;
fig. 5 is a schematic diagram illustrating an imaging principle of a fisheye camera provided in an embodiment of the invention;
FIG. 6 is a schematic diagram of an indoor overhead image including extensions of line segments perpendicular to the ground provided by an embodiment of the invention;
FIG. 7 is a schematic diagram of another indoor top image provided by an embodiment of the invention;
FIG. 8 is another flow chart of a method for adjusting a direction of travel according to an embodiment of the present invention;
FIG. 9 is another flow chart of a method for adjusting a direction of travel according to an embodiment of the present invention;
FIG. 10(a) is a schematic diagram of another indoor top image provided by an embodiment of the present invention;
FIG. 10(b) is a schematic diagram of another indoor top image provided by an embodiment of the invention;
FIG. 10(c) is a schematic diagram of another indoor top image provided by an embodiment of the invention;
FIG. 11 is a schematic diagram of another indoor top image provided by an embodiment of the invention;
fig. 12 is a schematic structural diagram of a mobile robot system according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of a travel direction adjustment device according to an embodiment of the present invention;
fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order for the mobile robot to perform tasks efficiently on the floor in a room, the moving path of the mobile robot generally needs to be parallel or perpendicular to the wall surface in the room. For example, in order for an indoor sweeping robot to effectively sweep corners of a house, the sweeping path of the sweeping robot is generally parallel or perpendicular to the wall surface.
Therefore, when the mobile robot starts to work, the moving direction of the mobile robot needs to be adjusted so that the moving direction of the mobile robot is parallel to or perpendicular to the wall surface. Therefore, how to adjust the traveling direction of the mobile robot to be parallel or perpendicular to the wall surface is an urgent technical problem to be solved.
In order to adjust a traveling direction of a mobile robot to be parallel to or perpendicular to a wall surface, embodiments of the present invention provide a traveling direction adjustment method, a mobile robot system, and an electronic device.
It should be noted that, in a specific application, the method for adjusting a traveling direction provided by the embodiment of the present invention may be applied to a mobile robot, such as a sweeping robot or a greeting robot. Alternatively, the method for adjusting a traveling direction provided by the embodiment of the present invention may also be applied to various other electronic devices, for example, a smart phone, a personal computer, a server, and other devices with data processing capability, and when being applied to various other electronic devices, the electronic device may communicate with a mobile robot. In addition, it is understood that the method for adjusting the traveling direction provided by the embodiment of the present invention may be implemented by software, hardware, or a combination of software and hardware.
The method for adjusting the advancing direction provided by the embodiment of the invention can comprise the following steps:
acquiring an indoor top image obtained by upward acquisition aiming at the indoor position of the mobile robot;
determining a plurality of line segments contained in the indoor top image;
identifying at least one target line segment from the line segments; each target line segment is a line segment with the line segment direction parallel to the ground;
screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment;
and adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized.
In the above-mentioned scheme of the embodiment of the present invention, for the indoor environment, the contour lines of common indoor objects such as wall seams at the indoor top, top outlines of a wardrobe, and window edges are generally parallel to or perpendicular to the wall surface, and the indoor top image acquired upward contains the contour of such an object, so that the plurality of determined line segments contain the line segment corresponding to the contour of such an object, and then the line segment corresponding to the vertical outline contained in the indoor top image is eliminated by identifying the target line segment parallel to the ground, so that the interference of the indoor vertical outline can be avoided, and further, the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, is screened out according to the length in the line segment direction of each target line segment, and the traveling direction of the mobile robot is adjusted based on the line segment direction to be utilized. Therefore, by the scheme, the traveling direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface.
The following describes the method for adjusting the traveling direction according to the embodiment of the present invention in detail with reference to the drawings.
As shown in fig. 1, an embodiment of the present invention provides a method for adjusting a traveling direction, including steps S101 to S105, where:
s101, acquiring an indoor top image acquired by upward acquisition in a room where the mobile robot is located;
the room where the mobile robot is located refers to a room where the mobile robot executes a task, for example, the mobile robot is a floor sweeping robot, and if the floor sweeping robot needs to execute a cleaning task to a living room, the room where the floor sweeping robot is located is the living room.
The upward collection means that the lens of the camera faces upward, so that the sampling visual field of the camera faces upward, when the sampling visual field of the camera faces upward, and when image collection is performed indoors where the mobile robot is located, the collected images are indoor top images, namely images containing a roof, of the indoor where the mobile robot is located. In the embodiment of the present invention, the camera for capturing the indoor top image may be a large-field camera, and in order to make the lens of the camera face upward, the lens of the camera may be faced upward and laid flat on the capturing plane. Alternatively, the large field of view camera may be a fisheye camera.
As shown in fig. 2, an embodiment of the present invention provides a schematic diagram of an image capturing scene. In the figure, the camera used for acquiring the indoor top image is a camera carried on the mobile robot, and the camera is placed towards the right upper side, so that the mirror image of the camera faces upwards. After the mobile robot is started, a camera carried by the mobile robot may capture an indoor overhead image.
Of course, it should be emphasized that the indoor top image may also be acquired by using a camera independent from the mobile robot, for example, when the mobile robot and other indoor devices form an intelligent home system, the indoor top image of the indoor where the mobile robot is located may be acquired by using other devices with an image acquisition function in the intelligent furniture system, or the indoor top image of the indoor where the mobile robot is located may be captured by the user through a smart phone.
When the execution main body of the scheme is the mobile robot, in one implementation mode, if a camera for acquiring indoor images is a camera configured for the mobile robot, the mobile robot can directly control the camera to acquire indoor top images; if the camera for acquiring the indoor image is a camera independent from the mobile robot, the mobile robot may send an image acquisition request to the camera or the electronic device where the camera is located, so as to acquire the image acquisition device acquired by the camera.
When the main execution body of the scheme is an electronic device independent of the mobile robot, for example, a backend server, in an implementation manner, if a camera for acquiring an indoor image is a camera carried by the mobile robot, the electronic device may send an image acquisition instruction to the mobile robot to control the camera in the mobile robot to acquire the indoor top image, and after the mobile robot acquires the indoor top image, send an image acquisition instruction to the mobile robot to acquire the indoor top image acquired by the mobile robot. And if the camera for acquiring the indoor image is a camera independent of the mobile robot, acquiring the indoor top image from the camera or the electronic equipment where the camera is located in a similar manner.
Generally, the indoor top image obtained by upward capturing includes a roof in a room, and of course, may include things such as walls, furniture, doors and windows. As shown in fig. 3, an embodiment of the invention provides a schematic diagram of an indoor top image, which includes a roof, a wall surface, a door and window, and decorations.
S102, determining a plurality of line segments contained in the indoor top image;
for any image, the line segment included in the image refers to the line segment corresponding to the outline of each object in the image. For the embodiment of the present invention, since the indoor top image includes objects such as a roof and wall surfaces, and since an obvious contour exists at a boundary between the roof and the wall surfaces and an obvious contour also exists at a boundary between wall surfaces of different surfaces, a plurality of line segments included in the indoor top image include a line segment corresponding to a contour at a boundary between the wall surfaces and the roof and a line segment corresponding to a contour at a boundary between wall surfaces of different surfaces.
Meanwhile, because objects such as doors and windows, furniture, and decorations may be included in the indoor top image, a plurality of line segments included in the indoor top image often include line segments corresponding to outlines of such objects.
For example, as shown in fig. 4, in another schematic diagram of an indoor ceiling image according to an embodiment of the present invention, each black line segment in fig. 4 is a line segment obtained after determining a line segment of the image shown in fig. 3, where the obtained line segment includes a line segment corresponding to a contour of a boundary between a roof and a wall surface, a line segment corresponding to a curtain contour, a line segment corresponding to a contour of a boundary between different wall surfaces, a line segment corresponding to each contour in an entrance door, a line segment corresponding to a contour of each type of ornament, and a line segment corresponding to a contour of a human body.
The method for determining the plurality of Line segments included in the indoor top image includes various manners, for example, the plurality of Line segments included in the indoor top image may be determined by a pre-trained Line Segment determination model, or the indoor top image may be processed by a Line identification algorithm to obtain the plurality of Line segments included in the indoor top image, where the Line identification algorithm may include a hough transform algorithm, an LSD (Line Segment Detector) Line detection algorithm, an FLD (Fast Line Detector) Line detection algorithm, and the like.
In order to improve the accuracy of line segment determination and the proportion of line segments corresponding to contours of junctions of wall surfaces and roofs in the determined line segments, before the indoor top images are input into a line segment determination model or processed by a straight line detection algorithm, contour optimization processing can be performed on the indoor top images, so that the contours of the processed indoor top images are purer and more obvious. The specific implementation will be described in detail in the following embodiments, which are not described herein again.
S103, identifying at least one target line segment from the line segments; each target line segment is a line segment with the line segment direction parallel to the ground;
the moving direction of the mobile robot is required to meet two conditions, namely, the moving direction is parallel to the ground, and the moving direction is parallel to or perpendicular to the wall.
As described above, it is determined that the indoor ceiling image may include segments such as segments corresponding to the outlines of the junctions between different wall surfaces, the outlines corresponding to the vertical edge outlines of doors and windows, the vertical edge outlines of furniture, and the vertical edge outlines or the oblique edge outlines of other decorations, and that such segments are not parallel to the ground. Since such line segments are not parallel to the ground and cannot be used to adjust the traveling direction of the mobile robot, they need to be eliminated from the determined line segments.
In the embodiment of the present invention, after determining a plurality of line segments included in the indoor top image, a target line segment of at least one line segment parallel to the ground needs to be identified from the line segments, for example, a line segment corresponding to a contour of a boundary between a roof and a wall surface, a contour corresponding to a contour of a horizontal edge of a door or window, a line segment corresponding to a contour of a top of a wardrobe, a line segment corresponding to a contour of a decoration on the roof, and the like.
There are various ways to identify a target line segment from line segments, for example, a pre-trained line segment identification model may be used to identify a target line segment from each line segment, or an implementation manner of the present invention may be used to determine a line segment corresponding to a numerical outline from each line segment by using an imaging feature of a vertical outline, including:
at least one target line segment is identified from the line segments based on the relative position between the line segments and the center point of the indoor top image.
The relative position between each line segment and the center point of the indoor top image may be the relative direction between the two end points of the line segment and the center point of the indoor top image, and based on the imaging principle of the vertical profile, the relative directions of the two end points of the line segment of the vertical profile in the indoor top image and the center point are substantially the same, and the following explanation is made by using the fish-eye camera imaging principle schematic diagram shown in fig. 5
In fig. 5, a three-dimensional coordinate system O-XcYcZc is a camera coordinate system, a two-dimensional coordinate system uv is a pixel coordinate system, a two-dimensional coordinate system xy is a pixel coordinate system of a fisheye camera, a two-dimensional coordinate system uv is a pixel coordinate system after distortion correction corresponding to the fisheye camera, two end points of a vertical line PQ are a point P and a point Q, respectively, projections of the two end points PQ in the two-dimensional coordinate system uv are line segments PQ (the end points are the point P and the point Q), and an image center point O imaged by the fisheye camera is in the two-dimensional coordinate system uv.
Let P point coordinate be (X, Y, Z)P) Q coordinate is (X, Y, Z)Q) According to the fisheye camera model and the distortion correction principle, the projection point P point coordinate of the P point on the corrected image is as follows:
(fx*x/ZP)+u0，fy*y/ZP+v0)
the Q point projection point Q is:
(fx*x/ZQ+u0，fy*y/ZQ+v0)
wherein f isxAnd fyIs the focal length of the fisheye camera, the coordinate of the image center o is (u)0,v0)；
The vector op cross product oq results:
op×oq＝0
so that three points of o, p and q are collinear.
That means, in the indoor ceiling image, the extension line of the line segment perpendicular to the ground will approximately pass through the image center point, for example, the extension line of the line segment perpendicular to the ground in fig. 4 is extended, so as to obtain a schematic diagram of the extension line of the line segment perpendicular to the ground as shown in fig. 6, and the extension line of the line segment perpendicular to the ground in fig. 6 will approximately pass through the image center point, that is, the relative directions of the two end points of the line segment perpendicular to the ground and the image center point are consistent, so that at least one target line segment can be identified from each line segment based on the relative position between each line segment and the center point of the indoor ceiling image.
On the basis, in an implementation manner, the minimum internal angle of the triangle corresponding to each line segment may be determined as the minimum internal angle corresponding to the line segment, and then at least one line segment, of which the corresponding minimum internal angle is greater than a preset angle threshold, is selected from each line segment to obtain at least one target line segment. Wherein, three endpoints of the triangle corresponding to each line segment are respectively: the two end points of the line segment and the center point of the indoor top image. The preset angle threshold may be determined according to actual requirements, and may be 5 ° for example.
Illustratively, after determining the target line segment from the line segments included in the image shown in fig. 6, the target line segment included in the indoor ceiling image shown in fig. 7 is obtained, wherein each black line segment in the image shown in fig. 7 represents a target line segment.
S104, based on the line segment length of each target line segment, screening the line segment direction to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment;
after the target line segments parallel to the ground are determined, the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, can be further screened from the line segment directions of the target line segments.
Since the target line segments are only line segments parallel to the ground, and the traveling direction of the mobile robot to be determined in the embodiment of the present invention needs to be parallel to or perpendicular to the wall surface, after the target line segments are determined, the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, needs to be further screened from the line segment directions of the target line segments.
Generally speaking, in the indoor top image, the length of the contour at the boundary between the wall surface and the roof is the longest in each contour parallel to the ground, and the line segment of the contour at the boundary between the wall surface and the roof is a line segment parallel to or perpendicular to the wall surface, so that the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, can be screened from the line segment direction of each target line segment based on the line segment length of each target line segment.
In one implementation, the longest target line segment may be screened out from the target line segments, and the line segment direction of the longest target line segment is used as the line segment direction to be utilized.
Optionally, in order to provide accuracy of the screened line segment direction to be utilized, lengths of line segments in the same line segment direction may also be considered comprehensively, and a specific implementation manner will be described in detail in the following embodiments and will not be described herein again.
And S105, adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized.
After determining the direction of the line segment to be utilized, which is parallel or perpendicular to the indoor wall surface, the traveling direction of the mobile robot may be adjusted based on the direction of the line segment to be utilized. Optionally, the traveling direction of the mobile robot may be adjusted to the direction of the line segment to be utilized; alternatively, the traveling direction of the mobile robot is adjusted to a direction perpendicular to the direction of the line segment to be utilized.
Illustratively, if the direction of the line segment to be utilized is 100 °, the directions perpendicular to the direction of the line segment to be utilized are 10 ° and 190 °, and the directions parallel to the direction of the line segment to be utilized are 100 ° and 280 °, if the traveling direction of the mobile robot is 50 ° before the adjustment, the angle of the traveling direction of the mobile robot can be adjusted from 50 ° to the above-mentioned at least one angle, for example, 100 °, the angle of the traveling direction of the mobile robot is increased by 50 °, so as to be adjusted from 50 ° before the adjustment to 100 °, and parallel to the direction of the line segment to be utilized.
In the above-mentioned scheme of the embodiment of the present invention, for the indoor environment, the contour lines of common indoor objects such as wall seams at the indoor top, top outlines of a wardrobe, and window edges are generally parallel to or perpendicular to the wall surface, and the indoor top image acquired upward contains the contour of such an object, so that the plurality of determined line segments contain the line segment corresponding to the contour of such an object, and then the line segment corresponding to the vertical outline contained in the indoor top image is eliminated by identifying the target line segment parallel to the ground, so that the interference of the indoor vertical outline can be avoided, and further, the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, is screened out according to the length in the line segment direction of each target line segment, and the traveling direction of the mobile robot is adjusted based on the line segment direction to be utilized. Therefore, by the scheme, the traveling direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface.
Based on the embodiment shown in fig. 1, as shown in fig. 8, another embodiment of the present invention further provides a method for adjusting a traveling direction, where the step S104 may include steps S801 to S802, where:
s801, summing the line segment lengths of the target line segments belonging to the line segment direction aiming at each line segment direction in the line segment directions of the target line segments to obtain the length corresponding to the line segment direction;
illustratively, the target line segment includes a line segment 1, a line segment 2, a line segment 3, a line segment 4, and a line segment 5, where the line segment direction of the line segment 1 is direction 1, the line segment direction of the line segment 2 is direction 2, the line segment direction of the line segment 3 is direction 1, the line segment direction of the line segment 4 is direction 1, and the line segment direction of the line segment 5 is direction 2. Then for direction 1 it corresponds to length segment 1+ segment 3+ segment 4 and for direction 2 it corresponds to length segment 2+ segment 5.
S802, based on the length corresponding to each line segment direction, screening the line segment direction to be utilized which is parallel or vertical to the indoor wall surface from each line segment direction.
In one implementation, the line segment direction with the longest length can be selected from all the line segment directions, and the selected line segment direction is used as the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface.
Illustratively, if the length corresponding to the direction 1 is 10, the length corresponding to the direction 2 is 20, and the length corresponding to the direction 3 is 30, the direction 3 is taken as a line segment direction to be utilized which is parallel or perpendicular to the wall surface in the room.
In the above scheme of the embodiment of the invention, the moving direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface. Further, the line segment lengths of the target line segments belonging to the line segment direction are summed, and the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, is screened from the line segment directions based on the lengths corresponding to the line segment directions, so that a realization basis is provided for adjusting the traveling direction of the mobile robot to be parallel to or perpendicular to the wall surface.
Optionally, in another embodiment of the present invention, after the step S801 and before the step S802, for each line segment direction, if there is a target direction perpendicular to the line segment direction in each line segment direction, length compensation may be performed on a length corresponding to the line segment direction based on a length corresponding to the target direction perpendicular to the line segment direction.
Optionally, length compensation may be performed on the length corresponding to the line segment direction according to the following formula:
Ls′＝Ls+α*Lv
wherein L issIs the length corresponding to the line segment direction, Ls' is the length corresponding to the line direction after the length compensation, alpha is the preset compensation coefficient, LvThe length corresponding to the target direction.
Optionally, α may be determined according to needs and scenarios, and may be 0.8, for example.
In the above scheme of the embodiment of the invention, the moving direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface. Further, by performing length compensation for a line segment direction in which a target direction perpendicular to the line segment direction exists, the accuracy of determining the line segment direction to be utilized can be improved.
Based on the embodiment shown in fig. 1, as shown in fig. 9, another embodiment of the present invention further provides a method for adjusting a traveling direction, where the step S102 may include steps S901 to S902, where:
s901, carrying out edge detection on the indoor top image to obtain an indoor top image after edge detection;
the edge detection algorithm may be used to perform edge detection on the indoor top image, and optionally, the edge detection algorithm may include a Canny (kaien) edge detection algorithm, a Laplacian edge detection algorithm, and the like.
Optionally, in an implementation manner, edge detection may be directly performed on the indoor top image, and after the edge detection is finished, the obtained image is used as the indoor top image after the edge detection.
Optionally, in an implementation manner, in order to improve the accuracy of determining the line segments and improve the proportion of the line segments corresponding to the contour of the boundary between the wall surface and the roof in each determined line segment, the step S901 may include steps 1 to 3, where:
step 1, performing edge detection on an indoor top image to obtain an indoor top image after initial detection;
for example, after performing edge detection on the image shown in fig. 3, another schematic diagram of the indoor top image shown in fig. 10(a) is obtained, where the indoor top image shown in fig. 10(a) is the indoor top image after initial detection.
Step 2, performing morphological closed operation processing on the indoor top image subjected to the initial detection to obtain a processed indoor top image;
after the first edge detection is performed on the indoor top image, in order to reduce the number of scattered line segments in the image, morphological closed operation processing can be performed on the indoor top image after the initial detection, so that the internal regions of objects in the indoor top image are fused together, internal straight lines are removed, and the processed indoor top image is obtained.
The morphological closing operation is an operation of expanding and then corroding a sample defined in data morphology, wherein expansion (usually expressed by ^ is one of basic operators in mathematical morphology. It is initially defined for a binary image and then expanded to a grayscale image and a subsequent complete grid. Etching (usually with)
Figure BDA0003514940030000141
Representation), is one of the two basic operators in mathematical morphology. It is initially defined for a binary image and then expanded to a grayscale image and a subsequent complete grid.
For example, after the image shown in fig. 10(a) is subjected to the morphological closing operation, another schematic diagram of the indoor top image shown in fig. 10(b) is obtained, and the indoor top image shown in fig. 10(b) is the processed indoor top image.
And 3, carrying out edge detection on the processed indoor top image to obtain the indoor top image after edge detection.
After the processed indoor top image is obtained, a second time of edge detection may be performed on the processed indoor top image, and the obtained image may be used as the indoor top image after edge detection.
For example, after the image shown in fig. 10(b) is subjected to edge detection, another schematic diagram of the indoor top image shown in fig. 10(c) is obtained, and the indoor top image shown in fig. 10(c) is the indoor top image after edge detection.
And S902, identifying a plurality of line segments contained in the indoor top image after the edge detection.
Optionally, the indoor top image after the edge detection may be subjected to straight line recognition, and a plurality of line segments included in the indoor top image after the edge detection are determined.
In the above scheme of the embodiment of the invention, the moving direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface. Furthermore, edge detection is carried out on the indoor top image, and then a plurality of line segments contained in the indoor top image after edge detection are identified, so that the accuracy of line segment determination can be improved.
Optionally, in another embodiment of the present invention, the indoor top image is an image acquired by using a fisheye camera, and as shown in fig. 11, as the indoor top image acquired by using the fisheye camera has a large distortion, distortion correction may be performed on the indoor top image before determining a plurality of line segments included in the indoor top image, so that an accurate recognition result due to distortion may be reduced, and accuracy of determining the line segments may be improved.
Corresponding to the method for adjusting the traveling direction provided in the foregoing embodiment of the present invention, as shown in fig. 12, an embodiment of the present invention further provides a mobile robot system, including:
an image acquisition module 1201, configured to acquire an indoor top image of an indoor environment where the mobile robot is located;
a processor 1202, configured to determine, according to the indoor top image, a plurality of line segments included in the indoor top image, and identify at least one target line segment from the line segments, where each target line segment is a line segment whose line segment direction is parallel to the ground; screening a line segment direction to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment;
and a power module 1203, configured to adjust a traveling direction of the mobile robot based on the direction of the line segment to be utilized.
The image capturing module 1201 may be a camera integrated with the mobile robot, or may be a camera independent from the mobile robot. If the image capturing module 1201 is a camera independent of the mobile robot, it may establish a communication connection with the mobile robot, and send the image to the mobile robot after capturing the image. The power module 1203 may be a moving component carried by the mobile robot, and may include a motor, a tire, and the like.
Optionally, the processor 1202, based on the segment length of each target segment, selects, from the segment directions of each target segment, a segment direction to be utilized that is parallel to or perpendicular to the indoor wall surface, and may include:
for each line segment direction in the line segment directions of all the target line segments, summing the line segment lengths of the target line segments belonging to the line segment direction to obtain the length corresponding to the line segment direction; and screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the direction of each line segment based on the length corresponding to the direction of each line segment.
Optionally, the processor 1202 filters, from the line segment directions, a line segment direction to be utilized that is parallel to or perpendicular to the indoor wall surface based on the length corresponding to each line segment direction, and may include:
and screening the line segment direction with the longest corresponding length from all the line segment directions to be used as the line segment direction to be utilized which is parallel to or vertical to the indoor wall surface.
Optionally, the processor 1202 may further sum, in each of the line segment directions for each target line segment, the line segment lengths of the target line segments belonging to the line segment direction, to obtain a length corresponding to the line segment direction, and screen, from the line segment directions, a line segment direction parallel to or perpendicular to the indoor wall surface, before taking the line segment direction as the line segment direction to be utilized, and perform, for each line segment direction, length compensation on the length corresponding to the line segment direction based on the length corresponding to the target direction perpendicular to the line segment direction if there is a target direction perpendicular to the line segment direction in each line segment direction.
Optionally, the processor 1202 performs length compensation on the length corresponding to the line segment direction based on the length corresponding to the target direction perpendicular to the line segment direction, and the length compensation may include:
and length compensation is carried out on the length corresponding to the line segment direction according to the following formula:
Ls′＝Ls+α*Lv
wherein L issIs the length corresponding to the line segment direction, Ls' is the length corresponding to the line direction after the length compensation, alpha is the preset compensation coefficient, LvThe length corresponding to the target direction.
Optionally, the identifying, by the processor 1202, at least one target line segment from the line segments may include:
at least one target line segment is identified from the line segments based on the relative position between the line segments and the center point of the indoor top image.
Optionally, the processor 1202 identifies at least one target line segment from the line segments based on the relative position between the line segments and the central point of the indoor top image, which may include:
determining the minimum internal angle of the triangle corresponding to each line segment as the minimum internal angle corresponding to the line segment; wherein, three endpoints of the triangle corresponding to each line segment are respectively: two end points of the line segment and the center point of the indoor top image;
and selecting at least one corresponding line segment with the minimum internal angle larger than a preset angle threshold value from all the line segments to obtain at least one target line segment.
Optionally, the determining, by the processor 1202, a plurality of line segments included in the indoor top image may include:
performing edge detection on the indoor top image to obtain the indoor top image after edge detection;
and identifying a plurality of line segments contained in the indoor top image after edge detection.
Optionally, the performing, by the processor 1202, edge detection on the indoor top image to obtain the indoor top image after edge detection may include:
performing edge detection on the indoor top image to obtain the indoor top image after initial detection;
performing morphological closed operation processing on the indoor top image after the initial detection to obtain a processed indoor top image;
and carrying out edge detection on the processed indoor top image to obtain the indoor top image after edge detection.
Optionally, the image acquisition module 1201 may be a fisheye camera;
a processor 1202, further configured to perform distortion correction on the indoor top image before the determining the plurality of line segments included in the indoor top image.
Optionally, the power module 1203 adjusts the traveling direction of the mobile robot based on the direction of the line segment to be utilized, and the adjusting may include:
adjusting the traveling direction of the mobile robot to be the direction of the line segment to be utilized; or adjusting the traveling direction of the mobile robot to be perpendicular to the direction of the line segment to be utilized.
In the above-mentioned scheme of the embodiment of the present invention, for the indoor environment, the contour lines of common indoor objects such as wall seams at the indoor top, top outlines of a wardrobe, and window edges are generally parallel to or perpendicular to the wall surface, and the indoor top image acquired upward contains the contour of such an object, so that the plurality of determined line segments contain the line segment corresponding to the contour of such an object, and then the line segment corresponding to the vertical outline contained in the indoor top image is eliminated by identifying the target line segment parallel to the ground, so that the interference of the indoor vertical outline can be avoided, and further, the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, is screened out according to the length in the line segment direction of each target line segment, and the traveling direction of the mobile robot is adjusted based on the line segment direction to be utilized. Therefore, by the scheme, the traveling direction of the mobile robot can be adjusted to be parallel or perpendicular to the wall surface.
Corresponding to the method for adjusting the advancing direction provided by the above embodiment of the present invention, as shown in fig. 13, an embodiment of the present invention further provides an advancing direction adjusting device, where the device includes:
the image acquisition module 1301 is used for acquiring an indoor top image acquired by upward acquisition of the indoor environment where the mobile robot is located;
a line segment determining module 1302, configured to determine a plurality of line segments included in the indoor top image;
the line segment identification module 1303 is used for identifying at least one target line segment from the line segments; each target line segment is a line segment with the line segment direction parallel to the ground;
a direction screening module 1304, configured to screen, from the line segment directions of the target line segments, a line segment direction to be utilized that is parallel to or perpendicular to the indoor wall surface, based on the line segment length of each target line segment;
a direction adjusting module 1305, configured to adjust a traveling direction of the mobile robot based on the direction of the line segment to be utilized.
Optionally, the direction screening module includes:
the length summing submodule is used for summing the line segment lengths of the target line segments belonging to the line segment direction aiming at each line segment direction in the line segment directions of all the target line segments to obtain the length corresponding to the line segment direction;
and the direction screening submodule is used for screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the direction of each line segment based on the length corresponding to the direction of each line segment.
Optionally, the direction screening submodule is specifically configured to screen a line segment direction with the longest corresponding length from among the line segment directions, and the line segment direction is used as a line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface.
Optionally, the direction filtering module further includes:
and the length compensation submodule is used for summing the line lengths of the target line segments belonging to the line segment direction in each line segment direction executed by the length summation submodule after the length corresponding to the line segment direction is obtained, executing the length corresponding to each line segment direction in the direction screening submodule, and executing length compensation on the length corresponding to the line segment direction based on the length corresponding to the target direction vertical to the line segment direction if a target direction vertical to the line segment direction exists in each line segment direction before the line segment direction is screened for the line segment direction to be utilized which is parallel to or vertical to the indoor wall surface.
Optionally, the length compensation submodule is specifically configured to perform length compensation on the length corresponding to the line segment direction according to the following formula:
Ls′＝Ls+α*Lv
wherein L issIs the length corresponding to the line segment direction, Ls' is the length corresponding to the line direction after the length compensation, alpha is the preset compensation coefficient, LvThe length corresponding to the target direction.
Optionally, the line segment identification module is specifically configured to identify at least one target line segment from the line segments based on a relative position between each line segment and a central point of the indoor top image.
Optionally, the line segment identifying module is specifically configured to determine a minimum internal angle of a triangle corresponding to each line segment, and use the minimum internal angle as the minimum internal angle corresponding to the line segment; selecting at least one corresponding line segment with the minimum internal angle larger than a preset angle threshold value from all the line segments to obtain at least one target line segment; wherein, three endpoints of the triangle corresponding to each line segment are respectively: the two end points of the line segment and the center point of the indoor top image.
Optionally, the line segment determining module includes:
the image detection submodule is used for carrying out edge detection on the indoor top image to obtain the indoor top image after the edge detection;
and the line segment identification submodule is used for identifying a plurality of line segments contained in the indoor top image after the edge detection.
Optionally, the image detection submodule is specifically configured to perform edge detection on the indoor top image to obtain the indoor top image after initial detection; performing morphological closed operation processing on the indoor top image after the initial detection to obtain a processed indoor top image; and carrying out edge detection on the processed indoor top image to obtain the indoor top image after edge detection.
Optionally, the indoor top image is an image acquired by using a fisheye camera;
the device further comprises:
a distortion correction module for performing distortion correction on the indoor top image before the line segment determination module performs the determination of the plurality of line segments included in the indoor top image.
Optionally, the direction adjusting module is specifically configured to adjust the traveling direction of the mobile robot to the direction of the line segment to be utilized; or adjusting the traveling direction of the mobile robot to be perpendicular to the direction of the line segment to be utilized.
In the above-mentioned scheme of the embodiment of the present invention, for the indoor environment, the contour lines of common indoor objects such as wall seams at the indoor top, top outlines of a wardrobe, and window edges are generally parallel to or perpendicular to the wall surface, and the indoor top image acquired upward contains the contour of such an object, so that the plurality of determined line segments contain the line segment corresponding to the contour of such an object, and then the line segment corresponding to the vertical outline contained in the indoor top image is eliminated by identifying the target line segment parallel to the ground, so that the interference of the indoor vertical outline can be avoided, and further, the line segment direction to be utilized, which is parallel to or perpendicular to the indoor wall surface, is screened out according to the length in the line segment direction of each target line segment, and the traveling direction of the mobile robot is adjusted based on the line segment direction to be utilized. Therefore, the moving direction of the mobile robot can be adjusted to be parallel to or perpendicular to the wall surface through the scheme.
The embodiment of the present invention further provides an electronic device, as shown in fig. 14, which includes a processor 1401, a communication interface 1402, a memory 1403, and a communication bus 1404, wherein the processor 1401, the communication interface 1402, and the memory 1403 complete communication with each other through the communication bus 1404,
a memory 1403 for storing a computer program;
the processor 1401 is configured to implement the steps of the method for adjusting the moving direction according to the above-described embodiment of the present invention when executing the program stored in the memory 1403.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In still another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program realizes the steps of any one of the above travel direction adjustment methods when executed by a processor.
In a further embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the travel direction adjustment methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the mobile robot system, the apparatus, the electronic device, the readable storage medium, and the computer program, since they are substantially similar to the embodiments of the method, the description is simple, and the relevant points can be referred to the partial description of the embodiments of the method.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (14)

1. A method of adjusting a direction of travel, the method comprising:
acquiring an indoor top image obtained by upward acquisition aiming at the indoor position of the mobile robot;
determining a plurality of line segments contained in the indoor top image;
identifying at least one target line segment from the line segments; each target line segment is a line segment with the line segment direction parallel to the ground;
based on the line segment length of each target line segment, screening the line segment direction to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment;
and adjusting the traveling direction of the mobile robot based on the direction of the line segment to be utilized.
2. The method of claim 1, wherein the screening the direction of the line segment to be utilized, which is parallel or perpendicular to the indoor wall surface, from the direction of the line segment of each target line segment based on the length of the line segment of each target line segment comprises:
for each line segment direction in the line segment directions of all the target line segments, summing the line segment lengths of the target line segments belonging to the line segment direction to obtain the length corresponding to the line segment direction;
and screening the direction of the line segment to be utilized which is parallel to or vertical to the indoor wall surface from the direction of each line segment based on the length corresponding to the direction of each line segment.
3. The method of claim 2, wherein the screening the direction of the line segment to be utilized, which is parallel or perpendicular to the indoor wall surface, from the direction of each line segment based on the length corresponding to the direction of each line segment comprises:
and screening the line segment direction with the longest corresponding length from all the line segment directions to be used as the line segment direction to be utilized which is parallel to or vertical to the indoor wall surface.
4. The method according to claim 2 or 3, wherein after summing the line lengths of the target line segments belonging to each target line segment in each of the line segment directions for the target line segments to obtain the length corresponding to the line segment direction, and before screening, based on the length corresponding to each line segment direction, the line segment direction parallel or perpendicular to the indoor wall surface from the line segment directions as the line segment direction to be utilized, the method further comprises:
and for each line segment direction, if a target direction perpendicular to the line segment direction exists in each line segment direction, length compensation is carried out on the length corresponding to the line segment direction based on the length corresponding to the target direction perpendicular to the line segment direction.
5. The method of claim 4, wherein the length compensation for the length corresponding to the line segment direction based on the length corresponding to the target direction perpendicular to the line segment direction comprises:
and length compensation is carried out on the length corresponding to the line segment direction according to the following formula:
Ls′＝Ls+α*Lv
wherein L issIs the length corresponding to the line segment direction, Ls' is the length corresponding to the line direction after the length compensation, alpha is the preset compensation coefficient, LvThe length corresponding to the target direction.
6. The method of any of claims 1-3, wherein identifying at least one target line segment from the line segments comprises:
at least one target line segment is identified from the line segments based on the relative position between the line segments and the center point of the indoor top image.
7. The method of claim 6, wherein identifying at least one target line segment from the line segments based on the relative position of the line segments to the center point of the overhead indoor image comprises:
determining the minimum internal angle of the triangle corresponding to each line segment as the minimum internal angle corresponding to the line segment; wherein, three endpoints of the triangle corresponding to each line segment are respectively: two end points of the line segment and the center point of the indoor top image;
and selecting at least one line segment with the corresponding minimum internal angle larger than a preset angle threshold value from all the line segments to obtain at least one target line segment.
8. The method of any of claims 1-3, wherein the determining a plurality of line segments included in the indoor top image comprises:
performing edge detection on the indoor top image to obtain the indoor top image after edge detection;
and identifying a plurality of line segments contained in the indoor top image after edge detection.
9. The method of claim 8, wherein the edge detecting the indoor top image to obtain the indoor top image after edge detection comprises:
performing edge detection on the indoor top image to obtain the indoor top image after initial detection;
performing morphological closed operation processing on the indoor top image after the initial detection to obtain a processed indoor top image;
and carrying out edge detection on the processed indoor top image to obtain the indoor top image after edge detection.
10. The method according to any one of claims 1 to 3, wherein the indoor ceiling image is an image acquired by a fisheye camera;
prior to the determining a plurality of line segments included in the indoor top image, further comprising:
and carrying out distortion correction on the indoor top image.
11. The method according to any one of claims 1-3, wherein the adjusting the direction of travel of the mobile robot based on the direction of the line segment to be utilized comprises:
adjusting the traveling direction of the mobile robot to be the direction of the line segment to be utilized; alternatively, the first and second electrodes may be,
and adjusting the traveling direction of the mobile robot to be perpendicular to the direction of the line segment to be utilized.
12. A mobile robotic system, comprising:
the image acquisition module is used for acquiring indoor top images of the indoor where the mobile robot is located;
the processor is used for determining a plurality of line segments contained in the indoor top image according to the indoor top image and identifying at least one target line segment from the line segments, wherein each target line segment is a line segment of which the line segment direction is parallel to the ground; screening a line segment direction to be utilized which is parallel to or vertical to the indoor wall surface from the line segment direction of each target line segment based on the line segment length of each target line segment;
and the power module is used for adjusting the advancing direction of the mobile robot based on the direction of the line segment to be utilized.
13. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1 to 11 when executing a program stored in the memory.
14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 11.
CN202210161527.7A 2022-02-22 2022-02-22 Travel direction adjustment method, mobile robot system and electronic device Active CN114569004B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210161527.7A CN114569004B (en) 2022-02-22 2022-02-22 Travel direction adjustment method, mobile robot system and electronic device
PCT/CN2023/072374 WO2023160305A1 (en) 2022-02-22 2023-01-16 Travelling direction adjusting method, mobile robot system and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210161527.7A CN114569004B (en) 2022-02-22 2022-02-22 Travel direction adjustment method, mobile robot system and electronic device

Publications (2)

Publication Number Publication Date
CN114569004A true CN114569004A (en) 2022-06-03
CN114569004B CN114569004B (en) 2023-12-01

Family

ID=81773470

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210161527.7A Active CN114569004B (en) 2022-02-22 2022-02-22 Travel direction adjustment method, mobile robot system and electronic device

Country Status (2)

Country Link
CN (1) CN114569004B (en)
WO (1) WO2023160305A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023160305A1 (en) * 2022-02-22 2023-08-31 杭州萤石软件有限公司 Travelling direction adjusting method, mobile robot system and electronic device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102929280A (en) * 2012-11-13 2013-02-13 朱绍明 Mobile robot separating visual positioning and navigation method and positioning and navigation system thereof
CN109074084A (en) * 2017-08-02 2018-12-21 珊口（深圳）智能科技有限公司 Control method, device, system and the robot being applicable in of robot
CN110801180A (en) * 2018-08-03 2020-02-18 速感科技（北京）有限公司 Operation method and device of cleaning robot
CN112034837A (en) * 2020-07-16 2020-12-04 珊口（深圳）智能科技有限公司 Method for determining working environment of mobile robot, control system and storage medium
CN112263188A (en) * 2020-10-22 2021-01-26 湖南格兰博智能科技有限责任公司 Correction method and device for moving direction of mobile robot
CN112683266A (en) * 2019-10-17 2021-04-20 科沃斯机器人股份有限公司 Robot and navigation method thereof
CN112712584A (en) * 2019-10-25 2021-04-27 阿里巴巴集团控股有限公司 Wall line determining method, space modeling method, device and equipment
CN112773272A (en) * 2020-12-29 2021-05-11 深圳市杉川机器人有限公司 Moving direction determining method and device, sweeping robot and storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102398330B1 (en) * 2015-06-12 2022-05-16 엘지전자 주식회사 Moving robot and controlling method thereof
JP7311832B2 (en) * 2018-12-27 2023-07-20 ブラザー工業株式会社 Image processing device, reading device and computer program
CN114569004B (en) * 2022-02-22 2023-12-01 杭州萤石软件有限公司 Travel direction adjustment method, mobile robot system and electronic device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102929280A (en) * 2012-11-13 2013-02-13 朱绍明 Mobile robot separating visual positioning and navigation method and positioning and navigation system thereof
CN109074084A (en) * 2017-08-02 2018-12-21 珊口（深圳）智能科技有限公司 Control method, device, system and the robot being applicable in of robot
CN110801180A (en) * 2018-08-03 2020-02-18 速感科技（北京）有限公司 Operation method and device of cleaning robot
CN112683266A (en) * 2019-10-17 2021-04-20 科沃斯机器人股份有限公司 Robot and navigation method thereof
CN112712584A (en) * 2019-10-25 2021-04-27 阿里巴巴集团控股有限公司 Wall line determining method, space modeling method, device and equipment
CN112034837A (en) * 2020-07-16 2020-12-04 珊口（深圳）智能科技有限公司 Method for determining working environment of mobile robot, control system and storage medium
CN112263188A (en) * 2020-10-22 2021-01-26 湖南格兰博智能科技有限责任公司 Correction method and device for moving direction of mobile robot
CN112773272A (en) * 2020-12-29 2021-05-11 深圳市杉川机器人有限公司 Moving direction determining method and device, sweeping robot and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023160305A1 (en) * 2022-02-22 2023-08-31 杭州萤石软件有限公司 Travelling direction adjusting method, mobile robot system and electronic device

Also Published As

Publication number Publication date
WO2023160305A1 (en) 2023-08-31
CN114569004B (en) 2023-12-01

Similar Documents

Publication Publication Date Title
JP6710426B2 (en) Obstacle detection method and device
CN107636680B (en) Obstacle detection method and device
US9773313B1 (en) Image registration with device data
WO2017054314A1 (en) Building height calculation method and apparatus, and storage medium
US10026183B2 (en) Method, system and apparatus for determining distance to an object in a scene
CN111008597A (en) Space identification method and device for CAD drawing, electronic equipment and storage medium
CN105303514A (en) Image processing method and apparatus
CN107978017B (en) Indoor structure rapid modeling method based on frame line extraction
JP2004334819A (en) Stereo calibration device and stereo image monitoring device using same
JPH0528273A (en) Method and device for processing picture
CN111614867B (en) Video denoising method and device, mobile terminal and storage medium
CN112001298B (en) Pedestrian detection method, device, electronic equipment and storage medium
CN109447986B (en) Welding image acquisition method and device, electronic equipment and storage medium
CN108229583B (en) Method and device for fast template matching based on main direction difference characteristics
WO2023160305A1 (en) Travelling direction adjusting method, mobile robot system and electronic device
JP4427052B2 (en) Image processing apparatus and area tracking program
CN114298902A (en) Image alignment method and device, electronic equipment and storage medium
CN111986214A (en) Construction method of pedestrian crossing in map and electronic equipment
CN116208862A (en) Self-adaptive image processing method, system, electronic device and storage medium
KR101348681B1 (en) Multi-sensor image alignment method of image detection system and apparatus using the same
CN110826364A (en) Stock position identification method and device
CN109388131B (en) Robot attitude control method and system based on angular point feature recognition and robot
CN113592947B (en) Method for realizing visual odometer by semi-direct method
CN115473999A (en) Method and apparatus for imaging liquid-based cell, and storage medium
CN114049399B (en) Mirror positioning method combining RGBD image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant