WO2021031406A1

WO2021031406A1 - Automatic working system, and automatic walking device and control method therefor

Info

Publication number: WO2021031406A1
Application number: PCT/CN2019/117871
Authority: WO
Inventors: 陈泓; 朱绍明
Original assignee: 苏州科瓴精密机械科技有限公司
Priority date: 2019-08-21
Filing date: 2019-11-13
Publication date: 2021-02-25
Also published as: CN112486151A

Abstract

An automatic working system, and an automatic walking device and a control method therefor. The control method comprises: acquiring a real-time image photographed in real time by an automatic walking device in a forward direction; determining, according to a color layout of the real-time image, whether the automatic walking device has reached or approached a boundary; and when it is determined that the automatic walking device has reached or approached the boundary, the automatic walking device stopping moving forward, rotating a pre-set angle, and then continuing to move forward and work. The automatic walking device determines, by means of photographing the color layout of the real-time image in the forward direction, whether the automatic walking device has reached or approached the boundary. The detection is relatively convenient, and the boundary also does not need to be buried underground, but is arranged on the ground to facilitate disassembly and assembly.

Description

Automatic working system, automatic walking equipment and control method thereof

Technical field

The invention relates to the field of intelligent control, in particular to an automatic working system, an automatic walking device and a control method thereof.

Background technique

With the continuous advancement of computer technology and artificial intelligence technology, automatic walking equipment and automatic working systems of intelligent robots have slowly entered people's lives, such as intelligent sweeping robots and intelligent lawn mower robots. Generally, such intelligent robots are small in size, and are integrated with sensing devices, driving devices, batteries, etc., without manual manipulation, and can travel and work in a prescribed area. Moreover, when the battery power is insufficient, it can automatically return to the charging station, dock with the charging station and charge, and continue to travel and work after the charging is completed.

For the existing intelligent lawn mowing robot, the working area of the existing automatic working system is large lawn, and the boundary is mostly energized equipment buried under the ground, so that the intelligent lawn mowing robot can sense . Moreover, in addition to setting the boundary, it is also necessary to set up a charging station, so the construction of the entire automatic working system is relatively complicated. Especially for small lawns, such as less than 100 square meters, the lawn mower robot can mow a complete lawn in two hours. If the above-mentioned working system is also to be built for this type of lawn, the cost will be higher and also More troublesome.

Therefore, it is necessary to design a more convenient automatic working system, automatic walking equipment and its control method suitable for small area lawns.

Summary of the invention

In order to solve one of the above-mentioned problems, the present invention provides a control method of an autonomous walking device. The control method includes: acquiring real-time images taken by the autonomous walking device in the forward direction in real time; and judging the automatic walking based on the color layout in the real-time image Whether the device has reached or approached the boundary; when it is judged that it has reached or approached the boundary, the automatic traveling device stops traveling forward and rotates a preset angle before continuing to travel and work.

As a further improvement of the present invention, the step of "determining whether the autonomous device has reached or approaching the boundary based on the color layout in the real-time image" includes: judging the range of the boundary based on the color layout in the real-time image; calculating the boundary area and real-time image The ratio between the total area of the image is R1; when R1≥T1, it is judged that the automatic traveling equipment has reached or approached the boundary.

As a further improvement of the present invention, the step of "determining the range of the boundary based on the color layout in the real-time image" includes: detecting or acquiring the color of the boundary; converting the real-time image into HSV format or HSI format or LAB format; A collection of pixels representing the color of the border.

In order to solve one of the above-mentioned problems, the present invention provides a self-propelled equipment, which includes a main body, a walking module, and a power supply module; the self-propelled equipment includes: a camera device, the camera device is arranged on the front side of the main body, and used Acquire real-time images taken by the autonomous vehicle in the forward direction; the control device is connected to the camera device and used to determine whether the autonomous vehicle has reached or approached the boundary according to the color layout in the real-time image, and when it is judged to have reached or approached After the boundary, the automatic traveling device is controlled to stop moving forward and rotate a preset angle before continuing to move forward and work.

As a further improvement of the present invention, the autonomous walking device further includes a calculation module, the control module can also determine the range of the boundary according to the color layout in the real-time image, and the calculation module is connected with the control module to calculate the boundary in the real-time image The ratio R1 between the area of the real-time image and the total area of the real-time image; and when R1≥T1, the control module determines that the automatic walking equipment has reached or is close to the boundary.

As a further improvement of the present invention, the control module includes a format conversion unit and a color detection unit, the format conversion unit is used to convert the real-time image into HSV format or HSI format or LAB format; the color detection unit can be used to detect or Obtain the color of the boundary, and obtain a collection of all pixels that present the color of the boundary in the real-time image.

As a further improvement of the present invention, the step "after the automatic traveling equipment stops traveling forward and rotating a preset angle, then continues to travel and work" includes: the automatic traveling equipment suspends forward traveling; The evasion distance is moved in the opposite direction of the forward direction; the automatic traveling equipment rotates by a preset angle; the automatic traveling equipment continues to travel and work.

In order to solve one of the above-mentioned problems, the present invention provides an automatic working system, including: an automatic traveling device; a boundary, the enclosure is in a ring shape and is formed to limit the working area of the automatic traveling device, and the boundary is detachably arranged at On the ground; the color of the border is different from the ground color.

As a further improvement of the present invention, the boundary includes boundary piles arranged at intervals and an enclosure connected between adjacent boundary piles, and the boundary piles extend upward from the ground and can be partially inserted under the ground or fully supported on the ground.

As a further improvement of the present invention, the enclosure is made of flexible material.

Compared with the prior art, the self-propelled device of the present invention judges whether it has reached or is close to the boundary by shooting the color distribution on the real-time image in the forward direction, and the detection is more convenient. The boundary may not be buried in the ground, but set On the ground, easy to disassemble.

Description of the drawings

Figure 1 is a schematic diagram of the structure of the automatic working system of the present invention;

Fig. 2 is a flow chart of the control method of the autonomous walking equipment in the present invention.

Specific embodiment

In order to enable those skilled in the art to better understand the technical solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

The self-propelled equipment of the present invention may be an automatic lawn mower, or an automatic vacuum cleaner, etc., which automatically walks in the work area for mowing and vacuuming. In the specific example of the present invention, the self-propelled equipment is a lawn mower as an example. Specifically, correspondingly, the working area may be a lawn. Of course, the self-propelled equipment is not limited to lawn mowers and vacuum cleaners, but can also be other equipment, such as spraying equipment, snow removal equipment, monitoring equipment, etc., suitable for unattended equipment.

As shown in Fig. 1 and Fig. 2, the present invention provides a control method of an autonomous walking device, the control method includes:

Obtain real-time images taken by the automatic walking device in the forward direction in real time;

According to the color layout in the real-time image, judge whether the autonomous device has reached or is close to the boundary;

When it is judged that it has reached or is close to the boundary, the autonomous vehicle stops traveling forward and rotates a preset angle before continuing to travel and work.

Different from the existing self-propelled equipment, the self-propelled equipment in the present invention does not sense the current on the boundary, but takes real-time images of the self-propelled equipment in the forward direction and analyzes the color distribution in the real-time images. , To determine whether it has reached or is close to the boundary. Therefore, in order to be able to capture a real-time image including the boundary, the boundary must be significantly different from the grass. Specifically, it will be described in detail later.

In addition, the above-mentioned steps "after the autonomous vehicle stops traveling forward and rotates a preset angle, then continue to travel and work" includes: the autonomous vehicle stops traveling forward; the autonomous vehicle reverses the original forward direction Directional movement avoidance distance; the automatic traveling equipment rotates a preset angle; the automatic traveling equipment continues to travel and work. Of course, the self-propelled equipment can also directly stop traveling and rotate a preset angle. The above avoidance distance is determined by the walking speed of the automatic traveling equipment, the shooting angle, the size of the boundary, and so on.

Therefore, the autonomous walking device of the present invention judges the distance from the boundary through real-time images, so there is no need to bury the boundary in the ground, and the boundary can be set on the ground, so that the boundary is easier to build or disassemble.

Further, the step of "judging whether the autonomous vehicle has reached or approached the boundary according to the color layout in the real-time image" includes:

Judge the range of the boundary based on the color layout in the real-time image;

Calculate the ratio R1 between the area of the boundary in the real-time image and the total area of the real-time image;

When R1≥T1, it is judged that the automatic traveling equipment has reached or approached the boundary.

Since the boundary is set on the ground, if the ratio of the range of the boundary to the area in the real-time image is larger, it means that the autonomous vehicle is closer to the boundary. In addition, the threshold T1 is also determined based on the walking speed of the autonomous vehicle, the angle of shooting, the size of the boundary, and the like. In this embodiment, the boundary extends upward from the ground and the height exceeds the height of the autonomous vehicle, and the value of the threshold T1 is 100%.

Of course, in the above-mentioned embodiment of the present invention, the range of the boundary is determined by the color layout, and the ratio R1 between the area of the boundary in the real-time image and the total area of the real-time image is used to determine that the autonomous vehicle has reached or approached the boundary. Of course, in another embodiment, the separation line between the boundary and the ground can also be detected by the color layout, so as to determine the distance between the autonomous vehicle and the boundary by the position of the separation line in the real-time image. If the dividing line is closer to the bottom of the real-time image, the autonomous device is closer to the boundary; on the contrary, if the dividing line is closer to the top of the real-time image, the autonomous device is farther from the boundary. In particular, in this other embodiment, it can be performed by a method such as Hough transform straight line detection by the lower edge of the boundary having obvious linear characteristics.

Specifically, the step of "determining the boundary range according to the color layout in the real-time image" includes:

Detect or obtain the color of the border;

Convert real-time images to HSV format or HSI format or LAB format;

Obtain a collection of all pixels that present the border color in the real-time image.

In order to be able to distinguish clearly from the lawn, the color of the border is usually different from the color of the lawn. Therefore, in the present invention, the range of the border in the real-time image is recognized by acquiring pixels of the color in the real-time image. Further, the real-time image can be converted into HSV format or HSI, LAB format, which is more conducive to color detection, and then extracted as pixels of that color on the photo. After obtaining the range of the boundary, as described above, the ratio R1 between the area of the boundary and the total area of the real-time image is calculated.

It should be noted that “detecting or acquiring the color of the boundary” means that the color of the boundary can be detected through image recognition or the like, or the color of the boundary can be directly retrieved from the memory of the autonomous walking device. In addition, in the “acquisition of the collection of all pixels presenting the boundary color in the real-time image”, the color presenting the boundary can be multiple colors, for example, satisfying the uniformity of colors within a certain range of chromaticity or saturation within a certain range. Can be used as the color of the border.

For example, in an embodiment of the present invention, if the color of the border is red, red is clearly distinguished from lawn. Specifically, in the above-mentioned control method of the present invention, red pixels can be obtained in real-time images. And form the boundary of the range, and then calculate. If in the real-time image, the ratio between the area of the red range and the total area of the real-time image is R1 ≥ T1, it means that the autonomous vehicle has reached or approached the boundary.

The present invention also provides an automatic traveling equipment, which includes a main body, a traveling module, and a power supply module, and, in the present invention, the automatic traveling equipment further includes a camera device and a control device, and the camera device is arranged on the front side of the main body , And used to obtain real-time images taken by the aforementioned autonomous walking equipment in real-time in the forward direction. The control device can be connected to the camera device and used to determine whether the autonomous vehicle has reached or approached the boundary according to the color layout in the real-time image, and when it is determined that the autonomous vehicle has reached or approached the boundary, control the autonomous vehicle to stop moving forward And rotate the preset angle, then continue to move forward and work. Of course, this corresponds to the control method of the above-mentioned autonomous walking equipment, and will not be repeated here.

Further, the autonomous walking device further includes a calculation module, the control module can also determine the range of the boundary according to the color layout in the real-time image, and the calculation module is connected with the control module to calculate the boundary area and the real-time image. The ratio between the total area of the image is R1; and when R1≥T1, the control module judges that the autonomous walking equipment has reached or is close to the boundary. Specifically, the control module includes a format conversion unit and a color detection unit, the format conversion unit is used to convert the real-time image into HSV format or HSI format or LAB format; the color detection unit can be used to detect or acquire the color of the boundary , And obtain the collection of all pixels that present the border color in the real-time image. In addition, as shown in Fig. 1, the present invention also provides an automatic working system for automatic walking equipment, which includes:

Automatic walking equipment

The boundary is enclosed in a ring shape and is formed to define the working area of the automatic walking equipment, the boundary is detachably set on the ground; the color of the boundary is different from the color of the ground.

The boundary is detachably set on the ground, so the boundary can be dismantled after the construction is completed, so it can be reused, and for a small area of lawn, only the boundary needs to be built, and no arrangement is required Charging station, the construction efficiency of the automatic working system is higher. The self-propelled equipment can also walk and work freely in the working area enclosed by the boundary. In addition, the boundary is set on the ground and can be matched with the automatic traveling equipment, and when the automatic traveling equipment is approaching, the boundary is within the shooting range of the camera.

Further, the boundary includes boundary piles 1 arranged at intervals and an enclosure 2 connected between adjacent boundary piles 1. The boundary piles 1 extend upward from the ground and can be partially inserted under the ground or fully supported on the ground. Therefore, the boundary pile 1 can be fixed on the ground. In addition, as described above, the height of the enclosure 2 in the vertical direction is higher than that of the self-propelled device, which is more favorable for the self-propelled device to take pictures. Of course, the boundary pile 1 is designed to be higher than the enclosure 2. In this embodiment, the boundary pile 1 is in the shape of a hard rod with a sharp lower end to be inserted into the ground to support the enclosure 2.

Of course, as described above, the color of the boundary is different from the color of the ground, so that the camera of the self-propelled device can perform better recognition. Specifically, the color of the border is red. Moreover, in this embodiment, the colors of the enclosure 2 and the boundary pile 1 are the same and both are red. Of course, the colors of the enclosure 2 and the boundary pile 1 may also be different.

In addition, the enclosure 2 is made of flexible material, so as to facilitate crimping and folding. Preferably, the enclosure 2 is made of a lightweight elastic material, which is convenient for saving space during storage, and also facilitates tightening when fixed on the boundary pile 1 so that the enclosure 2 can be effectively enclosed on the boundary pile 1.

In summary, the present invention provides a method for controlling an autonomous vehicle, and can judge whether the autonomous vehicle reaches or is close to the boundary according to the color layout in the real-time captured real-time image, so that the boundary can be detachably set at On the ground, it is easier to build the automatic working system. Further, in the present invention, the pixels presenting the color of the boundary in the real-time photo can be detected to obtain the size of the boundary. When the range of the boundary is larger, it can be judged that the autonomous walking device is closer to the boundary. , So that it is possible to control the steering of the automatic walking equipment immediately.

In addition, it should be understood that although this specification is described in accordance with the implementation manners, not each implementation manner only contains an independent technical solution. This narration in the specification is only for clarity, and those skilled in the art should regard the specification as a whole The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

The series of detailed descriptions listed above are only specific descriptions of feasible implementations of the present invention, and are not intended to limit the scope of protection of the present invention. Any equivalent implementations or changes made without departing from the technical spirit of the present invention All should be included in the protection scope of the present invention.

Claims

A control method of autonomous walking equipment, characterized in that the control method includes:

Obtain real-time images taken by the automatic walking device in the forward direction in real time;

According to the color layout in the real-time image, judge whether the autonomous device has reached or is close to the boundary;

When it is judged that it has reached or is close to the boundary, the autonomous vehicle stops traveling forward and rotates a preset angle before continuing to travel and work.
The control method according to claim 1, wherein the step of "determining whether the autonomous vehicle has reached or approached the boundary according to the color layout in the real-time image" comprises:

Determine the range of the boundary according to the color layout in the real-time image; calculate the ratio R1 between the area of the boundary in the real-time image and the total area of the real-time image;

When R1≥T1, it is judged that the automatic traveling equipment has reached or approached the boundary.
The control method according to claim 2, wherein the step of "determining the range of the boundary according to the layout of colors in the real-time image" comprises:

Detect or obtain the color of the border;

Convert the real-time image to HSV format or HSI format or LAB format; obtain a collection of all pixels that present border colors in the real-time image.
The control method according to claim 1, characterized in that, the step of "after the automatic traveling equipment stops forward traveling and rotates a preset angle, then continues to travel and work" comprises:

The automatic traveling equipment stops moving forward;

The automatic traveling equipment moves in an evasion distance opposite to the original forward direction;

The automatic traveling equipment rotates by a preset angle;

The autonomous walking device continues to travel and work.
A self-propelled equipment, which includes a main body, a walking module, and a power supply module; characterized in that, the self-propelled equipment includes: a camera device, the camera device is arranged on the front side of the main body, and used to obtain the moving direction of the self-propelled equipment Real-time images taken in real time on the Internet;

The control device is connected to the camera device and used to determine whether the autonomous vehicle has reached or approached the boundary according to the color layout in the real-time image, and when it is determined that the autonomous vehicle has reached or approached the boundary, it controls the autonomous vehicle to stop moving forward and After rotating the preset angle, continue to move forward and work.
The autonomous walking device according to claim 5, wherein the autonomous walking device further comprises a calculation module, and the control module can also determine the range of the boundary according to the layout of colors in the real-time image, and the calculation module and the control module Connected to calculate the ratio R1 between the area of the boundary in the real-time image and the total area of the real-time image; and when R1≥T1, the control module determines that the autonomous walking device has reached or approached the boundary.
The autonomous walking device according to claim 6, wherein the control module comprises a format conversion unit and a color detection unit, and the format conversion unit is used to convert the real-time image into HSV format or HSI format or LAB format; The color detection unit can be used to detect or obtain the color of the border, and obtain a set of all pixels that present the color of the border in the real-time image.
An automatic working system for automatic walking equipment as claimed in claims 5 to 7, characterized in that it comprises:

Automatic walking equipment

The boundary is enclosed in a ring shape and is formed to define the working area of the automatic walking equipment, the boundary is detachably set on the ground; the color of the boundary is different from the color of the ground.
The automatic working system according to claim 8, wherein the boundary comprises boundary piles arranged at intervals and an enclosure connected between adjacent boundary piles, and the boundary piles extend upward from the ground and can be partially inserted into the ground Or fully supported on the ground.
The automatic working system according to claim 9, wherein the enclosure is made of flexible material.