WO2021169192A1

WO2021169192A1 - Automatic working system, automatic walking device and control method therefor, and computer-readable storage medium

Info

Publication number: WO2021169192A1
Application number: PCT/CN2020/108926
Authority: WO
Inventors: 朱绍明; 任雪
Original assignee: 苏州科瓴精密机械科技有限公司
Priority date: 2020-02-28
Filing date: 2020-08-13
Publication date: 2021-09-02
Also published as: CN111339961A

Abstract

Provided are an automatic working system, an automatic walking device and a control method therefor, and a computer-readable storage medium. The control method comprises: acquiring a photographed image; processing the photographed image to acquire a processed image; acquiring a contrast image; comparing the processed image with the contrast image, and calculating a similarity d therebetween; and if d > Vc, determining that the photographed image belongs to a lawn area, and if d < Vc, determining that the photographed image belongs to a non-lawn area, wherein Vc is a preset comparison threshold value. The contrast image may be an image of the lawn region or a derivative image thereof. Thus, whether an automatic walking device encounters a boundary or travels to the outside of a working area can be analyzed by analyzing a photographed image, which is more convenient, and thus making the control also more sensitive and effective.

Description

Automatic working system, automatic walking equipment and control method thereof, and computer readable storage medium

Technical field

The invention relates to the field of intelligent control, in particular to an automatic working system, an automatic walking device, a control method thereof, and a computer-readable storage medium.

Background technique

With the continuous advancement of computer technology and artificial intelligence technology, automatic walking equipment and automatic working systems of intelligent robots have slowly entered people's lives, such as intelligent sweeping robots and intelligent lawn mower robots. Generally, such smart robots are small in size, and are integrated with sensing devices, driving devices, batteries, etc., without manual manipulation, and can travel and work in a prescribed area. In addition, when the battery power is insufficient, it can automatically return to the charging station, dock with the charging station and charge, and continue to travel and work after the charging is completed.

For existing intelligent lawn mowing robots, the working area of the existing automatic working system is large lawn, and the boundary is mostly energized equipment buried under the ground, so that the intelligent lawn mowing robot can sense . However, if the boundary line is buried under the ground, much manpower and material resources are required. And burying the boundary line requires certain requirements, for example, the angle of the corner cannot be less than 90 degrees, etc., which also limits the shape of the lawn for the intelligent lawnmower robot to work to a certain extent.

Therefore, in view of the above problems, an automatic working system built on the ground can be designed, and the boundary line can be recognized by a visual recognition method. However, due to the different types of lawns, different seasons or the influence of sunlight in the morning and evening, the lawn may show different colors such as yellow-green, light green, dark green, dark green, gray-green, etc., and there may be dead grass or small pieces in the lawn. Bare plots. All these factors will affect the accuracy of visual recognition by automatic walking equipment.

Therefore, it is necessary to design an automatic working system that can be built on the ground and has a higher degree of visual recognition accuracy for an automatic walking device, a control method thereof, and a computer-readable storage medium.

Summary of the invention

In order to solve one of the above-mentioned problems, the present invention provides a control method of an autonomous walking device, and the control method includes:

Obtain a photographed image;

Process the captured image to obtain the processed image;

Obtain a comparison image;

Compare the processed image with the contrast image and calculate the similarity d;

If d>Vc, it is judged that the captured image belongs to the lawn area;

If d<Vc, it is determined that the captured image belongs to a non-turf area;

Among them, Vc is the preset comparison threshold.

As a further improvement of the present invention, the processed image and the contrast image are both in histogram format.

As a further improvement of the present invention, the step of "processing the captured image to obtain the processed image" includes:

Obtain the captured image in HSV format and record it as HSV image;

Calculate the histogram of the HSV image and generate the histogram image;

Normalize the histogram image to generate a normalized histogram and record it as a processed image.

As a further improvement of the present invention, the step of "obtaining a contrast image" includes:

Preset contrast histogram;

Recall the contrast histogram and record it as the contrast image.

Obtain sample images in HSV format and record them as HSV sample images;

Calculate the histogram of the HSV sample image and generate the sample histogram image;

Normalize the sample histogram image to generate a normalized sample histogram and record it as a contrast image.

As a further improvement of the present invention, the step of "compare the processed image and the contrast image and calculate the similarity d" includes: calculating the similarity d between the processed image and the contrast image through correlation analysis or chi-square test or intersection method or distance analysis method.

To solve one of the above-mentioned problems, the present invention also provides an automatic working system, including:

Automatic walking equipment can work according to the above-mentioned control method;

The working area is provided with a non-working area outside the edge of the working area, and the geology of the working area and the non-working area are different.

The boundary is surrounded in a ring shape and formed to define the working area of the autonomous vehicle, and the boundary extends upward from the ground.

In order to solve one of the above-mentioned problems, the present invention also provides an automatic walking device, including a main body, a walking module, a power supply module, and a memory and a processor arranged in the main body. The memory stores a computer that can run on the processor. Program, the autonomous walking device further includes a camera arranged on the body, the shooting direction of the camera is facing the front side of the autonomous walking device along the traveling direction; when the processor executes the computer program, it can be implemented as described above The steps of the control method of autonomous walking equipment.

In order to solve one of the above-mentioned problems, the present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it can realize the steps in the control method of the autonomous walking device as described above. .

Compared with the prior art, the present invention can process and analyze the photographed images taken by the autonomous walking equipment, and determine the degree of similarity between the processed image and the contrast image by calculating the similarity between the processed image and the contrast image And, the contrast image may be an image of the lawn area or a derivative image thereof. Therefore, if the similarity d is greater than the comparison threshold Vc, it means that the captured image and the contrast image are relatively similar, and the captured image belongs to the lawn area, and the automatic walking device can continue to work freely; if the similarity d is less than the comparison threshold Vc, then It shows that the similarity between the captured image and the contrast image is low, and the captured image belongs to a non-turf area, and the automatic walking device needs to perform operations such as backing and turning to avoid the boundary or non-working area. Therefore, by analyzing the captured images, it can be analyzed whether the autonomous walking device has encountered a boundary or traveled outside the working area, which is more convenient and makes the control more sensitive and effective.

Description of the drawings

Figure 1 is a schematic structural diagram of an automatic working system in a second embodiment of the present invention;

FIG. 2 is a schematic flowchart of an embodiment of the automatic working method of the present invention;

Specific embodiment

In order to enable those skilled in the art to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

In the drawings of the present application, for the convenience of illustration, some dimensions of the structure or part are exaggerated relative to other structures or parts, and therefore, they are only used to illustrate the basic structure of the subject matter of the present application.

The self-propelled equipment 1 of the present invention may be an automatic lawn mower, or an automatic vacuum cleaner, etc., which automatically walks in the work area for mowing and vacuuming. In a specific example of the present invention, the self-propelled equipment 1 is used as a lawn mower. An example is specifically described, and accordingly, the working area may be a lawn. Of course, the self-propelled equipment 1 is not limited to lawn mowers and vacuum cleaners, and can also be other equipment, such as spraying equipment, snow removal equipment, monitoring equipment, etc., suitable for unattended equipment.

As shown in Fig. 1 and Fig. 2, in the present invention, a control method of an autonomous walking device 1 is provided, and the control method includes:

Obtain the captured image orgImage;

Process the captured image orgImage to obtain the processed image;

Obtain a comparison image;

If d>Vc, it is judged that the captured image orgImage belongs to the lawn area;

If d<Vc, it is judged that the captured image orgImage belongs to a non-turf area;

Among them, Vc is the preset comparison threshold.

Therefore, in the present invention, the captured image orgImage captured by the autonomous vehicle 1 can be processed and analyzed, and the similarity d between the processed image and the contrast image can be calculated to determine the degree of similarity between the processed image and the contrast image, and , The contrast image can be an image of the lawn area or a derivative image thereof. Therefore, if the similarity d is greater than the comparison threshold Vc, it means that the captured image orgImage and the contrast image are relatively similar, and the captured image orgImage belongs to the lawn area, and the autonomous walking device 1 can continue to work freely; if the similarity d is less than the comparison threshold Vc means that the similarity between the captured image orgImage and the contrast image is low. The captured image orgImage belongs to a non-turf area, and the automatic walking device 1 needs to perform operations such as backing and turning to avoid the boundary 2 or the non-working area. Therefore, by analyzing the captured image orgImage, it can be analyzed whether the autonomous walking device 1 encounters the boundary 2 or travels outside the working area, which is more convenient and makes the control more sensitive and effective.

In addition, in this specific embodiment, if d=Vc, it is also determined that the captured image orgImage belongs to the lawn area.

Further, the processed image and the contrast image are both in histogram format. In the field of image processing and image recognition, the histogram can reflect the proportion of different colors in the entire image, but it does not reflect the spatial position of each color. It is suitable for describing images that cannot be automatically segmented. Since the autonomous vehicle 1 in the present invention travels in the grass, the images taken by the autonomous vehicle 1 in its working area are all related to the grass, so the color is relatively uniform on the whole, approximately green, thus, The processed image and the contrast image can be adjusted to histogram format and then the two can be compared to calculate the gap between the processed image and the contrast image. If the gap is larger, it means that there are fewer green-related pixels in the processed image. , It can be judged that the captured image orgImage belongs to a non-turf area.

Specifically, the step of "processing the captured image orgImage to obtain a processed image" includes:

Obtain the captured image orgImage in HSV format and record it as HSV image hsvImage1;

Calculate the histogram of the HSV image hsvImage1, and generate the histogram image hist1;

The histogram image hist1 is normalized, and a normalized histogram H1 is generated and recorded as a processed image.

Among them, the HSV format is a color space, which includes hue H, saturation S, and lightness V. The hue H is measured by an angle, and its value range is 0°～360°. It is calculated in a counterclockwise direction from red. Red is 0°, green is 120°, and blue is 240°. Saturation S indicates the degree to which the color is close to the spectral color. The greater the proportion of the spectral color, the higher the degree of the color close to the spectral color, and the higher the saturation of the color. The value of saturation ranges from 0% to 100%. The larger the value, the more saturated the color. The lightness V indicates the brightness of the color. For the color of an object, the lightness value V is related to the transmittance or reflectance of the object. The usual value ranges from 0% to 100%, with 0% being black and 100% being white.

In addition, usually, the captured image orgImage is an RGB format image. Therefore, the step of "acquiring a captured image orgImage in HSV format" specifically includes: converting the captured image orgImage into an HSV image. Of course, if the captured image orgImage itself is in the HSV format, no additional format conversion is required, and the objective of the present invention can also be achieved.

In addition, in "normalizing the histogram image hist1", the normalization refers to the process of performing a series of standard processing transformations on the image and transforming the image into a fixed standard format. Similarity analysis can be carried out more conveniently.

Through the above method, the captured image orgImage can be converted into the normalized histogram H1, that is, the processed image, so as to facilitate the comparison with the contrast image and calculate the similarity.

In addition, as described above, the comparison image is also in the histogram format. Therefore, in the present invention, two specific embodiments are provided to obtain the comparison image in the histogram format.

In an embodiment, the step of "obtaining a comparison image" includes:

Preset contrast histogram;

Recall the contrast histogram and record it as the contrast image.

In this embodiment, a comparison histogram is directly preset, and the comparison image is obtained by directly calling the comparison histogram. The comparison histogram can be pre-stored in the automatic walking device, and the comparison histogram can be retrieved and directly compared with the processed image, so that there is no need to calculate the comparison histogram separately. The comparison histogram is universal and can be applied to the lawn in that place. The comparison histogram can be directly fitted through simulation calculations and other methods.

Or, in the second embodiment, the step of "obtaining a comparison image" includes:

Obtain the sample image samImage in HSV format and record it as HSV sample image hsvImage2;

Calculate the histogram of the HSV sample image hsvImage2, and generate the sample histogram image hist2;

The sample histogram image hist2 is normalized to generate a normalized sample histogram H2 and recorded as a contrast image.

In this second embodiment, the steps are similar to the above-mentioned steps for acquiring the processed image of the captured image orgImage. Before work, through human-computer interaction, the user can issue a shooting instruction to the automatic walking device 1 so that the automatic walking device 1 can shoot the current lawn to obtain a sample image samImage, and perform similar processing steps to obtain a comparison image . In this embodiment, each lawn can be individually photographed in advance, so that the comparison with the above-mentioned processed image is more accurate, more suitable for the current situation of the lawn, and more accurate. In order to be more precise, multiple images can be obtained by taking multiple shots of multiple lawns in different environments, such as morning and evening under different light, cloudy or rainy, and then fitting these multiple images to obtain sample images. Alternatively, a default lawn image can also be preset as the sample image samImage. When the default lawn image is used as the sample image samImage, the default lawn image is also universal and can be applied to lawns in various situations.

Similarly, in the step "Obtain a sample image in HSV format", the sample image samImage in the HSV format can be directly obtained, or the sample image samImage can be converted into the HSV format.

Further, the step of "compare the processed image and the contrast image and calculate the similarity d" includes: calculating the similarity d between the processed image and the contrast image through correlation analysis or chi-square test or intersection method or distance analysis method. Of course, if other methods are used to calculate the similarity d, the objective of the present invention can also be met.

In the present invention, a method for calculating similarity d is proposed, wherein correlation analysis is a statistical analysis method for studying the correlation between two or more random variables in the same position. The chi-square test includes: the chi-square test for comparing two rates or two component ratios; the chi-square test for comparing multiple rates or multiple component ratios and the correlation analysis of classification data. The intersection method and distance analysis method are also methods for similarity analysis, and the details will not be repeated.

The comparison threshold Vc is related to factors such as the photographing effect of the camera. After determining the automatic working system, the developer can test different values and then use an optimal value as the value of the comparison threshold Vc. In a specific embodiment of the present invention, the value of the comparison threshold Vc is taken to be zero.

The present invention also provides an automatic working system. In order to enable the automatic traveling device 1 to recognize the boundary 2 or its working area during walking, and to better calculate the similarity in the present invention, the present invention provides Two embodiments of automatic working systems are presented.

Specifically, in the first embodiment, the automatic working system includes:

The automatic walking equipment 1 can work according to the above-mentioned control method;

In the working area, the autonomous vehicle 1 can travel and work in the working area, and a non-working area is arranged outside the edge of the working area, and the geology of the working area and the non-working area are different.

Since in the present invention, the autonomous walking device 1 acquires the captured image orgImage and then performs similarity analysis, the boundary 2 may not be additionally provided in the present invention. Since the autonomous walking device 1 of the present invention is applied to a lawn mower, the grass is the working area. Obviously, the non-working area can be bare soil, floor, cement board, etc., and its geology is larger than that of grass. Its color is also quite different from that of grass. Therefore, the control method in the present invention can be applied to analyze whether the autonomous vehicle 1 reaches the edge of the working area.

Or, in the second embodiment, the automatic working system includes:

The boundary 2 is surrounded in a ring shape and formed to define the working area of the autonomous vehicle 1, and the boundary 2 extends upward from the ground.

In this embodiment, the autonomous vehicle 1 obtains the captured image orgImage, and then processes and analyzes the captured image orgImage to obtain the area where the autonomous vehicle 1 travels. Therefore, the boundary 2 of the automatic working system of the present invention can be upward from the ground. Extend, so as to be photographed and recognized by the autonomous vehicle 1.

The present invention also provides an automatic walking device 1, including a main body, a walking module, a power supply module, and a memory and a processor arranged in the main body. The memory stores a computer program that can run on the processor. The walking device 1 also includes a camera arranged on the body, and the shooting direction of the camera faces the front side of the automatic walking device 1 along the traveling direction; the processor can realize the automatic walking as described above when the computer program is executed. The steps of the control method of the device 1. That is to say, when the processor executes the computer program, the steps of the control method of any embodiment of the autonomous walking device 1 described above can be implemented.

As described above, the body of the autonomous vehicle 1 in the present invention is provided with a camera, so that the captured image orgImage can be photographed and obtained. In addition, the photographing direction of the camera is toward the front side of the autonomous vehicle 1 in the traveling direction, so that the camera captures a view of the front side of the autonomous vehicle 1. Therefore, the following motion trajectory of the autonomous walking device 1 can be analyzed according to the captured image orgImage captured by the autonomous walking device 1: If it is determined that the captured image orgImage belongs to the lawn area, the autonomous walking device 1 is controlled to further walk and work; if the shooting is determined If the image orgImage belongs to a non-grass area, the automatic walking device 1 is controlled to stop, turn, or retreat.

Similarly, the present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it can realize the steps in the control method of the autonomous walking device 1 as described above. That is to say, when the processor executes the computer program, the steps of the control method of any embodiment of the autonomous walking device 1 described above can be implemented.

In summary, the present invention provides an automatic working system, an automatic driving device 1 and a control method thereof, and a computer readable storage medium. Among them, in the control method of the automatic driving device 1, the image of the automatic driving device 1 can be captured. The captured image orgImage is processed and analyzed, and the similarity d between the processed image and the contrast image is calculated to determine the degree of similarity between the processed image and the contrast image, and the contrast image can be the image of the lawn area or its extended image . Therefore, if the similarity d is greater than the comparison threshold Vc, it means that the similarity between the captured image orgImage and the contrast image is low, and the captured image orgImage belongs to the lawn area, and the autonomous walking device 1 can continue to work freely; the similarity d If it is less than the comparison threshold Vc, it means that the captured image orgImage and the contrast image are not very similar. The captured image orgImage belongs to a non-turf area, and the automatic walking device 1 needs to perform operations such as backing and turning to avoid the boundary 2. Therefore, by analyzing the captured image orgImage, it can be analyzed whether the autonomous walking device 1 encounters the boundary 2 or travels outside the working area, which is more convenient and makes the control more sensitive and effective.

Furthermore, the processed image and the contrast image in the present invention are both in histogram format, and the histogram can effectively describe the global distribution characteristics of colors in the image, and has the invariance of translation, scale and rotation, so the recognition is more accurate. In a specific embodiment, by taking an image of the lawn in advance as a sample image and processing it as a comparison image, the interference of the environment on visual recognition can be eliminated, the lawn area and the non-turf area can be effectively identified, and the work efficiency of the autonomous walking device 1 can be improved. .

In addition, the automatic working system of the present invention does not need to lay out additional boundary lines under the ground, reducing the cost of manpower and material resources, the shape of the working area is flexible and unrestricted, the recognition speed is fast, and the accuracy is high. It can also improve the automatic walking equipment of the present invention. 1 Especially the working efficiency of the automatic lawn mower robot.

In addition, it should be understood that although this specification is described in accordance with the implementation manners, not each implementation manner only includes an independent technical solution. This narration in the specification is only for the sake of clarity, and those skilled in the art should regard the specification as a whole The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

The series of detailed descriptions listed above are only specific descriptions of feasible implementations of the present invention, and are not intended to limit the scope of protection of the present invention. Any equivalent implementations or changes made without departing from the technical spirit of the present invention All should be included in the protection scope of the present invention.

Claims

A control method of autonomous walking equipment, characterized in that the control method includes:

Obtain a photographed image;

Process the captured image to obtain the processed image;

Obtain a comparison image;

Compare the processed image with the contrast image and calculate the similarity d;

If d>Vc, it is judged that the captured image belongs to the lawn area;

If d<Vc, it is determined that the captured image belongs to a non-turf area;

Among them, Vc is the preset comparison threshold.
The control method according to claim 1, wherein the processed image and the contrast image are both in histogram format.
The control method according to claim 1, wherein the step of "processing the captured image to obtain the processed image" comprises:

Obtain the captured image in HSV format and record it as HSV image;

Calculate the histogram of the HSV image and generate the histogram image;

Normalize the histogram image to generate a normalized histogram and record it as a processed image.
The control method according to claim 1, wherein the step of "obtaining a contrast image" comprises: preset a contrast histogram;

Recall the contrast histogram and record it as the contrast image.
The control method according to claim 1, wherein the step of "obtaining a comparison image" comprises: obtaining a sample image in HSV format and recording it as an HSV sample image;

Calculate the histogram of the HSV sample image and generate the sample histogram image;

Normalize the sample histogram image to generate a normalized sample histogram and record it as a contrast image.
The control method according to claim 1, wherein the step of "compare the processed image and the contrast image and calculate the similarity d" comprises: calculating the processing image and the comparison through correlation analysis or chi-square test or intersection method or distance analysis method. Image similarity d.
An automatic working system, characterized in that it comprises:

An automatic walking device that can work according to the control method according to any one of claims 1 to 6; a working area, a non-working area is arranged outside the edge of the working area, and the geology of the working area and the non-working area are different .
An automatic working system, characterized in that it comprises:

The autonomous vehicle can work according to the control method according to any one of claims 1 to 6; the boundary is ring-shaped and formed to define the working area of the autonomous vehicle, and the boundary extends upward from the ground .
A self-propelled equipment includes a main body, a walking module, a power supply module, and a memory and a processor arranged in the main body. The memory stores a computer program that can run on the processor, and is characterized in that the self-propelled equipment also It includes a camera arranged on the main body, and the shooting direction of the camera faces the front side of the automatic traveling equipment along the traveling direction; the processor can implement the computer program described in any one of claims 1 to 6 when the processor executes the computer program. Steps of the control method of autonomous walking equipment.
A computer-readable storage medium storing a computer program, wherein the computer program can implement the steps in the method for controlling an autonomous walking device according to any one of claims 1 to 6 when the computer program is executed by a processor.