WO2021139397A1 - 自移动设备的控制方法 - Google Patents

自移动设备的控制方法 Download PDF

Info

Publication number
WO2021139397A1
WO2021139397A1 PCT/CN2020/128396 CN2020128396W WO2021139397A1 WO 2021139397 A1 WO2021139397 A1 WO 2021139397A1 CN 2020128396 W CN2020128396 W CN 2020128396W WO 2021139397 A1 WO2021139397 A1 WO 2021139397A1
Authority
WO
WIPO (PCT)
Prior art keywords
self
mobile device
boundary
image
analyzed
Prior art date
Application number
PCT/CN2020/128396
Other languages
English (en)
French (fr)
Inventor
多尔夫·达维德
Original Assignee
苏州宝时得电动工具有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州宝时得电动工具有限公司 filed Critical 苏州宝时得电动工具有限公司
Publication of WO2021139397A1 publication Critical patent/WO2021139397A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0242—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using non-visible light signals, e.g. IR or UV signals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0225—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving docking at a fixed facility, e.g. base station or loading bay
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow

Definitions

  • the invention relates to a control method of self-mobile equipment, in particular to a control method of image recognition based on artificial intelligence.
  • intelligent self-mobile devices have gradually become familiar to people. Since mobile devices can automatically perform related tasks according to preset programs without manual operation and intervention, they are widely used in industrial applications and household products. For example, industrial applications include robots that perform various functions, and household products Applications include lawn mowers, vacuum cleaners, etc. These intelligent self-moving devices greatly save people's time, reduce people's labor intensity, and improve production efficiency or quality of life.
  • the mobile device works in an area set by the user, and the working area needs to be set by the user in advance.
  • the user before the mobile device starts to work, the user is required to set a boundary line at the boundary of the work area, and the boundary line is generally an energized wire that can send a signal.
  • the setting of the boundary line increases the user's early intervention, and also limits the possibility for the user to change the working area of the mobile device. Therefore, at present, it appears that an image acquisition device is set on a mobile device, and the working area of the mobile device is identified by comparing characteristic values such as image color and gray scale with preset values.
  • these disturbances include: shadows of objects; shadows of the lawnmower itself; reflected light and refraction; presence of dry grass; underexposure or overexposure of the camera; differences caused by weather phenomena Conditions; different perceptions due to the direction of travel; grains with non-nominal heights; perspective distortion due to uneven soil or tilted vehicles, etc.
  • Current image recognition methods are difficult to accurately recognize the boundaries of the work area.
  • Another situation is that when the existing self-mobile devices return to charging, they usually return along the boundary line or the wall, which can guarantee the return of the self-mobile device to the charging station within a certain period of time, but if there is no boundary line, Or use other methods to guide the return of the self-mobile device, and the coverage of these guide signals is relatively small compared to the working area, then the self-mobile device will spend a relatively long time searching for the guide signal, and even cannot return to the charging station.
  • the problem to be solved by the present invention is to provide a control method for improving the working efficiency of the self-moving device.
  • a method for edge control of a self-moving device that moves and works in a work area characterized in that the method includes:
  • the distance between the self-mobile device and the boundary of the work area is controlled to control the self-mobile device to move along the boundary of the work area.
  • the training-based neural network performing operations on the digital image includes performing image segmentation on the digital image.
  • the self-moving device is controlled to move so that the image to be analyzed meets a preset condition to control the distance between the self-moving device and the boundary of the working area; the preset condition is based on the self-moving The installation position and installation angle of the image acquisition device of the equipment.
  • the self-moving device is controlled to move parallel to the boundary of the working area to cut the boundary of the working area.
  • the self-moving device is controlled to maintain a first preset distance when moving along the boundary of the working area for the Kth time.
  • the second preset distance is maintained when moving along the boundary of the working area for the K+1th time.
  • controlling the distance between the self-moving device and the boundary of the working area includes at least two preset distances.
  • a regression control method from a mobile device characterized in that the method includes:
  • the charging station is identified based on at least one of a preset shape, a preset mark, and a preset object in the image to be analyzed.
  • the shortest path between the location of the self-mobile device and the charging station is generated based on the image to be analyzed, and the self-mobile device is controlled to move toward the charging station.
  • the training-based neural network is used to calculate the working environment image of the self-mobile device to identify the relative position relationship between the self-mobile device and the boundary, or the relative position relationship between the self-mobile device and the charging station. Control the distance between the self-moving device and the boundary based on the positional relationship between the self-moving device and the boundary, and optionally realize cutting to the edge or reduce the indentation.
  • a path between the self-mobile device and the charging station is generated based on the positional relationship between the self-mobile device and the charging station, so that the self-mobile device is controlled to return to the charging station along the path, and the return efficiency of the self-mobile device can be improved.
  • Figure 1 is a schematic diagram of an automatic working system in an embodiment
  • Figure 2 is a schematic diagram of a self-moving device in an embodiment
  • Figure 3 is a schematic diagram of a digital image and an image to be analyzed in an embodiment
  • Fig. 4 is a schematic diagram of a movement path of a self-mobile device in an embodiment.
  • the automatic working system of this embodiment includes a self-mobile device 1 and a charging station 5.
  • the self-mobile device 1 walks and works in a working area, wherein the boundary 3 is used to limit the working area of the automatic working system.
  • the charging station 5 is used to park the mobile device 1, especially when the energy is insufficient, to return to supplement energy.
  • the self-moving device 1 may be an automatic lawn mower, an automatic snow sweeper, etc., which automatically walk on the ground or surface of the work area to perform work such as mowing or snow sweeping. In this embodiment, the self-moving device 1 takes an automatic lawn mower as an example.
  • Boundary 3 is the collective term for the outer and inner boundaries of the working area.
  • the outer boundary is the periphery of the entire work area, usually connected end to end, enclosing the work area.
  • the inner boundary includes the boundary of obstacles, which are parts or areas that cannot be walked on within the working range, such as indoor sofas, bed cabinets, or outdoor ponds, flower stands, etc.
  • the boundary 3 includes the boundary line between the lawn and other vegetation, the boundary line between the lawn and the pond, the edge line of the fence, the edge line of the special object placed on the lawn, and so on.
  • the self-mobile device 1 includes a walking module 11, a working module 13, an image acquisition module 15, an energy module 17, a control module 19, and so on.
  • the walking module 11 is used to drive the self-mobile device 1 to walk in the working area 7, and is usually composed of a wheel set installed on the self-mobile device 1 and a walking motor that drives the wheel set.
  • the wheel set includes a driving wheel connected to the walking motor and an auxiliary wheel mainly playing an auxiliary supporting role.
  • the number of driving wheels is two, which are located at the rear of the mobile device 1, each The driving wheel is connected with a walking motor, the number of auxiliary wheels is one or two, and is located at the front of the self-moving device.
  • the working module 13 is used to perform specific working tasks of the mobile device 1.
  • the working module 13 includes a mowing blade, a cutting motor, etc., and may also include a mowing height adjustment mechanism and other components for optimizing or adjusting the mowing effect.
  • the image acquisition module 15 is used to detect the relative positional relationship between the mobile device 1 and the boundary 3, which may specifically include one or more of the distance, the angle, and the inner and outer directions of the boundary.
  • the image acquisition module 15 specifically includes one or more cameras for acquiring images from the working surface of the mobile device.
  • the camera can capture more or less part of the surrounding working surface based on its number, its position, and the geometry of the lens that characterizes its field of view.
  • Cameras and similar image acquisition modules can receive grayscale images or preferably color-coded images of the visible spectrum.
  • the image acquisition module can also be configured to operate in the infrared and ultraviolet spectra, or use a channel dedicated to depth to complete image acquisition.
  • the energy module 17 is used to provide energy for various tasks of the self-mobile device 1, and includes a rechargeable battery and a charging connection structure.
  • the charging connection structure is usually a charging electrode sheet that can be exposed outside the self-mobile device.
  • the control module 19 is used to control the self-mobile device 1 to automatically walk and work. It is the core component of the self-mobile device 1. Its functions include controlling the work module 13 to start or stop, generate a walking path and control the walking module to determine energy according to walking The power of the module 17 is instructed to return from the mobile device 1 to the charging station 5 for automatic docking and charging and so on.
  • the control module 19 usually includes a single-chip microcomputer, a memory, and other peripheral circuits.
  • the self-mobile device 1 also includes a housing for accommodating and installing various modules, a control panel for users to operate, etc.
  • the self-mobile device 1 may also include various environmental sensors, such as humidity sensors, temperature sensors, and acceleration sensors. Light sensors, etc., these sensors can help the mobile device to determine the working environment to execute the corresponding program.
  • the charging station 5 is usually located within the working range, generally located near or on the boundary 3, and is connected to the mains or other power supply systems for charging from the mobile device 1.
  • the charging station 5 is equipped with charging electrode pads for and The corresponding electrode pads of the mobile device 1 are docked.
  • the charging station may also be arranged outside the working area.
  • the image acquisition module 15 includes a camera 151, which is installed on the front side of the housing for acquiring images from the front working surface of the mobile device 1.
  • the image acquisition module 15 may include two or more cameras, and the control module 19 may perform processing or stitching processing on the images acquired by different cameras, respectively.
  • the control module 19 receives the image acquired by the camera 151, and performs edge control on the mobile device 1 based on this.
  • the control method includes:
  • the distance between the self-mobile device 1 and the boundary 3 is controlled to control the self-mobile device 1 to move along the boundary 3.
  • processing a digital image based on a trained neural network mainly includes performing image segmentation on the digital image to obtain an image to be analyzed.
  • Image segmentation refers to the division of an image into several disjoint areas based on features such as grayscale, color, spatial texture, and geometric shapes, so that these features show consistency or similarity in the same area, but show in different areas Make a clear difference.
  • a large amount of working environment data collected by the camera is used to train the neural network, so that the neural network can distinguish various types of objects such as lawns, roads, soil, shrubs, ponds, etc., so as to recognize the boundary 3.
  • Figure 3 is a schematic diagram of the digital image and the image to be analyzed in an embodiment. As shown in Figure 3, in one embodiment, the area A on the left of the image is a lawn, and the area B on the right of the image is a road. They are in the image to be analyzed. The two adjacent categories are separated by a boundary line L.
  • the set of points at the boundary between the two categories can be approximated with a straight line, for example by linear regression, such a straight line is characterized by an offset and an angle coefficient. If the self-mobile device 1 is perfectly aligned with the boundary line L, then such a straight line will actually be vertical (a zero angle factor in the selected coordinate system of the image). Conversely, if the self-mobile device 1 is not aligned with respect to the aforementioned boundary line L, such a straight line will be inclined (positive or negative angle coefficient in the selected coordinate system of the image).
  • the self-moving device 1 if the coefficient is negative, the self-moving device 1 is controlled to rotate clockwise, and if the coefficient is positive, the self-moving device 1 is controlled to rotate counterclockwise.
  • the offset of the straight line characterizes the degree of deviation from the mobile device 1 and the boundary 3.
  • the self-mobile device 1 is controlled to move so that the image to be analyzed meets a preset condition, which specifically includes actions such as rotating, forward, and backward, to control the distance between the self-mobile device 1 and the boundary 3.
  • a preset condition which specifically includes actions such as rotating, forward, and backward, to control the distance between the self-mobile device 1 and the boundary 3.
  • the preset conditions may include the relative position of the boundary between the working surface and the non-working surface in the image to be analyzed, specifically including the offset and the angle relationship.
  • the self-mobile device 1 is controlled to move based on the image to be analyzed, so that the self-mobile device 1 moves parallel to the boundary 3.
  • the self-moving device 1 moves parallel to the boundary 3 and performs cutting work, the grass on the boundary 3 of the working area can be evenly cut.
  • the image to be analyzed includes area A, area B and boundary line L, and the movement of the self-mobile device is controlled so that the boundary line L is parallel to the direction of movement of the self-mobile device, and the self-mobile device 1 is parallel to the direction of movement of the self-mobile device. Border 3.
  • the self-mobile device 1 is controlled to move based on the image to be analyzed, so that the self-mobile device maintains a first preset distance from the boundary 3 when the self-mobile device moves along the boundary 3 for the Kth time.
  • the distance between the mobile device 1 and the boundary 3 is controlled to be the first preset distance.
  • the distance between the control self-mobile device 1 and the boundary 3 is still the first preset distance. That is to say, every time the mobile device 1 is controlled to move along the boundary 3, the distance from the boundary 3 is the first preset distance.
  • the first preset distance here can make the self-mobile device 1 as close as possible to the boundary of the working area under safe conditions, thereby cutting the grass at the boundary of the working area cleanly, reducing or avoiding the area that cannot be cut by the self-moving device 1, thereby Reduce or avoid manual intervention.
  • the distance between the mobile device 1 and the boundary 3 when moving along the boundary 3 for the Kth time, is controlled to be the first preset distance.
  • the distance between the mobile device 1 and the boundary 3 is still controlled to be the second preset distance. In other words, when the mobile device 1 is controlled to move along the boundary 3 at different times, the distance from the boundary 3 is not completely the same.
  • the distance between the mobile device 1 and the boundary 3 when moving along the boundary 3 for the K+2th time, the distance between the mobile device 1 and the boundary 3 is controlled to be the first preset distance or the second preset distance, or it may be the third preset distance.
  • the wheels of the self-mobile device 1 will cause a certain degree of pressure loss on the grass. If the path along the boundary 3 is the same every time, it will form on the grass. Severe indentation. However, in this embodiment, the path that the mobile device 1 moves along the boundary 3 is changed, so that damage caused by repeated pressure can be reduced.
  • the relationship between the boundary line L and the selected coordinate system represents the positional relationship between the mobile device 1 and the boundary 3.
  • the self-mobile device 1 is controlled to move based on the image to be analyzed, so that the self-mobile device 1 changes the control distance from the boundary 3 when the self-mobile device 1 moves along the boundary 3 for the Kth time.
  • the distance between the mobile device 1 and the boundary 3 is controlled to change periodically. If the starting point of moving along the edge from the mobile device 1 is changed, the route along which the mobile device 1 moves along each time is changed.
  • the change of the distance can be controlled according to different logics, such as randomly setting a distance within a certain distance, and controlling the movement from the mobile device 1; another example is from the starting point to the ending point. Progressively reduce the distance from boundary 3 and so on.
  • the automatic working system includes a self-mobile device 1 and a charging station 5.
  • the control module 19 controls the self-mobile device 1 to return to the charging station 5.
  • the control module 19 also needs to control the self-mobile device to return to the charging station 5.
  • the image acquisition module 15 of the mobile device 1 includes a camera 153, which is used to capture images of the surrounding environment of the mobile device 1.
  • the camera 153 in this embodiment is mainly used to obtain environmental images from the front of the mobile device 1. In order to obtain a larger range of images, the camera is mainly oriented to the front side. Therefore, the camera 153 in this embodiment and the camera used to identify the boundary 3 different. In some cases, the camera 153 may be the same as the camera that recognizes the boundary 3.
  • the control module 19 receives the image obtained by the camera, and performs regression control on the mobile device 1 based on this.
  • the control method includes:
  • the mobile device 1 is controlled to move toward the charging station 5.
  • processing a digital image based on a trained neural network mainly includes performing image segmentation on the digital image to obtain an image to be analyzed.
  • a large amount of working environment data collected by the camera is used to train the neural network.
  • the working environment data includes the charging station.
  • the position of the charging station 5 in the image to be analyzed is identified through the special shape or mark of the charging station 5 or the characteristics of the object.
  • the shortest path between the mobile device 1 and the charging station 5 is generated. It can be understood that the shortest path here refers to a path that avoids non-working areas such as obstacles, and is also an optimal path obtained based on the digital image currently acquired from the mobile device 1.
  • the camera 153 is continuously collecting images of the surrounding environment, and the control module 19 performs processing and analysis again after receiving the new image, which can continuously optimize the self Return path of mobile device 1.
  • image segmentation includes semantic segmentation.
  • the neural network includes a convolutional neural network (CNN), which includes at least the following layers:
  • the input layer is used to receive at least one down-sampling of the digital image acquired by the image acquisition device 15;
  • At least one convolutional layer At least one convolutional layer
  • At least one deconvolution layer At least one deconvolution layer
  • An output layer which is configured to make available the semantically segmented soil images in at least two categories.
  • the neural network includes a fully convolutional neural network (FCN).
  • FCN fully convolutional neural network
  • the last three layers in the CNN network are all one-dimensional vectors, and the calculation method no longer uses convolution.
  • FCN network all these three layers are converted into multi-channels with the same vector length corresponding to the 1*1 convolution kernel.
  • Convolutional layer so that the last three layers are all calculated by convolution.
  • all are convolutional layers and there are no vectors.
  • the output of the fully convolutional neural network is clearer than the simple turf soil category, and is given by segmented images of different types of grass or obstacles that the mobile device may encounter during work.

Abstract

一种自移动设备（1）的沿边控制方法，自移动设备（1）在工作区域（7）内移动和工作，控制方法包括：获取自移动设备（1）工作表面的数字图像；基于训练的神经网络处理数字图像，以获取待分析图像；基于待分析图像识别工作区域（7）的边界（3）；控制自移动设备（1）与工作区域（7）的边界（3）的距离，以控制自移动设备（1）沿工作区域（7）的边界（3）移动，提高自移动设备（1）的工作效率。

Description

自移动设备的控制方法
本申请要求了申请日为2020年01月07日，申请号为202010014727.0的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及一种自移动设备的控制方法，尤其是一种基于人工智能进行图像识别的控制方法。
背景技术
随着科学技术的发展，智能化的自移动设备逐渐为人们所熟知。由于自移动设备可以自动按照预设程序执行相关任务，无须人工操作与干预，因此在工业应用及家居产品上的应用非常广泛，例如，工业上的应用有执行各种功能的机器人，家居产品上的应用有割草机、吸尘器等。这些智能的自移动设备极大地节省了人们的时间，降低了人们的劳动强度，从而提高了生产效率或生活品质。
一般的，自移动设备在一个用户设定的区域内工作，工作区域需要由用户提前设定。现有技术中，在自移动设备开始工作前，需要用户在工作区域边界设置边界线，边界线一般为可以发出信号的通电导线。边界线的设置增加了用户的前期干预，也限制了用户更改自移动设备工作区域的可能性。因此，目前出现了在自移动设备上设置图像获取设备，通过对图像颜色、灰度等特征值与预设值进行对比，来识别自移动设备的工作区域。
但是，由于实际操作环境中存在的多个扰动，这些干扰包括：物体的阴影；割草机本身的阴影；反射光和折射；存在干枯的草；相机曝光不足或曝光过度；天气现象引起的不同情况；由于行进方向而产生不同的感知；具有非标称高度的谷物；由于土壤不平坦或车辆倾斜而导致的透视变形等等。目前的图像识别方法很难准确地识别工作区域的边界。
另一种情况是，现有的自移动设备回归充电时多沿着边界线或墙边回归，能够在一定时间内保证自移动设备回归到充电站充电，但是如果在没有边界线的情况下，或是采用其他方式引导自移动设备回归，而这些引导信号的覆盖面相对于工作区域比较小，那么自移动设备会在寻找引导信号时耗费比较长的时间，甚至无法回归充电站。
发明内容
为克服现有技术的缺陷，本发明所要解决的问题是提供一种提高自移动设备工作效率的控制方法。
本发明解决现有技术问题所采用的一种技术方案是：
一种自移动设备的沿边控制方法，所述自移动设备在工作区域内移动和工作，其特征在于，所述方法包括：
获取所述自移动设备工作表面的数字图像；
基于训练的神经网络处理所述数字图像，以获取待分析图像；
基于所述待分析图像识别所述工作区域边界；
控制所述自移动设备与所述工作区域边界的距离，以控制所述自移动设备沿所述工作区域边界移动。
在一实施例中，所述基于训练的神经网络对所述数字图像进行运算包括对所述数字图像进行图像分割。
在一实施例中，控制所述自移动设备移动使所述待分析图像满足预设条件，以控制所述自移动设备与所述工作区域边界的距离；所述预设条件基于所述自移动设备的图像获取装置的安装位置和安装角度。
在一实施例中，控制所述自移动设备平行于所述工作区域边界移动，以切割所述工作区域边界。
在一实施例中，控制所述自移动设备在第K次沿所述工作区域边界移动时保持第一预设距离。
在一实施例中，在第K+1次沿所述工作区域边界移动时保持第二预设距离。
在一实施例中，在第K次沿所述工作区域边界移动时，控制所述自移动设备与所述工作区域边界的距离至少包括两个预设距离。
本发明解决现有技术问题所采用的另一种技术方案是：
一种自移动设备的回归控制方法，其特征在于，所述方法包括：
获取所述自移动设备周围环境的数字图像；
基于训练的神经网络处理所述数字图像，以获取待分析图像；
基于所述待分析图像识别充电站，所述充电站供所述自移动设备停靠或充电；
控制所述自移动设备朝向所述充电站移动。
在一实施例中，基于待分析图像中预设形状、预设标记、预设物体中的至 少一种识别所述充电站。
在一实施例中，基于所述待分析图像生成所述自移动设备所在位置与所述充电站之间的最短路径，控制所述自移动设备朝向所述充电站移动。
与现有技术相比，本发明的有益效果是：
通过基于训练的神经网络对自移动设备的工作环境图像进行运算，从而识别自移动设备与边界的相对位置关系，或识别自移动设备与充电站的相对位置关系。基于自移动设备与边界的位置关系控制自移动设备与边界的距离，可选的实现切割到边或减少压痕。基于自移动设备与充电站点的位置关系生成自移动设备与充电站的路径，从而控制自移动设备沿该路径回归充电站，可以提高自移动设备的回归效率。
附图说明
以上所述的本发明的目的、技术方案以及有益效果可以通过下面附图实现：
图1是一实施例中自动工作系统的示意图；
图2是一实施例中自移动设备的示意图；
图3是一实施例中数字图像及待分析图像的示意图；
图4是一实施例中自移动设备移动路径的示意图。
具体实施方式
如图1，本实施方式的自动工作系统包括自移动设备1和充电站5，自移动设备1在工作区域内行走并工作，其中边界3用于限制自动工作系统的工作区域。充电站5用于供自移动设备1停泊，尤其是在能源不足时返回补充能量。自移动设备1可以是自动割草机、自动扫雪机等，它们自动行走于工作区域的地面或表面上，进行割草或扫雪等工作。本实施例中，自移动设备1以自动割草机为例。
边界3是工作区域外边界和内边界的统称。外边界是整个工作区域的外围，通常首尾相连，将工作区域封闭。内边界包括障碍物的边界，障碍是位于工作范围内的无法在其上行走的部分或区域，如室内的沙发、床柜，或室外的水塘、花台等。本实施例中，边界3包括草坪与其他植被的分界线、草坪与池塘的分界线、篱笆边缘线、草坪上放置的特殊物体边缘线等等。
本实施例中，自移动设备1包括行走模块11、工作模块13、图像获取模块15、能量模块17、控制模块19等。
行走模块11用于带动自移动设备1在工作区域7内行走，通常由安装在自 移动设备1上的轮组和驱动轮组的行走马达组成。轮组包括连接行走马达的驱动轮和主要起辅助支撑作用的辅助轮，优选的，在本发明的具体实施方式中，驱动轮的数量为两个，位于自移动设备1的后部，每个驱动轮连接有一个行走马达，辅助轮的数量为一个或两个，位于自移动设备的前部。
工作模块13用于执行自移动设备1的具体工作任务，本实施例中，工作模块13包括割草刀片、切割马达等，也可以包括割草高度调节机构等优化或调整割草效果的部件。
图像获取模块15用于侦测自移动设备1和界限3的相对位置关系，具体可能包括距离、角度，界限内外方位中的一种或几种。图像获取模块15具体包括1个或1个以上的摄像头，用于获取自移动设备的工作表面的图像。摄像头可以基于其数量，其位置以及表征其视场的透镜的几何形状来获取或多或少的周围工作表面的一部分。摄像头和类似的图像获取模块可以接收灰度级的图像或优选地以颜色编码的可见光谱的图像。图像获取模块也可以被配置为在红外和紫外光谱中操作，或者利用专用于深度的通道来完成图像获取。
能量模块17用于为自移动设备1的各项工作提供能量，其包括可充电电池和充电连接结构，充电连接结构通常为可露出于自移动设备外的充电电极片。
控制模块19用于控制自移动设备1自动行走和工作，是自移动设备1的核心部件，它执行的功能包括控制工作模块13启动工作或停止，生成行走路径并控制行走模块依照行走，判断能量模块17的电量并及时指令自移动设备1返回充电站5自动对接充电等等。控制模块19通常包括单片机和存储器以及其它外围电路。
除了上述模块，自移动设备1还包括容纳和安装各个模块的壳体、供使用者操作的控制面板等，自移动设备1还可能包括各种环境传感器，如湿度传感器，温度传感器，加速度传感器，光线传感器等，这些传感器可以帮助自移动设备判断工作环境，以执行相应的程序。
充电站5通常位于工作范围内，一般位于边界3附近或边界3上，和市电或其它电能提供系统连接，供自移动设备1返回充电，充电站5上设有充电电极片，用于和自移动设备1的相应的电极片对接。在一些实施例中，为了保持工作区域的美观，充电站可也设置于工作范围外。
如图2所示，本实施例中，图像获取模块15包括1个摄像头151，摄像头151安装于壳体前侧，用于获取自移动设备1前侧工作表面的图像。在其他实 施例中，图像获取模块15可以包括2个或两个以上摄像头，控制模块19可对不同摄像头获取的图像分别进行处理或拼接处理。本实施例中，控制模块19接收摄像头151获取的图像，并基于此对自移动设备1进行沿边控制，控制方法包括：
获取自移动设备1工作表面的数字图像；
基于训练的神经网络处理该数字图像，以获取待分析图像；
基于待分析图像识别边界3；
控制自移动设备1与边界3的距离，以控制自移动设备1沿边界3移动。
本实施例中，基于训练的神经网络处理数字图像主要包括对该数字图像进行图像分割，获得待分析图像。图像分割是指根据灰度、彩色、空间纹理、几何形状等特征把图像划分成若干个互不相交的区域，使得这些特征在同一区域内表现出一致性或相似性，而在不同区域间表现出明显的不同。利用摄像头采集的大量工作环境数据对神经网络进行训练，使得该神经网络能够区分草坪、道路、泥土、灌木、池塘等各种不同类型的物体，从而识别到边界3。
图3是一实施例中数字图像及待分析图像的示意图，如图3所示，在一个实施例中，图像左侧区域A为草坪，图像右侧区域B为马路，它们是待分析图像中的两个相邻的类别，并且由界线L分开。
在一个实施例中，两个类别之间的边界处的点集可以例如通过线性回归用直线近似，这样的直线的特征在于偏移和角度系数。如果自移动设备1与界线L完全对齐，则这样的直线实际上将是垂直的(在图像的选定坐标系中为零角系数)。相反，如果自移动设备1相对于前述界线L未对准，则这样的直线将倾斜(在图像的所选坐标系中为正或负角度系数)。
在一个实施例中，如果系数为负，则控制自移动设备1将顺时针旋转，如果系数为正，则控制自移动设备1逆时针旋转。根据选定坐标系，直线的偏移量表征了自移动设备1与边界3的偏离程度。
在一个实施例中，控制自移动设备1移动使待分析图像满足预设条件，具体的包括旋转、前进、后退等动作，以控制自移动设备1与边界3的距离。基于摄像头151的安装位置和安装角度，预设条件可以包括待分析图像中工作表面和非工作表面分界线的相对位置，具体包括偏移量和角度关系等。
在一个实施例中，基于待分析图像控制自移动设备1移动，使自移动设备1平行于边界3移动。当自移动设备1平行于边界3移动且进行切割工作时， 工作区域边界3上的草能够被均匀地切割。在一个具体的实施例中，如图3，待分析图像中包括区域A、区域B和界线L，控制自移动设备移动使得界线L平行于自移动设备的移动方向，则自移动设备1平行于边界3。
在一个实施例中，基于待分析图像控制自移动设备1移动，使自移动设备在第K次沿边界3移动时与边界3保持第一预设距离。
在一个具体的实施例中，在第K次沿边界3移动时，控制自移动设备1与边界3的距离为第一预设距离。在第K+1次沿边界3移动时，控制自移动设备1与边界3的距离仍为第一预设距离。也就是说，控制自移动设备1每一次沿边界3移动时，与边界3的距离都为第一预设距离。这里的第一预设距离能够使自移动设备1在安全的情况下尽可能地靠近工作区域边界，从而将工作区域边界的草切割干净，减少或避免自移动设备1无法切割到的区域，从而减少或避免人工干预。
在一个具体的实施例中，如图4所示，在第K次沿边界3移动时，控制自移动设备1与边界3的距离为第一预设距离。在第K+1次沿边界3移动时，控制自移动设备1与边界3的距离仍为第二预设距离。也就是说，控制自移动设备1在不同次沿边界3移动时，与边界3的距离不完全相同。在其他实施例中，在第K+2次沿边界3移动时，控制自移动设备1与边界3的距离为第一预设距离或第二预设距离，也可以为第三预设距离。对于自移动设备1沿边界3移动次数较多的情况，自移动设备1的轮子会对草地造成一定程度的压损，若每一次沿边界3移动的路径都是相同的，会在草地上形成严重的压痕。而本实施例中，自移动设备1沿边界3移动的路径是变化的，则可以减少重复压力形成的损坏。
在一个具体的实施例中，如图3所示，界线L与选定坐标系的关系表示自移动设备1与边界3的位置关系。通过控制界线L在选定坐标系中的位置，可以控制自移动设备1与边界3的距离。
在一个实施例中，基于待分析图像控制自移动设备1移动，使自移动设备1在第K次沿边界3移动时与边界3控制距离改变。在一个具体的实施例中，在第K次沿边界3移动时，控制自移动设备1与边界3的距离周期性地改变。若自移动设备1开始沿边移动的起点是变化的，则自移动设备1每一次沿边移动的路线都是变化的。在其他实施例中，在第K次沿边界3移动时，距离的变化可以按照不同的逻辑控制，如在一定距离范围内随机给定距离，控制自移动 设备1移动；又如从起点到终点渐进式减小与边界3的距离等等。
如图1所示，自动工作系统包括自移动设备1和充电站5，当自移动设备1能量小于设定值时，控制模块19控制自移动设备1返回充电站5。在下雨或工作完成等情况下，控制模块19也需要控制自移动设备返回充电站5。在一个实施例中，自移动设备1的图像获取模块15包括摄像头153，该摄像头用于拍摄自移动设备1周围环境的图像。本实施例中的摄像头153主要用于获取自移动设备1前方的环境图像，为了获取更大范围的图像，摄像头主要朝向前侧，因此，本实施例中摄像头153与用于识别边界3的摄像头不同。在一些情况中，摄像头153可以与识别边界3的摄像头相同。控制模块19接收摄像头获取的图像，并基于此对自移动设备1进行回归控制，控制方法包括：
获取自移动设备1周围环境的数字图像；
基于训练的神经网络处理数字图像，以获取待分析图像；
基于待分析图像识别充电站5；
控制自移动设备1朝向充电站5移动。
本实施例中，基于训练的神经网络处理数字图像主要包括对该数字图像进行图像分割，获得待分析图像。利用摄像头采集的大量工作环境数据对神经网络进行训练，这些工作环境数据中包括充电站，通过充电站5的特殊形状或标记或物体的特性来识别待分析图像中充电站5的位置。
基于待分析图像获取自移动设备1与充电站5的相对位置关系，生成自移动设备1与充电站5之间的最短路径。可以理解的是，这里的最短路径是指避开障碍物等非工作区域的路径，也是基于自移动设备1当前获取的数字图像得到的最优路径。
在一个实施例中，由于自移动设备1在回归充电站5的过程中，摄像头153在不断采集周围环境的图像，控制模块19接收到新的图像后再次进行处理和分析，可以不断地优化自移动设备1的回归路径。
在一个实施例中，图像分割包括语义分割。本实施例中，神经网络包括卷积神经网络(CNN)，至少包括以下几层：
输入层，用于接收图像获取装置15获取的数字图像的至少一个下采样；
至少一个卷积层；
至少一个反卷积层；
输出层，其被配置为使在至少两个类别中在语义上分割的土壤图像可用。
在一个实施例中，神经网络包括完全卷积神经网络(FCN)。CNN网络中的后三层，都是一维的向量，计算方式不再采用卷积，而FCN网络中，将这三层全部转化为1*1的卷积核所对应等同向量长度的多通道卷积层，使后三层也全部采用卷积计算，整个模型中，全部都是卷积层，没有向量。完全卷积神经网络的输出相对于简单的草皮土壤类别更为清晰，并且由自移动设备在工作期间可能遇到的不同类型的草或障碍物中的分段图像给出。
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。

Claims (8)

  1. 一种自移动设备的控制方法，所述自移动设备在工作区域内移动和工作，其特征在于，所述控制方法包括：
    获取所述自移动设备工作表面的数字图像；
    基于训练的神经网络处理所述数字图像，所述处理所述数字图像包括对所述数字图像进行图像分割，以获取待分析图像；
    基于所述待分析图像识别所述工作区域边界，控制所述自移动设备移动，使所述自移动设备与所述工作区域边界的距离满足预设条件，所述预设条件基于所述自移动设备的图像获取装置的安装位置和安装角度。
  2. 根据权利要求1所述的控制方法，其特征在于，控制所述自移动设备平行于所述工作区域边界移动，以切割所述工作区域边界。
  3. 根据权利要求1所述的控制方法，其特征在于，控制所述自移动设备在第K次沿所述工作区域边界移动时保持第一预设距离。
  4. 根据权利要求3所述的控制方法，其特征在于，在第K+1次沿所述工作区域边界移动时保持第二预设距离。
  5. 根据权利要求1所述的控制方法，其特征在于，在第K次沿所述工作区域边界移动时，控制所述自移动设备与所述工作区域边界的距离至少包括两个不同的预设距离。
  6. 一种自移动设备的控制方法，其特征在于，所述方法包括：
    获取所述自移动设备周围环境的数字图像；
    基于训练的神经网络处理所述数字图像，以获取待分析图像；
    基于所述待分析图像识别充电站，所述充电站供所述自移动设备停靠或充电；控制所述自移动设备朝向所述充电站移动。
  7. 根据权利要求6所述的控制方法，其特征在于，基于待分析图像中预设形状、预设标记、预设物体中的至少一种识别所述充电站。
  8. 根据权利要求6所述的控制方法，其特征在于，基于所述待分析图像生成所述自移动设备所在位置与所述充电站之间的最短路径，控制所述自移动设备朝向所述充电站移动。
PCT/CN2020/128396 2020-01-07 2020-11-12 自移动设备的控制方法 WO2021139397A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010014727.0A CN113156924A (zh) 2020-01-07 2020-01-07 自移动设备的控制方法
CN202010014727.0 2020-01-07

Publications (1)

Publication Number Publication Date
WO2021139397A1 true WO2021139397A1 (zh) 2021-07-15

Family

ID=76787730

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/128396 WO2021139397A1 (zh) 2020-01-07 2020-11-12 自移动设备的控制方法

Country Status (2)

Country Link
CN (1) CN113156924A (zh)
WO (1) WO2021139397A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023231022A1 (zh) * 2022-06-02 2023-12-07 深圳市正浩创新科技股份有限公司 图像识别方法、自移动设备及存储介质
CN117678412A (zh) * 2022-09-07 2024-03-12 苏州宝时得电动工具有限公司 自动割草机、自动割草机的控制方法及割草控制装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740792A (zh) * 2016-01-25 2016-07-06 浙江生辉照明有限公司 目标检测方法和装置
CN106558058A (zh) * 2016-11-29 2017-04-05 北京图森未来科技有限公司 分割模型训练方法、道路分割方法、车辆控制方法及装置
US20180035606A1 (en) * 2016-08-05 2018-02-08 Romello Burdoucci Smart Interactive and Autonomous Robotic Property Maintenance Apparatus, System, and Method
CN109631903A (zh) * 2019-01-04 2019-04-16 丰疆智慧农业股份有限公司 谷物处理自动驾驶系统及其自动驾驶方法和路径规划方法
CN109814551A (zh) * 2019-01-04 2019-05-28 丰疆智慧农业股份有限公司 谷物处理自动驾驶系统、自动驾驶方法以及自动识别方法
CN110334648A (zh) * 2019-07-02 2019-10-15 北京云迹科技有限公司 适用于机器人的充电桩识别系统和方法
CN110632915A (zh) * 2018-06-21 2019-12-31 科沃斯机器人股份有限公司 机器人回充路径规划方法、机器人及充电系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH03148708A (ja) * 1989-11-02 1991-06-25 Ishikawajima Shibaura Kikai Kk 自動走行作業車の操向制御装置
US6339735B1 (en) * 1998-12-29 2002-01-15 Friendly Robotics Ltd. Method for operating a robot
US7991521B2 (en) * 2006-02-01 2011-08-02 Jervis B. Webb Company Variable path automated guided vehicle
CN105785986A (zh) * 2014-12-23 2016-07-20 苏州宝时得电动工具有限公司 自动工作设备
CN109859158A (zh) * 2018-11-27 2019-06-07 邦鼓思电子科技（上海）有限公司 一种基于视觉的工作区域边界的检测系统、方法及机器设备
CN110018686A (zh) * 2019-03-26 2019-07-16 宁波大叶园林设备股份有限公司 一种智能割草机的路径规划方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740792A (zh) * 2016-01-25 2016-07-06 浙江生辉照明有限公司 目标检测方法和装置
US20180035606A1 (en) * 2016-08-05 2018-02-08 Romello Burdoucci Smart Interactive and Autonomous Robotic Property Maintenance Apparatus, System, and Method
CN106558058A (zh) * 2016-11-29 2017-04-05 北京图森未来科技有限公司 分割模型训练方法、道路分割方法、车辆控制方法及装置
CN110632915A (zh) * 2018-06-21 2019-12-31 科沃斯机器人股份有限公司 机器人回充路径规划方法、机器人及充电系统
CN109631903A (zh) * 2019-01-04 2019-04-16 丰疆智慧农业股份有限公司 谷物处理自动驾驶系统及其自动驾驶方法和路径规划方法
CN109814551A (zh) * 2019-01-04 2019-05-28 丰疆智慧农业股份有限公司 谷物处理自动驾驶系统、自动驾驶方法以及自动识别方法
CN110334648A (zh) * 2019-07-02 2019-10-15 北京云迹科技有限公司 适用于机器人的充电桩识别系统和方法

Also Published As

Publication number Publication date
CN113156924A (zh) 2021-07-23

Similar Documents

Publication Publication Date Title
WO2021139397A1 (zh) 自移动设备的控制方法
US20210136993A1 (en) Method for controlling a soil working means based on image processing and related system
WO2022021630A1 (zh) 自动行走设备及其控制方法和系统及可读存储介质
EP3438778A1 (en) Automatic working system, automatic traveling device and steering method thereof
WO2017198222A1 (zh) 自动工作系统，自移动设备及其控制方法
CN104111651A (zh) 自动行走设备及其向停靠站回归的方法
WO2021169192A1 (zh) 自动工作系统、自动行走设备及其控制方法及计算机可读存储介质
CN105785986A (zh) 自动工作设备
WO2021169190A1 (zh) 自动工作系统、自动行走设备及其控制方法及计算机可读存储介质
CN109247118A (zh) 基于全景摄像头构建电子地图的智能割草机
CN111353431A (zh) 自动工作系统、自动行走设备及其控制方法及计算机可读存储介质
CN107463167B (zh) 自动行走设备及目标区域识别方法
US11882787B1 (en) Automatic sensitivity adjustment for an autonomous mower
CN111413977A (zh) 一种基于机器视觉的割草机器人的路径规划系统及方法
CN113156929B (zh) 自移动设备
WO2021042487A1 (zh) 自动工作系统、自动行走设备及其控制方法及计算机可读存储介质
WO2021139683A1 (zh) 自移动设备
CN107561079B (zh) 一种基于结构光的草地识别系统
EP4123406A1 (en) Automatic working system, automatic walking device and method for controlling same, and computer-readable storage medium
CN204143255U (zh) 自动行走设备
WO2021042486A1 (zh) 自动工作系统、自动行走设备及其控制方法及计算机可读存储介质
WO2021031406A1 (zh) 自动工作系统、自动行走设备及其控制方法
WO2022223023A1 (zh) 自移动设备、移动轨迹调整方法及计算机可读存储介质
WO2022117001A1 (zh) 移动机器人智能避障方法及移动机器人
WO2022000878A1 (zh) 自动行走设备及其控制方法及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20912391

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20912391

Country of ref document: EP

Kind code of ref document: A1