JP2021040589A - Harvesting robot system - Google Patents

Abstract

To provide a harvesting robot system organically combining a cultivation management harvesting system capable of planning a best harvesting scheduling with a harvesting robot.SOLUTION: A harvesting robot system includes a harvesting robot 102 harvesting an object in a farm and a server 101 configured to be communicable with the harvesting robot 102. The server 101 is provided with: an input/output part 202 receiving input of a harvesting given condition including farm map information on a farm and harvesting determination reference information determining a period of harvesting of an object 306; a determination part 203 which determines an area to be harvested on the basis of an image of the object 306 photographed by the harvesting robot 102 every unit harvesting area in the farm and the harvesting given condition; and a first transmission/reception part 201 transmitting to the harvesting robot 102 harvesting instruction information including positional information of an area to be harvested.SELECTED DRAWING: Figure 1

Description

The present invention relates to a harvesting robot system that automatically harvests agricultural products including fruits such as apples, fruit vegetables such as tomatoes, and fruit vegetables such as strawberries based on cultivation management information.

As a future social issue in Japanese society, there is concern that the working-age population will decrease due to aging. Especially for agriculture, which is the primary industry, the environment surrounding Japan is extremely aging and is in a very difficult situation from the viewpoint of labor shortage. According to the basic data of the Ministry of Agriculture, Forestry and Fisheries, the number of key agricultural workers in 2014 was 1.68 million, a decrease of about 230,000 in five years. In addition, the average age is 66.5 years old, and the number of new farmers is decreasing, which highlights the labor shortage due to aging. As a result, the abandoned cultivated land has reached 400,000 ha, which adversely affects the local farming environment and living environment. Especially in rural areas where there are many cultivated lands, depopulation is progressing due to the declining birthrate and aging population, and this problem is becoming more apparent because there are no farmers.

Under such circumstances, expectations are rising for responding to labor shortages through agricultural automation. According to the Ministry of Economy, Trade and Industry's "2012 Robot Industry Market Trends," the domestic market for agricultural robots is expected to grow significantly from 2018 to 2024, reaching a scale of approximately 220 billion yen. In fact, developments that lead to the automation of agriculture, such as management using drones, automatic operation of tractors, and systems for navigating cultivation, are becoming active. In addition, the Ministry of Agriculture, Forestry and Fisheries has also enhanced the subsidy system to support the automation of agriculture, and the government has high expectations for this field.

Among them, research on harvest automation technology has been advanced in companies and universities, and many harvesting robots that automatically harvest various fruits, fruit vegetables, and fruits and vegetables have been proposed. .. However, a proposal regarding a cultivation management system and a harvesting robot system in which the harvesting robot is linked, how to automatically obtain cultivation status information in the farm and how the harvesting robot determines the area to be harvested. Is few.

Among them, as a conventional system for collecting and managing cultivation status information, there is a system in which a plurality of monitoring devices and one server are installed in a remote place in a farm and connected to them via the Internet (for example, patent documents). See 1.). In the conventional self-sustaining operation control system for collecting cultivation status information in a farm described in Patent Document 1, the monitoring device has an imaging unit and captures an image of a predetermined crop. The server is connected to the plurality of monitoring devices via the Internet, and the operator can view images of agricultural products in remote areas via terminals.

Patent No. 4586182

However, in the configuration of the conventional Patent Document 1, since the monitoring device is fixedly installed, the imaging location is fixed, and the number of monitoring devices is limited, so that the entire farm cannot be covered. Therefore, the maturity status information of the crop is fragmented. Therefore, there is a problem that the optimum harvest schedule cannot be set for the farm as a whole from limited information. In addition, if a wide range can be photographed, the details of the crop cannot be known due to the limitation of the resolution of the imaging unit. In particular, there is a problem that the degree of maturity of small individual fruits cannot be determined and the purpose cannot be achieved in agricultural products in which the fruits connected to one main stem or one bunch mature in stages and the harvest time is different. .. Furthermore, the configuration of the conventional Patent Document 1 has only a monitoring function, and no cooperation with the harvesting robot is made. That is, it is difficult to operate the harvesting robot efficiently.

The present invention solves the above-mentioned conventional problems, and is a cultivation management system that can set an optimum harvest schedule by covering the entire farm with a harvesting robot and obtaining detailed maturity status information of individual crops. It is an object of the present invention to provide a harvesting robot system in which a harvesting robot is efficiently operated by combining with and.

The harvesting robot system of the present invention for achieving the above object is
A harvesting robot system that includes a harvesting robot that harvests objects in a farm and a server that is configured to communicate with the harvesting robot.
The server
An input / output unit that accepts input of harvesting conditions including farm map information of the farm and harvest judgment standard information for determining the harvest time of the object.
A determination unit that determines the harvest target area based on the image of the object taken by the harvest robot for each unit harvest area in the farm and the harvest conditions.
A first transmission / reception unit that transmits harvest instruction information including the position information of the harvest target area to the harvest robot, and
It is a harvesting robot system equipped with.

As described above, according to the harvesting robot system of the present invention, the server can set the optimum harvesting schedule based on the necessary and sufficient information on the cultivation status of the crops, and the harvesting robot receiving the instruction is the most efficient. Harvesting work can be carried out in a targeted manner.

Overall system configuration diagram of the harvesting robot system according to the embodiment of the present invention Block diagram of the server according to the embodiment of the present invention Configuration diagram of the harvesting robot according to the embodiment of the present invention Configuration diagram of the recovery robot according to the embodiment of the present invention Block diagram of a pickup station according to an embodiment of the present invention Block diagram of a home station according to an embodiment of the present invention Schematic layout of the entire farm according to the embodiment of the present invention Harvesting / collecting / storage basket supply basic operation flow chart according to the embodiment of the present invention Tool change operation flow chart according to the embodiment of the present invention Charging operation flow chart according to the embodiment of the present invention Harvest target area determination flow chart in the embodiment of the present invention Photographing rotation operation flow chart according to the embodiment of the present invention Learning function application maturity determination flow chart in embodiment of the present invention Flow diagram at the time of manual programming in the embodiment of the present invention Harvest prediction management processing flow chart according to the embodiment of the present invention Real-time feedback processing flow chart according to the embodiment of the present invention Cultivation management processing flow chart in embodiment of this invention Other farm cooperation system operation flow chart in embodiment of this invention

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Embodiment)
FIG. 1 is an overall system configuration diagram of the harvesting robot system according to the embodiment of the present invention.

In FIG. 1, the server 101 is input / output by a plurality of harvesting robots 102, one or more collection robots 103, a collection station 104, a home station 105 in which the harvesting robot 102 or the collection robot 103 stands by, and an operator 106. The harvesting robot system is organically related to the input / output means 107 for performing the above and the Internet 109 which can be connected to the server 108 of another farm.

FIG. 2 is a block diagram of the server 101.

As shown in FIG. 2, the server 101 transmits and receives information to the harvesting robot 102 and the collecting robot 103 by wireless LAN or the like, the first transmission / reception unit 201, the collection station 104, the home station 105, and the input / output means. Based on the input / output unit 202 that outputs or inputs information to the 107 and the server 108 of another farm via the Internet 109 by wire or wirelessly, the image data with the map address, and the harvesting conditions, the harvest target area is determined. Judgment unit 203 that determines, determines the collection order and collection route of stored storage baskets, and determines the empty storage cage supply and allocation plan, and a large-capacity storage device 204 that can store a large amount of information including image data with map addresses. And consists of.

FIG. 3 is a block diagram of the harvesting robot 102 according to the embodiment of the present invention.

As shown in FIG. 3, the harvesting robot 102 has a photographing unit 301 that photographs an object 306 and a storage basket with a camera, and a first self that identifies its own position by GPS, a laser radar, a white line marker, a landscape recognition collation, or the like. The position detection means 302, the second transmission / reception unit 303 that sends information including image data with a map address to the server by wireless LAN or the like, and receives instruction information from the server, and the object 306 are collected by the manipulator and stored in the storage basket 307. The harvesting section 304 to be stored in, the first lifter mechanism 305 that grips and lifts the lowermost storage basket 307 of the stacked storage basket 307 placed on the ground, and the second stage from the bottom of the stacked storage basket 307. It is composed of a third lifter mechanism 309 that sequentially grips and lifts and separates the empty storage basket 307 above, and a self-propelled trolley unit 308 that is equipped with all the above-mentioned devices and can automatically travel on the ridge 705 and the main passage 703.

It is desirable that the photographing unit 301 is a stereo camera. A normal two-dimensional area camera may be used if it is used only for acquiring image data with a map address for determining a harvest target area, but in the embodiment of the present invention, the photographing unit 301 is used for harvesting work and storage cage 307. Also used when picking up. The distance information to the object 306 is required for harvesting, and the distance information to the storage basket 307 is required to pick up the storage basket 307. If a normal two-dimensional area camera is used, a distance measuring device such as a laser sensor is required separately.

The form of the harvesting unit 304 differs depending on the type of the object 306 to be harvested. Since various harvesting robots, harvesting manipulators, and harvesting hands have been proposed in the past, detailed description thereof will be omitted. FIG. 3 shows an example in which the harvesting portion 304 has a twin-armed articulated manipulator attached to a rotatable body portion and a harvesting hand attached to the tip of the manipulator. As the harvesting hand, the most suitable harvesting hand is selected according to the type, shape, size, etc. of the object 306, and the harvesting hand is appropriately attached. In FIG. 3, the manipulator that collects the object 306 directly stores the object 306 in the storage basket 307. However, the manipulator that collects the object 306 and the storage mechanism that stores the object 306 in the storage basket 307 are provided. It may be divided into. In that case, the manipulator to be collected and the storage mechanism are combined to form the harvesting unit 304.

The storage basket 307 is, for example, a so-called container-type box that is made of plastic and can be stacked with objects stored therein.

The automatic guided vehicle unit 308 is generally called an AGV (Automatic Guided Vehicle), and detailed description thereof will be omitted. Although FIG. 3 shows an example of traveling on wheels, a crawler or the like may be adopted depending on the characteristics of the farm.

FIG. 4 is a configuration diagram of the recovery robot 103 according to the embodiment of the present invention.

As shown in FIG. 4, the recovery robot 103 includes a second lifter mechanism 401 having the same function as the first lifter mechanism 305, a second self-position detecting means 402 having the same function as the first self-position detecting means 302, and a second self-position detecting means 402. 2. It is composed of a third transmission / reception unit 403 having the same function as the transmission / reception unit 303, and a self-propelled trolley unit 404 that is equipped with all the above devices and can automatically travel in the main passage 703 (see FIG. 7).

FIG. 5 is a block diagram of the pickup station 104 according to the embodiment of the present invention.

As shown in FIG. 5, the collection station 104 receives a plurality of harvested storage baskets 307 from the collection robot 103, and receives, for example, a harvested storage basket stock unit 501 that can be stocked on a roller conveyor and a plurality of empty storage baskets 307. For example, it is composed of an empty storage basket stock unit 502 that can be stocked on a roller conveyor and can deliver an empty storage basket 307 to the collection robot 103 at the request of the server 101 or the collection robot 103.

The collection station 104 transmits the empty space information of the harvested storage basket stock unit 501 to the server 101, and transmits the inventory information including the inventory amount of the empty storage basket existing in the empty storage basket stock unit 502 to the server 101. It also has a fourth transmitter / receiver (not shown).

FIG. 6 is a block diagram of the home station 105 according to the embodiment of the present invention.

The home station 105 is a place where the harvesting robot 102 or the collecting robot 103 is moored when the harvesting work or the collecting work is not performed, and is generally called a dock, and parking spaces for a plurality of cars are prepared.

As shown in FIG. 6, the home station 105 replaces the tool at the tip of the manipulator for harvesting the harvesting unit 304 of the harvesting robot 102 with a tool for harvesting other types of objects or a tool used for other work. It is composed of a tool change unit 601 for charging, a charging unit 602 for charging the rechargeable battery for driving the harvesting robot 102 or the collecting robot 103, and a fifth transmitting / receiving unit (not shown) for transmitting / receiving data to / from the server 101. To.

FIG. 7 is a schematic layout diagram of a typical farm as a whole in the embodiment of the present invention. In FIG. 7, the same reference numerals are used for the same components as those in FIGS. 1 and 5, and the description thereof will be omitted.

As shown in FIG. 7, the farm is largely composed of a cultivation yard 701 for cultivating crops and a sorting and shipping yard 702 for sorting and shipping the harvested crops. The cultivation yard 701 may be an open field or a greenhouse such as a greenhouse. Further, the cultivation yard 701 and the sorting and shipping yard 702 may be directly connected or may be connected by an aisle.

In the center of the cultivation yard 701, there is a relatively wide main passage 703 in which the harvesting robot 102 and the collecting robot 103 run. The main aisle 703 is directly connected to the sorting and shipping yard 702. Several ridges 704 on which agricultural products such as tomatoes and strawberries are cultivated are vertically arranged on both sides of the main passage 703. The ridges 704 may be literally raised ridges, a cultivation box for elevated cultivation found in Dutch Venlo-type facility cultivation, or a row of standing fruit trees. The area where the object 306 to be cultivated and harvested in this ridge 704 exists is the harvest area. There is a ridge 705 between the ridges 704 and 704 so that the harvesting robot 102 can pass through. The harvesting robot 102 can harvest the object 306 from the ridges 704 on both sides. The ridge 705 may be laid with a hot water pipe on the soil surface or in the case of a greenhouse. When the hot water pipe is laid, the harvesting robot 102 can use the hot water pipe as a track for running on the ridge. As shown in FIG. 7, the end opposite to the main passage 703 of the ridge 705 may be a dead end. After harvesting, the harvesting robot 102 turns back to the ridge 705, goes out to the main passage 703, lowers the harvested storage basket 307, and heads for the next ridge 705. At this time, the harvesting robot 102 is efficient if, for example, the outward route is the harvest of the ridge 704 on the right side and the return route is the harvest of the ridge 704 on the left side. By the way, many farms have a dead end on the opposite side of the ridge as shown in FIG. This is to increase the area productivity of the cultivation yard 701.

The storage basket 307 or the empty storage basket 307 containing the harvested object 306 is temporarily placed at a position that does not interfere with other work of the main aisle 703, for example, at a ridge on the side of the main aisle 703.

The harvested storage basket 307 temporarily placed on the side of the main passage 703 by the harvesting robot 102 is collected by the collection robot 103, carried to the harvested storage basket stock section 501 of the collection station 104, and stocked. The harvested storage basket stock unit 501 stocks the storage basket 307 and conveys the storage basket 307 to the vicinity of the sorting and bagging worker 706 by, for example, a roller conveyor. There is a step-off unit 708 on the way, and after separating the stacked storage baskets 307 one by one, they are transported to the sorting and bagging worker 706.

The sorting bagging worker 706 takes out the object 306 from the storage basket 307, sorts it, stores it in a bag according to a predetermined procedure, and puts it on the shipping conveyor 707. The bag containing the object 306 is transported to the next process by a conveyor, and then the bag is packed in a box and shipped.

On the other hand, the empty storage basket 307 is placed by the sorting bag packing worker 706 on the conveyor connected to the empty storage basket stock section 502. The empty storage basket 307 is conveyed to the empty storage basket stock section 502 by a conveyor, but there is a stacking unit 709 in the middle, and the empty storage basket 307 is stacked in a predetermined number of stages, and then the empty storage basket stock section 502. Transported to and stocked. After that, according to the request from the server 101, the empty storage basket 307 is transported to the cultivation yard 701 by the recovery robot 103 and placed in a designated place beside the main aisle 703. The harvesting robot 102 picks up the placed empty harvest basket 307 and heads for harvesting. In this way, the circulation distribution system of the storage basket 307 functions.

The server 101 and the input / output means 107 are installed at appropriate locations in the farm.

The home station 105 is installed in a space close to the main passage 703, and a plurality of harvesting robots 102 or recovery robots 103 can be moored. Tools are replaced and rechargeable batteries are charged at the moored area.

Since a plurality of harvesting robots 102 and recovery robots 103 come and go in the main passage 703, it is advisable to determine the basic traveling route and rules in the main passage 703. For example, the main passage 703 is divided into two lanes to limit the direction of travel. That is, the outward route and the return route are completely separated so that they can always pass each other. The basic travel route of the recovery robot 103 is folded back at the end of the main aisle 703, and is connected to the main aisle 703 → harvested storage basket stock section 501 → empty storage basket stock section 502 → main aisle 703 in the sorting and shipping yard 702 to form a closed circuit. It is desirable to do. The plurality of recovery robots 103 circulate in one direction along the closed circuit. The travel route may be indicated by a white line, a magnetic tape may be attached, or may be defined by the farm map information.

Of course, the recovery robot 103, which has completed the installation of the empty storage basket 307 and the collection of the harvested storage basket 307 near the sorting and shipping yard 702, should immediately head to the collection station 104 without tracing the closed circuit until the end. Shortcut. This route instruction is also issued from the server 101.

The form of the farm is not limited to the form illustrated in FIG. 7. The collection robot 103 collects with the harvesting robot 102 on the main passage 703 in a form in which the collection robot 103 comes out from the ridge to the main passage 703 after the harvesting robot 102 finishes harvesting in the ridges and the main passage 703 where the collection robot 103 can go to and from the collection station 104. All that is required is that there is a contact point that the robot 103 can share.

Further, the form of the harvested storage basket stock section 501 and the empty storage basket stock section 502 of the collection station 104 is a roller conveyor, but this is not particular. For example, it may be a simple space in which each storage basket 307 is placed. In that case, the sorting and bagging worker 706 goes to the space to pick up the storage basket, and after the work is completed, returns it to the empty storage basket stock space. The recovery robot 103 detects the position of an empty storage basket in a predetermined space by, for example, a photographing device, and picks it up by the second lifter mechanism 401.

Further, in the embodiment of the present invention shown in FIG. 7, the sorting and shipping work is performed by the worker, but an automatic sorting and bagging machine may be used. By the way, automation of sorting and shipping work has been realized for some fruits. In addition to the present embodiment, these automatic sorting and bagging machines and an automatic boxing machine are introduced and connected to the server 101 as a shipping system, and the server 101 manages the shipping plan in addition to the harvesting plan and the collection plan. By doing so, an automatic harvesting and shipping system can be constructed, and a series of unmanned harvesting operations will be further promoted.

Further, when the sorting and bagging worker 706 operates only in the daytime, while the harvesting robot 102 and the collecting robot 103 operate both day and night, a shift difference occurs. Therefore, the harvested storage basket stock section 501 and the empty storage basket stock The unit 502 needs a conveyor length capable of stocking the number of storage baskets sufficient to absorb the shift difference. A larger amount of storage basket 307 is required. As described above, if all the operations including sorting and shipping are automated, the shift difference is eliminated and the operation can be performed with a shorter conveyor length and the number of storage cages.

An operation example of the harvesting robot system configured as described above will be described with reference to FIG.

FIG. 8 is a basic operation flow diagram of harvesting / collecting / empty storage basket supply according to the embodiment of the present invention.

As illustrated in FIG. 8, the server 101 first accepts the input of the harvesting conditions (S1). The harvesting condition is input to the input / output unit 202 by the operator 106 operating the input / output means 107.

The "harvesting condition" input by the operator 106 is information to be given in advance for the determination unit 203 to determine the harvest target area, and is the farm map information and the harvest determination standard for determining the harvest time of the object. Contains information. Here, the harvest determination standard information is information regarding a threshold value for determining whether or not the object 306 should be harvested from the image of the object 306 captured by the harvesting robot 102. Further, the farm map information is, for example, information on the cultivation yard 701, the travelable area, the position of the collection station 104, the position of the home station 105, and the like in the farm.

In the farm map information, the cultivation yard 701 is divided into a plurality of unit harvest areas, and address information is attached to each unit harvest area and stored. Then, in the harvesting robot system according to the present embodiment, the harvesting target area is determined for each unit harvesting area.

Next, the determination unit 203 determines the route to be photographed by the harvesting robot 102 based on the farm map information and the like (S2). Next, the server 101 outputs the shooting instruction information including the shooting route to the harvesting robot 102 through the first transmission / reception unit 201.

The harvesting robot 102, which has been input with the shooting instruction information through the second transmission / reception unit 303, shoots the object 306 in the farm using the shooting unit 301 while moving on the self-propelled bogie unit 308 according to the instruction (R1). The self-position of the harvesting robot 102 at the time of photographing is acquired by the first self-position detecting means 302. The own position may be the farm coordinate data or the address code defined in the farm map. When the photographing unit 301 is not a fixed camera but a camera whose orientation can be freely changed, the image data with a map address includes the own position in the farm and the posture information of the camera. That is, it is necessary to specify the absolute position of the object 306 reflected in the image data in the cultivation yard 701. The harvesting robot 102 transmits the acquired image data with a map address to the server 101 via the second transmission / reception unit 303.

The "image data with a map address" is an image of an object 306 existing in the unit harvesting area taken by the harvesting robot 102 for each unit harvesting area in the farm. The “image data with map address” is typically photographed so as to show the entire unit harvest area so that the maturity of the object 306 existing in the unit harvest area can be grasped from the image (FIG. 12). See below). However, the maturity of the object 306 existing in the entire unit harvesting area may be grasped by a plurality of images.

The "image data with map address" includes, for example, the position where the image was taken (for example, the own position of the harvesting robot 102) and the direction in which the image was taken (for example, the posture of the harvesting robot 102) in association with the image. ) Is stored. The position information stored in association with the image may be the position information related to the position where the image was taken and the position of the shooting target specified based on the direction in which the image was taken.

The determination unit 203 of the server 101, which receives the image data with the map address in the farm from the harvesting robot 102 by the first transmission / reception unit 201, is the most efficient, that is, the most mature object based on the harvesting conditions input in advance. The area with many 306s is determined as the harvest target area (S3). Specifically, the number of the ridge 705 facing the determined harvest target area is assigned to each harvesting robot 102 in consideration of the harvesting ability of the harvesting robot 102. Basically, one harvesting robot 102 is assigned to one ridge 705. In principle, only one harvesting robot 102 is allowed to enter one ridge 705 at a certain moment. The harvesting robot 102 may be assigned a plurality of ridges 705. In addition, all the ridges 704 on both sides of the ridge 705 may be the harvest target area, or only one part of the ridge 704 on one side may be the harvest target area. These are the harvest instruction information.

The first transmission / reception unit 201 of the server 101 transmits the harvest instruction information including the harvest target area to the harvest robot 102.

In the present embodiment, as described above, the harvest target area to be harvested by the harvesting robot 102 is determined by the determination unit 203 of the server 101 based on the image data with the map address of the harvested product and the harvesting conditions. Visually confirms the growing condition of the harvested object in the farm, determines the harvested area based on the conventional experience, and determines the number of the ridge 705 and the harvested section to be harvested by each harvesting robot 102 through the input / output means 107. It may be directly input as harvest instruction information.

The harvesting robot 102 enters the ridge 705 designated by the self-propelled trolley unit 308 based on the harvest instruction information received by the second transmission / reception unit 303, and makes full use of the harvesting unit 304 to obtain the object 306 in the harvest target area. Harvest and store in storage basket 307 (R2). In the harvesting robot 102, it is assumed that the first lifter mechanism 305 holds the lowermost storage basket 307 of the empty storage baskets 307 laminated in advance. This inherits the state of R5, which will be described later. When the harvesting robot 102 receives the harvesting instruction information, the third lifter mechanism 309 grips and lifts the second-stage storage basket 307 from the bottom from that state. Then, a space is formed above the storage basket 307 in the lowermost layer. Utilizing this space, the harvesting unit 304 stores the object 306 in the storage basket 307 of the lowermost layer. When the lowermost storage basket 307 is full, the third lifter mechanism 309 is lowered, and once all the storage baskets 307 are stacked, the grip of the second storage basket 307 from the bottom is released, and the height of the storage basket 307 is released. It rises by one piece, grips the third storage basket 307 from the bottom, and rises. As a result, the harvesting unit 304 resumes the harvesting operation using the space formed above the storage basket 307 in the second stage from the bottom, and the object 306 is stored in the storage basket 307 in the second stage from the bottom. .. FIG. 3 shows the state at this time. Hereinafter, the harvesting robot 102, which repeats this operation and fills all the mounted storage baskets 307 with the object 306, or completes the planned harvesting of the harvesting target area, is transferred from the second transmission / reception unit 303 to the server 101. The harvest status information including the harvest completion signal and the current position information are transmitted to.

The server 101, which has received the harvest status information including the harvest completion signal in the first transmission / reception unit 201, heads to the next harvest target area if the storage basket 307 mounted on the server 101 still has free space. Is given to the harvesting robot 102. If all of the loaded harvest baskets 307 are full of the object 306, information for collecting the storage baskets is collected (S4). The operator 106 has already input to the input / output unit 202 through the input / output means 107 in advance the collection conditions necessary for making the collection plan of the harvested storage basket 307. The collection condition is the farm map information including the position of the collection station 104, and the collection robot specifications including the number of the collection robots 103 in operation, the traveling speed, and the like. From the collection station 104, the vacant space information of the harvested storage basket stock unit 501 is obtained through the input / output unit 202, and it is confirmed whether or not the collection station 104 has room to accept the harvested storage basket 307. However, the harvested storage basket stock section 501 is designed to have ample space at all times, and in the worst case, the recovery robot 103 may wait for the harvested storage basket 307 to be loaded and unloaded in front of the collection station 104. This free space information is not needed.

Next, the server 101 transmits a request signal from the first transmission / reception unit 201 to the recovery robot 103 in order to grasp the status of the recovery robot 103.

The collection robot 103 that received the request signal in the third transmission / reception unit 403 detects the own position by the second own position detecting means 402, and the own position data, the presence / absence of the current storage basket loading, and the work content currently being executed. For example, the third transmitter / receiver 403 transmits the loading status information such as whether the battery is being charged at the home station 105, whether it has been harvested or empty, and which storage basket 307 is on the way to somewhere. (K1).

In this way, the server 101, which has obtained all the information necessary for the collection work of the harvested storage basket 307, determines the position of the harvesting robot 102 that has completed the harvesting work by the judgment unit 203 based on the information. The basket 307 is unloaded, and which collection robot 103 determines which route the storage basket 307 placed by the harvesting robot 102 is to be collected and which route is taken to the collection station 104 (S5). These are the collection instruction information, and the first transmission / reception unit 201 outputs the collection instruction information to the harvesting robot 102.

The harvesting robot 102 that received the collection instruction information from the server 101 in the second transmission / reception unit 303 moves to the position instructed by the collection instruction information by the self-propelled bogie unit 308, and when it arrives, lowers the first lifter mechanism 305. The harvested storage basket 307 is lowered to the ground (R3). When the harvesting robot 102 transmits the fact that the storage basket 307 has been lowered from the second transmitting / receiving unit 303 to the server 101 (harvest status information including the completion signal and the current position information), the server 101 sends the first transmitting / receiving unit 201 to the collecting robot 103. Send the collection instruction information (S6). Based on the collection instruction information received by the third transmission / reception unit 403, the collection robot 103 heads to the instructed place by the self-propelled bogie unit 404, raises the harvested storage basket 307 placed on the ground, and raises the second lifter mechanism 401. Let me pick it up (K2).

Here, the collection instruction information transmitted to the harvesting robot and the collection instruction information transmitted to the collection robot are the same information, but the collection instruction information transmitted to the harvesting robot includes the full storage basket itself. It suffices to include information on the placement position to be placed separately from the harvesting robot, and the collection instruction information sent to the collection robot is a collection route for collecting the storage basket on which the harvesting robot is placed and transporting it to the collection station. It suffices if the information of is included.

After that, the collection robot 103 heads for the collection station 104 (K3) through the route instructed by the server 101 by the self-propelled bogie unit 404, and puts the storage basket 307 in the harvested storage basket stock unit 501 of the collection station 104 and the second lifter. The mechanism 401 is lowered and lowered (K4).

Next, the third transmission / reception unit 403 of the recovery robot 103 transmits the loading status information including the completion signal and the current position information to the server 101. The server 101, which receives the loading status information including the completion signal of the collection robot 103 in the first transmission / reception unit 201, obtains the inventory information of the empty storage basket stock unit 502 of the collection station 104 (S7). At this time, if there is a sufficient distribution amount of storage baskets and a sufficient stock capacity of empty storage baskets, and in the worst case, the collection robot 103 can be allowed to stand by in front of the collection station 104, this inventory information can be obtained. Can be omitted.

If the empty storage basket is in stock, the determination unit 203 determines the empty storage basket supply / dispatch plan for loading the empty storage basket 307 and directing the collection robot 103 to (S8). These are the storage basket supply instruction information, and the server 101 transmits the storage basket supply instruction information from the first transmission / reception unit 201 to the collection robot 103. The collection robot 103 that received the storage basket supply instruction information in the third transmission / reception unit 403 goes to the empty storage basket stock unit 502 of the collection station 104 by the self-propelled trolley unit 404, and lifts the empty storage basket 307 by the second lifter mechanism 401. , Installed (K5).

After that, the recovery robot 103 moves to the place instructed by the server 101 by the self-propelled bogie unit 404, and when it arrives, the second lifter mechanism 401 descends and the empty storage basket 307 is lowered (K6). The recovery robot 103 transmits the loading status information including the completion signal and the current position information from the third transmission / reception unit 403 to the server 101.

The server 101 receives the loading status information including the completion signal and the current position information in the first transmission / reception unit 201, and the first transmission / reception unit 201 transmits the storage basket supply instruction information to the harvesting robot 102 (S9).

The harvesting robot 102 that received the storage basket supply instruction information in the second transmission / reception unit 303 moves to the designated place by the self-propelled bogie unit 308, and lifts the empty storage basket 307 placed on the ground by the first lifter mechanism 305. Hold (R4). The harvesting robot 102 waits until the next instruction is given from the server 101 (R5). This state is a state at the time of photographing the harvesting area of R1 or immediately before moving to the harvesting / storing operation of R2. After that, this is repeated.

The above operation flow is a basic pattern, and in reality, the plurality of harvesting robots 102 and the plurality of collection robots 103 operate in a complex manner according to the harvesting situation, and are operated so as to operate efficiently as a whole.

For example, in the operation flow diagram of FIG. 8, the operations of one harvesting robot 102 and one collecting robot 103 are shown separately for the basic harvesting flow, the basic collecting flow, and the basic empty storage basket supply flow. , Actually, it may be operated as one. That is, the server 101 constantly grasps the statuses of the plurality of harvesting robots 102 and the recovery robot 103 to determine the harvest target area, the collection order and the collection route, and the empty storage basket supply / dispatch plan at the same time. Do. For example, if the point where the harvested storage basket 307 should be collected at a certain point and the point where the empty harvesting basket 307 should be supplied are relatively close to each other, the collection robot 103 supplies the empty storage basket 307 on the outbound route from the collection station 104. However, if the harvested storage basket 307 is collected on the return route to the collection station 104, the collection robot 103 has almost no section in which it travels with no load, and the collection robot 103 does not have a wasteful waiting time. Similarly, after unloading the harvested storage basket 307, the harvesting robot 102 immediately heads to the nearby empty storage basket mounting point, picks up the storage basket 307, and ridges 705 of the next harvest target area instructed by the server 101. If the harvesting work is started by entering the harvesting robot 102, the harvesting robot 102 does not have to wait a long time.

Since the harvesting robot 102 and the collecting robot 103 are not paired and the storage basket 307 is once placed on the ground, the harvesting robot 102 and the collecting robot 103 do not have to synchronize the delivery operation of the storage basket 307. Further, since there may be a considerable time lag between the placement of the harvested storage basket 307 of the harvesting robot 102 and the picking up by the collecting robot 103, the server 101 can develop such efficient operation programming. As a result, the minimum number of recovery robots 103 can be used for the number of harvesting robots 102. This is because the harvesting span from the start of harvesting of the harvesting robot 102 to the completion of harvesting is basically long, while the collection and supply span from the collection of the harvested storage basket 307 of the recovery robot 103 to the installation of the empty storage basket 307 is short. ..

In this harvesting robot system, the harvesting plan, the collecting plan, and the storage cage supply plan are optimally programmed by the server 101 so that the harvesting robot 102 and the collecting robot 103 do not wait. Game theory, queuing theory, etc. may be applied to the process.

In addition, AI or the like may be applied to make some predictions in advance and program the harvest schedule. However, unlike industrial products, fruits do not grow uniformly and uniformly, and fluctuate greatly depending on the weather and other factors. It is possible to make a prediction first, but the prediction is wrong, and it is necessary to have a feedback mechanism that constantly changes the program flexibly with the latest information.

Further, as shown in FIG. 3, when the harvesting robot 102 has the third lifter mechanism 309, a plurality of storage baskets 307 can be mounted on the harvesting robot 102, and more objects 306 are harvested at one time. Can be stored. That is, the collection lot size by the collection robot 103 increases, and the collection frequency of the storage basket 307 of the collection robot 103 decreases. That is, the number of required harvesting robots 103 is reduced. If possible, the stock amount is preferably larger than the yield amount per ridge 705.

In that case, the position where the storage baskets 307 stacked so as to correspond one-to-one with each of the ridges 705 can be determined in advance. Further, in that case, the storage basket 307 may not be placed directly on the ground, but a dedicated fixedly installed storage basket stand may be provided. The fixed storage basket stand makes it possible to reliably and easily deliver the storage basket 307 to and from each robot. In either case, if it is determined that the existence of an empty storage basket 307 at that location is a steady state, management is easy and more efficient operation is possible. That is, as soon as the harvest target area, that is, the ridge 704 is determined, the harvesting robot 102 picks up the empty storage basket 307 at the position corresponding to the ridge 704, enters the ridge 705, harvests, and finishes the harvest. After placing the harvested storage basket 307 in the same position as it was originally and sending a harvest completion signal to the server 101, a new harvest instruction was received from the haircut side dish server 101 and it was paired with the next harvest target area. Head to the place where the empty storage basket 307 is placed. The recovery robot 103 places an empty storage basket 307 in a fixed position where the nearby harvested storage basket 307 has been collected and becomes empty, and picks up the harvested storage basket 307 on the return route in response to a collection instruction from the server 101. I'll give you. After that, after issuing a collection signal to the server 101, it heads for the collection station 104. After that, this is repeated.

As a mechanism in which the harvesting robot 102 mounts a plurality of storage baskets 307 and sequentially stores them in each storage basket 307, the first lifter mechanism 305 and the third lifter mechanism 309 are coaxial in the vertical direction as shown in FIG. Although the structure in which the harvested object is stored by utilizing the gap created by lifting the empty storage basket 307 that is arranged and the third lifter mechanism 309 remains in sequence is illustrated, this is not particular. In short, the structure may be such that a plurality of storage baskets 307 can be mounted, the object 306 can be stored in each of them, and the storage baskets 307 stacked from the ground or a fixed stand can be loaded and unloaded.

Further, in an extreme case, the storage baskets 307 are not laminated, and only one storage basket 307 may be used. In that case, the third lifter mechanism 309 is unnecessary. However, since the harvesting span is shortened, the recovery efficiency is deteriorated, and the number of required recovery robots 103 is increased. Of course, if the ridge length is short and the object 306 to be harvested is a small fruit such as cherry, one storage basket can be used for each ridge, so this is not the case.

Next, the option group in the present embodiment will be sequentially described with reference to FIGS. 9 to 18. These option groups more effectively improve, reinforce or supplement the function as a harvesting robot system based on the basic operation flow of harvesting / collecting / storing basket supply in FIG. 8 above. The option group may be appropriately combined with the embodiment as long as there is no contradiction.

FIG. 9 is a flow chart of a tool change operation according to the embodiment of the present invention.

When a plurality of types of crops are cultivated in the cultivation yard 701, the harvesting robot 102 generally attaches a dedicated harvesting hand according to the characteristics of each crop to the tip of the manipulator of the harvesting unit 304 to perform the harvesting operation. Various harvesting hands form a part of the work tool group, and the exchange is generally called tool change. When this option is applied, the manipulator and the harvesting hand at the tip of the harvesting unit 304 of the harvesting robot 102 have a structure in which the harvesting hand can be automatically replaced by a mechanism called an auto tool changer of an industrial robot, for example. A detailed description of the auto tool changer will be omitted.

In the tool change operation flow, as illustrated in FIG. 9, the server 101 receives the input of the tool change giving condition in advance (S101). The tool change condition is input to the input / output unit 202 by the operator 106 operating the input / output means 107. The tool change giving condition input by the operator 106 is information to be given in advance for the determination unit 203 to determine the tool exchange, for example, planting information, tool code information, and the like. The planting information is linked with the farm map information to specify the position of the harvest area for each variety of the object 306 cultivated in the cultivation yard 701, that is, the ridge number, and the type of harvesting hand to be applied for each variety. It is specified in the tool code. The tool code information is a code number assigned to each type of harvesting hand, and using this tool code, the server 101 constitutes tool change information as to which harvesting hand the harvesting robot 102 wears.

In the state where the tool change giving condition is input, the determination unit 203 of the server 101 in which the image data with the map address is input from step R1 in FIG. 8 determines the harvest target area. At the same time, the tool code of the harvesting hand corresponding to the type of the object 306 determined in the harvesting target area is determined (S102). On the other hand, the server 101 obtains the robot state information including the tool state from the harvesting robot 102 at the same time as the image data with the map address. If the harvesting hand worn by the current harvesting robot 102 and the tool code of the determined harvesting hand match (Y in S103), the harvesting instruction information is displayed according to the basic operation flow of harvesting / collecting / storing basket supply in FIG. The harvesting robot 102 harvests and stores (R2). If the tool codes do not match (N in S103), the server 101 outputs a request signal to the home station 105, and inquires the home station 105 of the availability of the standby position and the inventory status of the tool including the harvesting hand. The home station 105 returns vacant position information, that is, which dock is currently vacant, and tool inventory information, that is, how many tools of which code are left (H101). The determination unit 203 of the server 101, which has confirmed the availability of the standby position of the home station 105 and the inventory of the necessary tools, decides to replace the tools (S104). The server 101 includes the tool change information, and outputs the instruction information for the home station to which dock to go to the harvesting robot 102. Upon receiving this, the harvesting robot 102 moves to the home station 105 and stores it in the designated dock (R101). After arriving at the home station 105, the harvesting robot 102 transmits an arrival signal to the server 101. Upon receiving this, the server 101 transmits the tool change information to the home station 105 (S105). The received home station 105 is linked with the harvesting robot 102 and exchanged for a designated tool (H102). When the home station 105 notifies the server 101 of the completion of the exchange as a tool change completion signal, the server 101 transmits the harvest instruction information to the harvest robot 102 (S106). After that, the process returns to the basic operation flow of harvesting / collecting / storing basket supply shown in FIG.

The tool change H102 at the home station 105 may be based on the tool change information from the harvesting robot 102, not the tool change information from the server 101. However, the home station 105 needs a transmission / reception unit capable of directly transmitting / receiving information to / from the harvesting robot 102 via a wireless LAN or the like, or a structure capable of exchanging information with contacts. Similarly, the tool change completion signal may be transmitted from the harvesting robot 102 to the server 101.

FIG. 10 is a flow chart of a charging operation according to the embodiment of the present invention. The harvesting robot 102 and the recovery robot 103 have self-propelled trolleys 308 and 404, and are equipped with a rechargeable battery such as a lead storage battery or a lithium ion secondary battery as a power source. These are used not only as a power source for self-propelling, but also as another power source such as a harvest manipulator, and need to be charged regularly. Since an automatic charging device that automatically charges is generally used in an AGV (Automatic Guided Vehicle), the description thereof will be omitted.

In FIG. 10, when the remaining battery level of the rechargeable battery reaches a predetermined warning level, the harvesting robot 102 or the collecting robot 103 transmits robot state information including the charging state to that effect to the server 101 (R151). The warning level must be at least the capacity that the remaining capacity of the rechargeable battery can reach the home station 105 by interrupting the work no matter where in the farm. Upon receiving the warning of the dead battery in the robot status information, the server 101 sends a request signal to the home station 105 to consult the vacancy of the dock equipped with the automatic charging device (S151). If there is a vacant dock, the home station 105 returns the vacant dock number to the server 101 as vacant position information (H151). The determination unit 203 of the server 101 that has confirmed the availability of the dock determines charging (S152). Next, the server 101 transmits the instruction information for the home station including the charging instruction information for which dock to go to the harvesting robot 102 or the collecting robot 103. The harvesting robot 102 or the collecting robot 103 that has received the instruction information for the home station moves to the designated dock of the home station 105 (R152). After arriving at the home station 105, the harvesting robot 102 or the collecting robot 103 transmits an arrival signal to the server 101, and the server 101 sends charging instruction information to the home station 105 (S153). Upon receiving the charging instruction information, the home station 105 starts charging (H152). After the charging is completed, the home station 105 sends a charging completion signal to the server 101. When the harvesting robot 102 or the collecting robot 103 has no work, the server 101 that receives the charge completion signal transmits the standby instruction information that orders the standby as it is (S154). The harvesting robot 102 or the collecting robot 103 that has received the standby instruction information stands by at the designated position (R153). If the harvesting robot 102 or the collecting robot 103 has a job, the determination unit 203 of the server 101 allocates the job and gives the harvesting robot 102 or the collecting robot 103 harvest instruction information and other instruction information (S155).

The charging instruction information from the server 101 to the home station 105 may be transmitted from the harvesting robot 102 or the collecting robot 103. However, the home station 105 needs a structure capable of exchanging information with a transmission / reception device or a contact point. Similarly, the charging completion signal may be transmitted from the harvesting robot 102 or the collecting robot 103 to the server 101.

It should be noted that this charging operation may be systematically performed at the same time as the tool change operation shown in FIG.

In addition, although it was stated that the remaining battery level reached the warning level and the charging operation was started, the server 101 periodically monitored the status of the rechargeable batteries of the harvesting robot 102 and the collecting robot 103, and started the charging operation during the work. In order to avoid it, it is possible to systematically dock the work at a well-separated place as soon as possible.

FIG. 11 is a flow chart for determining a harvest target area according to the embodiment of the present invention.

Although there are various procedures for determining the harvest target area (S3) in the basic operation flow diagram of harvesting / collecting / storing basket supply in FIG. 8, an example of the procedure for objectively and efficiently determining the harvest target area is shown in FIG. Shown in.

As illustrated in FIG. 11, when the image data with the map address is input to the server 101 from the harvesting robot 102 that has taken a picture of the harvesting area and specified its own position (R1), the determination unit 203 first receives an unnecessary background. The noise of the branches and leaves is removed, and only the fruits are extracted (S201). Since various image processing methods have been proposed for this method, detailed description thereof will be omitted. Next, the determination unit 203 ranks the maturity of each of the extracted fruits based on the maturity ranking standard (N201) previously input by the operator 106 through the input / output means 107 (S202). The maturity ranking standard is, for example, that most fruits continuously change from the first color, which is immature, to the second color, which is ripe. Become. The first color is, for example, green, and the second color is, for example, red. The combination of the first color and the second color differs depending on the fruit. Further, depending on the fruit, the first color and the second color may be separated depending on the location in one fruit during ripening, or the color between them may change continuously. In general, the size increases as it matures. The maturity ranking standard is a pattern classification of the characteristics according to the maturity of the object 306 according to the characteristics of the object 306. The determination unit 203 compares these rank reference patterns with the object 306, and ranks the rank of the closest reference pattern as the rank of the object 306.

Next, since a rank or higher that can be harvested in that rank is input in advance as a threshold value (N202), the judgment unit 203 extracts fruits above the threshold value from the ranked fruits and harvests each unit harvesting area. Estimate the amount (S203). Basically, based on the estimated yield, the target harvest area may be set from the ridge with the highest yield, and the harvest robot 102 may be served in order, but some correction may be made here (S204). ). For example, instead of uniformly counting the number of fruits having a rank equal to or higher than the threshold value, the fruits having a higher maturity rank are weighted (N203). This is because fruits of a high rank of ripeness need to be harvested immediately before the ripeness progresses and the product power declines. The weighting coefficient for each rank is input to the server 101 from the input / output means 107 in advance.

On the other hand, a certain variety may be prioritized according to the harvest schedule that has been formulated in advance and is input to the server 101 from the input / output means 107 (N204). This is based on the judgment that it is better to prioritize the varieties that sell because the harvest schedule reflects the needs from the market. In that case, set a large weighting coefficient for the varieties that sell. In addition, if the shipping plan for each variety is predetermined based on a contract with a large-scale delivery destination of agricultural products, the quantity of that variety may be insufficient if it is normally cut at a predetermined threshold. In that case, some young fruits may be mixed, so it is necessary to lower the threshold rank to satisfy a predetermined quantity. On the contrary, if it is normally cut at a predetermined threshold, the quantity of the variety may exceed the plan. In that case, it is necessary to raise the threshold rank and reduce the planned number of harvests. For the farm, the disposal of crops must be avoided. Reflecting these circumstances, the determination unit 203 corrects the evaluation value. Based on the result, the determination unit 203 finally determines the harvest target area (S205). After that, the process proceeds as shown in the basic operation flow diagram of harvesting / collecting / storing basket supply in FIG.

Although the above-mentioned method for determining the maturity of the fruit is based on the image data captured by the harvesting robot 102, it may be determined based on other information. For example, a probe of a sugar content meter may be attached instead of the harvesting hand of the harvesting unit 304 of the harvesting robot 102, the sugar content of the fruit may be measured, and the harvest target area may be determined based on the sugar content.

Further, although the above operation flow has been described with the example of fruits that are easily colored, the same method can be applied to other agricultural products. However, depending on the type of crop, it may be judged not only by color but also by shape and size. In short, depending on the characteristics of the crop, a criterion that makes it easy to determine the maturity is selected.
FIG. 12 is a photographing rotation operation flow diagram according to the embodiment of the present invention.

In the step of photographing the image data with the map address in the flow chart of FIG. 8, the harvesting robot 102 first photographs based on the photographing instruction information from the server 101 (R1). This routine is mainly used for crops that are harvested only once each year within the same harvest area for a limited period of time. On the other hand, in some high-rise greenhouse cultivation facilities such as cherry tomatoes and strawberries, harvesting work continues all year round or for several months. In these crops, innumerable fruits grow in sequence from one seedling and stem during the harvest period, so the same ridge is harvested many times during the harvest period. That is, a fruit that has not been harvested because it is immature at a certain harvesting operation reaches the maturity period, for example, one week later. By utilizing this characteristic, it is possible to omit the scanning run of the harvesting robot 102 for the purpose of taking a picture only. The operation will be described below with reference to FIG.

In FIG. 12, the harvesting robot 102 that receives the harvesting instruction information from the server 101 carries out harvesting and storage (R2). More specifically, for harvesting and storage by the harvesting robot 102, first, the photographing unit 301 recognizes the object 306, determines whether or not the object can be harvested, identifies the position of the object 306 that can be harvested (R251), and then harvests. Harvesting is carried out in part 304 (R252) and stored in the storage basket 307 (R253). When all the on-board storage baskets 307 are filled with the object 306 or the harvesting of the ridges instructed by the server 101 is completed, the harvesting robot 102 transmits a harvest completion signal to the server 101 (R254). In the above flow, in this option, the image of the object 306 taken to harvest the harvested product 306 is transmitted to the server 101 as image data with a map address each time (R251). Since the harvesting robot 102 selects the object 306 to be harvested in the unit harvesting area at the time of harvesting, the entire unit harvesting area is first photographed, so that information on all fruits including immature can be obtained.

Since the image data obtained by the server 101 includes fruits to be harvested by the harvesting robot 102, the determination unit 203 first deletes the harvested fruits from the image data (S251). Harvest success information on which fruit was harvested or failed is obtained from the harvesting robot 102.

The server 101 accumulates the processed image data in the image data sent from the harvesting robot 102 (S252). When a certain amount is accumulated, the determination unit 203 creates a fruit maturation distribution map (S253). A certain amount is, for example, when the harvesting work of the day is completed, or when the harvesting of one ridge is completed. The maturity distribution map is data that ranks the maturity of all fruits and shows which level of maturity of fruits is distributed in which region. From this maturation distribution map, the determination unit 203 estimates, for example, the total amount of harvestable fruits of a certain ridge after a certain period of time (S254). The threshold value for harvestability at this time is input separately. Based on this estimated amount, the determination unit 203 determines the harvest target area after a lapse of a certain period of time (S255). From these, the determination unit 203 formulates a harvest schedule after a certain period of time has elapsed (S256), and the server 101 sends harvest instruction information to the harvest robot 102 at any time based on this harvest schedule. By repeating this as a rotation at a fixed time cycle, image data with a map address is sequentially obtained at the same time as harvesting, so that the harvesting robot 102 does not have to bother to scan the farm just for shooting. Therefore, the work efficiency of the harvesting robot 102 is improved, and the number of harvesting robots 102 can be reduced accordingly.

Further, in FIG. 12, it is assumed that the maturity distribution map is created after accumulating the image data, but the distribution map may be sequentially created after the image data is obtained.

The rotation span is appropriately set to a span with high operational efficiency and high prediction accuracy based on the characteristics of fruits and farms and past data.

FIG. 13 is a learning function application maturity determination flow diagram according to the embodiment of the present invention.

In the basic operation flow chart of harvesting / collecting / storing basket supply shown in FIG. 8, the determination unit 203 determines the harvest target area based on the harvesting conditions including the harvest determination criteria input in advance. More specifically, as illustrated in the harvest target area determination flow chart of the embodiment of the present invention in FIG. 11, a maturity ranking criterion and a harvestable threshold are given as harvesting conditions. In this option, the maturity ranking standard and the threshold value are not set by a person based on the experience and intuition of the person, but are set by the determination unit 203 based on the accumulated image data with a map address (S300). The operation will be described below with reference to FIG.

In FIG. 13, the server 101 accumulates image data with a map address taken by the harvesting robot 102 (S301). An image of a single crop randomly extracted from the accumulated image data regardless of the maturity is displayed on the input / output means 107 (N301). Upon seeing this, the operator 106 determines whether or not the crop is suitable for harvesting (P301), labels it with two options, and inputs it through the input / output means 107 (N302). The server 101 associates the image data of the crop with a label corresponding to the image data (S302), and accumulates the associated data (S303). When sufficient accumulation is obtained, the judgment unit 203 of the server 101 evaluates them by the judgment factors given in advance, for example, hue, lightness, saturation, degree of color mixture (variation), size, shape, and the like. Generate an index-based maturity evaluation function. Next, an evaluation function that divides the ranks of harvestability most clearly is extracted, and the boundary is used as a threshold value (S304). These are newly set as harvesting conditions, applied to the image data with a map address from the harvesting robot 102, the determination unit 203 determines the harvesting target area (S3), and transmits the harvesting instruction information to the harvesting robot 102.

As the evaluation function, a neural network, a Bayesian discriminator, or a discriminator using machine learning such as SVM (Support Vector Machine) is typically used. Then, the classifier is subjected to learning processing using the teacher data that associates the image of the object with the maturity of the object, and can be harvested for each unit harvesting area using the trained classifier. The amount of the object will be estimated.

In the process of generating the evaluation function and the threshold value, the operator 106 may intervene at any time to make corrections.

FIG. 14 is a flow chart at the time of manual programming according to the embodiment of the present invention. In the basic operation flow diagram of harvesting / collecting / storing basket supply according to the embodiment of the present invention of FIG. 8, as described above, the harvesting target area to be harvested by the harvesting robot 102 is the image data with the map address of the object 306 and harvesting. The determination unit 203 of the server 101 decides based on the given conditions, but in some cases, this option also allows the operator 106 to make this decision. In an actual farm, it may be inconvenient if the harvest target area is automatically determined. If the inconvenience is not caused by a single occurrence, the factor is added to the S204 correction term.

Hereinafter, the operation flow will be described with reference to FIG.

As shown in FIG. 14, in this option, a new judge of "Is the determination of the harvest target area automatic?" (S351) is added. YES or NO is a parameter, which is input by the operator 106 in advance. If YES, the harvesting work proceeds according to the flow of FIG. 8 as before. When the parameter is NO, the image of the image data with the map address and the area information are displayed on the input / output means 107 (N351). Looking at this, the operator 106 determines which harvesting area the object 306 is to be harvested (P351) and inputs it to the input / output means 107 (N352). Upon receiving this, the server 101 determines the input area as the harvest target area (S3), and transmits the harvest instruction information to the harvest robot 102.

FIG. 15 is a harvest prediction management processing flow chart according to the embodiment of the present invention.

In the harvesting robot system according to the embodiment of the present invention, the harvesting plan, the collecting plan, and the storage cage supply plan are optimally programmed by the server 101 so that the harvesting robot 102 and the collecting robot 103 do not wait. Game theory, queuing theory, etc. may be applied to the process. In addition, the harvest schedule may be programmed by making some predictions in advance based on a huge amount of past data. An example thereof will be described with reference to FIG.

In FIG. 15, all the image data with map addresses taken by the harvesting robot 102 are stored in the large-capacity storage device 204 (S401). The shooting date and time are also recorded as attached information in these image data. Based on the accumulated image and the time and place information associated with the image, the determination unit 203 systematically analyzes the image data in space and time (S402). For example, when focusing on a specific crop at a certain place, the change over time can be extracted, and the maturity curve of the time axis of the crop can be obtained. You can also see when the crop was harvested. In other words, from this data, it is possible to predict the future progress of maturity of certain crops. Next, the judgment unit 203 predicts the future maturity progress for each unit harvesting area, and creates a time-series maturity rank prediction distribution map (S403). Based on this forecast distribution map, the determination unit 203 estimates the daily yield in the near future, for example, in the next week (S404). Next, the determination unit 203 determines an optimized harvest schedule in the near future based on the estimation (S405), and transmits the harvest instruction information to the harvest robot 102. Optimization means, for example, adjusting the harvest schedule so that the daily yields for the next week are leveled. On the farm side, it will be easier to predict the yield in the near future and make a shipping plan. For farm management, a shortage of shipments leads to a loss of sales opportunities, and an excess of shipments leads to a loss of disposal.

The harvest schedule in the near future does not have to be a weekly schedule, and the optimum planning span is set as appropriate according to the characteristics of the crops, market needs, etc., such as the monthly schedule.

Further, if the harvesting robot 102 is equipped with various sensors in addition to the photographing unit 301 to collect various information other than the image data at the same time, the maturity prediction accuracy is further improved. The various sensors are a thermometer, a hygrometer, an illuminance meter, a gas concentration meter, and a soil sensor. The harvesting robot 102 is equipped with one or a plurality of the sensors in combination according to the purpose. All the items measured by these sensors have a great influence on the growth of plants. For example, in general, the higher the temperature and the higher the carbon dioxide concentration, the faster the growth. The harvesting robot 102 attaches the information obtained by these sensors to the image data with a map address and sends it to the server 101. Judgment unit 203 creates a maturity rank prediction distribution map by adding these information to the above-mentioned image data spatiotemporal analysis and future maturity progress prediction. The following follows the flow described with reference to FIG. 15 above.

FIG. 16 is a real-time feedback processing flow diagram according to the embodiment of the present invention.

The options shown in FIG. 15 allow for forecasting harvests for the near future, eg, the next week, and thus for weekly harvesting schedules. This will improve the efficiency of work on all farms, including harvesting. However, unlike industrial products, agricultural products do not grow uniformly and evenly, and fluctuate greatly depending on the weather and other factors. It is possible to make a prediction first, but the prediction is wrong, and it is necessary to have a feedback mechanism that constantly changes the program flexibly with the latest information. An example thereof will be described with reference to FIG.

In FIG. 16, S401 to S405 have the same flow as in FIG. In this option, the harvesting robot 102 photographs the harvesting target area before the actual harvesting is performed based on the harvesting schedule. The harvesting robot 102 sends the latest image data with a map address to the server 101. The judgment unit 203 of the server 101 compares the obtained image data with the map address with the maturity forecast map at the time of creating the harvest schedule plan in the near future (S451), and if the deviation from the forecast is less than the allowable value, it is as it is. The original harvest plan is carried out (S453). If the deviation exceeds the permissible value, the harvest target area is corrected (S452).

The modification may be performed by the operator 106 directly via the input / output means 107.

FIG. 17 is a cultivation management processing flow chart according to the embodiment of the present invention.

So far, the target of the main harvesting robot system implementation work has been limited to the harvesting work, but this option extends the target of the implementation work to agricultural work other than harvesting. Agricultural work other than harvesting includes, for example, pruning, chemical spraying, fruit thinning, and leaf thinning. Many conventional harvesting hands are equipped with scissors for harvesting, and the harvesting robot 102 can be used for pruning, fruit thinning, leaf thinning, etc. as it is or with a slight arrangement of morphology. If the picked leaves, branches, thinned immature fruits, etc. can be stored in the storage basket 307 as they are, the work of cleaning up the residue can be reduced. Chemical application can be handled by changing the harvesting hand to a spray gun and equipping a chemical tank instead of a storage basket. In this way, this harvesting robot system can be applied to agricultural work other than harvesting simply by exchanging tools, and has a wide range of applications. Tools in this case include not only various harvesting hands, but also pruning scissors, spray guns, and the like.

The cultivation management flow including the harvesting centering on the exchange of tools will be described with reference to FIG.

In FIG. 17, the same reference numerals are used for the same elements as in FIG. 9, and the description thereof will be omitted. The information necessary for exchanging tools is given in advance as a given condition (S501). From the given conditions of S101 of FIG. 9, agricultural work information other than harvesting is added. The tool code information is extended to tools used for agricultural work other than the harvesting hand. Agricultural work information other than harvesting includes, for example, pruning, fruit picking, and state pattern data of trees to be picked and judgment criteria. The contents are based on the harvest conditions. It should be noted that the chemical application is not judged from the condition of the photographed tree, but is performed multiple times at a fixed time every year, so that time is input as agricultural work information. The period of chemical application is allowed to vary, and the determination unit 203 of the server 101 decides to execute the chemical application when the harvesting robot 102 has spare capacity while avoiding the busy period of other agricultural work such as harvesting.

The determination unit 203 of the server 101, which has received the image data with the map address from the harvesting robot 102, determines the farm work to be performed and the target area based on the given conditions (S502). At the same time, the robot state information including the tool state is received from the harvesting robot 102, and it is determined whether or not the farm work determined by the tool can be carried out (S503). If it is OK, the server 101 transmits the farm work instruction information including the harvesting work to the harvesting robot 102. If it is NG, a request signal is transmitted to the home station 105. After that, the tool change operation flow of FIG. 9 is followed. However, the scope of the tool exchange decision is extended to agricultural work tools other than the harvesting hand (S504).

FIG. 18 is an operation flow diagram of another farm cooperative system according to the embodiment of the present invention.

So far, the target of this harvesting robot system has been limited to one farm managed by the server 101, but this option extends the target of information management to other farms managed by other servers. Then, a system is constructed in cooperation with the server 108 of another farm via the Internet 109 (see FIG. 1). Except for some high-rise greenhouse cultivation facilities such as cherry tomatoes, the harvest period of one crop is usually limited to a certain period of one year, but the harvest time differs depending on the type of crop. For example, the harvest time of tomatoes and cucumbers is generally from October to July, while the harvest time of eggplants and peppers is from June to October, and the peak harvest is shifted. Similarly, strawberries are harvested from December to May, and blueberries are harvested from June to September. Apples are from August to November, and cherries are from May to July.

Since the harvesting robot 102 can handle the harvesting of many crops by exchanging tools, that is, harvesting hands, the harvesting robot 102 can be time-shared among the above crops. Time-sharing can increase the year-round utilization rate of the expensive harvesting robot 102 and build a more cost-effective harvesting robot system.

An operation example of the cooperative system with other farms will be described with reference to FIG. First, an annual harvest schedule is created (S551). The annual harvest schedule is input to the server 101 by the operator 106 through the input / output means 107 based on the past results and taking into account the fluctuation factors of this year. Next, the determination unit 203 calculates the number of harvesting robots 102 and recovery robots 103, which are particularly necessary during the busy harvest season, and determines whether or not it can be handled by its own resources (S552). If sufficient, the judgment unit 203 formulates a robot serving plan with the robot on hand (S553) and sets it as a harvesting condition. If it is insufficient, a request signal is sent to the server 108 of the other farm to obtain information on the other farm including the robot operation plan of the other farm (T551). Next, the determination unit 203 incorporates surplus resources of other farms, formulates a robot serving plan (S554), and displays it on the input / output means 107 (N551). The operator 106 sees it, attaches the approval of the robot rental of another farm, and inputs the confirmation (N552). Upon receiving the input, the determination unit 203 determines the robot serving plan (S555), sends the information of the own farm including the operation plan of the robot to the server 108 of the other farm, and sets it as the harvesting condition (S556). After receiving the input of the information (T552), the server 108 of the other farm utilizes the information in the other farm.

In the above, the server 101 formulates the robot serving plan, but the operator 106 may browse and decide the situation data of other farms. For this system, it doesn't matter which one. A mechanism for exchanging information and sharing information on a network between servers, that is, a cooperative system is important.

Although the communication network between the servers is the Internet, a dedicated LAN line may be used.

In addition, although the flow was explained above on the premise of an annual plan, in cases where the distance between farms is relatively short, detailed time sharing on a weekly or daily basis is also possible, and the effect of flexible response by networking is possible. Is high. Furthermore, if the cooperation between farms is expanded and expanded on the premise that the harvesting robot 102 is also used for farm work other than harvesting, the farm management effect will be further enhanced.

[effect]
As described above, the harvesting robot system according to the present embodiment is
A harvesting robot system including a harvesting robot 102 that harvests an object in a farm and a server 101 that is configured to communicate with the harvesting robot 102.
Server 101
An input / output unit 202 that accepts input of harvesting conditions including farm map information of the farm and harvest judgment standard information for determining the harvest time of the object, and
Judgment unit 203 that determines the harvest target area based on the image of the object taken by the harvest robot 102 for each unit harvest area in the farm and the harvest conditions.
It includes a first transmission / reception unit 201 that transmits harvest instruction information including position information of a harvest target area to the harvest robot 102.

As described above, in the harvesting robot system according to the present embodiment, a plurality of harvesting robots 102 are centrally managed by the server 101, and the movable harvesting robot 102 photographs the harvesting area in the farm with the photographing device 301 before harvesting. By doing so, it is possible to obtain precise maturity status information of the crops in the entire farm, which cannot be obtained by the monitoring system using the fixed fixed point camera, and the server 101 can always formulate the optimum harvest schedule based on the information. Then, it is possible to realize a flow in which the harvesting robot 102 that receives the harvesting instruction from the server 101 executes the harvesting based on the harvesting schedule. This makes it possible to greatly improve the harvesting work efficiency of the harvesting robot.

In addition, the above-mentioned wide variety of options can be added to this system, which makes it possible to further improve the operating efficiency of the harvesting robot, which further improves cost effectiveness.

The harvesting robot system of the present invention is a cultivation management harvesting system that collects cultivation status information in a wide range and precision, formulates and executes an optimum work plan based on the information, and efficiently wastes such as harvests and weeds. It has a circulation type distribution system that collects harvesting, not only for automating the harvesting work of facility-type fruit cultivation farms, but also for harvesting almost all crops including open-field cultivation and standing trees, and for various other farming operations other than harvesting. It can also be applied to automation of applications.

101 Server 102 Harvesting robot 103 Harvesting robot 104 Collection station 105 Home station 106 Operator 107 Input / output means 108 Servers of other farms 109 Internet 201 1st transmission / reception unit 202 Input / output unit 203 Judgment unit 204 Large-capacity storage device 301 Imaging unit 302 1st Self-position detection means 303 2nd transmission / reception unit 304 Harvesting unit 305 1st lifter mechanism 306 Object 307 Storage cage 308 Self-propelled trolley unit 309 3rd lifter mechanism 401 2nd lifter mechanism 402 2nd self-position detection means 403 3rd transmission / reception unit 404 Self-propelled trolley part 501 Harvested storage basket stock part 502 Empty storage basket stock part 601 Tool change part 602 Charging part 701 Cultivation yard 702 Sorting shipping yard 703 Main passage 704 Ridge 705 Ridge road 706 Sorting bagging worker 707 Shipping conveyor 708 Step-disassembling unit 709 Stacking unit

Claims (16)

A harvesting robot system that includes a harvesting robot that harvests objects in a farm and a server that is configured to communicate with the harvesting robot.
The server
An input / output unit that accepts input of harvesting conditions including farm map information of the farm and harvest judgment standard information for determining the harvest time of the object.
A determination unit that determines the harvest target area based on the image of the object taken by the harvest robot for each unit harvest area in the farm and the harvest conditions.
A first transmission / reception unit that transmits harvest instruction information including the position information of the harvest target area to the harvest robot, and
Harvesting robot system. The harvesting robot
An imaging unit that photographs the object and generates the image,
The first self-position detecting means for detecting its own current position,
A second transmission / reception unit that associates the image with information related to the shooting position and transmits the image to the server.
Based on the harvest instruction information received from the server, the harvesting unit that harvests the object and
The harvesting robot system according to claim 1. A home station for the harvesting robot to stand by is further provided in the farm.
The home station
A tool change unit that replaces the arm of the harvesting unit of the harvesting robot from the first tool to the second tool based on the tool change information from the server.
A charging unit that charges the rechargeable battery mounted on the harvesting robot,
A third transmitter / receiver that transmits vacant position information indicating vacant dock information for mooring the harvesting robot and tool inventory information including the type and number of tools currently held by the home station to the server. When,
The harvesting robot system according to claim 1 or 2. The determination unit receives the vacant position information and the tool inventory information received from the home station, tool change conditions for determining a tool to be attached to the arm to be used in the harvesting robot, and the harvesting robot. Based on the robot state information including the remaining battery level of the harvesting robot and the tool state of the arm, it is determined to charge the harvesting robot or replace the tool.
The first transmission / reception unit transmits instruction information for the home station including charging instruction information and tool change information to the harvesting robot.
The harvesting robot system according to claim 3. The image is one or more images taken so as to show the object existing in the entire area of the unit harvesting area.
The harvesting robot system according to any one of claims 1 to 4. The determination unit estimates the amount of the target object that can be harvested for each unit harvesting area based on the color of the object that appears in the image, and determines the harvest target area.
The harvesting robot system according to any one of claims 1 to 5. When the harvesting robot harvests the object in the harvesting target area instructed by the server, the photographing unit takes a picture of the unit harvesting area corresponding to the harvesting target area.
The harvesting robot system according to claim 2. The determination unit extracts the object existing in the entire unit harvest area from the image, creates a maturity rank distribution map of the object in the unit harvest area, and is based on the maturity rank distribution map. To estimate the harvestable amount of the unit harvesting area,
The harvesting robot system according to any one of claims 1 to 7. The determination unit has a discriminator that has been subjected to learning processing using teacher data that associates the image of the object with the maturity of the object.
Using the classifier, the amount of the object that can be harvested is estimated for each unit harvest area, and the harvest target area is determined.
The harvesting robot system according to any one of claims 1 to 8. The input / output unit is configured to be able to receive input of the harvest target area by the operator.
The harvesting robot system according to any one of claims 1 to 9. Based on the time-series images, the determination unit creates a harvest schedule that defines when to harvest the object in the unit harvest area.
The harvesting robot system according to any one of claims 1 to 10. The determination unit harvests the object in the unit harvesting area based on at least one of the sensor data of the thermometer, the hygrometer, the luminometer, the gas concentration meter, and the soil sensor provided in the harvesting robot. Create a timed harvest schedule,
The harvesting robot system according to any one of claims 1 to 11. The harvesting robot includes a recovery robot that collects the harvested object and transports it to a collection station in the farm.
The harvesting robot system according to any one of claims 1 to 12. The determination unit determines the harvest target area based on the position and operating state of the harvest robot.
The harvesting robot system according to any one of claims 1 to 13. Based on the image, the determination unit determines when agricultural work other than harvesting is required in the unit harvesting area, and transmits instruction information for the home station and tool change information to the harvesting robot based on the time. The harvesting robot system according to claim 3. The input / output unit acquires information on other farms, including an operation plan for harvesting robots on other farms.
The determination unit creates an operation plan of the harvesting robot of the farm based on the information of the other farm.
The harvesting robot system according to any one of claims 1 to 15.

Images