JP7223977B2 - Harvest robot system - Google Patents

Description

The present invention relates to a harvesting robot system that automatically harvests agricultural products including fruits such as apples, fruit vegetables such as tomatoes, and fruity vegetables such as strawberries based on cultivation management information.

As a future social issue in Japanese society, there is concern about a decrease in the working population due to aging. In particular, agriculture, which is the primary industry, is in a very severe situation in terms of the aging population and labor shortage. According to the basic data of the Ministry of Agriculture, Forestry and Fisheries, the number of core farmers in 2014 was 1.68 million, a decrease of about 230,000 in five years. In addition, the average age is 66.5 years old, and the number of new farmers is also declining, which highlights the labor shortage due to the aging population. As a result, the amount of abandoned farmland reaches 400,000 ha, and this situation adversely affects the farming environment and living environment of the region. Especially in rural areas where there is a lot of arable land, depopulation is progressing due to the declining birthrate and aging population, and this problem is becoming more apparent because there are no people to bear the burden of agriculture.

Under such circumstances, expectations are rising for the automation of agriculture to deal with labor shortages. According to the Ministry of Economy, Trade and Industry's 2012 Robot Industry Market Trends, the domestic market for agricultural robots is expected to grow significantly from 2018 to 2024, and is expected to reach a scale of approximately 220 billion yen. In fact, developments that lead to the automation of agriculture, such as management using drones, automatic driving of tractors, and cultivation navigation systems, are becoming active. In addition, the Ministry of Agriculture, Forestry and Fisheries has a substantial subsidy system that supports the automation of agriculture, and the government has very high expectations for this field.

Among them, research on technologies related to automated harvesting has been progressing in companies and universities, and many harvesting robots that automatically harvest various fruits, fruity vegetables, and fruit vegetables have been proposed. . However, a proposal for a cultivation management system and a harvesting robot system in which the harvesting robots are interlocked, how to automatically obtain the cultivation status information in the farm and how to determine the area to be harvested by the harvesting robot. is less.

Among them, as a conventional system for collecting and managing cultivation status information, there is a system in which multiple monitoring devices are installed in the farm and one server is installed in a remote location, and these are connected to the Internet (for example, patent document 1). In the conventional independent operation control system that collects cultivation status information in a farm described in Patent Document 1, a monitoring device has an imaging unit and photographs images of predetermined crops. The server is connected to a plurality of monitoring devices via the Internet, and operators can view images of agricultural products in remote locations via terminals.

Patent No. 4586182

However, in the configuration of the conventional patent document 1, since the monitoring device is fixedly installed, the shooting location is fixed, and the number of monitoring devices is limited, so the entire farm cannot be covered. Therefore, there is a problem that the optimum harvest schedule for the entire farm cannot be set up from limited information. In addition, if the wide range can be photographed, the details of the crops cannot be recognized due to the limitation of the resolution of the imaging unit. In particular, in agricultural crops in which the fruits connected to one main stem or one cluster mature in stages and the harvesting times are different, there is a problem that the degree of maturity of each small fruit cannot be determined and the purpose cannot be achieved. . Furthermore, the conventional configuration of Patent Document 1 has only a monitoring function, and does not have any cooperation with a harvesting robot. In other words, it is difficult to operate harvesting robots efficiently.

The present invention solves the above-mentioned conventional problems, and is a cultivation management system that covers a wide farm with a harvesting robot and obtains detailed maturity status information of individual crops to establish an optimal harvesting schedule. It is an object of the present invention to provide a harvesting robot system in which the harvesting robot is efficiently operated by combining the above.

The harvesting robot system of the present invention for achieving the above object comprises:
A harvesting robot system including a harvesting robot for harvesting an object in a farm and a server configured to communicate with the harvesting robot,
Said server
an input/output unit that receives input of harvesting conditions including farm map information of the farm and harvest judgment criteria information for judging the harvest time of the object;
a determination unit that determines a harvest target area based on the image of the object photographed by the harvest robot for each unit harvest area in the farm and the harvest condition;
a first transmitting/receiving unit that transmits harvesting instruction information including position information of the harvesting target area to the harvesting robot;
A harvesting robot system comprising

As described above, according to the harvesting robot system of the present invention, the server can set up an optimum harvesting schedule based on the necessary and sufficient cultivation status information of the crops, and the harvesting robot that receives the instruction is most efficient. Harvesting work can be done systematically.

1 is an overall system configuration diagram of a harvesting robot system according to an embodiment of the present invention; FIG. Block diagram of a server according to an embodiment of the present invention 1 is a configuration diagram of a harvesting robot according to an embodiment of the present invention; FIG. Schematic diagram of a collection robot according to an embodiment of the present invention 1 is a block diagram of a pickup station according to an embodiment of the present invention; FIG. A block diagram of a home station according to an embodiment of the present invention Schematic layout diagram of the entire farm according to the embodiment of the present invention Harvesting/recovery/storage basket supply basic operation flow diagram in the embodiment of the present invention Tool change operation flow diagram in the embodiment of the present invention Charging operation flowchart in the embodiment of the present invention Harvesting target area determination flowchart in the embodiment of the present invention FIG. 2 is a flow chart of shooting rotation operation according to the embodiment of the present invention; Learning function application maturity determination flow diagram in the embodiment of the present invention Flow chart for manual programming according to the embodiment of the present invention Harvest prediction management processing flow diagram in the embodiment of the present invention Real-time feedback processing flow diagram in the embodiment of the present invention Cultivation management processing flowchart in the embodiment of the present invention Operation flow chart of other farm cooperation system in the embodiment of the present invention

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.
(Embodiment)
FIG. 1 is an overall system configuration diagram of a harvesting robot system according to an embodiment of the present invention.

In FIG. 1, a server 101 includes a plurality of harvesting robots 102, one or more collecting robots 103, a collection station 104, a home station 105 where the harvesting robot 102 or the collecting robot 103 stands by, and an operator 106. An input/output means 107 for performing the above operations and an Internet 109 that can be connected to a server 108 of another farm are connected by wire or wirelessly and organically related to constitute a harvesting robot system.

FIG. 2 is a block diagram of server 101 .

As shown in FIG. 2, the server 101 transmits and receives information to the harvesting robot 102 and the collection robot 103 via a wireless LAN or the like, a first transmission/reception unit 201, a collection station 104, a home station 105, and input/output means. 107, and a server 108 of another farm via the Internet 109, an input/output unit 202 for outputting or inputting information by wire or wirelessly, and an area to be harvested based on image data with a map address and harvesting conditions. A judging unit 203 that determines the collection order and collection route of stored storage baskets and determines an empty storage basket supply and distribution plan, and a large-capacity storage device 204 that can store a large amount of information including image data with map addresses. and consists of

FIG. 3 is a configuration diagram of the harvesting robot 102 according to the embodiment of the invention.

As shown in FIG. 3, the harvesting robot 102 includes a photographing unit 301 that photographs an object 306 and a storage basket with a camera, and a first self-positioning robot that identifies its own position by GPS, laser radar, white line marker, landscape recognition collation, and the like. a position detecting means 302; a second transmitting/receiving section 303 for transmitting information including image data with a map address to a server via a wireless LAN or the like, and receiving instruction information from the server; a first lifter mechanism 305 that grips and lifts the bottom storage basket 307 of the stacked storage baskets 307 placed on the ground, and the second stage from the bottom of the stacked storage baskets 307 It consists of a third lifter mechanism 309 that sequentially grasps and lifts and separates empty storage baskets 307 , and a self-propelled carriage 308 that can automatically travel along the ridge 705 and the main passage 703 with all of the above equipment mounted.

Note that the imaging unit 301 is preferably a stereo camera. A normal two-dimensional area camera may be used if it is used only for obtaining image data with map addresses for determining the harvest target area. Also used when picking up. Distance information to the object 306 is required for harvesting, and distance information to the storage basket 307 is required to pick up the storage basket 307 . If a normal two-dimensional area camera is used, a separate distance measuring device such as a laser sensor is required.

The harvesting unit 304 has different forms depending on the type of the target object 306 to be harvested. Since various harvesting robots, harvesting manipulators, and harvesting hands have been proposed in the past, detailed description thereof will be omitted. FIG. 3 shows an example in which the harvesting unit 304 has a dual-arm articulated manipulator attached to a rotatable body and a harvesting hand attached to the tip of the manipulator. As for the harvesting hand, the most suitable harvesting hand is selected according to the type, shape, size, etc. of the target object 306 and is appropriately attached. In FIG. 3, the manipulator that collects the target object 306 stores the target object 306 in the storage basket 307 as it is. may be divided into In that case, a harvesting unit 304 is formed by combining the manipulator for harvesting and the storage mechanism.

The storage basket 307 is made of plastic, for example, and is a type of box called a container that can be stacked with objects stored therein.

Also, the self-propelled carriage unit 308 is generally called an AGV (Automatic Guided Vehicle), and a detailed description thereof will be omitted. Although FIG. 3 shows an example of running on wheels, a crawler or the like may be employed depending on the characteristics of the farm.

FIG. 4 is a configuration diagram of the recovery robot 103 according to the embodiment of the present invention.

As shown in FIG. 4, the recovery robot 103 includes a second lifter mechanism 401 having the same function as the first lifter mechanism 305, a second self-position detection means 402 having the same function as the first self-position detection means 302, and a It consists of a third transceiver 403 having the same function as the second transceiver 303, and a self-propelled carriage 404 on which all of the above devices are mounted and capable of automatically traveling along the main passage 703 (see FIG. 7).

FIG. 5 is a block diagram of pickup station 104 in accordance with an embodiment of the present invention.

As shown in FIG. 5, the collection station 104 receives a plurality of harvested storage baskets 307 from the collection robot 103, for example, a harvested storage basket stock section 501 that can be stocked on a roller conveyor, and a plurality of empty storage baskets 307. For example, an empty storage basket stock unit 502 capable of stocking on a roller conveyor and capable of delivering an empty storage basket 307 to the recovery robot 103 upon request from the server 101 or the recovery robot 103 is provided.

Incidentally, the collection station 104 transmits to the server 101 the empty space information of the harvested storage basket stock unit 501, and the inventory information including the inventory amount of the empty storage baskets existing in the empty storage basket stock unit 502 to the server 101. It also has a fourth transmitter/receiver (not shown).

FIG. 6 is a block diagram of home station 105 according to an embodiment of the present invention.

The home station 105 is a place where the harvesting robot 102 or the collecting robot 103 is moored when not performing the harvesting work or the collecting work.

As shown in FIG. 6, the home station 105 replaces the tool at the tip of the manipulator for harvesting in the harvesting section 304 of the harvesting robot 102 with a tool for harvesting another type of object or for a tool used for other work. a tool changing unit 601 that performs the operation, a charging unit 602 that charges the rechargeable battery for driving the harvesting robot 102 or the collection robot 103, and a fifth transmitting/receiving unit (not shown) that transmits/receives data to/from the server 101. be.

FIG. 7 is a schematic layout diagram of the entire representative farm in the embodiment of the present invention. In FIG. 7, the same reference numerals are used for the same components as those in FIGS. 1 and 5, and description thereof is omitted.

As shown in FIG. 7, the farm is broadly composed of a cultivation yard 701 for cultivating crops and a sorting and shipping yard 702 for sorting and packing harvested crops into bags for shipment. The cultivation yard 701 may be an open field or a greenhouse such as a vinyl house. Moreover, the cultivation yard 701 and the sorting and shipping yard 702 may be directly connected or may be connected by a passage.

In the center of the cultivation yard 701, there is a relatively wide main passage 703 along which the harvesting robot 102 and the collection robot 103 travel. The main passage 703 is directly connected to the sorting and shipping yard 702 . A few ridges 704 for growing crops such as tomatoes and strawberries are vertically arranged on both sides of the main passage 703 . The ridge 704 may be a literal ridge of raised soil, a cultivation box for elevated cultivation as seen in Dutch-style Venlo-type greenhouse cultivation, or an array of standing fruit trees. The area where the object 306 to be cultivated and harvested on this ridge 704 is present is the harvesting area. Between the ridges 704 and 704 there is a ridgeway 705 to allow the harvesting robot 102 to pass through. Harvesting robot 102 can harvest objects 306 from ridges 704 on both sides. The ridge road 705 may be a soil surface or a hot water pipe may be laid in the case of a greenhouse. If hot water pipes are laid, the harvesting robot 102 can use the hot water pipes as tracks for furrow running. As shown in FIG. 7, the end of ridge 705 opposite main passage 703 may be a dead end. After harvesting, the harvesting robot 102 turns back along the ridge road 705 , once exits the main passage 703 , unloads the harvested storage basket 307 , and heads for the next ridge road 705 . At this time, it is efficient if the harvesting robot 102, for example, harvests the ridges 704 on the right side on the forward path and harvests the ridges 704 on the left side on the return path. By the way, many farms have a dead end on the opposite side of the ridge road as shown in FIG. This is for increasing the area productivity of the cultivation yard 701 .

The storage baskets 307 containing the harvested objects 306 or the empty storage baskets 307 are temporarily placed at positions where they do not interfere with other operations in the main passage 703, for example, at the ends of the ridges beside the main passage 703.

The harvested storage basket 307 temporarily placed on the side of the main passage 703 by the harvesting robot 102 is collected by the collection robot 103, transported to the harvested storage basket stock section 501 of the collection station 104, and stocked. The harvested storage basket stock unit 501 stocks the storage baskets 307 and transports the storage baskets 307 to the vicinity of the sorting bagging operator 706 by, for example, a roller conveyor. There is a destacking unit 708 on the way, and after separating the stacked storage baskets 307 one by one, they are transported to a sorting bagging operator 706 .

A sorting bagging worker 706 takes out the objects 306 from the storage basket 307 , sorts them, puts them in bags according to a predetermined procedure, and puts them on the shipping conveyor 707 . The bag containing the object 306 is conveyed to the next process by a conveyor, after which the bag is boxed and shipped.

On the other hand, the empty storage basket 307 is placed on the conveyor connected to the empty storage basket stock section 502 by the sorting bagging operator 706 . Empty storage baskets 307 are conveyed to the empty storage basket stock section 502 by a conveyor, and there is a stacking unit 709 on the way. transported to and stocked. After that, according to a request from the server 101 , the empty storage basket 307 is transported to the cultivation yard 701 by the collection robot 103 and placed at a specified location beside the main passage 703 . The harvesting robot 102 picks up the placed empty harvesting basket 307 and heads for harvesting. Thus, the circulation distribution system of the storage basket 307 functions.

The server 101 and the input/output means 107 are installed at appropriate locations within the farm.

The home station 105 is installed in a space adjacent to the main corridor 703 and can moor multiple harvesting robots 102 or collecting robots 103 . Tools are exchanged and rechargeable batteries are charged at moored locations.

Since a plurality of harvesting robots 102 and collecting robots 103 come and go in the main passage 703, it is preferable to determine a basic running route and rules in the main passage 703. For example, the main passage 703 is divided into two lanes, and the direction of travel is limited. In other words, the outbound and return trips are completely separated so that they can always pass each other. The basic travel route of the collection robot 103 turns around at the end of the main passage 703, and in the sorting and shipping yard 702, it connects the main passage 703→harvested storage basket stock section 501→empty storage basket stock section 502→main passage 703, forming a closed circuit. It is desirable to A plurality of collection robots 103 circulate in one direction along the closed circuit. The travel route may be displayed with white lines, attached with magnetic tape, or defined by farm map information.

Of course, the collection robot 103, which has finished installing the empty storage baskets 307 and collecting the harvested storage baskets 307 at a location near the sorting and shipping yard 702, should immediately head for the collection station 104 without tracing the closed circuit to the end. shortcut. This route instruction is also issued from the server 101 .

Note that the form of the farm is not limited to the form illustrated in FIG. The harvesting robot 102 and the harvesting robot 102 are collected on the main passage 703 in a form in which the collection robot 103 can come and go to the collection station 104, and the harvesting robot 102 can exit the main passage 703 from the ridge after completing the harvesting on the ridge. All that is necessary is to have a point of contact that the robots 103 can share.

Moreover, although the form of the harvested storage basket stock section 501 and the empty storage basket stock section 502 of the cargo collection station 104 is a roller conveyor, it is not limited to this. For example, it may be a simple space divided for each storage basket 307 only. In that case, the sorting bagging worker 706 goes to the space to pick up the storage basket, and after the work is completed, returns it to the empty storage basket stock space. The collection robot 103 detects the position of an empty storage basket in a predetermined space using, for example, an imaging device, and picks it up with the second lifter mechanism 401 .

Also, in the embodiment of the present invention shown in FIG. 7, the sorting and shipping work is performed by an operator, but an automatic sorting and bagging machine may be used. By the way, automation of sorting and shipping work has been realized for some fruits. In addition to the present embodiment, these automatic sorting bagging machines and further automatic boxing machines are introduced and connected to the server 101 as a shipping system, and the server 101 manages the shipping plan in addition to the harvest plan and collection plan. By doing so, an automatic harvesting and shipping system can be constructed, further advancing unmanned harvesting operations.

In addition, if the sorting bagging worker 706 works only in the daytime and the harvesting robot 102 and the collection robot 103 operate both day and night, a shift difference occurs. The section 502 requires a conveyor length capable of stocking the number of baskets to accommodate the shift difference. Furthermore, a large amount of storage baskets 307 are required. As described above, automating all operations, including sorting and shipping, eliminates shift differences and allows operation with shorter conveyor lengths and the number of storage baskets.

An operation example of the harvesting robot system configured as described above will be described with reference to FIG.

FIG. 8 is a basic operation flow diagram of harvesting, collecting, and supplying empty storage baskets in the embodiment of the present invention.

As exemplified in FIG. 8, the server 101 first receives an input of yield conditions (S1). The operator 106 operates the input/output means 107 to input the yield conditions to the input/output unit 202 .

The "harvest condition" input by the operator 106 is information that should be given in advance for the determination unit 203 to determine the harvest target area, and includes farm map information and harvest determination criteria for determining the harvest time of the target object. Contains information. Here, the harvesting determination reference information is information about a threshold for determining whether or not to harvest the object 306 from the image of the object 306 captured by the harvesting robot 102 . The farm map information is, for example, information about the cultivation yard 701 in the farm, the travelable area, the position of the collection station 104, the position of the home station 105, and the like.

In the farm map information, the cultivation yard 701 is divided into a plurality of unit harvesting areas, and address information is attached to each unit harvesting area and stored. In the harvesting robot system according to the present embodiment, a harvesting target area is determined for each unit harvesting area.

Next, the determination unit 203 determines a route for the harvesting robot 102 to shoot based on the farm map information and the like (S2). Next, the server 101 outputs photographing instruction information including the photographing route to the harvesting robot 102 through the first transmitting/receiving section 201 .

Harvesting robot 102, which has received the photographing instruction information through second transmitting/receiving section 303, follows the instruction while moving on self-propelled cart section 308 and photographs target object 306 in the farm using photographing section 301 (R1). The self-position of the harvesting robot 102 at the time of photographing is acquired by the first self-position detecting means 302 . The own position may be farm coordinate data or an address code defined in a farm map. If the photographing unit 301 is not a fixed type camera but a camera whose direction can be freely changed, image data with a map address including the own position in the farm and the attitude information of the camera is used. In other words, it is necessary to identify the absolute position of the object 306 in the cultivation yard 701 shown in the image data. The harvesting robot 102 transmits the acquired image data with map address to the server 101 via the second transmitting/receiving section 303 .

The “image data with map address” is an image of the target object 306 present in each unit harvesting area in the farm, captured by the harvesting robot 102 for each unit harvesting area. The "image data with map address" is typically photographed so that the entire unit harvesting area is captured so that the degree of maturity of the target object 306 present in the unit harvesting area can be grasped from the image (see FIG. 12). see below). However, the degree of maturity of the target object 306 present in the entire unit harvesting area may be grasped from a plurality of images.

The "image data with map address" includes, for example, the position where the image was taken (for example, the position of the harvesting robot 102) and the direction where the image was taken (for example, the posture of the harvesting robot 102) in association with the image. ) is stored. The position information stored in association with the image may be position information relating to the position of the object to be photographed, which is specified based on the position where the image was photographed and the direction in which the image was photographed.

The determination unit 203 of the server 101, which receives the image data with the map address of the farm from the harvest robot 102 by the first transmission/reception unit 201, selects the most efficient, that is, the most mature target based on the pre-input harvest conditions. An area with many 306 is determined as a harvest target area (S3). Specifically, the number of the ridge road 705 facing the determined harvesting target area is assigned to each harvesting robot 102 in consideration of the harvesting ability of the harvesting robot 102 . Basically, one harvesting robot 102 is assigned to one ridge 705 . The principle is that only one harvesting robot 102 is allowed to enter one ridge 705 at a given moment. A harvesting robot 102 may be assigned multiple furrows 705 . Note that all of the ridges 704 on both sides of the ridge road 705 may be the harvesting target area, or only a portion of the ridges 704 on one side may be the harvesting target area. These are the harvest instruction information.

The first transmission/reception unit 201 of the server 101 transmits the harvest instruction information including the harvest target area to the harvest robot 102 .

In the present embodiment, as described above, the harvest target area to be harvested by the harvest robot 102 is determined by the determination unit 203 of the server 101 based on the image data of the harvest with the map address and the harvest conditions. confirms the growth state of the harvest target in the farm with the naked eye, determines the harvest target area based on conventional experience, and inputs the number of the ridge road 705 to be harvested by each harvest robot 102 and the harvest section through the input/output means 107. Harvesting instruction information may be directly input.

Based on the harvesting instruction information received by the second transmitting/receiving unit 303, the harvesting robot 102 enters the ridge road 705 designated by the self-propelled carriage unit 308, and makes full use of the harvesting unit 304 to pick up the object 306 in the harvesting target area. Harvest and store in the storage basket 307 (R2). In the harvesting robot 102, the first lifter mechanism 305 is gripping the lowest storage basket 307 of the empty storage baskets 307 stacked in advance. This takes over the state of R5, which will be described later. When the harvesting robot 102 receives the harvesting instruction information, the third lifter mechanism 309 grips and lifts the storage basket 307 on the second stage from the bottom. Then, a space is formed above the storage basket 307 on the bottom layer. Using this space, the harvesting section 304 stores the objects 306 in the storage basket 307 on the bottom layer. When the storage basket 307 on the bottom layer is full, the third lifter mechanism 309 descends, and once all the storage baskets 307 are stacked, the storage basket 307 on the second stage from the bottom is released from the grip and the storage basket 307 is raised. It rises by one size, grasps the storage basket 307 on the third stage from the bottom, and rises. As a result, the harvesting unit 304 restarts the harvesting operation using the space formed above the storage basket 307 on the second stage from the bottom, and stores the object 306 in the storage basket 307 on the second stage from the bottom. . FIG. 3 shows the state at this time. Thereafter, this operation is repeated until all of the storage baskets 307 mounted thereon are filled with the objects 306, or when the harvesting robot 102 has completed the harvesting of the planned harvesting target area, Harvest status information including harvest completion signal and current position information are sent to

When the server 101 receives the harvesting status information including the harvesting completion signal by the first transmitting/receiving unit 201, the server 101 sends new harvesting instruction information to head to the next harvesting target area if there is still free space in the mounted storage basket 307. is given to the harvesting robot 102 . When all of the loaded harvest baskets 307 are full of objects 306, information for collection of storage baskets is collected (S4). The operator 106 has entered the input/output section 202 through the input/output means 107 in advance with respect to the collection and giving conditions necessary for making a collection plan for the harvested storage basket 307 . The collection conditions are farm map information including the positions of the collection stations 104, collection robot specifications including the number of collection robots 103 in operation, running speeds, and the like. Via the input/output unit 202 , the collection station 104 obtains the empty space information of the harvested storage basket stock unit 501 and confirms whether the collection station 104 has room for receiving the harvested storage baskets 307 . However, the harvested storage basket stock section 501 is designed to always have an extra space. This free space information is not required.

Next, the server 101 transmits a request signal from the first transmission/reception unit 201 to the collection robot 103 in order to ascertain the status of the collection robot 103 .

The collection robot 103 that has received the request signal by the third transmitting/receiving unit 403 detects its own position by the second own position detection means 402, and detects its own position data, whether or not the storage basket is currently loaded, and the work currently being executed. For example, the third transmitting/receiving unit 403 transmits to the server 101 the loading status information such as whether it is being charged at the home station 105, or whether it has been harvested or is empty, and which storage basket 307 is on the way to somewhere. (K1).

In this way, the server 101, which has obtained all the information necessary for collecting the harvested storage baskets 307, makes a judgment unit 203 based on the information, and determines the position of the harvested storage basket 102 that has completed the harvesting operation. After unloading the basket 307, it is determined which collection robot 103 will take the storage basket 307 placed by the harvesting robot 102 and which route it will take to collect the storage basket 307 (S5). These are recovery instruction information, which the first transmission/reception unit 201 outputs to the harvesting robot 102 .

The harvesting robot 102, which receives the collection instruction information from the server 101 by the second transmission/reception unit 303, moves to the position indicated by the collection instruction information by the self-propelled carriage unit 308, and when it arrives, the first lifter mechanism 305 is lowered. The harvested storage basket 307 is lowered to the ground (R3). When the harvesting robot 102 unloads the storage basket 307 is transmitted from the second transmitting/receiving unit 303 to the server 101 (harvesting status information including the completion signal and the current position information), the server 101 transmits the information from the first transmitting/receiving unit 201 to the collection robot 103. Send collection instruction information (S6). Based on the collection instruction information received by the third transmission/reception unit 403, the collection robot 103 goes to the instructed place by the self-propelled carriage unit 404, lifts the harvested storage basket 307 placed on the ground, and lifts the second lifter mechanism 401. Let the child pick it up (K2).

Here, the collection instruction information to be transmitted to the harvesting robot and the collection instruction information to be transmitted to the collection robot are assumed to be the same information. The collection instruction information to be sent to the collection robot should include the information of the placement position where the harvest robot is to be placed separately from the collection station. information should be included.

After that, the collection robot 103 moves to the cargo collection station 104 through the route instructed by the server 101 by the self-propelled carriage unit 404 (K3), and the harvested storage basket 307 is placed in the harvested storage basket stock unit 501 of the cargo collection station 104 by the second lifter. The mechanism 401 is lowered (K4).

Next, the third transmission/reception unit 403 of the collection robot 103 transmits the loading status information including the completion signal and the current position information to the server 101 . The server 101, which receives the loading status information including the completion signal of the collection robot 103 at the first transmission/reception unit 201, obtains the inventory information of the empty storage basket stock unit 502 of the collection station 104 (S7). At this time, if there is a sufficient storage basket distribution volume and a sufficient empty storage basket stock capacity, and in the worst case of stockout, if it is acceptable for the collection robot 103 to be in a standby state in front of the collection station 104, this inventory information can be obtained. can be omitted.

If there is an inventory of empty storage baskets, the determination unit 203 determines an empty storage basket supply and distribution plan for where to load the empty storage baskets 307 and direct the recovery robot 103 (S8). These are storage basket supply instruction information, and the server 101 transmits the storage basket supply instruction information from the first transmission/reception unit 201 to the collection robot 103 . The collection robot 103 that has received the storage basket supply instruction information by the third transmission/reception unit 403 goes to the empty storage basket stock unit 502 of the cargo collection station 104 by the self-propelled carriage unit 404, and lifts the empty storage basket 307 by the second lifter mechanism 401. , is loaded (K5).

After that, the collection robot 103 moves to the place instructed by the server 101 by the self-propelled carriage 404, and when it arrives, the second lifter mechanism 401 descends to unload the empty storage basket 307 (K6). The collecting robot 103 transmits the loading status information including the completion signal and the current position information to the server 101 from the third transmitting/receiving section 403 .

The server 101 receives the loading status information including the completion signal and the current position information through the first transmission/reception unit 201, and the first transmission/reception unit 201 transmits storage basket supply instruction information to the harvesting robot 102 (S9).

The harvesting robot 102 that has received the storage basket supply instruction information by the second transmitting/receiving unit 303 moves to the designated place by the self-propelled carriage unit 308, and lifts the empty storage basket 307 placed on the ground by the first lifter mechanism 305. Hold (R4). The harvesting robot 102 waits for the next instruction from the server 101 (R5). This state is the state at the time of photographing the harvesting area of R1 or immediately before moving to the harvesting/storage operation of R2. After that, this process is repeated.

The operation flow described above is a basic pattern, and in practice, the plurality of harvesting robots 102 and the plurality of collecting robots 103 operate in a composite manner according to the harvesting situation, and are operated so as to operate efficiently as a whole.

For example, in the operation flow diagram of FIG. 8, the operations of one harvesting robot 102 and one collecting robot 103 are shown separated into a basic flow for harvesting, a basic flow for collecting, and a basic flow for supplying an empty storage basket for easy understanding. , may actually be operated together. In other words, the server 101 constantly grasps the situation of the plurality of harvesting robots 102 and the collection robots 103 one by one, thereby simultaneously determining the harvesting target area, determining the collection order and collection route, and planning the delivery of empty storage baskets. conduct. For example, if the point at which the harvested storage baskets 307 should be collected and the point at which the empty harvested baskets 307 should be supplied are relatively close at a certain point, the collection robot 103 supplies the empty storage baskets 307 on the outbound route from the collection station 104 . However, if the harvested storage baskets 307 are collected on the return route to the cargo collection station 104, the collection robot 103 runs almost empty, and the collection robot 103 does not waste time waiting. Similarly, after unloading the harvested storage basket 307 , the harvesting robot 102 immediately moves to a nearby empty storage basket placing point, picks up the storage basket 307 , and picks up the storage basket 307 . If the harvesting robot 102 enters the forest and starts the harvesting work, the harvesting robot 102 does not waste time waiting.

Since there is no pairing relationship between the harvesting robot 102 and the collecting robot 103 and the storage basket 307 is once placed on the ground, the harvesting robot 102 and the collection robot 103 do not need to synchronize the handing over operation of the storage basket 307 . Furthermore, since there may be a considerable time lag between the placement of the harvested storage basket 307 by the harvesting robot 102 and the picking up by the collection robot 103, the server 101 can develop such efficient operation programming. As a result, the minimum number of collecting robots 103 can be used for the number of harvesting robots 102 . This is because basically the harvesting span from the start of harvesting to the completion of harvesting by the harvesting robot 102 is long, while the recovery supply span from the collection of the harvested storage basket 307 to the installation of the empty storage basket 307 by the recovery robot 103 is short. .

In this harvesting robot system, the server 101 optimally programs the harvesting plan, the collection plan, and the storage basket supply plan so that the harvesting robot 102 and the collecting robot 103 do not have to wait. Game theory, queuing theory, etc. may be applied to the process.

Alternatively, AI or the like may be applied to make a certain degree of prediction in advance and program the harvest schedule. However, unlike industrial products, fruits do not grow uniformly and uniformly, and fluctuate greatly depending on the weather and other factors. Predictions can be made at the beginning, but assuming that predictions are not correct, it is also necessary to have a feedback mechanism that allows the program to be changed flexibly according to the latest information.

Moreover, if the harvesting robot 102 has the third lifter mechanism 309 as shown in FIG. Can be stored. That is, the collection lot size of the collection robot 103 is increased, and the collection frequency of the storage basket 307 by the collection robot 103 is reduced. That is, the number of required harvesting robots 103 is reduced. If possible, the stock amount should be larger than the harvest amount per one furrow 705 .

In this case, it is possible to predetermine the positions at which the storage baskets 307 stacked in a one-to-one correspondence with each ridge 705 are to be placed. In that case, the storage basket 307 may not be placed directly on the ground, but may be provided with a dedicated fixed storage basket placement table. The storage basket placement table fixedly installed enables the delivery of the storage basket 307 to and from each robot reliably and easily. In either case, if it is determined that the presence of an empty storage basket 307 at that location is a steady state, management becomes easier and more efficient operation can be achieved. That is, as soon as the harvest target area, that is, the ridge 704 is determined, the harvesting robot 102 picks up the empty storage basket 307 at the position corresponding to the ridge 704, enters the ridge road 705, harvests, and finishes the harvest. Then, the harvested storage basket 307 is placed in the same position as before, and after sending a harvesting completion signal to the server 101, a new harvesting instruction is received from the side dish server 101, and the next harvesting target area is paired with it. head to the place where the empty storage basket 307 is stored. The collection robot 103 places an empty storage basket 307 at a fixed position that has become vacant due to the collection of nearby harvested storage baskets 307, and picks up the harvested storage basket 307 in response to a recovery instruction from the server 101 on the return trip. give. After that, after issuing a collection signal to the server 101 , it goes to the collection station 104 . After that, this process is repeated.

The harvesting robot 102 is equipped with a plurality of storage baskets 307 and sequentially stores them in each storage basket 307. As shown in FIG. Although the structure in which harvest objects are stored using the gaps generated by sequentially lifting the empty storage baskets 307 that are arranged and the third lifter mechanism 309 remains, it is not limited to this. In short, it is sufficient that a plurality of storage baskets 307 can be mounted, objects 306 can be stored in each of them, and the stacked storage baskets 307 can be loaded and unloaded from the ground or a fixed stand.

In an extreme case, the storage baskets 307 may not be stacked, and only one storage basket 307 may be used, in which case the third lifter mechanism 309 is unnecessary. However, since the harvesting span is shortened, the recovery efficiency is deteriorated, and the number of required recovery robots 103 is increased. Of course, if the ridge length is short and the object 306 to be harvested is a small fruit such as a cherry, this is not the case because one storage basket per ridge road can be used.

Next, the option group in the present embodiment will be sequentially explained using FIGS. 9 to 18. FIG. These option groups are based on the basic operation flow of harvesting, collecting, and supplying baskets shown in FIG. 8, to more effectively improve, reinforce, or supplement the functions of the harvesting robot system. The option group may be appropriately combined with the embodiment within a consistent range.

FIG. 9 is a flow diagram of tool change operation in the embodiment of the present invention.

When a plurality of types of crops are grown in the cultivation yard 701, the harvesting robot 102 generally attaches a dedicated harvesting hand corresponding to the characteristics of each crop to the tip of the manipulator of the harvesting unit 304 to perform the harvesting work. Various harvesting hands form a corner of the work tool group, and their replacement is generally called tool change. When this option is applied, the manipulator of the harvesting unit 304 of the harvesting robot 102 and the harvesting hand at the tip have a structure in which the harvesting hand can be automatically exchanged by a mechanism called an auto tool changer of an industrial robot, for example. A detailed description of the automatic tool changer is omitted.

In the tool change operation flow, as exemplified in FIG. 9, the server 101 receives input of tool change given conditions in advance (S101). The operator 106 operates the input/output means 107 to input the tool change conditions to the input/output unit 202 . The tool change condition input by the operator 106 is information that should be given in advance in order for the judgment unit 203 to decide to change the tool, such as planting information and tool code information. The planting information is interlocked with the farm map information to identify the position of the harvesting area, that is, the ridge number, for each variety of the object 306 cultivated in the cultivation yard 701, and to specify the type of harvesting hand to be applied to each variety. Specified in the tool code. The tool code information is a code number assigned to each type of harvesting hand. Using this tool code, the server 101 configures tool change information indicating which harvesting hand the harvesting robot 102 is to be equipped with.

In the state where the tool change given condition is inputted, the judgment section 203 of the server 101 to which the image data with the map address is inputted from step R1 in FIG. 8 determines the harvest target area. At the same time, the tool code of the harvesting hand corresponding to the type of the target object 306 determined as the harvesting target area is determined (S102). On the other hand, the server 101 obtains the robot state information including the tool state from the harvest robot 102 at the same time as the image data with the map address. If the harvesting hand currently attached to the harvesting robot 102 and the tool code of the determined harvesting hand match (Y in S103), the harvesting instruction information is output according to the basic operation flow for harvesting, collecting, and supplying a storage basket in FIG. The harvesting robot 102 harvests and stores (R2). If the tool codes do not match (N of S103), the server 101 outputs a request signal to the home station 105 to inquire of the home station 105 about the availability of waiting positions and the stock status of tools including harvesting hands. The home station 105 returns vacant position information, that is, which docks are currently vacant, and tool inventory information, that is, how many tools of which codes are left (H101). The determination unit 203 of the server 101, having confirmed the vacancy of the standby position of the home station 105 and the inventory of the necessary tools, decides to replace the tools (S104). The server 101 contains tool change information and issues to the harvesting robot 102 instructions to go to the home station indicating which dock to go to. The harvesting robot 102 that receives it moves to the home station 105 and enters the designated dock (R101). After arriving at the home station 105 , the harvesting robot 102 transmits an arrival signal to the server 101 . The server 101 receiving it transmits tool change information to the home station 105 (S105). The home station 105 that has received the tool interlocks with the harvesting robot 102 and replaces it with the designated tool (H102). When the home station 105 notifies the server 101 of the completion of tool change as a tool change completion signal, the server 101 transmits harvesting instruction information to the harvesting robot 102 (S106). Thereafter, the flow returns to the basic operation flow for harvesting, collecting, and supplying baskets shown in FIG.

The tool change H102 in the home station 105 may be based on the tool change information from the harvesting robot 102 instead of the tool change information from the server 101 . However, the home station 105 needs to have a transmitting/receiving unit capable of directly transmitting/receiving information to/from the harvesting robot 102 via a wireless LAN or the like, or a structure capable of exchanging information with a contact. Similarly, the tool change completion signal may also be sent from the harvesting robot 102 to the server 101 .

FIG. 10 is a flow chart of charging operation in the embodiment of the present invention. The harvesting robot 102 and the collection robot 103 have self-propelled carriages 308 and 404, and are equipped with rechargeable batteries such as lead-acid batteries or lithium-ion secondary batteries as power sources. These are used not only as a power source for self-running, but also as other power sources such as harvesting manipulators, and need to be charged periodically. An automatic charging device that automatically performs charging is generally put into practical use in AGVs (Automatic Guided Vehicles), so a description thereof will be omitted.

In FIG. 10, when the remaining battery level of the rechargeable battery reaches a predetermined warning level, the harvesting robot 102 or the collecting robot 103 transmits robot status information including the charging status to the server 101 (R151). The warning level must be at least sufficient to stop the work and reach the home station 105 anywhere in the farm. The server 101, which has received the battery exhaustion warning in the robot status information, sends a request signal to the home station 105 to inquire about availability of a dock equipped with an automatic charging device (S151). If there is an empty dock, the home station 105 returns the empty dock number to the server 101 as empty position information (H151). The determination unit 203 of the server 101 that has confirmed the availability of the dock determines charging (S152). Next, the server 101 transmits to the harvesting robot 102 or the collecting robot 103 home station go instruction information including charging instruction information indicating which dock to go to. The harvesting robot 102 or collecting robot 103 that has received the instruction information to go to the home station moves to the designated dock of the home station 105 (R152). After the harvesting robot 102 or collecting robot 103 arrives at the home station 105, it transmits an arrival signal to the server 101, and the server 101 sends charging instruction information to the home station 105 (S153). The home station 105 that received the charging instruction information starts charging (H152). After completing charging, the home station 105 sends a charging completion signal to the server 101 . The server 101 that has received the charging completion signal transmits standby instruction information to instruct the harvesting robot 102 or the collection robot 103 to wait (S154). The harvesting robot 102 or collecting robot 103 that has received the standby instruction information waits at the specified position (R153). If the harvesting robot 102 or the collecting robot 103 has a job, the determination unit 203 of the server 101 allocates the job and gives the harvesting instruction information and other instruction information to the harvesting robot 102 or the collecting robot 103 (S155).

The charging instruction information from the server 101 to the home station 105 may be sent from the harvesting robot 102 or the collecting robot 103 . However, the home station 105 must have a transmitting/receiving device or a structure capable of exchanging information with a contact. Similarly, a charging completion signal may be sent from harvest robot 102 or collection robot 103 to server 101 .

This charging operation may be systematically performed at the same time as the tool change operation in FIG.

In addition, the charging operation is started when the remaining battery level reaches the warning level. In order to avoid this, it is possible to systematically dock at a good work breakpoint early.

FIG. 11 is a flowchart for determining a harvest target area according to the embodiment of the present invention.

There are various procedures for determining the harvest target area (S3) in the basic operation flow chart for harvesting, collecting, and supplying a storage basket in FIG. shown in

As exemplified in FIG. 11, when image data with a map address is input to the server 101 from the harvesting robot 102 that has photographed the harvesting area and specified its own position (R1), the determination unit 203 first detects an unnecessary background image. Noises such as leaves and branches are removed, and only fruits are extracted (S201). Since various image processing methods have been proposed for this method, detailed description thereof will be omitted. Next, the judging section 203 ranks the degree of maturity of each extracted fruit based on the maturity ranking criteria (N201) previously input by the operator 106 through the input/output means 107 (S202). The maturity ranking criteria are, for example, most fruits continuously change from the immature first color to the fully ripe second color, so if the period is divided into 10 stages, the ranking is 10 ranks. Become. The first color is green, for example, and the second color is red, for example. The combination of the first color and the second color varies from fruit to fruit. In addition, depending on the fruit, it may be divided into the first color and the second color depending on the location in one fruit during maturation, or the color in between may change continuously. Also, as they mature, they generally increase in size. The maturity ranking standard is obtained by comprehensively classifying the features according to the maturity level according to the characteristics of the target object 306 . The determination unit 203 compares these rank reference patterns with the target object 306 and ranks the target object 306 based on the rank of the closest reference pattern.

Next, since the harvestable rank or higher among the ranks is input in advance as a threshold value (N202), the determination unit 203 extracts the fruits of the threshold value or higher from the ranked fruits, and harvests them for each unit harvesting area. Estimate the amount (S203). Basically, based on the estimated yield, the ridge road with the highest yield is set as the target harvest area, and the harvesting robot 102 serves food in order. ). For example, rather than uniformly counting the number of fruits with ranks equal to or higher than the threshold, the fruits with higher maturity ranks are weighted (N203). This is because it is necessary to harvest the fruits with a high degree of maturity as soon as possible before the degree of maturity progresses and the product appeal declines. A weighting coefficient for each rank is input to the server 101 from the input/output means 107 in advance.

On the other hand, a certain variety may be prioritized according to a harvest schedule that is prepared in advance and input to the server 101 through the input/output means 107 (N204). This is based on the judgment that, since the harvest schedule reflects the needs of the market, it is better to give priority to varieties that sell well. In that case, a large weighting coefficient is set for the product that sells well. In addition, if a shipping plan for each variety is determined in advance based on a contract with a large-scale delivery destination of agricultural products, the quantity of the variety may be insufficient if the crop is normally cut at a predetermined threshold. In that case, some young fruits may be mixed, so it is necessary to lower the threshold rank and satisfy the predetermined quantity. Conversely, if you cut at a predetermined threshold normally, the quantity of the variety may exceed the plan. In that case, it is necessary to raise the rank of the threshold value and reduce the planned number of harvests. Farmers must avoid discarding crops. Reflecting these circumstances, the determination unit 203 corrects the evaluation value. Based on the result, the determination unit 203 finally determines the harvest target area (S205). After that, the process proceeds as shown in the basic operation flow chart of harvesting, collecting, and supplying baskets in FIG.

Although the above-described fruit maturity determination method is performed based on image data captured by the harvesting robot 102, determination may be performed based on other information. For example, instead of the harvesting hand of the harvesting unit 304 of the harvesting robot 102, a saccharimeter probe may be attached to measure the sugar content of the fruit, and the harvest target area may be determined based on the sugar content.

Also, although the above operation flow has been explained using an example of fruits that tend to have a colored image, the same method can be applied to other crops. However, depending on the type of crop, it may be determined not only by color but also by shape, size, and the like. The key is to select a criterion that makes it easy to determine the degree of maturity according to the characteristics of the crop.
FIG. 12 is a flow chart of shooting rotation operation according to the embodiment of the present invention.

In the step of photographing image data with a map address in the flowchart of FIG. 8, first, the harvesting robot 102 photographs based on photographing instruction information from the server 101 (R1). This routine is mainly used for crops that are harvested only once each year within the same harvesting area for a limited period of time. On the other hand, in some high-level greenhouse cultivation facilities such as cherry tomatoes and strawberries, harvesting continues throughout the year or for several months. In these agricultural crops, countless fruits grow in order from one seedling or stem during the harvesting period, so the same ridge is harvested many times during the harvesting period. That is, a fruit that is not harvested because it is immature at a certain harvesting operation reaches maturity after, for example, one week. By utilizing this characteristic, it is possible to omit the scanning movement of the harvesting robot 102 only for photographing. The operation will be described below with reference to FIG.

In FIG. 12, the harvesting robot 102 that has received the harvesting instruction information from the server 101 performs harvesting and storage (R2). More specifically, the harvesting/storage by the harvesting robot 102 is performed by first recognizing the object 306 by the imaging unit 301, determining whether or not harvesting is possible, and identifying the position of the harvestable object 306 (R251). Harvesting is performed in the section 304 (R252), and storage is carried out in the storage basket 307 (R253). When all the loaded storage baskets 307 are filled with the objects 306 or when the harvesting of ridges instructed by the server 101 is completed, the harvesting robot 102 transmits a harvesting completion signal to the server 101 (R254). In the above flow, in this option, the image of the object 306 photographed for harvesting the crop 306 is sent to the server 101 as image data with a map address each time (R251). The harvesting robot 102 selects the target object 306 to be harvested in the unit harvesting area at the time of harvesting, so the entire unit harvesting area is photographed first, so information on all fruits including immature fruits can be obtained.

Since the image data obtained by the server 101 includes the fruit to be harvested by the harvesting robot 102, the determination unit 203 first deletes the harvested fruit from the image data (S251). Harvest success information about which fruits have been harvested or failed is obtained from the harvesting robot 102 .

The server 101 accumulates image data obtained by processing the image data sent from the harvesting robot 102 (S252). When a certain amount is accumulated, the determination unit 203 creates a fruit ripening distribution map (S253). A certain amount is, for example, when the harvesting work for the day is completed, or when the harvesting of one ridge is completed. The maturity distribution map is data indicating the degree of maturity ranking of all fruits and indicating in which region the fruits with what level of maturity are distributed. From this maturity distribution map, the determination unit 203 estimates, for example, the total amount of harvestable fruit on a certain ridge after a certain period of time has elapsed (S254). A threshold for harvesting at this time is separately input. Based on this estimated amount, the determination unit 203 determines a harvest target area after a certain period of time has elapsed (S255). Based on these, the determination unit 203 formulates a harvesting schedule after a certain period of time (S256), and the server 101 transmits harvesting instruction information to the harvesting robot 102 at any time based on this harvesting schedule. By repeating this as a rotation at the fixed time period, image data with map addresses are sequentially obtained at the same time as harvesting, so that the harvesting robot 102 does not need to bother scanning the farm just for photographing. Therefore, the working efficiency of the harvesting robots 102 is improved, and the number of harvesting robots 102 can be reduced accordingly.

Also, in FIG. 12, the maturity distribution map is created after the image data is accumulated, but the distribution map may be created sequentially after obtaining the image data.

The rotation span is appropriately set to a span with high operational efficiency and high prediction accuracy based on the characteristics of the fruits and farms and past data.

FIG. 13 is a learning function application maturity determination flow diagram according to the embodiment of the present invention.

In the flowchart of the basic operation of harvesting, collecting, and supplying a storage basket in FIG. 8, the determination of the harvest target area by the determination unit 203 is performed based on the harvest conditions including the harvest determination criteria input in advance. More specifically, maturity ranking criteria and harvestable thresholds are given as harvest conditions, as illustrated in the harvest target area determination flow diagram in the embodiment of the present invention in FIG. 11 . In this option, the determination unit 203 sets the maturity level ranking criteria and thresholds based on the accumulated image data with map addresses instead of setting them based on human experience and intuition (S300). The operation will be described below with reference to FIG.

In FIG. 13, the server 101 accumulates image data with map addresses photographed by the harvesting robot 102 (S301). Images of individual crops randomly extracted from the accumulated image data regardless of maturity are displayed on the input/output means 107 (N301). Seeing this, the operator 106 judges whether or not the crop is suitable for harvesting (P301), labels it with two options, and inputs it through the input/output means 107 (N302). The server 101 associates the image data of the crop with the corresponding label (S302), and accumulates the associated data (S303). When sufficient accumulation is obtained, the determination unit 203 of the server 101 evaluates them based on predetermined determination factors such as hue, brightness, saturation, degree of color mixture (dispersion), size, shape, and the like. Generate a maturity evaluation function by index. Next, an evaluation function that divides the ranks of whether or not harvesting is possible most clearly is extracted, and the boundary is determined as a threshold value (S304). These are newly set as harvesting conditions, applied to image data with a map address from the harvesting robot 102 , the determination unit 203 determines a harvesting target area ( S<b>3 ), and transmits harvesting instruction information to the harvesting robot 102 .

As the evaluation function, a classifier using machine learning such as a neural network, a Bayesian classifier, or an SVM (Support Vector Machine) is typically used. Then, the classifier is subjected to learning processing using teacher data that associates the image of the target with the maturity of the target, and can be harvested for each unit harvest area using the learned classifier. The amount of objects will be estimated.

Note that the operator 106 may intervene at any time during the process of generating the evaluation function and the threshold to make corrections.

FIG. 14 is a flow diagram during manual programming according to an embodiment of the present invention. In the flowchart of basic operations for harvesting, collecting, and supplying a storage basket in the embodiment of the present invention in FIG. Although the judgment unit 203 of the server 101 makes a decision based on given conditions, this option allows the operator 106 to make this decision depending on the situation. In an actual farm, if the harvest target area is automatically determined, inconvenience may occur. If the inconvenience is not due to a single occurrence, the factor is added to the S204 correction term.

The operation flow will be described below with reference to FIG.

As shown in FIG. 14, in this option, a new judgment child "Is the determination of the area to be harvested automatic?" (S351) is newly added. YES or NO is a parameter, which the operator 106 inputs in advance. In the case of YES, the harvesting operation proceeds according to the flow of FIG. 8 as usual. If the parameter is NO, the image and area information of the image data with map address are displayed on the input/output means 107 (N351). Seeing this, the operator 106 determines in which harvesting area the object 306 is to be harvested (P351), and inputs it to the input/output means 107 (N352). The server 101 receiving this determines the input area as a harvesting target area (S3) and transmits harvesting instruction information to the harvesting robot 102 .

FIG. 15 is a flowchart of harvest prediction management processing according to the embodiment of the present invention.

In the harvesting robot system according to the embodiment of the present invention, the server 101 optimally programs the harvesting plan, the collection plan, and the storage basket supply plan so that the harvesting robot 102 and the collecting robot 103 are not left waiting. Game theory, queuing theory, etc. may be applied to the process. Harvesting schedules can also be programmed based on a certain amount of prediction in advance based on a huge amount of past data. An example thereof will be described with reference to FIG.

In FIG. 15, all image data with map addresses photographed by the harvesting robot 102 are stored in the mass storage device 204 (S401). The shooting date and time are also recorded as attached information in these image data. Based on the accumulated images and the time and place information associated with the images, the determination unit 203 systematically analyzes the image data in time and space (S402). For example, focusing on a specific agricultural product in a certain location, changes over time can be extracted, and a maturity curve of the agricultural product on the time axis can be obtained. You can also see when the crop was harvested. In other words, this data can be used to predict the future maturity evolution of a given crop. Next, the determination unit 203 predicts future maturity progress for each unit harvest area, and creates a time-series maturity rank prediction distribution map (S403). The determination unit 203 estimates the daily yield in the near future, for example, the next week, based on this forecast distribution map (S404). Next, the determination unit 203 determines an optimized harvesting schedule in the near future based on the estimation (S405), and transmits harvesting instruction information to the harvesting robot 102. FIG. Optimization means, for example, correcting the harvest schedule so that the daily harvest amount for the next week is leveled. Farmers can predict the amount of harvest in the near future, making it easier to make shipping plans. For farm management, a shortage of shipments leads to lost sales opportunities, and an excess of shipments leads to disposal losses.

The harvest schedule in the near future does not have to be a weekly schedule, and an optimal plan span such as a monthly schedule is set as appropriate according to the characteristics of agricultural products, market needs, and the like.

Furthermore, if the harvesting robot 102 is equipped with various sensors in addition to the photographing unit 301 and various types of information other than the image data are collected at the same time, the maturity prediction accuracy is further improved. Various sensors are a thermometer, a hygrometer, an illuminance meter, a gas concentration meter, and a soil sensor. The harvesting robot 102 is equipped with one of the above sensors or a combination of a plurality of sensors depending on the purpose. All of the items measured by these sensors have a great influence on the growth of plants. For example, generally the higher the temperature and the higher the carbon dioxide concentration, the faster the growth. The harvesting robot 102 attaches the information obtained by these sensors to image data with a map address and sends it to the server 101 . The judging unit 203 creates a maturity level rank expected distribution map by adding these information to the image data spatio-temporal analysis and future maturity level progress prediction. The following steps conform to the flow described with reference to FIG. 15 above.

FIG. 16 is a real-time feedback processing flowchart in the embodiment of the present invention.

The options shown in FIG. 15 allow the harvest forecast to be made for the near future, eg, the next week, and thus the weekly harvest schedule. This will improve the efficiency of all plantation operations, including harvesting. However, unlike industrial products, agricultural products do not grow uniformly and uniformly, and fluctuate greatly depending on the weather and other factors. Predictions can be made at the beginning, but assuming that predictions are not correct, it is also necessary to have a feedback mechanism that allows the program to be changed flexibly according to the latest information. An example thereof will be described with reference to FIG.

In FIG. 16, the flow from S401 to S405 is the same as in FIG. In this option, the harvesting robot 102 photographs the harvesting target area before actually harvesting based on the harvesting schedule. Harvesting robot 102 sends the most recent image data with a map address to server 101 . The determination unit 203 of the server 101 compares the obtained image data with map address and the maturity forecast map at the time of planning the harvest schedule in the near future (S451). The initial harvest plan is executed (S453). If the deviation exceeds the allowable value, the harvest target area is corrected (S452).

The operator 106 may directly perform the correction via the input/output means 107. FIG.

FIG. 17 is a flowchart of cultivation management processing according to the embodiment of the present invention.

Until now, the target of the implementation work of this harvesting robot system was limited to harvesting work, but with this option, the target of implementation work is expanded to agricultural work other than harvesting. Agricultural work other than harvesting includes, for example, pruning, chemical spraying, fruit thinning, and leaf thinning. There are many conventional harvesting hands equipped with scissors for harvesting, and these can be used as they are or by slightly arranging their forms, the harvesting robot 102 can be used for pruning, fruit picking, leaf picking, and the like. If the picked leaves and branches, thinned immature fruits, etc. can be stored in the storage basket 307 as they are, the work of cleaning up the residues can be reduced. Chemical application can be handled by replacing the harvesting hand with a spray gun and equipping a chemical tank instead of a storage basket. In this way, this harvesting robot system can be applied to agricultural work other than harvesting simply by exchanging tools, and has a wide range of applications. Tools in this case include not only various harvesting hands but also pruning scissors and spray guns.

A cultivation management flow including harvest centering on tool exchange will be described with reference to FIG. 17 .

17, the same reference numerals are used for the same elements as in FIG. 9, and description thereof is omitted. Information necessary for tool exchange is given in advance as given conditions (S501). Farm work information other than harvesting is added from the given conditions of S101 in FIG. Note that the tool code information is extended to tools used for agricultural work other than harvesting hands. Farm work information other than harvesting includes, for example, pruning, thinning, and state pattern data of trees to be thinned, and judgment criteria. The content is based on the harvest conditions. It should be noted that chemical application is not determined from the state of the photographed tree, but is performed multiple times at a predetermined time each year, so that time is input as agricultural work information. The period of chemical application is given some latitude, and the determination unit 203 of the server 101 determines execution of chemical application when the harvesting robot 102 has spare capacity while avoiding the busy season of other agricultural work such as harvesting.

The determination unit 203 of the server 101 that has received the image data with the map address from the harvesting robot 102 determines the farm work to be performed and the target area based on the given conditions (S502). At the same time, robot state information including the tool state is received from the harvesting robot 102, and it is determined whether or not the determined farm work can be performed with the tool (S503). If OK, the server 101 transmits farm work instruction information including harvest work to the harvest robot 102 . If NG, a request signal is transmitted to the home station 105 . Thereafter, the tool change operation flow of FIG. 9 is followed. However, the target range of the decision to replace the tool is expanded to agricultural work tools other than the harvesting hand (S504).

FIG. 18 is an operation flow diagram of the other farm cooperation system according to the embodiment of the present invention.

Until now, the scope of implementation of this harvesting robot system has been limited to one farm managed by the server 101, but with this option, the target of information management can be expanded to other farms managed by other servers. and construct a system that cooperates with servers 108 of other farms via the Internet 109 (see FIG. 1). Except for some high-level cultivation facilities in greenhouses such as cherry tomatoes, the harvesting period for one agricultural product is usually limited to a certain time of the year, but the harvesting period varies depending on the type of agricultural product. For example, tomatoes and cucumbers are generally harvested from October to July, while eggplants and bell peppers are harvested from June to October, with the peak harvest time shifted. Similarly, strawberries are harvested from December to May, and blueberries from June to September. Apples are from August to November, and cherry trees are from May to July.

Since this harvesting robot 102 can handle the harvesting of many crops by exchanging tools, that is, harvesting hands, the harvesting robot 102 can be time-shared among the above crops. Through time sharing, the operating rate of the expensive harvesting robot 102 can be increased throughout the year, and a more cost-effective harvesting robot system can be constructed.

An operation example of the cooperation system with other farms will be described with reference to FIG. First, an annual harvest schedule is created (S551). The annual harvest schedule is input to the server 101 through the input/output means 107 by the operator 106 based on the past results and considering the current year's variable factors. Next, the determination unit 203 calculates the number of harvesting robots 102 and collection robots 103 that are required especially during the busy harvest season to achieve the plan, and determines whether or not it can be handled by its own resources (S552). If it is sufficient, the determination unit 203 formulates a robot food distribution plan with the robot on hand (S553), and sets it as a harvest condition. If it is not enough, a request signal is sent to the server 108 of the other farm to obtain the information of the other farm including the robot operation plan of the other farm (T551). Next, the determination unit 203 incorporates the surplus resources of other farms, formulates a robot catering plan (S554), and displays it on the input/output means 107 (N551). The operator 106 sees it, obtains approval for robot rental from other farms, and inputs confirmation (N552). Upon receipt of the input, the determination unit 203 confirms the robot food distribution plan (S555), sends the information of the own farm including the robot operation plan to the server 108 of the other farm, and sets it as a harvest condition (S556). After receiving the input of the information (T552), the server 108 of the other farm utilizes the information in the other farm.

In the above description, the server 101 draws up the robot food distribution plan, but the operator 106 may browse the status data of other farms and decide. For the present system, it doesn't matter, it doesn't matter. A mechanism that allows information to be exchanged and shared on a network between servers, that is, a cooperative system, is important.

Also, although the communication network between the servers is the Internet, a dedicated LAN line may also be used.

In addition, the flow above was explained based on the assumption of an annual plan, but if the distance between farms is relatively close, fine-grained time-sharing on a weekly or daily basis is also possible. is high. Furthermore, if cooperation between farms is expanded on the premise that the harvesting robot 102 is used for farm work other than harvesting, the management effect of the farm will be further enhanced.

[effect]
As described above, the harvesting robot system according to the present embodiment
A harvesting robot system including a harvesting robot 102 for harvesting an object in a farm and a server 101 configured to communicate with the harvesting robot 102,
The server 101
an input/output unit 202 that receives input of harvesting conditions including farm map information of a farm and harvesting criteria information for determining the harvesting time of an object;
a determination unit 203 that determines a harvest target area based on an image of an object photographed in each unit harvest area in the farm by the harvest robot 102 and harvest conditions;
and a first transmitting/receiving unit 201 that transmits harvesting instruction information including position information of the harvesting target area to the harvesting robot 102 .

As described above, in the harvesting robot system according to the present embodiment, a plurality of harvesting robots 102 are centrally managed by the server 101, and the harvesting robots 102 capable of traveling photograph the entire harvesting area in the farm with the photographing device 301 before harvesting. By doing so, it is possible to obtain precise ripening status information of crops throughout the farm, which cannot be obtained with a monitoring system using fixed fixed point cameras, and based on this information, the server 101 can always formulate an optimal harvest schedule. Then, based on the harvesting schedule from the server 101, it is possible to realize a flow in which the harvesting robot 102 receives harvesting instructions and performs harvesting. As a result, the harvesting work efficiency of the harvesting robot can be greatly improved.

In addition, the system can be supplemented with a wide variety of options as described above, which can further increase the operating efficiency of the harvesting robot, making it more cost-effective.

The harvesting robot system of the present invention is a cultivation management and harvesting system that collects cultivation status information extensively and precisely, formulates and executes an optimal work plan based on that information, and efficiently collects waste such as harvests and weeds. It has a recycling-type logistics system that collects and not only automates the harvesting work of facility-type fruit cultivation farms, but also the harvesting work of almost all agricultural crops, including open-field cultivation and standing trees, and various other agricultural work other than harvesting. It can also be applied to automation of applications.

101 Server 102 Harvesting Robot 103 Collection Robot 104 Collection Station 105 Home Station 106 Operator 107 Input/Output Means 108 Servers of Other Farms 109 Internet 201 First Transmission/Reception Unit 202 Input/Output Unit 203 Judgment Unit 204 Mass Storage Device 301 Photographing Unit 302 First Self-position detecting means 303 Second transmitting/receiving section 304 Harvesting section 305 First lifter mechanism 306 Object 307 Storage basket 308 Self-propelled carriage section 309 Third lifter mechanism 401 Second lifter mechanism 402 Second self-position detecting means 403 Third transmitting/receiving section 404 Self-propelled carriage unit 501 Harvested storage basket stock unit 502 Empty storage basket stock unit 601 Tool change unit 602 Charging unit 701 Cultivation yard 702 Sorting and shipping yard 703 Main passage 704 Ridge 705 Ridge 706 Sorting bagging worker 707 Shipping conveyor 708 Stacking unit 709 Stacking unit

Claims (16)

A harvesting robot system including a harvesting robot for harvesting an object in a farm and a server configured to communicate with the harvesting robot,
Said server
an input/output unit that receives input of harvesting conditions including farm map information of the farm and harvest judgment criteria information for judging the harvest time of the object;
a determination unit that determines a harvest target area based on the image of the object photographed by the harvest robot for each unit harvest area in the farm and the harvest condition;
a first transmitting/receiving unit that transmits harvesting instruction information including position information of the harvesting target area to the harvesting robot;
Harvesting robot system. The harvesting robot is
a photographing unit that photographs the object and generates the image;
a first self-position detection means for detecting the current position of itself;
a second transmitting/receiving unit that transmits the image to the server in association with information related to a shooting position;
a harvesting unit that harvests the target based on the harvest instruction information received from the server;
The robotic harvesting system of claim 1, comprising: further comprising a home station for the harvesting robot to wait in the farm;
The home station is
a tool changer for exchanging the arm of the harvesting section of the harvesting robot from the first tool to the second tool based on the tool change information from the server;
a charging unit that charges a rechargeable battery mounted on the harvesting robot;
A third transmitting/receiving unit for transmitting to the server vacant position information indicating vacant dock information for mooring the harvesting robot, and tool inventory information including the types and numbers of tools currently held by the home station. and,
The harvesting robot system according to claim 1 or 2, comprising: The determination unit receives the vacant position information and the tool inventory information received from the home station, the tool change conditions for determining the tool to be attached to the arm used in the harvesting robot, and the tool change conditions received from the harvesting robot. determining whether to charge the harvesting robot or replace the tool based on robot status information including the remaining battery level of the harvesting robot and the tool status of the arm;
The first transmission/reception unit transmits home station-bound instruction information including charging instruction information and tool change information to the harvesting robot.
The harvesting robot system according to claim 3. The image is one or a plurality of images photographed so as to show the object present in the entire unit harvesting area.
A harvesting robot system according to any one of claims 1 to 4. The determination unit determines the harvest target area by estimating the amount of the target that can be harvested for each unit harvesting area based on the color of the target that appears in the image.
Harvesting robot system according to any one of claims 1 to 5. When the harvesting robot harvests the target object within the harvesting target area instructed by the server, the imaging unit captures an image of the unit harvesting area corresponding to the harvesting target area.
The harvesting robot system according to claim 2. The determination unit extracts the objects existing in the entire unit harvesting area from the image, creates a maturity rank distribution map of the objects in the unit harvesting area, and based on the maturity rank distribution map to estimate the harvestable amount of the unit harvesting area,
Harvesting robot system according to any one of claims 1 to 7. The determination unit has a classifier that has undergone learning processing using teacher data that associates the image of the object with the maturity level of the object,
estimating the amount of the target that can be harvested for each unit harvesting area using the discriminator, and determining the harvesting target area;
Harvesting robot system according to any one of claims 1 to 8. The input/output unit is configured to be able to receive input of the harvest target area by an operator.
Harvesting robot system according to any one of claims 1 to 9. The determining unit creates a harvesting schedule that defines a time for harvesting the object in the unit harvesting area based on the time-series images.
Harvesting robot system according to any one of claims 1 to 10. The determination unit harvests the target object in the unit harvesting area based on sensor data from at least one of a thermometer, a hygrometer, an illuminance meter, a gas concentration meter, and a soil sensor provided on the harvesting robot. create a harvest schedule that defines when;
Harvesting robotic system according to any one of claims 1 to 11. a collection robot that collects the target objects harvested by the harvest robot and transports them to a collection station in the farm;
Harvesting robotic system according to any one of claims 1 to 12. The determination unit determines the harvest target area based on the position and operating state of the harvest robot.
Harvesting robotic system according to any one of claims 1 to 13. Based on the image, the determination unit determines when agricultural work other than harvesting is required in the unit harvesting area, and based on the timing, transmits instruction information to the harvesting robot to go to the home station and tool change information. The harvesting robot system according to claim 3. The input/output unit acquires information of other farms including operation plans of harvesting robots of other farms,
The determination unit creates an operation plan for the harvest robot of the farm based on the information of the other farm.
Harvesting robotic system according to any one of claims 1 to 15.

Images