JP7075171B2 - Computer systems, pest detection methods and programs - Google Patents

Description

The present invention relates to a computer system for detecting pests by image analysis, a pest detection method and a program.

Conventionally, pests and pests generated in a field have been exterminated by a drone or the like. As such an extermination method, there is a configuration in which the position of a pest in a field is specified in real time, the drone is moved to the specified position, and the extermination is performed.

As an example of such a technique, a system for exterminating pests is disclosed based on a live image of a field taken in real time by a drone (see Patent Document 1). This system acquires the live image and the position information of the shooting point of the live image, and detects the presence or absence of the pest by comparing the specific image in which the pest or the like is taken in advance with the live image. When the system detects a pest, it outputs the position information of the shooting point where the live image was taken to a drone or the like to exterminate the pest.

Japanese Unexamined Patent Publication No. 2017-16271

However, in the configuration of Patent Document 1, the detection of pests is limited to live images taken in real time, and there is a possibility that it is not effective for captured images other than such live images.

An object of the present invention is to provide a computer system, a pest detection method and a program that facilitate detection and extermination of pests based on captured images, not limited to live images.

The present invention provides the following solutions.

The present invention is a computer system that includes a pest database in which feature points and amounts of pests are associated with and registered pest identifiers, and that pests are detected by image analysis.
An acquisition means for acquiring a photographed image of a field and position information of a photographing point, and
A detection means for detecting a pest by analyzing the acquired image and comparing the extracted feature points and feature amounts with a pest database to determine the identifier of the pest.
A position information output means for outputting the position information of the shooting point in the captured image in which the pest is detected, and
Based on the determined pest identifier and the output position information, the device or device for spraying or fertilizing the pesticide is specified, and the instruction output means for outputting the instruction for spraying or fertilizing the pesticide effective for the pest. ,
Provided is a computer system characterized by being provided with.

A computer system that has a pest database registered by associating the characteristic points and amounts of pests with the identifiers of the pests and detects the pests by image analysis acquires the captured images of the field and the position information of the captured points. , The acquired image is analyzed, and the extracted feature points and feature amounts are compared with the pest database to detect the pest and determine the identifier of the pest, and the shooting point in the captured image in which the pest is detected. Based on the determined pest identifier and the output position information, the device or device for spraying or fertilizing the pesticide is specified, and the instruction for spraying or fertilizing the pesticide is effective for the pest. Is output.

Although the present invention is in the category of systems, the same actions and effects are exhibited in other categories such as methods and programs according to the categories.

According to the present invention, it is possible to provide a computer system, a pest detection method and a program that facilitate detection and extermination of pests based on captured images, not limited to live images.

FIG. 1 is a diagram showing an outline of a pest detection system 1. FIG. 2 is an overall configuration diagram of the pest detection system 1. FIG. 3 is a diagram showing a flowchart of position information output processing executed by the computer 10. FIG. 4 is a diagram showing a flowchart of an instruction output process executed by the computer 10. FIG. 5 is a diagram schematically showing an example of a pest map created by the computer 10.

Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. It should be noted that this is just an example, and the technical scope of the present invention is not limited to this.

[Overview of pest detection system 1]
An outline of a preferred embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram for explaining an outline of a pest detection system 1 which is a preferred embodiment of the present invention. The pest detection system 1 is a computer system composed of a computer 10 and detecting pests by image analysis.

The pest detection system 1 includes drones, agricultural machinery, high-performance agricultural machinery, worker terminals (for example, smartphones, tablet terminals, personal computers) possessed by workers who grow crops, and other terminals such as other computers. Devices may be included. Further, the pest detection system 1 may be realized by one computer such as a computer 10, or may be realized by a plurality of computers such as a cloud computer.

The computer 10 is connected to a drone, an agricultural machine, a high-performance agricultural machine, a worker terminal, another computer, etc. via a public network or the like so as to be capable of data communication, and executes necessary data transmission / reception.

The computer 10 acquires a photographed image of the field and position information of the imaged point. The computer 10 acquires, for example, a photographed image of each point of the field by a drone. The computer 10 acquires, for example, the position information of the shooting point where the drone shot the shot image from the drone.

The computer 10 analyzes the acquired captured image and detects the pests appearing in the captured image. For example, as an image analysis, the computer 10 extracts feature points (for example, shape, contour, hue) and feature quantities (for example, statistical numerical values such as average, variance, and histogram of pixel values) of a captured image. The computer 10 detects the pests shown in the captured image based on the extracted feature points and feature quantities.

The computer 10 identifies the detected pest position information based on the acquired position information of the photographing point. For example, the computer 10 identifies the position information of the pest, assuming that the position information of the detected pest matches the position information of the photographing point. As a result, the computer 10 will identify the location of the detected pest in the field.

The computer 10 outputs the position information of the identified pest. At this time, the computer 10 outputs this position information to the above-mentioned drone, agricultural tool, high-performance agricultural machine, worker terminal, other computer and the like.

The computer 10 can also be configured to output an instruction for pesticide spraying or fertilization based on this position information. For example, the computer 10 moves the drone to the position of the pest based on this position information, and sends an instruction to spray the pesticide or fertilizer to the drone as a command by the pesticide spraying device or the fertilizer applying device possessed by the drone. The drone receives this command and sprays pesticides or fertilizers against the pests. As a result, the computer 10 outputs an instruction of pesticide spraying or fertilization based on this position information.

Next, the outline of the processing executed by the pest detection system 1 will be described.

The computer 10 acquires a photographed image of the field and position information of the imaged point as photographed data (step S01). The computer 10 acquires, for example, a photographed image of each point in the field by the drone. The computer 10 acquires, for example, the position information of the shooting point where the drone shot the shot image from the drone. The computer 10 acquires such a captured image and position information of a captured point as captured data.

The computer 10 analyzes the captured image and detects the pests appearing in the captured image (step S02). The computer 10 extracts feature points and feature quantities of a captured image, for example, as image analysis. The computer 10 detects the pests shown in the captured image based on the extracted feature points and feature quantities.

The computer 10 identifies the detected pest position information based on the acquired position information of the photographing point (step S03). The computer 10 identifies the position information of the pest, assuming that the position information of the detected pest matches the position information of the photographing point. As a result, the computer 10 will identify the location of the detected pest in the field.

The computer 10 outputs the position information of the identified pest (step S04). The computer 10 outputs this position information to, for example, the above-mentioned drone, agricultural tool, high-performance agricultural machine, worker terminal, other computer, and the like.

The above is the outline of the processing executed by the pest detection system 1.

[System configuration of pest detection system 1]
Based on FIG. 2, the system configuration of the pest detection system 1 which is a preferred embodiment of the present invention will be described. FIG. 2 is a diagram showing a system configuration of a pest detection system 1 which is a preferred embodiment of the present invention. In FIG. 2, the pest detection system 1 is a computer system composed of a computer 10 and detecting pests by image analysis.

The computer 10 is connected to a drone, an agricultural machine, a high-performance agricultural machine, a worker terminal, another computer, etc. via a public network or the like so as to be capable of data communication, and executes necessary data transmission / reception.

The pest detection system 1 may include a drone, a farm tool, a high-performance agricultural machine, a worker terminal, other computers and the like, and other terminals and devices (not shown). Further, the pest detection system 1 may be realized by one computer such as a computer 10, or may be realized by a plurality of computers such as a cloud computer.

The computer 10 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and enables communication with other terminals and devices as a communication unit. A device for this purpose, for example, a Wi-Fi (Wiress-Fidelity) compatible device compliant with IEEE802.11 and the like. Further, the computer 10 includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, and a memory card as a recording unit. Further, the computer 10 includes various devices and the like that execute various processes as a processing unit.

In the computer 10, the control unit reads a predetermined program to realize the photographing data acquisition module 20, the pest data output module 21, and the command output module 22 in cooperation with the communication unit. Further, in the computer 10, the control unit reads a predetermined program and cooperates with the recording unit to realize the recording module 30. Further, in the computer 10, the control unit reads a predetermined program, and in cooperation with the processing unit, the image analysis module 40, the position identification module 41, the countermeasure data identification module 42, the pest map creation module 43, and the command creation module. Achieve 44.

[Location information output processing]
The position information output process executed by the pest detection system 1 will be described with reference to FIG. FIG. 3 is a diagram showing a flowchart of position information output processing executed by the computer 10. The process executed by each of the above-mentioned modules will be described together with this process.

The shooting data acquisition module 20 acquires the shooting image of the field and the position information of the shooting point as shooting data (step S10). In step S10, the photographing data acquisition module 20 is, for example, a photographed image taken by the drone at each of a plurality of preset shooting points in the field, and position information of each shooting point where the drone has taken a photographed image. And is acquired as shooting data. The drone shoots directly under itself as a shot image. That is, the drone captures the captured image from a position perpendicular to the field. The drone shoots a shot image at the shooting point and acquires its own position information from GPS or the like. The drone treats the acquired position information of itself as the position information of the shooting point. The drone transmits the captured image and the position information of the captured point to the computer 10 as captured data. The shooting data acquisition module 20 acquires shooting data by receiving the shooting data transmitted by the drone. As a result, the computer 10 acquires the photographed image of the field and the position information of the photographing point.

The image analysis module 40 analyzes the captured image based on the captured data (step S11). In step S11, the image analysis module 40 extracts feature points and feature quantities in the captured image. The image analysis module 40 extracts, for example, the shape, hue, and the like of an object existing in a captured image as image analysis.

The image analysis module 40 determines whether or not the pest can be detected based on the result of the image analysis (step S12). In step S12, the image analysis module 40 compares the extracted feature points and feature quantities with the pest database registered in advance in association with the pest feature points and feature quantities and the pest identifier to obtain a photographed image. Determine if there are pests. The image analysis module 40 determines that the pest could be detected if the feature points and feature quantities extracted this time match the feature points and feature quantities registered in the pest database, and if they do not match, the pest cannot be detected. It will be judged that it was. When the image analysis module 40 can detect a pest, the image analysis module 40 together determines the identifier of the pest based on the feature points and amounts of the pest and the pest database.

In step S12, when the image analysis module 40 determines that the pest could not be detected as a result of the image analysis (step S12 NO), the computer 10 ends this process. In this case, the computer 10 may be configured to execute the process of step S10 described above and acquire shooting data different from the shooting data obtained by performing the image analysis this time.

On the other hand, in step S12, when the image analysis module 40 determines that the pest can be detected as a result of the image analysis (step S12 YES), the position specifying module 41 detects the pest based on the position information in the imaging data. The position information of is specified (step S13). In step S13, the position specifying module 41 identifies the position information of the shooting point corresponding to the captured image obtained by the image analysis this time based on the captured data. The position specifying module 41 identifies the position information of the photographing point as the position information of this pest. Further, the position specifying module 41 identifies the detailed position of the pest based on the position information of the pest and the coordinates in the captured image. For example, the position specifying module 41 sets an orthogonal coordinate system for the captured image, and specifies the position in the captured image in which the pest is detected as the X coordinate and the Y coordinate in the captured image. The position specifying module 41 identifies the position information of the pest in the actual field based on the position information at the photographing point and the X coordinate and the Y coordinate. At this time, in the position specifying module 41, the position of the center of the captured image corresponds to the position information at the imaging point, and the X coordinate and the Y coordinate are specified as the positions with respect to the position of the center. As a result, the location identification module 41 will identify the location of this pest in the field.

The countermeasure data specifying module 42 specifies the identifier of the pesticide or fertilizer effective against the pests identified this time and the spraying amount or fertilizer application amount of the pesticide or fertilizer as countermeasure data (step S14). In step S14, the countermeasure data specifying module 42 refers to the pest control database registered in advance by associating the identifier of the pest with the identifier of the effective pesticide or fertilizer, and identifies the pesticide or fertilizer effective for the pest detected this time. do. Further, the countermeasure data specifying module 42 specifies the spraying amount or fertilizer application amount of the pesticide or fertilizer based on the extracted feature points and feature amounts. For example, the countermeasure data specifying module 42 specifies the required amount of pesticide sprayed and the amount of fertilizer applied according to the size and shape of the extracted pest.

The recording module 30 records the identified pest identifier, the position information of the pest, and the countermeasure data as pest data (step S15).

The computer 10 repeats the process of steps S10-S14 described above, and records the identifiers, location information, and countermeasure data of the detected pests for the entire field.

The computer 10 may be configured to execute the process described later without recording the pest data for the entire field. For example, the computer 10 may be configured to record the one pest data when the one pest data is specified and use the one pest data in a process described later.

The pest map creation module 43 creates a pest map showing the identifier and position of the detected pest in this field based on the recorded pest data (step S16). In step S16, the pest map creation module 43 creates an overall map of the preset area or the entire field. This whole map is created based on the position information of the preset area or the whole field. For example, when a rectangular area is set, an entire map is created based on the position information of each vertex of this rectangle. The pest map creation module 43 creates a pest map in which a pest icon is superimposed on a position corresponding to the position information of the specified pest in the entire map. This pest icon is, for example, a symbol such as a circle, a square, or a star, a text indicating an identifier of the pest, or an illustration of the pest. This pest icon can also indicate the type of pest and the degree of damage in this place, depending on its shape and size.

Further, the pest map creation module 43 superimposes or nears the pest map, the identifier of the drug effective as a countermeasure against the pest, the identifier of the fertilizer, and the countermeasure icon indicating the fertilizer application amount, based on the recorded pest data. It is also possible to place it in. This countermeasure icon is, for example, an identifier of a necessary pesticide or fertilizer such as "A 1 liter spray of pesticide A", a text indicating the amount of spray or fertilizer applied, and an illustration showing these.

The pest map creation module 43 can also indicate the positions of the crops to be cultivated on this overall map by icons, texts, illustrations, and the like.

A pest map created by the pest map creation module 43 will be described with reference to FIG. FIG. 5 schematically shows an example of a pest map. In FIG. 5, in the pest map 100, the pest icon 110 and the countermeasure icon 120 are superimposed on the entire map. The pest icon 110 is arranged at a position based on the above-mentioned pest data on the entire map. The pest icon 110 also displays a text indicating the identifier of the pest. Further, the pest icon 110 schematically shows the damage situation of the pest depending on the size of the icon itself. Further, the pest icon 110 schematically indicates the type of pest by the shape of the icon. The countermeasure icon 120 is arranged at a position in the vicinity of the pest icon 110. The countermeasure icon 120 displays an identifier of a required pesticide or fertilizer and a text indicating a spraying amount or a fertilizer application amount.

The display mode of the pest icon 110 is not limited to that described above, and can be changed as appropriate. Further, the position, display mode, and the like of the countermeasure icon 120 are not limited to those described above, and can be appropriately changed.

The pest data output module 21 outputs the recorded pest data in the field (step S17). In step S17, the pest data output module 21 outputs the pest data by outputting the pest map described above. The pest data output module 21 outputs this pest map to a drone, a farm tool, a high-performance agricultural machine, a worker terminal, another computer, or the like. At this time, the pest data output module 21 transmits the pest map to a drone, a farm tool, a high-performance agricultural machine, a worker terminal, another computer, or the like.

The pest data output module 21 may be configured to output pest data instead of outputting a pest map. In this case, the pest data output module 21 may omit the process of step S16 described above and output the recorded pest data.

A case where the pest data output module 21 outputs this pest map to the worker terminal will be described.

The computer 10 transmits the pest map to the worker terminal, displays the pest map on the worker terminal, and causes the worker terminal to output the pest map.

The pest data output module 21 transmits the pest map to the worker terminal. The worker terminal receives this pest map. The worker terminal displays the received pest map on its display unit. By browsing this pest map, the operator can easily grasp the position and amount of the field where pesticide spraying or fertilization is required.

In addition, the worker terminal records the received pest map in its own recording unit or the like, and displays it by receiving an input from the worker. As a result, the worker can easily browse the pest map at the time of his / her work, and can easily apply the pesticide or fertilize without using the real-time live image.

The case where the pest data output module 21 outputs this pest map to the drone will be described.

The computer 10 causes the drone to output this pest map by transmitting the pest map to the drone.

The pest data output module 21 transmits the pest map to the drone. The drone receives this pest map. By recording this pest map in its own recording unit or the like, the drone can easily fly or move to the position of the pest based on the operation input or the like from the operator or the like of the drone. Furthermore, if pesticide spraying or fertilizer application is set in advance, pesticide spraying or fertilizer application is performed based on the recorded pest map, so that pesticide spraying or fertilizer application is performed without using real-time live images. Will be easy.

The pest data output module 21 may be configured to output this pest map after a predetermined time has elapsed from the time when the recording module 30 recorded the pest data. The predetermined time is, for example, several hours later, one day later, and several days later.

The above is the position information output process.

[Instruction output processing]
The instruction output process executed by the pest detection system 1 will be described with reference to FIG. FIG. 4 is a diagram showing a flowchart of an instruction output process executed by the computer 10. The process executed by each of the above-mentioned modules will be described together with this process. The detailed description of the same processing as the above-mentioned processing will be omitted.

The command creation module 44 creates an instruction to spray or fertilize a pesticide or fertilizer as a countermeasure against the detected pest as a command based on the recorded pest data (step S20). In step S20, the command creation module 44 creates this command according to the device or device of the output destination. For example, when the output destination is a device or device for spraying or fertilizing a preset pesticide or fertilizer, the command creation module 44 identifies the device or device for spraying or fertilizing the pesticide or fertilizer in the countermeasure data, and this device. And create the commands necessary for the device to spray or fertilize. The commands in this case are commands for the equipment or device to spray or fertilize pesticides or fertilizers, commands for specifying the amount of pesticides sprayed or fertilizers applied, and the location of the pest. Command for. Further, when the output destination is a device or device for spraying or fertilizing a plurality of pesticides or fertilizers, the command creation module 44 identifies the device or device for spraying or fertilizing the pesticide or fertilizer in the countermeasure data, and this pesticide or fertilizer. To create the necessary commands to spray or fertilize this pesticide or fertilizer. The command in this case is for the device or device to specify a command for selecting this pesticide or fertilizer, a command for spraying or fertilizing this pesticide or fertilizer, and a command for spraying or fertilizing this pesticide. And the command to fly or move to the location of this pest.

If the output destination device or device is not movable independently, it is not necessary to create a command for flying or moving to the position of the pest as a command created by the command creation module 44, and other than that. Any configuration may be used.

A case where the command creation module 44 creates a command for a drone capable of spraying or fertilizing a plurality of pesticides or fertilizers will be described as an example. The command creation module 44 identifies a drone capable of spraying or fertilizing a pesticide or fertilizer in the countermeasure data based on the countermeasure data in the pest data. The command creation module 44 creates commands for the drone to spray or fertilize pesticides or fertilizers. Further, the command creation module 44 creates a command for the drone to specify the amount of the pesticide applied or the amount of the fertilizer applied. Further, the command creation module 44 creates a command for flying or traveling to this position based on the position of the pest in the countermeasure data.

The command creation module 44 creates this command for each of the pest positions on the pest map.

The command output module 22 outputs the created command (step S21). In step S21, the command output module 22 outputs the created command to the specified device or device. As a result, the computer 10 outputs an instruction of pesticide spraying or fertilization to the device or device. That is, the computer 10 outputs a command to the device or device to cause the device or device to execute pesticide spraying or fertilization.

A case where the command output module 22 outputs a command to the drone will be described as an example.

The command output module 22 outputs the created command to the drone. At this time, the command output module 22 transmits the created command to the drone specified at the time of creating the command. The drone receives this command. Based on this command, the drone will select pesticides or fertilizers, identify the amount of application or fertilizer applied, and fly or run to the location of the pest. The drone flies or travels to the location of the pest, moves to the location of the pest, and then sprays or fertilizes the pest based on the amount of pesticide applied or the amount of fertilizer applied to the pest. That is, the computer 10 outputs a command to the drone so that the drone can perform pesticide spraying or fertilization.

The computer 10 executes the processing of steps S20 and S21 described above at the same time as the processing of step S17 described above or after executing the processing of step S17. That is, the computer 10 simultaneously executes the output of the pest data and the output of this command, or executes the output of this command after the output of the pest data.

The command output module 22 may be configured to output this command after a predetermined time has elapsed from the time when the recording module 30 recorded the pest data. The predetermined time is, for example, several hours later, one day later, and several days later.

The above is the instruction output process.

The above-mentioned means and functions are realized by a computer (including a CPU, an information processing device, and various terminals) reading and executing a predetermined program. The program is provided, for example, in the form of being provided from a computer via a network (Software as a Service). Further, the program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, a CD (CD-ROM or the like), or a DVD (DVD-ROM, DVD-RAM or the like). In this case, the computer reads the program from the recording medium, transfers it to an internal recording device or an external recording device, records the program, and executes the program. Further, the program may be recorded in advance on a recording device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and the program may be provided from the recording device to a computer via a communication line.

Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments described above. Further, the effects described in the embodiments of the present invention merely list the most suitable effects arising from the present invention, and the effects according to the present invention are limited to those described in the embodiments of the present invention. is not.

1 pest detection system, 10 computers

Claims (5)

It is a computer system that has a pest database registered by associating feature points and amounts of pests with the identifiers of the pests, and detects pests by image analysis.
An acquisition means for acquiring a photographed image of a field and position information of a photographing point, and
A detection means for detecting a pest by analyzing the acquired image and comparing the extracted feature points and feature amounts with a pest database to determine the identifier of the pest.
A position information output means for outputting the position information of the shooting point in the captured image in which the pest is detected, and
Based on the determined pest identifier and the output position information, the device or device for spraying or fertilizing the pesticide is specified, and the instruction output means for outputting the instruction for spraying or fertilizing the pesticide effective for the pest. ,
A computer system characterized by being equipped with. When the pesticide spraying or fertilization is plural, the instruction output means selects pesticide spraying or fertilization.
The computer system according to claim 1, wherein the computer system is characterized in that. The acquired image is not limited to a live image.
The computer system according to claim 1, wherein the computer system is characterized in that. It is a pest detection method executed by a computer system that has a pest database registered by associating the characteristic points and amounts of pests with the identifiers of the pests and detects the pests by image analysis.
Steps to acquire the photographed image of the field and the position information of the filming point,
A step of detecting a pest by analyzing the acquired image and comparing the extracted feature points and feature amounts with a pest database, and determining the identifier of the pest.
A step of outputting the position information of the shooting point in the shot image in which the pest is detected, and
Based on the determined pest identifier and the output location information, a step of specifying a device or device for spraying or fertilizing a pesticide and outputting an instruction for spraying or fertilizing a pesticide effective against the pest.
A pest detection method comprising. A computer system that has a pest database registered by associating feature points and amounts of pests with the identifiers of the pests and detects pests by image analysis.
Step to acquire the photographed image of the field and the position information of the filming point,
By analyzing the acquired captured image and comparing the extracted feature points and feature amounts with the pest database, the pest is detected and the identifier of the pest is determined.
A step of outputting the position information of the shooting point in the captured image in which the pest is detected,
A step of identifying a device or device for spraying or fertilizing a pesticide based on the determined pest identifier and the output position information, and outputting an instruction for spraying or fertilizing a pesticide effective against the pest.
A computer-readable program for running.

Images