WO2020022215A1 - Information processing device, information processing method, and program - Google Patents

Abstract

Using a feature quantity acquired from an image in which a set area of a farm field is imaged and the actual number of target objects present in the set area as learning data, the present invention learns an estimation parameter for estimating the actual number of target objects present in a designated area of a farm field.

Description

Information processing apparatus, information processing method and program

The present invention relates to an information processing device, an information processing method, and a program.

技術 There is a technique for calculating the number of specific objects in a specific area.

For example, Patent Document 1 proposes a method of detecting a flower area from a captured image and calculating the number of flowers by using an image processing technique. Further, by using the partial detector of Patent Document 2, even when the object is partially hidden (for example, when a part of the crop, which is the object, is hidden by leaves or the like), the object can be detected. it can. Thus, even when the object is partially hidden, the number of objects can be obtained with higher accuracy.

However, there are cases where the object whose number is to be obtained cannot be detected. For example, when the object is completely hidden (for example, when the crop that is the object is completely hidden by leaves), the object cannot be detected using the techniques of Patent Documents 1 and 2.

JP 2017-77238 A JP 2012-108785 A

There is a demand to support a mechanism for estimating the total number of objects even when some or all of the objects whose number is to be obtained cannot be detected.

However, Patent Documents 1 and 2 cannot support the realization of such a mechanism.

An information processing apparatus according to the present invention includes: a feature obtaining unit configured to obtain a feature amount of the region related to the number of target objects detected from the image from an image of a region that is a part of a field where a crop is grown, A number acquisition unit that acquires an actual number of the target objects existing in a set area of the field, the feature amount acquired by the feature acquisition unit from an image obtained by capturing the set area, Learning means for learning an estimation parameter for estimating the actual number of the target objects present in a designated area of the field, with the actual number acquired by the number acquisition means as learning data, It is characterized by having.

According to the present invention, it is possible to support realization of a mechanism for estimating the total number of objects even when some or all of the objects whose number is to be obtained cannot be detected.

FIG. 3 is a diagram illustrating an example of a hardware configuration of an estimation device. It is a figure showing an example of functional composition of an estimation device. FIG. 4 is a diagram illustrating an example of a table for managing learning data. FIG. 7 is a diagram illustrating an example of a state in which a part of an object is hidden by a leaf. FIG. 4 is a diagram illustrating an example of a table for managing estimation data. It is a flowchart which shows an example of a learning process. It is a flowchart which shows an example of an estimation process. It is a figure showing an example of functional composition of an estimation device. It is a figure showing an example of a field of a section. It is a figure showing an example of a field of a section. FIG. 4 is a diagram illustrating an example of a table for managing learning data. FIG. 4 is a diagram illustrating an example of a table for managing estimation data. It is a flowchart which shows an example of a learning process. It is a flowchart which shows an example of an estimation process. It is a figure showing an example of functional composition of an estimation device. FIG. 4 is a diagram illustrating an example of a table for managing correction information. It is a flowchart which shows an example of an estimation process. FIG. 2 is a diagram illustrating an example of a system configuration of an information processing system. It is a figure showing an example of a display screen of an estimation result. It is a figure showing an example of a display screen of an estimation result. It is a figure showing an example of a display screen of an estimation result.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First embodiment>
In the present embodiment, the estimation device 100 learns an estimation parameter which is a parameter used for estimating the number of objects included in the specified area, and determines the number of objects included in the specified area based on the learned estimation parameter. The process of estimating the value will be described.

FIG. 1 is a diagram illustrating an example of a hardware configuration of the estimation device 100 according to the present embodiment. The estimation device 100 is an information processing device such as a personal computer, a server device, and a tablet device that estimates the number of objects included in a designated area.

The estimation device 100 includes a CPU 101, a RAM 102, a ROM 103, a network I / F 104, a VRAM 105, an input controller 107, an HDD 109, and an input I / F 110. The components are communicably connected to each other via a system bus 111.

The CPU 101 is a central processing unit that controls the entire estimating apparatus 100 as a whole. The RAM 102 is a Random Access Memory, and functions as a main memory of the CPU 101, a work memory necessary for loading an execution program and executing a program, and the like.

The $ ROM 103 is a Read Only Memory, and stores, for example, various programs and various setting information. The ROM 103 includes a program ROM in which basic software (OS), which is a system program for controlling equipment of the computer system, is stored, and a data ROM in which information necessary for operating the system is stored. Also, the HDD 109 may store programs and information stored in the ROM 103.

The network I / F 104 is a network interface, and is used for input / output control of data such as image data transmitted and received via a network such as a local area network (LAN). It is assumed that the network I / F 104 is an interface corresponding to a network medium such as a wired or wireless network.

The VRAM 105 is a video RAM in which image data displayed on the screen of the display 106 as a display device is expanded. The display 106 is a display device, for example, a liquid crystal display or a liquid crystal panel. The input controller 107 is a controller used for controlling an input signal from the input device 108. The input device 108 is an external input device for receiving an operation instruction from a user, and is, for example, a touch panel, a keyboard, a pointing device, a remote controller, or the like.

The HDD 109 is a hard disk drive and stores application programs and data such as moving image data and image data. The application program stored in the HDD 109 is, for example, a highlight moving image creation application or the like. The input I / F 110 is an interface used for connection with an external device such as a CD (DVD) -ROM drive, a memory card drive, etc., and is used, for example, for reading image data captured by a digital camera. The system bus 111 is an input / output bus for connecting the respective hardware components of the estimation device so as to be able to communicate with each other, and is, for example, an address bus, a data bus, a control bus, or the like.

The CPU 101 executes a process based on a program stored in the ROM 103, the HDD 109, or the like, and thereby functions of the estimating apparatus 100 described later in FIGS. 2, 8, and 15, and a flowchart described later in FIGS. 6, 7, 13, 14, and 17. And the like are realized.

In the present embodiment, the object whose number is to be estimated is a crop (for example, a bunch of fruits, flowers, grapes, and the like). Hereinafter, the object whose number is to be estimated is referred to as a target object. In the present embodiment, an object that can hinder detection of the target object is referred to as an obstruction. In the present embodiment, the inhibitor is a leaf. However, the inhibitors may be not only leaves but also trees and stems. Further, the target object is not limited to agricultural products, and may be a person or a car. In that case, the obstruction may be, for example, a building.

Further, in the present embodiment, the estimation device 100 detects the target object from the captured image of the target region for which the number of target objects is to be estimated, and determines the characteristics of the region determined based on the detected number of target objects. The characteristic amount shown is obtained. Then, based on the obtained feature amount and the number of target objects actually included in the region, the estimation device 100 estimates the parameter used for estimating the actual number of target objects included in the region. We will learn the parameters. Hereinafter, the actual number of target objects included in a region is defined as the actual number of target objects in the region.

The estimating apparatus 100 also detects a target object from an image of a specified region, which is a target for estimating the number of target objects, and indicates a feature amount indicating a feature of the region based on the detected number of target objects. Ask for. Then, the estimating apparatus 100 estimates the actual number of target objects included in the area based on the obtained feature amounts and the learned estimation parameters.

FIG. 2 is a diagram illustrating an example of a functional configuration of the estimation device 100 according to the present embodiment. The estimation device 100 includes a number acquisition unit 201, an image acquisition unit 202, a learning unit 203, a feature amount acquisition unit 204, a parameter management unit 205, an estimation unit 206, and a display control unit 207.

The number acquisition unit 201 acquires the actual number (actual number) of target objects included in a preset region obtained by counting manually or the like. The number acquisition unit 201 acquires the actual number by reading, for example, a text file in which the actual number of target objects in a preset area is recorded from the HDD 109 or the like. Further, the number obtaining unit 201 may receive an input of the actual number via the input device 108.

The image acquisition unit 202 acquires, for example, from an external imaging device or the like, an image in which a preset area including a target object has been photographed, and stores the acquired image in the HDD 109 or the like. In the present embodiment, it is assumed that each of the preset regions is the entire region photographed in the corresponding image.

The feature amount acquiring unit 204 detects a target object from the image acquired by the image acquiring unit 202 using an object detection technique, and based on the number of detected target objects, a preset target object in which the detected target object exists is set. A feature amount indicating the feature of the region that has been acquired is acquired. In the following, the number of target objects detected from a certain area by the feature amount acquiring unit 204 is defined as the number of detections in that area. In the present embodiment, the feature amount acquisition unit 204 acquires the number of detections of a certain region as a feature amount indicating the feature of the region. The process of acquiring a feature by the feature acquiring unit 204 is an example of a feature acquiring process.

The learning unit 203 performs the following processing for each image received by the number acquisition unit 201. That is, the learning unit 203 corresponds to the actual number of target objects included in the preset area corresponding to the image acquired by the number acquiring unit 201 and the image acquired by the feature amount acquiring unit 204. And a feature amount indicating a feature of the region set in advance. Then, the learning unit 203 learns, by machine learning, estimation parameters used for estimating the actual number of target objects included in the designated area, based on the acquired number and the feature amount. In the present embodiment, the learning unit 203 uses linear regression as a method of machine learning, and learns parameters used for linear regression as estimation parameters. However, the learning unit 203 may learn a parameter in another method such as a support vector machine as an estimated parameter.

The parameter management unit 205 stores the estimated parameters learned by the learning unit 203 in the HDD 109 or the like and manages them.

The estimating unit 206 includes a feature amount acquired by the feature amount acquiring unit 204 from an image obtained by photographing an area in which the number of target objects is to be estimated, and a learned estimation parameter managed by the parameter managing unit 205. , The following processing is performed. That is, the estimating unit 206 estimates the actual number of target objects included in the target area for estimating the number of target objects.

FIG. 3 is a diagram illustrating an example of a table that manages the actual number of target objects acquired by the number acquiring unit 201 and the number of target objects detected by the feature amount acquiring unit 204 as learning data. The table 301 includes items of ID, image file, number of detections, and actual number. The item of ID indicates identification information for identifying learning data. The item of the image file indicates which image the corresponding learning data is generated using. The item of the number of detections indicates the number of target objects detected from the image indicated by the item of the corresponding image file. The item of the actual number is the number of target objects actually included in a specific region photographed in the image indicated by the item of the corresponding image file (for example, the number including target objects hidden in leaves and not shown in the image). Is shown. The table 301 is stored in, for example, the HDD 109 or the like.

{For example, an image (IMG_0001.jpg) indicated by the image file corresponding to the learning data with the ID of 1 will be described with reference to FIG. IMG_0001. Looking at the image 701 as a jpg, it can be seen that three target objects are shown and four are hidden behind leaves and are not shown. Therefore, the detection number and the actual number corresponding to the learning data with the ID of 1 in the table 301 are 3, 7 (= 3 + 4), respectively.

The number acquisition unit 201 acquires the actual number of target objects in one or more specific areas in advance. In addition, the feature amount acquisition unit 204 detects a target object from each of a plurality of images in which any of the one or more specific regions is captured, and obtains the number of detections in advance. Then, the number acquiring unit 201 and the characteristic amount acquiring unit 204 store the acquired actual number and the detected number in the HDD 109 or the like as learning data in the format of the table 301 shown in FIG. Thus, learning data used for learning is prepared in advance.

Note that, among the learning data in the present embodiment, the image file or its feature amount is referred to as input data. The actual number corresponding to the input data is called correct data. The learned estimation parameter is also called a learned model.

FIG. 5 illustrates the number of detected target objects detected by the feature amount acquiring unit 204 from an image of a region in which the actual number of target objects for which estimation is desired is to be estimated, and the actual number of target objects in the region estimated by the estimating unit 206. FIG. 3 is a diagram showing an example of a table for managing the values of. The table 401 includes items of ID, image file, number of detections, and estimated value. The item of ID indicates identification information for identifying an area in which the actual number of target objects is estimated. The item of the image file indicates the image used for estimating the actual number. The item of the number of detections indicates the number of target objects (the number of detections) detected by the feature amount acquiring unit 204 from the image indicated by the item of the corresponding image file. The item of the estimated value indicates the number of target objects estimated by the estimating unit 206. The table 401 is stored in, for example, the HDD 109 or the like.

FIG. 6 is an example of a flowchart showing an example of the estimation parameter learning process.

In step S <b> 501, the number acquisition unit 201 acquires, for example, from a text file stored in the HDD 109, the file name of an image file in which a predetermined area is captured and the actual number of target objects included in the area. I do. Then, the number acquiring unit 201 registers the acquired file names and the actual number in the table 301 stored in the HDD 109. It is assumed that the HDD 109 previously stores a text file in which image file names and actual numbers are recorded in association with each other in a format such as a CSV format.

In the present embodiment, for each of a plurality of preset regions, the number acquisition unit 201 determines, for each of a plurality of preset regions, a file name of an image file in which the region is photographed, and a target object included in the region. And the actual number. Then, the number acquiring unit 201 registers, in the table 301, each set of the file name and the actual number acquired for each of the plurality of areas.

In step S502, the feature amount acquisition unit 204 detects the target object from the image indicated by the image file name for each of the image file names registered in the table 301 in step S501, and determines the number of detections in the area captured in the image. Is obtained as the feature value of

In step S503, the feature amount acquiring unit 204 registers, for example, the number of detections (feature amounts) acquired in step S502 in the table 301 stored in the HDD 109.

In step S <b> 504, the learning unit 203 learns an estimation parameter (in this embodiment, a parameter of linear regression) using a set of the detected number (feature amount) and the actual number registered in the table 301. For example, the linear regression is represented by the following equation (1). The learning unit 203 learns the estimation parameters by using the parameters w0 and w1 in Equation (1) as the estimation parameters. For example, values such as w0 = 7.0 and w1 = 1.2 are learned.
Actual number (estimated value) = w0 + (w1 x number of detections) Expression (1)

In step S505, the parameter management unit 205 starts management by, for example, storing the estimated parameters learned in step S504 in the HDD 109.

FIG. 7 is a flowchart showing an example of the estimation process using the estimation parameters learned by the process of FIG.

In step S601, the estimation unit 206 requests the parameter management unit 205 for the estimation parameters learned in the process of FIG. In response to this request, the parameter management unit 205 acquires the estimated parameters learned in S504 stored in S505 from the HDD 109, and transmits the acquired estimated parameters to the estimation unit 206.

In step S <b> 602, the feature amount acquiring unit 204 detects a target object from an image in which an area designated as a target for estimating the number of target objects is captured, and acquires the number of detections. In the present embodiment, an image obtained by photographing at least a part of the field in S602 is supplied as a processing target, that is, an area photographed in the image is designated as a target of a process of estimating the number of target objects. Is equivalent to If there are a plurality of designated images, the same processing is performed for all of them. Then, the feature amount acquiring unit 204 registers the acquired number of detections in the table 401 stored in the HDD 109, for example, in association with the image file name of the image.

In step S603, the estimating unit 206 estimates the number of target objects included in the target region in which the number of target objects is to be estimated based on the estimation parameters acquired in step S601 and the number of detections acquired in step S602. . The estimating unit 206 calculates the number of target objects included in the area using Expression (1) based on, for example, w0 and w1 that are the estimation parameters acquired in S601 and the number of detections acquired in S602. Find an estimate. The estimating unit 206 outputs the obtained estimated value by registering it in the table 401. The estimating unit 206 may output the obtained estimated value by displaying it on the display 106.

The estimated value registered in the table 401 may be used for, for example, predicting the yield of the crop, which is the target object, and visualizing the information of the predicted high- and low-yield areas on a map. .

As described above, in the present embodiment, the estimation device 100 detects a target object from an image in which a preset region is captured, and indicates a characteristic of the region based on the number of detected target objects (the number of detections). Take the quantity. Then, the estimation device 100 learns the estimation parameters based on the acquired feature amount and the actual number (actual number) of the target objects included in the area. By using the learned estimation parameters, it is possible to estimate the actual number of target objects included in a region based on a feature amount corresponding to the number of detected target objects detected from the region. That is, the estimating apparatus 100 can support the realization of a mechanism for estimating the total number of objects even when some or all of the target objects cannot be detected by learning the estimation parameters.

Further, in the present embodiment, the estimating apparatus 100 obtains the learned estimation parameters and the number of target objects in the region obtained based on the number of detected target objects detected from the captured image of the target region. The following processing was performed based on the feature amounts of That is, the estimation device 100 estimates the actual number of target objects included in the area. As described above, the estimation device 100 can estimate the actual number of target objects included in a region from the feature amount of the region based on the number of target objects detected from the region. Accordingly, the estimation device 100 can support the realization of a mechanism for estimating the total number of objects even when some or all of the target objects cannot be detected.

In the present embodiment, the estimating apparatus 100 generates learning data used for learning the estimation parameters in advance, and stores the learning data in the HDD 109 as the table 301. In this way, the estimation device 100 can support the realization of a mechanism for estimating the total number of objects even when some or all of the target objects cannot be detected by preparing learning data used for learning the estimation parameters.

In the present embodiment, the number of detections is the number of target objects detected from the image. However, the number of detections may be, for example, the number of people who have visually detected the target object. In that case, the estimation device 100 accepts the designation of the number of detections, for example, based on a user operation via the input device 108.

Further, suppose that the target object is a person, and a human sensor is installed in a part of a preset area. In that case, the estimation device 100 may use the number of people detected via the human sensor as the number of detections, and use the number of people actually existing in the area as the actual number. For example, the estimating apparatus 100 estimates the actual number of persons included in the region based on the feature amount of the region acquired from the number of detections, based on the set of the detected number and the actual number at each of a plurality of time points in the region. May be learned. In addition, the estimation device 100 may estimate the actual number of people included in the area at the designated time using the learned estimation parameters. In addition, the estimation device 100 may generate a set of the detected number and the actual number at each of a plurality of time points in advance and generate learning data used for learning the estimation parameter.

<Example of use>
An example of use of a system for presenting to a user the number of target objects obtained by the processing of the estimating apparatus 100 of the present embodiment and the yield of crops that can be predicted based on the number will be described. This system includes an estimation device 100. The user of this system can utilize the number of target objects estimated by the estimating apparatus 100 in a work to be performed later and a production plan of a processed product. For example, the processing of the present embodiment can be suitably applied to a case where grapes for wine production are cultivated as agricultural products. Hereinafter, the production control of grapes for wine production will be described as an example.

When cultivating grapes for wine production, it is necessary to accurately predict the yield at the stage of grape cultivation in order to control the types and amounts of wine that can be produced in that year (also called vintage). Desired. Therefore, in a grape field, at predetermined timings of a plurality of growth stages, grape buds, flowers, and bunches are counted to repeatedly predict the amount of grape finally obtained in the year. . When the harvest amount is smaller or larger than the predicted value, the content of work in the field is changed, or the type or amount of wine to be produced or the sales plan is adjusted.

(4) In order to count the actual number of target objects, a person (operator) counts target objects visually while avoiding obstacles such as grape leaves. However, when the field is vast, the human load becomes excessive when performing a counting operation in which timing is emphasized on all the trees in the field.

Therefore, conventionally, sampling surveys at a plurality of locations in a field or a plurality of trees have been performed. However, even in a field, the growth condition of a cared tree may vary depending on the place or year, for example, because the geographical and climatic conditions are not uniform. In order to perform more accurate sampling surveys in consideration of such variations, it is necessary to determine a sampling position that reflects differences in geographical and climatic conditions in a field each time a survey is performed.

On the other hand, even when the field is vast, taking a picture of the vine in the entire field or a part of the field under certain conditions requires counting the target object while turning over the leaves of all the trees. Work load is low compared to. Even when the actual number of target objects can be counted only in a part of the field, if an image captured in a wider range can be obtained, the processing of the present embodiment allows the number of target objects to be calculated. Can be estimated with higher accuracy. As described above, in the present embodiment, for at least a part of the field, a learned model obtained by learning a set of a result of counting the actual number of target objects and an image of the part is used. Then, the actual number of target objects is estimated based on the image. In this case, the learned model is learned so that if the number of detected target objects in the image is small, the estimated actual number tends to be small. Therefore, for example, even if there is a tree whose growth state is worse than that of the tree on which the sampling survey was performed due to the influence of geographical conditions, the number of target objects estimated from an image of the tree is determined by the sampling survey. Will be less than trees made. As described above, the processing of the present embodiment enables more accurate estimation processing irrespective of the position where the sampling investigation is performed.

FIGS. 19 to 21 are diagrams each showing an example of a display screen showing an estimation result output by the estimation device 100 when the system according to this usage example is introduced at a wine production grape production site. In the present embodiment, the display control unit 207 generates the display screens of FIGS. 19 to 21 based on the estimation result in S603 and displays the display screens on the display 106. However, the display control unit 207 generates the display screens of FIGS. 19 to 21 based on the estimation result in S603, transmits the generated display screen to an external device, and displays the display unit (display or the like) of the transmission destination device. ) May be controlled.

The screen 1900 is a screen showing, for each of the seven blocks included in the field, an identifier (ID), an area, and an estimated value of the grape harvest amount for the corresponding block. The display control unit 207 estimates the weight (unit: t) of the grapes to be harvested based on the total of the results of the processing in S603 (the estimated number of the grapes bunches to be harvested) for all the images of the corresponding blocks. Find the value. Then, the display control unit 207 causes the obtained weight to be included in the screen 1900 as an estimated value of the grape harvest amount. Expressing the grape yield by weight, rather than by the number of bunches, makes it easier to use the grape yield for estimating wine production. In the example of FIG. 19, a predicted value that 19.5 (t) grape is harvested in the block B-01 represented by the area 1901 is shown.

Further, in the present embodiment, when the display control unit 207 detects a pointing operation (for example, a selection operation such as a click or a tap) to the area 1901 via the input device 108, the display control unit 207 changes the screen displayed on the display 106 to an The screen is switched to a screen 2000 shown in FIG.

Referring to FIG. 20, screen 2000 displayed when area 1901 (corresponding to block B-01) is selected on screen 1900 will be described. The screen 2000 is a screen for presenting information that is the basis of the prediction of the yield for the block B-01.

Screen 2000 includes area 2001 at a position corresponding to area 1901 on screen 1900. Each of the 66 squares in the area 2001 is a marker indicating one unit (unit) to be subjected to the counting survey. In the example of FIG. 20, the target object is a grape cluster. The pattern of each marker is a pattern corresponding to the average number of tufts detected in each unit. That is, the area 2001 shows the geographic distribution of the number of detected target objects existing in the block B-01.

領域 An area 2002 surrounded by a broken line shows detailed information on the selected block B-01. For example, area 2002 indicates information indicating that the average number of detected cells for all units in block B-01 is 8.4. In the area 2002, information indicating that the average estimated number of cells for all units in the block B-01 is 17.1 is shown.

Here, the number of detected target objects detected by the feature amount acquisition unit 204 does not include the number of target objects whose detection is hindered by obstacles. On the other hand, the actual number acquired by the number acquiring unit 201 is a number including the number of target objects whose detection is inhibited by the obstacle. That is, in the present embodiment, there may be a difference between the actual number serving as a pair of learning data and the number of detections. As a result, the number of target objects estimated by the estimating unit 206 is larger than the number of detections, as indicated by the area 2002 in FIG. 20, by the number of target objects whose detection is inhibited by the obstacle. There is.

The area 2003 indicates, for each set of the set of the number of detected cells and the estimated number of cells divided into a plurality of stages, the total number of markers belonging to the corresponding stage. The method of expressing the information shown in the area 2003 may be a histogram format as in the example of FIG. 20 or may be various graph formats.

20. In the example of FIG. 20, the area 2003 shows a histogram in which the pattern of the bin is changed for each stage. The pattern of the bins in this histogram corresponds to the pattern of the marker in the area 2001.

20. Further, in the example of FIG. 20, below the area 2002, the stages of the number of detected and estimated numbers of cells corresponding to the patterns used in the area 2003 and the area 2001 are shown. In the example of FIG. 20, the display control unit 207 assigns different patterns to the bins and the markers. However, the bins and the markers may be colored in different colors.

As described above, in the area 2001, the distribution of the number of detected tufts is represented by a pseudo heat map. The display in the heat map format allows the user to intuitively understand the magnitude of the number of detections and the distribution thereof.

In the example of FIG. 20, the estimated cell number is also shown in the area 2002 in association with the detected cell number. Since the user who actually looks at the field sometimes sees the vine before the leaves grow, the user may intuitively recognize the number of bunches hidden in the leaves. For such a user, the number of clusters detected from the image may be felt to be less than the number known to one's own sensation.

Therefore, in this usage example, the display control unit 207 associates not only the number of actually detected bunches but also the actual number estimated by the learned model with the number of detections as the basis of the predicted value of the yield. Display. For example, the user first looks at the screen 1900 and knows the predicted value of the yield. Then, when making a subsequent plan for each block, if the user wants to know the basis of the predicted value just in case, he or she clicks the target block. Then, the user uses the screen 2000 corresponding to the clicked block to display the number of clusters detected from the image (the number of clusters that are surely present) and the number of clusters not detected from the image. It is possible to confirm both the estimated and expected values. For example, if the user who knows the actual field condition has an impression that the predicted value of the harvest amount is too small on the screen 1900, if the user checks the screen 2000, the cause is that the number of detections is small, or It is possible to quickly determine whether the cause is a small estimated number (estimation processing).

{Circle around (2)} The virtual button 2004 in the area 2002 is a button used to clearly indicate the position where the sampling survey was actually performed among the markers shown in the area 2001. When detecting the pointing operation on the virtual button 2004, the display control unit 207 switches the screen displayed on the display 106 to a screen 2100 shown in FIG.

In the example of FIG. 21, 66 markers included in the block B-01 are displayed in the area 2001 of the screen 2100 continuously from the screen 2000. Then, the display control unit 207 makes the ten markers corresponding to the positions where the sampling survey is actually performed out of the sixty-six markers into markers highlighted by a thick line like the marker 2101.

(5) At the same time, the display control unit 207 also changes the display state of the virtual button 2004 on the screen 2000 by changing the color or the like. Thereby, the display state of the virtual button 2004 is associated with whether or not the virtual button 2004 is selected. By confirming the virtual button 2004, the user can easily determine whether or not the virtual button 2004 has been selected.

Note that the function related to the virtual button 2004 is particularly effective when learning data from a sampling survey for that year is used as the basis of the estimation processing. For example, when a learned model in which only data obtained as a result before last year is learned as learning data is used, it is not necessary to confirm the sampling position, and thus the virtual button 2004 may be omitted. For example, after the year in which it is determined that sufficient learning data has been obtained based on the past sampling survey results, the sampling survey for each year may be omitted, and only the counting survey by the estimation processing according to the present embodiment may be performed.

<Embodiment 2>
In the present embodiment, a case will be described in which the estimating apparatus 100 specifies an area of a predetermined specific section set in advance and estimates the actual number of target objects included in the specified area.

ハ ー ド The hardware configuration of the estimation device 100 of the present embodiment is the same as that of the first embodiment.

点 Hereinafter, points different from the first embodiment will be described.

FIG. 8 is a diagram illustrating an example of a functional configuration of the estimation device 100 according to the present embodiment. The estimating apparatus 100 of the present embodiment is different from the case of the first embodiment shown in FIG. 2 in that it includes a section specifying unit 801 for specifying a preset area of a section. Further, in the present embodiment, the image acquired by the image acquisition unit 202 is an image indicating a region of a preset section.

The section identification unit 801 detects, from an image, an object indicating an area of a preset section (eg, an area of a section set in a field), and based on the position of the detected object, determines an Identify the area. The image acquisition unit 202 cuts out the area of the section specified by the section specifying unit 801 from the input image and stores the cut out area in the HDD 109 or the like.

The processing of the section specifying unit 801 will be described with reference to FIG. An image 901 in FIG. 9 is an image obtained by photographing a region of a section, and an image 902 is an image obtained by cutting out the area of a section. The section specifying unit 801 detects an object indicating the area of the section from the image 901 and specifies an area surrounded by the detected object in the image 901 as the area of the section. Then, the image obtaining unit 202 cuts out the area specified by the section specifying unit 801 from the image 901, obtains the image 902, and stores the obtained image 902 in the HDD 109.

If the width of the section is widened, the section may not fit in one image and may be divided into a plurality of images to capture one section. In this case, the section specifying unit 801 arranges a plurality of images of the section, detects objects indicating both ends of the section from the plurality of images, and specifies the area of the section for each image. The image acquiring unit 202 combines the areas of the sections in each of the plurality of images specified by the section specifying unit 801 and stores the combined image in the HDD 109 or the like.

処理 The processing of the section identification unit 801 in this case will be described with reference to FIG. A plurality of images 1001 in FIG. 10 are images obtained by capturing the area of the section, and an image 1002 is an image showing the area of the combined section. The section specifying unit 801 detects an object indicating the area of the section from the plurality of images 1001, and specifies an area sandwiched by the detected objects in the plurality of images 1001 as the area of the section. Then, the image obtaining unit 202 synthesizes the area specified by the partition specifying unit 801 from the image 1001, obtains the image 1002, and stores the obtained image 1002 in the HDD 109.

Further, the section specifying unit 801 combines a plurality of images obtained by capturing the sections, detects an object indicating the area of the section from the combined image as one combined image, and combines the objects based on the detected object. The area of the section in the image may be specified.

In the present embodiment, the feature amount acquiring unit 204 detects a target object from the image of the area of the preset section specified by the section specifying unit 801 obtained by the image obtaining unit 202. Then, the feature amount acquiring unit 204 acquires the feature amount of the area of this section based on the number of detected target objects (the number of detections).

Therefore, each of the feature amounts included in the learning data used for learning the estimation parameter and the feature amounts used for the estimation process is a feature amount of any of the preset sections. That is, in the present embodiment, the estimation parameter is learned for each section defined in the field.

The image acquiring unit 202 may acquire an image similar to that of the first embodiment without acquiring an image of the area of the section specified by the section specifying unit 801. In this case, the feature amount acquiring unit 204 detects the target object from the region of the preset section specified by the section specifying unit 801 in the image obtained by the image obtaining unit 202. Then, the feature amount acquiring unit 204 acquires the feature amount of the area of this section based on the number of detected target objects (the number of detections).

As described above, in the present embodiment, the estimating apparatus 100 can specify an area of a preset section and estimate the actual number of target objects included in the specified area of the section.

場合 When it is desired to estimate the actual number of target objects included in an area of a certain section, it is desirable to reduce the influence of the target object that can be detected from an area other than the area of the section. By the processing of the present embodiment, the estimation device 100 can reduce the influence of the target object that can be detected from an area other than the area in which the actual number of the target objects is to be estimated.

The section specifying unit 801 specifies the area of the section by detecting an object indicating the area of the section set in advance. However, the area of the section is measured using position information such as GPS data or an image measurement technique. May be specified. Furthermore, the estimating apparatus 100 may display a virtual frame on a camera finder that generates an input image so that the user can confirm an area of the section specified at the time of shooting, or a composite image in which the frame is superimposed. May be generated. Further, the estimation device 100 may store the frame information as metadata of the image in the HDD 109 or the like.

<Embodiment 3>
In the present embodiment, a case will be described in which the estimation device 100 acquires a feature amount indicating a feature of the area based on other attributes of the area in addition to the number of target objects detected from the area.

推定 The hardware configuration and the functional configuration of the estimation device 100 of the present embodiment are the same as those of the first embodiment.

In the present embodiment, the estimation device 100 uses a set of the number of target objects detected from a region and other attributes of the region as a feature amount of the region. Then, the estimation device 100 performs a learning process of the estimation parameter using the feature amount, and a process of estimating the number of target objects using the estimation parameter.

11 is a table used for registering learning data, and is stored in, for example, the HDD 109 or the like. The table 1101 is a table in which information used for learning the estimation parameter is added to the table 301 shown in FIG.

The table 1101 includes items of ID, image file, number of detections, number of adjacent detections, soil, leaf volume, and actual number. The item of the number of adjacent detections is an average value of the target object detected from each of one or more areas adjacent to the area indicated by the corresponding ID. In the present embodiment, the target object is a crop. In the present embodiment, the estimating apparatus 100 includes, in the feature amount of the region, a feature of a region around the region (for example, a statistic (for example, a statistical value (for example, , Average, total, variance, etc.). As a result, the estimation device 100 can learn an estimation parameter capable of estimating the actual number of target objects in consideration of the surrounding area, and use the estimation parameter to evaluate the target object in consideration of the surrounding area. Can be estimated.

The item of soil indicates an index value indicating the goodness (easiness of fruiting) of the soil in the area indicated by the corresponding ID. The larger the index value, the better the soil. In the present embodiment, the estimation device 100 causes the feature value to include an index value indicating the goodness of the soil in which the crop, which is the target object, is planted. Thereby, the estimating apparatus 100 can learn an estimation parameter capable of estimating the actual number of target objects in consideration of the characteristics of the soil, and use the estimation parameters to consider the characteristics of the soil in consideration of the characteristics of the soil. Can be estimated.

The item of 量 leaf amount indicates an index value indicating the amount of leaf detected from the area. The larger the index value, the larger the amount of leaves. In the present embodiment, it is assumed that the obstacle that can inhibit the detection of the target object is a leaf. In the present embodiment, the estimation device 100 includes the amount of the detected obstacle in the feature amount. Accordingly, the estimation device 100 can learn an estimation parameter capable of estimating the actual number of target objects in consideration of the amount of the obstruction, and use the estimation parameter to consider the amount of the obstruction. The actual number of target objects can be estimated.

In the present embodiment, it is assumed that the feature amount of the region is a set of the number of detections, the number of adjacent detections, the amount of leaves, and an index value indicating the goodness of soil. However, the feature amount of the region may be a set of the number of detections and a part of the index value indicating the number of adjacent detections / leaf volume / good soil. Further, the feature amount of the region may include an attribute of the region other than the number of detections, the number of adjacent detections, the amount of leaves, and the index value indicating the goodness of the soil.

テ ー ブ ル Table 1201 in FIG. 12 is a table for managing the feature amount of the area in which the actual number of target objects is to be estimated and the value of the actual number of target objects in the area estimated by estimating section 206. The table 1201 is a table in which information used for estimating the actual number of target objects is added to the table 401 in FIG. The table 1201 includes items of ID, image file, number of detections, number of adjacent detections, soil, leaf volume, and estimated value. The items of the number of adjacent detections, soil, and leaf amount are the same items as in the table 1101.

FIG. 13 is a flowchart showing an example of the estimation parameter learning process. In the processing of FIG. 13, unlike the processing of FIG. 6, a table 1101 is used instead of the table 301.

Steps S1301 to S1304 are processes in which a feature amount is acquired by the feature amount acquiring unit 204 for each image file registered in the table 1101, and the acquired result is registered.

In step S1301, the feature amount acquiring unit 204 detects a target object and leaves from the images registered in the table 1101. The feature amount acquiring unit 204 may detect the leaf using the object detection technique, or may detect the leaf simply by detecting a pixel having a leaf color.

In step S1302, the feature amount acquiring unit 204 registers the number of detected target objects and the leaf amount detected in step S1301 in the table 1101. The feature amount obtaining unit 204 obtains a leaf amount, which is an index value indicating a leaf amount, based on a ratio between the number of pixels of the detected leaf region and the number of pixels of the entire image.

In step S1303, the feature amount obtaining unit 204 obtains position information such as GPS data from the metadata of the image registered in the table 1101. Then, the feature amount obtaining unit 204 obtains information on the number of adjacent detections and the goodness of the soil based on the obtained position information. The feature amount obtaining unit 204 obtains position information of an image corresponding to IDs before and after the target ID from the table 1101, and determines whether the image is an image of an adjacent area. Then, the feature amount obtaining unit 204 obtains the number of detections for the image determined to be an adjacent area, and obtains an average value as the number of adjacent detections. For example, the feature quantity acquiring unit 204 sets the number of adjacent detections of ID2 to 3.5, which is the number of detections of ID1 and 4, the number of detections of ID3, on average.

In the present embodiment, it is assumed that images captured consecutively in position are registered in the table 1101 as images having consecutive IDs. Therefore, it is assumed that an object with a close ID is photographed at a close position. However, when the edge of the region to be photographed is photographed, the photographing may be interrupted and the IDs before and after the photographing may not always be adjacent. Therefore, the feature amount acquiring unit 204 is the region that is actually adjacent using the position information. Was determined.

{Circle around (4)} In order to cope with the case where the IDs are close and the positions are not close due to the fact that a plurality of locations are photographed in parallel, the estimation device 100 may perform the following. That is, assuming that the table 1101 includes the position information of the area of the corresponding ID, the feature amount acquiring unit 204 specifies the data of the number of detections of the area around the certain area from the table 1101 based on the position information. You may do it.

と す る It is assumed that information indicating the relationship between the position obtained empirically from the past growth state and the goodness of the soil is managed in a database or the like stored in advance in the HDD 109 or the like. The feature amount acquiring unit 204 acquires, for example, an index value indicating the goodness of the soil corresponding to the shooting position from the database or the like.

In step S <b> 1304, the feature amount acquiring unit 204 registers information on the number of adjacent detections acquired in step S <b> 1303 and an index value indicating soil goodness in the table 1101.

In S504 of FIG. 13, the learning unit 203 learns an estimation parameter using the number of detections, the number of adjacent detections, an index value indicating soil goodness, and the amount of leaves in the table 1101. In the present embodiment, it is assumed that the estimation parameter is a parameter of linear regression. For example, it is assumed that the linear regression is represented by the following equation (2).
Real number (estimated value) = w0 + (w1 x number of detections) + (w2 x number of adjacent detections) + (w3 x index value indicating goodness of soil) + (w4 x leaf volume) ... Equation (2)

In this case, the learning unit 203 learns w0, w1, w2, w3, and w4 as the estimation parameters. For example, values such as w0 = 7.0, w1 = 0.7, w2 = 0.5, w3 = 1.6, and w4 = 1.2 are learned.

In step S505 of FIG. 13, the parameter management unit 205 stores and manages the estimated parameters learned in step S504 in the HDD 109 or the like.

FIG. 14 is a flowchart illustrating an example of an estimation process for estimating the actual number of target objects using the estimation parameters learned in the process of FIG.

In step S601 in FIG. 14, the estimating unit 206 acquires the estimation parameters learned in step S504 in FIG.

In S1401, similarly to S1301, the feature amount acquiring unit 204 detects the target object and the leaves from the image in which the region for which the actual number of target objects is to be estimated is photographed.

In step S1402, the feature amount acquiring unit 204 registers the number of detected target objects and the leaf amount detected in step S1401 in the table 1201 as in step S1302. The feature amount obtaining unit 204 obtains a leaf amount, which is an index value indicating a leaf amount, based on a ratio between the number of pixels of the detected leaf region and the number of pixels of the entire image.

In step S1403, the feature amount obtaining unit 204 obtains position information such as GPS data from metadata of an image of a region in which the actual number of target objects is to be estimated, as in step S1303. Then, the feature amount obtaining unit 204 obtains information on the number of adjacent detections and the goodness of the soil based on the obtained position information. Then, the feature amount acquiring unit 204 registers information on the acquired number of detected neighbors and an index value indicating the goodness of soil in the table 1201.

In step S603 of FIG. 14, the estimating unit 206 executes the following process based on the estimation parameters acquired in step S601 and the feature amounts of the regions registered in the table 1201, using equation (2). That is, the estimating unit 206 estimates the number of target objects that are actually included in the region in which the actual number of target objects is to be estimated.

In the example of FIG. 12, the number of target objects detected (the number of detections) of ID836 is smaller than that of ID835 or ID837. However, the ID 836 is an estimated value similar to the ID 835 and the ID 837. Therefore, in the region of ID836, the number of target objects that happened to be hidden by leaves may have increased as compared with ID835 or ID837, and the number of detected objects may have decreased as compared with ID835 or ID837.

(4) It can be assumed that agricultural products have a bias in the easy-to-harvest and hard-to-harvest locations even in the same field, and that there is a correlation between the actual number of target objects among those close in position. The estimating apparatus 100 can supplement the information on the number of detections by using the number of adjacent detections as the feature quantity, so that the estimated value does not become too small even if the number of detections decreases. In addition, since the possibility that the target object is hidden increases as the leaf amount increases, the estimation device 100 uses the leaf amount as the feature amount, thereby preventing the estimation value from becoming too small even when the leaf amount is hidden. be able to.

As described above, in the present embodiment, the estimating apparatus 100 uses, as the feature amount, other attributes of the region such as the positional deviation of the yield (good soil) and the amount of leaves in addition to the number of detections. did. Thereby, the estimating apparatus 100 can estimate the real number of the target with higher accuracy than in the first embodiment.

In addition to the feature amounts used in the present embodiment, the estimating apparatus 100 may use information on the size of the target object detected on the assumption that the target object is large in an easy-to-produce location as the feature amount of the region. Good. In addition, the estimation device 100 may use, as a feature amount, a variety of agricultural crops, a fertilizer application state, the presence or absence of a disease, and the like, which are factors of the yield.

<Embodiment 4>
Agricultural crops tend to be fruitful depending on the weather. In the present embodiment, a description will be given of a process of estimating the actual number of the crops that are the target objects and correcting them in consideration of weather conditions.

で は In the present embodiment, the hardware configuration of the estimation device 100 is the same as that of the first embodiment.

Hereinafter, differences between the present embodiment and the first embodiment will be described.

FIG. 15 is a diagram illustrating an example of a functional configuration of the estimation device 100 according to the present embodiment. The functional configuration of the estimation device 100 of the present embodiment is different from the functional configuration of FIG. 2 in that a correction information acquisition unit 1501 that acquires correction information is included. The correction information acquisition unit 1501 includes a learned estimation parameter used for estimating the actual number of target objects by the estimation unit 206, and correction information (for example, an estimated value And the like for multiplication). In the present embodiment, the estimating unit 206 estimates the actual number of target objects using the estimation parameters and the correction information.

FIG. 16 is a diagram showing an example of a table for managing learned parameters and coefficients prepared in advance. The table 1601 includes a year, an average number of detections, and parameters. The item of the year is the year corresponding to the learning data registered in the table 301. The item of the average number of detections is the average number of detections in the corresponding year. The parameter is an estimated parameter learned by the learning unit 203 using the learning data corresponding to the corresponding year. The table 1601 is a table for managing a plurality of estimation parameters learned by the learning unit 203, each of which is associated with an index value of a preset index called an average detection number. The table 1601 is stored in, for example, the HDD 109 or the like.

The table 1602 includes items of the percentage of fine weather and the coefficient. The item of sunny ratio indicates the ratio of sunny days during the period in which the crop, which is the target object actually included in a certain area, grew. The item of the coefficient indicates a value for correcting the estimated value, and the larger the ratio of the corresponding sunny day, the larger the value.

FIG. 17 is a flowchart illustrating an example of an estimation process using estimation parameters.

In step S1701, the correction information acquisition unit 1501 averages the number of detections in the table 401 to acquire the average number of detections for the year corresponding to the estimation target of the actual number of target objects. The correction information acquisition unit 1501 acquires, from the estimation parameters registered in the table 1601, the estimation parameter whose corresponding average detection number value is closest to the acquired average detection number. The correction information acquisition unit 1501 selects the acquired estimation parameter as the estimation parameter used for the estimation processing.

The estimation device 100 can acquire the estimation parameter learned under the condition close to the condition where the region to be subjected to the estimation processing is located without learning the estimation parameter again by the process of S1701. Accordingly, the estimation device 100 can prevent the accuracy of the estimation process from decreasing while reducing the load of the process related to learning.

In step S1702, the correction information acquisition unit 1501 uses, for example, the weather information acquired using an external weather service or the like to generate a target object in the year corresponding to the region to be estimated, during the growing period of the crop, which is the target object. Get the percentage of sunny days. The correction information acquisition unit 1501 acquires, from the table 1602, a coefficient corresponding to the acquired percentage of a sunny day.

In S1703, the estimating unit 206 obtains an estimated value of the actual number of target objects by using Expression (1), for example, using the estimation parameter selected in S1701. Then, the estimating unit 206 corrects the estimated value by multiplying the obtained estimated value by the coefficient obtained in S1702, and sets the corrected estimated value as a final estimated value.

As described above, in the present embodiment, the estimation device 100 acquires a coefficient used for correcting the estimated value of the object based on the weather information (the ratio of a sunny day). Then, the estimation device 100 uses the obtained coefficient to correct the estimated value of the actual number of target objects. Thereby, the estimating apparatus 100 can obtain an estimated value of the actual number of target objects with higher accuracy than in the first embodiment.

Further, in the present embodiment, harvesting of agricultural products is performed once a year. However, the harvest is not limited to once a year, but may be applied to agricultural products harvested a plurality of times a year. In that case, the data managed for each year may be changed to be managed for each growth period.

Further, in the present embodiment, the estimation device 100 uses the ratio of sunny days to obtain the coefficient. However, the estimation device 100 may use an average value, an integrated value, or the like of sunshine hours, precipitation, temperature, and the like. .

Furthermore, the estimation device 100 uses the average number of detections used to acquire the estimation parameters described in the present embodiment and the ratio of sunny days used to acquire correction information as one of the feature amounts. Is also good. Further, the estimation device 100 may acquire the estimation parameter and the correction information by using a part of the feature amount described in the third embodiment.

<Embodiment 5>
In the first embodiment, the estimation device 100 performs a process of generating learning data used for learning an estimation parameter, a process of learning an estimation parameter, and a process of estimating the actual number of target objects. However, these processes need not be executed by a single device.

In the present embodiment, a case will be described in which the processing of generating learning data, the processing of learning estimation parameters, and the processing of estimating the actual number of target objects are performed by separate devices.

FIG. 18 is a system configuration of an information processing system that executes a process of generating learning data used for learning an estimation parameter, a process of learning an estimation parameter, and a process of estimating the actual number of target objects using the estimation parameter in the present embodiment. FIG.

The information processing system includes a generating device 1801, a learning device 1802, and an estimating device 1803. The hardware configuration of each of the generating device 1801, the learning device 1802, and the estimating device 1803 is the same as the hardware configuration of the estimating device 100 of the first embodiment illustrated in FIG.

The function of the generation device 1801 and the processing of the generation device 1801 illustrated in FIG. 18 are realized by the CPU of the generation device 1801 executing the processing based on the program stored in the ROM, the HDD, or the like of the generation device 1801. The function of the learning device 1802 and the processing of the learning device 1802 illustrated in FIG. 18 are realized by the CPU of the learning device 1802 executing the processing based on the program stored in the ROM, the HDD, or the like of the learning device 1802. The functions of the estimating device 1803 and the processing of the estimating device 1803 shown in FIG. 18 are realized by the CPU of the estimating device 1803 executing the processing based on the programs stored in the ROM, the HDD, or the like of the estimating device 1803.

The functional configuration of each of the generation device 1801, the learning device 1802, and the estimation device 1803 will be described.

The generation device 1801 includes a number acquisition unit 1811, an image acquisition unit 1812, a feature amount acquisition unit 1813, and a generation unit 1814. The number acquisition unit 1811, the image acquisition unit 1812, and the feature amount acquisition unit 1813 are the same as the number acquisition unit 201, the image acquisition unit 202, and the feature amount acquisition unit 204 in FIG. The generation unit 1814 generates learning data, and stores the generated learning data in a format such as a table 301 or CSV in an HDD or the like of the generation device 1801. The generation unit 1814 generates learning data by executing, for example, processing similar to S501 to S503 in FIG.

The learning device 1802 includes a learning unit 1821 and a parameter management unit 1822. The learning unit 1821 and the parameter management unit 1822 are functional components similar to the learning unit 203 and the parameter management unit 205 in FIG. 2, respectively. That is, the learning unit 1821 obtains the learning data generated by the generation device 1801 from the generation device 1801, and executes the same processing as S504 to S505 in FIG. 6 based on the obtained learning data (information in the table 301). Then, the estimation parameters are learned. Then, the parameter management unit 1822 stores the estimated parameters learned by the learning unit 1821 in the HDD or the like of the learning device 1802.

The estimation device 1803 includes an image acquisition unit 1831, a feature amount acquisition unit 1832, an estimation unit 1833, and a display control unit 1834. The image acquisition unit 1831, the feature acquisition unit 1832, the estimation unit 1833, and the display control unit 1834 are the same as the image acquisition unit 202, the feature acquisition unit 204, the estimation unit 206, and the display control unit 207 of FIG. That is, the image acquisition unit 1831, the feature amount acquisition unit 1832, and the estimation unit 1833 execute the same processing as in FIG. 7 to thereby determine the actual number of target objects included in the target region in which the number of target objects is estimated. Is performed.

As described above, in the present embodiment, the respective devices execute the processing of generating the learning data used for learning the estimated parameters, the processing of learning the estimated parameters, and the processing of estimating the actual number of target objects. This makes it possible to distribute the burden of each process to a plurality of devices.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

For example, a part or all of the functional configuration of the estimation device 100 described above may be implemented as hardware in the estimation device 100, the generation device 1801, the learning device 1802, the estimation device 1803, and the like.

Although an example of the embodiment of the present invention has been described above, the present invention is not limited to the specific embodiment. For example, the above embodiments may be arbitrarily combined.

The present invention is not limited to the above embodiment, and various changes and modifications can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to make the scope of the present invention public.

This application claims priority based on Japanese Patent Application No. 2018-141430 filed on Jul. 27, 2018 and Japanese Patent Application No. 2019-097874 filed on May 24, 2019, The entire contents of that description are incorporated herein.

Claims (20)

1. A feature acquisition unit configured to acquire a feature amount of the region related to the number of target objects detected from the image from an image of a region that is a part of a field where a crop grows,
   Number acquisition means for acquiring the actual number of the target objects present in a set area of the field,
   The feature amount obtained by the feature obtaining unit from the image obtained by capturing the set region, and the actual number obtained by the number obtaining unit as learning data, in the designated area of the field Learning means for learning an estimation parameter for estimating the actual number of existing target objects;
   An information processing apparatus comprising:
2. The information processing apparatus according to claim 1, wherein the target object is at least one of a bud, a fruit, a flower, and a bunch of the crop.
3. The target object included in the designated area based on the feature amount acquired by the feature acquiring unit from the image obtained by photographing the designated area and the estimation parameter learned by the learning unit. 2. The information processing apparatus according to claim 1, further comprising an estimating unit for estimating the actual number.
4. The estimating means corresponds to the feature quantity from the feature quantity of the designated area and a plurality of the estimation parameters learned by the learning means each associated with an index value in a preset index. 4. The method according to claim 3, further comprising: estimating an actual number of the target objects included in the designated area based on the selected estimation parameter based on the index value of the index. Information processing device.
5. The estimating means is based on the feature amount of the designated area, the estimation parameter learned by the learning means, and correction information used for correcting a value estimated using the estimation parameter. The information processing apparatus according to claim 4, wherein an actual number of the target objects included in the designated area is estimated.
6. A feature acquisition unit configured to acquire a feature amount of the region related to the number of target objects detected from the image from an image of a region that is a part of a field where a crop grows,
   The feature amount acquired by the feature acquiring unit from an image of a designated area in the field, and the feature amount acquired by the feature acquiring unit from the image of the field region, Estimating the actual number of the target objects included in the specified area based on an estimation parameter that is a parameter learned in advance used for estimating the actual number of the target objects included Estimating means;
   An information processing apparatus comprising:
7. The estimation means, the feature amount acquired by the feature acquisition means from an image of the designated area, from the plurality of estimation parameters each associated with an index value in a preset index, An actual number of the target objects included in the set area is estimated based on the selected estimation parameter based on the index value of the index corresponding to the feature amount. The information processing device according to claim 5.
8. The estimating means includes: the feature amount acquired by the feature acquiring means from an image obtained by photographing the designated area; the estimation parameter; and correction information used for correcting a value estimated using the estimation parameter. The information processing apparatus according to claim 5, wherein an actual number of the target objects included in the set area is estimated based on:
9. 9. The apparatus according to claim 3, further comprising a display control unit configured to display on a predetermined display a harvest amount of the crop estimated based on an actual number of the target objects estimated by the estimation unit. The information processing device according to claim 1.
10. The display control unit may further include, in response to an operation by a user, the number of the target objects detected in a predetermined range serving as a basis for predicting the yield of the crop and the target object estimated by the estimation unit. The information processing apparatus according to claim 9, wherein control is performed to display the number and the number on the predetermined display.
11. A feature acquisition unit configured to acquire a feature amount of the region related to the number of target objects detected from the image from an image of a region that is a part of a field where a crop grows,
    Number acquisition means for acquiring the actual number of the target objects present in a set area of the field,
    The feature amount acquired by the feature acquisition unit from the image obtained by photographing the set region and the actual number acquired by the number acquisition unit are associated with each other, and a designated region in the field is designated. Learning data used for learning estimation parameters used in an estimation process for estimating the actual number of the target objects included in the specified region from the feature amount acquired by the feature acquisition unit from the image obtained by capturing the image. Generating means for generating
    An information processing apparatus comprising:
12. The feature acquisition unit may include a number of the target objects detected from a region included in the image determined based on a predetermined object appearing in an image of the part of the region. The information processing apparatus according to claim 1, wherein the feature amount is obtained based on the information.
13. The feature acquiring unit is determined based on one preset object that partially appears in a plurality of images obtained by capturing the region that is the part, and is detected from an area included in the plurality of images. The information processing apparatus according to claim 12, wherein the feature amount is acquired based on the detected number, which is the number of target objects.
14. The feature acquiring unit is configured to perform the feature based on the detected number and information indicating a preset amount of an obstruction that can impede detection of the target object existing in the part of the region. 14. The information processing apparatus according to claim 1, wherein an amount is acquired.
15. 15. The method according to claim 1, wherein the characteristic acquiring unit acquires the characteristic amount based on the detected number and a characteristic of soil in the part of the region. An information processing apparatus according to claim 1.
16. 2. The feature acquiring unit according to claim 1, wherein the feature acquiring unit acquires the feature amount based on the detected number and a feature of a region set as a region around the partial region. The information processing apparatus according to any one of claims 15 to 15.
17. An information processing method executed by an information processing apparatus,
    A feature acquisition step of acquiring a feature amount of the area related to the number of target objects detected from the image from an image of an area that is a part of a field where a crop grows,
    A number acquisition step of acquiring the actual number of the target objects present in a set area of the field,
    The feature amount obtained in the feature obtaining step from the image obtained by capturing the set area, and the actual number in the number obtaining step as the learning data, the learning quantity is present in a designated area in the field. A learning step of learning estimation parameters for estimating the actual number of target objects,
    An information processing method comprising:
18. An information processing method executed by an information processing apparatus,
    A feature acquisition step of acquiring a feature amount of the area related to the number of target objects detected from the image from an image of an area that is a part of a field where a crop grows,
    The feature amount obtained in the feature obtaining step from an image obtained by capturing a designated area in the field, and the feature amount obtained in the feature obtaining step from the image obtained by capturing the field of the field into the area. Estimating the actual number of the target objects included in the specified area based on an estimation parameter that is a parameter learned in advance used for estimating the actual number of the target objects included An estimation step;
    An information processing method comprising:
19. A feature acquisition step of acquiring a feature amount of the area related to the number of target objects detected from the image from an image of an area that is a part of a field where a crop grows,
    A number acquisition step of acquiring the actual number of the target objects present in a set area of the field,
    The feature amount obtained in the feature obtaining step from the image obtained by capturing the set area and the actual number obtained in the number obtaining step are associated with each other, and a designated area in the field is designated. Learning data used for learning estimation parameters used in an estimation process for estimating the actual number of the target objects included in the specified area from the feature amount acquired in the feature acquisition step from an image obtained by photographing A generating step of generating
    An information processing method comprising:
20. A program for causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 16.

Images