JP2012198688A - Crop image processing program, crop image processing method and crop image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a crop image processing program, a crop image processing method and a crop image processing system in which images of crop are processed.SOLUTION: Images of crop planted in a ridge are captured at appropriate timing of a sprouting period, a growing period and a harvest period, and the obtained images are acquired by an image acquisition part 11. An area separation processing part 13 separates an area expressing crop from an area expressing background such as soil/weed etc. In the case where the area separation is performed for images captured in the sprouting period, a ridge line detection part 14a detects a ridge line by performing straight line detection using an image after the area separation. Further, in the case where the area separation is performed for images in the growing period and the harvest period, a growth line detection part 14b detects a growth line by performing straight line detection. A calculation part 15 calculates the height of crop by using the ridge line and the growth line which are detected.

Description

  The present invention relates to a crop image processing program, a crop image processing method, and a crop image processing apparatus for processing crop images.

  In agricultural corporations with large-scale fields, it is important to predict the yield from a management perspective. At present, the yield is predicted based on the intuition and experience of farmers.

  The yield depends on various factors such as the weather, pests, and work performed. For example, weather factors (lightning strikes, strong winds, heavy rain, temperature, etc.), growth factors (physiological disorders, pests, wildlife damage, etc.) and human factors (forgetting planting, cultivating soil mistakes, etc.) can be mentioned.

  On the other hand, a yield prediction method using aerial / satellite images is disclosed (see, for example, Patent Document 1). In this case, a spectral reflected light image is used to acquire a specific wavelength reflected by the crop. By analyzing this image, the crop area and the degree of growth in the field to be observed are estimated, and the yield is predicted.

JP 2010-140296 A

  However, in the prediction method described in Patent Document 1, since an image is taken from directly above the field scene, the crop yield calculated from this image only reflects the crop area. It is not possible to measure the exact size of crops based on the height of the plant.

  In order to solve the above-described problem, the present application calculates the height of a crop planted linearly in a farm scene using an image, and uses the crop image processing program and crop that can be used for crop yield prediction An object is to provide an image processing method and a crop image processing apparatus.

  The crop image processing program disclosed in the present application is a first image captured at a first growth stage of the crop, with reference to storage means in which a plurality of crop images to be planted linearly in a field are stored in a computer. An image of a region representing the crop is separated from the image based on information on a hue range representing a predetermined color of the crop, and a field scene is derived from the image of the region representing the crop separated from the first image. The first straight line asymptotically approaching the planting point of the crop is detected and stored in the storage means. With reference to the storage means, the first straight line of the crop is detected from a position substantially the same as the first image. Separating an image of a region representing a crop on the first line based on the information on the hue range and the first line from a second image captured in a second growth stage slower than the one growth stage; Separated from the second image A second line passing through the upper end of the crop based on the image of the area representing the crop, and the height of the crop based on the distance between the first line and the second line. It is characterized in that a process for calculating is executed.

  According to the present application, the height of a crop planted linearly in a farm scene can be calculated using an image, and can be used, for example, for crop yield prediction.

It is explanatory drawing explaining the outline | summary of the crop height calculation system which concerns on this Embodiment. It is a schematic diagram which shows the functional structure of the calculation apparatus which concerns on this Embodiment. It is a schematic diagram which shows the hardware constitutions of a calculation apparatus. It is a flowchart explaining the procedure which detects a shoreline. It is a schematic diagram which shows an example of the image of a process target. It is a schematic diagram which shows an example of the image which performed background separation and segmentation. It is a schematic diagram which shows an example of the detection result of a shoreline. It is a flowchart explaining the procedure which calculates the height of agricultural products. It is a schematic diagram which shows an example of the image of a process target. It is explanatory drawing explaining the calculation method of the height using perspective transformation. It is a schematic diagram which shows the functional structure of the yield prediction apparatus which concerns on Embodiment 2. FIG. It is a schematic diagram which shows an example of a weight function. It is a flowchart explaining the calculation procedure of the predicted yield by the harvest amount prediction apparatus. It is a figure which shows an example of the actual value of a crop area.

Hereinafter, the present application will be specifically described with reference to the drawings illustrating embodiments thereof.
Embodiment 1 FIG.
FIG. 1 is an explanatory diagram for explaining the outline of the crop height calculation system according to the present embodiment. The crop height calculation system according to the present embodiment includes an imaging device 1 and a calculation device 10. The imaging device 1 is a digital camera, for example. The imaging device 1 is installed at a fixed point in the field or in the vicinity of the field, periodically images crops planted in the field, and sends the obtained image (digital color image) to the calculation device 10. For this reason, the imaging device 1 has a wired or wireless communication function, and is configured to transmit an image obtained by imaging to the calculation device 10 by the communication function.

It is assumed that cocoons are formed in the field, and crops are planted along the cocoons at appropriate intervals. That is, it is assumed that a plurality of lines of linear crops are formed in the field. The imaging device 1 takes a bird's-eye view of the field from a fixed position higher than the height of the grown crop so that the imaging direction (the optical axis of the camera) is substantially parallel to the cocoon. The imaging device 1 is installed at a fixed point in the field or in the vicinity of the field with the camera angle fixed.
Note that the field range captured by the imaging device 1 may be a part of the field. Moreover, it is good also as a structure which installs the imaging device 1 in several places, and images the whole agricultural field.

The crop planted in the field is, for example, a crop belonging to leafy vegetables. In the crop height calculation system according to the present embodiment, it is possible to suitably calculate the height of a crop that grows in a vertical direction up to a certain height during the seedling period, the growth period, or the harvest period of the crop. In addition, the target crop is preferably a crop that is removed from the field after harvesting.
For example, it is possible to suitably calculate the heights of crops belonging to Lamiaceae leafy vegetables, Lariaceae leafy vegetables, leek, leek and the like. Also, the height can be calculated for crops such as rice, wheat, corn and other cereals, non-heading lettuce, cauliflower, cabbage, Chinese cabbage, broccoli, lettuce and the like.
On the other hand, root vegetables such as potatoes and carrots are excluded from the height calculation targets of the present embodiment.

  The calculation device 10 is an information processing device such as a personal computer or a server device. The calculation device 10 acquires an image sent from the imaging device 1 and calculates the height of the crop using the characteristics of the image. Although the calculation method of the height will be described in detail later, the calculation device 10 according to the present embodiment separates the region obtained from the image capturing device 1 from the region representing the agricultural product and the background region. Execute. Further, the calculation device 10 detects a straight line (a cocoon line) representing the position of the cocoon from the image captured by the imaging device 1 during the seedling raising period, and the crop of the crop from the image captured by the imaging device 1 during the growing season or the harvesting season. Processing for detecting a straight line (growth line) representing the upper end portion (straight line detection processing) is executed. Furthermore, the calculation device 10 executes a calculation process for calculating the height of the crop by applying perspective transformation using the detected shoreline and growth line.

  In the present embodiment, the imaging device 1 is configured to periodically image crops planted on the farm field. However, the calculation device 10 periodically gives an imaging instruction to the imaging device 1 to perform imaging. It may be configured that the imaging device 1 captures an image of a crop when an instruction is given. Moreover, it is good also as a structure which images at an appropriate timing in the seedling growing season, the growing season, and the harvesting season. In this case, it is good also as a structure imaged at the timing which a user instruct | indicates. Furthermore, the period of covering such as a passlight or a vinyl tunnel may be excluded from the imaging target.

  In the present embodiment, the image captured by the imaging device 1 is configured to be transmitted to the calculation device 10 by a wired or wireless communication function. However, the image captured by the imaging device 1 in an appropriate period is used. A configuration may be adopted in which an image is stored in a portable recording medium and an image is taken into the calculation apparatus 10 using the portable recording medium.

  FIG. 2 is a schematic diagram showing a functional configuration of the calculation apparatus 10 according to the present embodiment. The calculation device 10 according to the present embodiment includes an image acquisition unit 11, an image storage unit 12, a region separation processing unit 13, a straight line detection unit 14, a calculation unit 15, and an output unit 16.

  The image acquisition unit 11 acquires an image captured by the imaging device 1. Therefore, the image acquisition unit 11 has at least one of a function of receiving an image transmitted from the imaging device 1 and a function of taking an image stored in a portable recording medium into the calculation device 10. The image acquisition unit 11 outputs the acquired image to the image storage unit 12.

  The image storage unit 12 acquires and stores the image output by the image acquisition unit 11. At this time, the image storage unit 12 stores an image by designating an appropriate storage location in a storage area (not shown) included in the calculation device 10, information for identifying the image, information for identifying the storage location of the image, image Are stored in association with each other. In addition, when acquiring and storing images of a plurality of imaging devices 1, the image storage unit 12 specifies a field that has been captured at the same camera angle, such as the imaging device 1 that captured the images and the imaging position of the images. Preferably, possible information is stored in further association. For example, a directory may be divided and stored in the storage area for each image captured at the same imaging device 1 or shooting position.

The information for identifying the image is, for example, a file name of the image, and a name given when the imaging apparatus 1 generates the image can be used. The information for identifying the storage location of the image is information for specifying the position in the storage area of the image storage unit 12, and is represented by hierarchical information including directories, folders, and the like. The imaging date and time is information (metadata) included in imaging conditions embedded by the imaging device 1 when generating an image. The image storage unit 12 extracts information indicating the imaging date and time from the metadata of the image, and stores the information in association with information for identifying the image and information for identifying the storage location of the image.
Further, in addition to these pieces of information, the image storage unit 12 may be configured to associate and store information that distinguishes the sowing period, the seedling period, the growing period, and the harvest period of the crop. For example, each period is determined in advance, and the image storage unit 12 determines which period it belongs to when acquiring information on the imaging date and time from the metadata, and determines the determined period (seeding period, seedling period, growth period) , Harvest period) can be associated with other information and stored in a storage area (not shown) of the calculation device 10.

  The region separation processing unit 13 performs processing for separating a region representing a crop and a background region on the processing target image. In the present embodiment, the crop is planted on a bank called a reed, but weeds often occur between the reeds. Simply extracting the crop color (green) with image processing will simultaneously extract the weeds that exist in the furrows. Therefore, in the calculation device 10 according to the present embodiment, the region separation processing unit 13 detects the position of the cocoon, and uses the detected position of the cocoon as a reference, the region representing the crop on the image and the others (soil and weeds). ) Separate the background area.

  The region separation processing unit 13 outputs the processed image (that is, an image obtained by separating the region representing the crop and the background region from the captured image) to the straight line detection unit 14. At this time, the region separation processing unit 13 outputs an image (for example, an image captured during the seedling raising period) that is a target for detecting the wrinkle to the wrinkle detection unit 14a of the straight line detection unit 14, and The resulting image (for example, an image captured during the growing season or the harvesting season) is output to the growth line detector 14b of the straight line detector 14.

  Note that the region separation processing unit 13 may perform region separation within a range specified by the user. When the user sets an appropriate range, the region separation processing unit 13 can perform processing by excluding a region in which it is difficult to determine wrinkles by perspective, for example. Hereinafter, the range set by the user during the region separation process is referred to as an observation region.

The straight line detection unit 14 includes a shoreline detection unit 14a and a growth line detection unit 14b.
The shoreline detection unit 14a detects the shoreline of an image captured during the seedling raising period among the images processed by the region separation processing unit 13. For detection of the shoreline, Hough transform, which is generally known as a straight line detection method, can be used. When the shape (straight line) of a line to be detected is determined in advance as in the present embodiment and the shape can be expressed by an algebraic equation, the shoreline detection unit 14a has a parameter space representing the shape of the line. Straight line detection can be performed using Hough transform that maps feature points in an image.
For example, the shoreline detection unit 14a can detect a straight line (a shoreline) by applying Hough transform using a region portion representing an agricultural product as a feature point in the image among the region separated regions. At this time, the shoreline detection unit 14a applies a known hierarchical clustering with the threshold value of the clustering distance ΔdLth or less with respect to the slope ΔLi of each straight line, and for each cluster, the centroid value of the cluster Can be obtained as a straight line that passes through the region representing the crop, that is, a straight line that is asymptotic to the planting points of a large number of crops that exist in the cocoon region.
In the present embodiment, the shoreline is detected using the Hough transform, but the straight line detection method is not limited to the Hough transform. For example, a method of filtering an image using a weighted matrix, a method of extracting a region where a differential value is large using a differential image, a method of thinning, a pixel having a high differential value is sequentially connected using a differential image A technique for extracting a boundary line can be used.

An image captured by the imaging device 1 often includes a plurality of wrinkles. Even if the region separation processing unit 13 performs the region separation processing, the region separation processing unit 13 may not be able to completely remove regions other than agricultural products (for example, regions including weeds). Therefore, the straight line detected by the shoreline detection unit 14a includes a plurality of straight lines other than the shoreline and the shoreline. Therefore, the shoreline detection unit 14a selects a straight line (coastline) corresponding to the heel among the detected straight lines, and excludes other straight lines.
The selection of the shoreline may be automatically performed by the shoreline detection unit 14a, or the selection by the user may be received by the shoreline detection unit 14a. As a method for automatically selecting the shoreline by the shoreline detection unit 14a, a selection method using a vanishing point in the perspective method can be used. Since wrinkles are often formed in parallel and at equal intervals, there are points (disappearance points) where the detected straight lines overlap most in the image. Therefore, the shoreline detection unit 14a can detect only the shoreline by excluding the straight line that does not pass through the vanishing point as a straight line detected by an element other than the heel, and selecting the remaining straight lines. .
The shoreline detection unit 14a stores the detected shoreline information in a storage area (not shown) included in the calculation device 10. For example, the coordinates of the end points on the image of each shoreline, information on algebraic equations representing each shoreline, and the like are stored.

On the other hand, the growth line detection unit 14b detects a growth line for an image captured in the growing season or the harvesting season among the images processed by the region separation processing unit 13.
The growth line detection unit 14b selects a shoreline that can easily calculate the height of the crop among the shorelines detected by the shoreline detection unit 14a. Selection of the shoreline may be performed automatically by the growth line detection unit 14b, or the selection by the user may be received by the growth line detection unit 14b. When the growth line detection unit 14b automatically selects the shoreline, for example, the growth line detection unit 14b loses the shoreline among the two shorelines existing on both sides of the ridgeline having the largest inclination on the image. It is possible to select a shoreline whose end farther from the point exists in the observation region.

The growth line detection unit 14b extracts a region representing a crop portion existing on the selected shoreline using the image obtained by the region separation processing unit 13 performing region separation. Also, using the image captured by the imaging device 1, the color of the crop on the selected shoreline is sampled, the color region representing the crop portion is extracted, and the crop portion and other portions (soil and weeds) are extracted. You may perform the process which isolate | separates an area | region newly.
The growth line detection unit 14b detects a straight line other than the shoreline for the crop region including only the selected shoreline. The straight line detection method may be the same as the straight line detection method performed by the shoreline detection unit 14a. For example, the growth line detection unit 14b can detect a straight line (growth line) by applying the Hough transform using the contour of the region representing the crop as a feature point in the image. At this time, the growth line detection unit 14b applies a known hierarchical clustering with respect to the slope ΔLi of each straight line with the threshold of the distance for clustering being equal to or less than ΔdLth, and is farthest from the center of gravity of the cluster for each cluster. A straight line having a side inclination and passing through the vicinity of the vanishing point of the shoreline can be detected as a growth line (that is, a straight line passing through the upper end portion of the crop).
The growth line detection unit 14b stores information on the detected growth line in a storage area (not shown) included in the calculation device 10. For example, the growth line detection unit 14b stores the coordinates of end points on the growth line image, information on algebraic equations representing the growth line, and the like.

  The calculation unit 15 calculates the height of the crop using the shoreline detected by the shoreline detection unit 14a and the growth line detected by the growth line detection unit 14b. The calculation unit 15 obtains a line segment that vertically connects the end point on the near side of the shoreline selected when detecting the growth line and the growth line, and sets the length of the line segment as the height on the crop image. be able to. Moreover, the calculation unit 15 can calculate the actual height of the crop by converting the obtained height into the actual height using the positional relationship of the imaging device 1 and perspective transformation.

  The output unit 16 outputs the calculation result or the like by the calculation unit 15, that is, the calculated height of the crop. For example, the output of the calculation result by the output unit 16 is performed by displaying the height of the crop calculated by the calculation unit 15 on the screen as character information. The output unit 16 may be configured to notify the height of the crop calculated by the calculation unit 15 to an external device (for example, a terminal device used by the user).

  FIG. 3 is a schematic diagram illustrating a hardware configuration of the calculation apparatus 10. The calculation device 10 includes a CPU 101, ROM 102, RAM 103, communication interface 104, hard disk drive 105, optical disk drive 106, keyboard 107, and display 108.

  The ROM 102 stores in advance a control program necessary for controlling the operation of each hardware unit described above. The hard disk 105D stores in advance a calculation program that causes the calculation apparatus 10 to execute processing for calculating the height of the crop from the captured image.

  The CPU 101 reads out a computer program stored in the ROM 102 or the hard disk 105D at an appropriate timing on the RAM 103 and executes it, thereby controlling the operation of each of the above-described hardware units and realizing the calculation method of the present application as a whole. To act as.

  The RAM 103 is, for example, a DRAM (Dynamic RAM), an SRAM (Static RAM), a flash memory, or the like, and various data generated when the CPU 101 executes a computer program (for example, various parameters, calculation progress, calculation results, etc.) Is temporarily stored.

  The communication interface 104 is an interface for communicating with an external device via a communication network (not shown).

  The hard disk drive 105 controls data writing to the hard disk 105D and data reading from the hard disk 105D. The hard disk drive 105 stores various information in the hard disk 105D by writing information received through the keyboard 107, information received by the communication interface 104, information read from the optical disk 106D by the optical disk drive 106, and the like to the hard disk 105D. .

  The optical disc drive 106 controls writing of data to the optical disc 106D and reading of data recorded on the optical disc 106D. In this embodiment, it is assumed that the computer program of the present application is stored in the hard disk 105D, but it may be provided in a state recorded in the optical disk 106D.

  The keyboard 107 accepts user operations and character input. The display 108 displays information notified to the user.

  Note that the hardware configuration of the calculation device 10 need not be limited to the above. For example, when remote control is possible via a communication network, the optical disk drive 106, the keyboard 107, the display 108, etc. may be omitted.

  The calculation device 10 causes the CPU 101 to execute the calculation program stored in the hard disk 105D and controls each part of the hardware according to the control program stored in the ROM 102, whereby the image acquisition unit 11 and the image storage unit illustrated in FIG. 12, each process by the area separation processing unit 13, the straight line detection unit 14, the calculation unit 15, and the output unit 16 is executed.

  Next, the calculation procedure of the crop height by the calculation device 10 will be described. In the following, a procedure for detecting a wrinkle using an image captured during the seedling period, a procedure for detecting a growth line using an image captured during the growing season or the harvesting period, and then calculating the height of the crop The calculation procedure will be described separately.

  FIG. 4 is a flowchart for explaining a procedure for detecting a shoreline. The region separation processing unit 13 of the calculation device 10 acquires an image captured during the seedling raising period stored in the image storage unit 12 (step S11). FIG. 5 is a schematic diagram illustrating an example of an image to be processed. Fig.5 (a) shows an example of the image imaged in the seedling raising period. In the seedling raising period (especially at the beginning of the seedling raising period), the height of the crop is not so high, so that the cocoon near the center of the image (the cocoon having a large inclination on the image) is easily separated from other cocoons. On the other hand, in the vicinity of the edge of the image, many wrinkles appear close to each other by perspective, and it is difficult to separate them from each other.

  Therefore, the region separation processing unit 13 designates a region in which wrinkles are easily separated as an observation region (step S12). For example, the area indicated by hatching in FIG. 5B is designated as the observation area. The observation region is designated by the region separation processing unit 13 by accepting a user's selection operation. By designating the observation area, the area separation processing unit 13 can exclude areas of wrinkles that appear to approach each other and are difficult to separate from each other.

  Next, the region separation processing unit 13 performs background separation and segmentation (steps S13 and S14). The region separation processing unit 13 can separate the crop candidate portion and the soil portion (background) by adding a process of extracting only a hue range (green) representing a color unique to the crop. In the segmentation, a region where pixel values that are globally similar are continuous is extracted. For example, in order to remove a fine noise change, the image size is reduced, smoothing processing is performed, and a method such as image segmentation by an image pyramid is used. Such segmentation is a general method, and other methods can be used without depending on the above method. FIG. 6 is a schematic diagram illustrating an example of an image subjected to background separation and segmentation. FIG. 6A and FIG. 6B are examples of images that have undergone background separation and segmentation, respectively, where white portions in the images represent crop candidate portions and black portions represent soil portions. Here, FIG. 6A represents an image before the smoothing process, and FIG. 6B represents an image after the smoothing process.

  Next, the shoreline detection unit 14a detects a straight line included in the observation region in the segmented image (step S15). As described above, the shoreline detection unit 14a can detect a straight line by applying a Hough transform, which is generally known as a straight line detection method, to the image after segmentation. Since the detected straight line may include straight lines other than the shoreline, the shoreline detection unit 14a, for example, excludes a straight line that does not pass the vanishing point in the perspective method, and selects the remaining straight lines, A shoreline is selected (step S16). FIG. 7 is a schematic diagram showing an example of the detection result of the shoreline. FIG. 7 shows a state in which eight saddles L1 to L8 are detected by selecting only a straight line passing through the vanishing point Vp in the perspective method.

Next, a procedure for detecting a growth line will be described. FIG. 8 is a flowchart for explaining the procedure for calculating the height of the crop. The region separation processing unit 13 of the calculation apparatus 10 acquires an image captured during the growing season or the harvesting season stored in the image storage unit 12 (step S21). FIG. 9 is a schematic diagram illustrating an example of an image to be processed. FIG. 9A shows an example of an image captured during the growing season. In the present embodiment, the region separation processing unit 13 is configured to always image the field with the same camera angle by the imaging device 1 installed at a fixed point in the whole period of the seedling growing season, the growing season, and the harvesting season. The shoreline detected in the seedling stage can be used as it is as the shoreline in the seedling stage and the growing stage.
FIG. 9B shows a state in which the wrinkle detected in the seedling season is superimposed on the image captured in the growth season. As can be understood from FIG. 9 (b), there are cocoons that can easily calculate the height of the crop (the cocoons indicated by the shoreline L5) and cocoons that are difficult or impossible to calculate.

  Therefore, the region separation processing unit 13 selects an observation target shoreline from which the height of the crop can be easily calculated (step S22). For example, among the saddle lines L1 to L8 shown in FIG. 9B, the saddle line having the largest inclination is the saddle line L4. Since the growth line of the crop on the shoreline L4 substantially overlaps with the shoreline L4, it is difficult to detect the growth line. Therefore, in the present embodiment, attention is paid to the shorelines L3 and L5 existing on both sides of the ridgeline L4 having the largest inclination, and the end point far from the vanishing point is present in the observation region. The shoreline L5 to be selected is selected as the shoreline to be observed.

  The region separation processing unit 13 samples the color of the crop portion on the shoreline selected in step S22 (step S23), and performs segmentation of the crop region (step S24). That is, the region separation processing unit 13 performs region separation on the crop portion to be observed and the other portions (soil, weeds, crop portions that are not to be observed).

  Next, the growth line detection unit 14b detects a straight line other than the shoreline for the crop region including only the selected shoreline (step S25). As described above, the straight line detection method may be the same as the straight line detection method performed by the shoreline detection unit 14a. Further, the growth line detection unit 14b selects only a straight line passing through the vicinity of the vanishing point of the shoreline as a growth line (step S26). FIG. 9C shows a state where the growth line L0 is detected with respect to the shoreline L5 and the vanishing point Vp.

  Next, the calculation unit 15 calculates the height of the crop image on the image using the shoreline detected by the shoreline detection unit 14a and the growth line detected by the growth line detection unit 14b (step S27). For example, as shown in FIG. 9C, the calculation unit 15 obtains a line segment that vertically connects the end point X0 on the near side of the shoreline L5 selected when detecting the growth line L0 and the growth line L0. The length h1 of the line segment can be the height of the crop image.

  Then, the calculation unit 15 calculates the actual height of the crop by converting the obtained height to the actual height using perspective transformation (step S28). FIG. 10 is an explanatory diagram for explaining a height calculation method using perspective transformation. As shown in FIG. 10, the height of the imaging device 1 is y0, the horizontal distance from the imaging device 1 to the end of the shoreline to be observed is z2, the horizontal distance to the image projection plane is z1, and the crop calculated in step S27. When the height on the image is h1, the calculation unit 15 can calculate the actual height h2 of the crop from the similarity relationship.

As described above, in the present embodiment, the region separation process that separates the region representing the crop from the background region from the image captured from the field, and the shoreline, the growing season, or the harvester from the image captured during the seedling growing season. A straight line detection process for detecting a growth line is executed from the captured image, and the actual height of the crop can be calculated using the detected shoreline and growth line.
The state of the cocoon can change over time due to the influence of rainfall, irrigation, fertilization, etc., but in this embodiment, the cocoon line is determined in the initial stage (nursing stage), When detecting the height, the height of the crop is calculated based on a predetermined shoreline, so the actual height of the crop can be accurately calculated without being affected by the surrounding environment such as soil. Become.

Embodiment 2. FIG.
In the first embodiment, the height of the crop is calculated. However, it is possible to predict the crop yield by comparing the calculated height with the reference height at the same time. In the second embodiment, a mode of predicting the crop yield will be described.

FIG. 11 is a schematic diagram showing a functional configuration of the yield prediction device 20 according to the second embodiment. In addition to the image acquisition unit 11, the image storage unit 12, the region separation processing unit 13, the straight line detection unit 14, and the output unit 16 described in the first embodiment, the yield prediction device 20 according to the second embodiment has a height. A calculation unit 21, an area calculation unit 22, a standard index DB 23, an agricultural field information DB 24, and a harvest amount prediction unit 25 are provided.
Since the functions of the image acquisition unit 11, the image storage unit 12, the region separation processing unit 13, the straight line detection unit 14, and the output unit 16 are the same as those in the first embodiment, description thereof will be omitted. The height calculation unit 21 is exactly the same as the calculation unit 15 described in the first embodiment, and functions as a processing unit that calculates the height of the crop using the detected shoreline and growth line.

  The area calculation unit 22 calculates the area of the crop using the field image captured by the imaging device 1 in each of the seedling growing season, the growing season, and the harvesting season. The number of shore lines detected by the straight line detection unit 14 (the shore line detection unit 14a) is n, and each shore line is defined by Li (i = 1 to n). The area calculation unit 22 calculates pixels in the hue range (green) representing the color of the crops around each shoreline Li. The region for calculating the number of pixels may be a fixed region using a threshold in the horizontal direction around each shoreline Li, or may be expanded according to the growth stage of the crop. When the calculated number of pixels is S (Li), the area calculation unit 22 can calculate the area of the crop by a total ΣS (Li) of the number of pixels of the crop for all the shorelines.

  The standard index DB 23 is a database that stores a standard yield for each item and variety of agricultural products. The field information DB 24 is a database that stores an actual field area for each field. These databases are configured to return requested information in response to external access requests.

  The harvest amount prediction unit 25 predicts the harvest amount using the crop height calculated by the height calculation unit 21 and the crop area calculated by the area calculation unit 22. The harvest amount prediction unit 25 can calculate the predicted yield, for example, by the following equation.

Predicted yield = standard yield x field area x
{1+ (Crop area ratio-1) × Weight 1+ (Crop height ratio-1) × Weight 2}

  Here, since the standard yield varies depending on the items and varieties of crops planted in the field to be observed, the corresponding standard harvest is acquired from the standard index DB 23. Further, the field area to be observed is acquired from the field information DB.

  The crop area ratio represents the ratio of the crop area calculated by the area calculation unit 22 to the standard crop area (for example, the crop area in the year in which the yield was standard) in the observation region at the same time period. On the other hand, the crop height ratio is similarly calculated by the height calculation unit 21 with respect to the standard crop height (for example, the height of the crop in the year in which the yield was standard) in the observation region of the same period. Represents the ratio.

  The weights (weight 1 and weight 2) in the above calculation formula differ depending on the number of days elapsed from the planting date. In general, the variation in growth immediately after planting (seeding period to seedling period) has a low probability of contributing to the yield, but the variation in the growth status from the growth period to the harvest period has a high probability of contributing to the yield. Therefore, it is preferable that the harvest amount prediction unit 25 uses a function that increases as the number of elapsed days t from the planting date advances as the weight function Wi (t) (where 0 ≦ Wi (t) ≦ 1). FIG. 12 is a schematic diagram illustrating an example of a weight function. Since the value of the weight function Wi (t) in the sowing period is close to zero, the predicted yield is a value based on the standard yield. Further, since the value of the weight function Wi (t) approaches 1 as the harvest date approaches, the predicted yield that reflects the height of the crop calculated by the height calculation unit 21 and the area of the crop calculated by the area calculation unit 22 Is obtained by the above calculation formula.

Also, the weight function varies depending on the item and variety of the crop. For example, the timing at which the size and height change most in the growth stage varies depending on the item. In this way, a function that increases the weight is set at a timing at which the change appears remarkably. It is assumed that the weighting function is already known and is registered in advance in the standard index DB 23, and the yield prediction unit 25 extracts the weighting function corresponding to the item and the product type.
Further, the function of the weight 1 related to the crop area ratio and the function of the weight 2 related to the crop height ratio can be made different depending on the item or variety of the crop. For example, when the height change is large during the growing season and the area change is large during the harvesting season, the weighting function in the growing season is set so that weight 1 <weight 2 and the weighting function in the harvesting season is set. Can be set such that weight 1> weight 2. Therefore, as many weight functions as the number of items × number of products × weight types (weight 1 and weight 2) are registered in the standard index DB 23.

Hereinafter, the procedure for calculating the predicted yield will be described. FIG. 13 is a flowchart for explaining the procedure for calculating the predicted yield by the yield prediction device 20. It is assumed that the image acquisition unit 11 acquires images from the imaging apparatus 1 at regular or appropriate timings during the seedling period, the growing period, and the harvest period, and stores them in the image storage unit 12 as needed.
The region separation processing unit 13 acquires one image from the image storage unit 12 (step S101), and determines whether or not it is the first determination (step S102). Only in the case of the first determination (S102: YES), the region separation processing unit 13 accepts designation of the observation region as shown in FIG. 5B (step S103).

  Next, the region separation processing unit 13 determines whether or not the image acquired from the image storage unit 12 is an image captured in the sowing period (step S104). When it is determined that the image is picked up in the sowing period (S104: YES), the predicted yield at that time is calculated by the yield prediction unit 25 (step S110), and the output unit 16 outputs the prediction result. (Step S111).

  When it is determined that the image acquired from the image storage unit 12 is not an image captured in the sowing period (S104: NO), the region separation processing unit 13 is an image in which the image acquired from the image storage unit 12 is captured in the seedling period Is determined (step S105). When it is determined that the image is captured during the seedling raising period (S105: YES), the region separation processing unit 13 performs a region separation process for separating a region representing a crop and a region representing a background such as soil and weeds. The region-separated image is output to the shoreline detection unit 14a.

  The shoreline detection unit 14a performs shoreline detection using the image output from the region separation processing unit 13 (step S107). The shoreline detection procedure follows the flowchart shown in FIG. Next, the shoreline detection unit 14 a outputs the region-separated image and the detected shoreline information to the area calculation unit 22.

  The area calculation unit 22 calculates the area of the crop using the image and the information on the shoreline output from the shoreline detection unit 14a (step S109). That is, the area calculation unit 22 counts the pixels in the hue range (green) representing the color of the crop with respect to the periphery of each shoreline detected by the shoreline detection unit 14a, and totals all the shorelines. Thus, the area of the crop can be calculated. After calculating the area of the crop, the process proceeds to step S110, the predicted yield in the seedling season is calculated by the yield prediction unit 25 (S110), and the prediction result is output by the output unit 16 (S111). .

  In step S105, when the region separation processing unit 13 determines that the image acquired from the image storage unit 12 is not an image captured during the seedling raising period (S105: NO), the region separation processing unit 13 reads from the image storage unit 12. It is determined whether or not the acquired image is an image captured during the growing season or the harvesting season (step S106). When it is determined that the region separation processing unit 13 is an image captured in the growing season or the harvesting season (S106: YES), the region separation processing unit 13 represents a region representing a crop and a background such as soil / weed. A region separation process for separating the region is performed, and the region-separated image is output to the growth line detection unit 14b.

The growth line detection unit 14b detects a growth line using the image output from the region separation processing unit 13, and calculates the height of the crop using the detected growth line and the already detected shoreline. (Step S108). The procedure for calculating the height of the crop is assumed to follow the flowchart shown in FIG. Next, the growth line detection unit 14b outputs the region-separated image and shoreline information to the area calculation unit 22, and the area calculation unit 22 calculates the area of the crop (S109).
After calculating the area of the crop, the process proceeds to step S110, the yield prediction unit 25 calculates the predicted harvest amount in the growing season or the harvest season (S110), and the output unit 16 outputs the prediction result (S111). ). Note that the harvest amount predicting unit 25 may calculate the predicted harvest amount even when it is determined in step S106 that the image is not an image captured during the growing season or the harvesting season (S106: NO).

  As described above, according to the present embodiment, it is possible to predict the harvest amount according to the growth stage of the crop. In particular, in the growing season and the harvesting season, it is possible to predict the crop yield that reflects both the area and height of the crop, so obtain a predicted value that reflects the actual size of the crop and the density per unit area. It is possible to accurately and objectively predict the yield.

  Depending on the image capturing timing, the direction of the leaves and stems of the crop due to wind and rain changes, so the apparent crop area and height change. FIG. 14 is a diagram illustrating an example of an actual measurement value of a crop area. As shown in FIG. 14, the actual value of the crop area varies depending on the imaging timing. For example, the actual value is corrected based on the latest observed value, and the corrected value (corrected value) is used. The yield may be predicted. By using the correction value, it is possible to reduce the influence of the fluctuation of the actual measurement value accompanying the imaging timing, and it is possible to predict the harvest amount more accurately. For the above correction, for example, a regression analysis such as a least square method can be used.

In the present embodiment, it is desirable that the crop to be targeted for yield prediction has the following characteristics. Also in the second embodiment, since the height of the crop is calculated in order to predict the yield, it is a crop that grows vertically to a certain height as in the first embodiment. Is preferably removed from the field. For example, vegetables that crawl on the ground, such as melons, are not suitable for prediction. In addition, fruit trees are not suitable for prediction because trees remain after harvesting.
In addition, crops with edible parts on the ground are suitable. For example, root vegetables such as potato, sweet potato and carrot are not suitable for prediction.
Furthermore, crops in which the edible portion occupies the majority of the crop are suitable. For example, crops that are edible for fruits and seeds are not suitable for predicting the yield according to the present embodiment because the height and area of the crop may not be correlated with the yield.

DESCRIPTION OF SYMBOLS 1 Imaging device 10 Calculation apparatus 11 Image acquisition part 12 Image memory | storage part 13 Area separation process part 14 Straight line detection part 14a Saddle line detection part 14b Growth line detection part 15 Calculation part 16 Output part

Claims (10)

On the computer,
A hue representing a predetermined color of the crop from the first image captured in the first growth stage of the crop with reference to storage means storing a plurality of crop images to be planted linearly in the field Based on the information of the range of, the image of the area representing the crop is separated,
From the image of the region representing the crop separated from the first image, a first straight line asymptotic to the planting point of the crop in a farm scene is detected and stored in the storage means,
With reference to the storage means, information on the hue range from a second image captured in a second growth stage that is slower than the first growth stage of the crop, from substantially the same position as the first image, and the Separating an image of a region representing a crop on the first line based on the first line;
Detecting a second straight line passing through the upper end of the crop based on the image of the region representing the crop separated from the second image;
A crop image processing program for executing a process of calculating a height of the crop based on a distance between the first straight line and the second straight line. In the computer,
A plurality of straight lines are detected based on the image of the region representing the crop, and one end of the straight lines converges to the same point on the image among the plurality of straight lines, or the plurality of straight lines are 2. The crop image processing according to claim 1, wherein a process of detecting the first straight line is executed by selecting a plurality of straight lines that pass through the same point when extended outside the image. program. In the computer,
3. The process of detecting, as the first straight line, a straight line having an end farther from the same point on the image among the plurality of first straight lines is executed. Crop image processing program. In the computer,
A process of obtaining a line segment that vertically connects the end point on the imaging device side in the image of the first straight line and the second straight line, and calculating the height of the crop based on the length of the line segment The crop image processing program according to any one of claims 1 to 3, wherein the crop image processing program is executed. In the computer,
5. The process according to claim 1, wherein a process for predicting a yield of the crop is executed by comparing the height of the crop with a reference height stored in advance. A crop image processing program. In the computer,
From the image of the area representing the crop, calculate the crop area of the crop,
The height of the said crop and the said crop area are compared with the reference | standard height and reference | standard area which were memorize | stored previously, The process which estimates the crop yield of the said crop is performed. The crop image processing program according to any one of 4 above. In the computer,
The crop image processing program according to claim 5 or 6, wherein a process for weighting the predicted yield is performed according to a type of crop or a growth stage of the crop. In the computer,
The computer program according to any one of claims 5 to 7, wherein a process for correcting a harvest amount to be predicted is corrected using a harvest amount predicted in the past. Computer
A hue representing a predetermined color of the crop from the first image captured in the first growth stage of the crop with reference to storage means storing a plurality of crop images to be planted linearly in the field Based on the information of the range of, the image of the area representing the crop is separated,
From the image of the region representing the crop separated from the first image, a first straight line asymptotic to the planting point of the crop in a farm scene is detected and stored in the storage means,
With reference to the storage means, information on the hue range from a second image captured in a second growth stage that is slower than the first growth stage of the crop, from substantially the same position as the first image, and the Separating an image of a region representing a crop on the first line based on the first line;
Detecting a second straight line passing through the upper end of the crop based on the image of the region representing the crop separated from the second image;
A crop image processing method, wherein the height of the crop is calculated based on a distance between the first straight line and the second straight line. A storage unit for storing images of a plurality of crops to be planted linearly in a field;
With reference to the storage unit, from the first image captured in the first growth stage of the crop, an image of a region representing the crop is obtained based on information on a hue range representing a predetermined color of the crop. A first region separation processing unit for separation;
A first straight line detection unit that detects a first straight line asymptotic to a planting point of the crop in a farm scene from an image of a region representing the crop separated from the first image, and stores the first straight line in the storage unit;
With reference to the storage unit, information on the hue range from a second image captured in a second growth stage that is slower than the first growth stage of the crop, from substantially the same position as the first image, and the information A second region separation processing unit for separating an image of a region representing the crop on the first straight line based on the first straight line;
A second straight line detector for detecting a second straight line passing through the upper end of the crop based on the image of the region representing the crop separated from the second image;
A crop image processing apparatus comprising: a calculation unit that calculates a height of the crop based on a distance between the first straight line and the second straight line.