CN115392576A - Mushroom growth state space-time prediction method - Google Patents

Mushroom growth state space-time prediction method Download PDF

Info

Publication number
CN115392576A
CN115392576A CN202211044323.1A CN202211044323A CN115392576A CN 115392576 A CN115392576 A CN 115392576A CN 202211044323 A CN202211044323 A CN 202211044323A CN 115392576 A CN115392576 A CN 115392576A
Authority
CN
China
Prior art keywords
mushroom
image
mushrooms
predicted
growth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211044323.1A
Other languages
Chinese (zh)
Inventor
杨淑珍
黄杰
俞涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Polytechnic University
Original Assignee
Shanghai Polytechnic University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Polytechnic University filed Critical Shanghai Polytechnic University
Priority to CN202211044323.1A priority Critical patent/CN115392576A/en
Publication of CN115392576A publication Critical patent/CN115392576A/en
Priority to PCT/CN2023/098790 priority patent/WO2024045749A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G18/00—Cultivation of mushrooms
    • A01G18/60—Cultivation rooms; Equipment therefor
    • A01G18/69—Arrangements for managing the environment, e.g. sprinklers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/02—Agriculture; Fishing; Forestry; Mining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20088—Trinocular vision calculations; trifocal tensor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Business, Economics & Management (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Quality & Reliability (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Primary Health Care (AREA)
  • Animal Husbandry (AREA)
  • Operations Research (AREA)
  • Marine Sciences & Fisheries (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Development Economics (AREA)
  • Mining & Mineral Resources (AREA)
  • Biophysics (AREA)
  • Agronomy & Crop Science (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)

Abstract

The invention discloses a space-time prediction method for mushroom growth state; the method collects color map and depth map historical sequence images in the mushroom growth process, a mushroom growth state diagram at a certain future moment is predicted through an improved convolution long-time and short-time memory neural network (ConvLSTM), and the improved model fuses the ConvLSTM and an example segmentation network and combines an SSIM loss function to improve the quality and prediction accuracy of a predicted image. Then, the predicted color map is segmented through an example segmentation algorithm, mushroom fruiting bodies are extracted, and the mushroom fruiting bodies are matched with the depth map to calculate the size of each mushroom in the predicted image; finally, the growth vigor of the mushrooms is predicted through the predicted size and position characteristics of the mushrooms; the method can replace manual inspection, provide information basis and judgment basis for more accurately and effectively carrying out scientific and reasonable arrangement of crop greenhouse environmental factor control, intelligent picking and bud thinning of crops and the like, and improve the yield and quality of mushrooms.

Description

Mushroom growth state space-time prediction method
Technical Field
The invention belongs to the technical field of intelligent agriculture and the technical field of edible fungi, and particularly relates to a space-time prediction method for mushroom growth states.
Background
The current mushroom greenhouse environment monitoring system only collects greenhouse environment parameters, the environment parameters are preset by a user, a central control unit checks data collected by a sensor and then determines the equipment state in the middle of production by simple comparison according to the preset parameters of the user, and compared with the initial open-loop control system without environment monitoring, the mode obviously improves the real-time performance of the control of the growing environment of edible fungi. However, the growth of crops such as mushrooms is influenced by a lot of factors, the growth process and the growth state of the crops are not controllable like industrial products, the crops are difficult to grow according to the expected set state, the time is long, and the regulation and control effect is often poor. Therefore, the current mushroom culture bases still need to manually visit the mushroom houses at intervals to observe the growth states (growth vigor) of the mushrooms, and then manually perform fine adjustment intervention on environmental parameters according to the growth vigor, the manual observation mode has large workload, the prediction on the growth vigor of the mushrooms is not accurate enough, and the self-adaptive adjustment capability on the actual change of the growth vigor of the mushrooms is poor.
In addition, mushrooms are fruits which are easy to grow densely or in clusters, and the influence on the quality and yield of the fruits caused by growth space limitation and nutrition competition caused by over-dense fruits is generally avoided. Moreover, the follow-up picking time and picking amount of the fruits need to be predicted, so that picking tasks and bud thinning vegetables and fruits can be reasonably arranged better, and most fruits can be picked at the right time. Therefore, to achieve more accurate intelligent control of crop greenhouses, optimization of cultivation and picking management, and improvement of yield and quality, manual or growth state monitoring is insufficient, and automatic and intelligent prediction and early intervention on the growth state are needed.
The technology for predicting the future growth state of the fruit generally adopts a growth model modeling mode at present, and realizes the prediction of the fruit growth by establishing a growth model through researching the influence of fruit growth environmental factors on the growth of the fruit. However, the growing model is usually not accurate because of the strong randomness of crop growth and the many factors affecting the growing process, not only the influence of environmental factors, but also the influence of components, thickness and the like of the culture medium.
Disclosure of Invention
Aiming at the defects existing in the monitoring and prediction of the mushroom growth state in the prior art, the invention aims to provide a mushroom growth state space-time prediction method, which can replace manual monitoring of the mushroom growth state, particularly can perform image prediction on the mushroom growth state, considers the space-time change of the mushroom growth state, fuses a convolution long-time memory neural network ConvLSTM and an example segmentation network, and improves the quality and prediction accuracy of a predicted image of the mushroom growth state by combining an SSIM loss function. According to the invention, through accurate prediction of the mushroom growth state, the problems that manual fine regulation and control are difficult, the growth states of mushroom rooms are different, the regulation and control means are single, the mushroom rooms cannot be regulated and controlled all day after 24 hours and the like in the existing mushroom greenhouse environment regulation and control are solved, more accurate control of greenhouse environment factors is realized, and the mushroom yield and quality are improved; in addition, the method is beneficial to predicting the mature period and the growth aggregation degree of the mushrooms and assisting in carrying out more scientific and reasonable arrangement on cultivation and production management plans such as mushroom picking time, picking personnel, bud thinning plan arrangement and the like.
In order to realize the purpose of the invention, the invention adopts the following technical scheme:
a mushroom growth state space-time prediction method comprises the following steps:
(1) Shooting a plurality of images every x minutes by adopting an RGBD depth camera to obtain a color image and a depth image in the mushroom growth process;
(2) According to the growth tide number and the growth days, adopting a peak signal to noise ratio (PSNR) to perform data cleaning on the shot color image and depth image of the mushroom growth state to obtain clear processed image data;
(3) Identifying and extracting mushroom fruiting body by adopting example segmentation algorithm
Taking out the previous n times of images at a certain time interval t from the color image obtained after cleaning to form a color image sequence P in a growth state nt c0 (ii) a Extracting mushroom fruiting body image from each image of the sequence image by using example segmentation algorithm, removing background image to form color image sequence P only containing mushroom nt c ；
(4) Taking several sum-colour picture image sequences P nt c The corresponding continuous depth maps at the same time form a depth image sequence P nt d A color image sequence P nt c A sequence of depth images P nt d Respectively inputting the color images into a ConvLSTM network for prediction, and predicting the color image P after the K time To+K c And depth map image P To+K d ；
(5) Color map and depth map matching extraction of predicted image mushroom features
Color image P to be predicted To+K c Identifying and extracting each mushroom through an example segmentation algorithm, carrying out ellipse fitting on the extracted mushrooms to obtain the outline and the central point coordinates of each mushroom, and then matching the outline and the central point coordinates with a depth mapImage P To+K d Carrying out mushroom height matching, calculating the size of the mushroom, obtaining the size and the central point position of the mushroom after the K moment, and further predicting the growth characteristics of the mushroom, wherein the growth characteristics comprise: the method comprises the steps of mushroom growth speed prediction, mushroom maturation time prediction, prediction of mature mushroom quantity in the 1 st and 2 nd days, mushroom distribution uniformity degree, mushroom aggregation condition, mushroom total number and mushroom growth density in L days.
Further, in the step (2) and the step (5), the example division algorithm is independently selected from any one of SOLO v2, mask RCNN, YOLACT, and the like.
Further, in step (3), the color image sequence P nt c Depth image sequence P nt d Respectively inputting the information into a ConvLSTM network, respectively extracting and coding the characteristics through a convolutional network, inputting the coded characteristic information into an LSTM neural network, extracting long-sequence hidden information and short-sequence hidden information through an update state gate i, a forgetting gate f and a hidden layer h, then sending the information into a decoder to decode and output a prediction result, wherein the structural formula of the ConvLSTM network is as follows,
Figure BDA0003821760040000021
Figure BDA0003821760040000022
Figure BDA0003821760040000023
Figure BDA0003821760040000024
Figure BDA0003821760040000025
in the formula (I), the compound is shown in the specification,
Figure BDA0003821760040000031
is the product of the matrix elements, x t Indicating input at time t, i t Representing output gate state retention probability, f t Indicating the probability of forgetting to leave the door state, C t Represents the cell state at time t, o t Representing output probability of output gate at time t, H t Represents the hidden layer output at time t, W xi 、W hi 、W ci 、b i Respectively representing the weight and threshold of the input gate, W xf 、W hf 、W cf 、b f Weight and threshold respectively representing a forgetting gate, W xc 、W hc 、b c Respectively representing the weight and threshold of the status gate, W xo 、W ho 、W co 、b o Representing the weight and threshold of the output gate, respectively.
Further, in the step (4), in the ConvLSTM network, an MS-SSIM loss function is adopted for training and tuning, and a calculation formula of the MS-SSIM loss function is as follows:
Figure BDA0003821760040000032
wherein M represents different scales; mu.s p ,μ g Respectively representing the average values of the predicted image and the actual image; sigma p ,σ g Respectively representing the standard deviation between the predicted image and the actual image; sigma pg Representing a covariance between the predicted value and the actual image; beta is a m And gamma m Respectively, the relative importance between the two terms, c 1 And c 2 Is a constant term added to prevent the divisor from being 0.
Further, in step (4), in the ConvLSTM network, the live function is trained by using an Adam optimizer and the ReLU.
Further, in the step (5), the method for predicting the growth rate of the mushrooms comprises the following steps:
color image P after future K time to be statistically predicted To+K c The size of each mushroom pileus in the mushroom box corresponds To the size of each mushroom pileus at the present time ToThe average difference is divided by K to obtain the predicted mushroom growth rate value.
Further, in the step (5), the mushroom maturation time prediction method comprises the following steps:
according To the predicted growth state color image and depth image after 1 hour, 2 hours and 3 hours of 823030that are predicted, each mature mushroom is identified and extracted, the maturation time of the mature mushroom on the predicted image after m hours is mature after m hours, then the maturation time m of each mushroom is comprehensively marked on the mushroom growth state image at the current time To for displaying, and reference basis is provided for the growers To adjust environmental parameters, pick and arrange bud thinning. The size of the mushroom calculated based on the color image and the depth image is a key parameter for determining the maturity of the mushroom, whether the mushroom is mature or not is judged according to the calculated size of the mushroom, and when the size of the mushroom is larger than a determined value, the mushroom is mature.
Further, in the step (5), the method for predicting the amount of mature mushrooms in the next L day comprises the following steps:
according to the predicted growth state color image and depth image after the L days 1 and 2 \8230, identifying and extracting the number, shape and position information of mature mushrooms on the predicted image on each day, generating a predicted mature mushroom distribution map every day, and providing reference for the picking time of planting personnel and picking related arrangement of picking personnel and the like; the predicted daily mature mushroom distribution map highlights only the day's mature mushrooms, which is generated by matting the L-1 day's mature mushrooms from the predicted L-day's predictive map, and then marking the L-day's mature mushrooms with highlighting; wherein: judging whether the mushroom is mature according to the calculated size of the mushroom.
Further, in the step (5), the method for judging the uniform distribution degree of the mushrooms comprises the following steps:
carrying out region division on the predicted growth state color image from four directions of vertical, horizontal, 45 degrees and 135 degrees, as well as the center and the periphery to obtain 10 regions, namely, upper, lower, left, right, upper left, lower right, upper right, lower left, center and periphery, and counting the number y of mushrooms in the 10 regions i And each zone is groundSum of mushroom number y o I.e. the total number of mushrooms, the variance MSE of the regional statistical distribution vector is then calculated as:
Figure BDA0003821760040000041
the degree of uniformity of mushroom distribution is divided into three grades according to the magnitude of the value of MSE: the first grade is that MSE is between 0 and 13, the second grade is that MSE is between 14 and 21, the third grade is that MSE is above 21, wherein the first grade shows that mushroom distribution is relatively even, the second grade shows that mushroom distribution is general, and the third grade shows that mushroom distribution is very uneven.
Further, in the step (5), the method for judging the mushroom gathering condition comprises the following steps:
and (3) performing clustering judgment on the predicted growth state image and the extracted mushroom size and the position of a central point by adopting a clustering method based on local density according to the sum of the radiuses of the mushrooms, marking the clustered mushrooms in a prediction graph, and counting the number and the size of the clusters, namely the number of the mushrooms in one cluster to realize the judgment and prediction of the mushroom clustering degree.
Further, in the step (5), the method for judging the mushroom growth density comprises the following steps:
and identifying and extracting mushrooms from the predicted color image, and calculating the number of the mushrooms in a unit area to realize prediction of the growth density of the mushrooms.
In the above, the present invention proposes to directly predict the future growth state image of the continuous time sequence of the image (including color image and depth image) of the fruit growth. The method extracts features from an image, then predicts according to a historical sequence of feature values, and directly predicts a growth state picture, more growth relation influence details are loaded in the image to participate in prediction when the image is predicted, and spatial information such as the positions of fruits, surrounding fruits and the fruits also participate in prediction, so that the fruit growth state prediction based on two space-time dimensions can achieve a prediction effect closer to an actual state. Compared with the prior art, the invention has the beneficial effects that:
the method can predict the growth vigor of the mushrooms such as growth speed, maturation period, mushroom gathering condition, mushroom distribution uniformity and the like, can replace manual inspection, provides information basis and judgment basis for more accurately and effectively carrying out reasonable arrangement of crop greenhouse environment factor control, intelligent picking and bud thinning of crops and the like, realizes more accurate control of greenhouse environment factors, cultivation management and intelligent degree of picking management, and improves yield and quality of the mushrooms. The method is not limited to mushroom, and can be used for predicting the growth states of other edible fungi and fruits and vegetables.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the embodiments or the drawings in the prior art, and it is obvious that the drawings described below are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive efforts.
FIG. 1 is a flow chart of a mushroom growth state prediction method.
FIG. 2 Mushroom growth status prediction System.
Fig. 3 a mushroom growth state image prediction network structure.
FIG. 4 shows a color image prediction result image of a growth state.
FIG. 5 is a growing state depth map prediction result image.
FIG. 6 is a graph of the segmentation effect of the example segmentation algorithm.
FIG. 7 predicted maturation period profiles of various mushrooms.
FIG. 8 is a distribution diagram of the number of mature mushrooms per day as predicted.
FIG. 9 shows a method for determining the uniformity of mushroom distribution.
FIG. 10 is a diagram for predicting the uniformity of mushroom distribution.
FIG. 11 is a diagram for predicting mushroom aggregation.
Detailed Description
The technical scheme of the invention is explained in detail in the following by combining the drawings and the embodiment.
The flow chart of the space-time prediction of the mushroom growth state is shown in figure 1.
In a preferred embodiment of the present invention, a mushroom growth state spatiotemporal prediction system of the present invention is shown in fig. 2, and includes a depth camera, an image transmission device, a central main control system, and a growth state image prediction system. The depth camera is installed above the mushrooms, mushroom growth state images are collected, the collected images are transmitted to the central main control system in a 5G transmission mode through the image transmission device, the central main control system stores the collected images and predicts a growth state image at a certain future time K through the growth state image prediction system, then growth state features in the predicted images are extracted, and growth conditions such as growth speed, growth density, maturity, future gathering condition, distribution uniformity and the like of the mushrooms are predicted.
In a preferred embodiment of the present invention, the growth state image prediction model is a spatial-temporal prediction model fused with SOLOv2-ConvLSTM, and the structure of the model is shown in fig. 3, on one hand, a plurality of growth state color image sequences with time interval of t hours are taken, and an example segmentation algorithm SOLOv2 is adopted to extract individual images of mushrooms from each original image (containing background information such as soil, hypha, etc.) of the sequence images to form a new color image sequence containing mushrooms only for prediction, so as to focus the prediction task on the mushroom ontology only to obtain more accurate prediction of the mushroom state; on the other hand, a plurality of depth map image sequences with the same interval in the same time period are taken. And then respectively carrying out feature extraction and coding on the color image sequence and the depth image sequence only containing the mushrooms through a convolutional neural network, further inputting the coded feature information into an LSTM, extracting long sequence hidden information and short sequence hidden information through a state updating gate, a forgetting gate and a hidden layer prediction network structure, then sending the extracted long sequence hidden information and short sequence hidden information into a decoder for decoding and outputting a prediction result, wherein a loss function of the prediction network adopts MS-SSIM, and finally obtaining a predicted image. The growth state image prediction model SOLOV2-ConvLSTM is obtained by setting a training parameter as a learning rate alpha =0.0001 under a deep learning framework of a CPU (central processing unit) of i9-10900k-3.7GHz, an RTX3090GPU and a Pythroch, using an Adam optimizer beta 1=0.5 and beta 2=0.2, adopting a ReLU as an activation function and training 5000 times.
In a preferred embodiment of the present invention, the growth state image predicting step is:
(1) Respectively collecting three color pictures and three depths of the growth state of the agaricus bisporus at intervals of 10 minutes by using a depth camera;
(2) Transmitting to a central main control system through 5G;
(3) And (4) comparing the acquired 3 images through a PSNR image quality function to select an image with the optimal image quality.
(4) 5 continuous color image images are collected and screened to form a historical color image sequence P nt c0 And extracting mushroom individual images from each image (containing information such as background soil) of the sequence images by adopting an example segmentation algorithm SOLov2 to form a growth state color map historical image sequence P only containing mushrooms nt c 。
(5) Taking 5 continuous depth images corresponding to the color image sequence images at the same time to form a depth image historical image sequence P nt d From P to P nt c And P nt d Inputting the image prediction model ConvLSTM for prediction to obtain a color image P1 hour later To+1 c And a depth map image P To+1 d As shown in fig. 4 and 5.
(6) Color image P to be predicted To+K c And identifying and extracting each mushroom through an example segmentation algorithm SOLov2, carrying out ellipse fitting on the extracted mushroom to obtain the outline and the center point coordinates of each mushroom, then carrying out mushroom height matching with the depth map image, and calculating the size of the mushroom, thereby obtaining the size and the position of the center point of the mushroom in the predicted image after 1 hour.
In one embodiment of the invention, the inventors adopt different recognition algorithms to perform segmentation recognition on the collected mushroom mining images, the effect of segmentation by adopting a Mask RCNN (Mask RCNN) is shown in FIG. 6 (c), and the success rate and the precision of recognition are obviously superior to those of a traditional visual method (a traditional recognition method based on edge gray gradient features, FIG. 6 (a)) or a target detection and positioning algorithm (a target detection and positioning algorithm based on YOLO, FIG. 6 (b)).
In one embodiment of the present invention, the inventors predicted the growth status of mushrooms based on the example segmentation and ConvLSTM fusion methods, SOLOV2-ConvLST algorithm and ConvLSTM algorithm, respectively, and the results showed that the predicted effect of SOLOV2-ConvLST algorithm was significantly better than that of ConvLSTM algorithm alone, as shown in Table 1:
TABLE 1
Figure BDA0003821760040000061
In a preferred embodiment of the invention, the specific prediction of the growth vigor of the mushrooms is realized by predicting the growth speed of the mushrooms, predicting the maturation time of the mushrooms, predicting the mature mushrooms 1 st and 2 nd (8230); the mature mushroom amount in L days, the uniform distribution degree of the mushrooms, the aggregation condition of the mushrooms, the total number of the mushrooms and the like through the predicted mushroom size and central point position information.
In a preferred embodiment of the present invention, the method for predicting the maturation time of mushrooms comprises: according To the SOLOv2-ConvLSTM image prediction method, growth state color image and depth image after 1 hour and 2 hours \8230arepredicted respectively, mature mushrooms are extracted by adopting SOLOv2 recognition, the mature time of the mature mushrooms on the predicted image after the m hour is m hours later, and then the mature time m of each mushroom is comprehensively marked on the mushroom growth state image at the current time (To) To be displayed. As shown in figure 7, the number marked on each mushroom is the time when the mushroom is about to mature, if the number marked on each mushroom is 5, the mushroom is mature after 5 hours, and reference basis is provided for the growers to adjust environmental parameters, arrange bud thinning and the like.
In a preferred embodiment of the present invention, the method for predicting the amount of mature mushrooms per day on the next L-th day is: according to the SOLOV2-ConvLSTM image prediction method, the growth state color image and the depth image after 1 and 2 days are predicted respectively, SOLOv2 is adopted to identify and extract the number, the shape and the central point position information of the mature mushrooms on the predicted image on each day, a new mature mushroom distribution map in each day is generated, and reference is provided for the planting personnel to carry out picking time and picking related arrangement of picking personnel and the like. As shown in FIG. 8, in order to predict the distribution of the daily mature mushrooms, the plot of the daily mature mushrooms is highlighted in yellow. The left panel is the predicted distribution of mushrooms maturing after 1 day, and the right panel is the predicted distribution of mushrooms maturing after 2 days, wherein the mushrooms maturing the previous day are not included because the mushrooms of the previous day have been picked.
In a preferred embodiment of the present invention, the discrimination method for predicting the uniformity of mushroom distribution is as follows: the image is divided into 10 regions from four directions of vertical, horizontal, 45 degrees and 135 degrees, as well as from the center and the periphery, according to the growth state color map image predicted by the above SOLOv2-ConvLSTM image prediction method, and the 10 regions are as shown in fig. 9, namely, upper, lower, left, right, upper left, lower right, upper right, lower left, center and periphery. Counting the number y of mushrooms in the 10 regions i And the sum of the mushroom numbers of the respective regions y o (i.e. total number of mushrooms) and then calculate the variance MSE of the region statistical distribution vector as follows:
Figure BDA0003821760040000071
the degree of uniformity of mushroom distribution is divided into three grades according to the magnitude of the value of MSE: the first grade is that the MSE is between 0 and 13, the second grade is that the MSE is between 14 and 21, the third grade is that the MSE is above 21, and the visual display shows the even degree of distribution. As shown in fig. 10, the distribution diagrams of three mushrooms with different distribution uniformity levels are respectively provided, the left upper corner is marked with the total number of mushrooms, the MSE value and the uniformity level, wherein the first level indicates that the mushrooms are distributed more uniformly, the second level indicates that the mushrooms are distributed uniformly, and the third level indicates that the mushrooms are distributed very non-uniformly.
In a preferred embodiment of the present invention, the method for predicting the aggregation of mushrooms comprises: and (3) predicting a growth state image according to the SOLOV2-ConvLSTM image prediction method, identifying and extracting mushroom shapes and center point position information by using SOLOV2, performing clustering judgment by using a local density-based clustering method according to the condition that the center distance between mushrooms is less than or equal to the sum of the radii of the mushrooms, and displaying the clustered mushrooms, the number of clusters and the cluster size by using a visualization method. As shown in FIG. 11, the predicted image has a-F6 clusters after clustering calculation, the size of each cluster, i.e. the number of mushrooms in the cluster, is marked with a number behind the letter of the cluster, for example, C4 means C that the cluster contains 4 mushrooms grouped together. The method can predict the aggregation degree of the mushrooms, and the more the number of clusters and the larger the cluster size, the more serious the dense clustering phenomenon of the mushrooms is, and bud thinning is needed.

Claims (10)

1. A space-time prediction method for mushroom growth state is characterized in that: the method comprises the following steps:
(1) Shooting a plurality of images every x minutes by adopting an RGBD depth camera for obtaining a color image and a depth image in the mushroom growth process;
(2) Carrying out data cleaning on the color image and the depth image of the growth state of the mushroom by adopting a peak signal-to-noise ratio (PSNR) to obtain processed clear image data;
(3) Identifying and extracting color pattern obtained after cleaning mushroom fruiting body by adopting example segmentation algorithm, taking out the previous n images at a certain time interval t, and forming a color pattern image sequence in growth state
Column P nt c0 (ii) a Extracting mushroom fruiting body image from each image of the sequence image by using example segmentation algorithm, removing background image to form color image sequence P only containing mushroom nt c ；
(4) Taking several sum-colour picture image sequences P nt c The continuous depth maps with the same interval in the same time period form a depth image sequence P nt d A color image sequence P nt c Depth image sequence P nt d Inputting the color images into ConvLSTM network for prediction, and predicting the color image P of the growth state after K time To+K c And depth map image P To+K d ；
(5) Color map and depth map matching extraction predicted image mushroom feature
Color image P to be predicted To+K c Identifying and extracting each mushroom through an example segmentation algorithm, carrying out ellipse fitting on the extracted mushrooms to obtain the outline and central point coordinates of each mushroom, and then carrying out ellipse fitting on the obtained outlines and central point coordinates and obtaining the coordinates and the depth map image P To+K d Carrying out mushroom height matching, calculating the size of the mushroom, obtaining the size and position characteristics of the mushroom after the K moment, and further predicting the growth characteristics of the mushroom, wherein the growth characteristics comprise: the method comprises the steps of mushroom growth speed prediction, mushroom maturation time prediction, mature mushroom quantity prediction in the future 1 st and 2 nd times of 82303080%, mushroom distribution uniformity degree, mushroom gathering condition, mushroom total number and mushroom growth density in L days.
2. The mushroom growth status prediction method according to claim 1, wherein in the step (2) and the step (5), the example segmentation algorithm is independently selected from any one of SOLO v2, mask RCNN or YOLACT.
3. The mushroom growth state prediction method according to claim 1, wherein, in the step (3), the color image sequence P nt c A sequence of depth images P nt d Respectively inputting the information into a ConvLSTM network, respectively extracting and coding the characteristics through a convolutional neural network, inputting the coded characteristic information into an LSTM neural network, extracting long-sequence hidden information and short-sequence hidden information through an update state gate i, a forgetting gate f and a hidden layer h, then sending the information into a decoder to decode and output a prediction result, wherein the structural formula of the ConvLSTM network is as follows,
Figure 525733DEST_PATH_IMAGE001
in the formula (I), the compound is shown in the specification,
Figure DEST_PATH_BDA0003821760040000031
is the product of the elements of the matrix,x t which indicates the input at the time of the t,i t the output gate state retention probability is represented,f t indicating the probability of forgetting to leave the door state,C t indicating the state of the cell at time t,o t the output probability of the output gate at time t is shown,H t indicating the hidden layer output at time t,W xi 、W hi 、W ci 、b i respectively representing the weight and threshold of the input gate,W xf 、W hf 、W cf 、b f respectively representing the weight and threshold of a forgotten gate,W xc 、W hc 、b c respectively representing the weight and threshold of the status gate,W xo 、W ho 、W co 、b o representing the weight and threshold of the output gate, respectively.
4. The mushroom growth state prediction method according to claim 1, wherein in the step (4), in the ConvLSTM network, an MS-SSIM loss function is adopted for training and tuning, and the calculation formula of the MS-SSIM loss function is as follows:
Figure 785868DEST_PATH_IMAGE003
wherein M represents different scales;
Figure 250348DEST_PATH_IMAGE004
respectively representing the mean values of the predicted image and the actual image;
Figure 325751DEST_PATH_IMAGE005
respectively representing the standard deviation between the predicted image and the actual image;
Figure 584694DEST_PATH_IMAGE006
representing a covariance between the predicted value and the actual image;
Figure 870182DEST_PATH_IMAGE007
and
Figure 748139DEST_PATH_IMAGE008
respectively, the relative importance between the two terms,
Figure 802683DEST_PATH_IMAGE009
and
Figure 498106DEST_PATH_IMAGE010
is a constant term added to prevent the divisor from being 0.
5. The mushroom growth status prediction method according to claim 1, wherein in the step (4), in the ConvLSTM network, adam optimizer training is used, and ReLU is used as the activation function.
6. The mushroom growth state prediction method according to claim 1, wherein in the step (5), the method of predicting mushroom growth rate is: statistically predicting the growth state color image P after the future K time To+K c Dividing the average value of the difference between the size of each mushroom pileus in the mushroom culture medium and the size of each mushroom pileus corresponding To the current time To by K To obtain a predicted mushroom growth speed value; the method for predicting the growth density of the mushrooms comprises the following steps: and identifying and extracting mushrooms from the predicted color image, and calculating the number of the mushrooms in a unit area to realize prediction of the growth density of the mushrooms.
7. The mushroom growth state prediction method according to claim 1, wherein the mushroom maturation time prediction method in the step (5) is:
identifying and extracting mature mushrooms according To predicted growth state color images and depth image images after 1 hour, 2 hours and 3 hours of 8230that are predicted, predicting the mature time of the mature mushrooms after m hours, then comprehensively marking the mature time m of each mushroom on the mushroom growth state image at the current time To for display, and providing reference basis for the environment parameter adjustment, picking and bud thinning arrangement of growers; wherein: judging whether the mushroom is mature according to the calculated size of the mushroom.
8. The mushroom growth state prediction method according to claim 1, wherein in step (5), the amount of mature mushrooms per day on the next L-th day is predicted by:
according to the predicted growth state color image and depth image after the L days 1 and 2 \8230, identifying and extracting the number, shape and position information of mature mushrooms on the predicted image on each day, generating a predicted mature mushroom distribution map every day, and providing reference for the picking time of growers and picking related arrangement of pickers and the like; the predicted daily mature mushroom distribution map highlights only the day's mature mushrooms, which is generated by matting the L-1 day's mature mushrooms from the predicted L-day's predictive map, and then marking the L-day's mature mushrooms with highlighting; wherein: judging whether the mushroom is mature according to the calculated size of the mushroom.
9. The mushroom growth state prediction method according to claim 1, wherein the discrimination method of the degree of uniformity of mushroom distribution in the step (5) is:
carrying out area division on the predicted growth state color image from four directions of vertical, horizontal, 45-degree and 135-degree and the center and the periphery to obtain 10 areas, namely upper, lower, left, right, upper left, lower right, upper right, lower left, center and periphery, and counting the mushroom quantity y in the 10 areas i And the sum of the mushroom numbers of the respective regions y o I.e. the total number of mushrooms, the variance MSE of the regional statistical distribution vector is then calculated as:
Figure 146257DEST_PATH_IMAGE011
the degree of uniformity of mushroom distribution was divided into three grades according to the magnitude of the value of MSE: the first grade is that MSE is between 0-13, the second grade is that MSE is between 14-21, the third grade is that MSE is above 21, wherein the first grade shows that mushroom distribution is relatively even, the second grade shows that mushroom distribution uniformity degree is general, the third grade shows that mushroom distribution is very uneven.
10. The method for predicting the growth state of mushrooms according to claim 1, wherein the determination method of the aggregation of mushrooms in step (5) is:
and performing clustering judgment on the predicted growth state color image and depth image and the extracted mushroom size and central point position by adopting a local density-based clustering method according to the condition that the center distance between the mushrooms is less than or equal to the sum of the radii of the mushrooms, marking the clustered mushrooms on the prediction image, counting the clustering number and the clustering size, namely the number of the mushrooms in one cluster, and realizing the judgment and prediction on the mushroom clustering degree.
CN202211044323.1A 2022-08-30 2022-08-30 Mushroom growth state space-time prediction method Pending CN115392576A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202211044323.1A CN115392576A (en) 2022-08-30 2022-08-30 Mushroom growth state space-time prediction method
PCT/CN2023/098790 WO2024045749A1 (en) 2022-08-30 2023-06-07 Method for spatio-temporal prediction of growth state of mushrooms

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211044323.1A CN115392576A (en) 2022-08-30 2022-08-30 Mushroom growth state space-time prediction method

Publications (1)

Publication Number Publication Date
CN115392576A true CN115392576A (en) 2022-11-25

Family

ID=84123098

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211044323.1A Pending CN115392576A (en) 2022-08-30 2022-08-30 Mushroom growth state space-time prediction method

Country Status (2)

Country Link
CN (1) CN115392576A (en)
WO (1) WO2024045749A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116129193A (en) * 2023-02-27 2023-05-16 重庆大学附属肿瘤医院 Method, system and equipment for predicting organoid growth
CN116720633A (en) * 2023-08-11 2023-09-08 安徽农业大学 Method for optimizing edible fungus breeding growth parameters
WO2024045749A1 (en) * 2022-08-30 2024-03-07 上海第二工业大学 Method for spatio-temporal prediction of growth state of mushrooms

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220007589A1 (en) * 2017-04-17 2022-01-13 Iron Ox, Inc. Method for monitoring growth of plants and generating a plant grow schedule
KR102095539B1 (en) * 2017-12-05 2020-03-31 농업회사법인 원스베리 주식회사 Method for measuring growth amount by image analyzing ginseng
CN109741393B (en) * 2018-12-04 2023-06-09 上海大学 Diameter measurement and center point positioning method for agaricus bisporus
KR102504318B1 (en) * 2020-07-17 2023-02-28 주식회사 쎄슬프라이머스 Growth analysis method based on smart edge device and apparatus and system therefor
CN115392576A (en) * 2022-08-30 2022-11-25 上海第二工业大学 Mushroom growth state space-time prediction method

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024045749A1 (en) * 2022-08-30 2024-03-07 上海第二工业大学 Method for spatio-temporal prediction of growth state of mushrooms
CN116129193A (en) * 2023-02-27 2023-05-16 重庆大学附属肿瘤医院 Method, system and equipment for predicting organoid growth
CN116129193B (en) * 2023-02-27 2024-06-11 重庆大学附属肿瘤医院 Method, system and equipment for predicting organoid growth
CN116720633A (en) * 2023-08-11 2023-09-08 安徽农业大学 Method for optimizing edible fungus breeding growth parameters
CN116720633B (en) * 2023-08-11 2023-10-27 安徽农业大学 Method for optimizing edible fungus breeding growth parameters

Also Published As

Publication number Publication date
WO2024045749A1 (en) 2024-03-07

Similar Documents

Publication Publication Date Title
CN115392576A (en) Mushroom growth state space-time prediction method
CN110455340B (en) Agricultural product planting environment detection system based on big data
CN111460990B (en) Big data-based alpine pastoral area grassland insect pest monitoring and early warning system and method
CN108710766B (en) Greenhouse plant liquid manure machine fertilizer regulation parameter calculation method based on growth model
CN102789579A (en) Identification method for stressed state of water fertilizer of greenhouse crop on basis of computer vision technology
CN109978234A (en) Soil moisture content prediction, irrigation decision method and apparatus
CN113011221A (en) Crop distribution information acquisition method and device and measurement system
CN117893914B (en) Plant growth monitoring method and system based on image recognition
CN114581816A (en) Real-time detection and counting method for solanaceous vegetables and fruits in plant factory
CN116578047B (en) Fine intelligent control method and system for chilli production
CN116227758A (en) Agricultural product maturity prediction method and system based on remote sensing technology and deep learning
CN114937030A (en) Phenotypic parameter calculation method for intelligent agricultural planting of lettuce
CN116703083A (en) Controllable agricultural cultivation management system and method based on artificial intelligence
CN118411607A (en) Flower yield estimation method, device and medium based on computer vision
CN118097422A (en) Agricultural environment monitoring and early warning method
CN117933558A (en) Campus intelligent agricultural planting management method
CN116616019A (en) Oil tea water and fertilizer integrated drip irrigation method and system based on Internet of things and artificial intelligence
Santhosh Kumar et al. Review on disease detection of plants using image processing and machine learning techniques
CN114612661A (en) Vegetable planter intelligent control method based on neural network
CN116757867B (en) Digital village construction method and system based on multi-source data fusion
CN114663752B (en) Intelligent estimation method and system for yield of edible beans based on machine vision
CN116310846B (en) Disease and pest identification method and system based on image identification and data mining
CN116994001B (en) Rice leaf age mode cultivation method, system and equipment based on data intelligence
US20230214997A1 (en) Training method, evaluation method, electronic device and storage medium
CN118097372B (en) Crop growth visual prediction method based on artificial intelligence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination