WO2024045749A1 - 一种蘑菇生长状态时空预测方法 - Google Patents

一种蘑菇生长状态时空预测方法 Download PDF

Info

Publication number
WO2024045749A1
WO2024045749A1 PCT/CN2023/098790 CN2023098790W WO2024045749A1 WO 2024045749 A1 WO2024045749 A1 WO 2024045749A1 CN 2023098790 W CN2023098790 W CN 2023098790W WO 2024045749 A1 WO2024045749 A1 WO 2024045749A1
Authority
WO
WIPO (PCT)
Prior art keywords
mushroom
mushrooms
image
growth
predicted
Prior art date
Application number
PCT/CN2023/098790
Other languages
English (en)
French (fr)
Inventor
杨淑珍
黄杰
俞涛
Original Assignee
上海第二工业大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海第二工业大学 filed Critical 上海第二工业大学
Publication of WO2024045749A1 publication Critical patent/WO2024045749A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G18/00—Cultivation of mushrooms
    • A01G18/60—Cultivation rooms; Equipment therefor
    • A01G18/69—Arrangements for managing the environment, e.g. sprinklers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/02—Agriculture; Fishing; Forestry; Mining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20088—Trinocular vision calculations; trifocal tensor

Definitions

  • the invention belongs to the technical field of smart agriculture and the technical field of edible fungi. Specifically, it relates to a method for spatiotemporal prediction of mushroom growth status.
  • the current mushroom greenhouse environmental monitoring system only collects greenhouse environmental parameters.
  • the environmental parameters are preset by the user.
  • the central control unit determines the production center by checking the data collected by the sensor and then making a simple comparison based on the user's preset parameters.
  • this method has significantly improved the real-time control of the edible fungi growth environment.
  • crops such as mushrooms. Their growth process and state are not as controllable as industrial products. It is difficult for them to grow according to the expected set state. Over time, the control effect is often not good.
  • mushroom culture bases usually still need to manually inspect the mushroom room at regular intervals to observe the growth status (growth) of the mushrooms, and then manually fine-tune the environmental parameters based on the growth.
  • This method of manual observation has a large workload and is The prediction of mushroom growth is not accurate enough, and the adaptive adjustment ability to actual changes in mushroom growth is poor.
  • mushrooms are fruits that tend to grow densely or in clusters. It is usually necessary to avoid the restriction of growth space and nutritional competition caused by overly dense fruits, which will affect the quality and yield of the fruits. Moreover, it is necessary to predict the subsequent picking time and quantity of fruits so that the picking tasks and bud thinning of fruits and vegetables can be reasonably arranged so that most fruits can be picked in a timely manner. Therefore, in order to achieve more accurate intelligent control of crop greenhouses, optimize the intelligent level of cultivation and picking management, and improve yield and quality, it is not enough to rely solely on manual or growth status monitoring. It is necessary to automatically and intelligently predict the growth status and carry out advance analysis. intervention.
  • Technology for predicting the future growth status of fruits currently generally uses growth model modeling. By studying the impact of fruit growth environmental factors on its growth, a growth model is established to predict fruit growth. However, due to the strong randomness of crop growth and the many factors that affect the growth process, including not only environmental factors, but also the composition and thickness of the culture medium, the accuracy of growth models is usually not high.
  • the purpose of the present invention is to provide a spatio-temporal prediction method of mushroom growth status, which can replace the manual monitoring of mushroom growth status, and can especially monitor the growth status of mushrooms.
  • the growth state of mushrooms is used for image prediction, which takes into account the spatiotemporal changes in the growth state of mushrooms. It integrates the convolutional long short-term memory neural network ConvLSTM with the instance segmentation network, and combines the SSIM loss function to improve the quality and prediction of the predicted image of the mushroom growth state. Accuracy.
  • the present invention is helpful to solve the difficulties of manual fine-tuning in existing mushroom greenhouse environmental regulation, such as the different growth status of each mushroom house, single regulation means and the inability to provide 24-hour on-duty regulation, etc. problem, achieve more precise control of greenhouse environmental factors, and improve mushroom yield and quality; in addition, it is also helpful to predict the maturity period and growth aggregation degree of mushrooms, and assist in cultivation and arrangement of mushroom picking time, picking personnel, and bud thinning plans.
  • the production management plan is arranged more scientifically and rationally.
  • a method for spatiotemporal prediction of mushroom growth status including the following steps:
  • the peak signal-to-noise ratio PSNR is used to clean the color images and depth images of the mushroom growth status captured to obtain clear image data after processing
  • the first n images are taken out at a certain time interval t to form a growth state color image sequence P nt c0 ; and then the instance segmentation algorithm is used to extract the mushroom fruiting body image from each image of the sequence. , remove the background image to form a color image sequence P nt c containing only mushrooms;
  • the growth characteristics include: mushroom growth speed prediction, mushroom maturity time prediction, future 1st, 2nd... L-day prediction of mature mushroom quantity, mushroom distribution uniformity, mushroom aggregation, total number of mushrooms and mushroom growth density.
  • the instance segmentation algorithm is independently selected from any one of SOLOv2, MaskRCNN, YOLACT, etc.
  • step (3) the color image sequence P nt c and the depth image sequence P nt d are respectively input into the ConvLSTM network, and feature extraction and encoding are performed through the convolutional network respectively.
  • the encoded feature information is input into the LSTM neural network and updated.
  • the state gate i, the forgetting gate f, and the hidden layer h extract the long sequence hidden information and short sequence hidden information, and then send them to the decoder to decode and output the prediction results.
  • the ConvLSTM network structure formula is as follows,
  • x t represents the input at time t
  • i t represents the output gate state retention probability
  • f t represents the forgetting gate state retention probability
  • C t represents the unit state at time t
  • o t represents the output gate output probability at time t
  • H t represents the hidden layer output at time t
  • W xi , W hi , W ci , and bi represent the weight and threshold of the input gate respectively
  • W xf , W hf , W cf , and b f represent the weight and threshold of the forgetting gate respectively
  • W xc , W hc , and b c respectively represent the weight and threshold of the state gate
  • W xo , Who , W co , and bo respectively represent the weight and threshold of the output gate.
  • step (4) in the ConvLSTM network, the MS-SSIM loss function is used for training and tuning.
  • the calculation formula of the MS-SSIM loss function is as follows:
  • M represents different scales; ⁇ p and ⁇ g respectively represent the mean values of the predicted image and the actual image; ⁇ p and ⁇ g respectively represent
  • ⁇ pg represents the covariance between the predicted value and the actual image
  • ⁇ m and ⁇ m represent the relative importance between the two items respectively
  • c 1 and c 2 are to prevent A constant term added when the divisor is 0.
  • step (4) in the ConvLSTM network, the Adam optimizer is used to train the active function using ReLU.
  • step (5) the method for predicting the mushroom growth rate is:
  • step (5) the mushroom maturity time prediction method is:
  • each mature mushroom is identified and extracted.
  • the maturity time of the mature mushrooms on the predicted image after the m hour is m hours. Mature, and then the maturity time m of each mushroom is comprehensively marked and displayed on the mushroom growth status image at the current time To, which provides a reference basis for the growers to adjust environmental parameters and arrange picking and bud thinning.
  • the size of the mushroom calculated based on the color image and the depth image is a key parameter that determines its maturity. Whether the mushroom is mature is judged based on the calculated mushroom size. When the size of the mushroom is greater than a certain value, it is a mature mushroom.
  • step (5) the method for predicting the amount of mature mushrooms every day in the next L days is:
  • the predicted daily mature mushroom distribution map Based on the predicted growth status color images and depth images after the 1st, 2nd...L days, identify and extract the number, shape and position information of mature mushrooms on the predicted images for each day, and generate the predicted daily mature mushroom distribution map , providing a reference for planting personnel to make picking time and picking personnel and other related arrangements for picking; the predicted daily mature mushroom distribution map only highlights the mature mushrooms on that day, which is calculated by dividing the predicted L-th day in the prediction map. Mature mushrooms on day -1 are cut out from the picture, and then the mushrooms that are mature on day L are marked with a bright color; where: whether the mushroom is mature is judged based on the calculated mushroom size.
  • step (5) the method for judging the uniformity of mushroom distribution is:
  • the predicted growth status color image is divided into four directions: vertical, horizontal, 45 degrees and 135 degrees, as well as the center and periphery, and 10 regions are obtained, namely upper, lower, left, right, upper left and right. Bottom, upper right, lower left, center and periphery, count the number of mushrooms yi in these 10 areas and the sum of the number of mushrooms in each area y o , that is, the total number of mushrooms, and then calculate the variance MSE of the regional statistical distribution vector according to the following formula:
  • the uniformity of mushroom distribution is divided into three levels: the first level is MSE between 0-13, the second level is MSE between 14-21, and the third level is MSE above 21. , where the first level indicates that the mushroom distribution is relatively even, the second level indicates that the mushroom distribution is average, and the third level indicates that the mushroom distribution is very uneven.
  • step (5) the method for identifying mushroom aggregation is:
  • a clustering method based on local density is used to make clustering judgments based on the distance between the centers of the mushrooms being less than or equal to the sum of the radii of the mushrooms, and then in the prediction map Mark the clustered mushrooms, count the number of clusters and cluster size, that is, the number of mushrooms in a cluster, and realize the discrimination and prediction of the degree of mushroom aggregation.
  • step (5) the method for determining mushroom growth density is:
  • the present invention proposes to directly predict future growth status images from a continuous time series of fruit growth images (including color images and depth images). It first extracts features from the image, and then predicts according to the historical sequence of feature values, directly predicting the growth status picture. When predicting the image, more details of the growth relationship are included in the picture to participate in the prediction, and the fruit and surrounding Spatial information such as the fruit and its location are also involved in the prediction. This prediction of fruit growth status based on the two dimensions of space and time can make the prediction effect closer to the actual status. Compared with the prior art, the beneficial effects of the present invention are:
  • the method of the present invention can predict the growth rate, maturity period, mushroom aggregation, mushroom distribution uniformity and other growth conditions of mushrooms. It can replace manual inspection and provide a more accurate and effective way to control crop greenhouse environmental factors and intelligent picking and harvesting of crops. Reasonable arrangements such as bud thinning provide information basis and judgment basis to achieve more precise control of greenhouse environmental factors and intelligent cultivation management and picking management, thereby improving mushroom yield and quality.
  • This method is not limited to mushrooms. This method can be used to predict the growth status of other edible fungi, fruits and vegetables.
  • Figure 1 Flowchart of mushroom growth state prediction method.
  • Figure 7 Predicted maturity distribution diagram of each mushroom.
  • Figure 8 predicts the distribution of the number of mature mushrooms per day.
  • FIG. 1 A flow chart of spatiotemporal prediction of mushroom growth status according to the present invention is shown in Figure 1.
  • a mushroom growth state spatio-temporal prediction system of the present invention is shown in Figure 2, including a depth camera, an image transmission device, a central main control system and a growth state image prediction system.
  • the depth camera is installed above the mushroom to collect images of the mushroom's growth status.
  • the collected images are transmitted to the central main control system through the image transmission device using 5G transmission.
  • the central main control system stores the collected images and predicts them through the growth status image prediction system. Predict the growth status image of K at a certain time in the future, and then extract the growth status features in the predicted image to predict the growth rate, growth density, maturity period, future aggregation, and distribution uniformity of the mushrooms.
  • the growth state image prediction model is a SOLOv2-ConvLSTM fusion spatio-temporal prediction model. Its structure is shown in Figure 3. On the one hand, several growth state color images with a time interval of t hours are taken. sequence, the instance segmentation algorithm SOLOv2 is used to extract individual mushroom images from each original image of the sequence image (including background information such as soil, hyphae, etc.) to form a new color image sequence containing only mushrooms for prediction, so as to complete the prediction task. Only focusing on the mushroom itself can obtain a more accurate prediction of the mushroom state; on the other hand, several depth map image sequences at the same intervals in the same time period are taken.
  • the color image sequence and the depth image sequence containing only mushrooms are extracted and encoded through the convolutional neural network respectively, and then the encoded feature information is input into the LSTM and extracted through the updated state gate, forgetting gate, and hidden layer prediction network structure.
  • the long sequence hidden information and short sequence hidden information are then sent to the decoder to decode and output the prediction result.
  • the loss function of the prediction network uses MS-SSIM to finally obtain the predicted image.
  • the growth state image prediction model SOLOv2-ConvLSTM uses i9-10900k-3.7GHz CPU, RTX3090GPU, and Pytorch deep learning framework.
  • the activation function uses ReLU, and the number of training times is 5000 times.
  • the growth state image prediction step is:
  • the inventor uses different recognition algorithms to segment and identify the collected mushroom pictures.
  • the effect of segmentation using the instance segmentation algorithm (MaskRCNN) is shown in Figure 6(c).
  • the success rate of recognition and accuracy are better than traditional vision methods (traditional recognition methods based on edge gray gradient features, Figure 6(a),) or target detection and positioning algorithms (target detection and positioning algorithms based on YOLO, Figure 6(b)).
  • the inventor predicts the growth status of mushrooms based on the SOLOv2-ConvLST algorithm and the ConvLSTM algorithm, which are typical methods of merging instance segmentation and ConvLSTM.
  • the results show that the prediction effect of the SOLOv2-ConvLST algorithm is significantly better than The prediction effect of using only the ConvLSTM algorithm is shown in the following table:
  • mushroom growth speed prediction, mushroom maturity time prediction, mature mushroom amount prediction on the 1st, 2nd...L days in the future, and mushroom distribution uniformity are performed based on the predicted mushroom size and center point position information. , mushroom aggregation, total number of mushrooms, etc. to achieve specific predictions of mushroom growth.
  • the mushroom maturity time prediction method is: according to the above-mentioned SOLOv2-ConvLSTM image prediction method, predict the growth status color image and depth image after 1 hour, 2 hours...48 hours, and use SOLOv2
  • SOLOv2-ConvLSTM image prediction method predict the growth status color image and depth image after 1 hour, 2 hours...48 hours, and use SOLOv2
  • Each mature mushroom is identified and extracted.
  • the maturity time of the mature mushroom on the image is predicted to be mature after the m hour.
  • the maturity time m of each mushroom is comprehensively marked and displayed on the mushroom growth status image at the current moment (To). come out.
  • the number marked on each mushroom indicates the time when the mushroom is about to mature. If marked 5, it means that the mushroom will mature in 5 hours, providing a reference for growers to adjust environmental parameters and arrange bud thinning.
  • the method for predicting the amount of mature mushrooms every day on the Lth day in the future is: predicting the growth status color image and depth map image after the 1st and 2nd day respectively according to the above-mentioned SOLOv2-ConvLSTM image prediction method, SOLOv2 recognition is used to extract the number of mature mushrooms, mushroom shape and mushroom center point position information on the predicted images for each day, and a new daily mature mushroom distribution map is generated to provide planting personnel with picking time and picking personnel and other related arrangements. refer to.
  • it is the predicted distribution map of mature mushrooms every day. The map of mature mushrooms on that day is highlighted in yellow. The left picture shows the predicted distribution of mature mushrooms in 1 day, and the right picture shows the predicted distribution of mature mushrooms in 2 days. Mushrooms that matured the previous day are not included because the mushrooms from the previous day have already been picked.
  • the judgment method for predicting the uniformity of mushroom distribution is as follows: the growth status color map image predicted according to the above-mentioned SOLOv2-ConvLSTM image prediction method is divided into four categories: vertical, horizontal, 45 degrees and 135 degrees. The image is divided into regions according to the direction, center and periphery, and 10 regions are obtained as shown in Figure 9, namely upper, lower, left, right, upper left, lower right, upper right, lower left, center and periphery. Count the number of mushrooms yi in these 10 areas and the sum of the number of mushrooms in each area y o (that is, the total number of mushrooms), and then calculate the variance MSE of the regional statistical distribution vector according to the following formula:
  • the uniformity of mushroom distribution is divided into three levels: the first level is MSE between 0-13, the second level is MSE between 14-21, and the third level is MSE above 21. , and visually displays the uniformity of its distribution.
  • the first level is MSE between 0-13
  • the second level is MSE between 14-21
  • the third level is MSE above 21.
  • FIG 10 there are three mushroom distribution diagrams with different uniform distribution levels. The total number of mushrooms, MSE value, and uniformity level are marked in the upper left corner. The first level indicates that the mushrooms are relatively evenly distributed, and the second level indicates that the mushrooms are evenly distributed. The level is average, and the third level indicates that the mushrooms are very unevenly distributed.
  • the judgment method for predicting mushroom aggregation is: according to the growth state image predicted by the above-mentioned SOLOv2-ConvLSTM image prediction method and the mushroom shape and center point position information extracted by SOLOv2 recognition, using the method based on
  • the local density clustering method makes clustering judgments based on the fact that the center distance between mushrooms is less than or equal to the sum of the radii of the mushrooms, and displays the clustered mushrooms, the number of clusters, and the size of the clusters visually. As shown in Figure 11, after the predicted image is calculated through clustering, there are a total of AF 6 clusters.
  • each cluster that is, the number of mushrooms contained in the cluster
  • Cluster C contains 4 mushrooms clustered together. This method can predict the degree of mushroom aggregation. The greater the number of clusters and the larger the cluster size, the more serious the phenomenon of dense mushroom clusters and the need for thinning.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Business, Economics & Management (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Quality & Reliability (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Primary Health Care (AREA)
  • Animal Husbandry (AREA)
  • Operations Research (AREA)
  • Marine Sciences & Fisheries (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Development Economics (AREA)
  • Mining & Mineral Resources (AREA)
  • Biophysics (AREA)
  • Agronomy & Crop Science (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)

Abstract

本发明公开了一种蘑菇生长状态时空预测方法；该方法采集蘑菇生长过程中的彩图和深度图历史序列图像，通过改进的卷积长短时记忆神经网络(ConvLSTM)预测未来某一时刻的蘑菇生长状态图片，该改进模型将ConvLSTM与实例分割网络融合并结合SSIM损失函数来提高预测图像的质量与预测精度。然后，将预测出的彩图通过实例分割算法进行分割，提取出蘑菇子实体，并与深度图匹配计算出预测图像中各蘑菇的尺寸大小；最后，通过预测的蘑菇大小及位置特征对蘑菇长势进行预测；本发明方法能替代人工巡检，为更加精准有效的开展农作物温室环境因子控制及农作物的智能采摘及疏蕾等科学合理安排提供信息基础和判断依据，提高蘑菇产量和质量。

Description

一种蘑菇生长状态时空预测方法 技术领域
本发明属于智慧农业技术领域及食用菌技术领域，具体的说，涉及一种蘑菇生长状态时空预测方法。
背景技术
目前的蘑菇温室环境监控系统，只对温室环境参数进行采集，由用户预先设定好环境参数，中控单元通过对传感器采集的数据进行检查再根据用户的预设参数进行简单对比来确定生产中间的设备状态，该方式相对于最开始的无环境监测的开环控制系统，对于食用菌生长环境控制的实时性有了明显改善。但蘑菇等农作物的生长影响因素较多，其生长过程和状态不像工业产品那么可控，其很难按预计设定状态进行生长，时间长了，往往调控效果不好。所以，目前蘑菇培养基地通常还是需要人工隔一段时间去巡视菇房，观察菇菌的生长状态(长势)，然后根据长势对环境参数进行人工微调干预，这种人工观察的方式工作量大，而且对菌菇长势的预测不够准确，对菇菌长势实际变化的自适应调整能力较差。
此外，蘑菇是易密集或成簇生长的果实，通常还需要避免果实过密导致的生长空间限制以及营养竞争而对果实的质量和产量造成影响。并且，需要对果实的后续采摘时间和采摘量进行预测，便于较优地对采摘任务和疏蕾蔬果进行合理安排，使得大部分果实能适时被采摘。所以，要达到更精准的农作物温室智能控制、优化栽培和采摘管理智能化水平，提高产量和质量，仅靠人工或生长状态监测是不够的，需要对其生长状态进行自动、智能预测，提前进行干预。
对果实未来生长状态预测的技术目前一般采用生长模型建模方式，通过研究果实生长环境因子对其生长的影响，建立生长模型实现对果实生长的预测。但是由于农作物生长的随机性强、生长过程受影响因素非常多，不仅仅受环境因子的影响，还会受到培养基成分、厚度等影响，生长模型通常准确度不高。
发明内容
针对现有技术中对蘑菇生长状态监测和预测中存在的上述缺陷，本发明的目的是提供一种蘑菇生长状态时空预测方法，该方法能替代人工对蘑菇的生长状态进行监测，特别地能对蘑菇的生长状态进行图像预测，其考虑了蘑菇生长状态的时空变化，将卷积长短时记忆神经网络ConvLSTM与实例分割网络融合，并结合SSIM损失函数来提高蘑菇生长状态的预测图像的质量与预测精度。本发明通过对蘑菇生长状态的精准预测，有利于解决现有蘑菇温室环境调控中存在的人工精细化调控难、各菇房生长状态各有不同、调控手段单一且不能24小时全天值守调控等问题，实现温室环境因子的更加精准控制，提高蘑菇产量和质量；此外，还有利于对蘑菇的成熟期及生长聚集程度进行预测，辅助进行蘑菇采摘时间、采摘人员、疏蕾计划安排等栽培和生产管理计划进行更加科学合理的安排。
为实现本发明的目的，本发明采用以下技术方案：
一种蘑菇生长状态时空预测方法，包括以下步骤：
(1)采用RGBD深度相机每隔x分钟拍摄多张图像用于获取蘑菇生长过程中的彩图与深度图；
(2)按照生长潮数、生长天数，采用峰值信噪比PSNR对拍摄到的蘑菇生长状态的彩图与深度图进行数据清洗，得到处理后清晰的图片数据；
(3)采用实例分割算法识别提取蘑菇子实体
对清洗后得到的彩图，按一定时间间隔t取出前n次图像，构成生长状态彩图图像序列Pnt c0；再利用实例分割算法从该序列图像的每张图像中提取出蘑菇子实体图像，去除背景图像形成仅含蘑菇的彩图图像序列Pnt c；
(4)取若干张和彩图图像序列Pnt c同时刻对应的连续的深度图组成深度图像序列Pnt d，将彩图图像序列Pnt c、深度图像序列Pnt d分别输入ConvLSTM网络中进行预测，预测出K时刻后的彩图图像PTo+K c和深度图图像PTo+K d；
(5)彩图与深度图匹配提取预测图像蘑菇特征
将预测的彩图图像PTo+K c通过实例分割算法识别提取出各蘑菇并对提取的蘑菇进行椭圆拟合获得各蘑菇的轮廓和中心点坐标，然后与深度图图像PTo+K d进行蘑菇高度匹配，计算出蘑菇的尺寸大小，获得K时刻后的蘑菇大小及中心点位置，进而预测蘑菇的长势特征，长势特征包括：蘑菇生长速度预测、蘑菇成熟时间预测、未来第1、2…L天的成熟蘑菇量预测、蘑菇分布均匀程度、蘑菇聚集情况、蘑菇总数和蘑菇生长密度。
进一步的，步骤(2)和步骤(5)中，实例分割算法独立的选自SOLOv2、MaskRCNN、YOLACT等中任一种。
进一步的，步骤(3)中，彩图图像序列Pnt c、深度图像序列Pnt d分别输入ConvLSTM网络中，分别通过卷积网络进行特征提取和编码，编码后特征信息输入LSTM神经网络经更新状态门i、遗忘门f、隐藏层h提取出长序列隐藏信息与短序列隐藏信息，然后送入解码器解码输出预测结果，ConvLSTM网络结构公式如下，




式中，为矩阵元素的乘积，xt表示t时刻输入，it表示输出门状态保留概率，ft表示遗忘门状态保留概率，Ct表示t时刻单元状态，ot表示t时刻输出门输出概率，Ht表示t时刻隐含层输出，Wxi、Whi、Wci、bi分别表示输入门的权重和阈值，Wxf、Whf、Wcf、bf分别表示遗忘门的权重和阈值，Wxc、Whc、bc分别表示状态门的权重和阈值，Wxo、Who、Wco、bo分别表示输出门的权重和阈值。
进一步的，步骤(4)中，ConvLSTM网络中，采用MS-SSIM损失函数进行训练调优，MS-SSIM损失函数计算公式如下：
其中，M表示不同的尺度；μp,μg分别表示预测图像和实际图像的均值；σp,σg分别表
示预测图像和实际图像之间的标准差；σpg表示预测值和实际图像之间的协方差；βm和γm分别表示两项之间的相对重要性，c1和c2是为了防止除数为0而添加的常数项。
进一步的，步骤(4)中，ConvLSTM网络中，使用Adam优化器训练活函数使用ReLU。
进一步的，步骤(5)中，预测蘑菇生长速度的方法为：
将统计预测的未来K时刻后的彩图图像PTo+K c中的各蘑菇菌盖大小与当前时刻To时刻对应各蘑菇菌盖大小之差的平均值除以K，得到预测蘑菇生长速度值。
进一步的，步骤(5)中，蘑菇成熟时间预测方法为：
根据预测出的1小时、2小时、3小时…m小时后的生长状态彩图和深度图像，识别和提取出各成熟蘑菇，第m小时后预测图像上的成熟蘑菇的成熟时间为m小时后成熟，然后将每个蘑菇的成熟时间m综合标注在当前时刻To的蘑菇生长状态图像上显示，为种植人员进行环境参数调节和采摘、疏蕾安排提供参考依据。其中，基于彩图和深度图像配合计算的蘑菇大小是决定其成熟度的关键参数，蘑菇是否成熟根据计算出的蘑菇大小进行判断，当蘑菇的大小大于一个确定值时，即为成熟蘑菇。
进一步的，步骤(5)中，未来第L天中每天的成熟蘑菇量预测方法为：
根据预测出的第1、2…L天后的生长状态彩图和深度图像，识别和提取出各天预测图像上的成熟蘑菇个数、形状和位置信息，并生成预测出的每天成熟蘑菇分布图，为种植人员进行采摘时间和采摘人员等采摘相关安排提供参考；预测出的每天成熟蘑菇分布图仅高亮显示当天成熟的蘑菇，其通过将预测出的第L天的预测图中的第L-1天的成熟蘑菇从图片中抠除，然后用亮显色标记出第L天成熟的蘑菇生成；其中：蘑菇是否成熟根据计算出的蘑菇大小进行判断。
进一步的，步骤(5)中，蘑菇分布均匀程度的判别方法为：
对预测出来的生长状态彩图图像从竖直、水平、45度和135度四个方向以及中心和外围对图像进行区域划分，得到10个区域，即上、下、左、右、左上、右下、右上、左下、中心和外围，统计这10个区域内的蘑菇数量yi以及各区域磨菇数之和yo，即蘑菇总数，然后按如下公式计算区域统计分布向量的方差MSE：
根据MSE的值的大小将蘑菇分布的均匀程度划分为三个等级：第一等级为MSE在0-13之间、第二等级为MSE在14-21之间、第三等级为MSE在21以上，其中第一等级表示蘑菇分布比较均匀，第二等级表示蘑菇分布均匀程度为一般，第三等级表示蘑菇分布很不均匀。
进一步的，步骤(5)中，蘑菇聚集情况的判别方法为：
对预测出来的生长状态图像及提取出来的蘑菇大小及中心点位置，采用基于局部密度的聚类方法，按照蘑菇间中心距小于等于蘑菇的半径之和来进行聚类判断，然后在预测图中标示出聚类蘑菇，统计聚类个数和聚类大小，即一个聚类中的蘑菇个数，实现对蘑菇聚集程度的判别预测。
进一步的，步骤(5)中，蘑菇生长密度的判别方法为：
对预测出来的彩图图像识别并提取出蘑菇，计算单位面积的蘑菇个数，实现对蘑菇的生长密度进行预测。
以上，本发明提出对果实生长的图像(包括彩图和深度图)的连续时间序列直接进行未来生长状态图像进行预测。其从图像中先提取特征，然后按照特征值的历史序列进行预测，直接对生长状态图片进行预测，图像预测的时候更多的生长关系影响细节均载在图中一起参与预测，而且果实与周边果实及果实所处位置等空间信息也一起参与预测，这种基于时空两个维度对果实生长状态预测，其预测效果能更加接近实际状态。和现有技术相比，本发明的有益效果在于：
本发明方法可对蘑菇进行生长速度、成熟期、蘑菇聚集情况、蘑菇分布均匀程度等长势进行预测，其可替代人工巡检，为更加精准有效的开展农作物温室环境因子控制以及农作物的智能采摘及疏蕾等合理安排提供信息基础和判断依据，实现温室环境因子的更加精准控制和栽培管理以及采摘管理的智能化程度，提高蘑菇产量和质量。本方法不仅限于蘑菇，其它食用菌类、果蔬的生长状态预测均可采用此方法。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术的附图做简单地介绍，很显然下面描述的附图仅仅是对本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前题下，还可以根据这些附图获得其他附图。
图1蘑菇生长状态预测方法流程图。
图2蘑菇生长状态预测系统。
图3蘑菇生长状态图像预测网络结构。
图4生长状态彩图预测结果图像。
图5生长状态深度图预测结果图像。
图6实例分割算法分割效果图。
图7预测的各蘑菇成熟期分布图。
图8预测的每天成熟蘑菇的个数即分布图。
图9蘑菇分布均匀程度的判别方法。
图10蘑菇分布均匀程度预测图。
图11蘑菇聚集情况预测图。
具体实施方式
下面结合附图和实施例对本发明的技术方案进行详细阐述。
本发明的一种蘑菇生长状态时空预测流程图如图1所示。
在本发明的一个优选实施方案中，本发明的一种蘑菇生长状态时空预测系统如图2所示，包括深度相机、图像传输装置、中央主控系统和生长状态图像预测系统。其中深度相机安装在蘑菇上方，采集蘑菇生长状态图像，采集到的图像通过图像传输装置，采用5G传输方式传输至中央主控系统，中央主控系统存储采集到的图像并通过生长状态图像预测系统预测未来某一时刻K的生长状态图像，然后，提取预测图像中的生长状态特征，预测蘑菇的生长速度、生长密度、成熟期、未来聚集情况、分布均匀程度等长势。
在本发明的一个优选实施方案中，生长状态图像预测模型为SOLOv2-ConvLSTM融合的时空预测模型，其结构如图3所示，一方面，取若干张时间间隔为t小时的生长状态彩图图像序列，采用实例分割算法SOLOv2从该序列图像的每张原始图像中(含土壤、菌丝等背景信息)提取出蘑菇个体图像形成新的仅含蘑菇的彩图图像序列进行预测，以将预测任务只关注到蘑菇本体上而得到对蘑菇状态的更精准预测；另一方面，取同样时间段内若干张相同间隔的深度图图像序列。然后分别将仅含蘑菇的彩图图像序列和深度图像序列分别通过卷积神经网络进行特征提取和编码，进而将编码所得特征信息输入LSTM经更新状态门、遗忘门、隐藏层预测网络结构提取出长序列隐藏信息与短序列隐藏信息，然后送入解码器解码输出预测结果，其中预测网络的损失函数采用MS-SSIM，最终得到预测图像。该生长状态图像预测模型SOLOv2-ConvLSTM通过i9-10900k-3.7GHz的CPU、RTX3090GPU、Pytorch深度学习框架下，训练参数设置为学习率α＝0.0001，使用基于Adam优化器β1＝0.5、β2＝0.2，激活函数采用ReLU，训练次数为5000次训练而得。
在本发明的一个优选实施方案中，生长状态图像预测步骤为：
(1)采用深度相机分别对双孢蘑菇的生长状态每间隔10分钟采集三张彩图和三张深度；
(2)通过5G传输到中央主控系统；
(3)对采集的3张图像通过PSNR图像质量函数对比挑选出图像质量最优的图像。
(4)将采集并筛选5张连续彩图图像组成历史彩图图像序列Pnt c0，并采用实例分割算法SOLOv2从该序列图像的每张图像中(含背景土壤等信息)提取出蘑菇个体图像形成仅含蘑菇的生长状态彩图历史图像序列Pnt c。
(5)取以上彩图序列图像同时刻对应的5张连续深度图像组成深度图像历史图像序列Pnt d，将Pnt c和Pnt d输入图像预测模型ConvLSTM中进行预测，得到1小时后的彩图图像PTo+1 c和深度图图像PTo+1 d，如图4、5所示。
(6)将预测的彩图图像PTo+K c通过实例分割算法SOLOv2识别提取出各蘑菇并对提取的蘑菇进行椭圆拟合获得各蘑菇的轮廓和中心点坐标，然后与深度图图像进行蘑菇高度匹配，计算出蘑菇的尺寸大小，从而获得1小时后的预测图像中蘑菇大小及中心点位置。
在本发明的一个实施方案中，，发明人采用不同识别算法对采集的蘑菇采图进行分割识别，其采用实例分割算法(MaskRCNN)分割的效果如图6(c)所示，识别的成功率和精度均优于传统视觉方法(基于边缘灰度梯度特征的传统识别方法，图6(a),)或目标检测定位算法(基于YOLO的目标检测定位算法，图6(b))。
在本发明的一个实施方案中，发明人分别基于实例分割与ConvLSTM融合的典型方法SOLOv2-ConvLST算法、和ConvLSTM算法对蘑菇的生长状态进行预测，结果显示，SOLOv2-ConvLST算法的预测效果显著优于仅用ConvLSTM算法的预测效果，如下表所示：
在本发明的一个优选实施方案中，通过预测得到的蘑菇大小及中心点位置信息进行蘑菇生长速度预测、蘑菇成熟时间预测、未来第1、2…L天的成熟蘑菇量预测、蘑菇分布均匀程度、蘑菇聚集情况、蘑菇总数等实现对蘑菇长势的具体预测。
在本发明的一个优选实施方案中，蘑菇成熟时间预测方法为：按照上述SOLOv2-ConvLSTM图像预测方法分别预测出1小时、2小时…48小时后的生长状态彩图和深度图图像，并采用SOLOv2识别提取出各成熟蘑菇，第m小时后预测图像上的成熟蘑菇的成熟时间为m小时后成熟，然后将每个蘑菇的成熟时间m综合标注在当前时刻(To)的蘑菇生长状态图像上显示出来。如图7所示，每个蘑菇上标的数字为该蘑菇即将成熟的时间，若标5，则表示该蘑菇5小时后成熟，为种植人员进行环境参数调节和疏蕾安排等提供参考依据。
在本发明的一个优选实施方案中，未来第L天中每天的成熟蘑菇量预测方法为：按照上述SOLOv2-ConvLSTM图像预测方法分别预测出第1、2天后的生长状态彩图和深度图图像，并采用SOLOv2识别提取出各天预测图像上的成熟蘑菇个数、蘑菇形状和蘑菇中心点位置信息，并生成新的每天成熟蘑菇分布图，为种植人员进行采摘时间和采摘人员等采摘相关安排提供参考。如图8所示，为预测出的每天成熟蘑菇分布图，图中用黄色亮显示当天成熟的蘑菇图。左图为预测的1天后成熟的蘑菇分布，右图为预测的2天后成熟的蘑菇分布图，其中，不包含前一天成熟的蘑菇，因为前一天的蘑菇已经被采摘了。
在本发明的一个优选实施方案中，蘑菇分布均匀程度预测的判别方法为：对按照上述SOLOv2-ConvLSTM图像预测方法预测出来的生长状态彩图图像从竖直、水平、45度和135度四个方向以及中心和外围对图像进行区域划分，得到10个区域如图9所示，即上、下、左、右、左上、右下、右上、左下、中心和外围。统计这10个区域内的蘑菇数量yi以及各区域磨菇数之和yo(即蘑菇总数)，然后按如下公式计算区域统计分布向量的方差MSE：
根据MSE的值的大小将蘑菇分布的均匀程度划分为三个等级：第一等级为MSE在0-13之间、第二等级为MSE在14-21之间、第三等级为MSE在21以上，并可视化显示出其分布均匀程度。如图10所示，分别为三个不同分布均匀等级的蘑菇分布图，左上角标注了蘑菇总数、MSE值、均匀程度等级，其中第一等级表示蘑菇分布比较均匀，第二等级表示蘑菇分布均匀程度为一般，第三等级表示蘑菇分布很不均匀。
在本发明的一个优选实施方案中，蘑菇聚集情况预测的判别方法为：按照上述SOLOv2-ConvLSTM图像预测方法预测出来的生长状态图像及采用SOLOv2识别提取出的蘑菇形状和中心点位置信息，采用基于局部密度的聚类方法，按照蘑菇间中心距小于等于蘑菇的半径之和来进行聚类判断，并将聚类蘑菇、聚类个数及聚类大小用可视化方法显示。如图11所示，预测出来的图像通过聚类计算后，共有A-F 6个聚类，每个聚类的大小即聚类中所含蘑菇个数用数字标注在聚类字母后面，例如C4表示C这个聚类含有4个蘑菇聚集在一起。通过此方法可预测蘑菇聚集程度，聚类数量越多、聚类大小越大表示蘑菇密集丛生现象越严重，需要进行疏蕾。

Claims (10)

  1. 一种蘑菇生长状态时空预测方法，其特征在于：包括以下步骤：
    (1)采用RGBD深度相机每隔x分钟拍摄多张图像用于获取蘑菇生长过程中的彩图与深度图；
    (2)采用峰值信噪比PSNR对拍摄到的蘑菇生长状态的彩图与深度图进行数据清洗，得到处理后清晰的图片数据；
    (3)采用实例分割算法识别提取蘑菇子实体
    对清洗后得到的彩图，按一定时间间隔t取出前n次图像，构成生长状态彩图图像序列Pnt c0；再利用实例分割算法从该序列图像的每张图像中提取出蘑菇子实体图像，去除背景图像形成仅含蘑菇的彩图图像序列Pnt c；
    (4)取若干张和彩图图像序列Pnt c在同样时间段内、具有相同间隔的连续的深度图组成深度图像序列Pnt d，将彩图图像序列Pnt c、深度图像序列Pnt d分别输入ConvLSTM网络中进行预测，预测出K时刻后的生长状态的彩图图像PTo+K c和深度图图像PTo+K d；
    (5)彩图与深度图匹配提取预测图像蘑菇特征
    将预测的彩图图像PTo+K c通过实例分割算法识别提取出各蘑菇并对提取的蘑菇进行椭圆拟合获得各蘑菇的轮廓和中心点坐标，然后与深度图图像PTo+K d进行蘑菇高度匹配，计算出蘑菇的尺寸大小，获得K时刻后的蘑菇大小及位置特征，进而预测蘑菇的长势特征，长势特征包括：蘑菇生长速度预测、蘑菇成熟时间预测、未来第1、2…L天的成熟蘑菇量预测、蘑菇分布均匀程度、蘑菇聚集情况、蘑菇总数和蘑菇生长密度。
  2. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(2)和步骤(5)中，实例分割算法独立的选自SOLOv2、MaskRCNN或YOLACT中任一种。
  3. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(3)中，彩图图像序列Pnt c、深度图像序列Pnt d分别输入ConvLSTM网络中，分别通过卷积神经网络进行特征提取和编码，编码后特征信息输入LSTM神经网络经更新状态门i、遗忘门f、隐藏层h提取出长序列隐藏信息与短序列隐藏信息，然后送入解码器解码输出预测结果，ConvLSTM网络结构公式如下，




    式中，为矩阵元素的乘积，xt表示t时刻输入，it表示输出门状态保留概率，ft表示遗忘门状态保留概率，Ct表示t时刻单元状态，ot表示t时刻输出门输出概率，Ht表示t时刻隐含层输出，Wxi、Whi、Wci、bi分别表示输入门的权重和阈值，Wxf、Whf、Wcf、bf分别表示遗忘门的权重和阈值，Wxc、Whc、bc分别表示状态门的权重和阈值，Wxo、Who、Wco、bo分别表示输出门的权重和阈值。
  4. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(4)中，ConvLSTM网络中采用MS-SSIM损失函数进行训练调优，MS-SSIM损失函数计算公式如下：
    其中，M表示不同的尺度；μp,μg分别表示预测图像和实际图像的均值；σp,σg分别表示预测图像和实际图像之间的标准差；σpg表示预测值和实际图像之间的协方差；βm和γm分别表示两项之间的相对重要性，c1和c2是为了防止除数为0而添加的常数项。
  5. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(4)中，使用Adam优化器训练，激活函数使用ReLU。
  6. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(5)中，预测蘑菇生长速度的方法为：将统计预测的未来K时刻后的生长状态彩图图像PTo+K c中的各蘑菇菌盖大小与当前时刻To对应各蘑菇菌盖大小之差的平均值除以K，得到预测蘑菇生长速度值；预测蘑菇生长密度的方法为：对预测出来的彩图图像识别并提取出蘑菇，计算单位面积的蘑菇个数，实现对蘑菇的生长密度进行预测。
  7. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(5)中，蘑菇成熟时间预测方法为：
    根据预测出的1小时、2小时、3小时…m小时后的生长状态彩图和深度图图像，识别和提取出各成熟蘑菇，第m小时后预测图像上的成熟蘑菇的成熟时间为m小时后成熟，然后将每个蘑菇的成熟时间m综合标注在当前时刻To的蘑菇生长状态图像上显示，为种植人员进行环境参数调节和采摘、疏蕾安排提供参考依据；其中：蘑菇是否成熟根据计算出的蘑菇大小进行判断。
  8. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(5)中，未来第L天中每天的成熟蘑菇量预测方法为：
    根据预测出的第1、2…L天后的生长状态彩图和深度图图像，识别和提取出各天预测图像上的成熟蘑菇个数、形状和位置信息，并生成预测出的每天成熟蘑菇分布图，为种植人员进行采摘时间和采摘人员等采摘相关安排提供参考；预测出的每天成熟蘑菇分布图仅高亮显示当天成熟的蘑菇，其通过将预测出的第L天的预测图中的第L-1天的成熟蘑菇从图片中抠除，然后用亮显色标记出第L天成熟的蘑菇生成；其中：蘑菇是否成熟根据计算出的蘑菇大小进行判断。
  9. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(5)中，蘑菇分布均匀程度的判别方法为：
    对预测出来的生长状态彩图图像从竖直、水平、45度和135度四个方向以及中心和外围对图像进行区域划分，得到10个区域，即上、下、左、右、左上、右下、右上、左下、中心和外围，统计这10个区域内的蘑菇数量yi以及各区域磨菇数之和yo，即蘑菇总数，然后按如下公式计算区域统计分布向量的方差MSE：
    根据MSE的值的大小将蘑菇分布的均匀程度划分为三个等级：第一等级为MSE在0-13之间、第二等级为MSE在14-21之间、第三等级为MSE在21以上，其中第一等级表示蘑菇分布比较均匀，第二等级表示蘑菇分布均匀程度为一般，第三等级表示蘑菇分布很不均匀。
  10. 根据权利要求1所述的蘑菇生长状态预测方法，其特征在于，步骤(5)中，蘑菇聚集情况的判别方法为：
    对预测出来的生长状态彩图和深度图图像及提取出来的蘑菇大小及中心点位置，采用基于局部密度的聚类方法，按照蘑菇间中心距小于等于蘑菇的半径之和来进行聚类判断，然后在预测图中标示出聚类蘑菇，统计聚类个数和聚类大小，即一个聚类中的蘑菇个数，实现对蘑菇聚集程度的判别预测。
PCT/CN2023/098790 2022-08-30 2023-06-07 一种蘑菇生长状态时空预测方法 WO2024045749A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211044323.1 2022-08-30
CN202211044323.1A CN115392576A (zh) 2022-08-30 2022-08-30 一种蘑菇生长状态时空预测方法

Publications (1)

Publication Number Publication Date
WO2024045749A1 true WO2024045749A1 (zh) 2024-03-07

Family

ID=84123098

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/098790 WO2024045749A1 (zh) 2022-08-30 2023-06-07 一种蘑菇生长状态时空预测方法

Country Status (2)

Country Link
CN (1) CN115392576A (zh)
WO (1) WO2024045749A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115392576A (zh) * 2022-08-30 2022-11-25 上海第二工业大学 一种蘑菇生长状态时空预测方法
CN116129193B (zh) * 2023-02-27 2024-06-11 重庆大学附属肿瘤医院 一种类器官生长预测方法、系统及设备
CN116720633B (zh) * 2023-08-11 2023-10-27 安徽农业大学 一种食用菌育种生长参数寻优的方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109741393A (zh) * 2018-12-04 2019-05-10 上海大学 双孢菇直径测量和中心点定位方法
KR20190069648A (ko) * 2017-12-05 2019-06-20 농업회사법인 원스베리 주식회사 인삼의 영상 이미지 분석을 통한 생육량 측정 방법
US20220007589A1 (en) * 2017-04-17 2022-01-13 Iron Ox, Inc. Method for monitoring growth of plants and generating a plant grow schedule
KR20220010374A (ko) * 2020-07-17 2022-01-25 주식회사 쎄슬프라이머스 스마트 엣지 디바이스 기반의 생육 분석 방법 및 그를 위한 장치 및 시스템
CN115392576A (zh) * 2022-08-30 2022-11-25 上海第二工业大学 一种蘑菇生长状态时空预测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220007589A1 (en) * 2017-04-17 2022-01-13 Iron Ox, Inc. Method for monitoring growth of plants and generating a plant grow schedule
KR20190069648A (ko) * 2017-12-05 2019-06-20 농업회사법인 원스베리 주식회사 인삼의 영상 이미지 분석을 통한 생육량 측정 방법
CN109741393A (zh) * 2018-12-04 2019-05-10 上海大学 双孢菇直径测量和中心点定位方法
KR20220010374A (ko) * 2020-07-17 2022-01-25 주식회사 쎄슬프라이머스 스마트 엣지 디바이스 기반의 생육 분석 방법 및 그를 위한 장치 및 시스템
CN115392576A (zh) * 2022-08-30 2022-11-25 上海第二工业大学 一种蘑菇生长状态时空预测方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HAMAMOTO, TOMOHIRO ET AL.: "RGB-D Images Based 3D Plant Growth Prediction by Sequential Images-to-Images Translation with Plant Priors", COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS : 15TH INTERNATIONAL JOINT CONFERENCE, VISIGRAPP 2020, VALLETTA, MALTA, FEBRUARY 27–29, 2020 : REVISED SELECTED PAPERS, SPRINGER, CHAM, 29 February 2020 (2020-02-29) - 29 February 2020 (2020-02-29), Cham, pages 334 - 352, XP009553317, ISBN: 978-3-030-94892-4 *
HUANG HUIMIN; LIN LANFEN; TONG RUOFENG; HU HONGJIE; ZHANG QIAOWEI; IWAMOTO YUTARO; HAN XIANHUA; CHEN YEN-WEI; WU JIAN: "UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation", ICASSP 2020 - 2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), IEEE, 4 May 2020 (2020-05-04), pages 1055 - 1059, XP033793076, DOI: 10.1109/ICASSP40776.2020.9053405 *

Also Published As

Publication number Publication date
CN115392576A (zh) 2022-11-25

Similar Documents

Publication Publication Date Title
WO2024045749A1 (zh) 一种蘑菇生长状态时空预测方法
CN110455340B (zh) 一种基于大数据的农产品种植环境检测系统
Yin et al. Computer vision and machine learning applied in the mushroom industry: A critical review
CN111460990B (zh) 一种基于大数据的高寒牧区草原虫害监测预警系统及方法
Alencastre-Miranda et al. Robotics for sugarcane cultivation: Analysis of billet quality using computer vision
CN110427922A (zh) 一种基于机器视觉和卷积神经网络病虫害识别系统和方法
JP2019187259A (ja) 栽培支援方法、栽培支援プログラム、栽培支援装置、および栽培支援システム
CN102789579A (zh) 基于计算机视觉技术的温室作物水肥胁迫状态识别方法
CA3163818A1 (en) Crop scouting information systems and resource management
CN106768081A (zh) 一种判断果蔬生长状态的方法及系统
CN117575169A (zh) 一种基于数据处理的智慧农业管理系统
JP2021073860A (ja) 農作物生育推定装置、農作物生育推定システム、農作物生育推定方法、及びプログラム
CN108829762A (zh) 基于视觉的小目标识别方法和装置
CN116821630A (zh) 一种基于融合数据分析的农产品成熟度预测方法
CN114723667A (zh) 一种农业精细化种植及灾害预防控制系统
CN117197595A (zh) 基于边缘计算的果树生育期识别方法、装置及管理平台
CN116310806B (zh) 一种基于图像识别的智慧农业一体化管理系统及方法
CN111369497B (zh) 一种行走式树上果实连续计数方法及装置
CN111428990A (zh) 一种基于深度神经网络的水培花卉开花期花卉等级评估方法
CN113408334B (zh) 一种小龙虾全链条数据采集与智能检测方法及装置
EP3874936A1 (en) Plant growth monitoring system
CN112419356A (zh) 一种大棚中小树种植生长检测和记录方法及其装置
CN114663752B (zh) 一种基于机器视觉的食用豆产量智能估计方法、系统
CN116563654A (zh) 一种海鲜菇子实体生长质量特征提取及预测的系统和方法
CN116630707B (zh) 一种基于深度学习的番茄穴盘苗分级检测与数据传输方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23858788

Country of ref document: EP

Kind code of ref document: A1