CN117373016A - Tobacco leaf baking state judging method, device, equipment and storage medium - Google Patents
Tobacco leaf baking state judging method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117373016A CN117373016A CN202311367041.XA CN202311367041A CN117373016A CN 117373016 A CN117373016 A CN 117373016A CN 202311367041 A CN202311367041 A CN 202311367041A CN 117373016 A CN117373016 A CN 117373016A
- Authority
- CN
- China
- Prior art keywords
- tobacco leaf
- tobacco
- image
- model
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000002637 Nicotiana tabacum Nutrition 0.000 title claims abstract description 307
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 244000061176 Nicotiana tabacum Species 0.000 title 1
- 241000208125 Nicotiana Species 0.000 claims abstract description 306
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 34
- 238000000605 extraction Methods 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- XOFYZVNMUHMLCC-ZPOLXVRWSA-N prednisone Chemical compound O=C1C=C[C@]2(C)[C@H]3C(=O)C[C@](C)([C@@](CC4)(O)C(=O)CO)[C@@H]4[C@@H]3CCC2=C1 XOFYZVNMUHMLCC-ZPOLXVRWSA-N 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 60
- 238000004383 yellowing Methods 0.000 claims description 27
- 238000001035 drying Methods 0.000 claims description 24
- 238000013135 deep learning Methods 0.000 claims description 20
- 238000012850 discrimination method Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 16
- 230000002708 enhancing effect Effects 0.000 claims description 15
- 230000018044 dehydration Effects 0.000 claims description 12
- 238000006297 dehydration reaction Methods 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 claims description 11
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 11
- 238000002372 labelling Methods 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- 238000012795 verification Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 238000012800 visualization Methods 0.000 description 6
- 230000004075 alteration Effects 0.000 description 5
- 210000003462 vein Anatomy 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 229930002875 chlorophyll Natural products 0.000 description 3
- 235000019804 chlorophyll Nutrition 0.000 description 3
- ATNHDLDRLWWWCB-AENOIHSZSA-M chlorophyll a Chemical compound C1([C@@H](C(=O)OC)C(=O)C2=C3C)=C2N2C3=CC(C(CC)=C3C)=[N+]4C3=CC3=C(C=C)C(C)=C5N3[Mg-2]42[N+]2=C1[C@@H](CCC(=O)OC\C=C(/C)CCC[C@H](C)CCC[C@H](C)CCCC(C)C)[C@H](C)C2=C5 ATNHDLDRLWWWCB-AENOIHSZSA-M 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 101150104012 TOP2 gene Proteins 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 101100153581 Bacillus anthracis topX gene Proteins 0.000 description 1
- 101150041570 TOP1 gene Proteins 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000011049 filling Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000008961 swelling Effects 0.000 description 1
- 238000001429 visible spectrum Methods 0.000 description 1
- 238000004804 winding Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/54—Extraction of image or video features relating to texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention discloses a tobacco leaf baking state judging method, a device, equipment and a storage medium, and relates to the technical field of tobacco leaf baking. The tobacco leaf baking state judging method comprises the following steps: s1, acquiring and restoring a tobacco leaf image through a standard color chart and a color difference delta E debugging camera to generate an initial tobacco leaf image data set; s2, extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, and marking the key tobacco leaf pictures with degree grades in application processing software of a terminal by a user to generate a target tobacco leaf image data set; s3, preprocessing and data enhancement are carried out on the initial tobacco leaf image data set and the target tobacco leaf image data set. The invention solves the problems that in the existing tobacco leaf baking process, how to automatically and intelligently identify the baking state grade of tobacco leaf based on the baking tobacco leaf image in real time and give a next baking strategy according to the prediction result of the tobacco leaf state is not proposed.
Description
Technical Field
The invention relates to the technical field of tobacco leaf baking, in particular to a tobacco leaf baking state judging method, a device, equipment and a storage medium.
Background
The baking is a key link for determining the quality and economic benefit of tobacco leaves, is also the link which consumes the most labor and relates to a plurality of different procedures of baking (baking process execution and temperature and humidity control). The baking process is important in tobacco leaf making links. The yellowing degree and the drying water loss degree of the tobacco leaves are two Jin Biaoche which measure the quality of the tobacco leaf baking process. When the baking staff judges and controls the whole process, the actual baking process has a large amount of deviation due to the artificial factors and natural factors, and a large amount of baking loss occurs.
The baking room automatic control equipment is currently touted by China, but is a temperature and humidity control tool which is mainly operated by baking staff all the time. In the baking process, technicians need to continuously observe the color and shape changes of tobacco leaves so as to adjust the temperature and humidity of the baking room, and the whole process is particularly time-consuming and tedious. Besides consuming a great deal of manpower and material resources, the uncontrollable risk index brought by the artificial factors in the mode also rises linearly.
In recent years, with the progress of big data and artificial intelligence algorithms, the image acquisition and algorithm recognition capabilities are greatly improved. Through a deep learning image recognition algorithm, an intelligent recognition model of the yellowing degree, the drying dehydration degree and the browning degree of tobacco leaves in the baking process of automatic control equipment is constructed, tobacco leaf images in the baking process are collected in real time, the yellowing degree, the drying dehydration degree and the browning degree of tobacco leaves are predicted according to the model, and then the temperature and the humidity are reasonably regulated and controlled according to a baking process technology, so that the intelligent recognition model is an effective way for reducing tobacco leaf baking loss and improving tobacco leaf quality.
Aiming at the tobacco leaf baking problem, the related research is mainly focused on manually extracting image features, and establishing a mapping relation between the image features and chemical component content or baking stage through a regression or classification model. For example, by calculating the color characteristics of tobacco leaf images [ lightness value (L#) ], redness value (a#) ], yellowness value (b#) ], saturation (C#) ] and hue angle (H#) ] and image information technology, extracting texture characteristics [ texture mean (m), standard deviation (sigma), smoothness (R), third-order moment (mu 3), consistency (U) and texture entropy (e) ] and predicting the moisture content of tobacco leaves by using a regression analysis method, reference ([ 1] Chen Feicheng and the like ] "prediction of the moisture content of tobacco leaves in the flue-cured tobacco curing process based on image information", "southwest agricultural newspaper 34.11 (2021): 7.); the measurement of the chlorophyll content of tobacco based on images is realized by analyzing the relation between leaf image color index parameters R/(G+B), chl.a, chl.b, chl. (a+b) and chlorophyll content, and can be referred to in the literature ([ 2] Sun Zhiwei and the like), "a flue-cured tobacco leaf chlorophyll content estimation model based on visible spectrum parameters", "Chinese tobacco science 41.1 (2020): 6"). According to the national three-stage tobacco flue-curing process standard, the whole tobacco flue-curing process is divided into three stages of yellowing stage, color fixing stage and dry reinforcement stage. And (3) constructing an SVM model based on a genetic algorithm to identify the baking stage by 11 color features and 8 texture features of the whole tobacco leaf image in the image extraction baking process. The reference (3) Li Zengcheng et al, "tobacco flue-curing stage discrimination model based on image processing is preferably" chinese tobacco journal 28.2 (2022): 12.
However, in the existing tobacco leaf baking process, how to automatically and intelligently identify the baking state (including yellowing degree, drying dehydration degree and browning degree) grade of tobacco leaf based on the baked tobacco leaf image in real time is not proposed, and the problem of the next baking strategy is given according to the prediction result of the tobacco leaf state, and no effective solution is proposed at present.
Disclosure of Invention
The invention aims to: a tobacco leaf baking state judging method, device, equipment and storage medium are provided to solve the above problems existing in the prior art.
The technical scheme is as follows: a tobacco leaf baking state judging method comprises the following steps: s1, acquiring and restoring a tobacco leaf image through a standard color chart and a color difference delta E debugging camera to generate an initial tobacco leaf image data set; s2, extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, and marking the key tobacco leaf pictures with degree grades in application processing software of a terminal by a user to generate a target tobacco leaf image data set; s3, preprocessing and data enhancing are carried out on the initial tobacco leaf image dataset and the target tobacco leaf image dataset; and S4, training a tobacco baking state discrimination model by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm.
Preferably, the tobacco leaf image is collected and restored by a standard color chart and a color difference delta E debugging camera, and the generating of the initial tobacco leaf image data set comprises the following steps: the sRGB theoretical values of a standard 24-color chart are collected; based on a Python programming call Opencv CCM class, comparing sRGB theoretical values and actual values of 24 color blocks in a standard color chart, and calculating a CCM conversion matrix; calculating a color difference Δe value using a delta_e index in Lab space, Δe= [ (Δl) 2+ (Δa) 2+ (Δb) 2]1/2; wherein Δe represents color reproduction error of the acquired image, and Δe is controlled within a preset range.
Preferably, extracting a key tobacco leaf picture in the initial tobacco leaf image dataset based on a key frame extraction algorithm, and marking the key tobacco leaf picture with a degree grade in application processing software of a terminal by a user, wherein generating a target tobacco leaf image dataset comprises: differentiating the two frames of images based on a key frame extraction algorithm of the difference to obtain the average pixel intensity of the images, sequencing the differential intensity, extracting a part of images with larger change in the cured tobacco leaf images, and giving the part of images to an expert for marking; marking the key frame image set independently by n tobacco leaf baking experts, wherein the key frame image set comprises yellowing degree, drying dehydration degree and browning degree; screening out images with at least n-1 expert labeling results consistent as a target tobacco leaf image dataset; and arranging the target tobacco leaf image data set into an image format.
Preferably, preprocessing and data enhancing the initial tobacco leaf image dataset and the target tobacco leaf image dataset comprises: randomly segmenting an image data set into a training set, a verification set and a test set according to a preset proportion, and adjusting the image to a preset size; expanding the data set by adopting an image enhancement technology; the image enhancement technology comprises the following steps: geometric transformation, pixel transformation, and hybrid enhancement; and normalizing and standardizing the tobacco leaf image after data enhancement to accelerate the convergence of the model.
Preferably, training the tobacco baking state discrimination model based on the deep learning image recognition algorithm by using the preprocessed initial tobacco image dataset and the target tobacco image dataset comprises: preprocessing data, namely preprocessing and enhancing data of the cured tobacco leaf image according to the step S3; self-supervision pre-training, namely removing a target tobacco leaf image dataset from the preprocessed initial tobacco leaf image dataset, and adopting an MAE self-supervision pre-training method to obtain a pre-training model; model training, namely constructing a multi-task training model based on a multi-task Vision Transfomer algorithm by combining an MAE pre-trained encoder weight, and outputting yellowing degree, drying and water loss degree and browning degree information through the multi-task training model; and model deployment, namely deploying algorithm service by using a flash framework based on the centOS system, and providing access interfaces for mobile terminal and web terminal applications.
Preferably, the model deployment, based on the centOS system, uses a flash framework to deploy an algorithm service, and after providing an access interface for the mobile terminal and the web terminal application, further comprises: APP end demonstrates model prediction structure, include: and writing the accuracy and the loss function value into the tensorboard in the model training process, and obtaining a visual chart display of the model evaluation index at the local port.
Preferably, after training the tobacco baking state discrimination model by using the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm, the method further comprises: and inputting the tobacco leaf image data to be identified into the tobacco leaf baking state discrimination model, and outputting a temperature and humidity regulation strategy.
In order to achieve the above object, according to another aspect of the present application, there is provided a tobacco flue-curing state discriminating apparatus.
The tobacco flue-curing state distinguishing device according to the application comprises: the tobacco leaf image acquisition module is used for acquiring and restoring tobacco leaf images through the standard color chart and the color difference delta E debugging camera to generate an initial tobacco leaf image data set; the key tobacco leaf picture extraction and labeling module is used for extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, labeling the key tobacco leaf pictures in application processing software of a terminal through a user to generate a target tobacco leaf image data set; the tobacco image data processing module is used for preprocessing and enhancing the initial tobacco image data set and the target tobacco image data set; the model training module is used for training a tobacco baking state judging model based on a deep learning image recognition algorithm by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset.
To achieve the above object, according to another aspect of the present application, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the tobacco curing status discrimination method of any one of the present invention.
To achieve the above object, according to another aspect of the present application, there is provided a computer-readable storage medium having stored therein computer instructions for causing a processor to execute the tobacco flue-curing state discrimination method according to any one of the present inventions.
The beneficial effects are that: in the embodiment of the application, a model self-distinguishing mode is adopted, and a standard color chart and a color difference delta E are used for debugging a camera to collect and restore tobacco leaf images, so that an initial tobacco leaf image data set is generated; extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, and marking the key tobacco leaf pictures in application processing software of a terminal by a user to obtain a target tobacco leaf image data set; preprocessing and data enhancing the initial tobacco leaf image data set and the target tobacco leaf image data set; the tobacco baking state discrimination model is trained by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm, so that the purposes of training and generating the tobacco baking state discrimination model are achieved, the technical effect of outputting a temperature and humidity regulation strategy is achieved, and the technical problems that how to automatically and intelligently recognize the baking state (including yellowing degree, drying and dehydration degree and browning degree) grade of tobacco based on a baking tobacco image in real time and give a next baking strategy according to a prediction result of the tobacco state in the existing tobacco baking process are solved.
Drawings
Fig. 1 is a flow chart of a tobacco flue-curing state discrimination method according to an embodiment of the present application;
FIG. 2 is a schematic illustration of MAE pre-training process of a tobacco flue-curing status discrimination method according to a preferred embodiment of the present application;
FIG. 3 is a multi-task Vision Transfomer (ViT) based algorithm architecture for a tobacco flue-curing status discrimination method according to an embodiment of the present application;
FIG. 4 is a tensorboard model accuracy indicator visualization of a tobacco flue-state discrimination method according to an embodiment of the present application;
FIG. 5 is a schematic view of Grad-CAM visualization of a tobacco curing status discrimination method according to an embodiment of the present application;
FIG. 6 is a flow chart of a tobacco flue-curing status discrimination method according to yet another embodiment of the present application;
fig. 7 is a schematic structural view of a tobacco flue-curing state discrimination apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural view of an electronic device of the tobacco flue-curing state discrimination method according to the embodiment of the present application.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the present application described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Furthermore, the terms "mounted," "configured," "provided," "connected," "coupled," and "sleeved" are to be construed broadly. For example, it may be a fixed connection, a removable connection, or a unitary construction; may be a mechanical connection, or an electrical connection; may be directly connected, or indirectly connected through intervening media, or may be in internal communication between two devices, elements, or components. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
According to an embodiment of the present invention, there is provided a tobacco flue-curing state discriminating method, as shown in fig. 1 and 6, including steps S101 to S104 as follows:
step S101, acquiring and restoring tobacco leaf images through a standard color chart and a color difference delta E debugging camera to generate an initial tobacco leaf image data set;
a large number of tobacco curing images can be acquired, providing a data basis for subsequent analysis and processing.
According to the embodiment of the invention, preferably, the tobacco leaf image is collected and restored by the standard color chart and the color difference delta E debugging camera, and the generating of the initial tobacco leaf image data set comprises the following steps:
the sRGB theoretical values of a standard 24-color chart are collected;
such as: and purchasing the Spydecrcor 24 standard 24 color chart, and obtaining the sRGB theoretical values of 24 color patches in the standard color chart from an official channel.
Based on a Python programming call Opencv CCM class, comparing sRGB theoretical values and actual values of 24 color blocks in a standard color chart, and calculating a CCM conversion matrix;
Specifically, the camera parameters are adjusted using a color chart. Firstly, CCM enabling in camera debugging software MVS is closed, a real object color comparison card is put into a conveyor belt under a Bayer mode to collect a picture, and sRGB actual values of 24 color blocks in the color comparison card detected in the picture are identified for the collected color comparison card picture. Based on the Python programming call Opencv ColorCorrectionmodel class, the sRGB theoretical values and the actual values of 24 color blocks in the standard color chart are compared, and a CCM (Color Correction Matrix) conversion matrix is calculated. And then, opening CCM enabling in the camera debugging software MVS, and filling the calculated CCM matrix values one by one. The CCM enable button in the camera commissioning software MVS is kept in an open state, whereby the commissioning of the CCM parameters of the camera based on the 24-standard color card has been completed.
Calculating a color difference Δe value using a delta_e index in Lab space, Δe= [ (Δl) 2+ (Δa) 2+ (Δb) 2]1/2; wherein Δe represents color reproduction error of the acquired image, and Δe is controlled within a preset range. Wherein, the preset range can be within 15.
The color reduction effect of the tobacco leaf image can be well achieved, and therefore the image accuracy is improved.
Specifically, when image acquisition is performed on cured tobacco leaves, when a dark sealed curing barn is used, a camera shoots color with larger chromatic aberration, after the method is adopted, based on an ISP digital image processing flow, CCM matrix values obtained through calculation by using a standard color chart are embedded into MVS camera parameter adjusting software, a Colorcorrection function (between AWB white balance and Gamma calibration) in the ISP flow is realized, and color reproducibility is checked by using chromatic aberration delta E.
Step S102, extracting key tobacco leaf pictures in the initial tobacco leaf image dataset based on a key frame extraction algorithm, and marking the key tobacco leaf pictures with degree grades in application processing software of a terminal by a user to generate a target tobacco leaf image dataset;
by adopting the video key frame extraction algorithm, the image processing efficiency and accuracy can be improved.
According to the embodiment of the present invention, preferably, extracting the key tobacco leaf picture in the initial tobacco leaf image dataset based on a key frame extraction algorithm, and marking the key tobacco leaf picture in the application processing software of the terminal by the user to a degree level, and generating the target tobacco leaf image dataset includes:
differentiating the two frames of images based on a key frame extraction algorithm of the difference to obtain the average pixel intensity of the images, sequencing the differential intensity, extracting a part of images with larger change in the cured tobacco leaf images, and giving the part of images to an expert for marking;
Such as: the camera continuously takes flue-cured tobacco images at a frequency of one every 10 minutes throughout each baking oven pass, so there are a large number of repeated images without variation. Taking an image set of a single heat as a video, differentiating the two frames of images based on a key frame extraction algorithm of difference to obtain the average pixel intensity of the images, measuring the variation of the two frames of images, extracting a part of images with larger variation of the cured tobacco leaves from the difference intensity sequence, and giving the part of images to an expert for marking
Marking the key frame image set independently by n tobacco leaf baking experts, wherein the key frame image set comprises yellowing degree, drying dehydration degree and browning degree; screening out images with at least n-1 expert labeling results consistent as a target tobacco leaf image dataset;
such as: the key frame image sets are marked independently by 3 tobacco flue-curing experts, and the key frame image sets comprise yellowing degree (7 grades), drying water loss degree (9 grades) and browning degree (6 grades). And then screening out images with consistent labeling results of at least 2 experts as a data set.
And arranging the target tobacco leaf image data set into an image format.
Specifically, when the data set is constructed, because the acquisition frequency is high, if all the images are marked directly, the image repeatability is high and the manual marking cost is very high, so that a video key frame extraction algorithm is adopted, images of key nodes are extracted to carry out multi-bit expert marking, marked images with high consistency are selected as the constructed data set, and the high quality and low redundancy of the data set are ensured.
It is to be understood that the yellowing degree of the cured tobacco leaves is classified into 5-6 yellowing, 7-8 yellowing, yellow piece green basal portion with green, yellow piece white (main vein blushing and blushing), main vein partial shrinkage and purple and main vein purple of 7 degree grades. The degree of dehydration of baked tobacco leaves is divided into 9 degree grades of swelling and hardening, softening of leaf tips, softening of leaves, leaf slumping frames (complete withering), leaf tip hooking and curling, leaf dry sheets 1/2-2/3, leaf full drying (large winding drum), main vein drying 1/2 and main vein full drying. The browning degree of the cured tobacco leaves is divided into 6 degree grades of no browning, slight (within 10%), small (10% -20%), small (20% -30%), partial (30% -50%) and most (more than 50%). The yellowing degree and the browning degree of the cured tobacco depend on the image acquisition quality, and parameters of a camera and a light source are adjusted, so that the acquired cured tobacco image can truly reflect the real color of the tobacco, and the cured tobacco becomes an important premise for the effectiveness of an algorithm model. The scheme adopts standard white balance calibration and chromatic aberration delta E to calibrate camera parameters so as to achieve the effect of avoiding chromatic aberration and restoring color.
The distribution of the data is shown in Table 1-1, table 1-2, and Table 3 below:
Table 1-1 sample distribution of degree of yellowing of tobacco leaves:
table 1-2 sample distribution of tobacco leaf water loss:
table 1-3 sample distribution of degree of browning of tobacco leaves:
step S103, preprocessing and data enhancement are carried out on the initial tobacco leaf image dataset and the target tobacco leaf image dataset;
the method can realize good data processing and cleaning effects on the tobacco leaf image data, thereby avoiding interference with subsequent analysis and judgment of the picture image and further improving the judgment accuracy.
According to an embodiment of the present invention, preferably, preprocessing and data enhancing the initial tobacco leaf image dataset and the target tobacco leaf image dataset includes:
randomly segmenting an image data set into a training set, a verification set and a test set according to a preset proportion, and adjusting the image to a preset size;
expanding the data set by adopting an image enhancement technology; the image enhancement technology comprises the following steps: geometric transformation, pixel transformation, and hybrid enhancement;
and normalizing and standardizing the tobacco leaf image after data enhancement to accelerate the convergence of the model.
Specifically, the training set, the verification set and the test set are randomly segmented according to the proportion of 7:2:1 by the image data set. Scaling the image to a 224 x 3 size;
The dataset is augmented with image enhancement techniques including geometric transformations, pixel transformations, and hybrid enhancement.
(1) And (3) rotation: rotating the tobacco leaf image left and right by 45 degrees respectively;
(2) Mirror image flip: vertically and horizontally overturning the tobacco leaf image;
(3) Gaussian noise: adding random Gaussian noise into the tobacco leaf image;
(4) Mixing and enhancing: mix enhancement was performed using mix up and CutMix.
Step S104, training a tobacco baking state discrimination model by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm;
the tobacco leaf baking state judging model based on the deep learning algorithm is designed, and the tobacco leaf image is taken as input, and the yellowing degree, the drying and water loss degree and the browning degree information are taken as output, so that the end-to-end baking tobacco leaf state judging model is designed. The VIT algorithm is used as a basic structure for judging the state of the cured tobacco leaves. The overall flow is shown in fig. 1.
According to the embodiment of the present invention, preferably, training the tobacco baking state discrimination model based on the deep learning image recognition algorithm by using the preprocessed initial tobacco image dataset and the target tobacco image dataset includes:
Preprocessing data, namely preprocessing and enhancing data of the cured tobacco leaf image according to the step S3;
self-supervision pre-training, namely removing a target tobacco leaf image dataset from the preprocessed initial tobacco leaf image dataset, and adopting an MAE self-supervision pre-training method to obtain a pre-training model;
model training, namely constructing a multi-task training model based on a multi-task Vision Transfomer algorithm by combining an MAE pre-trained encoder weight, and outputting yellowing degree, drying and water loss degree and browning degree information through the multi-task training model; wherein, the browning degree model has the problem of unbalanced category, and the weight of the loss function is adjusted according to the category proportion;
and model deployment, namely deploying algorithm service by using a flash framework based on the centOS system, and providing access interfaces for mobile terminal and web terminal applications.
Specifically, as shown in fig. 2, by using the MAE self-supervision pre-training technology, image data without labels can be used for pre-training in a self-supervision manner, so as to obtain pre-training model weights, and the MAe pre-trained encoder part is used as a backstone of a downstream task, and the backstone automatically extracts high-level abstract features of an image without manually defining color and texture features. On the basis of pre-training, the convergence rate of the model in formal training can be increased, the training effect of the model is improved, the dependence on the sample size is reduced, and the cost of marking data is greatly reduced.
It should be appreciated that MAE (Masked Autoencoder) is a self-supervised pre-training method for feature learning and representation learning of image data. The following are general steps of the MAE self-supervised pretraining method: data preparation: first, it is necessary to prepare an image data set for self-supervised learning. This may be a dataset containing a large number of images, typically unlabeled data.
Image masking: in the MAE method, a portion of the image data may be masked, i.e., some regions may be hidden or blocked. This is to introduce a self-supervising signal that lets the model learn how to reconstruct the missing part.
Model architecture: a self-encoder architecture, typically an encoder-decoder architecture, is designed. The encoder encodes the image input into a low-dimensional representation, which the decoder restores to a complete image. This encoding-decoding process is used to reconstruct the mask portion.
Self-supervision target: the goal of the MAE is to minimize the reconstruction error, i.e., the difference between the output image of the model and the original image. Because a portion of the image is masked, the model needs to infer the masked portion from the remaining visible portions. During the training process, the object of the model is to make the reconstructed image as close as possible to the original image.
Super parameter setting: super parameters of the training MAE model, such as learning rate, batch size, training cycle number, etc., are set. The selection of these parameters may require experience or experimentation.
Training: self-supervised training is performed using the prepared dataset and model architecture. The training process will update the model parameters while continuously attempting to reduce the reconstruction error.
Feature extraction: once the MAE model is pre-trained, the encoder section may be used to extract image features. These features may be used for various visual tasks such as image classification, object detection, image generation, etc.
Fine tuning (optional): the pre-trained feature extractor may be further used for task-specific fine tuning to improve performance.
The key idea of the MAE self-supervised pre-training approach is to learn useful features by learning how to reconstruct images from partial information. The method is widely applied in the field of computer vision, and can improve the performance of various image related tasks. Of course, the specific use steps can be adjusted according to the actual use requirements.
Meanwhile, as shown in fig. 3, considering that certain correlation exists among three tasks of yellowing degree, drying and water loss degree and browning degree, the multi-task model based on ViT-base algorithm architecture is constructed because the multi-task model can improve the learning efficiency and quality of each task by learning the relation and difference of different tasks. The model prediction accuracy reaches 75%, and the accuracy of adjacent levels reaches 95% +; preliminarily meets the requirements of popularization and use.
In addition, when the model is deployed, three tasks of yellowing degree, drying water loss degree and browning degree are considered as a degree grade, and the model belongs to continuous variable properties. The actual tobacco curing status may not be of a particular grade, but rather intermediate between some two grades. Thus, at model deployment, the level and probability of the model predictive probability value top2 are output. Compared with the top1 prediction result, the top2 contains more abundant change trend information.
The invention aims at uncontrollable baking loss caused by artificial factors in a mode of continuously observing the color and shape changes of tobacco leaves by technicians in the tobacco leaf baking process so as to adjust the temperature and humidity of a baking room. The method for constructing the discrimination model of the yellowing degree, the drying dehydration degree and the browning degree in the tobacco leaf baking process based on the deep learning image recognition algorithm is provided. The camera parameters are adjusted by combining the white balance calibration and the chromatic aberration delta E to ensure the color reproducibility and the image data acquisition quality. And extracting a key frame image by a key frame extraction technology based on real-time stream data, and providing the key frame image for an expert to label the yellowing degree, the drying dehydration degree and the browning degree so as to reduce the cost of manual labeling of the expert. Therefore, a baking image data set with high quality and expert labels is obtained, an image recognition model is built based on a deep learning algorithm, and the yellowing degree, the drying and water loss degree and the browning degree of tobacco leaves in the baking process are predicted in real time, so that a temperature and humidity regulation strategy is provided.
From the above description, it can be seen that the following technical effects are achieved:
in the embodiment of the application, a model self-distinguishing mode is adopted, and a standard color chart and a color difference delta E are used for debugging a camera to collect and restore tobacco leaf images, so that an initial tobacco leaf image data set is generated; extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, and marking the key tobacco leaf pictures in application processing software of a terminal by a user to obtain a target tobacco leaf image data set; preprocessing and data enhancing the initial tobacco leaf image data set and the target tobacco leaf image data set; the tobacco baking state discrimination model is trained by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm, so that the purposes of training and generating the tobacco baking state discrimination model are achieved, the technical effect of outputting a temperature and humidity regulation strategy is achieved, and the technical problems that how to automatically and intelligently recognize the baking state (including yellowing degree, drying and dehydration degree and browning degree) grade of tobacco based on a baking tobacco image in real time and give a next baking strategy according to a prediction result of the tobacco state in the existing tobacco baking process are solved.
According to the embodiment of the invention, preferably, the model deployment, based on the centOS system, uses a flash framework to deploy algorithm service, and after providing an access interface for mobile terminal and web terminal applications, further comprises: APP end demonstrates model prediction structure, include:
and writing the accuracy and the loss function value into the tensorboard in the model training process, and obtaining a visual chart display of the model evaluation index at the local port.
Good visual effect can be realized, so that the running state is accurately known.
According to the embodiment of the present invention, preferably, after training the tobacco baking state discrimination model by using the preprocessed initial tobacco image dataset and the target tobacco image dataset based on the deep learning image recognition algorithm, the method further includes:
and inputting the tobacco leaf image data to be identified into the tobacco leaf baking state discrimination model, and outputting a temperature and humidity regulation strategy.
Specifically, as shown in fig. 4, the visualization includes an evaluation index visualization and an activation map visualization of the model training process. The accuracy and the loss function value are written into the tensorboard in the model training process, and a visual chart display of the model evaluation index can be obtained at the local port. Visualization of the presentation model using the Grad-CAM activation map is based on which features of the image are predictively output. It can be observed that the model focus is on the blade as shown in fig. 5.
In order to achieve the above object, according to another aspect of the present application, there is provided a tobacco flue-curing state discriminating apparatus. As shown in fig. 7, the tobacco flue-curing state discrimination apparatus includes:
the tobacco image acquisition module 701 is used for acquiring and restoring a tobacco image through a standard color chart and a color difference delta E debugging camera to generate an initial tobacco image data set;
a large number of tobacco curing images can be acquired, providing a data basis for subsequent analysis and processing.
The key tobacco leaf picture extraction and labeling module 702 is configured to extract a key tobacco leaf picture in the initial tobacco leaf image data set based on a key frame extraction algorithm, and label the key tobacco leaf picture by a user in application processing software of a terminal to generate a target tobacco leaf image data set;
by adopting the video key frame extraction algorithm, the image processing efficiency and accuracy can be improved.
A tobacco image data processing module 703, configured to perform preprocessing and data enhancement on the initial tobacco image dataset and the target tobacco image dataset;
the method can realize good data processing and cleaning effects on the tobacco leaf image data, thereby avoiding interference with subsequent analysis and judgment of the picture image and further improving the judgment accuracy.
According to an embodiment of the present invention, preferably, preprocessing and data enhancing the initial tobacco leaf image dataset and the target tobacco leaf image dataset includes:
randomly segmenting an image data set into a training set, a verification set and a test set according to the proportion of 7:2:1, and adjusting the image to a preset size;
expanding the data set by adopting an image enhancement technology; the image enhancement technology comprises the following steps: geometric transformation, pixel transformation, and hybrid enhancement;
and normalizing and standardizing the tobacco leaf image after data enhancement to accelerate the convergence of the model.
The model training module 704 is configured to train a tobacco baking state discrimination model based on a deep learning image recognition algorithm by using the preprocessed initial tobacco image dataset and the target tobacco image dataset.
The tobacco leaf baking state judging model based on the deep learning algorithm is designed, and the tobacco leaf image is taken as input, and the yellowing degree, the drying and water loss degree and the browning degree information are taken as output, so that the end-to-end baking tobacco leaf state judging model is designed. The VIT algorithm is used as a basic structure for judging the state of the cured tobacco leaves.
From the above description, it can be seen that the following technical effects are achieved:
In the embodiment of the application, a model self-distinguishing mode is adopted, and a standard color chart and a color difference delta E are used for debugging a camera to collect and restore tobacco leaf images, so that an initial tobacco leaf image data set is generated; extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, and marking the key tobacco leaf pictures in application processing software of a terminal by a user to obtain a target tobacco leaf image data set; preprocessing and data enhancing the initial tobacco leaf image data set and the target tobacco leaf image data set; the tobacco baking state discrimination model is trained by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm, so that the purposes of training and generating the tobacco baking state discrimination model are achieved, the technical effect of outputting a temperature and humidity regulation strategy is achieved, and the technical problems that how to automatically and intelligently recognize the baking state (including yellowing degree, drying and dehydration degree and browning degree) grade of tobacco based on a baking tobacco image in real time and give a next baking strategy according to a prediction result of the tobacco state in the existing tobacco baking process are solved.
As shown in fig. 8, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM12 and the RAM13 are connected to each other via a bus 14. An input/output (I/0) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/0 interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the respective methods and processes described above, such as the tobacco flue-curing state discrimination method.
In some embodiments, the tobacco curing status discrimination method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM12 and/or the communication unit 19. When the computer program is loaded into the RAM13 and executed by the processor 11, one or more steps of the above-described tobacco leaf curing state discrimination method may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the tobacco curing status discrimination method in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks (LA), wide area networks (WA), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. The tobacco leaf baking state judging method is characterized by comprising the following steps:
s1, acquiring and restoring a tobacco leaf image through a standard color chart and a color difference delta E debugging camera to generate an initial tobacco leaf image data set;
s2, extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, and marking the key tobacco leaf pictures with degree grades in application processing software of a terminal by a user to generate a target tobacco leaf image data set;
S3, preprocessing and data enhancing are carried out on the initial tobacco leaf image dataset and the target tobacco leaf image dataset;
and S4, training a tobacco baking state discrimination model by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm.
2. The method of claim 1, wherein the step of collecting and restoring the tobacco leaf image by the standard color chart and the color difference Δe debug camera to generate an initial tobacco leaf image dataset comprises:
the sRGB theoretical values of a standard 24-color chart are collected;
based on a Python programming call Opencv CCM class, comparing sRGB theoretical values and actual values of 24 color blocks in a standard color chart, and calculating a CCM conversion matrix;
calculating a color difference Δe value using a delta_e index in Lab space, Δe= [ (Δl) 2+ (Δa) 2+ (Δb) 2]1/2; wherein Δe represents color reproduction error of the acquired image, and Δe is controlled within a preset range.
3. The method of claim 1, wherein extracting key tobacco pictures in the initial tobacco image dataset based on a key frame extraction algorithm, and marking the key tobacco pictures with a degree level in application processing software of a terminal by a user, generating a target tobacco image dataset comprises:
Differentiating the two frames of images based on a key frame extraction algorithm of the difference to obtain the average pixel intensity of the images, sequencing the differential intensity, extracting a part of images with larger change in the cured tobacco leaf images, and giving the part of images to an expert for marking;
marking the key frame image set independently by n tobacco leaf baking experts, wherein the key frame image set comprises yellowing degree, drying dehydration degree and browning degree; screening out images with at least n-1 expert labeling results consistent as a target tobacco leaf image dataset;
and arranging the target tobacco leaf image data set into an image format.
4. The tobacco flue-curing status discrimination method of claim 1, wherein preprocessing and data enhancing the initial tobacco image dataset and the target tobacco image dataset includes:
randomly segmenting an image data set into a training set, a verification set and a test set according to a preset proportion, and adjusting the image to a preset size;
expanding the data set by adopting an image enhancement technology; the image enhancement technology comprises the following steps: geometric transformation, pixel transformation, and hybrid enhancement;
and normalizing and standardizing the tobacco leaf image after data enhancement to accelerate the convergence of the model.
5. The tobacco flue-curing state discrimination method according to claim 1, wherein training a tobacco flue-curing state discrimination model using a preprocessed initial tobacco image dataset and the target tobacco image dataset based on a deep learning image recognition algorithm includes:
preprocessing data, namely preprocessing and enhancing data of the cured tobacco leaf image according to the step S3;
self-supervision pre-training, namely removing a target tobacco leaf image dataset from the preprocessed initial tobacco leaf image dataset, and adopting an MAE self-supervision pre-training method to obtain a pre-training model;
model training, namely constructing a multi-task training model based on a multi-task Vision Transfomer algorithm by combining an MAE pre-trained encoder weight, and outputting yellowing degree, drying and water loss degree and browning degree information through the multi-task training model;
and model deployment, namely deploying algorithm service by using a flash framework based on the centOS system, and providing access interfaces for mobile terminal and web terminal applications.
6. The method according to claim 1, wherein the model deployment, after providing an access interface for mobile terminal and web terminal applications by using a flash framework deployment algorithm service based on the centOS system, further comprises: APP end demonstrates model prediction structure, include:
And writing the accuracy and the loss function value into the tensorboard in the model training process, and obtaining a visual chart display of the model evaluation index at the local port.
7. The tobacco flue-curing state discrimination method according to claim 1, wherein after training a tobacco flue-curing state discrimination model based on a deep learning image recognition algorithm using the preprocessed initial tobacco image dataset and the target tobacco image dataset, further comprising:
and inputting the tobacco leaf image data to be identified into the tobacco leaf baking state discrimination model, and outputting a temperature and humidity regulation strategy.
8. The tobacco leaf baking state distinguishing device is characterized by comprising:
the tobacco leaf image acquisition module is used for acquiring and restoring tobacco leaf images through the standard color chart and the color difference delta E debugging camera to generate an initial tobacco leaf image data set;
the key tobacco leaf picture extraction and labeling module is used for extracting key tobacco leaf pictures in the initial tobacco leaf image data set based on a key frame extraction algorithm, labeling the key tobacco leaf pictures in application processing software of a terminal through a user to generate a target tobacco leaf image data set;
the tobacco image data processing module is used for preprocessing and enhancing the initial tobacco image data set and the target tobacco image data set;
The model training module is used for training a tobacco baking state judging model based on a deep learning image recognition algorithm by utilizing the preprocessed initial tobacco image dataset and the target tobacco image dataset.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the tobacco curing status discrimination method of any one of claims 1-7.
10. A computer-readable storage medium, wherein computer instructions for causing a processor to execute the tobacco flue-curing state discrimination method according to any one of claims 1 to 7 are stored in the computer-readable storage medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311367041.XA CN117373016B (en) | 2023-10-20 | 2023-10-20 | Tobacco leaf baking state judging method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311367041.XA CN117373016B (en) | 2023-10-20 | 2023-10-20 | Tobacco leaf baking state judging method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117373016A true CN117373016A (en) | 2024-01-09 |
| CN117373016B CN117373016B (en) | 2024-04-30 |
Family
ID=89388737
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311367041.XA Active CN117373016B (en) | 2023-10-20 | 2023-10-20 | Tobacco leaf baking state judging method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117373016B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818827A (en) * | 2021-01-28 | 2021-05-18 | 贵州省烟草科学研究院 | Image recognition-based method for judging stage temperature control point in tobacco leaf baking process |
| CN113213016A (en) * | 2021-05-31 | 2021-08-06 | 南京工业大学 | Garbage recognition and classification algorithm based on ViT, and device and control method thereof |
| CN114331979A (en) * | 2021-12-16 | 2022-04-12 | 华南农业大学 | Intelligent decision method and system for tobacco leaf baking |
| CN114972770A (en) * | 2022-06-20 | 2022-08-30 | 中国烟草总公司郑州烟草研究院 | Tobacco leaf grading method and device and computer readable storage medium |
| CN115439690A (en) * | 2022-09-05 | 2022-12-06 | 湖南大学 | Flue-cured tobacco leaf image grading method combining CNN and Transformer |
| CN115661024A (en) * | 2022-08-16 | 2023-01-31 | 农芯(南京)智慧农业研究院有限公司 | Cigar coat color distinguishing method and device, storage medium and server |
| CN115731189A (en) * | 2022-11-22 | 2023-03-03 | 河北师范大学 | Enteroscopy image lesion identification method and device, electronic equipment and storage medium |
| WO2023071743A1 (en) * | 2021-10-25 | 2023-05-04 | 中兴通讯股份有限公司 | Network model training method and apparatus, and computer-readable storage medium |
| CN116363429A (en) * | 2023-03-28 | 2023-06-30 | 北京百度网讯科技有限公司 | Training method of image recognition model, image recognition method, device and equipment |
| CN116524224A (en) * | 2022-12-09 | 2023-08-01 | 云南省烟草农业科学研究院 | Machine vision-based method and system for detecting type of cured tobacco leaves |
| CN116863116A (en) * | 2023-06-30 | 2023-10-10 | 平安科技（深圳）有限公司 | Image recognition method, device, equipment and medium based on artificial intelligence |
-
2023
- 2023-10-20 CN CN202311367041.XA patent/CN117373016B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818827A (en) * | 2021-01-28 | 2021-05-18 | 贵州省烟草科学研究院 | Image recognition-based method for judging stage temperature control point in tobacco leaf baking process |
| CN113213016A (en) * | 2021-05-31 | 2021-08-06 | 南京工业大学 | Garbage recognition and classification algorithm based on ViT, and device and control method thereof |
| WO2023071743A1 (en) * | 2021-10-25 | 2023-05-04 | 中兴通讯股份有限公司 | Network model training method and apparatus, and computer-readable storage medium |
| CN114331979A (en) * | 2021-12-16 | 2022-04-12 | 华南农业大学 | Intelligent decision method and system for tobacco leaf baking |
| CN114972770A (en) * | 2022-06-20 | 2022-08-30 | 中国烟草总公司郑州烟草研究院 | Tobacco leaf grading method and device and computer readable storage medium |
| CN115661024A (en) * | 2022-08-16 | 2023-01-31 | 农芯(南京)智慧农业研究院有限公司 | Cigar coat color distinguishing method and device, storage medium and server |
| CN115439690A (en) * | 2022-09-05 | 2022-12-06 | 湖南大学 | Flue-cured tobacco leaf image grading method combining CNN and Transformer |
| CN115731189A (en) * | 2022-11-22 | 2023-03-03 | 河北师范大学 | Enteroscopy image lesion identification method and device, electronic equipment and storage medium |
| CN116524224A (en) * | 2022-12-09 | 2023-08-01 | 云南省烟草农业科学研究院 | Machine vision-based method and system for detecting type of cured tobacco leaves |
| CN116363429A (en) * | 2023-03-28 | 2023-06-30 | 北京百度网讯科技有限公司 | Training method of image recognition model, image recognition method, device and equipment |
| CN116863116A (en) * | 2023-06-30 | 2023-10-10 | 平安科技（深圳）有限公司 | Image recognition method, device, equipment and medium based on artificial intelligence |
Non-Patent Citations (4)
| Title |
|---|
| WEI C, ET AL.: "Masked feature prediction for self-supervised visual pre-training", IEEE, 24 June 2022 (2022-06-24), pages 14668 - 14678 * |
| ZHANG W, ET AL.: "Tobacco leaf segmentation based on improved MASK RCNN algorithm and SAM model", IEEE, 18 September 2023 (2023-09-18), pages 103102 * |
| 李翔, 等: "Transformer在计算机视觉领域的研究综述", 计算机工程与应用, 10 October 2022 (2022-10-10), pages 1 - 14 * |
| 鲁梦瑶, 等: "基于深度学习与多尺度特征融合的烤烟烟叶分级方法", 中国农机化学报, 15 January 2022 (2022-01-15), pages 158 - 166 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117373016B (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110479636B (en) | Method and device for automatically sorting tobacco leaves based on neural network | |
| CN110807760B (en) | Tobacco leaf grading method and system | |
| CN110648490B (en) | Multi-factor flame identification method suitable for embedded platform | |
| CN109308696A (en) | Non-reference picture quality appraisement method based on hierarchy characteristic converged network | |
| CN108875118B (en) | Method and device for evaluating accuracy of prediction model of silicon content of blast furnace molten iron | |
| CN112818827A (en) | Image recognition-based method for judging stage temperature control point in tobacco leaf baking process | |
| CN112163527B (en) | Fusion model-based tobacco leaf baking state identification method, device and system | |
| CN111008641B (en) | Power transmission line tower external force damage detection method based on convolutional neural network | |
| CN110717520B (en) | Solid waste identification method based on fusion of spatial features and spectral features of hyperspectral images | |
| CN111860639A (en) | System and method for judging quantized flue-cured tobacco leaf curing characteristics | |
| CN112730251B (en) | Device and method for detecting screen color defects | |
| CN108198176A (en) | A kind of method of discrimination based on image analysis tobacco maturity | |
| CN110224862A (en) | Multi-agent system network appearance based on multilayer perceptron invades capability assessment method | |
| CN109389569A (en) | Based on the real-time defogging method of monitor video for improving DehazeNet | |
| CN108960413A (en) | A kind of depth convolutional neural networks method applied to screw surface defects detection | |
| CN115049853A (en) | Tobacco leaf curl invariant characteristic feature extraction method and storage medium | |
| CN117373016B (en) | Tobacco leaf baking state judging method, device, equipment and storage medium | |
| CN117437277B (en) | Fruit and vegetable dehydration liquid level detection method and device | |
| CN109492701A (en) | A kind of congou tea withers appropriateness method of discrimination and system | |
| CN113358647A (en) | Tobacco baking data prediction monitoring model establishing method | |
| CN112651173A (en) | Agricultural product quality nondestructive testing method based on cross-domain spectral information and generalizable system | |
| CN110503092A (en) | The improvement SSD monitor video object detection method adapted to based on field | |
| CN115661024A (en) | Cigar coat color distinguishing method and device, storage medium and server | |
| CN114913100A (en) | Tobacco leaf baking degree detection method based on image analysis | |
| CN109726641B (en) | Remote sensing image cyclic classification method based on automatic optimization of training samples |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |