CN117351472A - Tobacco leaf information detection method and device and electronic equipment - Google Patents
Tobacco leaf information detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN117351472A CN117351472A CN202311392960.2A CN202311392960A CN117351472A CN 117351472 A CN117351472 A CN 117351472A CN 202311392960 A CN202311392960 A CN 202311392960A CN 117351472 A CN117351472 A CN 117351472A
- Authority
- CN
- China
- Prior art keywords
- processed
- image
- tobacco
- target
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 235000002637 Nicotiana tabacum Nutrition 0.000 title claims abstract description 242
- 238000001514 detection method Methods 0.000 title claims abstract description 142
- 244000061176 Nicotiana tabacum Species 0.000 title description 8
- 241000208125 Nicotiana Species 0.000 claims abstract description 241
- 239000011159 matrix material Substances 0.000 claims abstract description 93
- 238000002372 labelling Methods 0.000 claims abstract description 36
- 238000012549 training Methods 0.000 claims description 47
- 238000000034 method Methods 0.000 claims description 42
- 230000006870 function Effects 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 24
- 241000238631 Hexapoda Species 0.000 claims description 22
- 238000005286 illumination Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 16
- 238000011176 pooling Methods 0.000 claims description 16
- 238000001914 filtration Methods 0.000 claims description 8
- 239000000463 material Substances 0.000 claims description 6
- 238000004519 manufacturing process Methods 0.000 abstract description 15
- 238000010586 diagram Methods 0.000 description 18
- 238000003860 storage Methods 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 241000037488 Coccoloba pubescens Species 0.000 description 1
- 241000500891 Insecta Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/758—Involving statistics of pixels or of feature values, e.g. histogram matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a tobacco leaf information detection method, a tobacco leaf information detection device and electronic equipment. The specific scheme is as follows: acquiring an image to be processed comprising tobacco leaves and/or cut tobacco; inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category; the target matrix comprises tobacco leaf pixel information corresponding to the classification category; labeling the image to be processed based on the target matrix to obtain a labeling area corresponding to the classification category; and determining target information of the tobacco leaves and/or the tobacco shreds based on the marked areas and the corresponding quantity information. According to the invention, the image to be processed is processed by utilizing the target detection model, and the image to be processed is marked by utilizing the target matrix, so that the target information of the tobacco leaves and/or the tobacco shreds is obtained, the tobacco leaves and/or the tobacco shreds can be rapidly detected and the corresponding target information can be judged, so that the production process of the tobacco can be adjusted, and the production quality of the tobacco leaves and/or the tobacco shreds can be improved.
Description
Technical Field
The invention relates to the technical field of tobacco preparation, in particular to a tobacco information detection method and device and electronic equipment.
Background
In the tobacco preparation process, tobacco leaf information is required to be acquired, and relevant indexes of tobacco leaves are judged, so that auxiliary and lifting effects can be achieved on the production process research of the tobacco leaves and the control of production equipment.
At present, in the tobacco production process, various indexes of tobacco leaves are generally distinguished by manually photographing the tobacco leaves, so that the aim of judging the indexes is fulfilled. Under the condition that the tobacco leaves to be detected are too much, the efficiency is often low, and the subjective understanding of detection personnel is different, so that the difference of tobacco leaf index judgment is caused, and the accuracy and objectivity of tobacco leaf information detection are lacked. In addition, the method often can not detect the tobacco leaf information in real time by means of a third-party tool, so that the detection of the tobacco leaf information is delayed.
Disclosure of Invention
The invention provides a tobacco information detection method, a device and electronic equipment, which are used for processing an image to be processed by utilizing a target detection model and labeling the image to be processed by utilizing a target matrix so as to obtain target information of tobacco leaves and/or tobacco shreds, and can be used for rapidly detecting the tobacco leaves and/or the tobacco shreds and judging the corresponding target information so as to adjust the production process of the tobacco leaves and/or the tobacco shreds and improve the production quality of the tobacco leaves and/or the tobacco shreds.
According to an aspect of the present invention, there is provided a tobacco leaf information detection method, including:
acquiring an image to be processed comprising tobacco leaves and/or cut tobacco;
inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category; the target matrix comprises tobacco leaf pixel information corresponding to the classification category;
labeling the image to be processed based on the target matrix to obtain a labeling area corresponding to the classification category;
and determining target information of the tobacco leaves and/or the tobacco shreds based on the marked areas and the corresponding quantity information.
According to another aspect of the present invention, there is provided a tobacco leaf information detecting apparatus including:
the image acquisition module to be processed is used for acquiring an image to be processed comprising tobacco leaves and/or tobacco shreds;
the target matrix and quantity information acquisition module is used for inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category; the target matrix comprises tobacco leaf pixel information corresponding to the classification category;
The marking area acquisition module is used for marking the image to be processed based on the target matrix to obtain a marking area corresponding to the classification category;
and the target information determining module is used for determining target information of the tobacco leaves and/or the tobacco shreds based on the marked areas and the corresponding quantity information.
According to another aspect of the present invention, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the tobacco information detection method of any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to execute a tobacco leaf information detection method according to any one of the embodiments of the present invention.
According to the technical scheme, the target detection model is utilized to process the image to be processed, the target matrix is utilized to label the image to be processed, and then the target information of the tobacco leaves and/or the tobacco shreds is obtained, so that the problem that the detection efficiency of the tobacco leaf information is low when the tobacco leaves are detected based on a manual mode, and the objectivity of the detection result is lacking is solved, the tobacco leaves and/or the tobacco shreds can be detected quickly and accurately, the corresponding target information is judged, the production process of the tobacco leaves is adjusted, and the production quality of the tobacco leaves and/or the tobacco shreds is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a tobacco leaf information detection method provided by an embodiment of the present invention;
fig. 2 is an exemplary view of a to-be-processed image of tobacco leaves provided by an embodiment of the present invention;
fig. 3 is a diagram illustrating an image to be processed of cut tobacco provided by an embodiment of the present invention;
fig. 4 is a diagram showing a structural example of a target detection algorithm provided in an embodiment of the present invention;
FIG. 5 is an exemplary diagram of a stitching process using a CSPNet network, according to an embodiment of the present invention;
FIG. 6 is an exemplary diagram of pooling with SPP-Net provided by an embodiment of the present invention;
Fig. 7 is an exemplary view of an image to be processed of a large piece of tobacco leaves provided by an embodiment of the present invention;
FIG. 8 is an exemplary diagram of tobacco pixel information and a marking mode corresponding to an image to be processed of a large piece of tobacco provided by an embodiment of the present invention;
fig. 9 is an exemplary diagram of tobacco pixel information corresponding to a to-be-processed image of a large piece of tobacco provided by an embodiment of the present invention after a marking mode is processed by DropBlock regularization;
fig. 10 is an exemplary diagram of a region marked by selecting a paper sheet frame in an image to be processed of tobacco leaves according to an embodiment of the present invention;
fig. 11 is an exemplary diagram of a to-be-processed image of tobacco leaves after selecting and marking areas for green leaves, black leaves and insect spots;
fig. 12 is an exemplary diagram of a region marked by selecting an insect spot frame in an image to be processed of tobacco leaves according to an embodiment of the present invention;
fig. 13 is an exemplary diagram of a region marked by selecting an insect spot frame in an image to be processed of tobacco leaves according to an embodiment of the present invention;
fig. 14 is an exemplary diagram of a large-sized tobacco leaf frame in an image to be processed of cut tobacco provided by an embodiment of the present invention after marking an area;
fig. 15 is an exemplary diagram of a large-sized tobacco leaf frame in an image to be processed of cut tobacco provided by an embodiment of the present invention after marking an area;
FIG. 16 is a flow chart of a method for obtaining a target detection model provided by an embodiment of the present invention;
fig. 17 is a schematic structural diagram of a tobacco leaf information detecting device according to an embodiment of the present invention;
fig. 18 is a schematic structural diagram of an electronic device for implementing the tobacco leaf information detection method according to the embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a tobacco information detection method provided by the embodiment of the present invention, where the present embodiment is applicable to detecting tobacco and/or cut tobacco in a tobacco production process, and determining whether the tobacco and/or cut tobacco meets a certain condition, the method may be performed by a tobacco information detection device, where the tobacco information detection device may be implemented in the form of hardware and/or software, and the hardware may be an electronic device, such as a mobile terminal, a PC end, or a server. As shown in fig. 1, the method includes:
s110, acquiring an image to be processed comprising tobacco leaves and/or cut tobacco.
In the embodiment of the invention, the image to be processed can be an image acquired by a corresponding acquisition device in the tobacco leaf and/or tobacco shred processing process. When the acquisition device is used for acquiring the images of the tobacco leaves and/or the tobacco shreds, a plurality of images to be processed can be acquired, the subsequent processing modes of each image to be processed are the same, and in the embodiment, one acquired image of the tobacco leaves and/or the tobacco shreds is used as the image to be processed, namely, the current acquired image is used as the image to be processed.
Specifically, the tobacco leaves and/or the tobacco shreds are subjected to image acquisition through the corresponding image acquisition device, and corresponding images to be processed are obtained. When the image acquisition is performed on the tobacco leaves and/or the tobacco shreds, the real-time continuous image acquisition can be performed in an online mode, the purpose of detecting the information of the tobacco leaves and/or the tobacco shreds in real time is achieved, the centralized acquisition of the images can be performed at a certain time interval according to actual requirements, the acquired images to be processed are stored in corresponding storage equipment, and the embodiment is not limited to the above. Whether a real-time acquisition mode or a centralized acquisition mode is adopted, each image to be processed is analyzed and processed, the subsequent analysis and processing modes of each image to be processed are the same, and for convenience of description, the image to be processed is taken as the image to be processed in the follow-up process. For example, the currently collected image to be processed of tobacco leaf may be as shown in fig. 2, and the currently collected image to be processed of tobacco leaf may be as shown in fig. 3.
Optionally, acquiring the image to be processed including tobacco leaf and/or tobacco shred includes: deploying the image acquisition device at a first preset position and deploying the illumination device at a second preset position; the first preset position is determined based on the camera shooting range of the image acquisition device and the position information of the tobacco leaf conveying belt and/or tobacco shred conveying belt, and the second preset position is determined based on the illumination range of the illumination device and the position information of the tobacco leaf conveying belt and/or tobacco shred conveying belt; during the operation of the lighting device and the conveyor belt, the image to be processed comprising tobacco leaves and/or tobacco shreds is acquired based on the image acquisition device.
In the embodiment of the present invention, the first preset position may be a position for deploying the image capturing device, which is set according to actual requirements. The second preset position is a position for deploying the lighting device, which is set according to actual requirements. The image acquisition device is a device which is arranged according to actual requirements and is used for acquiring images of tobacco leaves and/or tobacco shreds, for example, the image acquisition device can be a digital CCD camera with a shooting area of 100cm x 100cm or other devices which can be used for acquiring images. In the image acquisition device, the shooting range is determined by various factors such as the focal length of a lens, the size of an aperture and the like of the device, and the corresponding shooting range can be artificially set according to actual requirements. The lighting device is a device which is actually required to illuminate tobacco leaves and/or tobacco shreds so that the light source condition meets the image acquisition requirement, for example, the lighting device can adopt a stable and high-brightness uniform LED light source with the color temperature of 5600K. The illumination range is related to the illumination device, and the illumination range corresponding to the illumination device is different from the illumination device.
Specifically, the image acquisition device is arranged according to the selected image acquisition device, a proper position is selected as a first preset position based on the image acquisition range and the position information of the tobacco leaf and/or tobacco shred conveying belt, and the image acquisition device is deployed according to the first preset position, so that the image acquisition device can be used for acquiring the image to be processed, which meets the requirements and comprises the tobacco leaf and/or tobacco shred. And determining a corresponding illumination range according to the selected illumination device, determining a second preset position based on the illumination range and the position information of the transported tobacco leaves and/or tobacco shred conveyor belts, and then disposing the illumination device at the second preset position so that the illumination device disposed at the position can be used for providing a proper light source for the image acquisition device during image acquisition. And then, when the tobacco leaf and/or tobacco shred conveying belt works, the lighting device provides a proper light source, and at the same time, the image acquisition device acquires images of tobacco leaves and/or tobacco shreds on the conveying belt to acquire corresponding images to be processed.
S120, inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category.
The target matrix comprises tobacco leaf pixel information corresponding to the classification category.
In the embodiment of the invention, the pre-trained target detection model is a model for analyzing an image to be processed based on a deep learning algorithm and determining a corresponding target matrix and quantity information. Alternatively, the image to be processed may be analyzed using a two-stage detection algorithm. It should be noted that, because the structures corresponding to different target detection algorithms are different, as shown in fig. 4, the two-stage detection algorithm is selected to have an input portion, a backbone network portion for extracting features, a feature extraction network portion for transmitting to the detection network, and a detection head portion responsible for detection, where the two-stage detection algorithm generally further includes a sparse prediction portion for spatial prediction. Therefore, an appropriate detection algorithm may be selected according to the actual requirement to analyze the image to be processed, which is only illustrated herein, and the embodiment is not limited thereto.
The target matrix can be understood as a position matrix corresponding to target information to be detected in the image to be processed. The quantity information can be understood as quantity information corresponding to each classification category in the image to be processed. The tobacco leaf pixel information may be pixel point information corresponding to a color or a pixel value obtained when each pixel point of an image to be processed of tobacco leaves is analyzed and processed. The classification category can be to divide the features to be detected in the image to be processed into different categories according to actual needs. For example, in the image to be processed of tobacco leaves and/or cut tobacco, features such as green leaves, material spots, insect spots, leaf stems and the like can be classified, and each feature serves as a classification category. In addition, some other classification categories can be set according to actual demands, for example, corresponding information of the large tobacco leaves is detected in the to-be-processed image of the tobacco shreds, and the large tobacco leaves can be used as one classification category. And detecting corresponding information of the paper sheets in the image to be processed of the cut tobacco, wherein the paper sheets can be used as a classification category. Specific requirements set this embodiment is not limited thereto, the foregoing is merely illustrative.
Specifically, a proper target detection algorithm is selected according to actual needs, and a target detection model is trained in advance based on the target detection algorithm to obtain the target detection model meeting the actual needs. And inputting the acquired image to be processed into a target detection model, and respectively outputting a target matrix and a number of information corresponding to the image to be processed according to different classification categories. The target matrix comprises tobacco leaf pixel information corresponding to the classification categories, so that the positions corresponding to each classification category can be determined by using the pixel information of the tobacco leaves.
Optionally, the classification category includes at least one of a first category identifier corresponding to a green leaf category, a second category identifier corresponding to a material spot category, a third category identifier corresponding to an insect spot category, and a fourth category identifier corresponding to a stem category, and the target matrix includes a plurality of element values, where the element values correspond to tobacco leaf pixel information of an area corresponding to the respective classification category in the image to be processed.
In the embodiment of the invention, the first category identifier can be understood as identification information corresponding to the green leaf category, and the second category identifier is identification information corresponding to the material spot category. The third category identification is identification information corresponding to the insect spot category, and the fourth category identification is identification information corresponding to the stem category. The identification information may be an identifier or other specific labels, which is not limited in this embodiment, and the number information corresponding to different classification categories in the image to be processed may be determined according to different category identifications. For example, the number of cyan leaves in the image to be processed may be determined according to the first category identification. Element values may be understood as values used in the target matrix to represent tobacco pixel information in different classification categories. For example, each element value in the target matrix may be used to represent whether certain tobacco pixel information in the image to be processed belongs to a certain classification category.
Specifically, when the image to be processed is analyzed, a classification category can be selected according to actual requirements, and the classification category can be one or more of a first category identifier corresponding to a green leaf category, a second category identifier corresponding to a material spot category, a third category identifier corresponding to an insect spot category and a fourth category identifier corresponding to a stem category. In addition, the target matrix contains a plurality of element values, and each element value can be used for representing tobacco leaf pixel information in a region corresponding to each classification category in the image to be processed.
Optionally, inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category, including: filtering the to-be-processed features corresponding to the to-be-processed images sequentially based on at least one filter in the target detection model to obtain to-be-spliced features corresponding to each filter; splicing the image to be processed according to the characteristics to be processed corresponding to the image to be processed and the corresponding characteristics to be spliced to obtain splicing characteristics; pooling the spliced features to obtain a fixed number of target features; and (3) carrying out target feature prediction processing, and determining a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category.
In the embodiment of the invention, the filter is used for carrying out feature filtering on the features to be processed corresponding to the images to be processed according to the preset filtering rules and algorithms so as to process the images by using the corresponding features later. The feature to be spliced can be understood as a feature obtained by processing the feature to be processed of the image to be processed through the filter, wherein each filter has the corresponding feature to be spliced after filtering the feature to be processed, and the features to be spliced corresponding to different filters are different, so that a plurality of features to be spliced can be obtained by utilizing at least one filter, and the subsequent feature analysis is more accurate. The splicing characteristic can be understood as a splicing characteristic obtained after the characteristic to be processed and the corresponding characteristic to be spliced are spliced in turn. The target features are features obtained after pooling the splice features.
Specifically, feature extraction is performed on an image to be processed to obtain a feature to be processed corresponding to the image to be processed, next, filtering processing is performed on the feature to be processed by using a filter in a target detection model to sequentially obtain a corresponding feature to be spliced, after the feature to be spliced is obtained, splicing processing is performed on the feature to be processed and the corresponding feature to be spliced sequentially, so as to obtain a spliced feature, pooling processing is performed on the spliced feature, so that spliced feature images with different sizes are pooled in a fixed size to obtain a fixed number of target features, and then prediction processing is performed on the target features to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to different classification categories.
The method comprises the steps of firstly extracting features of an image to be processed to obtain the features to be processed corresponding to the image to be processed, then filtering the features to be processed by using a filter in a target detection model to sequentially obtain the corresponding features to be spliced, then solving the problem of repeated gradient information of network optimization in a main network of other large convolutional neural network frames by using a CSPNet network (Cross Stage Partial Networks, cross-stage local network), and integrating the change of the gradient into a spliced feature map from beginning to end, so that the parameter number and floating point operation value of the model are reduced, the reasoning speed and accuracy are ensured, and the model size is reduced. As shown in fig. 5, fig. 5 is an exemplary diagram of a process of splicing a feature to be processed and a feature to be spliced sequentially using a CSPNet network. The rectangle at the lower left corner in the dotted line frame of the Dense Layer 1 (Dense Layer 1) represents the feature to be processed, the feature to be processed is processed by a filter to obtain the rectangle at the upper right side, the rectangle herein represents the feature to be spliced, the rectangle at the lower right side represents the feature obtained after the feature to be processed and the feature to be spliced are spliced, the subsequent processing is similar, and details are omitted herein, and the spliced feature, namely the rectangle at the rightmost side in the figure, can be obtained after the splicing processing is completed. Wherein, the CSPNet network is actually based on the ideas of DenseNet (Dense Convolutional Network ), copies the feature map of the base layer, and sends copies to the next stage through dense blocks, so that the feature map of the base layer is separated, the gradient vanishing problem can be effectively relieved, the feature propagation is supported, the network reuse feature is encouraged, and the number of network parameters is reduced. The CSPNet concept can be combined with ResNet (Residual Network), resNeXt (Residual Network Next, residual Network) and DenseNet, and two modified backbone networks, namely CSPResNext50 and CSPDarknet53, are mainly adopted at present. And then, pooling the spliced features by using SPP-Net (Spatial Pyramid Pooling Networks, a spatial pyramid pooling network), and directly pooling the spliced feature images with any size in a fixed size to obtain a fixed number of target features. As shown in fig. 6. Taking pooling of 3 sizes as an example, performing maximum pooling on the spliced feature images of any size through a pyramid pooling layer in fig. 6, namely taking the maximum value of one feature image to obtain 1*d (d is the dimension of the feature image); dividing the spliced feature map into grids of 2x2, and carrying out maximum pooling on each grid to obtain 4*d features; and similarly, dividing the spliced feature map into 4x4 grids, and carrying out maximum value pooling on each grid to obtain 16 x d features. And then combining the features obtained by each pooling to obtain the number of features with fixed length (the dimension of the feature map is fixed), namely a fixed number of target features, namely output in the map. And then, carrying out prediction processing on the target characteristics to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to different classification categories.
S130, labeling the image to be processed based on the target matrix to obtain a labeling area corresponding to the classification category.
In the embodiment of the invention, the marking area can be understood as an area marked by the element value of the target matrix.
Specifically, the image to be processed is judged according to the element values in the target matrix, if the pixel information is the tobacco leaf pixel information in the region corresponding to the corresponding classification category, the tobacco leaf pixel information in the region is marked, the marking region of the classification category is obtained, and the marking region corresponding to each classification category is obtained by judging the tobacco leaf pixel information in the region corresponding to each classification category.
Optionally, labeling the image to be processed based on the target matrix to obtain a labeled region corresponding to the classification category, including: determining position information corresponding to the corresponding classification category based on a plurality of element values in the target matrix; and carrying out frame selection marking on each classification category according to the marking mode corresponding to each classification category and the corresponding position information to obtain a plurality of marking areas.
In the embodiment of the present invention, the location information may be understood as the location of the region corresponding to each classification category in the image to be processed. The marking mode is a category identification mode corresponding to different categories in the classification categories.
Specifically, the element value in the target matrix corresponds to the tobacco leaf pixel information in the region corresponding to each classification category in the image to be processed, and the position information of the corresponding classification category in the image to be processed can be determined according to the element value corresponding to the tobacco leaf pixel information. And determining the corresponding classification category in the image to be processed according to the marking modes corresponding to the different classification categories, and simultaneously combining corresponding position information, and framing the marking area corresponding to each classification category in the image to be processed to obtain a plurality of marking areas.
As shown in fig. 7 to 11, fig. 7 is an image to be processed of a large piece of tobacco, and a hatched portion in fig. 8 is tobacco pixel information corresponding to the image to be processed of the large piece of tobacco, wherein "x" indicates a corresponding marking mode of the large piece of tobacco. Fig. 9 and fig. 8 are the same, but fig. 9 shows that the dropoblock regularization method is used to determine the position information corresponding to the classification category, namely the shaded portion in the graph, by using the tobacco leaf pixel information corresponding to the element value in the target matrix, and then, according to the set marking mode "x" of the large-piece tobacco leaf and the corresponding position information, the marking mode of the dropoblock regularization method is used to mark the adjacent areas of the large-piece tobacco leaf, so as to reduce complexity and avoid overfitting. And then, according to the marks of the blocky adjacent areas, the frame selection marks of the large tobacco leaves in the to-be-processed image of the tobacco shreds can be realized, and the marked areas are obtained, wherein in the to-be-processed image of the tobacco shreds, the leaves with the width exceeding 4 times of the tobacco shred width can be defined as the large tobacco leaves, and the frame selection marks can be realized by Labelme image marking software. In addition, the paper sheets in the image to be processed of the tobacco leaves can be marked as a classification category as shown in fig. 10, and the green leaves, the black leaves and the insect spots in the image to be processed of the tobacco leaves can be marked as classification categories as shown in fig. 11.
And S140, determining target information of tobacco leaves and/or tobacco shreds based on the marked areas and the corresponding quantity information.
In the embodiment of the invention, the target information can be understood as test result information obtained after analyzing the marked area and the number information corresponding to each classification category of the image to be processed of the tobacco leaf and/or the tobacco shred. For example, if the classification class is insect spots, the target information may include information about the number of early warning insect spots, the number of actual insect spots in the image to be processed, the false detection rate, the accuracy rate, the recall rate, and the like.
Specifically, target information of tobacco leaves and/or tobacco shreds is determined according to the marking areas and the quantity information corresponding to each classification category in the image to be processed.
Illustratively, in the image to be processed of the tobacco leaf, the Labelme image labeling software is utilized to carry out frame selection labeling on the classification category of the insect spots. Labeling the to-be-processed images of 20 tobacco leaves, wherein the labeled to-be-processed images can be shown in fig. 12 to 13. The target information obtained is shown in table 1 below.
TABLE 1 target information table for insect plaque detection
| Number of pictures | Number of early warning | True insect spot | False detection rate | Accuracy rate of |
| 20 | 502 | 485 | 3.4％ | 96.6％ |
According to the table, the number of pictures is 20, the early warning number of insect spots obtained based on the embodiment mode is 502, the number of real insect spots in the pictures is 485, the false detection rate obtained by calculation is 3.4%, and the accuracy is 96.6%.
Illustratively, in the image to be processed of the cut tobacco, the Labelme image labeling software is utilized to carry out frame selection marking on the large tobacco leaves as classification categories. The image to be processed of 160 cut tobacco is marked, wherein the marked image to be processed can be shown in fig. 14 to 15. The target information obtained is shown in table 2 below.
TABLE 2 target information for large leaf tobacco detection
| Number of pictures | Number of early warning | Real large sheet | False detection rate | Accuracy rate of |
| 160 | 12523 | 11395 | 9％ | 91.0％ |
According to the table, the number of pictures is 160, the early warning number of the large tobacco obtained based on the embodiment mode is 12523, the number of the real large tobacco in the pictures is 11395, the false detection rate obtained by calculation is 9%, and the accuracy rate is 91.0%.
According to the technical scheme, the target detection model is utilized to process the image to be processed, the target matrix is utilized to label the image to be processed, and then the target information of the tobacco leaves and/or the tobacco shreds is obtained, so that the problem that the detection efficiency of the tobacco leaf information is low when the tobacco leaves are detected based on a manual mode, and the objectivity of the detection result is lacking is solved, the tobacco leaves and/or the tobacco shreds can be detected quickly and accurately, the corresponding target information is judged, the production process of the tobacco leaves is adjusted, and the production quality of the tobacco leaves and/or the tobacco shreds is improved.
Example two
Fig. 16 is a flowchart of a method for obtaining a target detection model according to an embodiment of the present invention, where before determining target information of tobacco leaves and/or cut tobacco, the target detection model is further trained in advance, and then the solution according to the first embodiment is executed on the basis of obtaining the target detection model. As shown in fig. 16, the method includes:
s210, acquiring a plurality of training samples; the training sample comprises a sample image, the sample image comprises a plurality of labeling areas, the labeling areas correspond to at least one classification category, an area matrix corresponds to each labeling area, and the theoretical quantity corresponds to each classification category.
In this embodiment, the sample image may be understood as an image to be processed acquired by the corresponding acquisition device during the processing of tobacco leaves and/or cut tobacco. The labeling area can be understood as an area corresponding to the classification category in the determined sample image, and the labeling area can be obtained by manual labeling or by labeling by adopting a corresponding algorithm. Each classification category has a corresponding labeling area, and each labeling area corresponds to an area matrix, i.e. the area matrix is a position matrix for determining the labeling area. For example, the classification category is insect spots, and the area matrix corresponding to the labeling area of the insect spot can be a position matrix obtained by using the pixel point coordinates of the insect spot. The theoretical number may be understood as the actual number in each classification category. For example, in the image to be processed of the tobacco leaf, the actual number of insect spots is 20, and the classification category is that the theoretical number corresponding to the insect spots is 20.
Specifically, before training the target detection model, a plurality of training samples need to be acquired to train the model based on the training samples. In order to improve the accuracy of the model, as many training samples as possible, i.e. a plurality of sample images including the labeling area, need to be acquired. The labeling area corresponding to each classification category needs to be determined in the sample image, so that a corresponding area matrix is obtained based on the labeling areas. Correspondingly, the theoretical quantity corresponding to each classification category can be calculated and obtained according to the labeling area.
S220, obtaining an output matrix and the output quantity corresponding to each classification category from the detection model to be trained based on the plurality of training samples.
It should be noted that, for each training sample, the training may be performed in the manner of S220, so as to obtain the target detection model. The model parameters in the detection model to be trained are default values, and the model parameters to be trained are corrected through the training samples so as to obtain the target detection model.
In the embodiment of the invention, the output matrix can be understood as a position matrix of the training sample after being trained by the detection model to be trained. The output number can be understood as the number of each classification category in the training sample output after the training sample is trained by the detection model to be trained.
Specifically, the detection model to be trained is trained by using a plurality of training samples, and the prediction output of the detection model to be trained on each training sample is calculated, so that an output matrix corresponding to the training sample and the output quantity corresponding to each classification category in the training sample are obtained, and the detection model to be trained is subjected to adjustment and participation correction by using the output matrix and the output quantity.
And S230, determining a loss value based on the area matrix, the theoretical number, the output matrix and the output number of each training sample, so as to correct model parameters in the detection model to be trained based on the loss value.
In the embodiment of the present invention, the loss value may be understood as a difference value between the area matrix and the output matrix, and between the theoretical number and the output number.
Specifically, the output matrix and the area matrix of each training sample, and the output quantity and the theoretical quantity are subjected to loss processing according to a loss function in the detection model to be trained, so that a loss value is obtained. Wherein the loss function may be a function determined from the loss value for representing the degree of difference between the actual output and the theoretical output. The loss function may be any loss function, and optionally, may be at least one of an average binary cross entropy loss function, a cross entropy error loss function, a smooth L1 loss function, and the like. Then, the model parameters in the to-be-trained detection model are corrected according to the loss value, in general, the model parameters of the to-be-trained detection model are initial parameters or default parameters, and when the to-be-trained detection model is trained, each model parameter in the model can be corrected based on the output result of the to-be-trained detection model, that is, the loss value of the to-be-trained detection model can be corrected, so that the target detection model can be obtained.
It should be noted that, training of the model is often an iterative training process. The detection model to be trained is trained by a plurality of training samples, and model parameters of the detection model to be trained are continuously adjusted in the iterative process to obtain the target detection model.
S240, converging a loss function in the detection model to be trained as a training target to obtain a target detection model.
Specifically, the loss function is converged to be a training target, such as whether the training error is smaller than a preset error, whether the error change tends to be stable, or whether the current iteration number is equal to the preset number. If the detection reaches the convergence condition, for example, the training error of the loss function is smaller than the preset error, or the error change trend tends to be stable, which indicates that the training of the detection model to be trained is completed, and at the moment, the iterative training can be stopped. If the current condition is detected not to be met, other training samples can be further obtained to train the detection model to be trained until the training error of the loss function is within a preset range. When the training error of the loss function reaches convergence, the training-completed detection model to be trained can be used as a target detection model, namely, the target information corresponding to the image to be processed can be accurately obtained after the image to be processed is input into the target detection model.
Optionally, the loss function is:
wherein IoU characterizes the ratio of the intersection and union of two graphical areas, ρ 2 (b,b gt ) Representing the Euclidean distance of the center points of two rectangular frames, b representing the predicted rectangular frame, b gt Representing an actual rectangular box, c representing the diagonal length of two bounding rectangles, alpha representing a weight function, v representing the similarity of the measured aspect ratios,
wherein,
in the above formula, w represents the width of the rectangle, h represents the length of the rectangle, and w gt Represents the width of a real target rectangular frame, h gt Representing the length of the real target rectangular box.
Specifically, the above-mentioned loss function, namely CIoU (Complete Intersection over Union) loss function, can be utilized, wherein the CIoU loss function increases the consideration of the dimension of the detection frame and the loss calculation of the length and width of the prediction frame based on DIoU (Distance Intersection over Union) loss function, so that the prediction frame can better conform to the shape and size of the real frame, and the DIoU loss function is as follows.
Optionally, if the target detection model corresponds to one classification category, training a plurality of target detection models corresponding to different classification categories to process the image to be processed based on each target detection model, so as to obtain target information corresponding to the corresponding classification category.
Specifically, when the detection model to be trained is trained, the target detection model value obtained by each training is aimed at one classification category, so that in order to conveniently analyze each classification category in the tobacco leaf and/or tobacco shred to be processed, a plurality of target detection models need to be trained, and therefore the image to be processed is processed by utilizing a plurality of target detection models corresponding to the classification categories, and target information corresponding to each classification category in the image to be processed is obtained.
According to the technical scheme, the detection model to be trained is trained to obtain an output matrix and output quantity, a loss value is calculated based on the output matrix and the area matrix, the output quantity and the theoretical quantity, and then the detection model is corrected based on the loss value to obtain the target detection model. By training a plurality of target detection models corresponding to different classification categories, accurate acquisition of target information corresponding to each classification category in the image to be processed can be realized, so that the target detection models are used for accurately and completely processing the image to be processed to obtain the target information, and subsequent adjustment and control of the tobacco leaf and/or tobacco shred production process based on the target information are facilitated.
Example III
Fig. 17 is a schematic structural diagram of a tobacco leaf information detecting device according to an embodiment of the present invention. As shown in fig. 17, the apparatus includes: the image to be processed acquisition module 310, the target matrix and quantity information acquisition module 320, the marked area acquisition module 330 and the target information determination module 340.
A to-be-processed image acquisition module 310, configured to acquire to-be-processed images including tobacco leaves and/or cut tobacco;
the target matrix and quantity information obtaining module 320 is configured to input an image to be processed into a pre-trained target detection model, so as to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to a classification class; the target matrix comprises tobacco leaf pixel information corresponding to the classification category;
the marking area obtaining module 330 is configured to mark the image to be processed based on the target matrix, and obtain a marking area corresponding to the classification category;
the target information determining module 340 is configured to determine target information of tobacco leaves and/or tobacco shreds based on the marked area and the corresponding quantity information.
According to the technical scheme, the target detection model is utilized to process the image to be processed, the target matrix is utilized to label the image to be processed, and then the target information of the tobacco leaves and/or the tobacco shreds is obtained, so that the problem that the detection efficiency of the tobacco leaf information is low when the tobacco leaves are detected based on a manual mode, and the objectivity of the detection result is lacking is solved, the tobacco leaves and/or the tobacco shreds can be detected quickly and accurately, the corresponding target information is judged, the production process of the tobacco leaves is adjusted, and the production quality of the tobacco leaves and/or the tobacco shreds is improved.
On the basis of the above embodiment, optionally, the image acquisition module to be processed includes: the illumination device deployment unit is used for deploying the image acquisition device at a first preset position and deploying the illumination device at a second preset position; the first preset position is determined based on the camera shooting range of the image acquisition device and the position information of the tobacco leaf conveying belt and/or tobacco shred conveying belt, and the second preset position is determined based on the illumination range of the illumination device and the position information of the tobacco leaf conveying belt and/or tobacco shred conveying belt; the image acquisition unit to be processed is used for acquiring the image to be processed comprising tobacco leaves and/or tobacco shreds based on the image acquisition device in the working process of the lighting device and the conveyor belt.
Optionally, the target matrix and quantity information acquisition module includes: the to-be-spliced characteristic determining unit is used for filtering the to-be-processed characteristics corresponding to the to-be-processed images sequentially based on at least one filter in the target detection model to obtain to-be-spliced characteristics corresponding to each filter; the splicing characteristic determining unit is used for sequentially carrying out splicing processing on the basis of the to-be-processed characteristics corresponding to the to-be-processed images and the corresponding to-be-spliced characteristics to obtain splicing characteristics; the target feature determining unit is used for pooling the spliced features to obtain a fixed number of target features; and the target matrix and quantity information determining unit is used for predicting the target characteristics and determining quantity information corresponding to the target matrix and the classification category corresponding to the image to be processed.
Optionally, in the target matrix and quantity information obtaining module, the classification category includes at least one of a first category identifier corresponding to a green leaf category, a second category identifier corresponding to a material spot category, a third category identifier corresponding to an insect spot category, and a fourth category identifier corresponding to a stem category, the target matrix includes a plurality of element values, and the element values correspond to tobacco leaf pixel information of a region corresponding to the respective classification category in the image to be processed.
Optionally, the marking area obtaining module includes: the position information determining unit is used for determining position information corresponding to the corresponding classification category based on a plurality of element values in the target matrix; and the marking area acquisition unit performs frame selection marking on each classification category according to the marking mode corresponding to each classification category and corresponding position information to obtain a plurality of marking areas.
Optionally, the apparatus further includes an object detection model acquisition module, which includes: the training sample acquisition unit is used for acquiring a plurality of training samples; the training sample comprises a sample image, wherein the sample image comprises a plurality of labeling areas, the labeling areas correspond to at least one classification category, an area matrix corresponds to each labeling area, and theoretical quantity corresponds to each classification category; the output matrix and output quantity acquisition unit is used for obtaining an output matrix and output quantity corresponding to each classification category in the detection model to be trained based on a plurality of training samples; the loss value determining unit is used for determining a loss value based on the area matrix, the theoretical quantity, the output matrix and the output quantity of each training sample so as to correct model parameters in the detection model to be trained based on the loss value; and the target detection model determining unit is used for converging a loss function in the detection model to be trained as a training target to obtain a target detection model.
Optionally, in the target detection model acquisition module, the loss function is:
wherein IoU characterizes the ratio of the intersection and union of two graphical areas, ρ 2 (b,b gt ) Representing the Euclidean distance of the center points of two rectangular frames, b representing the predicted rectangular frame, b gt Representing an actual rectangular box, c representing the diagonal length of two bounding rectangles, alpha representing a weight function, v representing the similarity of the measured aspect ratios,
wherein,
the above formulaWherein w represents the width of the rectangle, h represents the length of the rectangle, and w gt Represents the width of a real target rectangular frame, h gt Representing the length of the real target rectangular box.
Optionally, the target detection model obtaining module is further configured to train a plurality of target detection models corresponding to different classification categories if the target detection model corresponds to one classification category, so as to process the image to be processed based on each target detection model, and obtain target information corresponding to the corresponding classification category.
The tobacco information detection device provided by the embodiment of the invention can execute the tobacco information detection method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 18 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 18, the electronic device 10 includes at least one processor 11, and a memory such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc. communicatively connected to the at least one processor 11, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the respective methods and processes described above, such as the tobacco information detection method.
In some embodiments, the tobacco information detection method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the tobacco information detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the tobacco information detection method in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
The computer program for implementing the tobacco information detection method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
Example five
The fifth embodiment of the present invention also provides a computer readable storage medium, where computer instructions are stored, where the computer instructions are configured to cause a processor to execute a tobacco leaf information detection method, and the method includes:
acquiring an image to be processed comprising tobacco leaves and/or cut tobacco; inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category; the target matrix comprises tobacco leaf pixel information corresponding to the classification category; labeling the image to be processed based on the target matrix to obtain a labeling area corresponding to the classification category; and determining target information of the tobacco leaves and/or the tobacco shreds based on the marked areas and the corresponding quantity information.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A tobacco information detection method, characterized by comprising:
acquiring an image to be processed comprising tobacco leaves and/or cut tobacco;
inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to classification categories; wherein the target matrix comprises tobacco leaf pixel information corresponding to the classification category;
labeling the image to be processed based on the target matrix to obtain a labeling area corresponding to the classification category;
and determining target information of the tobacco leaves and/or tobacco shreds based on the marked areas and the corresponding quantity information.
2. The method according to claim 1, wherein the acquiring the image to be processed comprising tobacco leaves and/or cut tobacco comprises:
deploying the image acquisition device at a first preset position and deploying the illumination device at a second preset position; the first preset position is determined based on the camera shooting range of the image acquisition device and the position information of the tobacco leaf and/or tobacco shred conveying belt, and the second preset position is determined based on the illumination range of the illumination device and the position information of the tobacco leaf and/or tobacco shred conveying belt;
And in the working process of the lighting device and the conveyor belt, acquiring an image to be processed comprising tobacco leaves and/or tobacco shreds based on the image acquisition device.
3. The method according to claim 1, wherein inputting the image to be processed into a pre-trained object detection model to obtain the number information corresponding to the object matrix and the classification category corresponding to the image to be processed, includes:
filtering the to-be-processed features corresponding to the to-be-processed image sequentially based on at least one filter in the target detection model to obtain to-be-spliced features corresponding to each filter;
based on the to-be-processed characteristics corresponding to the to-be-processed images and corresponding to-be-spliced characteristics, splicing the to-be-processed characteristics sequentially to obtain spliced characteristics;
pooling the spliced features to obtain a fixed number of target features;
and carrying out prediction processing on the target characteristics, and determining a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category.
4. A method according to claim 1 or 3, wherein the classification category comprises at least one of a first category identification corresponding to a green leaf category, a second category identification corresponding to a material spot category, a third category identification corresponding to a insect spot category, and a fourth category identification corresponding to a stem category, and the target matrix comprises a plurality of element values corresponding to tobacco leaf pixel information of a region corresponding to the respective classification category in the image to be processed.
5. The method according to claim 1, wherein the labeling the image to be processed based on the target matrix to obtain a labeled region corresponding to the classification category includes:
determining position information corresponding to the corresponding classification category based on a plurality of element values in the target matrix;
and carrying out frame selection marking on each classification category according to the marking mode corresponding to each classification category and the corresponding position information to obtain a plurality of marking areas.
6. The method as recited in claim 1, further comprising:
training to obtain a target detection model;
the training to obtain the target detection model includes:
acquiring a plurality of training samples; the training sample comprises a sample image, wherein the sample image comprises a plurality of labeling areas, the labeling areas correspond to at least one classification category, an area matrix corresponds to each labeling area, and the theoretical number corresponds to each classification category;
obtaining an output matrix and the output quantity corresponding to each classification category in the detection model to be trained based on the training samples;
determining a loss value based on the area matrix, the theoretical number, the output matrix and the output number of each training sample, so as to correct model parameters in the detection model to be trained based on the loss value;
And converging the loss function in the detection model to be trained as a training target to obtain the target detection model.
7. The method of claim 6, wherein the loss function is:
wherein IoU characterizes the ratio of the intersection and union of two graphical areas, ρ 2 (b,b gt ) Representing the Euclidean distance of the center points of two rectangular frames, b representing the predicted rectangular frame, b gt Representing an actual rectangular box, c representing the diagonal length of two bounding rectangles, alpha representing a weight function, v representing the similarity of the measured aspect ratios,
wherein,
in the above formula, w represents the width of the rectangle, h represents the length of the rectangle, and w gt Represents the width of a real target rectangular frame, h gt Representing the length of the real target rectangular box.
8. The method according to claim 1, wherein the object detection model corresponds to one classification category, and a plurality of object detection models corresponding to different classification categories are trained to process the image to be processed based on each object detection model, so as to obtain object information corresponding to the respective classification category.
9. A tobacco leaf information detection apparatus, characterized by comprising:
the image acquisition module to be processed is used for acquiring an image to be processed comprising tobacco leaves and/or tobacco shreds;
The target matrix and quantity information acquisition module is used for inputting the image to be processed into a pre-trained target detection model to obtain a target matrix corresponding to the image to be processed and quantity information corresponding to the classification category; wherein the target matrix comprises tobacco leaf pixel information corresponding to the classification category;
the marking area acquisition module is used for marking the image to be processed based on the target matrix to obtain a marking area corresponding to the classification category;
and the target information determining module is used for determining the target information of the tobacco leaves and/or the tobacco shreds based on the marked areas and the corresponding quantity information.
10. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the tobacco information detection method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311392960.2A CN117351472A (en) | 2023-10-25 | 2023-10-25 | Tobacco leaf information detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311392960.2A CN117351472A (en) | 2023-10-25 | 2023-10-25 | Tobacco leaf information detection method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117351472A true CN117351472A (en) | 2024-01-05 |
Family
ID=89369005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311392960.2A Pending CN117351472A (en) | 2023-10-25 | 2023-10-25 | Tobacco leaf information detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117351472A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117689660A (en) * | 2024-02-02 | 2024-03-12 | 杭州百子尖科技股份有限公司 | Vacuum cup temperature quality inspection method based on machine vision |
-
2023
- 2023-10-25 CN CN202311392960.2A patent/CN117351472A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117689660A (en) * | 2024-02-02 | 2024-03-12 | 杭州百子尖科技股份有限公司 | Vacuum cup temperature quality inspection method based on machine vision |
| CN117689660B (en) * | 2024-02-02 | 2024-05-14 | 杭州百子尖科技股份有限公司 | Vacuum cup temperature quality inspection method based on machine vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109613002B (en) | Glass defect detection method and device and storage medium | |
| CN111815564B (en) | Method and device for detecting silk ingots and silk ingot sorting system | |
| CN109671058B (en) | Defect detection method and system for large-resolution image | |
| CN112149543B (en) | Building dust recognition system and method based on computer vision | |
| CN115131283B (en) | Defect detection and model training method, device, equipment and medium for target object | |
| CN112926685A (en) | Industrial steel oxidation zone target detection method, system and equipment | |
| CN113095438A (en) | Wafer defect classification method and device, system, electronic equipment and storage medium thereof | |
| CN117351472A (en) | Tobacco leaf information detection method and device and electronic equipment | |
| KR102470422B1 (en) | Method of automatically detecting sewing stitch based on CNN feature map and system for the same | |
| CN117392042A (en) | Defect detection method, defect detection apparatus, and storage medium | |
| CN115272204A (en) | Bearing surface scratch detection method based on machine vision | |
| CN108460344A (en) | Dynamic area intelligent identifying system in screen and intelligent identification Method | |
| CN114359235A (en) | Wood surface defect detection method based on improved YOLOv5l network | |
| CN116385430A (en) | Machine vision flaw detection method, device, medium and equipment | |
| CN113313107A (en) | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge | |
| CN117455917B (en) | Establishment of false alarm library of etched lead frame and false alarm on-line judging and screening method | |
| TWI749714B (en) | Method for defect detection, method for defect classification and system thereof | |
| CN112967224A (en) | Electronic circuit board detection system, method and medium based on artificial intelligence | |
| CN115830302B (en) | Multi-scale feature extraction fusion power distribution network equipment positioning identification method | |
| CN116645351A (en) | Online defect detection method and system for complex scene | |
| TWM606740U (en) | Defect detection system | |
| CN117011216A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN115471494A (en) | Wo citrus quality inspection method, device, equipment and storage medium based on image processing | |
| CN115619725A (en) | Electronic component detection method and device, electronic equipment and automatic quality inspection equipment | |
| CN115311443A (en) | Oil leakage identification method for hydraulic pump |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |