CN115937533A - Aeroponic tomato feature extraction method based on semantic segmentation - Google Patents
Aeroponic tomato feature extraction method based on semantic segmentation Download PDFInfo
- Publication number
- CN115937533A CN115937533A CN202211578932.5A CN202211578932A CN115937533A CN 115937533 A CN115937533 A CN 115937533A CN 202211578932 A CN202211578932 A CN 202211578932A CN 115937533 A CN115937533 A CN 115937533A
- Authority
- CN
- China
- Prior art keywords
- features
- aeroponic
- semantic segmentation
- extracting
- extracting features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P60/00—Technologies relating to agriculture, livestock or agroalimentary industries
- Y02P60/20—Reduction of greenhouse gas [GHG] emissions in agriculture, e.g. CO2
- Y02P60/21—Dinitrogen oxide [N2O], e.g. using aquaponics, hydroponics or efficiency measures
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a semantic segmentation-based aeroponic tomato feature extraction method, and relates to the field of soilless culture. The aeroponic tomato feature extraction method based on semantic segmentation inputs the picture of plant leaves; preprocessing an input picture; transmitting the preprocessed pictures to an encoder for extracting features, extracting local abstract features of the pictures through a ResNet network, adding a context module before residual fusion of a ResLayer structure of the ResNet network and after the last convolution, and overlapping global context information of the pictures through the context module; the extracted features are transmitted to a decoder for fusion and decoding of the features, the ASPP optimized for small targets is used for decoding coarse-grained features, the coarse-grained features are converted into fine-grained features through a dynamic kernel updating mechanism, the feature information is mapped to classification information through a main classifier, classification is completed, the segmentation precision is effectively improved, and the requirements of subsequent plant growth state judgment on the leaf and root segmentation precision are met.
Description
Technical Field
The invention relates to the technical field of soilless culture, in particular to a semantic segmentation-based aeroponic tomato feature extraction method.
Background
Aeroponic cultivation (aeroponic cultivation for short) is an indispensable cultivation mode in soilless cultivation technology, and is considered to be in line with the development direction of future intelligent agricultural production. The nutrient solution is atomized into small fog drops by using an atomizing device and is directly sprayed to the root system of the plant to provide water and nutrients required by the growth of the plant. In order to reasonably control the irrigation time and irrigation quantity of the regulated and deficient irrigation, the accurate estimation of the physiological water demand condition of the aeroponics tomatoes is necessary. In the process of evaluating the physiological water requirement of the aeroponics tomatoes, the growth states of leaves and roots of the tomatoes need to be judged in time, so that the identification and the segmentation of the characteristics of the tomatoes are particularly important.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides a semantic segmentation-based aeroponic culture tomato feature extraction method, which effectively improves the segmentation precision and meets the requirements of subsequent plant growth state discrimination on the leaf and root system segmentation precision.
(II) technical scheme
In order to realize the purpose, the invention is realized by the following technical scheme:
in a first aspect, a method for extracting features of aeroponics tomatoes based on semantic segmentation is provided, which comprises the following steps:
inputting a picture of a plant leaf;
preprocessing an input picture;
transmitting the preprocessed picture to an encoder for extracting features, extracting local abstract features of the picture through a ResNet network, adding a context module before residual fusion of a ResLayer structure of the ResNet network and after the last convolution, and overlapping global context information of the picture through the context module;
the extracted features are transmitted to a decoder for fusion and decoding of the features, the ASPP optimized for small targets is used for decoding coarse-grained features, the coarse-grained features are converted into fine-grained features through a dynamic kernel updating mechanism, the classification accuracy of edge pixels and pixels difficult to classify is improved, the feature information is mapped to classification information through a main classifier, and classification is finished.
Preferably, the context module specifically includes:
the global context module is used for context modeling;
bottleneck layer translation to capture inter-channel dependencies;
element-by-element addition is used for feature fusion.
Preferably, the extracted features are transmitted to a decoder for fusion and feature decoding, and an auxiliary classifier is added in the process, wherein the auxiliary classifier is used for accelerating the optimization process and rapidly converging.
Preferably, the expansion coefficient used for the expansion convolution layer on each branch in the ASPP structure is set to [1,3,5,8].
Preferably, a dynamic core self-updating module is added in the ASPP structure.
Preferably, a dynamic core self-updating module is added to the ASPP structure, which specifically includes:
characteristic assembly: multiplying the feature mapping F by MaskMi-1 to serve as a new assembly feature FK;
dynamic kernel self-update: first, element multiplication is carried out between FK and Ki-1, two gating values are calculated, and a new group of cores are calculated by the two gating values;
kernel interaction: calculating a group of new Ki by multi-head attention and a feedforward neural network, and completing the updating of a convolution kernel K from Ki-1 to Ki;
a convolution kernel Ki passes through FC-LN-ReLU to generate a new binary mask MaskMi;
the generation of the binary mask MaskMi is repeated until the convolution kernel K is updated for a prescribed turn.
In a second aspect, a terminal device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and is characterized in that the memory stores the computer program capable of running on the processor, and when the processor loads and executes the computer program, the method for extracting features of aeroponics tomatoes based on semantic segmentation is adopted.
In a third aspect, a computer-readable storage medium storing a computer program is provided, wherein the program, when executed by a processor, implements the method for extracting features of aeroponic tomatoes based on semantic segmentation.
(III) advantageous effects
The aeroponic tomato feature extraction method based on semantic segmentation fuses a new semantic segmentation network of a self-attention module and a co-prime factor ASPP module aiming at small target improvement and dynamic kernel update, effectively improves the segmentation precision, and meets the requirement of subsequent plant growth state discrimination on the leaf and root system segmentation precision.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a diagram illustrating a context module for adding a self-attention mechanism in a feature extraction stage according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a comparison between NLBlock and GCBlock in the embodiment of the present invention;
FIG. 4 is a block diagram illustrating the addition of a dynamic core self-refresh module in an ASPPHead according to the present invention;
FIG. 5 is a flow chart of the image pre-processing according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention, and it is to be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Examples
Referring to fig. 1 and 5, an embodiment of the invention provides a method including:
inputting a picture of a plant leaf;
preprocessing an input picture;
transmitting the preprocessed picture to an encoder for extracting features, extracting local abstract features of the picture through a ResNet network, adding a context module before residual fusion of a ResLayer structure of the ResNet network and after the last convolution, and overlapping global context information of the picture through the context module;
the extracted features are transmitted to a decoder for fusion and decoding of the features, the ASPP optimized for small targets is used for decoding coarse-grained features, the coarse-grained features are converted into fine-grained features through a dynamic kernel updating mechanism, the classification accuracy of edge pixels and pixels difficult to classify is improved, the feature information is mapped to classification information through a main classifier, and classification is finished.
Most of the traditional CNN semantic segmentation model DeepLabV3 based on the CNN structure follows the framework of an encoder-decoder, the CNN in the encoder is used for feature extraction, the feature graph is rich in semantic information while the resolution of the feature graph is gradually reduced, and then the CNN in the decoder utilizes the encoder encoding features as input to decode the final segmentation prediction result;
such a frame has the following problems:
the semantic segmentation task needs detail information besides semantic information, so deep labv3 adopts an ASPP (feature pooling pyramid) structure to extract multi-scale information, but ignores global context information, and the expansion convolution coefficient in the ASPP (feature pooling pyramid) does not optimize any specific task of the leaf segmentation class.
Referring to fig. 2, a context module based on the self-attention mechanism (global context lock) is added in the feature extraction stage:
the following three steps of the module are:
the global context module is used for context modeling;
bottleneck layer translation to capture inter-channel dependencies;
element-by-element addition is used for feature fusion.
The position is added before residual fusion of the ResLayer structure of the feature extraction backbone network (ResNet-50) and after the last convolution.
Referring to fig. 3, GCBlock has the following advantages compared to the conventional self-attention module (e.g., NLBlock):
unlike the conventional non-local block (NLBlock) which has a large number of parameters, GCBlock replaces 1 × 1 convolution in the conventional transform module (Wv) with a bottleneck layer (depth separable convolution), greatly reducing the parameter values.
According to the self-attention mechanism-related research, the learned global context information is almost the same at different positions, namely the global context without position dependence is learned, so that the traditional NLBlock is simplified by calculating a global (Global) attention parameter, and the parameter is shared by all positions, namely the value of the query (Wq) is ignored.
To reduce the computational burden of the reduced version non-localblock, we move Wv outside the global context module and remove the linear transformation matrix (Wz) before element-by-element addition;
according to the sizes of the characteristics of the leaves and the root system, the expansion convolution module in the ASPP is redesigned, so that the ASPP is more suitable for small target detection.
In the original ASPP structure, in order to detect multi-scale information, a characteristic diagram is passed through four parallel expansion convolution layers, the expansion coefficients adopted by the expansion convolution layers on each branch are different, the original structure is set to be [1,6,12,18], but for the detection of a small target, such as a blade detection target, an overlarge expansion coefficient is not beneficial to accurate segmentation of small target edge information, and therefore the expansion coefficient is changed to be [1,3,5,8]. Meanwhile, the original expansion convolution coefficient has a theoretical problem, namely the problem of the grid effect, and after the problem is changed into a relatively prime number, the loss of local information caused by the grid effect is greatly reduced.
Referring to fig. 4, a dynamic kernel update module (dynamic kernel update) is added to the asppehead to solve the problem of limited effect caused by the difference between the input data and the training data, and the flexibility and performance of the model are improved by dynamic deformation of the kernel:
specifically, the dynamic self-update is performed by using N kernels (N is the number of classes of the target to be detected, such as leaf + root system + background, and N is 3), and the method includes the following three steps:
characteristic assembly: multiplying the feature map F by MaskMi-1 as a new assembly feature FK (red dotted frame on the upper graph)
Dynamic kernel self-update: first, perform an element multiplication between FK and Ki-1, then compute two gates (gates), and then compute a new set of kernels from the two gates;
kernel interaction: calculating a group of new Ki through an MSA-FFN (Multi-HeadSelf-Attention + Feed-Forward neural network) (Multi-head Attention + Feed-Forward neural network), and then completing the updating of the convolution kernel K from Ki-1 to Ki;
subsequently, the convolution kernel Ki passes through FC-LN-ReLU to generate a new binary mask MaskMi;
the generation of the binary mask MaskMi is repeated until the convolution kernel K is updated for a prescribed turn.
As another embodiment of the present invention, there is provided a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the method for extracting features of aeroponic tomato based on semantic segmentation in the above embodiments is adopted.
As a further embodiment of the present invention, there is provided a computer readable storage medium storing a computer program, wherein the program is executed by a processor to implement the method for extracting features of aeroponic tomatoes based on semantic segmentation in the above embodiments.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
Claims (8)
1. A method for extracting features of aeroponic tomatoes based on semantic segmentation is characterized by comprising the following steps:
inputting a picture of a plant leaf;
preprocessing an input picture;
transmitting the preprocessed pictures to an encoder for extracting features, extracting local abstract features of the pictures through a ResNet network, adding a context module before residual fusion of a ResLayer structure of the ResNet network and after the last convolution, and overlapping global context information of the pictures through the context module;
the extracted features are transmitted to a decoder for fusing and decoding the features, the ASPP optimized for small targets is used for decoding coarse-grained features, the coarse-grained features are converted into fine-grained features through a dynamic kernel updating mechanism, the classification accuracy of edge pixels and pixels difficult to classify is improved, and feature information is mapped to classification information through a main classifier to finish classification.
2. The method for extracting features of aeroponic tomatoes based on semantic segmentation as claimed in claim 1, wherein the method comprises: the context module specifically includes:
the global context module is used for context modeling;
bottleneck layer translation to capture inter-channel dependencies;
element-by-element addition is used for feature fusion.
3. The method for extracting features of aeroponic tomatoes based on semantic segmentation as claimed in claim 1, wherein the method comprises: the transmission of the extracted features to the decoder for fusion and addition of an auxiliary classifier in the decoding of the features, the auxiliary classifier for speeding up the optimization process and for fast convergence.
4. The method for extracting features of aeroponic tomatoes based on semantic segmentation as claimed in claim 1, wherein the method comprises: the expansion coefficient employed by the expansion convolution layer on each branch in the ASPP structure is set to [1,3,5,8].
5. The method for extracting features of aeroponic tomatoes based on semantic segmentation as claimed in claim 4, wherein the method comprises: and a dynamic core self-updating module is added in the ASPP structure.
6. The method for extracting features of aeroponic tomatoes based on semantic segmentation as claimed in claim 1, wherein the method comprises: the method for adding the dynamic core self-updating module in the ASPP structure specifically comprises the following steps:
characteristic assembly: multiplying the feature mapping F by Mask Mi-1 to serve as a new assembly feature FK;
dynamic kernel self-update: first, element multiplication is performed between FK and Ki-1, then two gates are calculated, and a new set of kernels is calculated by the two gates;
kernel interaction: calculating a group of new Ki by multi-head attention and a feedforward neural network, and updating the convolution kernel K from Ki-1 to Ki;
a convolution kernel Ki passes through FC-LN-ReLU to generate a new binary Mask Mi;
the generation of the binary Mask Mi is repeated until the convolution kernel K is updated for a specified turn.
7. A terminal device, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the memory stores the computer program capable of running on the processor, and the processor loads and executes the computer program, and the method for extracting features of aeroponics tomatoes based on semantic segmentation as claimed in any one of claims 1 to 6 is used.
8. A computer-readable storage medium storing a computer program, which when executed by a processor implements a method for extracting features of aeroponic tomatoes based on semantic segmentation as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211578932.5A CN115937533B (en) | 2022-12-05 | 2022-12-05 | Semantic segmentation-based aeroponic tomato feature extraction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211578932.5A CN115937533B (en) | 2022-12-05 | 2022-12-05 | Semantic segmentation-based aeroponic tomato feature extraction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115937533A true CN115937533A (en) | 2023-04-07 |
| CN115937533B CN115937533B (en) | 2023-08-25 |
Family
ID=86653644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211578932.5A Active CN115937533B (en) | 2022-12-05 | 2022-12-05 | Semantic segmentation-based aeroponic tomato feature extraction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115937533B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020215236A1 (en) * | 2019-04-24 | 2020-10-29 | 哈尔滨工业大学（深圳） | Image semantic segmentation method and system |
| CN112287940A (en) * | 2020-10-30 | 2021-01-29 | 西安工程大学 | Semantic segmentation method of attention mechanism based on deep learning |
| WO2021244621A1 (en) * | 2020-06-04 | 2021-12-09 | 华为技术有限公司 | Scenario semantic parsing method based on global guidance selective context network |
| CN113780296A (en) * | 2021-09-13 | 2021-12-10 | 山东大学 | Remote sensing image semantic segmentation method and system based on multi-scale information fusion |
| CN113807355A (en) * | 2021-07-29 | 2021-12-17 | 北京工商大学 | Image semantic segmentation method based on coding and decoding structure |
| CN114937148A (en) * | 2022-06-08 | 2022-08-23 | 华南理工大学 | Small target feature enhanced image segmentation method and system |
-
2022
- 2022-12-05 CN CN202211578932.5A patent/CN115937533B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020215236A1 (en) * | 2019-04-24 | 2020-10-29 | 哈尔滨工业大学（深圳） | Image semantic segmentation method and system |
| WO2021244621A1 (en) * | 2020-06-04 | 2021-12-09 | 华为技术有限公司 | Scenario semantic parsing method based on global guidance selective context network |
| CN112287940A (en) * | 2020-10-30 | 2021-01-29 | 西安工程大学 | Semantic segmentation method of attention mechanism based on deep learning |
| CN113807355A (en) * | 2021-07-29 | 2021-12-17 | 北京工商大学 | Image semantic segmentation method based on coding and decoding structure |
| CN113780296A (en) * | 2021-09-13 | 2021-12-10 | 山东大学 | Remote sensing image semantic segmentation method and system based on multi-scale information fusion |
| CN114937148A (en) * | 2022-06-08 | 2022-08-23 | 华南理工大学 | Small target feature enhanced image segmentation method and system |
Non-Patent Citations (1)
| Title |
|---|
| 杨鑫;于重重;王鑫;陈秀新;: "融合ASPP-Attention和上下文的复杂场景语义分割", 计算机仿真, no. 09 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115937533B (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xu et al. | Light-YOLOv3: fast method for detecting green mangoes in complex scenes using picking robots | |
| CN109800628B (en) | Network structure for enhancing detection performance of SSD small-target pedestrians and detection method | |
| CN110428428B (en) | Image semantic segmentation method, electronic equipment and readable storage medium | |
| CN108629288B (en) | Gesture recognition model training method, gesture recognition method and system | |
| CN112990116B (en) | Behavior recognition device and method based on multi-attention mechanism fusion and storage medium | |
| CN110929610A (en) | Plant disease identification method and system based on CNN model and transfer learning | |
| CN110032952B (en) | Road boundary point detection method based on deep learning | |
| CN112364931A (en) | Low-sample target detection method based on meta-feature and weight adjustment and network model | |
| Li et al. | Pitaya detection in orchards using the MobileNet-YOLO model | |
| AU2021105840A4 (en) | Method for predicting typhoon track | |
| CN116649191A (en) | Remote fertilization and irrigation control system and method based on PLC | |
| CN114187590A (en) | Method and system for identifying target fruits under homochromatic system background | |
| CN111783688B (en) | Remote sensing image scene classification method based on convolutional neural network | |
| CN115937533B (en) | Semantic segmentation-based aeroponic tomato feature extraction method | |
| CN115049786B (en) | Task-oriented point cloud data downsampling method and system | |
| CN116310128A (en) | Dynamic environment monocular multi-object SLAM method based on instance segmentation and three-dimensional reconstruction | |
| CN116309429A (en) | Chip defect detection method based on deep learning | |
| CN116246305A (en) | Pedestrian retrieval method based on hybrid component transformation network | |
| CN112507940B (en) | Bone action recognition method based on differential guidance representation learning network | |
| Si et al. | Image semantic segmentation based on improved DeepLab V3 model | |
| CN116152263A (en) | CM-MLP network-based medical image segmentation method | |
| CN112329697B (en) | Improved YOLOv 3-based on-tree fruit identification method | |
| CN114494284A (en) | Scene analysis model and method based on explicit supervision area relation | |
| CN114565639A (en) | Target tracking method and system based on composite convolutional network | |
| CN112396126A (en) | Target detection method and system based on detection of main stem and local feature optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |