CN110991454A - Blade image recognition method and device, electronic equipment and storage medium - Google Patents
Blade image recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN110991454A CN110991454A CN201911342356.2A CN201911342356A CN110991454A CN 110991454 A CN110991454 A CN 110991454A CN 201911342356 A CN201911342356 A CN 201911342356A CN 110991454 A CN110991454 A CN 110991454A
- Authority
- CN
- China
- Prior art keywords
- image
- leaf
- blade
- neural network
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
Abstract
The application provides a leaf image identification method, a leaf image identification device, electronic equipment and a storage medium, and relates to the technical field of image identification and leaf image identification; the method comprises the following steps: obtaining a leaf image, the leaf image comprising: a leaf region and a background region; removing a background area of the blade image to obtain a blade area of the blade image; and identifying the leaf region by using a pre-trained neural network model to obtain the classification of the leaf image. In the implementation process, the background area of the leaf image is removed, and then the pre-trained neural network model is used for identifying the leaf image with the background area removed, so that the accuracy of identifying the plant leaf image is improved.
Description
Technical Field
The present disclosure relates to the field of image recognition and leaf image recognition, and in particular, to a leaf image recognition method, device, electronic device, and storage medium.
Background
Image recognition, which refers to a technology for processing, analyzing and understanding images by using a computer to recognize various targets and objects in different modes; the leaf image recognition means that a leaf image of a plant is obtained first, and then the leaf image is recognized.
At present, the identification of plant leaves is carried out manually, namely, the plant leaves are collected manually, the plant leaves are photographed, then the plant leaves are observed through human eyes, relevant data are searched according to the plant leaves, and finally the classification of the plant leaves is obtained according to the identification experience or logic analysis of people. In the practical process of identifying the leaf images, the accuracy of identifying the plant leaf images is not high.
Disclosure of Invention
An object of the embodiments of the present application is to provide a leaf image recognition method, device, electronic device and storage medium, which are used to solve the problem that the accuracy of recognizing a plant leaf image is not high.
The embodiment of the application provides a blade image identification method, which comprises the following steps: obtaining a leaf image, the leaf image comprising: a leaf region and a background region; removing the background area of the blade image to obtain the blade area of the blade image; and identifying the blade region by using a pre-trained neural network model to obtain the classification of the blade image. In the implementation process, the background area of the leaf image is removed, and then the pre-trained neural network model is used for identifying the leaf image with the background area removed, so that the accuracy of identifying the plant leaf image is improved.
Optionally, in an embodiment of the present application, the removing the background area of the blade image includes: determining a leaf region of the leaf image; removing a background area of the leaf image according to the leaf area. In the implementation process, the blade area is obtained first, and then the background area is removed according to the blade area, so that the speed of obtaining and removing the background area of the blade image is improved.
Optionally, in an embodiment of the present application, the determining a leaf region of the leaf image includes: determining a gray scale image of the leaf image; determining a binary image of the blade image according to the gray level image; and determining a blade area of the blade image according to the binary image. In the implementation process, the gray level processing and the binarization processing are carried out on the leaf image, so that the accuracy of determining the leaf area of the leaf image is improved.
Optionally, in this embodiment of the present application, the removing the background area of the blade image according to the blade area includes: according to
The leaf area removes a background area of the leaf image; wherein i represents a color channel of the leaf image, and in the RGB color model, i is R, G, B, (x, y) represents an abscissa and an ordinate of a pixel point of the leaf image, i.e., a coordinate of the pixel point, respectively, and f isi(x, y) represents a pixel point value of coordinates (x, y) in the ith color channel, and C is a set of pixel points in the leaf region. In the implementation process, the speed of obtaining and removing the background area of the blade image is improved by reserving the pixel points of the blade area and removing the background area of the blade image.
Optionally, in this embodiment of the present application, the identifying the blade region by using a pre-trained neural network model to obtain the classification of the blade image includes performing convolution operation and pooling operation on the blade region by using at least one convolution layer and at least one pooling layer of the neural network model to obtain a first feature map; calculating the first feature map by using at least one perception module group of the neural network model to obtain a second feature map; and performing pooling, linear regression and normalization operations on the second feature map by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model to obtain the classification of the leaf image. In the implementation process, the convolution operation and the pooling operation are carried out on the leaf area, and the pooling, linear regression and normalization operation are carried out on the second characteristic diagram by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model, so that the accuracy of identifying the plant leaf image is improved.
Optionally, in an embodiment of the present application, the method further includes: obtaining a plurality of plant leaf images and a plurality of plant leaf labels, wherein the plant leaf images are leaf images of plants, and the plant leaf labels are classifications corresponding to the plant leaf images; and training a neural network by taking the plant leaf images as training data and the plant leaf labels as training labels to obtain the trained neural network model. In the implementation process, the trained neural network model is obtained by training the neural network, so that the speed of recognizing the plant leaf image is improved.
Optionally, in an embodiment of the present application, the neural network model is inclusion v3 or inclusion v 4. In the implementation process, the inclusion V3 or the inclusion V4 is used as a neural network model for identifying the leaf images, so that the accuracy of identifying the plant leaf images is improved.
The embodiment of the present application further provides a blade image recognition apparatus, including: an image obtaining module for obtaining a leaf image, the leaf image comprising: a leaf region and a background region; the background removing module is used for removing a background area of the blade image and obtaining a blade area of the blade image; and the classification identification module is used for identifying the blade area by using a pre-trained neural network model to obtain the classification of the blade image.
Optionally, in an embodiment of the present application, the background removing module includes: a blade determination module for determining a blade region of the blade image; and the area removing module is used for removing the background area of the blade image according to the blade area.
Optionally, in an embodiment of the present application, the blade determination module includes: the first determining module is used for determining a gray level image of the blade image; the second determining module is used for determining a binary image of the blade image according to the gray level image; and the third determining module is used for determining the blade area of the blade image according to the binary image.
Optionally, in an embodiment of the present application, the region removing module includes: an image removal module for removing a pattern based on
The leaf area removes a background area of the leaf image; wherein i represents a color channel of the leaf image, and in the RGB color model, i is R, G, B, (x, y) represents an abscissa and an ordinate of a pixel point of the leaf image, i.e., a coordinate of the pixel point, respectively, and f isi(x, y) represents a pixel point value of coordinates (x, y) in the ith color channel, and C is a set of pixel points in the leaf region.
Optionally, in an embodiment of the present application, the classification identifying module includes: the first operation module is used for performing convolution operation and pooling operation on the blade region by using at least one convolution layer and at least one pooling layer of the neural network model to obtain a first feature map; the second operation module is used for operating the first feature map by using at least one perception module group of the neural network model to obtain a second feature map; and the third operation module is used for performing pooling, linear regression and normalization operation on the second feature map by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model to obtain the classification of the leaf images.
Optionally, in an embodiment of the present application, the method further includes: a data obtaining module, configured to obtain a plurality of plant leaf images and a plurality of plant leaf labels, where the plant leaf images are leaf images of plants, and the plant leaf labels are classifications corresponding to the plant leaf images; and the model training module is used for training a neural network by taking the plant leaf images as training data and the plant leaf labels as training labels to obtain the trained neural network model.
Optionally, in an embodiment of the present application, the neural network model is inclusion v3 or inclusion v 4.
An embodiment of the present application further provides an electronic device, including: a processor and a memory storing machine-readable instructions executable by the processor, the machine-readable instructions when executed by the processor performing the method as described above.
The embodiment of the present application also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method as described above is executed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
FIG. 1 is a schematic flow chart of a blade image recognition method according to an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating a background removal method for a blade image according to an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a preset processing method for a blade image according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an inclusion B module in an inclusion V3 model according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a blade image recognition device according to an embodiment of the present application;
fig. 6 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
Before describing the blade image identification method provided by the embodiment of the present application, some concepts related to the embodiment of the present application are described below:
neural Networks (NN) are complex network systems formed by a large number of simple processing units (called neurons) widely interconnected, reflect many basic features of human brain functions, and are highly complex nonlinear dynamical learning systems. The neural network has the capabilities of large-scale parallel, distributed storage and processing, self-organization, self-adaptation and self-learning, and is particularly suitable for processing inaccurate and fuzzy information processing problems which need to consider many factors and conditions simultaneously. The neural network model refers to a neural network model obtained by training the neural network with preset training data, where the preset training data may be set according to specific actual conditions, for example: in the task of recognizing the character images, the preset training data refers to the character images to be recognized, and in the process of supervised learning training, correct labels need to be set for the training data.
Convolutional Neural Networks (CNN), which is a kind of feed-forward Neural network, allows artificial neurons to respond to surrounding cells and perform large-scale image processing. The convolutional neural network includes convolutional layers and pooling layers. The convolutional neural network includes a one-dimensional convolutional neural network, a two-dimensional convolutional neural network, and a three-dimensional convolutional neural network. One-dimensional convolutional neural networks are often applied to data processing of sequence classes; two-dimensional convolutional neural networks are often applied to the recognition of image-like texts; the three-dimensional convolutional neural network is mainly applied to medical image and video data identification.
Convolutional layer (Convolutional layer), each Convolutional layer in the Convolutional neural network is composed of a plurality of Convolutional units, and the parameters of each Convolutional unit are optimized through a back propagation algorithm. The convolution operation aims to extract different input features, the convolution layer at the first layer can only extract some low-level features such as edges, lines, angles and other levels, and more layers of networks can iteratively extract more complex features from the low-level features. The convolutional layer needs to set an activation function before connection, and common activation functions include: modified linear units, Sigmoid functions, and tanh functions.
The pooling layer is used for performing subarea sampling on data to downsample a large matrix into a small matrix, so that the calculated amount is reduced, and overfitting can be prevented; usually, pooling is performed on the feature map by using a pooling layer, and the result obtained after the pooling operation is sent to a correction linear unit for calculation.
The normalized exponential function (Softmax), or Softmax function, is in fact a gradient log normalization of a finite discrete probability distribution. In mathematics, particularly probability theory and related fields, a normalized exponential function, or Softmax function, is a generalization of logistic functions. It can "compress" a K-dimensional vector z containing arbitrary real numbers into another K-dimensional real vector σ (z) such that each element ranges between (0,1) and the sum of all elements is 1.
A server refers to a device that provides computing services over a network, such as: x86 server and non-x 86 server, non-x 86 server includes: mainframe, minicomputer, and UNIX server. Certainly, in a specific implementation process, the server may specifically select a mainframe or a minicomputer, where the mainframe refers to a dedicated processor that mainly supports a closed and dedicated device for providing Computing service of a UNIX operating system, and that uses Reduced Instruction Set Computing (RISC), single-length fixed-point instruction average execution speed (MIPS), and the like; a mainframe, also known as a mainframe, refers to a device that provides computing services using a dedicated set of processor instructions, an operating system, and application software.
It should be noted that the blade image identification method provided in the embodiment of the present application may be executed by an electronic device, where the electronic device refers to a device terminal having a function of executing a computer program or the server described above, and the device terminal includes, for example: a smart phone, a Personal Computer (PC), a tablet computer, a Personal Digital Assistant (PDA), a Mobile Internet Device (MID), a network switch or a network router, and the like.
Before describing the blade image identification method provided by the embodiment of the present application, an application scenario to which the blade image identification method is applicable is described, where the application scenario includes, but is not limited to: identifying the plant leaves to obtain the types of the plants, namely classifying the plants according to the leaf images, or identifying and classifying the leaves of the plants to obtain the classification of the leaves, and the like. The dependent characteristics of the classifications herein include, but are not limited to: the shape of the plant leaves, the nutritional value of the plants, the medicinal value and the economic value of the plants and the like. For convenience of explanation and understanding, the following description will be made in detail by taking as an example a classification of leaves obtained by identifying and classifying leaves of a plant.
Please refer to fig. 1, which illustrates a flowchart of a blade image recognition method provided in the embodiment of the present application; the blade image identification method can comprise the following steps:
step S100: obtaining a leaf image, the leaf image comprising: a leaf area and a background area.
The leaf image refers to an image of a leaf of a plant, and comprises: a leaf region and a background region; the leaf area refers to an area where most of the leaves in the leaf image are leaves, and the background area refers to an area where no plant leaves exist in the leaf image. Specific examples of the leaf image include: the image of the original trees and shrubs in the central european region, or the image of the trees and shrubs which are frequently planted, etc., wherein the specific format of the leaf image can be png format, jpg format, gif, pdf, psd, etc.
Blade image acquisition means are, for example: in the first mode, leaf images are obtained by manually photographing leaves of a plant; in a second mode, the existing leaf image is modified by using an image enhancement method to obtain the leaf image, specifically for example: the method comprises the following steps of rotating the leaf direction of a plant, adjusting the brightness of an image, adjusting the contrast of the image, adjusting the sharpening degree of the image and the like; in the third mode, leaf images and the like are downloaded from the internet through crawler software.
After step S100, step S200 is performed: and removing the background area of the blade image to obtain the blade area of the blade image.
Please refer to fig. 2, which is a schematic diagram illustrating a background removing method for a blade image according to an embodiment of the present application; removing the background area of the blade image, and obtaining one embodiment of the blade area of the blade image, for example: sometimes, the obtained leaf image is not a single leaf, and other leaves or impurities may be in the leaf image, such as an unprocessed original image on the left side in the figure; then, at this time, the original image needs to be processed, that is, the background region of the blade image is removed, here, the redundant blades and the impurities can be removed by reserving the blade region with the largest area and regarding the remaining regions except the blade region with the largest area as the background region, and the processed image is removed as the right processed picture in the image.
Please refer to fig. 3, which is a schematic diagram illustrating a preset processing method for a leaf image according to an embodiment of the present application; of course, in the implementation process, the preset processing may also be performed on the blade image, and then the preset processing when the background area of the blade image is removed, that is, step S200 may include the following steps:
step S210: a leaf region of the leaf image is determined.
The embodiment of determining the blade area of the blade image may include the following steps, that is, the embodiment of step S210 may include the following steps:
step S211: a gray scale image of the leaf image is determined.
Among these, the embodiment of determining the gray scale image of the leaf image is, for example: if the leaf image is in png format and the color channel model is an RGB model, the process of processing the leaf image to obtain a gray level image is also called gray level processing, which is a process of converting a color image into a gray level image; for example, in the RGB model, when R ═ G ═ B, the color represents a gray scale color, where the value of R ═ G ═ B is called the gray scale value, so that each pixel of the gray scale image only needs one byte to store the gray scale value (also called the intensity value, luminance value), and the gray scale range is 0 to 255. In a specific implementation process, the color image may be subjected to gray scale processing by using four methods, such as a component method, a maximum value method, an average value method, and a weighted average method.
Step S212: and determining a binary image of the blade image according to the gray level image.
Binary Image (Binary Image) refers to that each pixel in an Image has only two possible values or gray scale states, and people often represent Binary images by black and white or monochrome images. The binary image means that there are only two gray levels in the image, that is, the gray value of any pixel in the image is 0 or 255, which represents black and white respectively.
The binary image of the leaf image is determined from the grayscale image, for example: converting the gray level image into a binary image by using an Otsu algorithm (also called OTSU algorithm or maximum inter-class variance method); the OTSU algorithm here is an efficient algorithm for binarizing images proposed by OTSU in 1979.
Step S213: and determining a blade area of the blade image according to the binary image.
Embodiments of determining the blade region of the blade image from the binary image are for example: firstly, morphological processing is performed on the binary image, specifically for example: firstly, carrying out open operation on the binary image, wherein the iteration time is three times, and then carrying out closed operation, wherein the iteration time is three times; and then carrying out contour extraction on the binary image after morphological processing, determining the region where the blade is located, and carrying out image segmentation to extract the region. In the implementation process, the gray level processing and the binarization processing are carried out on the leaf image, so that the accuracy of determining the leaf area of the leaf image is improved.
Of course, in a specific implementation process, other preset processing may be performed on the blade image, specifically, for example: morphological contour extraction, background whitening, background dirt removal, redundant leaf removal in the background and the like; the background whitening here can be understood as the above-mentioned process of determining a binary image of the blade image according to the gray scale image, and the following process of removing the background area of the blade image according to the blade area can be understood by the operations of removing background dirt and removing unnecessary leaves in the background. In the obtained leaf images, the backgrounds of a plurality of leaf images are gray or dark gray, the backgrounds are dirty, the four corners are provided with redundant leaves, and the noise in the backgrounds is more interfered, which can affect the subsequent training result of the neural network. And carrying out multiple preprocessing on the leaf image by using contour extraction based on morphology to remove noise interference of the image background. Image morphological operations are powerful tools for extracting image features and include dilation, erosion, opening, closing, and the like.
The above morphological contour extraction method includes, for example: firstly, generating a structural operator, wherein the shape of the structural operator can be a square, a circle or a cross; for example, a 3 x 3 square structuring operator may be selected, where the structuring operator is denoted e; then, performing expansion operation on the blade image (the blade image is represented as I) to obtain an expanded image represented as Id; finally, performing erosion operation on the blade image to be represented as Ie; then the profile of the blade image can be obtained from Id-Ie.
After step S210, step S220 is performed: the background area of the leaf image is removed according to the leaf area.
The above-described embodiment of removing the background area of the blade image according to the blade area includes:
according to
The leaf area removes the background area of the leaf image; wherein i represents the color channel of the leaf image, and in the RGB color model, i is R, G, B, (x, y) represents the abscissa and ordinate of a pixel point of the leaf image, i.e., the coordinate of the pixel point, respectively, and fi(x, y) represents a pixel point value whose coordinate is (x, y) in the ith color channel, and C is a set of pixel points in the leaf region. In the implementation process, the speed of obtaining and removing the background area of the blade image is improved by reserving the pixel points of the blade area and removing the background area of the blade image; and by obtaining the leaf area first and then removing the background area according to the leaf area, the sum of the obtained result is improvedThe velocity of the background area of the blade image is removed.
After step S200, step S300 is performed: and identifying the leaf region by using a pre-trained neural network model to obtain the classification of the leaf image.
The neural network model may adopt an inclusion v3 or an inclusion v4, and certainly, in a specific implementation process, an inclusion v1 or an inclusion v2 may also be adopted, and different neural network models are adopted according to different training data. In the implementation process, the Inception V1, the Inception V2, the Inception V3 or the Incepton V4 are adopted as the neural network model for identifying the leaf images, so that the accuracy of identifying the plant leaf images is improved. For ease of understanding and illustration, the neural network model is described herein only by way of example as the incorporated v3, and the specific structure of the incorporated v3 is as follows:
as can be seen from the above table, the inclusion v3 model includes three types of inclusion modules, which are: inclusion a, inclusion B and inclusion C; the principle of design of inclusion a, inclusion B and inclusion C is similar, and different features can be simultaneously extracted by the network by using different convolution kernels, so that a richer feature expression can be finally obtained, the difference is only that part of the network structures are different, and for convenience of understanding and explanation, only inclusion B is taken as an example and explained as follows:
please refer to fig. 4, which illustrates a schematic structural diagram of an inclusion B module in an inclusion v3 model according to an embodiment of the present application; the inclusion B module is a multi-branch parallel structure, and each branch can be regarded as a sub-Network, so the inclusion V3 Network can also be regarded as a Network-in-Network (NIN). The Incep B module utilizes different convolution kernels to simultaneously extract different features, and finally a richer feature expression can be obtained, which is beneficial to the identification of complex objects. The module decomposes an n × n convolution into n × 1 and 1 × n convolutions, so that the receptive fields of the two modes before and after decomposition are the same, the result is equivalent, but the parameter quantity of the modes after decomposition is less, the required calculation quantity of the network is obviously reduced, and the training efficiency is improved. The features and advantages of the inclusion a and the inclusion C modules are similar to those of the inclusion B module, and thus are not described herein again.
In the implementation process, the background area of the leaf image is removed, and then the pre-trained neural network model is used for identifying the leaf image with the background area removed, so that the accuracy of identifying the plant leaf image is improved.
Optionally, before the neural network model is used to identify the leaf region, the neural network model may be trained, and the method for training the neural network model may include the following steps:
step S301: a plurality of plant leaf images and a plurality of plant leaf labels are obtained.
The plant leaf image is a leaf image of the plant, and the plant leaf label is a classification corresponding to the plant leaf image, specifically for example: the first image is an image of maple leaves, the tag corresponding to maple leaves is 1, the second image is shrub leaves, the tag corresponding to shrub leaves is 2, and so on.
Embodiments of obtaining a plurality of plant leaf images and a plurality of plant leaf labels are for example: in the first mode, a leaf image is obtained by manually photographing leaves of a plant, and the leaf image is labeled; in the second mode, the existing leaf image is modified by using an image enhancement method to obtain a leaf image, and the modified leaf image is labeled; in a third way, a source database and the like are downloaded from the internet by downloading software, for example: the plant leaf database MEW (middle European woods)2014 edition was used.
Of course, in a specific implementation process, before using the database, the preset processing in step S210 and step S220 may be performed on the pictures in the database to eliminate various factors affecting accuracy, such as: the background may contain impurities such as dirt and soil, and the area occupied by each plant leaf in the picture is different due to different shooting distances, wherein part of the picture leaves only occupy a small part, and the area occupied by some picture leaves is larger.
The MEW database comprises pictures of trees and shrubs which are native or frequently planted in the Central European region, wherein the total number of the pictures is 15180, the total number of the pictures is 201, all the pictures of plant leaves are scanned at the resolution of 300dpi, the pictures in the database have relatively uniform backgrounds, and the file formats of the pictures are lossless compressed png formats. It should be noted that 106 pictures of the 6 th (Acer _ creatoids) are completely repeated and removed, and a 41 th picture of the 190 th (viburn _ farreri) viburn _ farreri _41 png is 0 byte, which cannot be read normally and also removes the image which cannot be read normally.
It can be understood that, in the process of obtaining a plurality of plant leaf images and a plurality of plant leaf labels, in order to make the training model adapt to the identification of plant leaves under various conditions as much as possible, more plant leaf images and plant leaf labels need to be obtained, and in the case that the number of obtained plant leaf images is limited, the training data can be expanded according to the limited plant leaf images. Here, the training data is augmented by, for example: the data set is expanded by adopting a basic image enhancement method, and the expansion mode comprises the following steps: the plant leaf direction rotation, the picture brightness adjustment, the picture contrast adjustment, the picture sharpening degree adjustment and the like are carried out, the number of the plant leaf images is increased to be well trained on the neural network, and the good identification accuracy is obtained.
Step S302: and training the neural network by taking the plant leaf images as training data and the plant leaf labels as training labels to obtain a trained neural network model.
The implementation mode of training the neural network by using a plurality of plant leaf images as training data and plant leaf labels as training labels to obtain a trained neural network model is as follows: and training the IncepotionV 3 neural network by using the plant leaf images in the MEW database as training data and the plant leaf labels in the MEW database as training labels to obtain a trained IncepotionV 3 model. Of course, in a specific implementation process, the neural network model may also adopt inclusion v4, or may also adopt inclusion v1 or inclusion v2, and different neural network models are adopted according to different training data. In the implementation process, the trained neural network model is obtained by training the neural network, so that the speed of recognizing the plant leaf image is improved.
The above-mentioned identifying the leaf region by using the pre-trained neural network model to obtain the classification of the leaf image, that is, step S300 may include the following steps:
step S310: and performing convolution operation and pooling operation on the blade region by using at least one convolution layer and at least one pooling layer of the neural network model to obtain a first feature map.
The convolution operation and the pooling operation are carried out on the blade region by using at least one convolution layer and at least one pooling layer of the neural network model, and the embodiment of the first feature map is obtained by the following steps: referring to a specific structure table of Inception V3, a first feature map is obtained by performing convolution operation and pooling operation on a blade region using 6 convolution layers and 1 pooling layer.
Step S320: and operating the first characteristic diagram by using at least one perception module group of the neural network model to obtain a second characteristic diagram.
The sensing module group means that the inclusion V3 model includes any inclusion module, namely, the inclusion A, the inclusion B or the inclusion C; as can be seen from the specific structure table of inclusion v3, inclusion a includes 3 inclusions, inclusion B includes 5 inclusions, and inclusion C includes 2 inclusions.
The embodiment of using at least one perception module group of the neural network model to operate the first feature map to obtain the second feature map is as follows: with reference to the specific structure table of inclusion v3, the first feature map is calculated using the above-described inclusion a, inclusion b, and inclusion C, and a second feature map is obtained.
Step S330: and performing pooling, linear regression and normalization operation on the second feature map by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model to obtain the classification of the leaf images.
The second feature map is subjected to pooling, linear regression and normalization operations using a pooling layer, a linear regression layer and a normalization exponential function layer of the neural network model, and classification of the leaf image is obtained by, for example: referring to the specific structure table of inception v3, the second feature map is subjected to pooling, linear regression, and normalization operations using one pooling layer (i.e., the second pooling layer with an input size of 8 × 8 × 2048) of the neural network model, a linear regression layer (i.e., the linear regression layer with an input size of 1 × 1 × 2048), and a normalization index function layer (i.e., the normalization index function layer with an input size of 1 × 1 × 201) of the neural network model, so as to obtain the classification of the leaf image.
In the implementation process, the convolution operation and the pooling operation are carried out on the leaf area, and the pooling, linear regression and normalization operation are carried out on the second characteristic diagram by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model, so that the accuracy of identifying the plant leaf image is improved.
Please refer to fig. 5, which illustrates a schematic structural diagram of a blade image recognition apparatus provided in the embodiment of the present application; the embodiment of the present application provides a blade image recognition apparatus 500, including:
an image obtaining module 510 for obtaining a blade image, the blade image comprising: a leaf area and a background area.
And a background removing module 520, configured to remove a background area of the blade image, and obtain a blade area of the blade image.
And the classification identification module 530 is configured to identify a leaf region by using a pre-trained neural network model, so as to obtain a classification of the leaf image.
Optionally, in an embodiment of the present application, the background removing module includes:
and the blade determining module is used for determining the blade area of the blade image.
And the area removing module is used for removing the background area of the blade image according to the blade area.
Optionally, in an embodiment of the present application, the blade determination module includes:
the first determining module is used for determining a gray level image of the blade image.
And the second determining module is used for determining a binary image of the blade image according to the gray level image.
And the third determining module is used for determining the blade area of the blade image according to the binary image.
Optionally, in an embodiment of the present application, the region removing module includes:
an image removal module for removing a pattern based on
The leaf area removes the background area of the leaf image; wherein i represents the color channel of the leaf image, and in the RGB color model, i is R, G, B, (x, y) represents the abscissa and ordinate of a pixel point of the leaf image, i.e., the coordinate of the pixel point, respectively, and fi(x, y) represents a pixel point value whose coordinate is (x, y) in the ith color channel, and C is a set of pixel points in the leaf region.
Optionally, in an embodiment of the present application, the classification identifying module includes:
the first operation module is used for carrying out convolution operation and pooling operation on the blade area by using at least one convolution layer and at least one pooling layer of the neural network model to obtain a first characteristic diagram.
And the second operation module is used for operating the first characteristic diagram by using at least one perception module group of the neural network model to obtain a second characteristic diagram.
And the third operation module is used for performing pooling, linear regression and normalization operation on the second feature map by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model to obtain classification of the leaf images.
Optionally, in an embodiment of the present application, the method further includes:
the data acquisition module is used for acquiring a plurality of plant leaf images and a plurality of plant leaf labels, wherein the plant leaf images are leaf images of plants, and the plant leaf labels are classifications corresponding to the plant leaf images.
And the model training module is used for training the neural network by taking the plant leaf images as training data and the plant leaf labels as training labels to obtain a trained neural network model.
Optionally, in an embodiment of the present application, the neural network model is inclusion v3 or inclusion v 4.
It should be understood that the device corresponds to the blade image identification method embodiment described above, and can perform the steps related to the method embodiment described above, and the specific functions of the device can be referred to the description above, and the detailed description is appropriately omitted here to avoid redundancy. The device includes at least one software function that can be stored in memory in the form of software or firmware (firmware) or solidified in the Operating System (OS) of the device.
Please refer to fig. 6 for a schematic structural diagram of an electronic device according to an embodiment of the present application. An electronic device 600 provided in an embodiment of the present application includes: a processor 610 and a memory 620, the memory 620 storing machine readable instructions executable by the processor 610, the machine readable instructions when executed by the processor 610 perform the method as above.
The embodiment of the present application further provides a storage medium 630, where the storage medium 630 stores thereon a computer program, and the computer program is executed by the processor 610 to perform the blade image identification method as above.
The storage medium 104 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an alternative embodiment of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the embodiments of the present application, and all the changes or substitutions should be covered by the scope of the embodiments of the present application.
Claims (10)
1. A blade image recognition method is characterized by comprising the following steps:
obtaining a leaf image, the leaf image comprising: a leaf region and a background region;
removing the background area of the blade image to obtain the blade area of the blade image;
and identifying the blade region by using a pre-trained neural network model to obtain the classification of the blade image.
2. The method of claim 1, wherein the removing the background area of the leaf image comprises:
determining a leaf region of the leaf image;
removing a background area of the leaf image according to the leaf area.
3. The method of claim 2, wherein the determining the blade region of the blade image comprises:
determining a gray scale image of the leaf image;
determining a binary image of the blade image according to the gray level image;
and determining a blade area of the blade image according to the binary image.
4. The method of claim 2, wherein said removing a background region of the leaf image from the leaf region comprises:
according to
The leaf area removes a background area of the leaf image;
wherein i represents a color channel of the leaf image, and in the RGB color model, i is R, G, B, (x, y) represent abscissa of a pixel point of the leaf image, respectivelyAnd the ordinate, i.e. the coordinate of the pixel, fi(x, y) represents a pixel point value of coordinates (x, y) in the ith color channel, and C is a set of pixel points in the leaf region.
5. The method of claim 1, wherein the identifying the leaf region using a pre-trained neural network model to obtain the classification of the leaf image comprises
Performing convolution operation and pooling operation on the blade region by using at least one convolution layer and at least one pooling layer of the neural network model to obtain a first feature map;
calculating the first feature map by using at least one perception module group of the neural network model to obtain a second feature map;
and performing pooling, linear regression and normalization operations on the second feature map by using a pooling layer, a linear regression layer and a normalization index function layer of the neural network model to obtain the classification of the leaf image.
6. The method of claim 1, further comprising:
obtaining a plurality of plant leaf images and a plurality of plant leaf labels, wherein the plant leaf images are leaf images of plants, and the plant leaf labels are classifications corresponding to the plant leaf images;
and training a neural network by taking the plant leaf images as training data and the plant leaf labels as training labels to obtain the trained neural network model.
7. The method of any one of claims 1-6, wherein the neural network model is inclusion V3 or inclusion V4.
8. A blade image recognition apparatus, comprising:
an image obtaining module for obtaining a leaf image, the leaf image comprising: a leaf region and a background region;
the background removing module is used for removing a background area of the blade image and obtaining a blade area of the blade image;
and the classification identification module is used for identifying the blade area by using a pre-trained neural network model to obtain the classification of the blade image.
9. An electronic device, comprising: a processor and a memory, the memory storing machine-readable instructions executable by the processor, the machine-readable instructions, when executed by the processor, performing the method of any of claims 1-7.
10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911342356.2A CN110991454A (en) | 2019-12-23 | 2019-12-23 | Blade image recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911342356.2A CN110991454A (en) | 2019-12-23 | 2019-12-23 | Blade image recognition method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110991454A true CN110991454A (en) | 2020-04-10 |
Family
ID=70075878
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911342356.2A Pending CN110991454A (en) | 2019-12-23 | 2019-12-23 | Blade image recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110991454A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307983A (en) * | 2020-11-02 | 2021-02-02 | 深圳市中诺通讯有限公司 | Method and system for enhancing plant colors in image |
| CN112434631A (en) * | 2020-12-01 | 2021-03-02 | 天冕信息技术(深圳)有限公司 | Target object identification method and device, electronic equipment and readable storage medium |
| TWI837469B (en) | 2021-04-13 | 2024-04-01 | 宏正自動科技股份有限公司 | Method and device for removeing the background |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937698A (en) * | 2006-10-19 | 2007-03-28 | 上海交通大学 | Image processing method for image distortion automatic correction |
| CN101916382A (en) * | 2010-07-30 | 2010-12-15 | 广州中医药大学 | Method for recognizing image of plant leaf |
| CN103460244A (en) * | 2011-03-29 | 2013-12-18 | 富士通先端科技株式会社 | Biometric authentication apparatus, biometric authentication system, and biometric authentication method |
| CN108133186A (en) * | 2017-12-21 | 2018-06-08 | 东北林业大学 | A kind of plant leaf identification method based on deep learning |
| US10223610B1 (en) * | 2017-10-15 | 2019-03-05 | International Business Machines Corporation | System and method for detection and classification of findings in images |
| CN109977802A (en) * | 2019-03-08 | 2019-07-05 | 武汉大学 | Crops Classification recognition methods under strong background noise |
| CN110210434A (en) * | 2019-06-10 | 2019-09-06 | 四川大学 | Pest and disease damage recognition methods and device |
-
2019
- 2019-12-23 CN CN201911342356.2A patent/CN110991454A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1937698A (en) * | 2006-10-19 | 2007-03-28 | 上海交通大学 | Image processing method for image distortion automatic correction |
| CN101916382A (en) * | 2010-07-30 | 2010-12-15 | 广州中医药大学 | Method for recognizing image of plant leaf |
| CN103460244A (en) * | 2011-03-29 | 2013-12-18 | 富士通先端科技株式会社 | Biometric authentication apparatus, biometric authentication system, and biometric authentication method |
| US10223610B1 (en) * | 2017-10-15 | 2019-03-05 | International Business Machines Corporation | System and method for detection and classification of findings in images |
| CN108133186A (en) * | 2017-12-21 | 2018-06-08 | 东北林业大学 | A kind of plant leaf identification method based on deep learning |
| CN109977802A (en) * | 2019-03-08 | 2019-07-05 | 武汉大学 | Crops Classification recognition methods under strong background noise |
| CN110210434A (en) * | 2019-06-10 | 2019-09-06 | 四川大学 | Pest and disease damage recognition methods and device |
Non-Patent Citations (2)
| Title |
|---|
| 曹姝清等: "空间目标快速轮廓特征提取与跟踪技术", 《飞控与探测》 * |
| 郑一力等: "基于多特征降维的植物叶片识别方法", 《农业机械学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307983A (en) * | 2020-11-02 | 2021-02-02 | 深圳市中诺通讯有限公司 | Method and system for enhancing plant colors in image |
| CN112307983B (en) * | 2020-11-02 | 2024-03-26 | 深圳市中诺通讯有限公司 | Method and system for enhancing plant color in image |
| CN112434631A (en) * | 2020-12-01 | 2021-03-02 | 天冕信息技术(深圳)有限公司 | Target object identification method and device, electronic equipment and readable storage medium |
| TWI837469B (en) | 2021-04-13 | 2024-04-01 | 宏正自動科技股份有限公司 | Method and device for removeing the background |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110490850B (en) | Lump region detection method and device and medical image processing equipment | |
| CN111275046B (en) | Character image recognition method and device, electronic equipment and storage medium | |
| CN111079764B (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN111783749A (en) | Face detection method and device, electronic equipment and storage medium | |
| CN110070115B (en) | Single-pixel attack sample generation method, device, equipment and storage medium | |
| CN111738436A (en) | Model distillation method and device, electronic equipment and storage medium | |
| CN113919442B (en) | Tobacco maturity state identification method based on convolutional neural network | |
| US20200134382A1 (en) | Neural network training utilizing specialized loss functions | |
| CN109615614B (en) | Method for extracting blood vessels in fundus image based on multi-feature fusion and electronic equipment | |
| CN111680753A (en) | Data labeling method and device, electronic equipment and storage medium | |
| CN113435407B (en) | Small target identification method and device for power transmission system | |
| CN112418195A (en) | Face key point detection method and device, electronic equipment and storage medium | |
| CN105335760A (en) | Image number character recognition method | |
| CN112507897A (en) | Cross-modal face recognition method, device, equipment and storage medium | |
| CN110991454A (en) | Blade image recognition method and device, electronic equipment and storage medium | |
| CN110163206B (en) | License plate recognition method, system, storage medium and device | |
| CN110363103B (en) | Insect pest identification method and device, computer equipment and storage medium | |
| CN111652320B (en) | Sample classification method and device, electronic equipment and storage medium | |
| Shastry et al. | Classification of medicinal leaves using support vector machine, convolutional neural network and you only look once | |
| US11715288B2 (en) | Optical character recognition using specialized confidence functions | |
| CN115358952A (en) | Image enhancement method, system, equipment and storage medium based on meta-learning | |
| CN111695526B (en) | Network model generation method, pedestrian re-recognition method and device | |
| CN114283087A (en) | Image denoising method and related equipment | |
| CN117593610B (en) | Image recognition network training and deployment and recognition methods, devices, equipment and media | |
| CN112183212B (en) | Weed identification method, device, terminal equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200410 |
|
| RJ01 | Rejection of invention patent application after publication |