CN110148146B

CN110148146B - Plant leaf segmentation method and system by utilizing synthetic data

Info

Publication number: CN110148146B
Application number: CN201910439140.1A
Authority: CN
Inventors: 刘骥; 林艳
Original assignee: Chongqing University
Current assignee: Chongqing University
Priority date: 2019-05-24
Filing date: 2019-05-24
Publication date: 2021-03-02
Anticipated expiration: 2039-05-24
Also published as: CN110148146A

Abstract

The invention discloses a plant leaf segmentation method and a plant leaf segmentation system by utilizing synthetic data. The method comprises the following steps: constructing a plurality of blade three-dimensional models of different postures and different colors of the blade images based on the blade images; projecting the three-dimensional blade model to a two-dimensional plane to generate two-dimensional blade image data, and fusing the two-dimensional blade image data with different background images to obtain a training set; training the deep learning model through a training set to obtain a blade segmentation model; and inputting the image to be segmented containing the blade into a blade segmentation model to obtain a segmented blade image. The three-dimensional leaf model is used for generating segmentation data, leaf models with different shapes and colors are obtained through reconstruction of one plant leaf image and are fused with different backgrounds, a large number of training pictures and training labels are automatically generated to form a training set, energy consumption of manual labeling of images is reduced, the plant leaf image can be fully automatically segmented under a natural background condition, and the segmentation effect is good.

Description

Plant leaf segmentation method and system by utilizing synthetic data

Technical Field

The invention relates to the field of image segmentation, in particular to a plant leaf segmentation method and system by utilizing synthetic data.

Background

The study of plants, the identification, detection and segmentation of plant leaves is an important task for us in the computer vision of plants. However, the background conditions of plant leaves under natural conditions are complex, and the types of the plant leaves are color and complicated in texture, so that the identification, detection and segmentation of the plant leaves are a difficult task. The machine is able to identify leaves efficiently and is based on the ability to extract plant leaves from a natural background. Only if the leaves are extracted from the background, the subsequent analysis of the plant can continue. For example, whether the plant grows healthily or not and whether the plant suffers from diseases and insect pests or not is judged through the divided leaves. The leaf segmentation algorithm is also a common image segmentation method, and essentially divides an area on an image.

The principle of image segmentation is to divide an image into a plurality of parts according to the difference of feature information of the image, and the feature information of the same part is the same, and the feature information may be information such as color and shape. The current image segmentation algorithm roughly comprises the traditional image segmentation methods, such as region segmentation by utilizing threshold values, an algorithm using image edge detection, a segmentation method according to regions, segmentation using graph theory relevant knowledge, an energy functional-based method and the like. Based on the traditional image segmentation method, good effect can be obtained under the environment that the background is relatively simple or a single leaf is provided, but in the image with a complex background, the segmentation effect is greatly reduced. Moreover, many conventional methods need manual feature extraction and parameter setting, and cannot fully automatically segment, which obviously cannot meet actual requirements.

In recent years, with the improvement of relevant basic equipment and algorithms of big data and cloud computing, deep learning develops rapidly, and a large number of experts and scholars solve the problem of image segmentation or the problem of blade segmentation by using the relevant algorithms of deep learning. The related methods, which are also deep learning, currently achieve the best results on the public data set. However, training data is required in a deep learning-based mode, data sets published in the world are manually labeled at present, manually labeled data means that a large amount of manpower and financial resources are consumed, and manual labeling is easy to label errors at edges. Therefore, the lack of data seriously hinders the advancement of the deep learning method on the image segmentation algorithm. And to date, there is no segmented data set specifically targeted at plant leaves. Meanwhile, the development of deep learning is hindered by the lack of data.

Therefore, the traditional blade segmentation method needs to manually input some parameters, cannot complete full-automatic segmentation, or has poor segmentation effect. The segmentation method using deep learning can achieve full-automatic segmentation of leaves, but requires a large amount of training data.

Disclosure of Invention

The invention aims to at least solve the technical problems in the prior art, and particularly provides a plant leaf segmentation method and system by utilizing synthetic data.

In order to achieve the above object of the present invention, according to a first aspect of the present invention, there is provided a plant leaf segmentation method using synthetic data, comprising:

step S1, constructing blade three-dimensional models of different postures and different colors of a plurality of blade images based on the blade images;

projecting a three-dimensional model of a blade to a two-dimensional plane to generate two-dimensional image data of the blade, and fusing the two-dimensional image data of the blade with different background images to obtain a training set, wherein the training set comprises a plurality of training samples and sample labels corresponding to the training samples;

step S2, training the deep learning model through a training set to obtain a blade segmentation model;

step S3, the image to be segmented including the blade is input into the blade segmentation model, and the blade segmentation model outputs the segmented blade image.

The beneficial effects of the above technical scheme are: aiming at the problem of lack of training data of a plant leaf research algorithm under deep learning, a three-dimensional leaf model is used for generating segmentation data, leaf models with different shapes and different colors are obtained through reconstruction of one plant leaf image and are fused with different backgrounds, and a large number of training pictures and training labels are automatically generated to form a training set; the synthetic leaf data and the label are used for training the segmentation network to obtain the leaf segmentation model, energy consumption in manual image labeling is reduced, the problem that the current segmentation method based on deep learning is insufficient in training data is solved, the leaf segmentation model can fully automatically segment plant leaf images under the natural background condition, and the segmentation effect is good.

In order to achieve the above object, according to a second aspect of the present invention, there is provided a plant leaf segmentation system comprising a processor and an image providing unit, wherein the processor obtains an image to be segmented containing leaves from the image providing unit, and segments the leaf image from the image to be segmented according to the plant leaf segmentation method using synthesized data according to the present invention.

The beneficial effects of the above technical scheme are: the method for segmenting the plant leaves by utilizing the synthetic data has the advantages of fully automatically and accurately segmenting the leaf images from the images to be segmented.

It can be seen that, with the conventional blade segmentation method, some parameters need to be manually input, and full-automatic segmentation cannot be completed or the segmentation effect is poor. The segmentation method using deep learning can achieve full-automatic segmentation of leaves, but requires a large amount of training data. To address this problem, it is proposed herein to train a leaf segmentation network by using synthetic data.

Drawings

FIG. 1 is a schematic flow chart of a plant leaf segmentation method using synthetic data according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a blade profile line segment obtained by a grid in accordance with an embodiment of the present invention;

FIG. 3 is a schematic view of the shape of the blade profile before and after compression of the blade profile in one embodiment of the present invention;

FIG. 4 is a schematic diagram of a partial meshing process of a blade according to an embodiment of the present invention, wherein FIG. 4(a) is a schematic diagram of a blade skeleton and sampling points on a blade profile; FIG. 4(b) is a schematic diagram of the preliminary gridding result of the leaf surface part; FIG. 4(c) is a schematic diagram of the meshing result of further subdivision of the leaf surface part;

FIG. 5 illustrates a process for moving an operating point according to an embodiment of the present invention; wherein, fig. 5(a) is a schematic diagram of the operating point moving principle; FIG. 5(b) is a schematic diagram of the operating point moving process;

FIG. 6 is a schematic representation of a deformed planar mesh model of a blade according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of two-dimensional image data with different sizes and different viewing angles according to an embodiment of the present invention;

FIG. 8 is a schematic illustration of a specimen and a specimen label in accordance with an embodiment of the present invention;

FIG. 9 is a schematic view of pyramidal pooling of void space in one embodiment of the present invention;

FIG. 10 shows a single iteration of a CRF-RNN in accordance with an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, it is to be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention.

In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.

The invention discloses a plant leaf segmentation method using synthetic data, and in a preferred embodiment, a flow chart of the method is shown in fig. 1, and specifically comprises the following steps:

step S1, constructing a plurality of blade three-dimensional models with different postures and different colors of the blade images based on the blade images;

projecting the three-dimensional model of the blade to a two-dimensional plane to generate two-dimensional image data of the blade, and fusing the two-dimensional image data of the blade with different background images to obtain a training set, wherein the training set comprises a plurality of training samples and sample labels corresponding to the training samples;

In this embodiment, the leaf image is a picture containing plant leaves to be divided, and preferably, the plant leaves in the picture are as flat and complete as possible, and the background is relatively clean. The blade image can be acquired by a camera or can be obtained by scanning.

In a preferred embodiment, in step S1, the step of constructing a plurality of three-dimensional models of blades with different postures and different colors based on the blade image specifically includes:

step S11, acquiring a blade outline in the blade image; preferably, step S11 specifically includes:

step S111, carrying out gray processing on each pixel point in the blade image according to the following formula to obtain a gray image;

gray is 0.3R + 0.59G + 0.11B, wherein gray is the gray value of a pixel point, and R, G and B are the R channel value, the G channel value and the B channel value of the pixel point in the blade image respectively;

step S112, setting a gray threshold, and judging all pixel points in the gray image as follows to obtain a binary image:

if the gray value of the pixel point is greater than the gray threshold, the pixel point is considered as a leaf pixel point, namely the pixel point belongs to a leaf in the image, and if the gray value of the pixel point is less than or equal to the gray threshold, the pixel point is considered as a background pixel point;

and converting the gray level image into binaryzation to enable the blade outline to be clearer.

Step S113, extracting the blade outline from the binary image, wherein the specific process of the method for accurately extracting the blade outline is as follows:

setting a movable square on the binary image, wherein the square has four vertexes, each vertex stores a binary value of a pixel point of the binary image, the color of the vertex of the square corresponds to the binary value of the pixel point at the position of the vertex, if the binary value of the pixel point is 1, the vertex is black, and if the binary value of the pixel point is 0, the vertex is white;

and moving the blade part from the leftmost side of the blade part in the binary image along the blade edge in the anticlockwise direction until the blade part returns to the starting point, and connecting the midpoints of two sides adjacent to one or two black vertexes in each square to obtain a blade contour line segment, wherein all the blade contour line segments form a blade contour, and the endpoints of the blade contour line segment are the vertexes of the blade contour.

Fig. 2 is a schematic diagram showing an application scenario in which a blade contour line segment is obtained through squares, and in fig. 2, 16 cases are illustrated, where a blade contour line segment is not generated when 4 vertices of a square are not black, and a blade contour line segment is obtained by connecting midpoints of two sides adjacent to one or two black vertices in each square when there are black vertices. However, in

cases

8 and 16, two blade contour line segments are generated, and correction is required.

Therefore, when the black vertexes of the squares are two and are respectively positioned at the diagonal points of the squares, the generation of two blade contour line segments in one square is avoided by changing the size of the squares or neglecting one of the black vertexes.

Step S12, extracting veins as leaf skeletons; the blade framework comprises secondary and/or tripolar veins; the veins are in the structure of the plant leaves, support the whole leaf surface, play the role of a skeleton and keep the stability of the plant leaves; (ii) a The veins have a hierarchical structure like branches, with the largest main vein, the second lateral vein and the bottom with the smallest vessels. The skeleton of the leaf is generally obtained manually by selecting several points along the leaf vein of the leaf in the main vein and then connecting these points.

Step S13, forming a leaf surface part by the leaf contour and the area surrounded by the leaf framework, and meshing the leaf surface part to obtain a leaf plane mesh model; preferably, step S13 specifically includes:

step S131, compressing the vertex of the blade profile, preferably, step S131 specifically includes:

in step S1311, N1 is set as the set of vertices before the vertex compression on the blade profile, and N1 ═ N1₀,n1₁,n1₂,...,n1_m-1}，n1_pAnd n1_qThe sequence numbers of the vertex set N1 are p and q respectivelyP is more than or equal to 0 and less than or equal to m-1, q is more than or equal to 0 and less than or equal to m-1, and m is the number of vertexes of a vertex set N1; setting a set V as a vertex set after the vertex of the blade contour is compressed; let p be 0 and q be 2, let the vertex n1₀Adding the vertex set V into the vertex set V;

step S1312 determines whether or not q-m-1 is satisfied, and if so, ends the blade contour vertex compression processing, and if not, connects the vertices n1_pAnd n1_qForm a line segment l_pqCalculating all the vertices n1 satisfying p < r < q_rTo line segment l_pqR is a positive integer, and the maximum of r is selected and recorded as D_rmax；

Step S1313, if D_rmaxIf "T" is less than "q", the process returns to step S1312 by making q + 1;

if D is_rmaxT, will peak n1_rAdding the vertex set V, making p q-1 and q +1, and returning to step S1312;

t represents the maximum error value preset before and after the blade profile change, 0 < T < 50, and preferably T is 30.

By using the method for compressing the leaf contour, redundant nodes can be reduced, and the contour of the plant leaf is not influenced basically. In fig. 3, the blade contour shape after the compression process is a blade contour shape in which the number of redundant points decreases as T increases and the value of T is different, and before the compression, after the compression at T-10, after the compression at T-20, and after the compression at T-30 are sequentially arranged from left to right. The method for compressing the blade profile can effectively reduce redundant vertexes, avoid overlarge subsequent calculation amount and simultaneously ensure that the shape of the blade profile is not distorted.

S132, selecting a plurality of sampling points from the blade framework and the blade profile, and sequentially connecting the sampling points of the blade profile and the sampling points of the blade framework to form a plurality of polygons; preferably, as shown in fig. 4(a), a series of sampling points are manually selected on the blade framework and the blade profile, and the blade framework sampling points and the blade profile sampling points are sequentially connected to form a plurality of polygons.

Step S133, dividing the polygon into at least 2 triangles or quadrangles, and further subdividing the meshes formed by all or part of the triangles or quadrangles to obtain a blade plane mesh model. Step S133 specifically includes:

step S1331, dividing the polygon into at least 2 triangles by the Delaunay triangulation algorithm, and obtaining a mesh preliminary division result, as shown in fig. 4 (b).

Step S1332, marking the blade outline and the blade skeleton as boundaries;

step S1333, performing further subdivision processing on the mesh composed of all or part of triangles, for example, optionally subdividing the edge (with higher curvature) of the blade and the part where the blade is not flat enough, without subdividing all the blades, so as to reduce the computation amount. The Loop subdivision method is preferably, but not limited to, selected, and the following methods may be used, including:

step A: for the vertex v in any blade triangular mesh in step S1331, a set N is set as a set including the vertex v and all triangle vertices in the vertex v neighborhood, and each vertex in the set N is subdivided, specifically:

set the vertex v₀,v₁For two vertices in the set N of vertices, v₀≠v₁If the side v₀v₁Not a common edge, no new vertex is inserted;

if side v₀v₁For the common edge, the edge v is obtained₀v₁The included angle of the normal lines of the planes of the two triangles which are the common edge is greater than or equal to a first threshold value, and the included angle is set on the edge v₀v₁Do not insert a new vertex, if the included angle is smaller than the first threshold, at the side v₀v₁Up inserting new vertex v_new，v_newThe position coordinate calculation formula of (2) is as follows:

wherein v is_newRepresenting new vertex position coordinates; v. of₂And v₃Respectively represent by side v₀v₁Two triangles having a common side and a side v₀v₁Position coordinates of opposite vertexes(ii) a The first threshold value is a preset value, and the value range is as follows: 15 to 60 degrees, preferably 30 degrees;

and B: for any vertex v in step S1331, the flatness S of the vertex v is calculated by assuming that the set N is a set including the vertex v and all triangle vertices in the neighborhood of the vertex v_vIf S is_v< lambda, the position coordinates of the vertex v do not need to be adjusted, if S_vAnd lambda is larger than or equal to lambda, and the position coordinate of the vertex v is adjusted through the following formula:

wherein v' represents the position coordinate of the vertex v after adjustment; v. of_jRepresenting the jth vertex in the vertex set N; beta is a position adjustment coefficient, and beta is a position adjustment coefficient,

k is the number of vertexes in the vertex set N; flatness of vertex v

Normal vector of vertex v

N_jIs the normal vector of the jth triangle in k-1 triangles in the neighborhood of the vertex v; λ is a flatness threshold, 0 < λ < 1, preferably λ ═ 0.7;

step S1334 of generating a new mesh based on the new vertex, the adjusted vertex, and the original vertex inserted in step S1333;

and step S1335, executing step S1333 and step S1334 in an S-time loop, and further subdividing the grid, wherein S is a positive integer, preferably, S is 3, and the selection of the value of S is related to the required subdivision degree, and the subdivision degree can be judged by the size of the maximum grid unit.

Step S14, texture mapping processing is performed on the blade plane mesh model.

A blade plane mesh model is obtained, blades in the model only have rough shapes, no colors and no unevenness, the blades need to be further processed in order to obtain real blade information, and texture mapping is the most important method for obtaining the reality of the blades.

The texture mapping of the leaf maps leaf texture pixels (also called UV coordinates) into three-dimensional space pixels, which can be understood as that a mapping is pasted on a three-dimensional object, and the pasting of the texture mapping on the three-dimensional object needs a relatively complex algorithm, because the mapping is deformed, fortunately, the currently obtained leaf plane grid model is still in a two-dimensional plane, and the original leaf image can provide leaf texture materials, so that the texture can be directly pasted on the grid of the leaf. The texture mapping processing comprises the following specific processes:

extracting leaf texture from the leaf image, pasting the leaf texture on a leaf plane mesh model, and regarding texture coordinates { X } of any vertex v ═ X, y, z } on a mesh in the leaf plane mesh model_texture,Y_textureAnd z is:

wherein, X_min＝min{x₁,x₂,...,x_n}，X_max＝max{x₁,x₂,...,x_n}，Y_min＝min{y₁,y₂,...,y_n}，Y_max＝max{y₁,y₂,...,y_n}; { x, y, z } denotes the Euclidean coordinates of the vertex v in the blade plane mesh model, x₁,x₂,...,x_nRespectively representing the x-axis coordinate values, y, of all vertices on the mesh in the blade planar mesh model₁,y₂,...,y_nRespectively representing the coordinate values of the y axes of all vertexes on the grid in the blade plane grid model, and n represents the number of vertexes on the grid in the blade plane grid model; preferably, in order to avoid that some noise possibly existing outside the edge of the blade is also mapped into the grid model of the blade when texture mapping is carried out, the transparent channel of the texture is used for setting the pixels outside the blade to be transparent.

Step S15, selecting at least one operation point on the blade outline and/or the blade skeleton of the blade plane grid model after texture mapping, carrying out different movements on the operation point to enable the blade outline and the blade skeleton to generate different deformations, obtaining coordinate positions before and after the deformation of the operation point, and deforming the blade plane grid model after texture mapping by using a Laplace deformation algorithm to obtain a plurality of blade three-dimensional grid models with different postures; step S15 specifically includes:

step S151, selecting at least one operation point on the blade profile and/or the blade skeleton, as shown in FIG. 5(a), setting the operation point as Q_wThe number of the operation points is W, W represents the serial number of the operation points, and W is more than or equal to 1 and less than or equal to W; q_wIs Q_w-1Vector Q_wQ_w-1And vector Q_w-1Q_NForm a plane P, a vector Q_w-1Q_NIs the Z axis, and a vector Q is set_w-1Q_MPerpendicular to plane P, operating point Q_wIs moved around the vector Q_w-1Q_MRotating, wherein the rotating angle is set to be theta, theta is approximately equal to t, and the operating point Q_wThe position coordinates after the movement are:

Q(t)＝k(t)*(Q_w+t*(Q_N-Q_w))；

wherein k (t) ═ Q_w|/|Q_w+t*(Q_N-Q_w) If t is more than or equal to 0 and less than or equal to 1, different movement of the operation point can be realized by setting different parameters t, so that the blade profile and the blade framework are subjected to different deformations, and the vertex of the blade profile and the coordinates of the feature points on the blade framework after the different deformations are obtained; fig. 5(b) shows an example of the deformation of the blade profile or the blade skeleton, and a plurality of different θ can be set in the practical process, so that blades with different shapes can be obtained.

Step S152, set the vertex set of the mesh in the blade plane mesh model as V ═ V₁,v₂,...,v_n}，1≤i≤n，v_iThe coordinates of the ith vertex in the vertex set V are expressed, and the set N is set as the vertex V_iSet of adjacent vertices, v_jThe coordinate representing the jth vertex in the set N, vertex v_iThe laplace coordinates of (a) are:

wherein, w_ijIs an edge (v)_i,v_j) Has a weight of sigma_j∈Nw_ij＝1，w_ijα and β are edges (v)_i,v_j) In two adjacent triangles and side (v)_i,v_j) Two opposite corners; it can be seen that the laplacian coordinates can be regarded as information of the vertex, which represents an approximation of its average normal curvature. The laplacian coordinate can be regarded as an average weight of a point and all its neighboring points, so it can be used to express local information of a point.

The laplace coordinates can also be expressed by a matrix expression, and based on the formula, the matrix expression for obtaining the laplace coordinates is as follows:

l(x',y',z')＝L×V(x,y,z)；

wherein l (x ', y ', z ') is an n × 3 Laplace coordinate corresponding to a grid vertex coordinate in the blade plane grid model; v (x, y, z) represents the Euclidean coordinates of grid vertexes in the blade plane grid model and is an n multiplied by 3 order matrix; l is an nxn order laplacian matrix, which can be expressed specifically by the following piecewise expression:

e' is the vertex v_iAnd vertex v_jThe rank of the matrix L is n-1;

constructing an error function to minimize the mean square error of the features before and after deformation, wherein the error function is as follows:

Q_wis the first operation point w, Handles is the set of all operation points, k_wIs an operating point Q_wWeight of (0 < k)_w＜5,The error function value is minimized by solving the coordinates of V in the following equation:

h is a W multiplied by n order sparse matrix; each row containing only one non-zero element h_iiWhen v is equal to 1_iIs the operating point; h is_W×3The matrix is formed by the products of the coordinates of all the deformed operation points and the corresponding operation point weights;

the following equation is solved using the least squares method:

A^TAV＝A^Tb, obtaining the coordinates of V, namely the deformed coordinates of the vertex V in the blade plane mesh model, A^TAnd step S152 is repeatedly executed for the transposed matrix of the matrix a until transformed coordinates of all points in the blade plane mesh model are obtained, so as to obtain a plurality of blade three-dimensional mesh models with different postures. The Laplace deformation algorithm is used, so that the information such as the color, the texture and the like of the blade is unchanged as much as possible, and the detail information cannot be lost. Fig. 6 is a schematic diagram showing the deformation result of the grid of the blade.

And step S16, coloring the three-dimensional grid models of the blades in different postures to obtain a plurality of three-dimensional models of the blades in different postures and different colors.

After obtaining the three-dimensional mesh model of the blade, coloring the three-dimensional mesh model of the blade is needed to obtain the three-dimensional blade model with various colors. The color of the plant leaves is seen by the eyes because the leaves are specularly reflected, diffusely reflected and transmitted when light is irradiated on the leaves. Therefore, the optical characteristics of the leaves can be known, and the illumination model of the leaves can be modeled, so that a plurality of color models of the leaves are generated, and training sample data is enriched.

Acquiring a bidirectional reflection distribution function BRDF (bidirectional reflectance distribution function) and a bidirectional transmission distribution function BTDF (bidirectional transmittance distribution function) of a three-dimensional grid model of the blade at different postures and generating an irradiation rate texture; processing the irradiation rate texture by using a Gaussian fuzzy algorithm to obtain a processing result, and combining the processing result with a bidirectional reflection distribution function BRDF (bidirectional reflectance distribution function) and a bidirectional transmission distribution function BTDF to obtain coloring models of three-dimensional grid models of the blades in different postures; and randomly adjusting illumination and blade parameters in the coloring model of the three-dimensional grid model of the blade in different postures to obtain a plurality of three-dimensional blade models in different postures and different colors.

Step S16 specifically includes:

step S161, obtaining a reflection component fr of the coloring model of the plant leaf:

wherein fr_diffIs a diffuse reflection component; fr_specIs a specular component;

diffuse reflection component fr_diffThe calculation formula of (2) is as follows:

fr_diff＝k_dD_rgbeta (x), wherein k_dThe light intensity adjustment parameters are set by a user; η (x) denotes the normalized diffuse reflectance texture, D_rgbThe RGB vector of the bidirectional reflectance distribution function BRDF of the plant leaf is determined by chlorophyll, carotene and structural parameters of the leaf;

specular component fr_specThe calculation formula of (2) is as follows:

wherein F is a Fresnel coefficient, and when the angles of the sight line and the blades are different, the observed reflection effects are different; d_beckmannIs Beckmann distribution parameter, G is shielding term, theta_iAnd theta_vRespectively representing the incident angle and the reflection angle of the light;

step S162, obtaining a transmission component ft of the coloring model of the plant leaf:

ft＝k_tT′_rgbγ(x)e^-h；

wherein, T'_rgb(0.9g, g,0.2g) g represents T'_rgbOf green light component, k_tFor adjusting diffuse reflectionParameters of light intensity and parameters of parameters set by a user; gamma (x) is the normalized transmission texture; h represents the thickness of the blade;

step S163, generating an radiance texture by using the reflection component fr and the transmission component ft of the blade; processing the generated irradiation rate texture by using a Gaussian fuzzy algorithm to obtain a sub-surface dispersion component of the blade; obtaining a coloring model by combining the sub-surface dispersion component, the reflection component fr and the transmission component ft; in the convolution process of the Gaussian blur algorithm, the invention uses 7 convolutions, the size of the convolution kernel is 12, and the centers are {0.006,0.061,0.242,0.383,0.242,0.061,0.006}, respectively.

And S164, randomly adjusting the color and intensity of light in the coloring model and the transmissivity of the blades to obtain a plurality of color models, and overlapping the color models with different three-dimensional grid models of the blades to obtain a plurality of three-dimensional models of the blades with different postures and different colors. The illumination and various parameters of the leaves, such as the color and the intensity of the light, the transmissivity of the leaves and the like, are randomly adjusted and controlled within a certain range, and for the same leaf model, a plurality of leaf three-dimensional models with different colors can be obtained in the mode.

Step S1 further includes: and projecting the three-dimensional model of the blade to a two-dimensional plane to generate two-dimensional image data of the blade, and fusing the two-dimensional image data of the blade with different background images to obtain a training set, wherein the training set comprises a plurality of training samples and sample labels corresponding to the training samples.

The three-dimensional leaf model cannot be directly trained in the segmentation network model of the deep learning model, so that segmentation images and sample labels are required, the sample labels are the same as the sample images in the corresponding training set in size and are black and white, and the white area represents the leaf position information in the corresponding sample image. The method comprises the steps of firstly projecting a three-dimensional model of the blade onto a two-dimensional plane, generating data only with the blade without background information, generating the data too single and needing to be fused with complex backgrounds, fusing different backgrounds and two-dimensional blade graphic data by utilizing a Laplace pyramid algorithm, and solving the problem that the blade edge is very sharp when the blade is directly fused into a background image.

Projecting the three-dimensional model of the blade onto a two-dimensional plane is actually converted from a world coordinate system to a pixel coordinate system by a conversion formula:

wherein the pixel coordinate system is a coordinate system of the position of the depicted pixel in the two-dimensional image and is marked as O₀-uv with the abscissa and ordinate uv axes; image coordinate system-which is the physical location of a pixel on a plane, usually in millimeters-defines the image coordinate system O₁-xy, the x-axis being parallel to the u-axis in the pixel coordinate system and the y-axis being parallel to the v-axis in the pixel coordinate system; the camera coordinate system is a point of space relative to the camera optical center O_cThree-dimensional coordinates of (a). Suppose the camera coordinate system is O_c-X_cY_cZ_cThe shooting direction of the camera is Z_cThe shooting direction is then perpendicular to the pixel coordinate system and intersects at O₁And (4) point. X_cThe axis is the same as the direction of the u-axis in the image plane coordinate system, Y_cThe axis is the same as the direction of the v-axis in the image plane coordinate system. The world coordinate system can represent the coordinate position of each vertex of the grid blade in the three-dimensional world, and the coordinate system of the world coordinate system is O_w-X_wY_wZ_wIts coordinate position is relative to the position of other objects. f. of_xAnd f_yIs the focal length of the camera; k is a matrix of size 3X 3, which is the in-camera parameter matrix, f_x、f_y、u_oAnd v_oAre camera intrinsic parameter elements, each camera has its own intrinsic parameters, and the intrinsic parameters have a relationship with the camera itself. R and t are camera external parameter elements, R is a rotation parameter, and t is a translation parameter, which is equivalent to the posture of the camera relative to the world coordinate system.

By adjusting the external parameters of the camera, that is, the rotation parameters and the translation parameters, thousands of two-dimensional image data can be generated for one blade three-dimensional model, as shown in fig. 7, two-dimensional image data with different sizes and inconsistent view angles generated by one three-dimensional blade model are shown.

The background of the two-dimensional graph data of the leaf is pure color and cannot be used for manufacturing segmentation data, and background pictures are randomly selected to be fused in order to synthesize a segmentation data set, namely a sample of a training set. Since the pixel coordinates in the leaf two-dimensional graph data are known, the exemplar label can be made directly. The specific process of fusion is as follows:

firstly, constructing two-dimensional graph data of plant leaves and a Gaussian pyramid of a background to be fused, and then constructing a Laplacian pyramid of a B layer.

And secondly, setting a mask consistent with the shape and the size of the blade in the two-dimensional graphic data of the blade, and indicating the position to be fused.

And thirdly, fusing the Laplacian pyramid of the two-dimensional image data of the blade and the background picture according to the mask to obtain a new pyramid.

And fourthly, reconstructing the result obtained in the third step, wherein the operation method is the same as that of the laplacian pyramid but is up-sampling, and then fusing the images of all the layers to obtain a leaf two-dimensional graph image (namely, a sample) and a sample label of the synthetic background, as shown in fig. 8, wherein the first row and the second row are sample images, and the second row and the fourth row are sample labels corresponding to the sample images.

The deep learning model in the invention is improved based on a full convolution neural network (FCN) model as follows: the method comprises the steps that hole convolution is used for solving the problem of poor segmentation effect caused by a reduced characteristic diagram in a full convolution neural network (FCN) model; in order to solve the multi-scale problem of the object, a cavity convolution layer structure (ASPP) with a parallel structure is used; the edge information from the segmentation is not accurate enough and Conditional Random Field (CRF) techniques are employed. And finally, a network deep Lab-ASPP model integrating the FCN, the ASPP and the CRF is introduced, and the segmentation effect reaches unprecedented height.

The full convolution neural network FCN is subjected to convolution and pooling for multiple times, the image is smaller and smaller, the resolution is lower and lower, in order to obtain a segmentation image with the same size as the original image, the image is subjected to up-sampling by using deconvolution, and the deconvolution is that the convolution carries out reverse operation in forward or backward propagation.

On the basis of up-sampling by deconvolution, a layer-skipping structure is used, i.e. the result of combining the upper network layers is fused to obtain a finer segmentation result, and such a structure is called a layer-skipping structure. Such as VGG-16 networks, for a total of 5 pool operations, after each pool operation, the image becomes 1/2 size of the previous layer. After the 3 rd pool operation, the image becomes 1/8 original, at which time its results are preserved, 1/16 original after the fourth pool operation, and 1/32 original after the last pool operation, at which time the results are called heatmaps (heatmaps). Following the 3 convolutional layers, if the 1/32 size heatmap is up-sampled 32 times directly at this point, the results are not fine enough because some detail features are lost during pooling. At this point we iterate forward to fuse the feature map of 1/8, and the 2-fold upsampling of the feature map of 1/16 and the 4-fold upsampling of 1/32 to obtain a segmentation result of 1/8.

Removing the subsequent pooling layer in the full convolution neural network FCN, replacing at least one convolution layer with a cavity convolution layer by using the cavity convolution, avoiding the characteristic loss caused by down-sampling, wherein the calculation formula of the cavity convolution is as follows:

for a two-dimensional output feature y, it can be represented by the feature x containing the input as:

r is the sampling step length controlling the input features; it can be understood that 0 is inserted between every two features of the input and then convolution is carried out; w is the convolution kernel and k is the convolution kernel size, which can be 3 x 3 or 5 x 5.

The problem of multiscale property for target blades, that is, the problem of different sizes of blades on an image. A plurality of parallel hole convolution layers with different sampling rates are used for solving the multi-scale segmentation problem. This amounts to using multiple different convolutions on the original image and obtaining more contextual information on the image at multiple scales. Such parallel structured void convolution layers are called ASPP modules. Fig. 9 shows a rough flow of the void space pyramid, where 4 parallel void convolutions sample the features of the previous layer at the same time, and the parameter r of the void convolution of each branch is different to take different features. In general, the smaller r, the more local features are extracted, and the larger r, the more global information is extracted.

Aiming at the problem that the plant leaf can be segmented by the full convolution neural network FCN and the segmentation result at the edge part of the leaf is not good enough, the invention adopts a Conditional Random Field (CRF) and takes the relation between pixels into consideration, thereby improving the defect that the FCN does not pay attention to the information of adjacent pixels and further processing the segmentation result.

Random variable X ═ X₁,X₂,...,X_NDenotes a data set, Y ═ Y₁,Y₂,...,Y_NIs the label and N is the size of the data set. On blade division, X_iIndicating the pixel value of the ith pixel, Y_iIs the label value of the ith pixel, and the value is L ═ L₁,l₂,...,l_k}. For graph G ═ V, E, Y ═ Y_i) Wherein G is a probability undirected graph, V represents random variables, E represents the dependency relationship between the random variables, y_iIs a random variable of the ith pixel, the conditional probability of Y obeys the formula P (Y) if given X_i|X,y_i,j≠i)＝P(y_i|X,y_iJ: i), which is the undirected edge from j to i in G, i.e. the Markov property, (X, Y) constitutes a conditional random field P (Y | X), which can be expressed as Gibbs distribution:

where Z (X) is a normalizing term, the potential function for each group c on graph G is

For the result y ∈ L of the segmentation marker, the Gibbs energy thereof can be expressed by the following equation:

p (y | X) is the posterior probability, and the best segmentation marker y is obtained in order to obtain the best segmentation result^*I.e. the objective function P (y | X) is maximum, then Gibbs energy is minimum, i.e.:

y^*＝argminE(y|X)；

the essence of constructing the CRF model is the weighted summation of all potential functions, here only the first-order potential and the second-order potential are considered, and the corresponding Gibbs energy function is:

used herein as psi_c(y_c) Instead of psi_c(y_c| X), then the above expression can be converted to:

ψ_i(y_i) Is a first order potential function, and expresses that the pixel value of the ith pixel point is X_iThe degree of difference between this point and its class, which is only related to the pixel itself. Combining the output results of FCN,. psi_i(y_i) Can be calculated in detail as:

ψ_i(y_i)＝-ln(P_FCN(y_i＝l_k))；P_FCN(y_i＝l_k) When the FCN is used for image segmentation, the predicted value of the ith pixel point is l_kThe probability of (c).

But the FCN model does not take into account the consistency of label assignment and the smoothness of the split edges. To further utilize the pixel-to-pixel information between images, if the pixel value X of the adjacent pixel point_i,X_jCloser together, they may be considered to belong to the same class with a greater probability, so a second order potential function is introduced herein to take into account pixel-to-pixel information. The second order potential function may be defined as a weighted gaussian function:

wherein, mu (y)_i,y_j) Is a penalty term, and the penalty is larger when the similarity of the two characteristics is higher. For an image, the closer the two pixels are, the more the pixel values try, but the different categories, the higher their penalty. When the labels of the ith pixel point and the jth pixel point are different, the number is 1, otherwise, the number is 0. w is a^(m)The corresponding weights of the gaussian kernel are represented. Each k is^mUsing Gaussian kernels

Λ^(m)An accuracy matrix being a Gaussian kernel function, an eigenvector f_i,f_jThe pixel coordinates and RGB values can be used for composition, further simplification can be achieved, and the following can be obtained:

p_iis the position of the ith pixel, p_jIs the location of the jth pixel, I_iIs the color of the ith pixel, I_jIs the color of the jth pixel, λ_αAnd λ_βIs the distance of a pixel in color and position from other pixels on the image. To lowerLow computational complexity, using mean field approximation, to approximate P (YX) with Q (X),

representing the relative entropy, which is equivalent to the difference between the information entropies of the two probability distributions; and the KL divergence can be satisfied, and by combining a Lagrange multiplier method, the following can be solved:

as can be seen from the above equation, the posterior probability calculation for P (Y | X) is transformed into the pair Q_i(X_i) F (i) is exactly the bilateral filtering operation on the ith pixel.

As shown in FIG. 10, the mean field algorithm of conditional random field CRF is transformed into a generic layer of neural network, all parameters being trained. The parameter I is an input image, an output result Q is obtained through a subsequent stage by combining an output result U of the FCN_outAs input Q for the next time_inAnd continuously iterating until convergence.

An initialization stage:

in the first iteration process, the computational expression of the output result Q of the ith pixel belonging to the ith category is:

where U is the output of FCN, U_i(l) Representing the output result of the ith layer FCN; z_iIs a normalization constant, this layer can be seen as the Softmax layer of the convolutional neural network CNN.

A probability calculation stage:

this stage is the first stage in the iterative process, applying gaussian filtering to the Q value. Deriving filter coefficients from image features, e.g.Pixel coordinates and RGB values.

And

the top marks are only used for distinguishing and have no practical significance; because the CRF may be fully connected, the difficulty in calculating the Gaussian convolution kernel is too large, so that the Q value can be obtained by approximate calculation according to the following formula, and Gaussian filtering is applied, namely the convolution with the Gaussian kernel in the feature space is obtained, and the expression is as follows:

and a weight adjusting stage:

the phase directly performs weighted summation on the probability calculation phase, and the equation is as follows:

m is the number of Gaussian kernel layers.

And (3) a category transformation stage:

this layer is a class conversion layer and can be considered as a 1 x 1 convolutional layer:

and (3) probability integration stage:

this layer combines the output of the FCN and the output of the class conversion to integrate the probability of assigning a label to the ith pixel, which can be expressed by the mathematical equation:

a normalization stage:

normalizing the input to facilitate the next iteration:

Q_ithe normalization treatment is carried out on the mixture,

is probability integration.

The mean field approximation may be implemented by repeating the upper network layer, with the input to the network layer for each iteration being the original form of the last iteration network layer output and a first potential. This is equivalent to considering iterative mean-field algorithmic reasoning as a Recurrent Neural Network (RNN), whose iterative algorithm can be expressed as follows:

H₂(t)＝f_θ(U,H₂(t-1),I),0<t≤T；

the three formulas are mean field iterative algorithm of the CRF in the recurrent neural network, t is iterative times of the recurrent neural network, and U is a result of the full convolution network. Experiments show that when T is about 10, the mean field iterative algorithm is basically converged, and the segmentation effect is optimal. Y (T) is the result output of the Tth time.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A plant leaf segmentation method using synthetic data, comprising:

step S3, inputting the image to be segmented containing the blade into a blade segmentation model, and outputting the segmented blade image by the blade segmentation model;

in step S1, the step of constructing a plurality of different-pose, different-color blade three-dimensional models of the blade image based on the blade image includes:

step S11, acquiring a blade outline in the blade image;

step S12, extracting veins as leaf skeletons; the blade skeleton comprises secondary and/or tripolar veins;

step S13, forming a leaf surface part by the leaf contour and the area surrounded by the leaf framework, and meshing the leaf surface part to obtain a leaf plane mesh model;

step S14, performing texture mapping processing on the blade plane mesh model, specifically:

wherein, X_min＝min{x₁,x₂,...,x_n}，X_max＝max{x₁,x₂,...,x_n}，Y_min＝min{y₁,y₂,...,y_n}，Y_max＝max{y₁,y₂,...,y_n}; { x, y, z } denotes the Euclidean coordinates of the vertex v in the blade plane mesh model, x₁,x₂,...,x_nRespectively representing the x-axis coordinate values, y, of all vertices on the mesh in the blade planar mesh model₁,y₂,...,y_nRespectively representing the coordinate values of the y axes of all vertexes on the grid in the blade plane grid model, and n represents the number of vertexes on the grid in the blade plane grid model;

step S15, selecting at least one operation point on the blade outline and/or the blade skeleton of the blade plane grid model after texture mapping, carrying out different movements on the operation point to enable the blade outline and the blade skeleton to generate different deformations, obtaining coordinate positions before and after the deformation of the operation point, and deforming the blade plane grid model after texture mapping by using a Laplace deformation algorithm to obtain a plurality of blade three-dimensional grid models with different postures;

2. The method for plant leaf segmentation using synthetic data according to claim 1, wherein the step S11 includes:

if the gray value of the pixel is greater than the gray threshold, the pixel is considered as a leaf pixel, and if the gray value of the pixel is less than or equal to the gray threshold, the pixel is considered as a background pixel;

step S113, extracting a blade contour from the binary image, specifically:

setting a movable square on the binary image, wherein the color of the vertex of the square corresponds to the binary value of a pixel point at the position of the vertex, if the binary value of the pixel point is 1, the vertex is black, and if the binary value of the pixel point is 0, the vertex is white;

starting to move along the blade edge in the counterclockwise direction from the leftmost side of the blade part in the binary image until the blade part returns to the starting point, connecting the midpoints of two sides adjacent to one or two black vertexes in each square to obtain a blade contour line segment, wherein all the blade contour line segments form a blade contour, and the end points of the blade contour line segment are the vertexes of the blade contour;

when the black vertexes of the squares are two and are respectively positioned at the diagonal points of the squares, the generation of two blade contour line segments in one square is avoided by changing the sizes of the squares or neglecting one of the black vertexes.

3. The method for plant leaf segmentation using synthetic data according to claim 1, wherein the step S13 includes:

s131, compressing the top point of the blade outline;

s132, selecting a plurality of sampling points from the blade framework and the blade profile, and sequentially connecting the sampling points of the blade profile and the sampling points of the blade framework to form a plurality of polygons;

step S133, dividing the polygon into at least 2 triangles or quadrangles, and further subdividing the meshes formed by all or part of the triangles or quadrangles to obtain a blade plane mesh model.

4. A plant leaf segmentation method using synthetic data according to claim 3, wherein the step S133 includes:

step S1331, dividing the polygon into at least 2 triangles by a Delaunay triangulation algorithm;

step S1332, marking the blade outline and the blade skeleton as boundaries;

step S1333, performing further subdivision processing on the mesh composed of all or part of the triangles, specifically including:

set the vertex v₀，v₁For two vertices in the set N of vertices, v₀≠v₁If the side v₀ v₁Not a common edge, no new vertex is inserted;

if side v₀ v₁For the common edge, the edge v is obtained₀ v₁The included angle of the normal lines of the planes of the two triangles which are the common edge is greater than or equal to a first threshold value, and if the included angle is greater than or equal to the first threshold value, the included angle is on the edge v₀ v₁Do not insert a new vertex, if the angle is smaller than a first threshold, at the edge v₀v₁Up inserting new vertex v_new，v_newThe position coordinate calculation formula of (2) is as follows:

wherein v is_newRepresenting new vertex position coordinates; v. of₂And v₃Respectively represent by side v₀ v₁Two triangles having a common side and a side v₀ v₁The position coordinates of the opposite vertices; the first threshold is a preset value, and the value range is as follows: 15 to 60 degrees;

k is the number of vertexes in the vertex set N; flatness of vertex v

Normal vector of vertex v

N_jIs the normal vector of the jth triangle in k-1 triangles in the neighborhood of the vertex v; lambda is a flatness threshold value, and lambda is more than 0 and less than 1;

and step 1335, circularly executing step 1333 and step 1334S times to realize the further subdivision of the grid, wherein S is a positive integer.

5. A plant leaf segmentation method using synthetic data according to claim 3 wherein the step S131 comprises:

in step S1311, N1 is set as the set of vertices before the vertex compression on the blade profile, and N1 ═ N1₀,n1₁,n1₂,...,n1_m-1}，n1_pAnd n1_qThe two vertexes with the serial numbers p and q in the vertex set N1 are respectively, p is more than or equal to 0 and less than or equal to m-1, q is more than or equal to 0 and less than or equal to m-1, and m is the number of vertexes of the vertex set N1; setting a set V as a vertex set after the vertex of the blade contour is compressed; let p be 0 and q be 2, let the vertex n1₀Adding the vertex set V into the vertex set V;

and T represents a preset allowable maximum error value before and after the blade profile change, and T is more than 0 and less than 50.

6. The method for plant leaf segmentation using synthetic data according to claim 1, wherein the step S15 specifically includes:

step S151, selecting at least one operation point on the blade profile and/or the blade framework, and setting the operation point as Q_wThe number of the operation points is W, W represents the serial number of the operation points, and W is more than or equal to 1 and less than or equal to W; q_wIs Q_w-1Vector Q_wQ_w-1And vector Q_w-1Q_NForm a plane P, a vector Q_w-1Q_NIs the Z axis, and a vector Q is set_w-1Q_MPerpendicular to plane P, operating point Q_wIs moved around the vector Q_w-1Q_MRotating, wherein the rotating angle is set to be theta, theta is approximately equal to t, and the operating point Q_wThe position coordinates after the movement are:

Q(t)＝k(t)*(Q_w+t*(Q_N-Q_w))；

wherein k (t) ═ Q_w|/|Q_w+t*(Q_N-Q_w) L, t is more than or equal to 0 and less than or equal to 1, different movement of the operation point can be realized by setting different parameters t, so that the blade profile and the blade framework are subjected to different deformations, and the vertex of the blade profile and the characteristics of the blade framework after different deformations are obtainedCoordinates of the feature points;

wherein, w_ijIs an edge (v)_i,v_j) Has a weight of sigma_j∈Nw_ij＝1，w_ijα and β are edges (v)_i,v_j) In two adjacent triangles and side (v)_i,v_j) Two opposite corners;

based on the above formula, the matrix expression for obtaining the laplacian coordinates is:

l(x',y',z')＝L×V(x,y,z)；

wherein l (x ', y ', z ') is an n × 3 Laplace coordinate corresponding to a grid vertex coordinate in the blade plane grid model; v (x, y, z) represents the Euclidean coordinates of grid vertexes in the blade plane grid model and is an n multiplied by 3 order matrix; l is an nxn order laplacian matrix, specifically:

e' is the vertex v_iAnd vertex v_jThe rank of the matrix L is n-1;

constructing an error function, wherein the error function is as follows:

Q_wis the first operation point w, Handles is the set of all operation points, k_wIs an exerciseAs point Q_wWeight of (0 < k)_w< 5, minimizing the error function value by solving the coordinates of V in:

solving using a least squares method:

A^TAV＝A^Tb, obtaining the coordinates of V, namely the deformed coordinates of the vertex V in the blade plane mesh model, A^TAnd step S152 is repeatedly executed for the transposed matrix of the matrix a until transformed coordinates of all points in the blade plane mesh model are obtained, so as to obtain a plurality of blade three-dimensional mesh models with different postures.

7. The method for plant leaf segmentation using synthetic data according to claim 1, wherein the step S16 includes:

fr_diff＝k_dD_rgbeta (x), wherein k_dThe light intensity adjustment parameters are set by a user; η (x) denotes the normalized diffuse reflectance texture, D_rgbThe RGB vector of bidirectional reflectance distribution function BRDF of plant leaf is composed of chlorophyll, carotene and knot of leafDetermining a structural parameter;

specular component fr_specThe calculation formula of (2) is as follows:

ft＝k_tT′_rgbγ(x)e^-h；

wherein, T'_rgb(0.9g, g,0.2g) g represents T'_rgbOf green light component, k_tThe parameters for adjusting the diffuse reflection parameters and the light intensity are set by a user; gamma (x) is the normalized transmission texture; h represents the thickness of the blade;

step S163, generating an radiance texture by using the reflection component fr and the transmission component ft of the blade; processing the generated irradiation rate texture by using a Gaussian fuzzy algorithm to obtain a sub-surface dispersion component of the blade; obtaining a coloring model by combining the sub-surface dispersion component, the reflection component fr and the transmission component ft;

and S164, randomly adjusting the color and intensity of light in the coloring model and the transmissivity of the blades to obtain a plurality of color models, and overlapping the color models with different three-dimensional grid models of the blades to obtain a plurality of three-dimensional models of the blades with different postures and different colors.

8. A plant leaf segmentation system comprising a processor and an image providing unit, wherein the processor obtains an image to be segmented containing leaves from the image providing unit, and segments a leaf image from the image to be segmented according to the plant leaf segmentation method using synthesized data according to any one of claims 1 to 7.