KR102131243B1 - Plant Area Extraction System and Method Based on Deep Running and Connectivity Graphs - Google Patents

Abstract

A plant area extraction system is provided. The plant region extraction system includes a plant image generator that photographs a target plant to generate a plant image; A first plant region extraction unit for generating first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And a second plant region extraction unit that generates second plant region extraction data that maximizes interconnection between a plurality of pixels of the plant image by utilizing the plant image and the first plant region extraction data.

Description

Plant Area Extraction System and Method Based on Deep Running and Connectivity Graphs

The present invention relates to a plant area extraction system and method, and specifically to a system and method for extracting a plant area based on deep learning and connectivity graphs.

[Explanation on national support R&D]

This study was supported by the Rural Development Administration under the management of the next generation BioGreen 21 project, "Development of Technology for Estimating the Expression of Core Image-Based Crops" (Task ID: 1395056227, Detailed Task Number: PJ012281022018, Task Execution Organization: Korea Institute of Science and Technology) It is made by.

Phenonomics is a study of cell function based on examining the phenotype of living organisms such as RNA and protein and measuring the amount of them. In order to measure and document phenotypic information, individual analysis of objects is required, and when the object of expression morphology is a plant, individual analysis of plant leaves is required.

For the individual analysis of such plant leaves, a labor-intensive method of manually identifying individual plant leaves has been performed in the past, but recently, a method of deriving the characteristics of individual plant leaves from the acquired image by studying the entire plant image has been studied. .

In the method of deriving the characteristics of individual plant leaves from the plant image, a process of distinguishing the plant region and the background region in the plant image may be important. Since the analysis of the characteristics of individual plant leaves is performed based on the divided plant image, if the classification and extraction of the plant area are not performed accurately, the accuracy of the results that are performed afterwards is not guaranteed. Particularly, in the case of monocotyledonous plants, since the leaves are long and thin, it is not easy to distinguish the plant region from the background region, and in the divided plant region, there may be a hole that does not exist in the actual plant, or the leaf may be cut off. Can occur. That is, in a plant image having an elongated leaf, when separating a plant region from a background region, a separation technique capable of ensuring maximum connectivity of plant leaves is required.

The present invention has been devised to solve the above-mentioned problems, and provides a system and method for extracting plant regions based on deep learning and connectivity graphs that can classify plant regions with elongated leaves in a plant image as background regions.

A plant region extraction system according to an embodiment of the present specification includes a plant image generator configured to generate a plant image by photographing a target plant; A first plant region extraction unit for generating first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And a second plant region extraction unit that generates second plant region extraction data that maximizes interconnection between a plurality of pixels of the plant image by utilizing the plant image and the first plant region extraction data.

In one embodiment, the machine learning model based on the convolutional neural network of the first plant region extraction unit includes a convolution step (Conv) for generating a feature map for the plant image, and the resolution of the feature map. It includes a transition down (TD) step to reduce and a transition up (TU) step to increase the resolution of the feature map, the data input to the transition down can be skipped (skip connetion) with the data output from the transition up. have.

In one embodiment, the convolutional neural network-based machine learning model of the first plant region extractor includes a density block including a plurality of convolution steps, and within the density block The feature maps generated through convolution may be accumulated so that the initial stage information is not erased and continues with the later stage information.

In one embodiment, when the plant image is larger than the size of the image that can be processed by the first plant region extracting unit, the first plant region extracting unit divides the plant image into the size of the processable image, thereby generating a plurality of sub-plants. An image is generated, first sub plant region extraction data for each of the plurality of sub plant images are respectively generated, and the first first plant region extraction data is generated by combining each of the generated first sub plant region extraction data. can do.

In one embodiment, if the first plant region extraction data of each of the first sub-plant region extraction data includes overlapping pixel regions of the plurality of sub-plant images, the first plant region extraction unit is highest for the overlapping pixel regions. The first plant region extraction data may be combined by selecting first sub-plant region extraction data having a probability value.

In one embodiment, the second plant region extracting unit forms a graph of a probability value for each pixel corresponding to the first plant region extraction data and an interconnection graph between the plurality of pixels and between connected pixels. In consideration of the mutual influence, the second plant region extraction data for determining whether each pixel is a plant region may be generated.

In one embodiment, the mutual influence may be determined in consideration of a distance difference between connected pixels and similarity of R, G, and B values of the plant image.

In one embodiment, the plant may be monocotyledonous plants.

Plant region extraction method according to another embodiment of the present invention, the plant region extraction system, generating a plant image by photographing a target plant; Generating, by the plant region extraction system, first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And the plant region extracting system, using the plant image and the first plant region extracting data, generating second plant region extracting data that maximizes interconnection between a plurality of pixels of the plant image.

In one embodiment, the convolutional neural network-based machine learning model includes a convolution step (Conv) of generating a feature map for the plant image, and a transition down (TD) of reducing the resolution of the feature map. ) And a transition up (TU) step of increasing the resolution of the feature map, and data input to the transition down may be skipped concatenated with data output from the transition up.

In an embodiment, the machine learning model based on the convolutional neural network includes a density block including a plurality of convolution steps, and features generated through convolution within the density block The map is accumulated so that the initial stage information is not erased, but may be continued with the later stage information.

In one embodiment, generating the first plant region extraction data may include: dividing the plant image into a size of the processable image to generate a plurality of sub-plant images; Generating first sub plant region extraction data for each of the plurality of sub plant images; And combining the generated first sub-plant region extraction data to generate the first plant region extraction data.

In one embodiment, the step of generating the first plant region extraction data by combining the generated first sub plant region extraction data may include generating the first sub plant region extraction data of the plurality of sub regions. When the overlapping pixel region of the plant image is included, the first plant region extraction data may be combined by selecting the first sub-plant region extraction data having the highest probability value for the overlapping pixel region.

In one embodiment, the step of generating the second plant region extraction data forms both a probability value that each pixel according to the first plant region extraction data corresponds to the plant region and an interconnection graph between the plurality of pixels. And, considering the mutual influence between the connected pixels, it is possible to generate the second plant region extraction data for determining whether each pixel is a plant region.

In one embodiment, the mutual influence may be determined in consideration of a distance difference between connected pixels and similarity of R, G, and B values of the plant image.

In one embodiment, the plant may be monocotyledonous plants.

A computer program according to another embodiment of the present invention is stored in a medium in combination with hardware to execute the above-described plant region extraction method.

The plant region extraction system according to an embodiment of the present invention distinguishes a plant region and a background region from a plant image through a convolutional neural network, but the result data is input through a deconvolution process. It is guaranteed to have the same resolution as the image.

In addition, through a conditional probability model called a density conditional random field, it is possible to generate result data that maximizes the connectivity of pixels in a plant image.

Therefore, a plant region with elongated leaves can be more precisely extracted from the plant image.

1 is a schematic block diagram of a plant area extraction system according to an embodiment of the present invention.
2A and 2B show exemplary plant images of plants to be photographed in the plant image generation unit.
3A is a block diagram showing an operation process of the first plant region extraction unit, and FIG. 3B is a block diagram showing the configuration of the dense block of FIG. 3A.
4 is a graph showing a connection relationship between a plurality of pixels.
5A and 5B are result data obtained by extracting a plant region from the plant images of FIGS. 2A and 2B, respectively.
6 is a flowchart of a method for extracting plant regions according to another embodiment of the present invention.

For a detailed description of the present invention, which will be described later, reference is made to the accompanying drawings that illustrate, by way of example, specific embodiments in which the present invention may be practiced. These embodiments are described in detail enough to enable a person skilled in the art to practice the present invention. Various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain shapes, structures, and properties described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with one embodiment. In addition, the location or placement of individual components within each disclosed embodiment can be changed without departing from the spirit and scope of the invention. Therefore, the following detailed description is not to be described in a limiting sense, and the scope of the present invention is limited only by the appended claims, along with all ranges equivalent to the claims. In the drawings, similar reference numerals refer to the same or similar functions in various aspects.

The terminology used in the present specification is a general terminology that is currently widely used while considering functions, but this may be changed according to intentions or customs of technicians in the field or the appearance of new technologies. Also, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meanings will be described in the description of the corresponding specification. Therefore, the terms used in the present specification should be interpreted based on the actual meaning of the term and not the contents of the simple term, but the contents throughout the present specification.

1 is a schematic block diagram of a plant region extraction system according to an embodiment. 2A and 2B show exemplary plant images of plants to be photographed by the plant image generator 100. 3A is a block diagram showing an operation process of the first plant region extraction unit, and FIG. 3B is a block diagram showing the configuration of the dense block of FIG. 3A. 4 is a graph showing a connection relationship between a plurality of pixels. 5A and 5B are result data obtained by extracting a plant region from the plant images of FIGS. 2A and 2B, respectively.

Referring to FIG. 1, the plant region extraction system 10 according to the present embodiment includes a plant image generator 100, a first plant region extractor 110, and a second plant region extractor 120. The plant area extraction system according to embodiments and each device or unit constituting the same may have aspects that are entirely hardware, or partially hardware and partially software. For example, each component of the plant area extraction system is intended to refer to a combination of hardware and software driven by the hardware. The hardware may be a data processing device including a central processing unit (CPU) or other processor. Also, software driven by hardware may refer to a running process, an object, an executable, a thread of execution, a program, or the like. For example, the plant image generating unit 100 may refer to a combination of hardware for imaging an object and imaging it by controlling it, and software for converting the image into a form for subsequent processing by performing imaging and processing imaging.

The plant image generation unit 100 may photograph a target plant to generate a plant image I corresponding to a two-dimensional image. The plant image generation unit 100 may include an image sensor for photographing a plant image. The plant image generation unit 100 may generate a plant image I by photographing a target plant from the front. However, the present invention is not limited thereto, and in another embodiment, the plant image generating unit 100 may photograph in at least one of front, rear, left, right, and top surfaces of the target plant.

Here, the target plant photographed by the plant image generating unit 100 is a plant having a plurality of plant leaves, and each plant leaf may have a long and thin shape. The target plant may be monocotyledonous plants, but is not limited thereto.

Further, the space for photographing the target plant may be a space having a single color background. Specifically, it may be a space having a background of a color that contrasts with a plant leaf or is easily distinguishable from a color, forming a complementary color relationship, but is not limited thereto. The target plant may be photographed while sequentially passing through the photographing place through a conveyor system or the like, but is not limited thereto. 2A and 2B show exemplary plant images generated by the plant image generator 100. The plant image I may be composed of a plurality of pixels, and may have a resolution according to the number of pixels. The generated plant image I may be provided to the first plant region extractor 110 and the second plant region extractor 110.

The first plant region extracting unit 110 generates first plant region extraction data for distinguishing the plant region and the background region for each pixel of the plant image. The first plant region extraction data is data that stochastically divides the plant region and the background region in units of each pixel of the plant image I, and the probability that each of the plurality of pixels included in the plant image I corresponds to the plant region It may be calculated data. The first plant region extracting unit 110 may determine whether each of the plurality of pixels of the input plant image I corresponds to the plant region, and generate a probability value corresponding to the plant region as a result value.

The first plant region extraction unit 110 may generate first plant region extraction data through a convolutional neural network-based machine learning model. A machine learning model based on a convolutional neural network is composed of a plurality of convolutional stages and a plurality of polling stages, and the value of a filter that can reflect the characteristics of the input image in the process of repeatedly performing each stage It can learn automatically and analyzes and discriminates images according to the learned filters. However, in the case of a conventional machine learning model based on a convolutional neural network, the resolution of the input image is reduced in the polling step on the input image. As a result, a result of a plurality of pixels of each input image based on the original resolution May not be suitable for the present invention.

Accordingly, the first plant region extractor 110 according to an embodiment of the present invention corresponds to a polling step and a transition down (TD) step of reducing the resolution of the feature map and a transition up (TU) of increasing the resolution of the feature map ) Is composed of an encoder-decoder structure including a step. That is, the first plant region extraction data finally generated through the deconvolution process may be guaranteed to have the same resolution as the input plant image (I).

3A shows an overall operation process of a machine learning model based on a convolutional neural network performed by the first plant region extraction unit 110. The first plant region extraction unit 110 may be configured such that Conv, DB, TD, DB, TD, DB, TU, DB, TU, DB, Conv steps are sequentially performed as illustrated in FIG. It does not work.

In the block shown in FIG. 3A, Image I means plant image (I), and Lavel L is first plant region extraction data. In the block shown in FIG. 3A, Conv represents a convolution step and creates a feature map for the plant image. The input plant image I having undergone the convolution step may be formed as a plurality of feature maps according to the number of filters. The feature map generated through the convolution step may have the same size and resolution as the image input by zero padding.

TD represents a transition down step, and the feature map after TD may have a reduced resolution. The TU represents a transition up step, and the feature map after the TU can be increased in resolution at the same rate as the reduced rate in TD. Also, data input to the transition down may be skipped concatenated with data output from the transition up. That is, data output from the transition up may be supplemented with data provided from the transition down. The loss of the original data generated during the transition down process may be compensated through a skip connetion, thereby making it possible to judge and generate result data for all pixels of the original image.

DB stands for density block. The Dense Block is composed of a plurality of convolution steps, and feature maps generated through convolution in a density block are continuously accumulated, and the size of the feature maps in one density block is the same. These continuously accumulated points are indicated by small circular blocks (Concatenation) in FIGS. 3A and 3B.

3B is a block diagram showing the structure of a Dense Block according to an embodiment of the present invention. In the embodiment of the present invention, the Dense Block comprises two convolution steps, but is not limited thereto. As the feature maps generated through convolution are continuously accumulated and accumulated, the initial stage information may not be washed out, but may be continued with the later stage information, and connectivity between each stage is increased, so that learning of the filter can be performed more efficiently. Can.

According to the convolutional neural network structure illustrated in FIG. 3A, the first plant region extraction unit 110 may generate first plant region extraction data having the same resolution and size as the input image plant image (I). In addition, since it is determined whether it is a plant region or a background region for each pixel constituting the plant image I, even if the leaves of the plant are fine, it is possible to determine this. In addition, it may be possible to classify regions more accurately to perform judgment on each pixel through a learned filter that extracts characteristics of an image, such as comparison with surrounding pixels, not merely color differences.

Here, when the size (resolution) of the plant image I is larger than the maximum size (resolution) of the image that can be processed by the first plant region extracting unit 110, a pre-processing process for the plant image I may be required. have. However, if the resolution of the plant image I is downscaled to a processable image, pixel loss may occur. Due to the occurrence of pixel loss in the fine leaf portion of the plant image I, unintended breaks or holes may occur when extracting the plant region. Therefore, the first plant region extracting unit 110 according to the present embodiment divides the plant image I into a size of a processable image to generate a plurality of sub-plant images. The plurality of sub-plant images may be the same size as the image that can be processed by the first plant region extraction unit 110, and the first plant region extraction unit 110 may extract the first sub-plant region extraction data for each sub-plant image. The first plant region extraction data for the original plant image (I) can be generated by generating each and combining them again.

However, the plurality of sub-plant images may include pixel regions overlapping with other sub-plant images. Since the value of the first sub-plant region data for the overlapping pixel region is a result of machine learning for another type of image, the resulting probability value may be different. The first plant region extracting unit 110 has the highest probability value for the overlapping pixel regions when the generated first sub plant region extraction data includes overlapping pixel regions of the plurality of sub plant images. The first plant region extraction data may be selected to combine the first plant region extraction data.

The first plant region extraction data generated by the first plant region extraction unit 110 is provided to the second plant region extraction unit 120.

The second plant region extraction unit 120 receives the plant image (I) from the plant image generation unit 100 and the first plant region extraction data from the first plant region extraction unit 110, respectively, the plant image and the first Plant region extraction data is used to generate second plant region extraction data that maximizes interconnection between a plurality of pixels of a plant image. The second plant region extraction unit 120 may maximize and maximize the connectivity of the pixels of the plant image I through probabilistic graphical models. The probabilistic graphic model may be a conditional probability model called a dense conditional random field (Dense Conditional Random Field) forming all of the interconnection relationship between a plurality of pixels. Here, maximizing or maximizing the connectivity means, as illustrated in FIG. 4, forming an interconnect graph between a plurality of pixels and determining whether each pixel is a plant region in consideration of the mutual influence between the connected pixels. do.

More specifically, the second plant region extraction unit 120 considers the probability value that each pixel based on the first plant region extraction data corresponds to the plant region and other mutual influences on the connection between the pixels, so that each pixel is connected to the plant region. The second plant region extraction data, which determines whether or not it is applicable, is generated.

Here, the mutual influence considered between the connected pixels may be determined by considering the distance between the connected pixels, the position difference, whether the R, G, and B values of the plant image I are similar, and whether the plant image color is similar. In consideration of the relationship between one pixel and another pixel connected to the one pixel, the probability value of whether each pixel corresponds to a plant region is expressed by the Gibbs energy function defined by Equation 1 below, and the result of this The value can be calculated by optimizing equation (1).

[Equation 1]

는 식물 이미지,

는 모델 파라미터(식물 영역인지 여부를 판별),

는 식물 이미지의 제i 화소,

는 식물 이미지의 제j 화소를 의미한다.here,

Plant images,

Is the model parameter (determining whether it is a plant area),

Is the i-th pixel of the plant image,

Means j pixel of the plant image.

)은 로지스틱 회귀분석 모델(logistic regression model)로 정의되며, 제1 식물 영역 추출 데이터에서 생성된 각 화소에 대한 확률 값(

)이 하기 수학식 2와 같이 입력될 수 있다.In the Gibbs energy function of Equation 1, the unary term(

) Is defined as a logistic regression model, and a probability value for each pixel generated from the first plant region extraction data (

) May be input as shown in Equation 2 below.

[Equation 2]

: 제1 식물 영역 추출 데이터에서 생성된 제i 화소의 결과 데이터

: Result data of the i-th pixel generated from the first plant region extraction data

)은 연결된 상이한 화소(

,

)의 영향성을 나타내는 수식으로, 가우시안 커널(Gaussian kernel)과 바이라테랄 커널(bilateral kernel)을 혼합하여 정의될 수 있다. Also, in the Gibbs energy function of Equation 1, pairwise interaction term(

) Is a different connected pixel (

,

) Is an expression representing the influence of the Gaussian kernel (Gaussian kernel) and may be defined by mixing a bilateral kernel (bilateral kernel).

))은 하기 수학식 3과 같이 정의될 수 있다.Gaussian kernel,

)) may be defined as in Equation 3 below.

[Equation 3]

는 고정된 특성 벡터로 하기 수학식 4와 같이 정의되고,

도 동일한 방식으로 정의될 수 있다.here,

Is a fixed feature vector defined by Equation 4 below,

Can also be defined in the same way.

[Equation 4]

및

는

의 x, y 위치에 대한 Potts compatibility function으로

을 만족하며,

는 x위치에 대한 표준편차에 해당하는 파라미터 값,

는 y위치에 대한 표준편차에 해당하는 파라미터 값에 해당한다.)(here,

And

The

Potts compatibility function for x and y position of

Satisfying,

Is the parameter value corresponding to the standard deviation for the x position,

Corresponds to the parameter value corresponding to the standard deviation for y position.)

))을 통해 위치에 따른 영향성이 고려될 수 있다. That is, the Gaussian kernel (Gaussian kernel,

)) can be considered depending on the location.

)의 바이라테랄 커널(bilateral kernel,

)은 하기 수학식 5와 같이 정의된다. pairwise interaction term(

) Of the bilateral kernel (bilateral kernel,

) Is defined as in Equation 5 below.

[Equation 5]

는 고정된 특성 벡터로 하기 수학식 6와 같이 정의되고,

도 동일한 방식으로 정의될 수 있다.here,

Is defined as Equation 6 below as a fixed characteristic vector,

Can also be defined in the same way.

[Equation 6]

는 식물 이미지(I)의 제i 화소(

)의 R, G, B 값이며,

는 x위치에 대한 표준편차에 해당하는 파라미터 값,

는 y위치에 대한 표준편차에 해당하는 파라미터 값,

,

및

은 각각 R, G, B 값에 대한 표준 편차에 해당하는 파라미터 값이다.)(here,

Is the i-th pixel of the plant image I

), R, G, B values,

Is the parameter value corresponding to the standard deviation for the x position,

Is the parameter value corresponding to the standard deviation for y position,

,

And

Is a parameter value corresponding to the standard deviation for the R, G, and B values, respectively.)

The color difference between pixels and the distance between pixels may be considered through a bilateral kernel.

By combining Equation 3 and Equation 5, the pairwise interaction term in the Gibbs energy function of Equation 1 may be defined as Equation 7 below.

[Equation 7]

)를 손실 함수(loss function)로, L-BFGS(limited-memory Broyden-Fletcher-Goldfarb-Shanno)을 최적화 알고리즘으로 사용하여 수학식 1의 Gibbs energy function을 하기 수학식 8과 같이 최적화하여 각 화소에 대한 결과 값을 산출하여 제2 식물 영역 추출 데이터를 생성한다. The second plant region extracting unit 120 has a log likelihood loss (

) As the loss function and L-BFGS (limited-memory Broyden-Fletcher-Goldfarb-Shanno) as the optimization algorithm. The result value is calculated for the second plant region extraction data.

[Equation 8]

,

,

은 화소(

)의 결과 값, 제2 식물 영역 추출 데이터이다.)(here,

Silver pixel(

), the second plant area extraction data.)

The second plant region extraction data is data indicating whether each pixel of the plant image I is a plant region. 5A is second plant region extraction data as a result of extracting a plant region from the plant image of FIG. 2A, and FIG. 5B is second plant region extraction data as a result of extracting a plant region from the plant image of FIG. 2B. The second plant region extraction data is a binary image in which the plant region is displayed in white and the background region is displayed in black.

The plant region extraction system according to an embodiment of the present invention distinguishes a plant region and a background region from a plant image through a convolutional neural network, but the result data is input through a deconvolution process. It is guaranteed to have the same resolution as the image.

In addition, through a conditional probability model called a density conditional random field, it is possible to generate result data that maximizes the connectivity of pixels in a plant image.

Therefore, a plant region with elongated leaves can be more precisely extracted from the plant image.

Hereinafter, a method for extracting a plant region according to an embodiment of the present invention will be described. 6 is a flowchart of a method for extracting plant regions according to an embodiment of the present invention.

Referring to Figure 6, the plant leaf identification method according to an embodiment of the present invention includes a plant image generation step (S100), the first plant area data extraction step (S110), the second plant area data extraction step (S120) do.

Here, the plant region extraction system performing each step described above may be the plant region extraction system 10 of FIG. 1 described above, and detailed description thereof will be omitted. Also, FIGS. 1 to 5 may be referred to for the description of this embodiment.

First, a plant image is generated (S100).

The plant region extraction system 10 includes a plant image generation unit 100, a first plant region extraction unit 110, and a second plant region extraction unit 120. The step of generating a plant image (I) by photographing the target plant may be performed in the plant image generator 100.

The plant image generation unit 100 may photograph a target plant to generate a plant image I corresponding to a two-dimensional image. The plant image generation unit 100 may include an image sensor for photographing a plant image. Here, the target plant photographed by the plant image generating unit 100 is a plant having a plurality of plant leaves, and each plant leaf may have a long and thin shape. The target plant may be monocotyledonous plants, but is not limited thereto. The plant image I may be composed of a plurality of pixels, and may have a resolution according to the number of pixels. The generated plant image I may be provided to the first plant region extractor 110 and the second plant region extractor 110.

Subsequently, the first plant region extraction data is extracted (S110).

The extraction of the first plant region data may be performed by the plant region extraction unit 110. The first plant region extracting unit 110 generates first plant region extraction data for distinguishing the plant region and the background region for each pixel of the plant image. The first plant region extraction data is data that stochastically divides the plant region and the background region in units of each pixel of the plant image I, and the probability that each of the plurality of pixels included in the plant image I corresponds to the plant region It may be calculated data. The first plant region extracting unit 110 may determine whether each of the plurality of pixels of the input plant image I corresponds to the plant region, and generate a probability value corresponding to the plant region as a result value.

The first plant region extraction unit 110 may generate first plant region extraction data through a convolutional neural network-based machine learning model. The convolutional neural network-based machine learning model includes a convolution step (Conv) for generating a feature map for the plant image, a transition down (TD) step for reducing the resolution of the feature map, and a feature map It includes a transition up (TU) step of increasing the resolution, the data input to the transition down may be skipped (skip connetion) with the data output from the transition up.

In addition, the convolutional neural network-based machine learning model includes a density block including a plurality of convolution steps, and feature maps generated through convolution within the density block are cumulative. As the initial stage information is not erased, it can be continued with the later stage information.

The first plant region extraction unit 110 may generate first plant region extraction data having the same resolution and size as the input image plant image (I). In addition, since it is determined whether it is a plant region or a background region for each pixel constituting the plant image I, even if the leaves of the plant are fine, it is possible to determine this. In addition, it may be possible to classify regions more accurately to perform judgment on each pixel through a learned filter that extracts characteristics of an image, such as comparison with surrounding pixels, not merely color differences.

Here, the step of generating the first plant region extraction data includes: dividing the plant image into a size of the processable image to generate a plurality of sub-plant images; Generating first sub plant region extraction data for each of the plurality of sub plant images; And combining the generated first sub plant region extraction data to generate the first plant region extraction data. The generating of the first plant region extraction data by combining the generated first sub plant region extraction data may include generating pixels in which the generated first sub plant region extraction data overlaps the plurality of sub plant images. When the region is included, the first plant region extraction data may be combined by selecting the first sub-plant region extraction data having the highest probability value for the overlapping pixel region.

Subsequently, second plant region extraction data is generated (S120 ).

Generation of the second plant region extraction data is performed by the second plant region extraction unit 120. The second plant region extraction unit 120 may maximize and maximize the connectivity of the pixels of the plant image I through probabilistic graphical models. The probabilistic graphic model may be a conditional probability model called a dense conditional random field (Dense Conditional Random Field) forming all of the interconnection relationship between a plurality of pixels. Here, maximizing or maximizing the connectivity means forming an interconnect graph between a plurality of pixels and determining whether each pixel is a plant region in consideration of the mutual influence between the connected pixels.

In the generating of the second plant region extraction data, all of the pixels according to the first plant region extraction data form a probability value corresponding to the plant region and an interconnection graph between the plurality of pixels, and between the connected pixels. In consideration of the mutual influence, the second plant region extraction data for determining whether each pixel is a plant region may be generated. The mutual influence may be determined in consideration of a distance difference between connected pixels and similarity of R, G, and B values of the plant image. The mutual influence may be determined in consideration of a difference in distance between connected pixels and similarity of R, G, and B values of the plant image. In consideration of the relationship between one pixel and another pixel connected to the one pixel, the probability value of whether each pixel corresponds to a plant region is expressed as a Gibbs energy function, and optimization of the second plant region extraction data is calculated. Can.

The operation by the method for extracting plant regions according to the above-described embodiments may be at least partially implemented by a computer program and recorded on a computer-readable recording medium. A program for implementing an operation by a method for extracting plant regions according to embodiments is recorded, and a computer-readable recording medium includes all types of recording devices in which data that can be read by a computer are stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. In addition, the computer readable recording medium may be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present embodiment will be readily understood by those skilled in the art to which this embodiment belongs.

Although described above with reference to embodiments, the present invention should not be construed as being limited by these embodiments or the drawings, and those skilled in the art will appreciate the spirit and scope of the present invention as set forth in the claims below. It will be understood that various modifications and changes can be made to the present invention without departing from the scope.

10: Plant area extraction system
100: plant image generating unit
110: first plant region extraction unit
120: second plant area extraction unit

Claims (17)

A plant image generating unit that photographs a target plant to generate a plant image;
A first plant region extraction unit for generating first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And
And a second plant region extraction unit that generates second plant region extraction data that maximizes interconnection between a plurality of pixels of the plant image by utilizing the plant image and the first plant region extraction data,
The convolutional neural network-based machine learning model of the first plant region extractor,
A convolution step (Conv) for generating a feature map for the plant image, a transition down (TD) step for reducing the resolution of the feature map, and a transition up (TU) step for increasing the resolution of the feature map,
The data input to the transition down is skip connetion with the data output from the transition up,
The machine learning model based on the convolutional neural network of the first plant region extractor includes a density block including a plurality of convolution steps,
The feature map generated through convolution within the density block is accumulated so that the initial stage information is not erased, but the plant region extraction system is characterized in that it continues to be connected to the late stage information. delete delete A plant image generating unit that photographs a target plant to generate a plant image;
A first plant region extraction unit for generating first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And
And a second plant region extraction unit that generates second plant region extraction data that maximizes interconnection between a plurality of pixels of the plant image by utilizing the plant image and the first plant region extraction data,
When the plant image is larger than the size of the image that can be processed by the first plant region extracting unit, the first plant region extracting unit divides the plant image into the size of the processable image to generate a plurality of sub-plant images,
First sub plant region extraction data for each of the plurality of sub plant images are respectively generated,
Each of the generated first sub-plant region extraction data is combined to generate the first plant region extraction data,
When the first plant region extraction data of each of the first plant region extraction data includes overlapping pixel regions of the plurality of sub plant images, the first plant region extraction unit may include:
And selecting the first sub-plant region extraction data having the highest probability value for the overlapping pixel region and combining the first plant region extraction data. delete According to claim 1,
The second plant region extracting unit forms a probability value for each pixel corresponding to the first plant region extraction data and an interconnection graph between the plurality of pixels, and considers the mutual influence between connected pixels By generating the second plant region extraction data to determine whether each pixel is a plant region. The method of claim 6,
The mutual influence is determined by considering the distance difference between the connected pixels and whether the R, G, and B values of the plant image are similar. According to claim 1,
The plant is a monocotyledonous plant (Monocotyledonous Plants), characterized in that the plant area extraction system. The plant region extraction system, photographing a target plant to generate a plant image;
Generating, by the plant region extraction system, first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And
The plant region extraction system includes generating the second plant region extraction data that maximizes interconnection between a plurality of pixels of the plant image by utilizing the plant image and the first plant region extraction data,
The machine learning model based on the convolutional neural network,
A convolution step (Conv) for generating a feature map for the plant image, a transition down (TD) step for reducing the resolution of the feature map, and a transition up (TU) step for increasing the resolution of the feature map,
The data input to the transition down is skip connetion with the data output from the transition up,
The convolutional neural network-based machine learning model includes a density block including a plurality of convolution steps,
The feature map generated through convolution within the density block is accumulated, and the initial stage information is not erased, but the plant region extraction method is characterized in that it continues with the later stage information. delete delete The plant region extraction system, photographing a target plant to generate a plant image;
Generating, by the plant region extraction system, first plant region extraction data that separates a plant region and a background region for each pixel of the plant image through a machine learning model based on a convolutional neural network; And
The plant region extracting system, using the plant image and the first plant region extraction data, generating a second plant region extraction data that maximizes the interconnection between a plurality of pixels of the plant image,
Generating the first plant region extraction data,
Dividing the plant image into a size of a processable image to generate a plurality of sub-plant images;
Generating first sub plant region extraction data for each of the plurality of sub plant images; And
And combining the generated first sub plant region extraction data to generate the first plant region extraction data,
The step of generating the first plant region extraction data by combining the generated first sub-plant region extraction data may include overlapping the plurality of sub-plant images with each of the generated first sub-plant region extraction data. When a pixel region is included, a plant region extraction method characterized by selecting first sub-plant region extraction data having the highest probability value for the overlapping pixel region and combining the first plant region extraction data. delete The method of claim 9,
In the generating of the second plant region extraction data, all of the pixels according to the first plant region extraction data form a probability value corresponding to the plant region and an interconnection graph between the plurality of pixels, and between the connected pixels. A method of extracting plant regions, characterized in that the second plant region extraction data for determining whether each pixel is a plant region in consideration of mutual influence is generated. The method of claim 14,
The mutual influence is determined by considering the distance difference between the connected pixels and whether the R, G, and B values of the plant image are similar. The method of claim 9,
The plant is a monocotyledonous plant (Monocotyledonous Plants), characterized in that the plant region extraction method. A computer program stored in a medium to execute the plant area extraction method according to any one of claims 9, 12, 14 to 16 in combination with hardware.

Images