KR20160145319A

KR20160145319A - Method for temperature measuring of plant leaf using muti-image

Info

Publication number: KR20160145319A
Application number: KR1020150081726A
Authority: KR
Inventors: 유병문
Original assignee: 주식회사 엘앤와이비젼
Priority date: 2015-06-10
Filing date: 2015-06-10
Publication date: 2016-12-20

Abstract

The present invention relates to a method for measuring leaf temperature using multiple images, which enables more accurate measurement of leaf temperature in a non-contact manner using color images and thermal images taken from plants.
The method includes a first step of calculating a mapping parameter between a color image and a column image using an image of a same photographing region photographed through a color camera and a thermal image camera, A second step of acquiring a color image and a thermal image of a target plant photographed through a thermal camera, respectively; a step of separating a leaf area having a corresponding color based on the leaf color of the plant in the color image acquired in the step A fourth step of setting a region corresponding to the leaf area in the color image in the thermal image based on the mapping parameters calculated in the first step and the fourth step in the leaf area in the thermal image set in the fourth step, And a fifth step of calculating an average value of the pixel-by-pixel temperature values to obtain leaf temperature of the plant.

Description

[0001] METHOD FOR TEMPERATURE MEASURING OF PLANT LEAF USING MUTI-IMAGE [0002]

The present invention relates to a method and apparatus for measuring leaf temperature using multiple images, which can more accurately perform temperature measurement on a leaf temperature by clearly separating a leaf area in a non-contact manner using a color image and a thermal image captured by a plant .

The temperature of a plant is one of the important factors that indicate the physiological response of a living plant and the process of mass transfer between the plant and its environment. The temperature of the plant changes according to various environmental factors, and it reacts rapidly depending on the physiological situation of each plant, and has a characteristic that it differs depending on the kind of plant.

Therefore, it is essential to accurately measure the temperature of a plant in order to study the interrelationship between the plant and the environment, such as the influence of the insect pests of the plant, the pollution of the surrounding air,

Thermocouples or thermistors have been used to measure plant temperatures. Contact-type measuring methods such as thermocouple sensors and thermistors are methods for measuring temperature by contacting with plants. In experiments requiring repeated measurement, stress is applied to the plant, and accurate temperature measurement at the contact portion becomes unstable .

In order to overcome these disadvantages, in recent years, many researches have been made on the non-contact type measurement method using the radiation thermometer or the scanning infrared camera for the contactless temperature measurement with the development of the surface temperature measuring device .

In particular, the scanning infrared camera has an advantage of providing two-dimensional surface information on the plant temperature, and is being studied with many problems that have not been possible with the image processing technology.

However, the two-dimensional thermal image obtained from the scanning infrared camera is divided into the area to be measured and the other background area, and it is difficult to clearly distinguish the two areas from each other in the thermal image.

In other words, the temperature measurement using the scanning infrared camera is mainly a point measurement method in which the target area is directly observed and measured, but passive methods in which the temperature is obtained by directly dividing the measurement area by hand are used for the acquired two- ought.

As a result, the temperature information about the background area, not the leaf area of the plant, is applied to the leaf area temperature, resulting in a result of the leaf temperature, thereby causing an error in the result of the leaf temperature measurement.

This can have a negative effect on the research on the plant or the analysis of the plant cultivation environment using the chlorophyll.

1. Korean Patent Laid-Open Publication No. 2013-0035816 (entitled " Plant Temperature Measurement Using Image Processing and Method Thereof "

Accordingly, the present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a color image processing method and a color image processing method, in which a color image captured by a plant is matched with a thermal image, The temperature of the leaf can be measured by separating the area corresponding to the leaf area extracted from the color image, so that the temperature of the leaf area can be automatically measured in a non-contact manner by a simple method of analyzing the image The purpose of this paper is to provide a method for measuring the temperature of leaves using multiple images.

According to an aspect of the present invention, there is provided a color image processing method comprising the steps of: calculating a mapping parameter between a color image and a column image using an image captured by a color camera and a thermal image camera; A second step of acquiring a color image and a thermal image for a target plant photographed through a camera and a thermal camera, respectively, and a leaf area having the corresponding color is separated based on the leaf color of the plant in the color image acquired in the second step A fourth step of setting a region corresponding to a leaf area in the color image in the thermal image based on the mapping parameter calculated in the first step, And a fifth step of calculating an average value of the pixel-by-pixel temperature values for the leaf temperature of the plant to obtain the leaf temperature of the plant The yeopon measurement method using an image, is provided.

The first step may include a step of binarizing the brightness of the photographed image provided through the color camera and the thermal camera to extract a marker corresponding to the same object position, And a coordinate value of a marker extracted from a column image with respect to a corresponding object position, and setting the difference value as a mapping parameter. A method of measuring a leaf temperature using a multiple image is provided .

In addition, in the first step, the imaging object for acquiring the mapping parameters is composed of a test module in which black and white are arranged in a lattice form, and a white area of the test module in the color image and the column image, The method comprising the steps of:

According to the present invention, a leaf area of a plant is acquired in a color image, and a leaf area corresponding to a leaf area of a color image is acquired in a thermal image using a preset matching parameter, Measurement becomes possible.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic view showing a configuration of a temperature measuring system to which the present invention is applied; FIG.
FIG. 2 is a view illustrating the arrangement of the color camera 100 and the thermal image camera 200 shown in FIG. 1; FIG.
FIG. 3 is a block diagram showing the internal configuration of the temperature sensor 300 shown in FIG. 1 functionally separated.
FIG. 4 is a flowchart schematically illustrating a method of measuring the temperature of leaves using multiple images according to the present invention.
FIG. 5 is a flowchart for explaining the mapping parameter calculation process ST10 shown in FIG. 4; FIG.
FIG. 6 is a view illustrating an image generated in the process shown in FIG. 5; FIG.
FIG. 7 is a flow chart for explaining the leaf temperature obtaining process (ST20) shown in FIG.
8 is a view illustrating an image generated in the process shown in FIG.

The description of the present invention is merely an example for structural or functional explanation, and the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the present invention should be understood to include equivalents capable of realizing technical ideas. Also, the purpose or effect of the present invention should not be construed as limiting the scope of the present invention, since it does not mean that a specific embodiment should include all or only such effect.

Hereinafter, a method for measuring a temperature of a plurality of images using multiple images according to an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a diagram showing a schematic configuration of a temperature measuring system to which the present invention is applied.

As shown in FIG. 1, the temperature measuring system according to the present invention includes a color camera 100, a thermal image camera 200, a color image provided from the color camera 100, and a color image provided from the thermal image camera 200 And a leaf temperature measuring device 300 for obtaining the leaf temperature of the plant included in the image by matching the thermal image.

The color camera 100 and the thermal camera 200 may be installed so as to simultaneously photograph the same position. For example, the color camera 100 and the thermal image camera 200 may be installed as shown in FIG.

The thermal imager 200 is a camera which can be expressed in a different color depending on the temperature,

As shown in FIG. 3, the temperature measuring apparatus 300 includes an image mapping parameter calculator 310, an image mapping unit 320, and a leaf temperature calculator 330.

The image mapping parameter calculator 310 extracts a marker through a binarization process for the color image and the column image, and calculates a mapping parameter for mapping the color image and the column image using the marker. At this time, the image mapping parameter calculator 310 may calculate a mapping parameter using a marker in a state in which distortion correction is performed on the image using each camera distortion parameter.

The image synthesizer 320 separates the leaf area of the plant based on the color information in the color image provided from the color camera 100 and displays the color image corresponding to the separated leaf area using the mapping parameter And synthesizes the image to be mapped to the image.

The leaf temperature calculation unit 330 obtains the leaf temperature, that is, the leaf temperature, by separating the leaf area color image part from the thermal image and calculating the average value through the pixel value of the thermal image for the separated leaf area.

Next, a method of measuring a leaf temperature using multiple images according to the present invention will be described with reference to FIGS. 4 to 8. FIG. FIG. 4 is a flow chart for explaining a method of measuring the temperature of leaves using multiple images according to the present invention, FIG. 5 is a flowchart for explaining the mapping parameter calculation step ST10 shown in FIG. FIG. 7 is a flow chart for explaining the process of acquiring a leaf temperature (ST20) shown in FIG. 4, and FIG. 8 is a view illustrating an image generated in the process shown in FIG. 7 .

As shown in FIG. 4, the method of measuring a leaf temperature using multiple images according to the present invention includes mapping parameters for calculating a mapping parameter between a color image provided from a color camera 100 and a column image provided from a thermal image camera 200 (ST10), each of the measurement object images provided from the color camera 100 and the thermal camera 200 is mapped based on the pre-calculated mapping parameters and synthesized, and the synthesized image corresponds to the leaf area of the color image (ST20) which calculates the temperature of the leaf based on the temperature value of the corresponding pixel in the thermal image region.

1. Mapping parameter calculation process (ST10)

The mapping parameter calculation step ST10 shown in Fig. 4 will be described in more detail with reference to Figs. 5 and 6. Fig. In FIG. 6, (X) is a color image and (Y) is a thermal image.

First, a test object 1 for calculating a mapping parameter is placed in the photographing area in a state in which the color camera 100 and the thermal camera 200 are installed to photograph the same area at a predetermined position. At this time, the test object 1 may be configured to have a grid shape as shown in FIG. 6 (a) to facilitate camera calibration and mapping parameter calculation. For example, the test object 1 may be configured as a test module in which black and white are arranged in a lattice form. It should be noted that the test object 1 may be various images or objects including the object of the temperature measurement.

In the above state, the color camera 100 and the thermal imaging camera 200 photograph the test object 1 and provide the photographed image to the temperature measuring apparatus 300 (ST11). At this time, the color camera 100 and the thermal imaging camera 200 perform imaging operations sequentially or at the same time with respect to the test object 1. [ 6A is a diagram illustrating a color image X and a column image Y for the test object 1. In FIG.

The cut-off temperature measuring apparatus 300 extracts a marker from the color image provided from the color camera 100 and the thermal image provided from the thermal image camera 200 for the test object 1 (ST12). That is, the color temperature measuring apparatus 300 binarizes the color image and the thermal image based on a predetermined reference constant for brightness to extract the marker. For example, the cut-off temperature measuring apparatus 300 sets "0" when the brightness of the pixel is less than the reference constant, and "1" when the brightness of the pixel is greater than or equal to the reference constant. 6 (b) shows an example in which the markers extracted from each image are expressed in color hues. That is, in the text object 1, a bright area, that is, a white area is obtained as a marker.

The temperature measuring apparatus 300 performs a distortion correction process on a color image provided from the color camera 100 and a thermal image provided from the thermal image camera 200 (ST13). Generally, images captured by a camera include distortion components due to physical errors such as a lens and an installation environment. That is, the camera captures an image including a distortion component according to an external parameter including a camera height, a camera position, a reference plate width, and a reference plate height, and internal parameters such as a focal length, . Accordingly, the leaf temperature measuring apparatus 300 can perform correction processing on the output image by calculating internal parameters, external parameters, and distortion parameters for the camera through the known Zhang theory. 6 (c) illustrates a state in which the image shown in (a) is corrected.

Next, in step ST14, the leaf temperature measuring apparatus 300 acquires mapping parameters using the markers extracted in step ST12 in the respective corrected images. That is, the color temperature measuring apparatus 300 compares the first marker coordinate value of the color correction image with the second marker coordinate value of the thermal correction image, and calculates a mapping parameter corresponding to the difference. For example, if the first marker coordinate value of the color correction image is (1,10) and the second marker coordinate value of the thermal correction image is (5,12) for the same marker, the mapping parameter is (4,2) . &Lt; / RTI > At this time, the leaf temperature measuring apparatus 300 may calculate the mapping parameters for a plurality of markers, and obtain an average of the mapping parameters to obtain mapping parameters.

Also, the temperature measuring apparatus 300 stores the currently set mapping parameters. Thereafter, when the position of the camera is changed, the corresponding mapping parameter is reset through the above-described operation.

2. Obtaining the leaf temperature (ST20)

The step of acquiring the leaf temperature (ST20) shown in FIG. 4 will be described in detail with reference to FIG. 7 and FIG. In FIG. 8, (X) is a color image and (Y) is a thermal image.

First, a mapping parameter calculated through the above-described mapping parameter calculation process is set and stored in the temperature measuring device 300, a plant to be measured is positioned at a photographing position, and the color camera 100 and the thermal camera 200 And performs imaging of the target plant (ST21).

The photographic image including the target camera provided from the color camera 100 and the thermal camera 200 is provided to the temperature measuring device 300. 8 (a) is a view illustrating an image captured from the color camera 100 and the thermal camera 200. As shown in FIG.

The leaf temperature measuring apparatus 300 separates the leaf area of the plant based on the color information in the color image provided from the color camera 100 (ST22). For example, the leaf temperature measuring apparatus 300 separates a region having a "green" color corresponding to a leaf color into a region of interest, that is, a leaf region. At this time, the leaf temperature measuring apparatus 300 determines a green color distribution, and determines a green color that is searched for a predetermined distance or more as a background area. 8 (b) shows a leaf area separated on the basis of color information.

Then, the temperature measuring apparatus 300 acquires a thermal image corresponding to the corresponding leaf area using the mapping parameters obtained in the mapping parameter calculation step ST10 (ST23). As shown in FIG. 8 (a), the thermal image is more accurate than the color image, so that the thermal image ROI corresponding to the ROI in the color image can be obtained more accurately. That is, as shown in FIG. 8 (c), an area corresponding to the leaf area separated from the color image in the thermal image is set using a mapping parameter.

In addition, the lenticularity measuring apparatus 300 separates the region of interest corresponding to the color leaf region from the thermal image. 8 (d) shows a region of interest, i.e., a leaf area, which is separated from the thermal image.

The leaf temperature measuring apparatus 300 calculates the leaf temperature based on the temperature value of each pixel in the leaf area of the thermal image acquired in step ST23 (ST24). At this time, the leaf temperature measuring apparatus 300 adds the temperature values for each pixel, calculates the average value thereof, and determines the leaf temperature.

That is, according to the embodiment, the leaf area in the thermal image is set based on the leaf area in the color image, and the temperature for the leaf is calculated based on the pixel value of the leaf area in the thermal image .

100: color camera, 200: thermal camera,
300: Leaf temperature measuring apparatus, 310: Mapping parameter calculating unit,
320: video mapping unit, 330: blindness calculating unit.

Claims

A first step of calculating a mapping parameter between a color image and a thermal image using a captured image of the same photographing region photographed through a color camera and a thermal image camera;
A second step of acquiring a color image and a thermal image for a target plant photographed through the color camera and the thermal camera,
A third step of separating a leaf area having a corresponding color based on the leaf color of the plant in the color image acquired in the second step,
A fourth step of setting an area corresponding to the leaf area in the color image in the thermal image based on the mapping parameter calculated in the first step,
And a fifth step of calculating an average value of pixel values for the leaf areas of the thermal image set in the fourth step and obtaining the leaf temperature of the plant. .

The method according to claim 1,
Wherein the first step includes the steps of extracting at least one marker for the same object by performing a binarization conversion according to brightness for each shot image provided through the color camera and the thermal camera,
Comparing the coordinate value of the marker extracted from the color image with the coordinate value of the marker extracted from the column image with respect to the corresponding object position, and setting the difference value as a mapping parameter. Measurement of Leaf Temperature Using Image.

3. The method of claim 2,
In the first step, the imaging object for acquiring the mapping parameters is composed of a test module in which black and white are arranged in a lattice form, and a white area of the test module in the color image and the thermal image is set as a marker Wherein the method comprises the steps of: