KR102489183B1 - Method And Apparatus for Providing Plant Growth Information Based on Plant Image - Google Patents

Abstract

Disclosed are a method and device for providing plant growth information based on a plant image. The embodiment of the present invention provides a method and device for providing plant growth information based on a plant image, wherein plant types for new plant images are classified using a plant classification model pre-learned for plant images taken, and then plant growth information corresponding to the plant type is extracted and provided. Therefore, individual's competitiveness can be improved by receiving and learning a required learning process.

Description

Method and apparatus for providing plant growth information based on plant image {Method And Apparatus for Providing Plant Growth Information Based on Plant Image}

An embodiment of the present invention relates to a method and apparatus for providing plant growth information based on plant images.

The contents described below merely provide background information related to the present embodiment and do not constitute prior art.

With the spread of smart phones and the development of deep learning-based image recognition technology, it is possible to acquire information after recognizing various objects using a camera provided in a smart phone. In the field of plant recognition, various technological researches are being conducted, and services based on research in the field of plant recognition can be provided to mobile devices.

In the case of Europe and the Americas, various non-profit organizations collect and distribute plant data with the voluntary participation of users, providing the soil for technological progress in the field of plant recognition. When collecting data, the efficiency of data collection is increased by using technology developed based on data to select plants classified by machines by humans.

In Korea, it is difficult to find contributions from professional organizations that collect and distribute large amounts of plant images. In the case of plants, it is necessary to systematically collect, classify, and distribute data on plants inhabiting Korea, considering that the distribution of species inhabiting each continent is different.

It requires an artificial intelligence-based plant image discrimination technology for identifying plants based on plant images and determining pests and diseases of plants.

This embodiment classifies plant types for new plant images by using a plant classification model pretrained on plant images of plant images, and then extracts and provides plant growth information corresponding to the plant type based on plant images. An object of the present invention is to provide a method and apparatus for providing growth information.

According to one aspect of this embodiment, the plant image acquisition unit for obtaining a new plant image photographed plants; a plant type classification unit for classifying a plant type for the new plant image using a pre-learned plant classification model; and a plant growth information extractor extracting plant growth information corresponding to the plant type.

According to another aspect of this embodiment, the process of acquiring a new plant image photographed plants; Classifying a plant type for the new plant image using a pre-learned plant classification model; and extracting plant growth information corresponding to the plant type.

As described above, according to the present embodiment, plant types for new plant images are classified using a plant classification model pre-learned for plant images photographed, and then plant growth information corresponding to the plant type is extracted. There are effects that can be provided.

1 is a diagram schematically showing a plant growth information providing device according to an embodiment.
2 is a diagram showing growth characteristics for each plant type according to the present embodiment.
3 is a diagram showing a light-related decision tree for the growth of rosemary according to the present embodiment.
4 is a diagram showing a water-related decision tree for the growth of rosemary according to the present embodiment.
5 is a diagram illustrating recognition of a stem, leaf, fruit, and root of a plant according to the present embodiment.
6 is a flowchart illustrating a plant type classification method using plant correlation information according to an exemplary embodiment.
7 is a diagram showing classification of plant types based on the chlorophyll content of leaves and stems according to the present embodiment.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically showing a plant growth information providing device according to an embodiment.

The plant growth information providing device 100 according to the present embodiment includes a learning image acquisition unit 110, a plant information acquisition unit 120, a learning data preprocessing unit 130, a plant classification model generation unit 140, a learning unit ( 150), plant image acquisition unit 160, plant data preprocessing unit 170, plant type classification unit 180, plant growth information extraction unit 190, survey information acquisition unit 192, plant growth guide providing unit ( 194). Components included in the plant growth information providing device 100 are not necessarily limited thereto.

Each component included in the plant growth information providing device 100 is connected to a communication path that connects software modules or hardware modules inside the device and can organically operate with each other. These components communicate using one or more communication buses or signal lines.

Each component of the plant growth information providing apparatus 100 shown in FIG. 1 means a unit that processes at least one function or operation, and may be implemented as a software module, a hardware module, or a combination of software and hardware. .

The learning image acquiring unit 110 acquires a plant image for learning by photographing a plant. The learning image acquisition unit 110 acquires a plant image obtained by photographing the entire plant.

The plant information acquiring unit 120 acquires a reference image and additional information (photographing time, photographing date, plant part, area information) on the plant.

The learning data pre-processing unit 130 matches the reference image and additional information to the plant image for learning. The training data pre-processing unit 130 converts the training plant image into training image RGB pixel values, and matches the training image RGB pixel values with the reference image and additional information.

The learning data pre-processing unit 130 extracts photographing time, photographing date, plant part, and region information from the additional information. The training data pre-processor 130 converts the training plant image into training image RGB pixel values having a value of 0 to 255.

The training data pre-processing unit 130 calculates the average value (Mean), standard deviation value (STDV), minimum value (Min), and maximum value (Max) for the RGB pixel values of the training image, and calculates the average value, standard deviation value, minimum value, and maximum value. Normalize based on value.

The training data pre-processing unit 130 extracts only values between 0 and 1 from the normalized training image RGB pixel values and performs pre-processing to remove outliers to generate pre-processing data for training.

The learning data pre-processing unit 130 compares a plurality of plant images for learning by date and by temperature, and extracts a part in which a color difference exceeds a predetermined threshold value. The learning data pre-processing unit 130 divides the entire area of all plant images for learning, each taken by date and temperature, into preset areas. The learning data pre-processing unit 130 calculates RGB values for each region of all plant images for learning, each photographed for each date and temperature. The learning data pre-processing unit 130 compares the RGB values of each area with each other and extracts areas that occur above a preset threshold.

When a plurality of leaves, stems, fruits, and roots are recognized in the learning plant image, the learning data pre-processing unit 130 extracts dark values for each of the plurality of leaves, stems, fruits, and roots. The learning data pre-processing unit 130 recognizes the shadow object as a shadow object hidden in the recognized leaf when the shadow value is equal to or greater than the predetermined threshold value, and removes the shadow object from the learning candidates. The learning data pre-processing unit 130 extracts the shapes of objects other than the shadow objects among the plurality of leaves, stems, fruits, and roots. The learning data pre-processing unit 130 selects one object having the entire shape among the shapes of the remaining objects as the final object for learning. The training data pre-processing unit 130 converts the final training object into training image RGB pixel values.

The learning data pre-processing unit 130 recognizes the shape of a leaf in a plant image for learning. The learning data pre-processing unit 130 determines the type of plant according to the leaf shape. The learning data pre-processing unit 130 determines the number of leaves in the plant image for learning. The learning data pre-processing unit 130 compares the number of leaves according to the season with the number of leaves according to the type of plant to determine current season information.

The learning data pre-processing unit 130 recognizes the shape of the fruit in the plant image for learning. The learning data pre-processing unit 130 determines the type of plant according to the shape of the fruit. The learning data pre-processing unit 130 determines the number of fruits in the plant image for learning. The learning data pre-processing unit 130 compares the number of fruits by season with the number of fruits according to plant types to determine current time information.

The plant classification model generating unit 140 creates a plant classification model by modeling a correlation between a plant image for learning, a reference image, and additional information. The plant classification model generation unit 140 creates a plant classification model by modeling the correlation between pre-processing data for learning, a reference image, and additional information obtained by performing pre-processing on RGB pixel values of the training image.

The plant classification model generation unit 140 sets the preprocessing data for learning of the plant image for learning as an independent variable, sets the reference image and additional information as a dependent variable, and models the correlation between the independent variable and the dependent variable. Create a plant classification model.

The plant classification model generation unit 140 divides the plant image for learning into a leaf area, a stem area, a fruit area, and a root area. Thereafter, the plant classification model generating unit 140 correlates the leaf image, stem image, fruit image, root image and additional information of the plant included in the reference image by matching the photographing time, photographing date, plant part, and regional information. to create a plant classification model.

The plant classification model generation unit 140 generates a plant classification model by reflecting the season information determined by the learning data preprocessing unit 130 to additional information. The plant classification model generation unit 140 generates a plant classification model by reflecting the time information determined by the learning data preprocessing unit 130 to the additional information.

The learning unit 150 inputs a classification result of classifying plant types for a plant image for learning using the plant classification model as learning data, and reflects the learning result to the plant classification model.

The learning unit 150 forms a cluster as a result of classifying RGB pixel values for learning, a reference image, and additional information using SVM (Support Vector Machine) and KNN (K-Nearest Neighbor), sets the corresponding cluster as a threshold, and Classify plant types.

When the learning unit 150 performs learning by inputting the classification result of classifying the plant type for the plant image for learning using the plant classification model as learning data, the number of leaves according to the season (date) and the chlorophyll value of the leaf In addition, the result of learning the presence or absence of fruits is reflected in the plant classification model.

The plant image acquiring unit 160 acquires a new plant image by photographing the plant.

The plant data pre-processing unit 170 converts the new plant image into plant image RGB pixel values having a value of 0 to 255. The plant data preprocessing unit 170 calculates the average value (Mean), standard deviation value (STDV), minimum value (Min), and maximum value (Max) of the plant image RGB pixel values for the new plant image, and calculates the average value and standard deviation. Normalize based on value, minimum and maximum values.

The plant data pre-processing unit 170 extracts only values between 0 and 1 from the normalized plant image RGB pixel values and performs pre-processing to remove outliers to generate new pre-processed data.

The plant type classification unit 180 classifies the plant type of the new plant image by using a pre-learned plant classification model. The plant type classification unit 180 classifies plant types for new pre-processed data using a pre-learned plant classification model, and extracts a plant name corresponding to the plant type.

When classifying plant image RGB pixel values, reference images, and additional information using SVM and KNN, the plant type classification unit 180 performs clustering by firstly performing SVM and then secondarily performing KNN. The plant type classification unit 180 classifies the plant type of the corresponding plant based on the clustering result.

When classifying plant image RGB pixel values, reference images, and additional information using SVM and KNN, the plant type classification unit 180 performs clustering by firstly performing KNN and then secondarily performing SVM. The plant type classification unit 180 classifies the plant type of the corresponding plant based on the clustering result.

The plant type classification unit 180, when there is a plant type that matches the new plant image obtained using the plant classification model by a preset value (eg, 70%) or more, classifies the corresponding plant type as angiosperm, gymnosperm, It is determined as one of moss plants, vascular plants, seed plants, leaf plants, ferns, selective plants, conifers, and red algae plants.

The plant type classification unit 180 estimates a plant size based on a new plant image in order to increase accuracy when extracting plant growth information corresponding to a plant type (plant name), and determines whether the plant type is correct based on the plant size. additional authentication.

The plant type classification unit 180 extracts the shape, thickness, pattern, and color based on the new plant image in order to increase accuracy when extracting plant growth information corresponding to the plant type (plant name), and extracts the shape, thickness, pattern, and color. Based on this, it is additionally certified whether the plant type is correct.

The plant type classification unit 180 extracts a leaf shape based on a new plant image in order to increase accuracy when extracting plant growth information corresponding to a plant type (plant name). The plant type classification unit 180 additionally authenticates whether or not it has a shape similar to that of a leaf.

The plant type classification unit 180 extracts the color of a leaf based on a new plant image in order to increase accuracy when extracting plant growth information corresponding to a plant type (plant name). The plant type classification unit 180 converts pixel values for leaf colors. The plant type classification unit 180 determines the state of chlorophyll based on the pixel value of the leaf color. The plant type classification unit 180 additionally authenticates whether or not the plant type is correct based on the state of chlorophyll.

The plant type classification unit 180 classifies the new plant image into a leaf area, a stem area, a fruit area, and a root area. Thereafter, the plant type classification unit 180 checks the leaf area chlorophyll content based on the leaf area RGB pixel values for the leaf area. The plant type classification unit 180 checks the chlorophyll content of the stem region based on RGB pixel values of the stem region. The plant type classification unit 180 uses the plant classification model to determine the matching rate between the leaf area chlorophyll content and the stem area chlorophyll content and the leaf area chlorophyll content matched according to the photographing time, photographing date, and region information included in the additional information. Categorize plant types based on

The plant type classification unit 180 extracts date information, shooting time, and shooting location (GPS information) from the additional information. The plant type classification unit 180 extracts a first plant image feature from a new plant image by using a plant classification model. The plant type classification unit 180 generates plant association information based on a photographing time, a photographing location (GPS information), and first plant image characteristics. The plant type classification unit 180 classifies the plant type of the new plant image by combining the plant association information and the first plant image characteristics.

The plant type classification unit 180 extracts a first plant image feature normalized to a real value between -1 and 1 from a new plant image by a convolution layer in a convolutional neural network (CNN).

When plant type classification unit 180 generates plant correlation information, when classifying the species of plants photographed based on the photographing time and photographing location, flowering time and flowering place information are reflected as weights for classifying plant types. .

The plant growth information extraction unit 190 extracts plant growth information (light environment, preferred growth environment, non-preferred growth environment, optimal temperature for growth, ventilation or drainage, pests, and pest types) corresponding to the plant type (plant name).

The survey information acquisition unit 192 additionally acquires survey information corresponding to the plant type (plant name). The survey information acquisition unit 192 is a growing place (outdoor, indoor), survey light reception information (sunshine, half-sun, half-shade, shade), watering (three times a week, once a week, once a week, once a week, once a week) ), survey information including the amount of water is obtained.

The plant growth guide providing unit 194 provides a plant growth guide based on plant growth information and survey information.

The plant growth guide providing unit 194 extracts light environment, preferred growth environment, non-preferred growth environment, optimum temperature for growth, ventilation or drainage, pests, pests, and growth factors for types of pests from plant growth information corresponding to the plant type (plant name). do. The plant growth guide providing unit 194 extracts environmental factors related to a growing place, light receiving information, water cycle, and watering amount from survey information. The plant growth guide providing unit 194 compares the growth factor and the environmental factor and generates a plant growth guide based on information that does not match.

The plant growth guide providing unit 194 compares factors for the growing place (outdoor, indoor) and light reception information (sunlight, half-sun, half-shade, shade) extracted from the survey information with factors for the light environment extracted from the plant growth information. . The plant growth guide providing unit 194 extracts the watering cycle (three times a week, once a week, once a week, once a week, once a week, once a week) and the preferred growth factor extracted from the plant growth information. Plant growth guides are created by comparing with environmental factors.

The plant growth guide providing unit 194 determines whether the growth environment is sunny, half-sunny, half-shade, or shaded based on the growing place (outdoor, indoor) and survey light-receiving information (sunshine, half-sun, half-shade, or shade) included in the survey information. determine which one The plant growth guide provider 194 determines whether the growth environment is humid or dry based on the water cycle (three times a week, once a week, once a week, once a week, once a week) and the amount of watering included in the survey information. determine whether

The plant growth guide providing unit 194 determines whether the growth environment is sunny, half-sunny, half-shade, or shaded, and determines whether the growth environment is humid or dry. Inconsistent information is extracted by comparing with the environment, optimum temperature for growth, ventilation or drainage, respectively. The plant growth guide providing unit 194 generates a plant growth guide including growth guides for inconsistent information.

The plant growth guide providing unit 194 generates a growth guide instructing the plant to relocate when the growth environment is identified as sunny but the light environment is half-sun. The plant growth guide providing unit 194 generates a growth guide instructing to reduce the amount of water when the growth environment is determined to be dry but the preferred environment is high humidity.

2 is a diagram showing growth characteristics for each plant type according to the present embodiment.

The plant growth information providing device 100 stores growth characteristics of each plant type in a database.

The plant growth information providing device 100 includes 'Tillandsia', 'Dissidia', 'Geranium', 'Monstera', 'Olive tree', 'Eucalyptus', 'Bengal gum tree', and 'Lemon tree' as plant types. ', 'Golden Tree', 'Alocasia', 'Begonia', 'Areca Palm', 'Bird of Paradise', 'Rosemary', 'Yulma', 'Margaret', 'Skin Dapsus', 'Lucky Tree', 'Ivy', 'Trian' and the like are included.

The plant growth information providing device 100 is a light environment (half-sun, half-shade), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (direct sunlight), and an appropriate temperature for growth (15 to 25 ℃), high temperature (strong), low temperature (-), high humidity (strong), dryness (weak), ventilation or drainage (-), pest (rare), pest type (-), and other information (-) are matched and saved do.

The plant growth information providing device 100 is a light environment (half-sun, half-shade), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (direct sunlight), and an appropriate temperature for growth (18 to 28 ° C) as growth characteristics for 'Dissidia'. ), high temperature (-), low temperature (-), high humidity (weak), dryness (-), ventilation or drainage (-), pest (rare), pest type (-), and other information (-) are matched and stored. .

The plant growth information providing device 100 has growth characteristics for 'geranium', such as light environment (sunny, semi-sunny), preferred growth environment (-), non-preferred growth environment (high temperature and humidity), optimum growth temperature (16 ~ 20 ℃) , high temperature (weak), low temperature (-), high humidity (very weak), dry (very strong), ventilation or drainage (very important), pest (frequent), pest type (gray mold disease), others (low temperature and high humidity) Pest occurrence) information is matched and stored.

The plant growth information providing device 100 is a light environment (half shade), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (direct sunlight), an appropriate temperature for growth (20 ~ 25 ° C), and a high temperature as growth characteristics for 'Monstera'. (strong), low temperature (weak), high humidity (strong), dryness (-), ventilation or drainage (-), pest damage (-), pest type (mites, scale insects), and other information (-) are matched and stored. .

The plant growth information providing device 100 is a light environment (sunny), a preferred growth environment (high temperature and dry), a non-preferred growth environment (lack of light), an appropriate temperature for growth (18 ~ 23 ℃), High temperature (-), low temperature (-), high humidity (weak), dryness (strong), ventilation or drainage (very important), pest (-), pest type (-), and other information (-) are matched and stored.

The plant growth information providing device 100 is a light environment (sunny, semi-sunny), a preferred growth environment (light and dry), a non-preferred growth environment (-), and an appropriate growth temperature (18 ~ 23 ℃) as growth characteristics for 'eucalyptus' ), high temperature (-), low temperature (-), high humidity (-), dryness (-), ventilation or drainage (very important), pest (rare), pest type (mite, powdery mildew), other (winter blistering) information Match and save.

The plant growth information providing device 100 is a light environment (half shade), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (vulnerable to root over-humidity), and an appropriate temperature for growth (25 ~ 30℃), high temperature (-), low temperature (weak), high humidity (-), dryness (weak), ventilation or drainage (drainage is important), pest (-), pest type (-), and other (-) matching information and save it

The plant growth information providing device 100 is a light environment (sunny), a preferred growth environment (light and dry), a non-preferred growth environment (-), an appropriate temperature for growth (16 ~ 30 ℃), High temperature (-), low temperature (weak), high humidity (weak), dryness (-), ventilation or drainage (-), pest (frequent), pest type (ulcer disease, black spot disease), and other information (-) are matched and stored do.

The plant growth information providing device 100 is a light environment (half shade), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (low temperature and humidity), an appropriate growth temperature (15 ~ 30 ℃), High temperature (-), low temperature (weak), high humidity (-), dryness (strong), ventilation or drainage (very important), pests (-), pest types (mites, scale insects), and other (-) Save.

The plant growth information providing device 100 has growth characteristics for 'Alocasia', such as light environment (half-sun), preferred growth environment (high temperature and humidity), non-preferred growth environment (low temperature and humidity), and optimal growth temperature (15 ~ 25 ℃). , high temperature (-), low temperature (weak), high humidity (-), dry (moderate), ventilation or drainage (very strong), pest (-), pest type (soft rot, mite, scale insect), other (-) information Match and save.

The plant growth information providing device 100 has growth characteristics for 'Begonia', such as light environment (half-sun), preferred growth environment (slightly dry, ventilated), non-preferred growth environment (direct sunlight, high temperature and humidity), optimum temperature for growth (10 ~ 20℃), high temperature (weak), low temperature (weak), high humidity (weak), dry (-), ventilation or drainage (very important), pest (frequent), pest type (powder disease, wilt disease, gray mold disease), others (In case of high temperature and high humidity, damage increases) Match and save information.

Plant growth information providing device 100 is a light environment (half-sun, half-shade), preferred growth environment (high temperature and humidity), non-preferred growth environment (dry, direct sunlight, cold), optimum growth temperature ( 18 ~ 25℃), high temperature (-), low temperature (weak), high humidity (-), dryness (-), ventilation or drainage (very important), pest (frequent), pest type (mealy scales, mites), other ( Mite occurs when dry) information is matched and stored.

The plant growth information providing device 100 is a light environment (sunny, half-sun, half-shade), a preferred growth environment (high humidity, sunlight), and a non-preferred growth environment (direct sunlight, darkness, low temperature, excessive humidity) as growth characteristics for 'Bird of Paradise' , optimum temperature for growth (15 ~ 25℃), high temperature (-), low temperature (-), high humidity (-), dryness (strong), ventilation or drainage (very important), pest (-), pest type (-), Other information (a lot of front drying, air humidity maintenance) is matched and stored.

The plant growth information providing device 100 is a light environment (sunny, half sunny, half shade), a preferred growth environment (dry), a non-preferred growth environment (high temperature and humidity), and an appropriate growth temperature (15 to 25 ℃), high temperature (weak), low temperature (strong), high humidity (weak), dry (strong), ventilation or drainage (very important), pest (rare), pest type (powdery mildew), and other (-) Save.

The plant growth information providing device 100 is a light environment (sunny, semi-sunny), preferred growth environment (sunlight, low temperature, ventilation), non-preferred growth environment (high temperature and humidity), optimum growth temperature (15 ~ 20℃), high temperature (-), low temperature (weak), high humidity (-), dryness (weak), ventilation or drainage (important), pest (-), pest type (mealy scales, mites), other (-) Match and store information.

The plant growth information providing device 100 is a light environment (half-sun), a preferred growth environment (sunlight), a non-preferred growth environment (excessive humidity), an appropriate temperature for growth (15 ~ 23 ℃), and a high temperature (-), low temperature (strong), high humidity (-), dryness (-), ventilation or drainage (very important), pest (rare), pest type (-), and other information (-) are matched and stored.

The plant growth information providing device 100 is a light environment (half shade, shade), a preferred growth environment (high humidity, water), a non-preferred growth environment (-), and an appropriate growth temperature (20 ~ 25 ℃) as growth characteristics for 'Skindapsus'. ), high temperature (-), low temperature (-), high humidity (-), dry (strong), ventilation or drainage (very important), pest (rare), pest type (mealy scales, mites, thrips), other (roots) Too much damage) Match and save information.

The plant growth information providing device 100 is a light environment (half shade), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (direct sunlight), an appropriate growth temperature (21 ~ 25 ℃), and a high temperature ( -), low temperature (-), high humidity (weak), dryness (-), ventilation or drainage (-), pests (-), types of pests (mites, gnats), and other (-) information are matched and stored.

The plant growth information providing device 100 is a light environment (sunny, half sunny, half shade), preferred growth environment (high humidity), non-preferred growth environment (direct sunlight, excessive humidity), optimum temperature for growth (16 ~ 20℃), high temperature (weak), low temperature (-), high humidity (-), dry (weak), ventilation or drainage (very important), pest (-), pest type (mite, scale insect), others (high temperature dry City pest) information is matched and stored.

The plant growth information providing device 100 is a light environment (half-sun), a preferred growth environment (high temperature and humidity), a non-preferred growth environment (high temperature and dryness, poor ventilation), and an appropriate temperature for growth (16 ~ 30℃), high temperature (-), low temperature (-), high humidity (-), dryness (-), ventilation or drainage (very important), pests (frequent), pest types (mites, aphids), others (beware of excessive humidity in winter) ) information is matched and stored.

3 is a diagram showing a light-related decision tree for the growth of rosemary according to the present embodiment.

When 'rosemary' is selected, the plant growth information providing device 100 checks whether the growing place is 'outdoor' or 'indoor' (S310).

The plant growth information providing device 100 outputs 'outdoor' or 'indoor' on the screen to identify a place where 'rosemary' is grown, receives either 'outdoor' or 'indoor', and selects 'rosemary'. It is confirmed that the growing place is 'outdoor' (S320).

When the plant growth information providing device 100 confirms that the place where 'rosemary' grows is 'outdoor', the amount of light received by 'rosemary' is selected from 'sunny', 'half-shaded', 'half-shaded', and 'shaded'. Which one is confirmed (S330).

The plant growth information providing device 100 outputs 'sunny', 'half-sun', 'half-shade', and 'shade' on the screen to check the amount of light that 'rosemary' receives, and displays 'sun' and 'half-shade' on the screen. Any one of 'sunshine', 'half-shade', and 'shade' is input, and it is confirmed that the amount of light received by 'Rosemary' is 'sunshine' (S340).

Although it is described in FIG. 3 that steps S310 to S340 are sequentially executed, it is not necessarily limited thereto. In other words, since it will be applicable to changing and executing the steps described in FIG. 3 or executing one or more steps in parallel, FIG. 3 is not limited to a time-series sequence.

As described above, the light-related decision tree for the growth of rosemary according to the present embodiment described in FIG. 3 can be implemented as a program and recorded on a computer-readable recording medium. The computer-readable recording medium on which the program for implementing the light-related decision tree for the growth of rosemary according to the present embodiment is recorded includes all kinds of recording devices storing data that can be read by a computer system.

4 is a diagram showing a water-related decision tree for the growth of rosemary according to the present embodiment.

The plant growth information providing device 100 checks the watering cycle for 'rosemary' (S410).

The plant growth information providing device 100 displays 'three times a week', 'once a week', 'once two weeks', 'once three weeks' on the screen to check the watering cycle of 'rosemary'. , and input any one of 'three times a week', 'once a week', 'once two weeks', 'once a week', and the amount of watering 'rosemary' is 'three times a week' It is confirmed that it is (S420).

The plant growth information providing device 100 checks whether enough water is given to 'rosemary' so that water comes out from the bottom of the pot (S430).

The plant growth information providing device 100 determines whether the amount of water that is given to 'rosemary' under the pot is sufficient to come out from under the pot, 'yes', 'not so', 'yes', 'not so' is displayed on the screen, and one of 'yes', 'no', 'yes', and 'no' is input, and goes to 'Rosemary' to confirm that the amount of light received is ''not' (S440).

Although it is described in FIG. 4 that steps S410 to S440 are sequentially executed, it is not necessarily limited thereto. In other words, since it will be applicable to changing and executing the steps described in FIG. 4 or executing one or more steps in parallel, FIG. 4 is not limited to a time-series sequence.

As described above, the water-related decision tree for the growth of rosemary according to the present embodiment described in FIG. 4 can be implemented as a program and recorded on a computer-readable recording medium. A computer-readable recording medium on which a program for implementing a water-related decision tree for the growth of rosemary according to the present embodiment is recorded includes all kinds of recording devices storing data readable by a computer system.

5 is a diagram illustrating recognition of a stem, leaf, fruit, and root of a plant according to the present embodiment.

The plant growth information providing device 100 acquires a plant image. The plant growth information providing device 100 provides plant growth information based on plant images. The plant growth information providing device 100 acquires a plant image and a preset reference image together.

The plant growth information providing device 100 recognizes leaves, stems, fruits, and roots of a corresponding plant by comparing plant images with reference images. The plant growth information providing device 100 determines the corresponding plant type based on the recognized leaves, stems, fruits, and roots. The plant growth information providing device 100 extracts plant growth information corresponding to the determined plant type (plant name).

When acquiring a reference image, the plant growth information providing apparatus 100 may acquire additional information including plant part, year, month, date, and area information for the reference image. The plant growth information providing device 100 may extract plant part, year, month, date, and area information corresponding to the reference image from the web or a pre-stored database.

The plant growth information providing device 100 determines the type of plant (angiosperms, gymnosperms, moss plants, vascular plants, seed plants, fronds) when the obtained plant image and the reference image match a predetermined number (eg, 70%) or more. plants, ferns, selective plants, conifers, red algae plants).

The plant growth information providing device 100 estimates the plant size based on the plant image to increase accuracy, and additionally authenticates whether the plant type is correct based on the plant size. The plant growth information providing device 100 extracts the shape, thickness, pattern, and color based on the plant image to increase accuracy, and additionally authenticates whether the plant type is correct based on the shape, thickness, pattern, and color. . The plant growth information providing device 100 extracts a leaf shape based on a plant image to increase accuracy, and additionally verifies whether or not it has a shape similar to that of the leaf.

The plant growth information providing device 100 extracts the color of a leaf based on a plant image, converts a pixel value for the color of the leaf, and determines the state of chlorophyll based on the pixel value for the color of the leaf to increase accuracy. and additionally verify whether the plant type is correct based on the state of chlorophyll.

The plant growth information providing device 100 acquires a plant image. The plant growth information providing device 100 acquires a reference image and additional information. The plant growth information providing device 100 determines a plant by recognizing roots, stems, leaves, and fruits, respectively, based on plant images.

6 is a flowchart illustrating a plant type classification method using plant correlation information according to an exemplary embodiment.

The plant growth information providing device 100 may receive and store at least one plant image from an external device (S610).

The plant growth information providing device 100 additionally extracts date information and location information from the stored plant image (S620). The plant growth information providing device 100 receives a photographing time and a photographing position from a plant image. In step S620, the plant growth information providing apparatus 100 may extract date information and GPS information from the plant image.

The plant growth information providing apparatus 100 may extract at least one image feature from the received plant image using a deep learning algorithm for learning (S630). The plant growth information providing device 100 extracts a photographing time and a photographing position from the obtained plant image. The plant growth information providing device 100 extracts a first plant image feature from a plant image by using a pre-learned plant classification model. The plant growth information providing device 100 receives at least one plant image and extracts image features for learning. The plant growth information providing apparatus 100 extracts a first plant image feature from at least one plant image for learning. The plant growth information providing apparatus 100 extracts features using a plant image as an input value by using a convolutional neural network (CNN) for learning.

In step S630, the plant growth information providing apparatus 100 extracts a first plant image feature from the plant image by a convolution layer in the CNN for learning. The plant growth information providing device 100 extracts a first plant image feature normalized to a real value between -1 and 1.

The plant growth information providing device 100 generates plant correlation information based on the input values (S640).

The plant growth information providing device 100 generates plant correlation information based on the photographing time, location, and first plant image characteristics. The plant growth information providing apparatus 100 determines the plant type of the plant image by combining the plant relevance information and the first plant image feature.

The plant growth information providing device 100 generates plant association information including values expressing associations between plant categories. The plant growth information providing apparatus 100 generates a fixed low-dimensional second plant image feature using a first plant image feature, a capturing time, and a capturing location as input values.

The plant growth information providing device 100 performs deep learning learning. The plant growth information providing apparatus 100 uses a first plant image feature, a photographing time, and a photographing location as input values, and generates plant association information by a learning model. The plant growth information providing apparatus 100 extracts a first plant image feature and a second plant image feature using a photographing time and a photographing location as input values.

The plant growth information providing device 100 combines the generated plant association information and the first plant image feature (S650). In step S650, the plant growth information providing apparatus 100 may combine the second plant image feature and the first plant image feature.

The plant growth information providing device 100 classifies the type of at least one plant captured in the plant image based on the calculated probability data (S660). The plant growth information providing apparatus 100 classifies plant images with high precision and high reliability by considering the flowering time and flowering location information when classifying the species of the photographed plants by reflecting the photographing time and photographing location.

The plant growth information providing device 100 extracts plant growth information corresponding to the type of plant (S670).

Although it is described in FIG. 6 that steps S610 to S670 are sequentially executed, it is not necessarily limited thereto. In other words, since it will be applicable to changing and executing the steps described in FIG. 6 or executing one or more steps in parallel, FIG. 6 is not limited to a time-series sequence.

As described above, the plant type classification method using plant association information according to the present embodiment described in FIG. 6 is implemented as a program and may be recorded on a computer-readable recording medium. A computer-readable recording medium on which a program for implementing a plant type classification method using plant association information according to the present embodiment is recorded includes all types of recording devices storing data readable by a computer system.

7 is a diagram showing classification of plant types based on the chlorophyll content of leaves and stems according to the present embodiment.

The plant growth information providing device 100 obtains a plant image obtained by photographing a plant. The plant growth information providing device 100 acquires a reference image and additional information (photographing time, photographing date, plant part, area information) on the plant.

The plant growth information providing device 100 matches a plant image with a reference image and additional information. The plant growth information providing apparatus 100 creates a plant classification model by modeling a correlation between a plant image, a reference image, and additional information.

The plant growth information providing device 100 uses a plant classification model to determine the type of plant image (angiosperms, gymnosperms, moss plants, vascular plants, seed plants, frond plants, ferns, selective plants, conifers, red algae plants) decide

The plant growth information providing device 100 divides a plant image into a plurality of regions. The plant growth information providing device 100 extracts leaf, mid-stage, and fruit parts from a plurality of areas. The plant growth information providing device 100 converts RGB pixel values in order to identify colors corresponding to chlorophyll in leaves, mid-stage, and fruit parts. The plant growth information providing device 100 matches RGB pixel values with reference images and additional information.

The plant growth information providing apparatus 100 creates a plant classification model by modeling a correlation between preprocessed data obtained by preprocessing RGB pixel values, a reference image, and additional information. The plant growth information providing apparatus 100 inputs the result of determining the type of plant image as learning data, and reflects the learning result to the plant classification model. The plant growth information providing device 100 converts each of a plurality of plant images into RGB pixel values having a value of 0 to 255.

The plant growth information providing device 100 calculates the average value (Mean), standard deviation value (STDV), minimum value (Min), and maximum value (Max) of the RGB pixel values, and calculates the average value, standard deviation value, minimum value, and maximum value. Normalize based on .

The plant growth information providing device 100 extracts only values between 0 and 1 from a plurality of normalized RGB pixel values and then performs preprocessing to remove outliers to generate preprocessed data.

The plant growth information providing device 100 sets preprocessing data for plant images as independent variables, and sets reference images and additional information as dependent variables. The plant growth information providing device 100 creates a plant classification model by modeling a correlation between an independent variable and a dependent variable.

The plant growth information providing device 100 classifies RGB pixel values, reference images, and additional information using SVM (Support Vector Machine) and KNN (K-Nearest Neighbor) to form a cluster and set the cluster as a critical point. to classify the type of plant.

When a plant is photographed, the plant growth information providing device 100 acquires an entire plant image obtained by photographing the entire plant. The plant growth information providing device 100 classifies the captured plant image into a plurality of regions, and extracts leaves, stems, and fruits from among the plurality of regions. The plant growth information providing device 100 calculates RGB pixel values for each leaf, stem, and fruit portion.

The plant growth information providing device 100 extracts a leaf color based on a new plant image to increase accuracy when extracting plant growth information corresponding to a plant type (plant name). The plant growth information providing device 100 converts pixel values for leaf colors. The plant type classification unit 180 determines the state of chlorophyll based on the pixel value of the leaf color. The plant type classification unit 180 additionally authenticates whether or not the plant type is correct based on the state of chlorophyll.

The plant growth information providing device 100 divides the new plant image into a leaf area, a stem area, a fruit area, and a root area. Then, the plant growth information providing device 100 checks the chlorophyll content of the leaf area based on the RGB pixel values of the leaf area. The plant growth information providing apparatus 100 checks the chlorophyll content of the stem region based on RGB pixel values of the stem region. The plant growth information providing device 100 matches the chlorophyll content of the leaf area and the chlorophyll content of the stem area with the chlorophyll content of the leaf area matched according to the photographing time, photographing date, and region information included in the additional information using the plant classification model. Classify plant types based on

When classifying RGB pixel values, reference images, and additional information using SVM and KNN, the plant growth information providing apparatus 100 performs clustering by firstly performing SVM and then secondarily performing KNN, and clustering Based on the results, the type of plant is classified.

When classifying RGB pixel values, reference images, and additional information using SVM and KNN, the plant growth information providing apparatus 100 performs clustering by firstly performing KNN and then secondarily performing SVM, and clustering Based on the results, the type of plant is classified.

The plant growth information providing device 100 first classifies the type of the corresponding plant by using its own plant classification model for the plant image. The plant growth information providing device 100, when it is difficult to classify the captured plant image using its own plant classification model, goes through a process of secondarily learning the plant image, and uses the plant classification model reflecting the learning result to Classify plant types.

The above description is merely an example of the technical idea of the present embodiment, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of this embodiment should be interpreted according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of this embodiment.

100: plant growth information providing device
110: learning image acquisition unit
120: plant information acquisition unit
130: training data pre-processing unit
140: plant classification model generating unit
150: learning unit
160: plant image acquisition unit
170: plant data pre-processing unit
180: plant type classification unit
190: plant growth information extraction unit
192: survey information acquisition unit
194: plant growth guide providing unit

Claims (5)

Plant image acquisition unit for obtaining a new plant image photographed plants;
a plant type classification unit for classifying a plant type for the new plant image using a pre-learned plant classification model;
a plant growth information extractor extracting plant growth information corresponding to the plant type;
a learning image acquiring unit acquiring a plant image for learning by photographing the plant;
a plant information acquisition unit acquiring a reference image and additional information about the plant;
a learning data pre-processor matching the reference image and the additional information to the learning plant image;
a plant classification model generation unit generating a plant classification model by modeling a correlation between the training plant image, the reference image, and the additional information;
a learning unit inputting a classification result obtained by classifying plant types for the plant image for learning using the plant classification model as learning data, and reflecting the result of learning to the plant classification model;
The learning data pre-processing unit extracts a dark value for each of the plurality of leaves, stems, fruits, and roots when a plurality of leaves, stems, fruits, and roots are recognized in the learning plant image, and the dark value is If it is greater than the set threshold, it is recognized as a shadow object hidden by the shadow of the recognized leaf, the shadow object is removed from the learning candidate, and the shape of the remaining objects excluding the shadow object among the plurality of leaves, stems, fruits, and roots is determined. extract, select one object having the shape of the entire object among the shapes of the remaining objects as the final learning object, recognize the leaf shape in the plant image for learning, determine the plant type according to the leaf shape, and determine the plant for learning Discriminate the number of leaves in the image, determine the current season information by comparing the number of leaves with the number of leaves for each season according to the plant type, recognize the fruit shape in the learning plant image, and determine the plant type according to the fruit shape determining the number of fruits in the learning plant image, comparing the number of fruits by season according to the plant type with the number of fruits to determine current time information, converting the new plant image into plant image RGB pixel values, , The final learning object and the plant image for learning are converted into RGB pixel values of the learning image,
The plant classification model generation unit creates the plant classification model by modeling a correlation between preprocessing data for learning obtained by performing preprocessing on RGB pixel values of the training image, the reference image, and the additional information, and converts the season information to the additional information. generating the plant classification model by reflecting the current time information to the additional information to generate the plant classification model;
When the learning unit performs learning by inputting the classification result of classifying the plant type for the plant image for learning using the plant classification model as learning data, the number of leaves according to the season, the chlorophyll value of the leaf, and the fruit A plant growth information providing device characterized in that the result of learning the presence or absence of is reflected in the plant classification model. According to claim 1,
a survey information acquisition unit that additionally obtains survey information corresponding to the plant type;
A plant growth guide providing unit providing a plant growth guide based on the plant growth information and the survey information.
Plant growth information providing device characterized in that it further comprises. According to claim 2,
The plant growth guide providing unit,
From the plant growth information, light environment, preferred growth environment, non-preferred growth environment, optimum temperature for growth, ventilation or drainage, pests and pests, and growth factors for pest types are extracted, and the growing place, light reception information, water cycle, and water are extracted from the survey information. An apparatus for providing plant growth information, characterized in that for extracting an environmental factor for an amount giving , and generating the plant growth guide based on information that does not match by comparing the growth factor and the environment factor. According to claim 3,
The plant growth guide providing unit,
The factor for the growing place and the light reception information extracted from the survey information is compared with the factor for the light environment extracted from the plant growth information, and the factor for the watering cycle and the watering amount extracted from the survey information The plant growth information providing device characterized in that for generating the plant growth guide by comparing with the factor for the preferred growth environment extracted from the plant growth information. Acquiring a new plant image photographed by the plant image acquisition unit;
Classifying a plant type for the new plant image using a plant classification model pretrained in a plant type classification unit; and
extracting plant growth information corresponding to the plant type in a plant growth information extraction unit;
Acquiring a plant image for learning in which the plant is photographed by a learning image acquiring unit;
Acquiring a reference image and additional information about the plant in a plant information acquisition unit;
matching the reference image and the additional information to the plant image for learning in a learning data pre-processing unit;
generating a plant classification model by modeling a correlation between the plant image for learning, the reference image, and the additional information in a plant classification model generating unit;
a step of inputting a classification result of classifying plant types for the plant image for learning using the plant classification model in a learning unit as learning data, and reflecting the learning result to the plant classification model;
Including, but, when a plurality of leaves, stems, fruits, and roots are recognized in the learning plant image in the learning data preprocessing unit, dark values are extracted for each of the plurality of leaves, stems, fruits, and roots, and the dark values are If it is greater than or equal to a preset threshold, it is recognized as a shadow object hidden by the shadow of the recognized leaf, the shadow object is removed from the learning candidates, and the shape of the remaining objects excluding the shadow object among the plurality of leaves, stems, fruits, and roots is extracted, one object having the shape of the entire object among the shapes of the remaining objects is selected as the final learning object, the leaf shape is recognized in the learning plant image, the plant type is determined according to the leaf shape, and the learning Determines the number of leaves in a plant image, determines current season information by comparing the number of leaves for each season with the number of leaves according to the plant type, recognizes the shape of a fruit in the plant image for learning, and determines the type of plant according to the shape of a fruit determine the number of fruits in the plant image for learning, compare the number of fruits by season according to the plant type with the number of fruits to determine current time information, and convert the new plant image into a plant image RGB pixel value And converting the final learning object and the plant image for learning into RGB pixel values of the learning image,
The plant classification model generating unit creates the plant classification model by modeling the correlation between preprocessing data for learning, the reference image, and the additional information, which is preprocessed on the RGB pixel values of the learning image, and converts the season information into the additional information. to create the plant classification model, and reflect the current time information to the additional information to create the plant classification model;
When learning is performed by inputting the classification result of classifying the plant type for the plant image for learning using the plant classification model in the learning unit as learning data, the number of leaves according to the season, the chlorophyll value of the leaf, the A method for providing plant growth information, characterized in that the result of learning the presence or absence of fruits is reflected in the plant classification model.

Images