JP2002203242A - Plant recognition system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify flowering plants by performing image recognition of a plant that bears flowers. SOLUTION: An image of flowers and leaves being an object is extracted from a digital image of the flowers and the leaves by using a clustering method, and information obtained from the extracted image of the flowers and the leaves is defined as characteristic quantity. One or more characteristic quantities are calculated, and the calculated characteristic quantities and the characteristic quantities of various plants pre-registered in a database are analyzed by using a statistical method to discriminate the kind of a wild plant. A system in which one correct name of a wild plant can be obtained in a recognition result and a system in which several similar wild plants are displayed on the screen and a user can recognize what the wild plants are in the end by observing the plants are constructed, and the kinds of the wild plants and such information carried on a plant dictionary can be displayed on the basis of the recognition results.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image recognition, and more particularly to a system for recognizing an image of a flowering plant.

[0002]

BACKGROUND ART Wild grass can be seen everywhere, such as on roads, mountain roads, and fields. However, we often do not know the names and classifications of wildflowers. The name of the wildflower is tried to be known using the picture book, but it is difficult for an amateur to understand because it is arranged based on the botanical classification. In a portable pictorial book (for example, Yasaka Hayashi, “Sankei Handy Pictorial Book 1 Flowers Blooming in the Field”, Yamato Gorgesha, Tokyo, 1998), 8
Approximately 1000 species of wildflowers are recorded in 4 families. Due to the large variety of wildflowers, it is difficult to find not only specific wildflowers in the picture book, but also wildflowers of the same family. if,
If the technique of automatic recognition can be applied to the identification of wild grass, the name can be easily known even if it is not an expert. Furthermore, with the recent spread of portable computers and digital cameras, an environment for realizing an inexpensive and highly portable automatic recognition system is being prepared. There are many research cases related to the recognition of primitive objects and human faces from still images and the recognition of lung cancer and the like from CT images. But,
There are few reports on recognition of natural objects such as fish, insects, and plants. For example, Hiraoka et al. (Toru Hiraoka, Keiji Yano, Ryuzo Takiyama, "Image Recognition Method Based on Contour and Texture Information-Application to Fish Image Recognition", IEICE Technical Report, PRMU
96-148, no.1, pp.55-62, Jan. 1997) recognizes fish based on texture and shape, but uses images processed from pictorial books instead of images actually taken. ing.
Im et al. (Cholhong Im, Hirobumi Nishida, Toshiyasu L.
Kunii, "Recognizing Plant Species by Leaf Shapes
-A CaseStudy of the Acer Family ", ICPR'98, pp.117
1-1173, Brisbane, Australia, Aug. 1998) recognizes the maple tree from the leaf shape. Sekida and others (Sekida
Iwao, Takio Kurita, Nobuyuki Otsu, "Classification by Complex Autoregressive Model", IEICE (D-II), vol.J73-D-II,
No. 6, pp. 804-811, Jun. 1990) recognizes the leaves of the tree, but the data is small with 16 samples of 5 types. Kanayama et al. (Kazuyoshi Kanayama, Toshio Kawashima, Yoshinao Aoki, "Image Indexing for Plant Data," IEICE Technical Report, PRMU97-1
72, no. 11, pp. 151-158, Nov. 1997), Takemoto et al. (Takemoto
Kiyoka, Masakatsu Korogi, Yoichi Muraoka, "Flower Retrieval System Based on Corolla Features", 1999, IEICE, D.12-66) aims at indexing based on plant photos, but the recognition of wildflowers Not yet. An object of the present invention is to realize a series of systems from photographing to recognition using wild grasses that actually inhabit.

[0003]

SUMMARY OF THE INVENTION An object of the present invention is to recognize an image of a plant having a flower and to identify a flower.

[0004]

In order to achieve the above object, the present invention is a plant recognition system for discriminating the type of a plant based on characteristics of flowers and leaves. And / or extracting an image of a flower and / or a leaf as an object from a digital image of a leaf, obtaining one or more feature amounts from the extracted image of the flower and / or a leaf, and obtaining the obtained feature amount. Then, the type of the plant is determined from the feature amounts of various plants registered in the database in advance. As a result, as the image recognition for the plant having the flower, the image recognition can be automatically performed from the flower image and the leaf image, and the flower can be identified.

To extract the flower and / or leaf image, a clustering method performed in a color space can be used. The feature amount is a value obtained from the image of the extracted flower or leaf, and may be selected from the following definitions. 1. In a flower image, when the distance from the center of gravity G to the contour of the petal is d, and the angle θ with respect to the center of gravity is a horizontal axis and the vertical axis is d, a graph of the waveform is formed. Petal shape l when the average height of the petals is h
/ H. 2. Number of petals. However, when the number of petals is 7 or more, the number is set to 7. 3. When the area of the flower is S and the center of gravity is G (g _i , g _j )

(Equation 5) The moment M of the flower, defined by 4. When the perimeter of the flower is L

R = 4πS / L² The circularity R of the flower, defined in. 5. Half the HS color space when the hue is H and the saturation is S
Set a circle with a diameter of 1 and set the center of the circle at the origin of the xy plane.
Corresponds to the color that has the largest distribution of the flowers in the space
X-coordinate value of the color to perform. 6. In the space of the 5 above, to the color that is the largest distribution of the flowers
The y-coordinate value of the corresponding color. 7. In the space of 5, the space is further defined as −1 ≦ x ≦ 1
And -1≤y≤1
A color is a color set of one unit, and the color that is distributed the most in the flower
The distribution ratio of the colors contained in the color set containing. 8. In the space of 5, the flower is secondarily distributed in the flower
The x coordinate value of the color corresponding to the color. 9. In the space of 5, the flower is secondarily distributed in the flower
The y-coordinate value of the color corresponding to the color. Ten. In the 7 color set, the flower is secondarily distributed
The distribution ratio of the colors included in the color set that includes the colors. 11． In the leaf, near the outside formed by connecting the vertices of the sawtooth
Area S of similar shape_EInside approximation made by connecting the sawtooth valley
Shape area S_I, The leaf shape S_I/ S _E. 12． Leaf aspect ratio. 13. The area of the leaf is S, and the center of gravity is G (g_i, G_j)
Sometimes

(Equation 7) The leaf moment M, defined by 14. When the perimeter of the leaf is L

Equation 8] defined R = 4πS / ^{L 2,} the circularity of the leaf R. 15． When the center of gravity of the leaf is G, and the center of the central vein having a length 1 connecting the base and the tip is C, the distance from the center C to the center of gravity G when the tip direction of the leaf is a positive direction is b. (1 + 2b) / l 16． The opening angle of the base of the leaf. 17． The opening angle of the tip of the leaf. 18． 0 if the leaf is single leaf, 1 if double leaf. 19. An x-coordinate value of a color corresponding to a color that is maximally distributed on the leaves in the space of 5. 20. The y-coordinate value of the color corresponding to the color most distributed on the leaf in the space of 5. twenty one. The distribution ratio of the colors included in the color set including the color that is most distributed in the leaves in the color set of 7 above.

In the analysis for determining the type of the plant,
The feature amount can be normalized using a piecewise linear discriminant function. The feature quantity used for the analysis is:
Since a predetermined number of feature amounts that are a combination having a good recognition rate are used, a sufficient recognition rate can be obtained with 7, 8 feature amounts required for recognition. The recognition result may be displayed, and the type of the plant and its information may be displayed based on the recognition result. Note that a storage medium storing a program that can implement the above-described functions in a computer system is also an aspect of the present invention.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a plant recognition system according to the present invention will be described with reference to the drawings. The present invention provides a system for inputting a captured image using a digital camera or the like and displaying a name of a wild grass. However, the wildflowers handled by this system are flowering plants that humans can clearly recognize flowers and leaves, and it is assumed that images are taken when the flowers are blooming. Flowers and leaves are collected from a certain wildflower, and each is located near the center on a black cloth (or drawing paper) and photographed. Collect flowers and leaves that are not defective due to withering or insect eating. Further, the shape of the leaf margin may vary depending on the growth process and the sampling position. Here, a leaf of a typical shape is used for each wildflower. Leaves have a direction, so the base is on the left side of the image and the tip is on the right side. Furthermore, the shape of a flower often has a three-dimensional structure. The feature quantity defined in the present invention uses only a two-dimensional structure. In order to prevent the photographing direction from being different depending on the user, the photographing is performed from directly above or obliquely upward. The images used in this system are color images of red (R), green (G), and blue (B).
For example, each color of RGB is represented by 8 bits, and the image size is 480 pixels vertically and 640 pixels horizontally.

FIG. 1 shows the system of the present invention and the flow of its recognition processing. As shown in FIG. 1A, the system includes an image input device 10 such as a digital camera and a photo reader, an input device 30 for operating the system, and a display device for displaying an operation screen and a recognition result, as shown in FIG. 40, a storage device 50 in which data used for recognition and plant data are recorded is connected. Also, the flow of the recognition process is as shown in FIG.
First, an image of a flower and a leaf to be recognized is input to the input device 10.
Is input (S100), and the CPU 20 performs a background separation process of cutting out flowers and leaves from the input image (S1).
10). Further, the CPU 20 obtains a defined feature amount based on the extracted image of the object (S120), and recognizes the object (S130). The operation of extracting an object from an image is extremely important, and the system of the present invention uses a clustering method. The feature amount is defined for each flower and leaf. Then, recognition using a piecewise linear discriminant function is performed using the feature amount as input data.

[Background Separation] <Character and Policy of Input Image> FIG. 2 shows an image obtained by photographing the natural state of flowers and leaves as they are. Since most flowers and leaves are clustered, it is difficult to automatically extract only the target object from the image shown in FIG. Therefore, in order to extract an object, a method of setting the same imaging environment such as a light source and taking an image, or using two images of an image where the object does not exist and an image where the object exists, and subtracting the two-sided image to identify the object. An extraction method is exemplified. However, in the present invention, it is necessary to consider an inexpensive system and a reduction in operation for the user. Therefore, in the system of the present invention,
Black cloth (or drawing paper) was used for the background of the object. This is because a flower as an object is distributed in a vivid color or an achromatic color (white), leaves are generally green, and reflection of light such as sunlight at the time of photographing is minimized. FIG. 3 is an image photographed by placing the black cloth on the background of the object photographed by the above-described method. Since many of the features defined by the system of the present invention require an accurate contour of an object, accurate background separation is indispensable. Therefore, background separation was attempted on the input image by various methods such as edge detection and threshold value selection, but the background separation was not always easy due to the factors described below. The object (especially a flower) has a three-dimensional structure, and FIG.
And the difference from the background is reduced. As shown in FIG. 3B, the shadow of the object appears in the background area,
The background is not uniform. Light and dark may occur at the four corners on the background area. Therefore, background separation is performed using the clustering method described below.

<Clustering> In the present study, cluscling is performed in a color space. Clustering in color space has been studied for the purpose of limited color selection of still color images (Paul S. Heckbert, "Color image quan
tization for frame buffer display ", pp.297-307, SI
GGRAPH'82, Boston, Jul. 1982). Here, first, a three-dimensional color distribution space of red (R), green (G), and blue (B) is considered, and a seed (initial cluster value) is derived in this space, and then clustering is performed. (1) Initial Cluster Determination In the system of the present invention, the number of clusters is finally set to two, the background and the object. Clustering is in RGB space
The initial cluster is derived by the following procedure using the k-means method (k-means method). In obtaining the distribution seed in the RGB space, consider the RGB space distribution. FIG. 4 is a diagram showing an RGB space. As shown in FIG. 4, first, in order to disregard elements considered as noise components at the minimum and maximum ends of the distribution on each coordinate axis, the distribution of 0.1% (about 300 pixels) of the total number of pixels from both ends is ignored. Next, in the distribution, excluding both ends, consider n ³ pieces of small space equally divided into n along each coordinate axis. Seed is detected from this distribution space. However, in the embodiment of the present invention, n = 5
And Seed detection The seed detection is performed according to the following procedure. First, the small space with the highest frequency is selected. And the connected small spaces are both seeded
In order to avoid this, six small spaces adjacent to the selected small space as shown in FIG. 5 are not selected as seeds. After removing the selected small space and its surrounding six small spaces, the process of selecting the most frequent small space again is repeated. Let the median value in the selected small space be seed. The initial cluster number (seed number) k is all small spaces that satisfy the condition.

(2) k-means method (k-means method) The algorithm of this method is as follows. First, the center of gravity seed of k temporary clusters is determined. In this study, the seed is obtained by the method described in the previous section. Classify all data into clusters at the shortest distance. The Euclidean distance in the RGB space is used as the distance. The center of gravity of the cluster is newly obtained from the reclassified cluster. If the new cluster centers are all the same as before in the processing of, the processing ends, otherwise the processing returns.

FIG. 6 is a diagram showing the clustering result of the flower image of FIG. 3A in the RGB spatial distribution.
Is the result of expanding the clustering result on the image.
Even if they are classified into the same cluster in the color space as shown in FIG. 6, they are not always distributed in the same region on the image. Therefore, all unjoined areas in the developed image are independent clusters. The number of clusters in the color space of FIG. 6 is 6. However, in FIG. 7, the number of clusters increases to 475. In this figure, a cluster (small area) where the number of pixels is small at a place where color transition is considered in the input image
There are many. Then, these small areas are removed for the purpose of reducing the number of clusters.

<Cluster Integration> Cluster integration includes primary cluster integration for the purpose of removing small area clusters, secondary cluster integration for integrating clusters belonging to background areas, and tertiary cluster for removing shadows according to image conditions. It consists of three steps of integration. Finally, the object is extracted by removing the background area. (1) by examining the images belonging to the primary cluster integration each cluster, the object of integration threshold T ₁ following cluster where that number. Care should be taken that this value does not cause the contour of the object to be lost by the integration process.
In the integration, the average color of each cluster is obtained, and the cluster is absorbed by the cluster having the smallest color difference as compared with the surrounding clusters. The color difference is the Euclidean distance in the RGB space. FIG. 8 shows the result after the primary cluster integration in which the above-described processing is performed with T ₁ = 300 (0.1% of the total number of pixels). (2) Secondary cluster integration In the image of flowers and leaves used in the present system, the object is always placed near the center of the image, and the object does not protrude from the image. Under this condition, the cluster including the outer edge of the image always belongs to the background. Therefore by utilizing this property, belongs results of the primary cluster integration background A _b, the object A _o
Belong either, or fractionated to undecided A _t to. In order to classify the entire cluster A _b, A _o, any of A _t, the system of the present invention to define the edge intensity e. Then, the distribution of e at the cluster boundary is determined, and the distribution is tested to perform cluster integration processing. That is, at the present stage, the object is to integrate a background region composed of a number of clusters into one cluster. FIG. 9 shows the edge strength e.
FIG. 10 is a diagram for describing the derivation of, and is a diagram showing pixels by cluster. This will be described with reference to FIG. Detecting a boundary of the cluster C _l and C _2, numbered in the corner of the pixel on the boundary line. A 4 × 4 small area (referred to as a local area) is provided around the corner. In the p-th local region, the RGB average value (r ₁ , r ₁ , r ₂₎ in the input image of the pixels belonging to clusters C ₁ and C ₂
g ₁ , b ₁ ) and (r ₂ , g ₂ , b ₂ ) are calculated. Edge strength is the difference between the above average values

(Equation 9)｛E_p, P = 1,..., N}_1,2, Minutes
Scattering σ² _1,2Is calculated. Obtained by the above procedure
Cluster integration is performed using the distribution of the edge strength e. background
In the cluster boundary existing in the region, the brightness value on the input image
Changes slowly. Conversely, the background area and the object area
The luminance value changes suddenly at the star boundary. First, include the image edges.
A is all clusters_bBelonging to A_bClass belonging to
The distribution of the edge strength between the data is represented by a normal distribution N (μ_b, Σ_b)When
Assume. Then A_bCluster C belonging to_bAnd its neighbors
Raster C_tThe distribution of edge strength between clusters of the target distribution N
(Μ _t, Σ_t). Next, N (μ_b, Σ_b) And N
(Μ_t, Σ_tThe normal distribution test is performed in ()). The hypothesis is μ
_b= Μ_tage,

(Equation 10) Is calculated. z o _<α if the target distribution is to be the same as the background distribution, to belong to C _t to A _b. If _{α ≦ z o <β, C} t is the target of tertiary cluster integration to belong to _{A t.} Otherwise to belong to _{C t} to _{A o.} FIG.
9 is an image showing a result of applying the secondary cluster integration processing in FIG. 8. Thus, it can be seen that the object and the background are accurately separated. In FIG. 7, the number of clusters is four.
8, the number of clusters is 24 in FIG. 8, and 22 in FIG.

(3) Tertiary cluster integration The target and the background are almost accurately separated by the pre-integration processing.
You. However, in some images, the background area
Some have shadows. Clearly appearing shadows are paired with the background
Located in the middle of the elephant and can be processed by secondary cluster integration
May not be. Therefore, this process detects the shadow area
Then, a process of integrating with the background is performed. The process is as follows
Do with. A obtained by secondary cluster integration_tClass belonging to
TA C_iAverage luminance value of Y _iAnd A_bThe average luminance value of all clusters belonging to_bAnd Because shadows are darker than backgrounds,_i<Y_bIf
If the cluster of interest is a shadow, A_bBelong to. That's right
If it is A_oBelong to. The above processing is performed for all C_i(C_i∈A_tRow for
A, cluster A_bOr A_oClassify into. FIG.
Is 1 with respect to the image having the actual shadow (FIG. 11A).
Processes and results of cluster integration processing of the first to third order
Are shown in the order of FIGS. 11B to 11D.
is there. The shadow of the object is generated in the background area as shown in FIG.
The shadow area is detected even in the image in which
Can be integrated into the background.

(4) Background Area Removal The image is roughly represented by clusters by the processing described above. Finally to achieve an object extracted by removing the background cluster A _b a. FIG. 12 shows the state shown in FIG.
12B is an image showing a result of performing a background separation process from the image of the flower and the leaf of FIG. Fig. 3 (a) flower and Fig. 11
FIGS. 12A and 12B show images obtained by extracting a target object from the leaf image of FIG.

[Features] <Features of Flower> The structure of a flower is various and complicated. It is also difficult to recognize complex structures such as stamens and pistils from an input image. Therefore, in this study, mainly four features F1 to F4 based on the contour shape information, and features F5 to F4 based on the color information
The feature amounts of 10 flowers in total of 10 flowers are defined. (1) Shape information (flower) FIG. 13 is a diagram showing an analysis of a flower shape. Petals are an example of information that occupies the largest area in a flower structure and is reliably obtained from an input image. In this system,
The distance d from the center of gravity G to the contour is determined based on the contour information of the flower as shown in FIG. 3A, and the horizontal axis as shown in FIG. Convert to dimensional waveform.
The following two parameters are defined from this waveform. F1) Petal shape The petal width is defined as the average valley distance 1 in the waveform, the petal length is defined as the average peak height h, and the petal shape is defined as l / h. F2) Number of Petals The number of local maximum values is obtained with the waveform shown in FIG. For a flower with a large number of petals, it is difficult to accurately determine the number. Number of petals N
If N> 7, the numerical value N = 7 that explicitly indicates “the number of petals is large”. F3) Moment F3 uses the moment M defined by the following equation.

[Equation 11] Here, in the equation, S is an area, and G (g _i , g _j ) is the center of gravity. F4) Circularity As information indicating how close the outer shape of the petal is to a circle,

Equation 12] Request circularity than R = 4πS / ^{L 2.} Here, L indicates the perimeter of the flower, and 0
<R ≦ 1.

(2) Color information (flower) The color of a flower is often achromatic or achromatic, such as white.
Flowers can be classified into flowers mainly having one color distribution and flowers having two or more color distributions. In this study, two large colors and their area ratios are defined as six features. The input image information is an RGB color space value represented by three primary colors of light, red, green, and blue, but is converted into an HSV color space represented by three elements of hue, saturation, and brightness to obtain color information. FIG. 14 is a diagram showing a divided HS space. Then, six feature quantities are obtained from the HS space shown in FIG. Here, the distribution ratio is the sum of the number of distributions of the coordinate of interest and the distribution of the four neighboring coordinates. F5) x-coordinate value of the first color x-coordinate of the maximum number of distributions F6) y-coordinate value of the first color y-coordinate of the maximum number of distributions F7) 1st color ratio Distribution ratio including its coordinates and 4 neighboring coordinates F8) 2nd color x-coordinate x-coordinate value of second distribution number F9) y-coordinate value of second color y-coordinate value of second distribution number F10) second color ratio Distribution ratio including its coordinates and 4 neighboring coordinates However, in this study, HSV As a result of finding and recognizing the feature quantity in the HLS color space, which is expressed by the color space and the three elements of hue, luminance, and saturation, almost no difference was found. for that reason,
Here, the HSV color space was used.

<Features of Leaf> The structure of leaves is simpler than that of flowers, and the directions are clear. In this study, we first detect the leaflets to determine the feature values, and then examine the positions of the base and tip. From the shape information, eight feature amounts L1 to L8,
From the color information, three feature amounts L9 to L11, that is, a total of 11 feature amounts are defined. (1) Detection of leaflets FIG. 15 is an image showing a single leaf and a double leaf. As shown in FIG. 15, leaf images are roughly classified into two types: single leaves and double leaves composed of a plurality of small leaves. In the feature value, a single leaf and a double leaf are distinguished by the leaf structure (L8). However, L8 and the feature amount excluding the color information define a feature in a single leaf. Therefore, it is necessary to first classify single leaves and double leaves, and to detect small leaves in the case of double leaves. In the detection of the leaflets, the distance between the center of gravity and the contour point is obtained in the same manner as the procedure for obtaining the shape information of the flower, and the two-dimensional image is converted into a one-dimensional waveform. The minimum number of points is determined from this waveform to determine whether there are small leaves. (2) Determination of base / tip position The direction of the leaves is specified by the shooting conditions, such as the base is left and the tip is right, as described above. However, there are various types of base / tip shapes. In the system of the present invention, the shapes of the base and the tip are classified into a convex shape and a concave shape, and the positions are determined by obtaining pole points.

(3) Shape Information (Leaf) Since most leaves are green, it is considered that shape information is more effective than color information for classification. Therefore, in the present system, eight feature values such as an aspect ratio and a moment are defined as a leaf shape. L1) Shape of Leaf Edge FIG. 16 is a diagram showing a leaf having saw teeth. On the leaves,
As shown in FIG. 16, there are various shapes such as those with and without saw teeth, and differences in size, but it is difficult to analyze and distinguish these. Therefore, in this system, the ratio of the sawtooth is defined, and this value is used as a feature amount. The ratio of the sawtooth generates an outer approximate shape and an inner approximate shape as shown in FIG. 16 and gives them as the ratio (S _I / S _E ) of the areas S _E and S _I. L2) Aspect ratio The aspect ratio of a leaf is defined. The moment and circularity are obtained in the same manner as in the case of a flower. L3) Moment Similar to F3, the moment M defined in Expression [Equation 11]
Is used. L4) Similar to F4, R defined in Expression [12] is obtained. L5) Bias of the Center of Gravity FIG. 17 is a diagram showing an example of leaf patterns. The leaf centers of gravity are roughly classified into three patterns as shown in FIG. Let G be the center of gravity of the leaf, C be the center of the central vein connecting the base and the tip,
The bias of the center of gravity

L5 = (l + 2b) / l (where -1
/ 2 ≦ L5 ≦ １ ／).

The base and tip of the leaf are useful information indicating the characteristics of the leaf as well as the shape of the edge. In this system, the angles θ ₁ and θ ₂ are obtained as the shapes of the base and the tip, respectively.
These are defined as features. FIG. 18 shows the definition of the angle between the base and the tip of the leaf. As shown in FIG. 18, the angle is determined at a point d away from each of the base A and the tip B to both sides (base: A ₁ , A ₂ , tip: B ₁ , B ₂ ). Then, L6 and L7 are obtained. L6) as leaves handled at the base angle _{θ 1 = ∠A 1 AA 2 L7} ) tip angle _{θ 2 = ∠B 1 BB 2 L8} ) leaf structure the system shown in FIG. 15, is classified into a monoplane and biplane You. Therefore, in the former case, L8
= 0 and L8 = 1 in the latter case. The classification method is as described above in “(1) Detection of leaflets”.

(4) Color Information (Leaf) The color of a leaf is generally one green color distribution. Therefore, the color information defines only the maximum distribution color unlike the case of the flower. The definition is the same as for flowers. L9) x-coordinate value of the first color x-coordinate of the maximum number of distributions L10) y-coordinate value of the first color y-coordinate of the maximum number of distributions L11) first color ratio distribution ratio including the coordinates and four neighboring coordinates

<Recognition method> At present, many recognition methods have been proposed. In the system of the present invention, the feature amount obtained by the above definition is normalized. Then, recognition is performed using a piecewise linear discriminant function. For details, refer to the following examples.

<Example> Next, as an example, an experiment of plant recognition was performed using the system of the present invention. For the experiment, 20 sets of various input images were prepared. Recognition is performed by leaving Leave-
One-out method was used. That is, the number of prototypes of the piecewise linear discriminant function is 19, and the number of experimental data is 1. [Background Separation] The inventors of the present invention photographed wild grasses living near the university campus, and performed background separation processing using 20 sets of 34 kinds of flower images and leaf images. The parameters set in the experiment are shown in [Table 1], and the processing results are shown in [Table 2]. The extraction rate was 98.53% for the flower image and 99.12% for the leaf image.

[Table 1]

[Table 2] As described above, the clustering method was adopted as a method of cutting out an object from a captured image, and the effectiveness of the method was verified. Generally, in clustering processing in a color space, it is necessary to perform cluster integration as post-processing. Therefore, by performing a three-stage integration process,
Accurate object extraction has become feasible.

[Recognition] Next, each feature amount was calculated based on the object surface image extracted by the background separation processing. Then, recognition was performed using these as inputs. In the present embodiment, the recognition rate was obtained by using three sets of feature amounts, ie, all feature amounts, flower-only feature amounts, and leaf-only feature amounts. The results are shown in [Table 3].

[Table 3] Table 3 shows recognition rates when only the first candidate is considered in the recognition result and when the second candidate and the third candidate are considered. 96.03 when all feature values are used
% Recognition rate.

The present system does not consider the size information of the object. However, as can be seen from Table 3, a sufficient recognition rate is obtained without the size information. This is considered because the number of types of wildflowers used in the present embodiment is small, and even if the size is unknown, only the color and shape information are sufficient. As a result of conducting a preliminary experiment including manually measured size information, 96.91% (that is, 0.88%
Improvement).

Next, an experiment for obtaining an effective feature amount was performed using the present system. First, one of the 21 feature values is selected for recognition. That is, 21 types of recognition are performed with the feature amount 1. A feature amount having the highest recognition rate is obtained from the result, and this is set as the most effective feature amount. Next, this feature and the remaining 2
Recognition is performed using 0 combinations (20 combinations). Similarly, a feature amount having the highest recognition rate is obtained from the recognition result, and this is set as a second effective feature amount. Thereafter, an effective feature amount is determined in the same manner. [Table 4] shows the determined effective feature amounts.

[Table 4] FIG. 19 is a graph showing changes in the number of feature values and the recognition rate. As can be seen from the graph of FIG. 19, from this result, a sufficient recognition rate is obtained with 7, 8 feature quantities. Recognition experiments using 20 sets of 16 wild grass species (Takeshi Sa
itoh, Toyohisa Kaneko, "Automatic Recognition of W
ild Flowers ", ICPR2000, vol.2, pp.507-510, Barcelo
na, Spain, Sep. 2000), it was found that out of the eight effective feature quantities, the proportion of flower features increased. This is because flower information is more effective than leaf information.

Next, erroneous recognition was examined. As a result, those who tend to misrecognize tend to be wild grasses of the same family. Even if the leaf shapes are different, erroneous recognition is likely to occur if the flowers have similar shapes or colors. This can be inferred from the recognition result described above, but is because the information on flowers is more effective than the information on leaves.

<Other Embodiments> In the above-described embodiment, wild grass from spring to early summer is used. Therefore, a sufficient recognition rate was obtained without inputting seasonal information, sampling location, etc., as feature quantities, but in addition to the feature quantities measured from the image, the season,
A higher recognition rate can be obtained by adding information such as a location and time as a feature amount. Further, a system that obtains one correct wildflower name based on the recognition result may be used, or a system that displays several similar wildflower names on a screen and finally performs recognition by human eyes may be used. Based on the recognition result, it is possible to display the kind of wild grass and information such as those described in a botanical dictionary. The system according to the present invention can be realized by reading and executing the program from a storage medium storing the program according to the present invention by the system.
This recording medium includes DVD, CD, MD, MO, floppy (registered trademark) disk, magnetic tape, ROM cassette and the like.

[0029]

According to the system of the present invention, wildflowers are picked up as image recognition for natural objects,
Image recognition was automatically performed using 21 feature amounts effective for discriminating the type of wildflowers measurable from the leaf image, and the flowers could be identified. Also, by using this system, a sufficient recognition rate could be obtained with 7, 8 feature amounts required for recognition.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system of the present invention and the flow of its recognition processing.

FIG. 2 is a diagram showing an image obtained by photographing the natural state of flowers and leaves as they are.

FIG. 3 is a diagram showing an image photographed with a black cloth placed on a background of an object.

FIG. 4 is a diagram showing an RGB space.

FIG. 5 is a diagram showing a small space having a maximum frequency and six small spaces adjacent thereto;

FIG. 6 shows a clustering result for a flower image in RGB.
It is the figure shown in the spatial distribution.

FIG. 7 is a diagram showing an image obtained by developing a clustering result.

FIG. 8 is a diagram showing an image obtained as a result of applying the primary cluster integration processing.

FIG. 9 is a diagram for explaining derivation of an edge strength e.

FIG. 10 is a diagram showing an image as a result of applying the secondary cluster integration processing.

FIG. 11 is a diagram illustrating a process and a result of performing first to third-order cluster integration processing on an image having a shadow.

FIG. 12 is a diagram showing an image obtained by performing a background separation process from an image of a flower and a leaf.

FIG. 13 is a diagram showing an analysis of a flower shape.

FIG. 14 is a diagram showing a divided HS space.

FIG. 15 is a diagram showing single leaf and double leaf images.

FIG. 16 shows a leaf having a saw tooth.

FIG. 17 is a diagram showing an example of a leaf pattern.

FIG. 18 is a diagram showing a definition of an angle between a base and a tip of a leaf.

FIG. 19 is a graph showing a change in the number of features and the recognition rate.

[Explanation of symbols]

 Reference Signs List 10 image input device 20 CPU 30 input device 40 display device 50 storage device

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 7/60 150 G06T 7/60 150S 150A

Claims (7)

[Claims] 1. A plant recognition system for discriminating a kind of a plant based on characteristics of flowers and leaves, wherein an image of a flower and / or a leaf as an object is extracted from a digital image of the flowers and / or leaves. Then, one or more feature amounts are obtained from the extracted image of the flower and / or leaf, and the feature amount of the plant is determined from the obtained feature amount and the feature amounts of various plants registered in advance in a database. A plant recognition system that distinguishes types. 2. The plant recognition system according to claim 1, wherein a clustering method performed in a color space is used to extract the flower and / or leaf image. 3. The plant recognition cis according to claim 1 or 2.
In the system, the feature amount is obtained from an image of the extracted flower or leaf.
This is the required value and can be selected from the following
A plant recognition system, characterized in that: 1. In the flower image, the distance from the center of gravity G to the petal outline
The wave where the separation is d, the angle θ from the center of gravity is the horizontal axis, and the vertical axis is d
Average valley distance in the waveform when plotted in a shape graph
Petal shape l when distance l and average of mountain height are h
/ H. 2. Number of petals. However, when the number of petals is 7 or more, the number is set to 7. 3. The area of the flower is S and the center of gravity is G (g_i, G_j)
[Equation 1]The moment M of the flower, defined by 4. When the circumference of the flower is L: R = 4πS / L² The circularity R of the flower, defined in. 5. Half the HS color space when the hue is H and the saturation is S
Set a circle with a diameter of 1 and set the center of the circle at the origin of the xy plane.
Corresponds to the color that has the largest distribution of the flowers in the space
X-coordinate value of the color to perform. 6. In the space of the 5 above, to the color that is the largest distribution of the flowers
The y-coordinate value of the corresponding color. 7. In the space of 5, the space is further defined as −1 ≦ x ≦ 1
And -1≤y≤1
A color is a color set of one unit, and the color that is distributed the most in the flower
The distribution ratio of the colors contained in the color set containing. 8. Within the space of 5 above, it is the second distribution of the flowers
The x coordinate value of the color corresponding to the color. 9. In the space of 5, the flower is secondarily distributed in the flower
The y-coordinate value of the color corresponding to the color. Ten. In the 7 color set, the flower is secondarily distributed
The distribution ratio of the colors included in the color set that includes the colors. 11． In the leaf, near the outside formed by connecting the vertices of the sawtooth
Area S of similar shape_EInside approximation made by connecting the sawtooth valley
Shape area S_I, The leaf shape S_I/ S _E. 12． Leaf aspect ratio. 13. The area of the leaf is S, and the center of gravity is G (g_i, G_j)
[Equation 3]The leaf moment M, defined by 14. Perimeter of the leaf
Let L = 4πS / L² Leaf circularity R, defined in 15． G is the center of gravity of the leaf, and the center of length l connecting the base and the tip
When the center of the pulse is C, the tip direction of the leaf is positive.
When the distance from the center C to the center of gravity G is b
Mushroom, deviation of the center of gravity of the leaves (l + 2b) / l. 16． The opening angle of the base of the leaf. 17． The opening angle of the tip of the leaf. 18． 0 if the leaf is single leaf, 1 if double leaf. 19. In the space of 5 above, to the color that is maximally distributed on the leaves
The x-coordinate value of the color. 20. In the space of 5 above, to the color that is maximally distributed on the leaves
The y-coordinate value of the corresponding color. twenty one. In the color set of 7 above, the color that is maximally distributed on the leaves is
The distribution ratio of the colors included in the included color set. 4. The plant recognition system according to any one of claims 1 to 3, wherein the feature amount is normalized for analysis for discriminating the type of the plant, and a piecewise linear discriminant function is used. A plant recognition system characterized by performing. 5. The plant recognition system according to claim 4, wherein the feature amount used for the analysis uses a predetermined number of feature amounts, which is a combination having a good recognition rate. . 6. The plant recognition system according to claim 1, wherein a recognition result is displayed, and the type of said plant and its information are displayed based on said recognition result. Plant recognition system. 7. A storage medium storing a program capable of implementing the function according to claim 1 in a computer system.