JP2007200020A - Object recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object recognition system for quickly performing retrieval. <P>SOLUTION: The distribution information of features is generated for every featured value from a refined database 50 by a distribution data formation part 52. A distribution value Ve is obtained for each featured value, and a data retrieval engine 40 retrieves the data in the descending order of the distribution value Ve. The retrieval engine 40 compares the featured values of the plant group of the refined database 50 and that of the object to identify the object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an object recognition system.

  A conventional object recognition system described in the following published patent publication is known.

Extracting the image of the target flower and leaf from the digital image of the flower and leaf using the clustering method, obtaining one or more feature quantities from the extracted flower and leaf images, and obtaining the obtained feature quantity Then, the feature quantity of various plants registered in the database in advance is analyzed using a statistical method to determine the type of wild grass.
Japanese Patent Laid-Open No. 2002-203242

  In the above-described conventional object recognition system, the probability of having a predetermined parameter is obtained from an image for every feature amount registered in the database, and the type of wild grass is determined. For this reason, a lot of calculation time is required until a discrimination result is obtained.

  This invention is made in view of such a situation, and is providing the target object recognition system which can search quickly.

  In order to solve the above-mentioned problem, an invention according to claim 1 is characterized in that an imaging unit that captures an image of an object, a feature amount extraction unit that extracts a feature amount of the object from an image captured by the imaging unit, A database in which feature quantity information corresponding to at least the feature quantity is registered for a plurality of species included in the category to which the object belongs, and database narrowing means for extracting a data group from the feature quantity information registered in the database; , A narrowing condition setting means for setting conditions for extraction by the database narrowing means, a distribution data forming means for forming feature quantity distribution data for each of the data groups extracted by the database narrowing means, and the distribution data forming Extracted by the feature quantity extraction means based on the feature quantity distribution data formed by the means. Feature quantity information having the same feature quantity as the feature quantity from the feature quantity information in the data group by comparing the feature quantity with the feature quantity information in the data group extracted by the database narrowing means And a search means for searching.

  According to a second aspect of the present invention, in the object recognition system according to the first aspect, the distribution data forming means distributes the feature quantity distribution data most evenly distributed among the feature quantity distribution data formed for each feature quantity information. The feature amounts are ranked in order, and the search means searches in the order ranked by the distribution data forming means.

  According to a third aspect of the present invention, in the object recognition system according to the first aspect, the distribution data forming means has a feature quantity having a parameter that appears most frequently among the feature quantity distribution data formed for each of the feature quantity information. The feature amounts are ranked in ascending order of the number of distribution data, and the search means searches in the order ranked by the distribution data forming means.

  According to a fourth aspect of the present invention, there is provided at least an imaging unit that captures an image of an object, an image display unit that displays an image captured by the imaging unit, and a plurality of species included in a category to which the object belongs. A database in which feature amount information corresponding to the feature amount of the object is registered; distribution data forming means for forming feature amount distribution data for the feature amount information registered in the database; and for each feature amount Display means for displaying the feature quantity distribution data, selection means for selecting a part of the image displayed on the display means based on the feature quantity distribution data, and one of the images selected by the selection means. Feature quantity extraction means for extracting the feature quantity of the object based on the information of the part, the feature quantity extracted by the feature quantity extraction means and the database registered in the database By comparing the symptoms amount information, characterized by comprising a search unit for searching the feature amount information having the same characteristic quantity and the characteristic quantity from the feature amount information in the data group.

  According to a fifth aspect of the present invention, in the object recognition system according to the fourth aspect, a part of the image is at least one part.

  A sixth aspect of the present invention is the object recognition system according to the fourth aspect, wherein a part of the image is at least one region surrounded by a closed curve selected by the selection means.

  A seventh aspect of the present invention is the object recognition system according to the fifth or sixth aspect, wherein the feature amount is color information extracted from a selected part or region in the image.

  The invention according to claim 8 is the object recognition system according to claim 5 or 6, wherein the feature amount is distance information between at least two selected parts or regions.

  According to a ninth aspect of the present invention, in the object recognition system according to the sixth aspect, the feature amount is at least one of a minimum diameter, a maximum diameter, an average diameter, and an area of at least one selected region in the image. It is characterized by being one.

  According to a tenth aspect of the present invention, in the object recognition system according to the sixth aspect, the feature amount is a shape of at least one selected region in the image.

  The invention according to claim 11 is the object recognition system according to any one of claims 4 to 6, wherein the feature amount is distribution information of a plurality of selected parts or / and regions. .

  According to a twelfth aspect of the present invention, in the object recognition system according to the fourth aspect, in the image display means, the outline of the part or area selected by the selection means, or the color or brightness of the non-selected part or area. It is characterized by the change.

  According to this invention, it is possible to search quickly.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram of an object recognition system according to the first embodiment of the present invention, FIG. 2 (a) is a diagram showing an example of regional vegetation data and an original database, and FIG. 2 (b) is a regional plant group. It is a figure which shows an example of the procedure which produces the narrowing-down database 50 from the information 71 and the original database 60. FIG.

  This object recognition system includes an imaging device (imaging unit) 10, an image display unit (image display unit) 20, a feature amount extraction unit (feature amount extraction unit) 30, a search engine (search unit) 40, a narrowing database 50, and an original. A database 60, a data extraction unit (database narrowing means) 70, and a GPS module (squeezing condition setting means) 80 are provided.

  The imaging device 10 includes a photographic lens (imaging optical system) 11, an imaging unit 12, and an image processing circuit 13.

  The taking lens 11 forms an optical image of an object such as a flower on the light receiving surface of the imaging unit 12. In FIG. 1, the photographic lens 11 is depicted as a single lens, but is actually composed of a plurality of lenses.

  As the imaging unit 12, a CCD (Charge Coupled Device) can be used. The CCD includes a plurality of light receiving elements, and outputs the charges accumulated in each light receiving element by the optical image of the object formed by the photographing lens 11 to the image processing circuit 13 as an analog electric signal.

  The image processing circuit 13 decomposes the analog electrical signal of the object received from the CCD into R, G, and B components. The image processing circuit 13 converts an analog electric signal decomposed into R, G, and B components into a digital signal, and stores digital image data indicating an image of the object in a memory (not shown) to perform various operations. After the processing, the digital signal is converted into an analog signal and analog image data is output to the image display unit 20.

  As the image display unit 20, a CRT display or an LCD panel can be used. Image data captured by the imaging device 10 is output to the image display unit 20, and an image of the target object (target plant image) 21 is provided to the searcher. The image display unit 20 displays data searched by the search engine 40, for example, information including images (search result image 22, search result image 23) and names.

  The feature amount extraction unit 30 extracts feature amounts from the image of the object. The feature amount is, for example, flower color information, the number of petals extracted from the contour of the flower, shape, size, and the like.

  The search engine 40 compares, for example, color information (feature amount) extracted by the feature amount extraction unit 30 with color information (feature amount) registered in the narrowing database 50, and has the same color information as the extracted color information. Search for data.

  Vegetation information for each region extracted by the data extraction unit 70 is input to the narrowing database 50. Based on this information, narrowing data (name, family, feature amount, flowering time (fruiting time), etc.) 51 narrowed down to a group of plants that grow in a predetermined area is registered in the narrowing database 50.

  In the narrowing-down database 50, distribution data (feature amount distribution data) is formed based on data narrowed down to a group of plants that grow in a predetermined area using the distribution data forming unit (distribution data forming means) 52.

  For example, data (name, family, feature amount (color information, shape information, etc.), time information such as flowering time (fruiting time), commentary, sample images) corresponding to wild grasses throughout Japan are registered in the original database 60. (See FIG. 2 (a)). In Japan, about 2000 kinds of plants are growing as typical wild grasses. For example, there are 17 species in the family Auriculariaceae, including 10 species and subspecies of Haksanfuuro, Iyofuuro, Asamafuuro, Tachifuuro, Gunkniuuro, Himefuuro, Ichigefuuro, Mitsubafuuro, Kofuuro (reference: Sankei Handy 2) flower").

  Therefore, it is difficult to identify a plant by comparing the feature amount such as the color and shape of the flower of the original database 60 with the feature amount such as the color and shape of the flower of the object image. However, the flowers are distributed in specific regions and mountain areas, and if limited to a predetermined mountain area, it can be narrowed down to several kinds of flowers at most.

  The data extraction unit 70 inputs the location information acquired by the GPS module 80. The data extraction unit 70 stores a plurality of regional data and wild grass group data growing in each region. The data extraction unit 70 selects only wild grass information (regional plant group information) 71 that grows in the selected region from the original database 60 based on the wild grass group data of the selected region, and creates narrowed data 51. The regional plant group information 71 is vegetation information of a plant group growing in each region a, b, c..., For example, as shown in FIG.

  The narrowing-down database 50 is created from the regional plant group information 71 and the original database 60. A procedure for creating the narrowed-down database 50 will be described by taking as an example a case where the position detected by the GPS module 80 is, for example, the region c. The plant group that grows in the region c is the plant group c (plants 1, 3, 5, 7,...) (See FIG. 2A). The data extraction unit 70 extracts the plants 1, 3, 5, 7,... The operation for obtaining the narrowed-down data is performed using the narrowed-down data when the search work is first performed when the region is not changed.

  The GPS module 80 acquires location information obtained from images based on signals from a plurality of artificial satellites.

  Next, a method for extracting the size of the flower and the height of the plant height from the image will be described.

  FIG. 3 is a diagram for explaining a method of extracting the size of the flower and the height of the plant height from the image.

  The diameter φ of the flower is calculated by the following equation, where the diameter φ ′ of the flower on the image 21, the magnification β, or the focal length f of the lens 11 and the distance d from the lens to the object.

φ = φ '/ β
= Fφ '/ (df) (1 formula)
Here, d can be obtained as the distance from the lens 11 to the object from the amount of extension of the lens 11 in an in-focus state by an autofocus mechanism (AF) mounted on the imaging device 10. The focal length f is a fixed value in the case of a single focus lens, and is stored together with a digital image usually taken as lens data of a digital camera even in a zoom lens.

  The diameter of the flower is measured in mm from the outer shape of the extracted flower image, collated with the flower size information in the database 50, and a search operation is performed along with other feature amounts.

In addition, for flowers and fruits that have a tuft shape, as represented by the Apiaceae and Euphorbiaceae, the size of the single flower and the tuft is input in advance into the feature quantity of the database 50 as size information. Thus, the size information can be used to measure the size of the flower from the whole image of the flower by the macro lens (lens 11) and the other flower (lens 11). Furthermore, the height of the plant height that is the object can also be obtained from the image of the entire plant.

  Next, the distribution data forming unit 52 will be described.

  FIG. 4 is a diagram illustrating an example of distribution data of feature amounts included in the narrowing-down database.

  The distribution data forming unit 52 creates distribution data for each feature amount of the narrowing database 50. For example, distribution data of a predetermined feature amount is formed from the narrow-down database 50 in a predetermined area of the area c. The distribution data forming unit 52 obtains a variance value Ve expressed by the following two formulas from distribution data for each feature quantity.

Ve = Se / (n-1) (2 formulas)
Here, Se is the magnitude of the difference between the data and is expressed by the following three equations.

Se = S _T −S _m (3 formulas)
(However, S _T = y ₁ ² + y ₂ ² + y ₃ ² +... + Y _n ² , Sm = (y ₁ + y ₂ + y ₃ +... + Y _n ) ² / n)
The variance value Ve was obtained for each feature amount shown in FIG. The variance value Ve is the variance value Ve of the feature value “flower shape and distribution” is 17.0, the variance value Ve of “flower color” is 24.1, and the “flower (or inflorescence) size” variance value. Ve was 97.8. The distribution data forming unit 52 searches the data in the order of the smallest variance of the data in the narrowing-down database 50, that is, “flower shape and distribution”, “flower color”, and “flower (or inflorescence) size”. The search engine 40 is instructed to do so.

  First, the search engine 40 performs a search based on “flower shape and distribution”. Data having the same characteristics is extracted by collating the shape and distribution of the flower extracted from the image of the object with the shape and distribution of the flower of the narrow-down database 50, and stored in a data storage unit (not shown). The search engine 40 shown in FIG. 1 performs an object identification operation by comparing feature data of the narrowed-down data of the plant group in the narrowed-down database 50 and the target object.

  Next, the search engine 40 performs a search based on “flower color”. Of the data existing in the data storage unit, data that matches the condition of the color data extracted from the image 21 of the target plant is extracted, and other data is removed from the data storage unit. Thereafter, the same extraction operation is performed for “the size of the flower (or inflorescence)”.

  A sample image of the selected object is displayed on the image display unit 20 such as an LCD panel.

  If there are a plurality of candidates for the searched object, the screen is divided and displayed on the image display unit 20 at one time, for example, as shown in FIG. Sample images can be displayed sequentially. The image display unit 20 also displays a captured image 21 of the object at the same time.

  The searcher selects the same sample image as the target object by comparing the sample image and the target object image 21 or the sample image and the target object image. In the case of a plant that is difficult to identify from an image, a sentence that describes the features that serve as identification points is displayed on the image display unit 20 together with the sample image. Features that serve as identification points are, for example, the presence or absence of stem hair and the color of the back of the leaves.

  When the object is in front of the eyes, the searcher observes the object and performs identification work. The sample image and the plant information accompanying the sample image are recorded in a storage unit (not shown) of the digital camera as information of the photographed image 21 of the object. Therefore, the photographer can organize photographs based on the information on the target plant recorded in the digital camera even after returning home.

  If the number of data is less than the predetermined number during the search operation, the search process is interrupted, and the feature quantity used for extraction and the number of identification candidate data are displayed on the image display unit 20. . If the searcher determines that the number of data is sufficient, the image of the candidate object (sample image) is displayed, and an image identified as the image of the target plant is found from among the images. At this time, if the number of plants that are candidates for the object is large, the screen may be scrolled and displayed. In addition, a candidate plant identified as an object is registered as accompanying data of the object image 21, and when the image 21 of the object is read later, the registered accompanying data is displayed at the same time. Also good.

  According to this embodiment, since the search is performed using the distribution data, the calculation time can be shortened and the search result can be quickly obtained. Further, distribution data for each feature amount of the refined data in the refinement database 50 is formed, a variance value Ve is obtained for each distribution, and search is performed by ranking the feature amounts in ascending order of the variance value Ve. Search can be performed.

  In this embodiment, the distribution data of each feature quantity in the narrowing-down database 50 is formed, and the search ranking is given based on the variance value Ve. For example, the largest number of features (of the distribution data of each feature quantity ( The number of data indicating parameters) may be compared, and the search may be performed in order from the smallest number of data. In this case, since it is not necessary to obtain the variance value Ve, the search result can be obtained more easily.

  In this embodiment, only the regional plant group information 71 is selected from the original database 60 to create the narrowed data 51 and the distribution data of the narrowed data 51 is formed. Alternatively, the narrowed-down data 51 may be created based on the shooting time and position information. If the timing information and the position information are linked, the accuracy of the search can be further improved. At this time, the distribution data of the feature amount to be formed changes depending on the conditions such as time and place.

  Furthermore, in this embodiment, the case where the narrowed-down database 50 according to the region is created from the original database 60 by taking 2000 kinds of plants nationwide as an example has been described. However, if the original database 60 has a limited population, the narrowed-down database 50 50 may not be created, and distribution data for each feature amount may be directly formed from the data of the original database 60 in advance to perform search ranking.

  FIG. 5 is an overall configuration diagram of the object recognition system according to the second embodiment of the present invention, and the same reference numerals are given to portions common to the first embodiment, and description thereof is omitted.

  The object recognition system includes an imaging device 10, an image display unit 20, a database 150, a distribution data creation unit (distribution data creation unit) 152, a touch pen (selection unit) 25, and a selection region image extraction unit (selection region image extraction unit). 90, a feature amount extraction unit (feature amount extraction unit) 30, a search engine (search unit) 40, and an identification candidate display unit (display unit) 45.

  In the database 150, for each of a plurality of species included in a category to which the object belongs (in this embodiment, a category of plants), a plurality of feature amount data (name, family, feature amount, flowering time) that are characteristics of the species. , (Time information such as (fruiting time)) 151 is registered. The distribution data creation unit 152 creates distribution data (for example, the distribution data in FIG. 4 of the first embodiment) for each feature amount based on the feature amount stored in the database 150. The distribution data creation unit 150 calculates a variance value for each feature amount based on the distribution data. In the distribution data of FIG. 4, as described above, the magnitude relationship between the variance values is in the order of “flower shape and distribution” <“flower color” <“flower (or inflorescence) size”. For example, the feature quantity acquired from the image 26 of the target plant is displayed on the image display unit 20 in descending order of the variance value. The searcher determines the feature amount to be acquired first with reference to the display on the image display unit 20.

  The selection area image extraction unit 90 extracts a specific area selected by the touch pen 25 from the display image 26 displayed on the image display unit 20.

  The feature amount extraction unit 30 extracts, for example, color information that is a feature amount for the pixels extracted by the selection region image extraction unit 90.

  The identification candidate display unit 45 displays data searched by the search engine 40, for example, an image, distribution data, and the like.

  Hereinafter, the display and search of the feature value based on the distribution data will be described.

  First, the extraction of the feature amount by selecting the searcher's object display screen will be described by taking an example of extracting color information. In the following description, a point-like or block-like area (for example, an area selected by using the touch pen 25 and including only one to several pixels) is referred to as a part. A portion selected by being surrounded by a closed curve drawn with the touch pen 25 is referred to as a region.

  First, an image of the object is formed on the light receiving surface of the imaging unit 12 by the photographing lens 11, and image data photoelectrically converted by the imaging unit 12 is output to the image processing circuit 13. The image data output from the image processing circuit 13 is displayed as a display image 26 of the image display unit 20.

  Next, a desired part is selected from the image of the object in the display image 26 using the touch pen 25. Since the color of the selected part in the display image 26 changes, the searcher can recognize the selected part.

  If the selected part is not correct, the searcher clears the selected part with a selection release button (not shown). On the other hand, if the selected part is correct, the searcher operates an extraction button (not shown) to give an extraction instruction for extracting the part of the selected region.

  When the feature amount is color information, the color or luminance (brightness) of the part not selected on the display image 26 changes, and only the selected area remains the color of the original image. This is to clarify the color of the part selected by the searcher. Therefore, a frame may be displayed so as to surround the outline of the selected part. In addition, when the number of pixels in the selected part is only 1 to several, an accurate color cannot be displayed. Therefore, a part including at least a pixel adjacent to the pixel in the selected part is selected. .

  By doing in this way, it can be judged whether the selected part is the color which the searcher selected. If this color is not the intended color, the searcher clears the part selected by an extraction cancellation button (not shown) (may be the same as the selection cancellation button), and performs the search operation again. On the other hand, if the color of the selected part is the color intended by the searcher, the search process is continued.

  The search engine 40 compares the extracted color information with the color information registered in the data group of the database 150, and selects data having the same color information as the extracted color information from the data group of the database 150.

  Information including the selected candidate image and name is displayed on the identification candidate display unit 45. If there are many selected candidates, the selected candidates are sequentially displayed in the scroll mode in accordance with the number of candidates, or only the number of selected candidates is displayed, or only the candidate names are displayed. It is displayed. The display method may be selected by the searcher.

  In addition, the searcher selects a part that can extract a feature quantity different from the feature quantity extracted first, and uses the feature quantity that is a combination of the feature quantity extracted first and the newly extracted feature quantity. It is also possible to search for images and names of objects. This search method will be described with reference to FIGS.

  6 and 7 are diagrams for explaining another feature amount extraction method of the object recognition system according to the second embodiment of the present invention.

  The searcher selects color information with the touch pen 25 in the menu screen MP of FIG. The color information becomes active, and the color information is extracted as a feature amount from the selected part.

  In FIG. 6, the object (flower) has a plurality of colors of white and yellow.

  When the searcher selects two parts 21-1 and 21-2 having different colors on the display image 26 using the touch pen 25, color information for each part is selected, and from the database 150 based on this feature amount. Candidate extraction is performed. As described above, since the color information for each part is extracted and the search is performed, more accurate search accuracy can be obtained.

  Here, the color processing of the selected parts 21-1 and 21-2 will be described. When the selected parts 21-1, 21-2 include a plurality of pixels, a plurality of combinations (pixels) of the pixels R, G, B in the part are selected, and the colors in the respective combinations are obtained. The average value of the colors is the color of the part. In the processing, RGB values obtained for each combination in advance may be converted into a uniform color space by HSV conversion, Lab conversion, or the like, and the average of each coordinate value may be taken. In the HSV conversion, H is heu (hue), S is saturation (saturation), and V is value (lightness). In the Lab, L represents a luminance element, a represents an element of hue from green to red, and b represents an element of hue from blue to yellow.

  Alternatively, RGB values may be obtained for each combination, and the average value may be converted into a uniform color space by HSV conversion, Lab conversion, or the like to obtain color coordinates. When the selected parts 21-1 and 21-2 are only a few pixels, the same process is performed with a part including at least neighboring pixels adjacent to the one pixel as a color part.

  In FIG. 6, the parts 21-1 and 21-2 that have been touched with the touch pen 25 are selected. For example, as shown in FIG. 7, the regions surrounded by the closed curve drawn with the touch pen 25 (22-1, 22-2, 22- 3) can also be selected. In this case, the color extracted from the selected region is the average value of the color of each pixel in the selected region 22-1 2-22-2 or 22-3 or the color of the pixel occupying the most region. is there. Alternatively, the color may be detected in units of pixels in which each of the pixels R, G, and B is a set, and the color for each pixel and the number of pixels corresponding to the color may be displayed. Further, the region 22-4 may be selected in addition to the regions 22-1, 22-2, and 22-3. One area may be selected.

  The searcher can select a candidate having the same characteristics as the target object from the candidates extracted in this manner, and display the candidate sample image and its data on the image display unit 20.

  According to this embodiment, the same effects as those of the first embodiment can be obtained. In addition, since the feature amount of the object is extracted by applying the touch pen 25 to the displayed display image 26, it is not necessary to perform a calculation for selection, and the feature amount can be extracted easily and accurately. Furthermore, since a predetermined feature amount is extracted by selecting a predetermined part of the displayed image with the touch pen 25, the feature amount can be extracted well even in an image with a complicated background. Further, by using the touch pen 25, a feature amount arbitrarily extracted by a searcher can be selected. Further, by using the touch pen 25, the contour can be freely extracted according to the shape of the object required by the searcher. Further, by using the touch pen 25, it is possible to freely select a region of a color and a region for obtaining size information, and a feature amount can be easily extracted.

  8, 9, and 10 are diagrams for explaining another feature amount extraction method of the object recognition system according to the third embodiment of the present invention. FIGS. 10A and 10B are diagrams showing only regions selected from the images of FIGS. 9A and 9B.

  This embodiment is different from the second embodiment in that a feature amount is extracted from a positional relationship between selected parts (for example, a distance between two parts).

  First, a method for obtaining the size of an object will be described with reference to FIG. The searcher selects the diameter of the flower on the menu screen MP with the touch pen 25 and activates it. The searcher selects two parts 23-1 and 23-2 for obtaining distance information as a feature amount from the image of the object. The two parts 23-1 and 23-2 are respectively located in the vicinity of the boundary line between the object and the background on the display image 26.

  The actual flower diameter φ is the distance φ ′ between the two parts 23-1 and 23-2 on the display image 26, the magnification β, or the distance φ ′ between the two parts 23-1 and 23-2, When the focal length f of the lens 11 and the distance d from the lens 11 to the object are set, the following four formulas are used.

φ = φ '/ β
= Fφ '/ (df) (4 formulas)
Here, d can be obtained as the distance from the lens 11 to the object by the amount of extension of the lens 11 in the focused state by an automatic focusing mechanism (AF) (not shown) mounted on the imaging apparatus 10.

  Further, the focal length f is a fixed value in the case of a single focus lens, but is stored together with a digital image photographed as lens data of a digital camera, even in the case of a zoom lens.

  A search is performed based on the distance information extracted in this way, for example, plant height information and flower size information. Although the diameter of the head flower is measured in FIG. 8, the extracted feature amount differs depending on the type of the object such as the diameter of the tubular flower part and the plant height. All the feature quantities that can be extracted can be selected on the menu screen MP.

  When the search target is a flower having an inflorescence shape, the searcher activates the distribution information on the menu screen MP, and the center of each flower is touched with the touch pen 25 as shown in FIGS. 9 (a) and 9 (b). By selecting the pixels at positions P1 to P7 and P11 to P17, the shape of the inflorescence can be determined from the distribution state of the flowers. In FIG. 9, an inflorescence outline (indicated by a dotted line) is designated with the touch pen 25, and the inflorescence regions A1 and A2 are selected. By selecting the inflorescence regions A1 and A2 in this way, the size of the entire inflorescence can be measured.

  The minimum diameter φ1 ′ and the maximum diameter φ2 ′ of the regions A1 and A2 extracted by the feature amount extraction unit 30 are measured (see FIGS. 10A and 10B). The actual inflorescence minimum diameter φ1 and maximum diameter φ2 are obtained from the inflorescences (regions A1, A2) φ1 ′ and φ2 ′ on the display image 26 using the above-described (Equation 4). Further, the feature amount extraction unit 30 determines whether the shape of the inflorescence of the target object is linear or planar by obtaining the ratio between the minimum diameter φ1 ′ and the maximum diameter φ2 ′. This is the feature quantity of the inflorescence. From the ratio of the minimum diameter φ1 ′ and the maximum diameter φ2 ′, the flowers are distributed linearly as shown in FIG. 10A, or the flowers are distributed in a plane (substantially circular) as shown in FIG. You can easily see that Further, since the number of pixels included in the selected regions A1 and A2 is measured and the area corresponding to one pixel is known, the areas of the selected regions A1 and A2 can be obtained.

  Further, more detailed flower distribution information can be extracted from the information on the center positions (P1 to P7, P11 to P17) of the selected flowers. The flower distribution information is extracted from the distance φ3 ′ between adjacent flowers obtained by the distance φ3 ′ on the display image 26 from the above (Equation 4) and the center positions (P1 to P7, P11 to P17) of the flowers. The The position of the flower may be the barycentric position for each small inflorescence in a plant having a distributed inflorescence such as some plants of the Apiaceae family. Further, for example, the extraction may be performed so that the outline of each flower is traced instead of the center position of the flower. The regularity of the flower distribution can be extracted from the flower distribution information, and the inflorescence shape can be specified.

  As in this embodiment, when the purpose is to detect the positions of the selected parts, unlike the color detection, even if one to several pixels are included in the selected part, There is no need to select a pixel adjacent to it.

  Feature quantity data corresponding to the various feature quantities thus obtained is registered in the database 150 in advance as a registered feature quantity, and at least one registered feature of the extracted feature quantity and feature quantity data in the database 150 is registered. By comparing the quantity, a highly accurate search is possible.

  According to the third embodiment, the same effect as that of the second embodiment is obtained, and the feature amount is obtained from easily obtained flower size information and distribution information. Therefore, it is not necessary to cut out an object from the obtained image, and a feature amount for identifying the object can be obtained more easily than in the first embodiment.

  11, 12, and 13 are diagrams for explaining another feature amount extraction method of the object recognition system according to the fourth embodiment of the present invention.

  In this embodiment, a region is selected by tracing the vicinity of the contour of the object with the touch pen 25, and the contour is extracted.

  When the image of the object to be searched is displayed on the display image 26, the searcher first presses a region selection mode switching button (not shown) provided on the display image 26 to set the contour extraction mode.

  Next, as shown in FIG. 11, the area 24 surrounded by the dotted line is selected by tracing the outline of the object with the touch pen 25. If the line tracing the outline of the object is a line intended by the searcher, a region selection button (not shown) is pressed. When an area selection instruction is issued by pressing the area selection button, the selected area selection image extraction unit 90 starts area selection based on the traced line position. The area selection is performed by the following method.

  First, at each point (for example, pixel) constituting the outline drawn with the touch pen 25, a discontinuous part closest to each point is selected. Here, the discontinuous part is, for example, a part where the color and contrast change abruptly or a part where the focus changes abruptly. The process of selecting the discontinuous part is performed along a line that traces the outline of the object, whereby the outline of the object can be extracted.

  After the contour extraction, feature quantities such as the size (diameter and area) and shape of the region surrounded by the contour can be automatically extracted. Further, since color information in this region and the ratio of each color (pixel number information, area information, etc.) can be obtained, color information can be automatically extracted. Thus, since the characteristic color, shape, and size in the contour can be automatically extracted, the feature amount of the object can be efficiently extracted.

  Note that if color information is registered by specifying a characteristic color part (for example, 24-1) in the contour before contour extraction, a color tone different from the registered color is obtained during contour extraction. Since it can be recognized as a discontinuous point, the efficiency of contour extraction work is improved (see FIG. 12).

  Further, as shown in FIG. 13, when the discontinuous second region 25-2 exists in the extracted first region 25-1, the extraction process is performed on the second region 25-2. Do it also. By extracting the colors of the two areas 25-1, 25-2 and the parts 25-3, 25-4 in this way, the positional relationship between the two areas 25-1, 25-2 is recognized. The recognition rate can be increased by comparing the recognized feature quantity with the feature quantity of the database 150.

  According to this embodiment, there exists an effect similar to 2nd Embodiment. In addition, the feature amount can be further automated by extracting the contour of the object.

  14 and 15 are diagrams for explaining another feature amount extraction method of the object recognition system according to the fifth embodiment of the present invention.

  In this embodiment, feature amounts are extracted and searched by selecting regions and parts based on distribution information.

  The database 150 has distribution data for each of various feature amounts stored in advance in the database 150, and the feature amounts extracted for search are ranked based on the distribution value of the distribution data. Yes.

  The searcher first takes an image 26 of the object. When searching for an imaged object, a search start button (not shown) is pressed. When the search start button is pressed, the extractable feature amounts are displayed on the menu screen MP (see FIG. 14A). The feature amounts of the database 150 are, for example, “flower color”, “flower size”, and “flower shape and distribution”. The searcher uses the touch pen 25 to activate the feature amount desired to be extracted first among the feature amounts displayed on the menu screen MP. For example, color information is activated first. The first part 26-1 on the display image 26 in FIG. 14A is selected with the touch pen 25, and the feature amount extraction button 20 a at the lower right of the display image 26 is pressed. Color information corresponding to the selected part 26-1 is registered in the feature amount extraction unit 30, and the recognized color information is displayed on the menu screen MP.

  If there is no other color information to be recognized, a search is performed with the color information selected by pressing the search execution button 20b at the lower left of the display image 26. When there is other color information to be extracted, the second part 26-2 is selected with the touch pen 25 (see FIG. 14B), and the feature amount extraction button 20a is pressed again. This process is repeated until there are no more colors to be extracted.

  When all necessary parts are selected and the feature amount extraction button 20a of the display image 26 is pressed with the touch pen 25, the search is executed (see FIG. 14C), and the database is based on the color information that is the registered feature amount. An identification candidate is selected from 150. A sample image, a name, and the number of candidates are displayed according to the number of selected identification candidates. When the number of identification candidates is small, images and names of identification candidates are displayed on the identification candidate display unit 45. When the number of identification candidates is large, the identification candidates can be sequentially displayed in a scroll mode, only the number of identified candidates can be displayed, or only the number and names of identification candidates can be displayed (FIG. 14 ( d)). It is also possible to display only the names of identification candidates and perform the identification work from the names stored by the searcher. Note that the name display can also be scrolled when there are many identification candidates.

  On the other hand, when the searcher determines that the number of identification candidates is large and identification by name or image is difficult, another feature amount is extracted while the previous information (color information) remains. In order to extract another feature amount, the erasure item 21e in the search result display screen EP is selected with the touch pen 25 to erase the search result display screen EP (see FIG. 14D). Then, the next feature amount ("flower size" in this embodiment) is newly extracted (see FIG. 15A). As described above, the extraction of the size of the flower is performed by selecting the two radial portions 26-3 and 26-4 in the outline of the flower with the touch pen 25 and pressing the feature amount extraction button 20a ( You may switch to the contour extraction mode and automatically extract the diameter from the extracted contour). After the flower size is extracted, the searcher presses the search execution button 20b (see FIG. 15B).

  The search is performed again based on the two feature values of the color extraction result and the flower size extraction result described above, identification candidates are narrowed down, and the search result is displayed on the search result display screen EP ( (Refer FIG.15 (c)). The searcher selects whether to display an image of the search result or to perform a search under the following conditions. When the searcher selects an image display item on the search result display screen EP, the display on the image display unit 20 is as shown in FIG. As shown in FIG. 15D, the sample images 27 of the identification candidates and the explanatory text 28 are displayed, and the searcher performs the identification work based on this display. When the object identification work is completed, for example, the touch pen 25 is applied to the check box 27a on the upper right of the sample image 27, and the sample image 27 and the object image 26 are linked. The image 26 of the object is stored in an image storage unit (not shown) together with predetermined data (information such as flowering time, fruiting time, and growing place).

  In this embodiment, the search is performed using the database 150 of a predetermined category as it is. However, as in the first embodiment, a narrowed-down database (not shown) in which data is extracted at the location and time of imaging is used. You may search using. In this case, distribution data is created for each feature quantity from the narrowed-down database created according to the conditions, and the feature quantities are ranked based on the created distribution data.

  According to this embodiment, there exists an effect similar to 2nd Embodiment.

  FIG. 16 is a flowchart showing an example of the flow of processing in the object recognition system according to the sixth embodiment of the present invention.

  When the power is turned on and information on the time and place for searching for an object is input, the identification operation is performed in the following sequence.

  First, a data group is extracted from the original database 60 based on the input information, and a narrowing database 50 is formed (step 1).

  Next, feature quantity distribution data of the narrowing-down database 50 is formed (step 2).

  Thereafter, the feature amounts are ranked based on the formed distribution data (step 3).

  Steps 1 to 3 are automatically performed by turning on the power and inputting predetermined information. Note that Steps 1 to 3 are omitted when a direct search is performed from the original database 60 or when a unique database (a database in which unique distribution data is formed in advance) is formed at a predetermined location.

  Steps 1 to 3 need only be performed once at first unless the type of the object is changed or the place where the image of the object is acquired is changed.

  Next, an image of the object to be identified is acquired (step 4). The acquired image of the object is displayed on the image display unit 20.

  The feature amount extraction button is pressed to extract the first feature amount (for example, color) (step 5). If the first feature value is automatically extracted from the image, the process proceeds to step 7.

  When the searcher arbitrarily designates a part from which an amount of feature is extracted from the image, it is determined whether there is another area to be extracted (step 6). If there is another area to be extracted, the process returns to step 5.

  When the feature quantity is sufficiently extracted and there are no other areas to be extracted, a search using the first feature quantity from the refined database 50 is performed (step 7). The number of candidates obtained from the narrowed-down database 50 as a result of the search is displayed on the display image 26.

  The searcher looks at the number of candidates displayed on the image display unit 20 and determines whether or not to extract another feature amount (step 8). When the identification operation is performed only with the first feature amount, the candidate image is displayed on the image display unit 20. The searcher can arbitrarily select the display method, such as dividing the candidate image into the image display unit 20 to display the image, or scrolling and displaying the candidate image sequentially on the image display unit 20. When browsing the identified identification candidates, the process proceeds to step 14.

  When the retrieved identification candidate is not browsed, the second feature amount (for example, distribution information) is extracted (step 9). If the feature amount is automatically extracted from the image as in step 5, the process proceeds to step 11.

  When the searcher arbitrarily designates a part from which an amount of feature is extracted from the image, it is determined whether there is another area to be extracted (step 10). When there is another area to be extracted, the process returns to step 9.

  When the feature quantity is sufficiently extracted and there are no other areas to be extracted, a search based on the first and second feature quantities from the narrowed-down database 50 is performed (step 11).

  The searcher looks at the number of candidates displayed on the image display unit 20 and determines whether or not to extract another feature amount (step 12). When browsing the identified identification candidates, the process proceeds to step 14.

  When the retrieved identification candidate is not browsed, the third feature amount (for example, the size of the object) is extracted (step 13).

  The above process is repeated by the number of feature quantities to be extracted, and identification work is performed (step 14).

The searcher identifies an object from the candidate images.

  According to this embodiment, the same effects as those of the first embodiment can be obtained.

  In the second embodiment, a part of the image is selected by bringing the touch pen 25 as an input unit into contact with the display image 26. However, instead of the touch pen 25, a pressure sensor, a temperature sensor, or the like is used for the display image 26. A part of the image may be selected by providing it with a finger or the like. In this way, the selection operation can be performed as part of the operation of the camera, and the labor for holding the touch pen 25 can be saved, and the search time can be shortened.

FIG. 1 is an overall configuration diagram of an object recognition system according to a first embodiment of the present invention. FIG. 2 (a) is a diagram showing an example of regional vegetation data and an original data base, and FIG. 2 (b) is a diagram showing an example of a procedure for creating narrowed-down data from positional group-specific plant group information and the original database. is there. FIG. 3 is a diagram for explaining a method of extracting the size of the flower and the height of the plant height from the image. FIG. 4 is a diagram illustrating an example of distribution data of feature amounts included in the narrowing-down database. FIG. 5 is an overall configuration diagram of an object recognition system according to the second embodiment of the present invention. FIG. 6 is a diagram for explaining a search method of the object recognition system according to the second embodiment of the present invention. FIG. 7 is a view for explaining a search method of the object recognition system according to the second embodiment of the present invention. FIG. 8 is a diagram for explaining a search method of the object recognition system according to the third embodiment of the present invention. FIG. 9 is a diagram for explaining a search method of the object recognition system according to the third embodiment of the present invention. FIG. 10 is a diagram for explaining a search method of the object recognition system according to the third embodiment of the present invention. FIG. 11 is a diagram for explaining a search method of the object recognition system according to the fourth embodiment of the present invention. FIG. 12 is a diagram for explaining a search method of the object recognition system according to the fourth embodiment of the present invention. FIG. 13 is a view for explaining a search method of the object recognition system according to the fourth embodiment of the present invention. FIG. 14 is a view for explaining a search method of the object recognition system according to the fifth embodiment of the present invention. FIG. 15 is a view for explaining the search method of the object recognition system according to the fifth embodiment of the present invention. FIG. 16 is a flowchart showing an example of the flow of processing in the object recognition system according to the sixth embodiment of the present invention.

Explanation of symbols

10: Imaging device (imaging unit), 20: Image display unit (image display unit), 21, 26: Display image, 25: Touch pen (input unit), 90: Selection region image extraction unit (selection region image extraction unit), 30: Feature extraction unit (feature extraction unit), 40: Search engine (search unit), 45: Identification candidate display unit (display unit), 50: Refinement database, 52, 152: Distribution data generation unit (distribution data generation) Means), 60: original database, 70: data extraction unit (database narrowing means), 80: data extraction unit (squeezing condition setting means), 150: database.

Claims (12)

Imaging means for capturing an image of the object;
Feature quantity extraction means for extracting the feature quantity of the object from the image captured by the imaging means;
A database in which feature amount information corresponding to at least the feature amount is registered for a plurality of species included in the category to which the object belongs;
Database narrowing means for extracting a data group from the feature amount information registered in the database;
Narrowing condition setting means for setting conditions for extraction by the database narrowing means;
Distribution data forming means for forming feature quantity distribution data for each of the data groups extracted by the database narrowing means;
Based on the feature quantity distribution data formed by the distribution data forming means, the feature quantity extracted by the feature quantity extracting means and the feature quantity information in the data group extracted by the database narrowing means. A target recognition system comprising: a search unit that searches for feature quantity information having the same feature quantity as the feature quantity from the feature quantity information in the data group.   The distribution data forming means ranks the feature quantities in order from the feature quantity distribution data distributed most evenly among the feature quantity distribution data formed for each feature quantity information, and the search means sets the distribution data formation means. 2. The object recognition system according to claim 1, wherein the search is performed in the order of ranking.   The distribution data forming means ranks the feature quantities in ascending order of the number of feature quantity distribution data having parameters that appear most frequently among the feature quantity distribution data formed for each feature quantity information, and the search means 2. The object recognition system according to claim 1, wherein searches are performed in the order ranked by the distribution data forming means. Imaging means for capturing an image of the object;
Image display means for displaying an image picked up by the image pickup means;
For a plurality of species included in the category to which the object belongs, a database in which feature amount information corresponding to at least the feature amount of the object is registered;
Distribution data forming means for forming feature quantity distribution data for the feature quantity information registered in the database;
Display means for displaying feature quantity distribution data for each feature quantity;
Selection means for selecting a part of the image displayed on the display means based on the feature amount distribution data;
Feature quantity extraction means for extracting the feature quantity of the object based on a part of information of the image selected by the selection means;
The feature quantity having the same feature quantity as the feature quantity from the feature quantity information in the data group by comparing the feature quantity extracted by the feature quantity extraction means with the feature quantity information registered in the database An object recognition system comprising: a search means for searching for information.   The object recognition system according to claim 4, wherein a part of the image is at least one part.   5. The object recognition system according to claim 4, wherein a part of the image is at least one region surrounded by a closed curve selected by the selection means.   The object recognition system according to claim 5 or 6, wherein the feature amount is color information extracted from a selected region or region in the image.   The object recognition system according to claim 5 or 6, wherein the feature amount is distance information between at least two selected parts or regions.   The object recognition system according to claim 6, wherein the feature amount is at least one of a minimum diameter, a maximum diameter, an average diameter, and an area of at least one selected region in the image.   The object recognition system according to claim 6, wherein the feature amount is a shape of at least one region selected in the image.   The object recognition system according to claim 4, wherein the feature amount is distribution information of a plurality of selected parts or / and regions.   5. The object recognition system according to claim 4, wherein in the image display means, the color of the part or area selected by the selection means or the color or brightness of the part or area not selected is changed.

Images