JP2008242712A - Image recognition apparatus and image recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition apparatus and an image recognition method facilitating identification of an image to be compared by means of another image, without technical knowledge. <P>SOLUTION: A kind identification server 4 has an image comparing part 4b for searching for an image similar to an image 11 to be compared, a feature point determining part 4d for determining whether or not the image 11 contains a feature point, a template database 4e for storing a feature point template 13 representing the feature part of the image, and a template output part 4f which, if the feature point determining part 4d cannot detect a feature point, outputs a feature point template 13 associated with the similar image. A camera 1 has a display part for displaying an image corresponding to the feature point template 13. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to an image recognition apparatus and an image recognition method, and more particularly to an image recognition apparatus and an image recognition method that can easily identify an image of a living organism such as an insect.

In recent years, with the spread of the Internet, anyone can easily obtain desired information by Web search. However, it is difficult to set appropriate keywords without specialized knowledge, such as when collecting collected insects or plants, or when trying to identify the type of photograph of captured insects or plants. It takes a lot of time and effort to identify the

Therefore, various apparatuses and methods for obtaining information by performing a search using an image recognition technique have been proposed. For example, in Patent Document 1, when a recognition mode is selected in a digital camera having a normal mode and a recognition mode, feature information of an object is extracted based on the captured front image data and side image data. It is disclosed that the name of an object is read by comparing information with reference information on a database.
JP 2002-247436 A

Patent Document 2 discloses a system that obtains a feature amount of an object from a plant image and discriminates the type from the plant feature amount registered in a database in advance. Here, for example, when the object is a flower image, the feature amount is an amount that can be numerically analyzed as determined from the distance from the center of gravity to the outline of the petal and the angle relationship with the center of gravity. . Further, Patent Document 3 discloses an information search apparatus that acquires a keyword from an image search in a database in which image features and keywords are associated, and acquires related information from an information search using the keyword.
Japanese Patent Laid-Open No. 2002-203242 JP 2002-297648 A

However, in the digital camera disclosed in Patent Document 1, since the feature information is limited to the front image data and the side image data, the images may not be identified because the feature portions are not always included in these images. . Further, in the system disclosed in Patent Document 2, a feature amount is extracted from an image, but a feature amount necessary for identification may not be extracted from a read image. Furthermore, the information search apparatus disclosed in Patent Document 3 has a problem that it cannot be searched for an image that does not include a feature effective for identifying an object. In other words, these prior arts do not always include the characteristic portions necessary for identification in image analysis. For this reason, without specialized knowledge, image identification cannot be performed easily. It was.

The present invention has been made in view of such circumstances, and provides an image recognition apparatus and an image recognition method that can easily identify an image to be compared with an image without specialized knowledge. The purpose is to do.

In order to achieve the above object, an image recognition apparatus according to a first invention includes a similar image recognition unit that recognizes a similar image similar to a predetermined image, an image recognition unit that recognizes the image, and a feature portion of the image. A template storage unit that stores a template to be represented; a template output unit that outputs the template associated with the similar image when the image recognition unit cannot recognize the image; and an image that inputs an image corresponding to the template An input unit is provided.

In the image recognition apparatus according to the second invention, in the first invention, the similar image recognition unit recognizes a similar image based on the contour information of the image.
According to a third aspect of the present invention, in the image recognition apparatus according to the first aspect, the image recognition unit compares the feature part representing the feature of the target image with the feature part of the image registered in advance. Recognize the target image.
The image recognition apparatus according to a fourth aspect of the present invention is the image output apparatus according to the first aspect, wherein the image recognition apparatus further outputs a keyword related to the image when the image recognition unit can recognize the image. It has a part.
Furthermore, the imaging apparatus according to a fifth aspect of the present invention is the imaging apparatus according to the first aspect, wherein the image recognition apparatus further includes an imaging unit that captures an image, wherein the imaging unit is the template output unit. The captured image is captured based on the template output by.

In order to achieve the above object, an image recognition method according to a sixth invention includes a similar image recognition step for recognizing a similar image similar to a predetermined image, an image recognition step for recognizing the image, and an image recognition unit step. When the image cannot be recognized, a template output step is provided for outputting a template that is associated with the similar image and that represents a feature of the image.

In order to achieve the above object, an image recognition system according to a seventh invention includes a similar image recognition unit for recognizing a similar image similar to a predetermined image, a template storage unit for storing a template for recognizing the image, and the image When the recognition unit cannot recognize the image, a recognition unit includes a template output unit that outputs the template associated with the similar image, an image input unit that inputs an image corresponding to the template, and an imaging unit that captures the image. .
In order to achieve the above object, a program according to an eighth invention includes a similar image recognition step for recognizing a similar image similar to a predetermined image, an image recognition step for recognizing the image, and the image recognition unit If it cannot be recognized, the computer is caused to execute a template output step for outputting a template representing the feature portion of the image associated with the similar image and an image input step for inputting an image corresponding to the template.

In order to achieve the above object, an image recognition apparatus according to a ninth aspect of the present invention provides an image database for storing type feature points and a template for storing feature point templates for displaying the feature points when determining the type of the compared image. Determining a presence of the feature point stored in the image database in the comparison image, a keyword database storing a keyword representing a type of the comparison target image based on the feature point, and determining As a result, when it is determined that there is no feature point, the corresponding feature point template is selected from the template database. On the other hand, when it is determined that the feature point is present, the corresponding keyword is selected from the keyword database. A feature point determination unit.

According to a tenth aspect of the present invention, in the ninth aspect, the image database stores a reference image group representing a rough type, and prior to the determination by the feature point determination unit, the compared image is An image comparison unit that selects a similar image by comparing with the reference image group stored in the image database.
An image recognition apparatus according to an eleventh aspect of the invention includes a display unit that displays the feature point template selected by the feature point determination unit in the ninth aspect of the invention.

In order to achieve the above object, an image recognition method according to a twelfth aspect of the present invention comprises a step of comparing a compared image with a feature point of a type stored in an image database, and when the feature point cannot be recognized in this comparison. And a step of displaying a template representing an image feature similar to the image to be compared.
In the image recognition method according to the thirteenth invention, in the twelfth invention, prior to the step of comparing the feature points, the compared image is compared with a reference image group, and a similar image is selected.

In order to achieve the above object, an image pickup apparatus according to a fourteenth aspect of the invention is an image pickup means for acquiring an image and outputting a compared image, an image database storing feature points for each type of image, and displaying the feature points as an image. In the template database for storing the feature point template and the compared image, the presence / absence of the feature point stored in the image database is determined in the image to be compared, and when it is determined that the feature point does not exist, A feature point determination unit that selects a corresponding feature point template from the template database; and a display unit that displays the feature point template selected by the feature point determination unit.
An image pickup apparatus according to a fifteenth aspect of the present invention is the image pickup apparatus according to the fourteenth aspect of the present invention, further comprising a keyword database that stores keywords representing the type of the image based on the feature points. If it is determined that there is a keyword, the corresponding keyword is selected from the keyword database, and the display unit displays the keyword.
In the imaging device according to a sixteenth aspect based on the fourteenth aspect, the display unit superimposes and displays the image captured by the imaging unit and the feature point template.
In the imaging device according to a seventeenth aspect according to the fourteenth aspect, the image database stores a reference image group representing a rough type, and the image to be compared is the above prior to determination by the feature point determination unit. An image comparison unit that compares the reference image group stored in the image database and selects a similar image is provided.

In order to achieve the above object, an imaging assisting method according to an eighteenth aspect of the present invention includes a step of inputting a template from the outside, a step of setting a template shooting mode for displaying the template, and a step of shooting in the template shooting mode. And a step of outputting the photographed image acquired in the photographing to the outside.
In order to achieve the above object, a program according to a nineteenth invention comprises a step of outputting a template to a camera, a step of receiving image data shot using the template from the camera, and the image data for image recognition. Sending image data to the server.

According to the present invention, a template representing a feature portion of an image is stored, and the template is output when the image recognition unit cannot recognize the image. It is possible to provide an image recognition apparatus and an image recognition method that can easily identify an image.

Hereinafter, a preferred embodiment will be described using an image recognition system to which the present invention is applied according to the drawings. In the present embodiment, the type of the comparison target image input directly from an image input device such as a digital camera or via a personal computer is identified by the type determination server. If the type can be determined, the type determination server feeds back the name to the digital camera or personal computer based on the identified type.

If the type cannot be determined, a template for displaying the characteristic part is fed back to the user so that the user can acquire an image including the characteristic part necessary for the type determination. The user obtains the image to be compared with reference to the template, and can thereby easily identify the type of the image to be compared. The camera or personal computer and the type determination server are connected via the Internet.

FIG. 1 is a block diagram of an image recognition system including a digital camera, a personal computer, and a server on the Internet according to an embodiment of the present invention. The camera 1 can acquire the subject 2 as digital image data by the image sensor and record it on a recording medium in the camera 1. A personal computer (PC) 3 is connected to the camera 1 wirelessly or by wire and can transmit and receive data. The type determination server 4 is a server arranged on the Internet, and can be connected to the camera 1 directly or via the PC 3.

In the type determination server 4, an image input unit 4 a for inputting a comparison target image to be identified from the PC 3 or the like, an image that is a database that stores a reference image, a feature point image, and an identification determination image. The DB 4c, an image comparison unit 4b that compares the image to be compared with the reference image is provided. Further, a feature point determination unit 4d that receives the outputs of the image comparison unit 4b and the image DB 4c and determines the presence / absence of feature points necessary for identification of the compared image is provided. A template DB 4e and a keyword DB 4g that receive the determination result of the feature point determination unit 4d are provided.

As will be described in detail later, the template DB 4e stores a feature point template 13 (image data) for displaying the feature part to be identified in an easily understandable manner. The output of the template DB 4e is connected to the template output unit 4f, and the output of the template output unit 4f is connected to the PC 3 or the camera 1 directly via the Internet.

The keyword DB 4g stores a name (keyword) representing the type determined by the feature point determination unit 4d. The output of the keyword DB 4g is connected to the keyword output unit 4h, and the output of the keyword output unit 4h is connected to the PC 3 or the camera 1 directly via the Internet.

The operation of the embodiment of the present invention configured as described above will be described. First, the user photographs the subject 2 using the camera 1. The captured image 11 to be compared (see FIG. 2) is transferred to the image input unit 4a of the type determination server 4 via an information input / output terminal such as the PC 3. When the camera 1 itself has a communication function, the image may be directly transferred from the camera without using the PC 3. The captured image may be temporarily stored in a storage device other than the PC 3 and directly transferred from the storage device, or the image may be scanned and transferred.

The compared image 11 transferred to the image input unit 4a is sequentially compared with the reference image group 12 (see FIG. 2) stored in the image DB 4c, which is a database, in the image comparing unit 4b. select. That is, the compared image 11 and the reference images 12a, 12b, and 12c in the reference image group 12 are sequentially compared, and similar images are selected. In this example, the compared image 11 is a registration mark, and the image comparison unit 4b selects the reference image 12c representing the registration mark as an image most similar to the compared image 11. In FIG. 2, the reference image group 12 depicts three types of reference image groups 12a to 12c, but a number of reference image groups are stored in the image DB 4c.

When the reference image group 12 is selected, the feature determination unit 4d next determines the feature points 13a to 13d of the substantially matched images in order to determine the presence / absence of the feature points necessary for identification of the compared image 11 (see FIG. 2), the identification effectiveness of the image to be compared is determined. Here, feature points of the image will be described. In the identification by the image, the shape information and the color information are to be compared, but the problem is which part can be identified by comparing. Therefore, in this embodiment, feature points are defined and identified.

First, the image comparison unit 4b compares the contour extracted from the image 11 to be compared with the contour of the reference image group 12, and defines each feature point according to a classification in which the contours substantially match. For example, in the case of an insect, the positions of the head, chest, abdomen, and the like are specified from the positional relationship between the eye (substantially circular portion) and the center of gravity of the contour. The positional relationship among the eyes, the contour centroid, and each part is individually defined by each classification. Further, in the case of flowers, there is a method in which the contour centroid is regarded as the center of the flower, and the petals are specified from the radial distance from the contour centroid to the contour.

For example, when the image to be compared 11 is a dragonfly, (1) head shape, (2) chest shape, (3) abdominal shape, and (4) wrinkle shape are required for identification. . Therefore, the above-described feature points 13a to 13d are prepared as feature points corresponding to the above (1) to (4). The feature determination unit 4d determines whether or not the feature points 13a to 13d necessary for identification are included in the compared image 11 input by the image input unit 4a.

As a result of the determination, if these feature points are included, the comparison image 11 and the identification determination image 31 (see FIG. 6) are further compared to perform identification determination. In the identification determination, the shape of the feature portion and the feature portion color are compared. In FIG. 6, the difference in feature color is indicated by the difference in the shaded pattern. A keyword 32 is associated with the identification determination image 31 shown in FIG. 6 as the type name.

The template DB 4e, which is a database in which the feature point template 13 is stored, sends the selected template to the PC 3 via the template output unit 4f. The keyword DB 4g, which is a database storing keywords similar to the keyword 32 in the identification determination image 31, sends the selected keyword to the PC 3 via the keyword output unit 4h.

Here, whether the feature point template 13 or the keyword 32 is sent depends on the determination result of the feature point determination unit 4d. That is, the keyword 32 is output when the comparison target image 11 can be identified effectively. On the other hand, if the comparison target image 11 cannot be identified effectively, the feature point template 13 is sent out.

If the comparison target image 11 cannot be identified effectively, the PC 3 receives the feature point template 13 from the type determination server 4 and displays the template image on the PC 3 screen. Forward to 3.

The user can obtain an image whose type can be discriminated by photographing the subject 2 so as to obtain an image including the feature point with reference to the feature point template 13 displayed on the PC 3. In addition, when photographing with the camera 1, the image display device such as a liquid crystal has a function of combining and displaying the feature point template 13 on the live image from the image sensor, so that the user can photograph the subject while superimposing the subject on the template. Therefore, it is possible to easily shoot a subject image including feature points. This composite display will be described later with reference to FIGS.

By specifying the background color, the contrast setting, the flash setting, the white balance setting, etc. as the template additional information, it is also possible to easily acquire an image that can easily obtain contour and color information. . The PC 3 displays the keyword received from the type determination server 4 as text on the screen. Further, as will be described later, the keyword is sent to the keyword search engine 21 (see FIG. 3) via the network.

Next, the operation in the type determination server 4 will be described with reference to FIG. First, the comparison target image 11 for identifying the type is input from the PC 3 or the camera 1 to the image input unit 4a (S101). The input comparison target image 11 is transferred to the image comparison unit 4b.

Subsequently, the contour portion of the transferred comparison target image 11 is extracted (S102). Note that the image may be transmitted after the contour extraction image processing is executed by the PC 3 or the camera 1. When the contour extraction process is completed, the image comparison unit 4b then compares the contour of the extracted image to be compared with the reference image 12 (see FIG. 2) stored in advance in the image DB 4c (S103). . In FIG. 2, insects are taken as an example. Of course, any of flowers, fish, buildings, works of art, etc. may be used.

The presence / absence of a similar image is determined based on the contour similarity determination result in step S103 (S104). As a result of the determination, if a similar image is not obtained, information indicating that identification is impossible is fed back to the PC 3 or directly to the camera 1 (S114). The PC 3 or the camera 1 receives this information and displays on the screen that the type of the compared image 11 cannot be identified. At this time, for example, “The outline of the target image is unclear, so identification could not be performed. Change the background and try again. Or the like may be displayed at the same time.

On the other hand, when it is determined in step S104 that a similar image is found and a similar image exists, that is, when the similarity between the compared image 11 and the reference image group 12c is confirmed, the feature point determination unit 4d In order to determine whether or not there is a feature portion associated with the similar image, a feature point image corresponding to the similar image (an image equivalent to the feature point template 13 in FIG. 2) is acquired from the image DB 4c (S105).

Next, the compared image 11 is compared with the selected feature point image (equivalent to the feature point template 13 of FIG. 2), and a comparison is made as to whether the compared image 11 includes the feature portions 13a to 13d. (S106). As a result of the feature image extraction, it is determined whether or not a feature part necessary for identification is included in the compared image 11 (S107). As a result of the determination, if it is determined that it is not included, the feature point template 13 associated with the similar image selected in the contour similarity determination in step S103 is acquired from the template DB 4e (S110). Thereafter, the selected feature point template 13 is sent to the PC 3 via the template output unit 4f.

On the other hand, if it is determined in step S107 that the comparison target image 11 includes a characteristic portion, the comparison target image 11 and the identification determination image 31 are compared in detail (S108). Here, the feature portion of the compared image 11 and the identification determination image 31 (see FIG. 6) are compared in the order of contour / pattern and color, and an image that matches the compared image 11 is searched.

Next, it is determined whether or not a feature necessary for identification has been retrieved as a result of the image comparison in step S108 (S109). If there is a matching feature as a result of the determination, the keyword 32 (see FIG. 6) associated with the feature image is acquired from the keyword DB 4g (S112), and is sent to the PC 3 via the keyword output unit 4h. Send out (S113). On the other hand, if it is determined in step S109 that the feature necessary for identification could not be retrieved, the user is prompted to perform re-shooting in the processing after step S110.

Next, the network configuration in the present embodiment will be described with reference to FIG. A PC 3 as an information input / output terminal and the type determination server 4 described with reference to FIG. A keyword search engine 21 and an image search engine 22 are connected to the Internet 20.

The PC 3 connected to the Internet 20 is an information input / output terminal having various information processing functions including image data exchange with the camera 1, image processing, and data display. As described above, the camera 1 communicates with the type determination server 4 via the PC 3 or directly, and acquires the keyword data 24 related to the compared image 11 and the data related to the image template 13. Although detailed description is omitted, it is also possible to acquire the key image 23 in addition to the keyword 24.

The PC 3 uses the data acquired from the type determination server 4, the information acquired via the Internet, the information acquired from the camera, etc., via the Internet 20, the keyword search engine 21, the image search engine, etc. 22 can be sent.

The keyword search engine 21 is a database for searching for a related URL (Uniform Resource Locator) from the keyword, and returns the URL related to the keyword 24 from the PC 3 to the PC 3. Further, the type determination server 4 does not send the searched keyword 24 to the keyword search engine 21 via the PC 3, but directly sends it from the type determination server 4 to the keyword search engine 21, and returns the related URL to the PC 3 via the type determination server. May be. The user can access the acquired URL from the PC 3 and obtain necessary information.

The image search engine 22 is a database that compares the input image with the image on the input URL and returns only the URL including the matched image. The user can further narrow down the information obtained by the keyword search by the image search, and obtain only the compared image 11 and the identified information.

The search performed via the Internet will be described with reference to the flowchart shown in FIG. First, the PC 3 sends a keyword from the type determination server 4 to the keyword search engine 21 via the Internet 20 (S201). Subsequently, the keyword search engine 21 executes a search based on the keyword, and returns the searched related URL to the PC 3 (S202).

Next, the image including the characteristic part to be identified (compared image 11) and the URL obtained in step S202 are sent to the image search engine 22 via the Internet 20 (S203). The image search engine 22 performs URL narrowing by image comparison, and returns the narrowed URL to the PC 3 (S204). The user can efficiently acquire the information identified as the compared image (original image) 11 by accessing the narrowed URL.

Next, the operation in which the camera 1 receives the feature point template 13 and sends the compared image 11 to the type determination server 4 will be described with reference to FIGS. As described above, in step S109, if the image to be compared 11 does not have a characteristic portion and cannot be identified, a template is output to the PC 3 or the camera 1 (S111). Here, a method will be described in which the camera 1 obtains a template and a user obtains an image in which a feature portion is captured.

As shown in FIG. 7, the camera 1 includes a liquid crystal monitor 1a for image display, operation members 1b, 1c, and 1d, a built-in flash 1e for subject illumination, and a release button 1f for instructing a photographing operation. . The camera 1 has a communication function with the PC 3. The communication may be wired or wireless, but the camera 1 and the PC 3 can bidirectionally exchange images and various data other than images according to a predetermined protocol. The subject 2 is an object to be identified by the user.

The user can arbitrarily change various settings of the camera 1 by performing predetermined operations on the operation members 1b, 1c, and 1d of the camera 1. The settings include white balance setting, contrast intensity setting, color balance setting, macro mode setting, etc., in addition to the presence or absence of flash emission by the built-in flash 1e. Further, setting of a template shooting mode described later is also included.

The liquid crystal monitor 1a has a reproduction function for confirming a photographed image, and further has a function as a substitute for a finder, and can display a live video (also referred to as a through image or an electronic view) of a subject on a monitor. it can. In addition, photographing assist information (template) such as an outline figure 41 (in this embodiment, as an example, an outline figure of a register mark), text 42, guideline 43, and the like can be displayed over the image.

Next, the flow of operations of the camera 1 and the PC 3 will be described using the flowchart shown in FIG. First, when the PC 3 transmits data related to the feature point template 13 received from the template output unit 4f to the camera 3 (S301), the camera 3 receives this data (S302).

Subsequently, the camera 3 shifts to the template shooting mode based on the received template data (S303). When the camera 1 shifts to the template shooting mode, the contour figure 41 of the target object, the text 42 for instructing points to be taken in shooting, the guideline 43 for instructing the composition where the object should be arranged, etc. are displayed on the liquid crystal monitor 1a. . The contour figure 41 is an assist for surely capturing the feature points necessary for identification, and the guideline 43 indicates the center of the screen, for example, so that the comparison priority area is used in the subsequent image comparison. This is useful for increasing the accuracy of identification. In addition, the text 42 can be expected to eliminate mistakes by expressing the above photographing points to be noted with sentences.

In addition, the template information includes contrast enhancement settings for improving the accuracy of contour extraction, flash photography for standardizing color reproduction, macro mode settings if the target is an insect, and the camera 1 is effective for identification. Perform all settings. The above may be set automatically by the camera 1 in accordance with the received template information, or may be set by the user operating the operation members 1b, 1c, and 1d in accordance with instructions.

Also, by displaying these template information on the liquid crystal monitor 1a of the PC 3 and shooting according to the display by the user, even a camera that does not have a template shooting mode or a function that does not have a data reception function from the PC 3 An image effective for identification may be acquired.

When the setting in step S303 is completed, the camera 1 captures the subject 2 according to this setting (S304). Shooting is executed when the user presses the release button 1f. However, it is also possible to detect that the contour of the subject substantially matches the contour figure 41 and use it as a trigger for shooting.

When shooting is completed, the camera 1 sends the shot image data to the PC 3 (S305). Accordingly, the PC 3 receives image data from the camera 1 (S306). When the image data is received, the PC 3 performs image processing (S307). The image processing is processing for improving the accuracy of subsequent image comparison, for example, extracting the contour of the subject and converting the remaining background data into a single color. Note that this process may be executed in a portion with a sufficient processing capability, and may be executed by the camera 1 or may be executed by the type determination server 4 in the subsequent stage, for example. When the image processing is completed, the PC 3 sends the image data to the type determination server 4 (S308).

When receiving the image data, the type determination server 4 executes the image comparison flow shown in FIG. 4 to identify the compared image 11. As shown in FIG. 7 and FIG. 8, since the user uses the template to acquire the image in which the feature points are imprinted, the type determination server 4 can easily determine the type, and the user Can be fed back to the PC 3 and the camera 1.

As described above, in the present embodiment, the type determination server 4 determines whether or not the compared image 11 has a feature, and if the result of the determination is that there is no feature in the image, the feature point template 13 is displayed. , Have feedback to the user. For this reason, since the user can take an image effective for identification based on the template, it is possible to easily identify the object and obtain information from the image without any special knowledge. In addition, if information related to a wide variety of objects is stored in the image database 4c, the names of the objects can be easily recognized even for a wide variety of objects without any special knowledge.

In the present embodiment, prior to the feature point determination, the image comparison unit 4b searches for a similar image from the reference image group 12, and whether or not there is a feature point in the searched similar image. For this, the compared image 11 is determined. For this reason, feature points can be narrowed down and identification can be facilitated.

Furthermore, when there is a feature point, it is possible to determine the type of the compared image 11 using the identification determination image 31 and detect the keyword from the keyword DB 4g. Therefore, the user can easily know the name of the image taken as the image to be compared 11.

In the present embodiment, an image is stored and output as the feature template 13. However, the present invention is not limited to this, and character data representing a feature portion such as “head”, “chest”, and “feather” may be used. . In the present embodiment, when the feature point determination unit 4d cannot detect a feature point (S107), it is determined that identification is not possible (S114). However, without making identification impossible, for example, a user may input a word such as “dragonfly” or “chow” as text data to search for a feature image and continue the identification operation.

Furthermore, in the present embodiment, the type determination server 4 is configured on the Internet. However, the configuration is not limited to this, and the configuration in the type determination server 4 may be built in the camera 1 or the PC 3. . In particular, when the camera 1 is provided integrally, it is not necessary to connect to the PC 3 when displaying the feature point template as shown in FIG.

1 is a block diagram of an image recognition system according to an embodiment of the present invention. It is a figure which shows the example of a to-be-compared image, a reference image group, and a feature point template in the image recognition system concerning one Embodiment of this invention. It is a block diagram which shows the network structure in one Embodiment of this invention. It is a flowchart figure which shows the operation | movement of the image comparison in one Embodiment of this invention. It is a flowchart which shows the search in one Embodiment of this invention. It is a figure which shows the example of the identification determination image in one Embodiment of this invention. It is a figure which shows the state which image | photographs using the feature point template in the camera in one Embodiment of this invention. It is a flowchart figure which shows the operation | movement of a camera and PC at the time of imaging | photography using the feature point template in one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera, 1a ... Liquid crystal monitor, 1b ... Operation member, 1c ... Operation member, 1d ... Operation member, 1e ... Built-in flash, 1f ... Release button, 2. ..Subject, 3 ... Personal computer (PC), 4 ... Type determination server, 4a ... Image input unit, 4b ... Image comparison unit, 4c ... Image DB, 4d ... Features Point determination unit, 4e ... template DB, 4f ... template output unit, 4g ... keyword DB, 4h ... keyword output unit, 11 ... compared image, 12 ... reference image group, 12a ... reference image, 12b ... reference image, 12c ... reference image, 13 ... feature point template, 13a ... feature point, 13b ... feature point, 13c ... feature point, 13d ... feature point, 20 ... in -Net, 21 ... keyword search engine, 22 ... image search engine, 23 ... key image, 24 ... keyword data, 31 ... identification determination image, 32 ... keyword, 41 ... Outline figure, 42 ... text, 43 ... guideline

Claims (19)

A similar image recognition unit for recognizing a similar image similar to a predetermined image;
An image recognition unit for recognizing the image;
A template storage unit for storing a template representing the feature portion of the image;
A template output unit that outputs the template associated with the similar image when the image recognition unit cannot recognize the image;
An image input unit for inputting an image corresponding to the template;
An image recognition apparatus comprising:   The image recognition apparatus according to claim 1, wherein the similar image recognition unit recognizes a similar image based on contour information of the image.   The image recognition apparatus according to claim 1, wherein the image recognition unit recognizes the target image by comparing a feature portion representing a feature of the target image with a feature portion of an image registered in advance.   The image recognition apparatus according to claim 1, further comprising a keyword output unit that outputs a keyword related to the image when the image can be recognized by the image recognition unit.   The image recognition apparatus further includes an imaging unit that captures an image, and the imaging unit captures an image captured based on the template output by the template output unit. The imaging device according to claim 1. A similar image recognition step for recognizing a similar image similar to the predetermined image;
An image recognition step for recognizing the image;
A template output step for outputting a template representing a characteristic portion of the image associated with the similar image when the image cannot be recognized in the image recognition unit step;
An image recognition method comprising: A similar image recognition unit for recognizing a similar image similar to a predetermined image;
A template storage unit for storing a template for recognizing the image;
A template output unit that outputs the template associated with the similar image when the image recognition unit cannot recognize the image;
An image input unit for inputting an image corresponding to the template;
An imaging unit that captures the image;
An image recognition system comprising: A similar image recognition step for recognizing a similar image similar to the predetermined image;
An image recognition step for recognizing the image;
A template output step of outputting a template representing a characteristic part of the image associated with the similar image when the image recognition unit cannot recognize the image;
An image input step for inputting an image corresponding to the template;
A program that causes a computer to execute. In determining the type of the compared image, an image database that stores the type feature points;
A template database for storing a feature point template for image display of the feature points;
A keyword database for storing keywords representing the types of the compared images based on the feature points;
The presence / absence of the feature point stored in the image database is determined in the image to be compared, and when it is determined that the feature point does not exist, the corresponding feature point template is selected from the template database. On the other hand, if it is determined that the feature point is present, a feature point determination unit that selects a corresponding keyword from the keyword database;
An image recognition apparatus comprising: The image database stores a reference image group representing a rough type,
10. The image comparison unit for comparing the reference image group with the reference image group stored in the image database and selecting a similar image prior to the determination by the feature point determination unit. The image recognition apparatus described in 1.   The image recognition apparatus according to claim 9, further comprising a display unit that displays the feature point template selected by the feature point determination unit. Comparing the image to be compared with the feature points of the type stored in the image database;
In this comparison, when the feature point cannot be recognized, displaying a template representing a feature portion of an image similar to the compared image;
An image recognition method comprising:   13. The image recognition method according to claim 12, wherein, prior to the step of comparing the feature points, the image to be compared and a reference image group are compared and a similar image is selected. An imaging means for acquiring an image and outputting a compared image;
An image database that stores feature points for each type of image;
A template database for storing feature point templates for displaying the feature points as images;
The presence / absence of the feature point stored in the image database is determined in the image to be compared, and if it is determined that the feature point does not exist, the corresponding feature point template is selected from the template database. A feature point determination unit,
A display unit for displaying the feature point template selected by the feature point determination unit;
An imaging apparatus comprising: A keyword database for storing keywords representing the type of the image based on the feature points;
When it is determined that the feature point is present, the feature point determination unit selects a corresponding keyword from the keyword database,
The imaging device according to claim 14, wherein the display unit displays the keyword.   15. The imaging apparatus according to claim 14, wherein the display unit superimposes and displays the image captured by the imaging unit and the feature point template. The image database stores a reference image group representing a rough type,
15. The image comparison unit according to claim 14, wherein the comparison target image is compared with the reference image group stored in the image database prior to determination by the feature point determination unit, and a similar image is selected. The imaging device described in 1. Steps to enter the template from outside,
Setting a template shooting mode for displaying the template;
Shooting in the template shooting mode;
Outputting the captured image acquired in the shooting to the outside;
A shooting assist method for a camera comprising: Outputting a template to the camera;
Receiving image data taken using the template from the camera;
Sending the image data to the image recognition server;
A program that causes a computer to execute.