KR20080036231A - Information presenting system, information presenting terminal, and server - Google Patents

Abstract

An information presenting system (10) comprises an imaging section (11) for capturing an image, a peripheral information acquiring section (14) for acquiring peripheral information, a storage section (15) for storing registration information, an analysis basic information creating section (16) for creating analysis basic information from the stored registration information according to the acquired peripheral information, and a related information acquiring section (13) for acquiring information related to the image from the captured image and the created analysis base information, and a presenting section (12) for presenting the acquired related information. The analysis basic information creating section (16) varies the analysis basic information according to at least one of the time, date, season, weather, air temperature, humidity, place, ambient brightness, average color/color distribution of the captured image, ambient noise, and type of the device.

Description

Information presentation system, information presentation terminal and server {INFORMATION PRESENTING SYSTEM, INFORMATION PRESENTING TERMINAL, AND SERVER}

The present invention relates to an information presentation system, an information presentation terminal, and a server for presenting information related to an image obtained from an imaging unit.

Japanese Patent Laid-Open No. 2003-15980 discloses a technique in which a server finds a corresponding URL by using an image of a photographed logo. That is, in this technique, first, a logo mark included in an advertisement printed on a magazine or the like is photographed on a portable device. Next, the logo mark acquired by this photography is sent to a search server via the Internet. The search server searches for the logo mark closest to the logo mark sent over the Internet. The search server then sends the URL corresponding to the searched logo mark to the portable device via the Internet. The portable device can access the URL and browse the home page.

Also, Japanese Patent Laid-Open No. 2004-179783 discloses a technique in which a user reads an ID with a portable client device without being aware of the ID type. That is, in this technique, the portable client device first acquires an image with a camera or a scanner. Next, the portable client device extracts the ID from the image even if the image is a one-dimensional barcode, a two-dimensional barcode, or an image containing an electronic watermark. The portable client device then wirelessly transmits the ID to an external ID information database. The portable client device wirelessly receives information from the ID information database.

Here, in the technique disclosed in Japanese Patent Laid-Open No. 2003-15980, a logo and a URL are fixedly associated with each other. However, regardless of the surrounding situation, the conditions of the device to be used, etc., the use of the common content or the common various parameters will lower convenience. For example, by the electric decoration, even if it is the same sign, the way of seeing in day and night changes. There are also changes in contacts during the day and night. In this manner, the information actually requested by the user differs depending on the environment in which the user photographs the logo. Therefore, in the technique disclosed in Japanese Patent Laid-Open No. 2003-15980, it is difficult to flexibly respond to a user's request.

In addition, in the technique disclosed in Japanese Patent Laid-Open No. 2004-179783, it is implicitly understood that various image processing parameters required for reading an ID are held fixed in the apparatus. However, in the case of supporting a plurality of models and a plurality of data sets in the same application, if the conditions for recognition are fixed, the corresponding environment is limited. That is, in an actual use situation, it is difficult to stably perform image processing unless the parameter is adjusted according to the difference in the brightness of the surroundings or the optical performance for each terminal.

By the way, as a method of judging the degree of agreement between two images, for example, "Kanazawa Yasushi and Kenichi Kanaya," Automatic Detection of Feature Point Correspondence Between Two Images, "" Image Lab, November 2004 "(Japanese Industrial Publication 2) ) 004, PP20-23 ", the following method is disclosed. First, feature points as recognition elements in the input image and the comparison image are respectively extracted. And the feature point in an input image is compared and contrasted with the feature point in a comparison image. In comparison between these feature points, it is determined by using the RANSAC method or the like whether the value of the similarity between the two images exceeds a threshold set to a predetermined value as the satisfaction of the data. However, this method mainly causes the following three points.

The first problem is that the above method is a method of comparing the images themselves, so that the entire object or the entire image on which the object is taken is registered in advance, and the registered image (comparative image) is compared with the input image thereon. It is going to be going. Therefore, when an object other than the assumed object was imprinted on the input image or the comparative image, or an object similar to the assumed object was imprinted, a false recognition may occur.

The second problem is that in the above method, the following problem may occur when a similar object is taken in an image when the object is recognized. For example, S company which makes OBJ1 (feature point group; A, B, C) its logo mark, and M company which makes OBJ2 (feature point group; E, F, G) its logo mark are assumed. Here, S company and M company are said to compete with each other. In this case, of course, confusion of both logo marks should be avoided as much as possible. However, in the above method, even when one of the feature point A and the feature points E to G is extracted from the same image, the determination is not made by special logic in consideration of avoiding confusion between OBJ1 and OBJ2. Therefore, in such a case, there exists a possibility that a problem may arise in said method.

The third problem is that even when the number of feature points that can be recognized is the same, the sentence expression that delivers the recognition result to the user is the same. For this reason, for example, when only some feature points can be recognized, that is, when the degree of matching between the input image and the comparison image is a degree of matching including uncertainty, the effect cannot be transmitted to the user.

<Start of invention>

This invention is made | formed in view of the said point, and an object of this invention is to provide the information presentation system, the information presentation terminal, and the server which can show the optimal information according to a situation by use environment.

One aspect of the information presentation system of the present invention,

An imaging unit for obtaining an image,

A peripheral information acquisition unit for acquiring peripheral information,

A storage unit that stores registration information,

An analysis basic information construction unit for constructing analysis basic information from the registered information stored in the storage unit based on the peripheral information acquired by the peripheral information acquisition unit;

A related information acquisition unit for acquiring information related to the image from an image obtained from the imaging unit and analysis basic information obtained from the analysis basic information construction unit;

Presenting unit for presenting the relevant information obtained by the related information acquisition unit

And

The analysis basic information constructing unit uses the analysis basic information based on at least one of time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. It is characterized by changing.

In addition, one aspect of the information presentation terminal of the present invention,

An imaging unit for obtaining an image,

A peripheral information acquisition unit for acquiring peripheral information related to the imaging unit;

An analysis basic information constructing unit for constructing analysis basic information based on the surrounding information acquired by the peripheral information obtaining unit from the registered information downloaded from the server via a communication line;

A related information acquisition unit for acquiring information related to the image from an image obtained from the imaging unit and analysis basic information obtained from the analysis basic information construction unit;

Presenting unit for presenting the relevant information obtained by the related information acquisition unit

And

The analysis basic information constructing unit uses the analysis basic information based on at least one of time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. It is characterized by changing.

Moreover, one aspect of the server of this invention is

A peripheral information acquisition unit for acquiring an image obtained from an imaging unit transmitted from the information presentation terminal via a communication line and / or peripheral information related to the imaging unit;

A storage unit that stores registration information,

An analysis basic information construction unit for constructing analysis basic information from the registered information stored in the storage unit based on the peripheral information acquired by the peripheral information acquisition unit;

And

The analysis basic information constructing unit uses the analysis basic information based on at least one of time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. Change,

By downloading the analysis basic information constructed by the analysis basic information construction unit to the information presentation terminal, the information presentation terminal acquires the information relating to the image from the analysis basic information and the image obtained from the imaging unit, It is characterized by presenting.

That is, in the information presentation system, the information presentation terminal, and the server of the present invention, as the surrounding information, time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient Well, at least one of the models is used. In this way, analysis basic information is constructed from registration information based on the surrounding information. Then, the analysis basic information is added to the image picked up by the imaging unit, and information related to the image is obtained and presented.

Examples of the peripheral information acquisition unit for acquiring the peripheral information include the following. For example, when time is adopted as the surrounding information, the surrounding information acquisition unit includes a built-in clock and a communication unit that obtains time information of a predetermined place through a communication line. For example, in the case of employing a date as the peripheral information, the peripheral information acquisition unit includes a built-in calendar information and a communication unit that obtains a date of a predetermined place through a communication line. For example, in the case of employing the weather, temperature, or humidity as the surrounding information, the surrounding information acquisition unit is provided with a predetermined temperature and humidity sensor and an electronic circuit for determining the weather from the output of the sensor, or via a communication line. There is communication department to get temperature, humidity, weather information of place. For example, in the case of adopting the brightness of the surroundings as the peripheral information, as the peripheral information obtaining unit, there are an illuminance sensor to be built in and an analysis circuit for analyzing the image picked up by the imaging unit to obtain illuminance. For example, when employ | adopting the average color / color distribution of an acquired image as said peripheral information, there exists an analysis circuit which analyzes the image information picked up by the imaging part as said peripheral information acquisition part. For example, when the surrounding sound is adopted as the peripheral information, there is a volume sensor as the peripheral information acquisition unit. For example, when employing a model as the peripheral information, there is an internal memory for recording model information as the peripheral information acquisition unit.

Moreover, as an imaging part, there exists an imaging element, such as CCD and CMOS, and the optical system image-forming on the light receiving surface of the said imaging element, for example.

As the storage unit, for example, a memory element or an HDD (hard disk drive) or the like accessible through a built-in or communication line.

Examples of the analysis basic information constructing unit include a built-in CPU and an analysis basic information constructing program operating on the CPU.

Moreover, as a display part, there exist display apparatuses, such as a liquid crystal and a plasma. In addition, when presenting necessary information by pronunciation, a speaker etc. are mentioned.

1 is a block diagram of an information presentation system according to a first embodiment of the present invention.

2 is a flowchart showing an operation flowchart of an analysis basic information construction unit;

3 is a block diagram of an information presentation system according to a second embodiment of the present invention;

4 is a diagram illustrating a configuration of an information presentation system when the information presentation terminal is configured as a mobile telephone.

Fig. 5 is a diagram showing the configuration of an information presentation system in the case where the information presentation terminal is configured as a mobile telephone using a plurality of servers.

6 is a block diagram of an information presentation system according to a third embodiment of the present invention;

7 is a block diagram of an information presentation system according to a fourth embodiment of the present invention;

8 is a block diagram of an information presentation system according to a fifth embodiment of the present invention;

9 is a diagram for explaining a relationship between an analysis basic information construction unit, a related information acquisition unit, and a second related information acquisition unit;

10 is a diagram showing the configuration of a search system according to a sixth embodiment of the present invention;

11 is a diagram illustrating an example of a target image.

12 is a diagram illustrating an example of a target image.

13 is a diagram illustrating an example of a target image.

14 is a diagram illustrating an example of a target image.

Fig. 15 is a flowchart showing a process of comparing and registering an input target image desired to be registered with an image registered in the TP data management DB.

Fig. 16 is a diagram showing an example of a display method for highlighting and displaying a portion determined to be similar to an image registered in the TP data management DB when segmentation and registration are performed.

FIG. 17 is a diagram showing an example of a display method of displaying, in a list format, an image similar to an image desired to be registered among the images registered in the TP data management DB.

FIG. 18 is a diagram showing an example of a display method of displaying, in a list form, an image similar to an image desired to be registered among the images registered in the TP data management DB. FIG.

19 shows a process of dividing and deleting similar portions sequentially in a target image, and finally leaving a dissimilar portion as a new registration portion when a plurality of images similar to the target image to be registered exist in the TP data management DB. Drawings showing.

20 is a diagram showing a process of dividing and re-registering an image already registered in the TP data management DB.

Fig. 21A is a diagram showing an example of a process of synthesizing similar parts so as to overlap each other when an image having a part similar to a part of the target image to be registered exists in the TP data management DB.

Fig. 21B is a diagram showing an example of a process of synthesizing similar parts so as to overlap each other when an image having a part similar to a part of the target image to be registered exists in the TP data management DB.

Fig. 22 is a diagram showing an example of using matching when registering a moving image in the TP data management DB.

23 is a flowchart showing a recognition element identification process.

Fig. 24 is a diagram explaining an object recognition method according to the twelfth embodiment.

Fig. 25 is a diagram explaining an object recognition method according to the twelfth embodiment.

Fig. 26 is a diagram explaining a system configuration of a matching processing unit and a TP data management DB in the fourteenth embodiment.

Fig. 27 is a flowchart for explaining the flow of image matching processing using the mobile phone with camera according to the fifteenth embodiment of the present invention.

Fig. 28 is a diagram showing an example of the display form of the marker image in the fifteenth embodiment of the present invention.

Fig. 29 is a diagram showing an example of the display form of the marker image in the fifteenth embodiment of the present invention.

<Best Mode for Carrying Out the Invention>

EMBODIMENT OF THE INVENTION Hereinafter, the best form for implementing this invention is demonstrated with reference to drawings.

[First Embodiment]

As shown in FIG. 1, the information presentation system 10 according to the first embodiment of the present invention includes the photographing unit 11, the related information obtaining unit 12, the presenting unit 13, and the surrounding information obtaining unit 14. ), A storage unit 15, and an analysis basic information construction unit 16. The photographing section 11 is an imaging section such as a camera. The related information acquisition unit 12 acquires information related to a predetermined image such as a marker image or a logo image obtained from the photographing unit 11. The presenting unit 13 presents the related information acquired by the related information obtaining unit 12. The peripheral information acquisition unit 14 acquires the peripheral information related to the photographing unit 11. The storage unit 15 stores registration information. The analysis basic information building unit 16 builds analysis basic information from the registration information stored in the storage unit 15 based on the peripheral information acquired by the peripheral information acquisition unit 14.

Here, the peripheral information acquired by the peripheral information acquisition unit 14 includes time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. At least one of the. The analysis basic information building unit 16 changes the analysis basic information based on the surrounding information. That is, as shown in FIG. 2, the said analysis basic information construction part 16 acquires the surrounding information from the said surrounding information acquisition part 14 (step S1). The analysis basic information construction unit 16 also sets various parameters in accordance with the surrounding information (step S2). Simultaneously with this step S2, the analysis basic information construction unit 16 selectively reads registration information from the storage unit 15 (step S3). The analysis basic information construction unit 16 then integrates the set parameter and the read registration information as analysis basic information (step S4). In addition, the analysis basic information construction unit 16 provides the related information acquisition unit 12 with the analysis basic information constructed in the step S4. The related information acquisition unit 12 acquires information related to the image from the image obtained from the photographing unit 11 and the analysis basic information supplied from the analysis basic information construction unit 16.

As a specific example of the information presentation system 10 of such a structure, the following are considered, for example.

At the start-up, the peripheral information acquisition unit 14 acquires the date and time information, and the analysis basic information construction unit 16 acquires the telephone number corresponding to the time from the address book in the storage unit 15, As described above, analysis basic information is constructed. In the analysis basic information construction unit 16, the optimal one can be selected from the information prepared in advance. In addition, the said analysis basic information contains output information. That is, the analysis basic information construction unit 16 changes the output information based on the surrounding information. Here, for example, when the photographing unit 11 photographs an album with a face picture, it is assumed that the application recognizes the face and displays the telephone number. In this case, if it is 9: 00-18: 00 of weekdays, the telephone number of an office is acquired and it is set as analysis basic information. However, if it is after 18:00 on a holiday or weekday, it is considered to obtain a home telephone number and use it as analysis basic information. In addition, if it is after 22:00, it is similarly to acquire a home telephone number, but may add the message "only at the time of emergency", and may use it as analysis basic information.

In addition, the peripheral information acquisition part 14 acquires positional information at the time of starting, and the said analysis basic information construction part 16 is represented by the said positional information among the registration information (signboard) of the memory | storage part 15. Moreover, as shown in FIG. Analysis basic information may be constructed from registration information (signboards) that are 1 km around from the position. In this case, the peripheral information acquisition unit 14 transmits the position information to the radio wave strength between the base stations, and the position information embedded in the marker or the two-dimensional code, for example, latitude and longitude or address, if there is a GPS or a communication device. , Etc. can be determined. With such a configuration, even if a signboard exists in a plurality of regions, it is possible to present information corresponding to the place. For example, when the photographing unit 11 photographs the logo mark of the company A, it is possible to provide an application in which the nearest repair reception window is displayed. In addition, when the photographing unit 11 photographs the signboard of the convenience store B, the current weather and the weather forecast information are acquired based on the location information, for example, "Umbrella is now in XXX yen and still in stock". As described above, an application for presenting promotion information of a corresponding convenience store may be provided. As described above, according to the information presentation system 10 according to the first embodiment, it is possible to present various types of information such as navigation and promotion.

Or at the time of starting, the surrounding information acquisition part 14 acquires the brightness of the surroundings or the brightness | luminance of the image acquired from the imaging | photography part 11, and the analysis basic information construction part 16 based on the acquired brightness information. It is also possible to perform a process such as adjusting a threshold for image processing, adjusting a parameter of the contrast filter, and the like. Thereby, even if the image acquired by the imaging | photography part 11 is a dim image, the marker of a recognition object, a face photograph, a logo mark, etc. can be recognized. For example, when photographing in a dark room, the threshold of binarization is reduced by 10% based on the brightness of the image. By this adjustment, even if the circumference | surroundings of a logo and a photo are slightly dark, they can extract them stably. Similarly, the matching process between the template supplied as the analysis basic information and the logo or photograph as the extracted recognition target can be stably recognized.

In addition, when starting, the peripheral information acquisition part 14 acquires the brightness of the surroundings or the brightness | luminance of the image acquired from the imaging | photography part 11, and when the analysis basic information construction part 16 is dark, illumination is reached. We build analysis basic information from registration information for signboard and shining signboard. Thereby, even if it is a signboard in which a visible method changes when it is dark, it becomes possible to recognize. In addition, since the color of the components in the screen can be adjusted to the color scheme according to the brightness of the surroundings, information can be presented easily.

Second Embodiment

The information presentation system according to the second embodiment of the present invention is divided into an information presentation terminal 10A and a server 10B, as shown in FIG. Then, the information presentation terminal 10A and the server 10B are connected by a communication line. In this case, the information presentation terminal 10A includes the photographing unit 11, the related information obtaining unit 12, and the presenting unit 13. The server 10B includes the peripheral information acquisition unit 14, the storage unit 15, and the analysis basic information construction unit 16.

In addition, when the information presentation terminal 10A is configured as a mobile telephone, the information presentation system according to the second embodiment has a configuration as shown in FIG. 4, for example. Here, the peripheral information acquisition unit 14 of the server 10B not only acquires the sensor or GPS information included in the information presentation terminal 10A as the peripheral information, but also the position information provided by the base station 21 and the like. Environmental information and the like are also acquired as the surrounding information.

Alternatively, when the information presentation terminal 10A is a portable information device having an Internet connection function such as a PDA, the communication line 20 includes the Internet. In this case, the information presentation terminal 10A and the server 10B cannot be said to exist in the same station. Accordingly, the peripheral information acquisition unit 14 of the server 10B may appropriately acquire the date and time information as the peripheral information from the clock included in the information presentation terminal 10A.

In such a configuration, optimal analysis basic information is constructed by the analysis basic information constructing unit 16 on the server 10B when the information presentation terminal 10A is activated or when the analysis basic information is downloaded from the server 10B. do.

For example, according to the model of the mobile telephone as the information presentation terminal 10A, the threshold value of the binarization (image processing) is adjusted on the server 10B to construct analysis basic information. Furthermore, according to the model of the mobile telephone as the information presentation terminal 10A, an image processing filter is set to construct analysis basic information. That is, a plurality of models exist in the mobile phone, and the optical performance of the photographing unit 11 is different for each model. For this reason, it is difficult to perform stable image recognition processing unless an appropriate image processing parameter is set. For example, since a mobile phone has a blurred image depending on the model, it is effective to set a threshold value of binarization as high. In addition, in the model of the mobile phone whose image is blurred, setting the edge emphasis filter as the image processing filter is effective for stable image recognition.

Alternatively, the template size may be adjusted in accordance with the model of the cellular phone. Further, the template size may be adjusted according to the number of registration information incorporated in the analysis basic information and / or the similarity of each other. For example, when the similarity of mutually registered registration information to be incorporated as analysis basic information is equal to or larger than a preset value, the template size for performing template matching is doubled horizontally and the value is included in the analysis basic information. By doing in this way, more accurate matching process can be implement | achieved even in the environment which is easy to recognize. If the similarity of registration information is low, the template size is made small according to the number of registrations. In other words, when the number of registrations is large, in order to shorten the time required for the matching process, an attempt is made to reduce the template size in a range in which misrecognition is not made (a range in which the similarity of mutuality does not exceed the set value). In addition, when the similarity of registration information is low, you may lower the data gradation of a template. For example, the gray scale is usually 256 gradations, but may be reduced to 16 gradations. In addition, when the similarity of registration information is low, you may set a high threshold of matching, in order to avoid misperception.

Note that the server 10B including the peripheral information acquisition unit 14, the storage unit 15, and the analysis basic information construction unit 16 is not limited to one, but may be provided in plural. In the case where the information presentation terminal 10A is a mobile telephone, the information presentation system according to the second embodiment has a configuration as shown in FIG. 5, for example. That is, as shown in FIG. 5, the some server 10B-1, 10B-2, 10B-3, ... becomes an information presentation system of the structure connected to the communication line 20. As shown in FIG. In addition, this communication line 20 is constituted by the base station 21 and the network 22. The servers 10B-1 to 10B-3, ... are connected to the network 22. In such a case, the related information acquisition unit 12 of the information presentation terminal 10A may acquire analysis basic information from the plurality of servers 10B-1, 10B-2, 10B-3, .... In addition, the analysis basic information construction part 16 of one server 10B (for example, server 10B-1) is at least one other than the said server 10B (for example, server 10B-1). Acquires information from the server 10B (for example, the servers 10B-2, 10B-3, ...), and combines the information, and then constructs analysis basic information to be transmitted to the information presentation terminal 10A. You may also do so.

Third Embodiment

The information presentation system according to the third embodiment of the present invention is divided into an information presentation terminal 10A and a server 10B similarly to the second embodiment. Then, the information presentation terminal 10A and the server 10B are connected by a communication line. In the present embodiment, however, as shown in FIG. 6, the information presentation terminal 10A includes the photographing section 11, the related information obtaining section 12, the presenting section 13, and the surrounding information obtaining section. (14) and the analysis basic information constructing unit 16. The server 10B is configured to include only the storage unit 15.

According to this structure, since the registration information acquired from the server 10B is constructed as analysis basic information on the information presentation terminal 10A, once the registration information is downloaded, it is possible to finely correspond to the surrounding information of the information presentation terminal 10A. Done.

[Example 4]

The information presentation system according to the fourth embodiment of the present invention is also divided into the information presentation terminal 10A and the server 10B similarly to the second and third embodiments. Then, the information presentation terminal 10A and the server 10B are connected by a communication line. In the present embodiment, however, as shown in Fig. 7, the information presentation terminal 10A has only the photographing section 11 and the presentation section 13. The server 10B has another related information acquisition unit 12, the peripheral information acquisition unit 14, the storage unit 15, and the analysis basic information construction unit 16. However, as shown in FIG. 7, the peripheral information acquisition unit 14 of the server 10B uses a communication line such as GPS information, time information, brightness, and the like from a sensor or the like of the information presentation terminal 10A. Peripheral information (including peripheral information transmitted from the base station 21) such as an image from the photographing unit 11 for acquiring the information is acquired.

With such a configuration, the user does not need to download the analysis basic information to the information presentation terminal 10A in advance. In addition, the server 10B can perform processing based on the optimal analysis basic information corresponding to the peripheral information of the information presentation terminal 10A which is transmitted together with the captured image.

[Example 5]

The information presentation system according to the fifth embodiment of the present invention is, in the configuration of the second embodiment, as shown in FIG. 8, having a function equivalent to that of the related information acquisition unit 12 in the server 10B. 2 The related information acquisition unit 17 is further provided. Here, the second related information acquisition unit 17 acquires the related information from more analysis basic information than the analysis basic information acquired by the related information acquisition unit 12.

That is, in this embodiment, as shown in FIG. 8, the peripheral information acquisition part 14 of the server 10B is GPS information and time information from the sensor etc. which the information presentation terminal 10A has, etc. via a communication line. Furthermore, peripheral information (including peripheral information transmitted from the base station 21) such as an image from the photographing unit 11 for acquiring information such as brightness is acquired. And, as shown in FIG. 9, the analysis basic information building part 16 has high priority analysis basic information based on the surrounding information acquired by the surrounding information acquisition part 14, and lower priority than that. Construct the second analysis basic information. And the said analysis basic information is downloaded to the related information acquisition part 12 of 10 A of information presentation terminals via a communication line. In addition, the second analysis basic information is supplied to the second related information acquisition unit 17.

By setting it as such a structure, even if the related information acquisition part 12 of the said information presentation terminal 10A was unable to acquire the related information, the result of the said 2nd related information acquisition part 17 is shown in the said presentation part 13. It is possible to present.

For example, when the portable telephone is considered as the information presentation terminal 10A, the portable telephone has a different storage capacity depending on the model. Moreover, the available storage capacity differs according to a user's usage method. Moreover, the processing speed differs depending on the model. Therefore, it is desirable to change the capacity of the analysis basic information to be downloaded for each model so as not to put more burden on the processing on the mobile phone than necessary. Therefore, in this embodiment, analysis basic information which has not been downloaded is maintained on the server 10B. Then, the second related information acquisition unit 17 of the server 10B receives the related information from the second analysis basic information which is the analysis basic information which has not been downloaded to the image or part of the image transmitted from the mobile phone. Acquisition processing is performed. When the relevant information can be obtained on the cellular phone, the information is displayed on the presentation unit 13. On the other hand, if the relevant information could not be obtained on the mobile phone, but could be acquired on the server 10B, the information acquired on the server 10B is presented on the presentation unit 13 of the mobile phone. Alternatively, the processing is also performed on the mobile phone, but even when the information on the server 10B can quickly acquire information, the information acquired on the server 10B is presented on the presentation unit 13 of the mobile phone.

For example, based on the positional information of the mobile phone, a template set of around 500m is acquired as analysis basic information and downloaded to the mobile phone. The user can walk the streets while getting information of nearby shops by this template set. If the user is 500m or more away from the position where the user originally acquired the template set, the corresponding information cannot be obtained from the template acquired in the mobile phone. However, in the present embodiment, the image acquired by the cellular phone is also transmitted to the second related information acquisition unit 17 of the server 10B. In the server 10B, when the user first downloads a template set of around 500m, the template set of around 4 km is transferred to the second related information acquisition unit 17 at the same time. Therefore, even if the user cannot acquire the information from within the cellular phone, the second related information acquisition unit 17 on the server can acquire the information from the template set of around 4 km. Information is acquired on the server 10B, and when the information is not acquired on the cellular phone, the information acquired on the server 10B is displayed. Therefore, the user can use the template data of the area exceeding the capacity of the cellular phone. At this time, a template set of around 500 m based on the current position may be transmitted to the cellular phone. In this way, since information can be acquired again only by a mobile telephone within 500m from the position, a light feeling of operation can be realized.

As described in the above embodiments, the template data to be recognized by the system, information associated with the template data, and various parameters in the recognition process are used based on various information such as environment and use model. By switching the information, it is possible to present the optimum information according to the situation according to the use environment.

Although this invention was demonstrated based on the above Example, this invention is not limited to the Example mentioned above, Of course, various deformation | transformation and application are possible within the scope of the summary of this invention.

For example, in the fifth embodiment, the server 10B includes a second related information acquisition unit 17, and the analysis basic information construction unit 16 constructs the analysis basic information and the second analysis basic information. I did it. However, the configuration is not limited to this configuration, and the third related information acquisition unit, the fourth related information acquisition unit,. The server 10B further comprises a plurality of related information acquiring units, and the analysis basic information constructing unit 16 performs the third analysis basic information, the fourth analysis basic information,. You can also build more.

[Example 6]

As shown in Fig. 10, the retrieval system according to the sixth embodiment of the present invention is composed of a camera-equipped portable terminal 23 such as a camera-equipped mobile phone or a PDA with a camera and a search means. The camera-equipped portable terminal 23 includes an image input unit (camera) 11 for inputting an image and a display 26 for outputting an image of a search result. The retrieval means retrieves an image from a database using a hierarchically managed template based on the image input by the image input unit 24. In this case, the searching means is realized by a portable application software 25 of the portable terminal 23 with a camera and a matching processing unit 29 configured in the server 27 that can communicate with the portable terminal 23 with a camera. do.

The server 27 further has a template (TP) data management database (DB) 28 in which a plurality of templates are registered and hierarchically managed them. The template registered in this TP data management DB 28 is the TP creation part 50 from the target image 41 arrange | positioned on the page 40 by desktop publishing (DTP) 30, as shown in FIG. )

That is, in the retrieval system according to the sixth embodiment, the target image 41 is printed on the paper sheet 40 by the DTP 30 in advance, and the target image 41 is printed by the TP creating unit 50. Create a template. The created template is registered in the TP data management DB 28 of the server 27. If there are many target images 41 to be registered, the creation and registration of such a template is repeated.

Then, when the user who wants to search for the target image 41 is acquired from the page 40 by using the image input unit 24 of the portable terminal 23 with a camera, the portable application software 25 uses the input image. Feature extraction of the image is performed. The portable application software 25 then sends the extracted feature data to the matching processing unit 29 of the server 27. The matching processing unit 29 then matches the template registered in the TP data management DB 28. In this case, following the template matching using the template of the upper layer, template matching using the template of the lower layer is performed to narrow the search image. When the matching result is obtained, the matching processing unit 29 sends the matching result information to the portable application software 25 of the portable terminal 23 with a camera. Portable application software 25 displays the result information on display 26.

Moreover, when using the target image with which the registration area | region, ie, the area | region as a recognition object, is clear like the said target image 41 shown in FIG. 10 as a target image, there are the following advantages. That is, when registering the target image 41 in the TP data management DB 28, the similarity evaluation between the target image to be registered and the image already registered in the TP data management DB 28 is performed. For example, when the evaluation result that the similarity between the two images (the numerical value indicating the similar degree) is 75% or more in matching, the target image 41 is not registered in the TP data management DB 28. It is possible to prevent re-registering the target image similar to the image registered in the TP data management DB 28. Thereby, misperception in said matching can be stably avoided. In addition, although the threshold of judgment of whether to perform a registration process is made 75% here conveniently, the threshold is not limited to 75%. Here, it is preferable to set the threshold value of the similarity judgment in matching as high as possible within the range of the numerical value which does not generate misperception in the use environment of the matching. In addition, the threshold value is not only statically set as an element of the system, but also the matching conditions (for example, the tilt of the camera at the time of shooting, the stability of the illumination, the number of pixels captured by the camera, the type of information used for the matching, It may be set dynamically according to the method of matching).

In addition, during the registration of the target image 41 in the TP data management DB 28, the target image 41 may be registered after appropriate modification (for example, addition of noise or the like) to the target image 41. By applying such a suitable deformation, the target image 41 comes close to the state obtained by photographing with an actual camera. This works effectively when evaluating the similarity. That is, the accuracy of the similarity evaluation is increased by evaluating the similarity not only for the image before applying the appropriate deformation but also for the image after applying the appropriate deformation.

Moreover, as said suitable deformation | transformation, various image processing operations, such as rotation, three-dimensional affine transformation, expansion, reduction, color information deterioration, provision of white noise, simulation of projection by various lighting conditions, are mentioned as an example, for example. Can be.

The processing in the case where an image similar to the image desired to be registered exists in the TP data management DB 28 is not limited to the processing for rejecting the registration as described above. In other words, for example, the user may be notified that an image similar to the image desired to be registered exists in the TP data management DB 28, and the user may be allowed to select whether or not to register. In addition, this processing is one of the effective processings when giving the same information with respect to mutually similar images.

It is also effective in the actual system to selectively present the above-described processing for rejecting the registration and the selection processing of the registration by the user in accordance with individual situations. That is, for example, suppose that a plurality of accounts having registration authority exist in the TP data management DB 28. At this time, generally, a user cannot see information related to accounts other than himself / herself. In such a system, for example, when account A wants to register image C with the TP data management DB, the image C considers the case where account B is similar to an image that has already been registered with the TP data management DB 28. In such a case, it is reasonable that the matching processing unit 29 rejects the registration of the image C because the similar image is found in the TP data management DB 28. Further, for example, consider a case where the account B wants to register the image C again in the TP data management DB 28. In such a case, the matching processing unit 29 displays the information of the image C already registered by the account B, and performs processing of leaving the user to decide whether or not to register, such as "This image is similar, but should it be registered?" It is reasonable to do. In this way, it is important for the actual service to perform the most rational processing according to the situation so that the user of the system can avoid the risk of misrecognition.

In addition, in such a matching system, the image used as the said target image 41 is not limited to area | region division in the form of a clear quadrangular structure like the said target image 41 shown in FIG. As the target image 41, of course, the target image 41 may be configured as shown in FIGS. 11 to 14. That is, the target image 41A (see FIGS. 11 and 12) having an arbitrary shape of a circle or curve, and the target images 41B and 41C not including the frame itself (see FIGS. 13 and 14, respectively). ) And various images, such as the target image 41C (see FIG. 14) in which a specific logo is placed in the vicinity of the specific design, can be used as the target image.

Here, when using the same image as the target image 41B and 41C which does not restrict the area | region which recognition and registration are subject to as a target frame, the following matching is performed. First, a plurality of feature points are extracted from the input image. Then, the feature point group is compared and compared with the feature point group registered in the TP data management DB 28 for each object in advance. By this comparison comparison, matching is performed whether or not they are the same target image.

In actual matching, when the value indicating similarity is a value exceeding a preset threshold value based on the difference in feature amounts between the feature points, the matching processing unit 29 is similar to the feature points. Judging from the feature point. For example, the matching processor 29 determines that a predetermined number of feature points among the objects registered in the TP data management DB 28 are similar to those of the input image. Is judged by

Here, the feature point refers to a point where the difference in feature amount with other pixels in a given image is equal to or higher than a predetermined level. For example, as said difference, difference, such as contrast of a contrast, a color, distribution of surrounding pixels, differential component value, or arrangement | positioning of feature points, is mentioned. In addition, in the case of using matching by comparison of the feature point group, the feature point is extracted for each object in advance, and registered in the TP data management DB 28 for each object.

When performing matching based on feature points as described above, when an image similar to the input image is found in the group of images registered in the TP data management DB 28, "deny of registration" or "similarity is accepted. In addition to the option of "register afterwards", the option of "subdivide and register the target image and / or registered image to be registered" is also considered. According to the option of "dividing and registering the target image and / or the registered image to be registered" with this option, after minimizing the number of images which cannot be registered, a database which can avoid misperception can be constructed. .

In the case where the above-mentioned option of "the target image and / or registered image which are desired to register and / or registered image is divided | segmented" is considered, for example, the said TP creation part 50 is based on the flowchart shown in FIG. The target image inputted for registration is matched with the comparison image registered in the TP data management DB 28. In addition, the flowchart shown in FIG. 15 has shown the operation control by the said TP preparation part 50 only in the case of "it divides and registers the target image and / or registered image which you want to register" for convenience of description. .

First, an input image which is an image desired to be registered input by the user and an image registered in the TP data management DB 28 (hereinafter referred to as a registration image) are compared (step S10). Subsequently, it is determined whether the input image and the registered image are similar images (step S11). When the step S11 branches to NO, the input image is registered in the TP data management DB 28 (step S12). In addition, in this step S12, as shown, for example in FIG. 16, you may let a user decide whether to register the said input image in the said TP data management DB28. In this case, registration is performed only when the user selects that the input image is registered in the TP data management DB 28.

By the way, when branching said step S11 to YES, the part similar to the said registration image in the said input image is determined, and the similar part is divided and deleted (step S13). In other words, this step S13 is a step of creating an image in which a portion similar to the registered image in the input image is deleted.

After completion of the step S13, the process returns to the step S10. In this way, the series of processes of the step S10, the step S11, and the step S13 are repeatedly performed to create an image obtained by dividing and deleting all portions similar to the registered image from the input image.

In addition, when presenting an image created by division and deletion to a user, a presentation method as shown in FIG. 16 is considered. That is, it is a presentation method which presents both images of an input image and the registration image judged to be similar to the input image, and specifies the part used for similar determination in both images. This illustration is effective in increasing the efficiency of registration. 17 and 18, the registration image determined to be similar to the input image may be presented to the user by displaying it in a list format.

That is, as shown in FIG. 17 and FIG. 18, the display screen is divided into three into an input image display area 251, a list display area 253, and a selected image display area 255. When a registered image is selected as shown by hatching in the list display area 253, the selected image is displayed on the selected image display area 255. FIG. Moreover, in the input image displayed on the said input image display area 251, identification display (hatching part in FIG. 17 and FIG. 18) with a registration image is performed.

By the way, when a plurality of registration images similar to the input image to be registered exist in the TP data management DB 28, as shown in FIG. 19, the similar portions are sequentially divided and deleted. Then, the last remaining part is used as an input image for newly registering.

In the above, the process which divides and deletes the input image which wants to register was demonstrated. However, it is not necessarily necessary to divide and delete only the input image. That is, of course, the process of dividing and deleting the said registration image may be performed. Furthermore, as shown in FIG. 20, you may divide | segment both and register each image as follows. First, since the logo called "ABCDEFGH" included in the input image 300A is similar to the logo called "ABC" included in the registered image 301A, "ABC" in the input image 300A is selected. The image is divided and deleted into an input image 300B. After that, since the logo called "DEFGH" in the input image 300B is similar to "GHIJ" included in the registered image 301B, the image 301B1 including the registered image 301B only with "GH". ) And an image 301B2 composed only of "IJ". Then, both images of the image 301B consisting only of "GH" and the image 301B2 consisting only of "IJ" are re-registered in the TP data management DB 28. The input image 300B is also registered in the TP data management DB 28 as the image 300C after the division.

According to the method of dividing and registering an image as described above, for example, a very advantageous embodiment is realized on an actual service as described below.

For example, in a product by the same maker or a product of the same brand, generally a brand logo peculiar to a product by the maker or a product of the brand in the package of the product concerned and the product original product name or icon Etc. (hereinafter, generic names, icons, etc. are collectively referred to as trade names) are printed. In this case, according to this embodiment, the brand logo and the product name are divided and registered in the TP data management DB 28. This produces the following effects.

For example, even when the image in which the brand logo and the product name "A" are described is already registered in the TP data management DB 28 as "product A", the same brand logo as the corresponding brand logo and the above-mentioned A The image in which the product name is described is not rejected because it is an image similar to the "product A". That is, according to the present embodiment, " merchandise A " which is a registered image is divided into the brand logo and the product name A. Then, the brand logo and the product name A are respectively re-registered. Specifically, the image in which the brand logo is described is registered as a "brand logo", and the image in which the product name A is described is registered as "the name of the product A".

In the sixth embodiment, the concept of "combination information" is introduced. This combination information is the following information. For example, consider a case where a combination of "brand logo" and "name of product A" is registered as "product A". In this case, the information indicating that the combination of "brand logo" and "name of goods A" is "goods A" is "combination information". In addition, in this case, when the combination information registers the image which shows the "brand logo" and the image which shows "the name of the goods A" in the said TP data management DB 28, the said TP data management DB ( 28).

In addition, when dividing an input image and / or a registered image to be registered, it is not necessary to register an image resulting from the division. For example, in a photograph of a used car or a product, each photograph has a similar background. Here, for example, consider a case where the car A is an image as it is with the background and is registered in the TP data management DB 28. In this case, in the case where it is desired to register the picture on which the car B has a background similar to the picture on which the car A is taken in the TP data management DB 28, according to the sixth embodiment, the background is divided and Can be extracted. In such a case, if the user does not particularly feel the necessity of registration, it may be of course possible to deviate from the object of registering the background part. In this way, the user, that is, the information provider, can efficiently register only the portion to be matched with the TP data management DB 28. Therefore, it is possible to avoid the occurrence of misperception due to the similarity in the background portion between the input image and the registered image.

The "information represented by the image" may also be managed in correspondence with the combination of the divided images as described above. Here, the information represented by the image may include, for example, a URL to which the image corresponds, a resource ID inside the application, music information, location information on a map, and the like. For example, in the example shown in FIG. 20, the image 301B containing "GHIJ" is the image 301B1 containing "GH" and the image 301B2 containing "IJ" as mentioned above. Although divided and registered, the information expressed by the image may be given to the set of the image 301B1 including "GH" and the image 301B2 containing "IJ". Similarly, in the example shown in FIG. 20, the information expressed by the image may be given to the image 300A including both "ABC", "GH", and "DEF".

That is, the information represented by the image is managed as the logical sum of the images including the predetermined logo, and the information expressed by the image is given only to the image satisfying the predetermined logical condition. According to this study, for example, even when a portion similar to the registered image does not exist in the input image as a result of the division of the input image, the TP is provided after providing information represented by the image as a set of predetermined registered images. It is also possible to register in the data management DB 28.

It is also possible, of course, to use not only logical sums, but also more complex logical expressions. For example, "GH" in the input image 300 shown in FIG. 20 is called an essential element (element in logical sum for giving the information which an image represents with respect to an arbitrary image). By doing so, a technique such as "GH" AND ("ABC" OR "DEF") is possible as a logical expression for managing this. In this way, the information expressed by the image can be managed by a free logical expression on the basis of the divided image.

For example, by using an essential element as a symbol indicating a target image to be photographed, it is possible to specify that the image is a target image to the user, and stable matching can be realized. In addition, the importance of the feature points of the area around the essential elements may be weighted to be higher than the other feature points, and in the matching process, similarity may be determined by adding the weighting. By doing in this way, a more stable and efficient matching process can be realized.

It is of course also possible to use a NOT expression as a logical expression. For example, when logo A is a registered image, a person who newly registers logo B describes the logo A, which is a registered image, and the logo B that he or she registers in a NOT relation, so that logo A and the logo are matched at the time of matching. You can prevent B from being recognized at the same time. Using the logical expression of NOT in this way is effective when the logo A and the logo B are each used by a company or the like that compete with each other. In addition, such contention relationship information is registered in the system in advance, and upon registration of the logo, the system automatically uses a logo that is used by a company or the like that has a contention relationship with a predetermined logo based on the contention relationship information. Of course, it may be described as a relationship and registered. Here, it is a database of various designs used by the competitive relationship information, a company in a competitive relationship in any field, and the like.

In the case where the logical sum is used in this manner, when a portion not included in the registered image is included in the input image 300A of the input image 300A in the example shown in FIG. 20, determination at the time of matching becomes easy. .

It goes without saying that the relationship between the images by various logical expressions as described above can be described and constructed in the image division process. In the sixth embodiment, matching is performed by the server 20 in cooperation with the application 25 on the mobile terminal 23, but the application 25 includes so-called e-mail. Of course. In other words, the user can of course perform matching by attaching an input image and sending it to the server by e-mail. In this case, the matching processing result is distributed to the user by electronic mail.

In addition, the process which can be taken when the registration image similar to the input image which wants to register is found is not limited to the process which divides the above-mentioned input image and / or registration image. That is, the following processing is also considered.

For example, as shown in FIG. 21A, when it is determined that the input image and the registered image are similar, a process of synthesizing the input image and the registered image is also considered by overlapping the similar portions. In addition, as shown in FIG. 21B, the case where matching is performed using the image 402 including a part of "B" and a part of "C" as an input image is considered. Here, before the matching, the input image 400 including "A" and "B" and the registration image 401 including the input image containing "A" and "C" are synthesized. If these synthesized images are registered in the TP data management DB 28, the matching can be performed more accurately. In other words, this study can return a highly accurate matching result to the user.

When dividing an input image and / or a registered image, each image element generated by the division is given a meaning such as a classification element or an individual element to perform hierarchical image recognition of the system. It is also possible to operate as a system. In addition, the TP data management DB 28 having various pieces of information on division can be effectively used in the determination at the time of matching by the above-described logical expressions based on the image elements generated by each division. have.

[Example 7]

An embodiment in the case where a moving picture is assumed as a template to be created by a TP creating unit that creates a template image used for matching processing will be described below.

The moving image is composed of, for example, a continuous still image of 30 sheets / second. Therefore, when a moving picture is registered as a TP in the TP data management DB, which is a means for recording a template created by the TP creating unit, the moving picture is treated as a continuous image of a still picture. However, it is not necessary to register still images of all commas among still images constituting a moving image. In other words, once the still image of any coma is registered in the TP data management DB, the still image of the coma is only displayed when the still image of the coma succeeds meets a predetermined criterion as described below. Add it again to the data management DB.

First, using the similarity as the constant criterion is considered. In this case, the similarity between the still image of the coma registered in the TP data management DB and the still image of the coma subsequent thereto is evaluated, and only when the similarity is equal to or less than a predetermined value, the still image of the coma is stored in the TP data. You can add it again to the management DB. As the predetermined value of the similarity, the similarity may be set to 75% or less, for example.

Second, it is considered to use the number of the feature points as the constant criterion. That is, if the number of feature points found in the registered still picture decreases to less than an arbitrary threshold value in the still picture following the registered still picture, it is a method of additional registration.

Third, it is considered to use the moving distance of the feature point as the constant reference. That is, if the feature point found in the registered still image is traced in the still image subsequent to it, and the moving distance of the feature point becomes more than a threshold value, what is necessary is just to register further.

According to the above method, even if the moving image is composed of 120 still images, only six images are actually registered, so that the original scene at any position within the moving image can be stably matched. The inventor confirmed.

A still image registered as a moving image may not only handle each still image independently but also include the sequence information of each still image in the moving image and register it in the TP data management DB. In this way, by using the order information also in matching, more stable matching is realized, and misrecognition can be avoided.

Hereinafter, with reference to one structural example of the object recognition system shown in FIG. 22, it demonstrates concretely. When the portable terminal 23 with a camera and the TP data management DB 28 in the server are connected by the line of a video telephone, for example, the image input part (not shown) in the portable terminal 23 with a camera. ) Is continuously transmitted to the TP data management DB 28 at the server. When the matching processing unit (not shown) in the server recognizes the still picture B in the continuously transmitted still picture, it is still registered in the registered moving picture (TP data management DB 28) corresponding to the moving picture. Moving picture) cannot be determined, but since the recognized images are E and G, the moving picture 400 can be determined.

In addition, the configuration of recognizing a moving image is not limited to a method that can continuously transmit an image like a video telephone. For example, the mobile terminal with a camera 23 captures a moving image, and the image is transferred from an application to a server ( 27). Of course, the application may be a so-called electronic mail. In this case, the user transmits the data required for matching by sending a mail with a moving picture to a specific mail address. The result of the matching is distributed to the user by mail. The method of dividing and registering the image described in the sixth embodiment is, of course, applicable to the case of a moving image.

[Example 8]

By the way, in image recognition using feature points, for example, matching processing, the larger the number of feature points extracted from an image, the easier it is to recognize an image. However, the larger the number of feature points, the more time is required for the matching process. For this reason, extracting only effective feature points and registering only the feature points in the matching process realizes an effective matching process.

Generally, a feature point can be extracted by performing various image processing with respect to the input image to register. Examples of this image processing include processing that gives a blur effect to the image, processing that changes the size of the image, and the like. The feature points extracted in this way are distinctly different from other pixels locally in the feature amount. However, it cannot be prevented that a plurality of points having similar feature amounts exist in the same image.

Here, the case where a plurality of points having similar feature amounts exist in the image Z is considered. In this case, in matching between the image Z and the image picked up the image Z, it is determined that the feature points extracted from the image Z are similar to the plurality of points in the image Z. For this reason, it becomes difficult to implement stable matching. As such, similar feature points included in the same image inhibit stable matching and slow down the processing speed of matching.

In addition, a plurality of similar feature points included in such a same image cannot be easily excluded except when the feature amounts are completely the same. This is because, in the same image, it is difficult to set threshold values for determining similarity based on the validity of the matching.

In view of the above circumstances, in the eighth embodiment, at the time of registration, simulation of photographing is performed on the target image, and the validity of each feature point is evaluated in matching with the target image itself. This selects the feature points to register and effectively realizes stable matching by the minimum number of feature points.

Simulation of the photographing may include two-dimensional and / or three-dimensional affine transformation including rotation and enlargement and reduction, a process of imparting noise and / or variation in photographing, a process of imparting noise of image compression in a photographing size, Includes various processes that can be realized by general image processing programs, such as level manipulation processing that reflects the lighting environment, contrast manipulation, gamma curve manipulation, color information deterioration manipulation, and color overlay manipulation. do.

The TP preparation unit 50 for creating a template used for the matching process is configured of, for example, a feature point extraction unit, a simulation image group preparation unit, a matching processing unit, and a feature point evaluation unit. The detail of the function of each part is as follows.

The feature point extracting unit extracts feature points from the target image to be registered.

The said simulation image group creation part performs the said simulation to the said target image, and produces | generates at least 1 simulation image. The simulation image group creating unit extracts feature points of these simulation image groups.

The matching processing unit compares the feature point extracted by the feature point extraction unit with the feature point extracted by the simulation image group creating unit. Here, in the comparison of the feature points, the feature points extracted by the feature point extraction unit may be treated as registered in the TP data management DB 28, and matching may be performed at the feature points extracted from each simulation image. Note that the feature points extracted by the simulation image group creating unit may be treated as registered in the TP data management DB 28, and the matching may be performed on the feature points extracted by the feature point extraction unit.

In the feature point evaluator, the amount contributed to the actual match is associated with the feature points extracted by the feature point extractor with reference to the matching processing result by the matching processor. The contribution amount is treated as the quality of the feature point, and the feature point is ranked. Further, only feature points having a quality equal to or greater than an arbitrary threshold value and / or a predetermined number of feature points above are selected, and the feature points are referred to as feature points of the target image to be registered.

As described above, by selecting the feature points to be registered, it is possible to preferentially register the points effective for the actual matching process. Therefore, the processing time of matching can be shortened and a result with high precision can be obtained.

[Example 9]

In the ninth embodiment, a plurality of feature points are extracted from the input image, and the same object is identified by comparing and comparing the feature point group formed from those feature points with the feature point group for each object registered in advance.

The feature point in the image here indicates that the difference from other pixels is at a certain level or more, and the feature amounts include, for example, contrast of light and dark, color, distribution of surrounding pixels, differential component values, and arrangement of feature points. Etc. can be mentioned. In the ninth embodiment, the feature points are extracted and registered for each object. At the time of actual identification, the feature points are searched out in the input image, and compared with the data registered in advance.

The flow of operation control of identification processing in the matching processing unit in the ninth embodiment will be described below with reference to FIG. First, the feature point data of the recognition element of the target object Z registered in advance is read out from the feature point data storage unit in which the feature point group is recorded (step S21). Subsequently, the feature point data is input to the feature point comparison unit for comparing the feature point data (step S22). Then, the feature point comparison section compares and compares the feature point data with the input feature point data (step S23).

Then, in the identity determination part which judges the identity of objects, the identification of the said object Z and the input object is performed (step S24). Then, it is determined whether or not the number of coincident feature points is equal to or greater than a predetermined value (here, referred to as X) (step S25). If this step S25 branches off, it will return to said step S23. On the other hand, when branching said step S25 to YES, it is determined whether the recognition element of the object Z currently compared with the input object is the same (step S27).

Thereafter, it is determined whether or not the comparison with respect to all the recognition elements has been completed (step S28). In the case where this step S28 branches to NO, the feature point data in the feature point group of the next recognition element is input as comparison data to the feature point comparison unit (step S26), and the flow returns to the step S23. By the way, in the case where the step S28 branches to YES, it is determined whether the number of matching feature points is equal to or greater than a predetermined value (herein referred to as Y) (step S29). Here, in the case where this step S29 branches to YES, it is determined that the input object and the object Z coincide, and the user is displayed on the display unit or the like to inform the user (step S30). On the other hand, when branching said step S29 to "no", it is determined that the input object and the object Z do not correspond (step S31).

In addition, in the case of actual identification, when the numerical value (the difference of each component between feature points) which shows similar degree exceeds the preset threshold value, it determines with the feature point as a similar feature point. Then, the object to which the plurality of feature points coincide is determined to be the same as the object of the input image. At this time, the feature point group in the input image and the feature point group registered in advance are compared as follows.

First, the object is divided into a plurality of elements and registered. This makes it possible to recognize by the determination logic that the object is not recognized if a plurality of elements (for example, three) are not recognized when comparing the objects.

Second, in the case of recognition of an object, when a similar object is photographed in an image, for example, S company having OBJ1 (feature point group A, B, C) as its logo mark, and OBJ2 (feature point group; E, We assume M company making F, G) its logo mark. Here, S company and M company are said to compete with each other. In this case, of course, confusion of both logo marks should be avoided as much as possible. In view of such circumstances, in the ninth embodiment, when A and E are simultaneously detected from within the same screen, neither of the objects is recognized. That is, the recognition judgment is made strictly.

Third, conventionally, even when the number of feature points that can be recognized is the same, the sentence expression that delivers the recognition result to the user is the same. For this reason, for example, when only some feature points can be recognized, that is, when the degree of matching between the input image and the comparison image is a degree of matching including uncertainty, the effect cannot be transmitted to the user. On the other hand, in the ninth embodiment, when the number of recognition elements is small, the display method (expression method) of the result is changed to an expression including uncertainty.

By each of the above studies, the following effects are obtained, respectively.

First, it is possible to reduce the probability of causing a misunderstanding by matching only a part of the object.

Second, the criterion for judging the misrecognition of the object in particular can be strictly enforced.

Third, even when the accuracy of the identity determination of the object is lower than the predetermined value, after the user has been alerted, the user can be informed of the result of the coincidence determination.

By the way, in the case of OBJ1 (feature point group; A, B, C) and OBJ2 (feature point group; E, F, G) which are the registered targets, the feature points in the object are divided and recognized by the following decision logic. .

First, it is not called OBJ1 recognition success unless it is A and B and C.

That is, when OBJ1 consisting of A, B, and C, which are recognition elements, is recognized, the recognition of OBJ1 is not successful in one or two recognition states.

In addition, as this modification, A, B, and C are weighted as evaluation points. For example, the weights are 1, 0.5, and 0.3, respectively. Here, if authentication is made when the total evaluation point exceeds 1.5, the total evaluation point becomes 1.5 when A and B are found as recognition elements, so the OBJ1 is authenticated. On the other hand, when B and C are found, OBJ1 does not authenticate.

The evaluation points of these recognition elements can be managed together with the feature points of the recognition elements.

It is also possible to change the priority of each element as a logical expression, and besides A and B and C, for example, a combination such as A and (B or C) or A or (B and C) is possible. These examples are examples in which A is an essential element in order to make recognition successful.

In addition, the said evaluation point and the example of a logical formula can be used in combination. That is, the priority of a logical expression and the weighting of each element can be combined and used.

Second, when E and A are extracted, it is never said that both OBJ1 and OBJ2 are recognized.

For example, if S company using OBJ1 as a logo and M company using OBJ2 as a competition are in a competitive relationship and want to avoid the confusion of both, S company's logo OBJ1 and M company's logo OBJ2 are in the same screen. If so, neither logo is recognized. In this case, the reason that the user cannot recognize the object is not that the target image is not detected, but the recognition elements are detected from both (A, B, C) and (E, F, G). It is displayed to the effect that there is.

As described above, in the ninth embodiment, regarding identification of logos of companies and the like that compete with each other, for example, only OBJ1, which is the logo of S company, or OBJ2, which is the logo of M company, only one of them is displayed on the screen. Recognize the logo only when it is in an arbitrary state. Specifically, OBJ1 or OBJ2 is recognized only when any one of (A, B, C) or only one of (E, F, G) is detected within the same screen. In other words, when either one of (A, B, C) and one of (E, F, G) are detected in the same screen, neither OBJ1 nor OBJ2 are recognized.

Third, if only a few, such as A and B, are extracted, change the way the results are presented (observe expressions that include uncertainty).

For example, regarding the recognition of OBJ1, when all of A, B, and C of the recognition element can be recognized, the recognition result is presented to the user with a strong expression "OBJ1 has been authenticated". In addition, when two recognition elements such as A, B, B, and C of the recognition elements can be recognized, the recognition result is presented to the user in a slightly weakened expression, for example, "we think it is OBJ1." do. If there was only one element of recognition that could be recognized, the recognition result is presented to the user in an expression including uncertainty so that "OBJ1 may have been recognized."

Further, as a modification of the ninth embodiment, when the above-described weighted evaluation points are used, the above-described studies in the expression method when presenting the recognition result based on the total evaluation points to the user are also shown. Is considered. In addition, the study of the expression method as described above when presenting the recognition result to the user is, of course, applicable to various scenes. For example, it is also applicable to the recognition of a desired recognition element group. Further, the expression method as described above when presenting the recognition result to the user can be applied, for example, by the number of matching feature points in the recognition element, and the degree of matching between the extracted feature points and the registered feature points.

In addition, in the case where the ninth embodiment is applied to the sixth embodiment, the TP creating unit 50 may be operating on the server 20. In addition, the paper 40 means a display surface, and is not necessarily limited to paper. For example, other materials, such as a metal and plastic, may be sufficient, and a video display apparatus, such as a liquid crystal monitor and a plasma television, may be sufficient. Naturally, the information displayed on them corresponds to that displayed in the visible light region in humans. However, as long as the information can be inputted to the image input unit, the information may be invisible to humans. In addition, since all that can be acquired as an image are objects, an image called an X-ray image or a thermography may be sufficient, for example.

10, the image which grasped the target image input from the said image input part 24 is transmitted to the matching processing part 29 in the server 20 from the portable terminal with a camera. At this time, the image obtained by the image input unit 24 may be transmitted as image data as it is, and of course, the image may be reduced and transmitted. It is also possible to extract feature data used for matching from the image and transmit it. Furthermore, it goes without saying that both the image and the feature data are transmitted. That is, as long as it is data of the aspect which can be derived from the said image, you may transmit data of what kind of aspect.

[Example 10]

In the sixth to ninth embodiments described above, the server includes an image storage unit that stores a registered image for comparison and comparison with an input image.

As the data used for the comparison and verification, the feature point group data extracted in advance or at the time of image input is usually used. For this reason, the information stored in the memory | storage part which stores the data used for comparison collation is also usually characteristic point group data. In the comparison and contrasting operation, the similarity is determined in consideration of the similarity of the data and the like. In such a case, when there are two very similar images in the data used for the comparison collation, there is a case where it is difficult to determine the similarity of the input images. In that case, if there exists an image for comparison and contrast (original data from which a feature point group was extracted; hereafter called an original image), the comparison reference of a feature amount is changed and a comparative precision with high precision is performed, or the feature point group is extracted. The user can visually check the result using the original image. In addition, the image itself may have a smaller capacity than the feature point group data extracted from the image.

In view of the above circumstances, as in the tenth embodiment, it is advantageous to assume an image as data to be used for comparison and to have the server store an image storage unit relating to the image. In addition, whenever a comparison collation operation of an image occurs, the feature point group may be extracted from the image used for comparison collation.

[Example 11]

In the eleventh embodiment, in the sixth to ninth embodiments described above, the server indicates, as comparative contrast data for the feature point group in the input image, a place where the feature point group and an image corresponding to the feature point group are recorded. An embodiment having a location storage section for indicating location information.

For example, consider the case where the image or the feature point group extracted from the image is recorded in the image recording section for recording the image as data for comparison and comparison with the feature point group in the input image. In such a case, there is a case in which the size of data used for comparison and comparison, which is a feature point group extracted from these images and images, is large, and it is not advantageous to maintain them as a system. In such a case, considering the improvement of the overall performance, it is considered that the image recording unit for recording the image is not provided in the system.

In view of the above circumstances, when the image determined to be the same is needed from the processing result of matching, the server has a location storage unit for recording location information for searching for the image. Here, the location information is, for example, link information to the image in a system different from the system, link information to the image on the network, a keyword for extracting the image on the network, and the like. For example, search information. Such location information is recorded and managed in the location storage unit as location information indicating a relationship with a feature point group registered for comparison and verification. In the eleventh embodiment, the above location information storage unit is provided in the system.

[Twelfth Example]

In the twelfth embodiment, the sixth to eleventh embodiments described above are characterized in that the feature point group for comparison and contrast with the input image adds three-dimensional arrangement information in the photographing object.

That is, in the twelfth embodiment, three-dimensional arrangement information in the photographing object is introduced. Here, the three-dimensional arrangement information refers to the coordinate value of each feature point on the so-called object local coordinate system. Hereinafter, an object recognition method according to the twelfth embodiment will be described with reference to FIGS. 24 and 25. Here, in FIG. 24 and FIG. 25, the point which attaches (1)-(11) has shown the characteristic point, respectively. These feature points are the feature points extracted in the input image. In addition, as shown in FIG. 24, among these characteristic points, the characteristic points (1)-(5) have shown the tree to be photographed on the left side, and the characteristic points (6) to (11) have shown the characteristic point in the house which is taken to the right, respectively. FIG. 25 is a diagram showing only the feature points shown in FIG. 24. In addition, these feature points are registered beforehand as the feature point group of the said imaging object in the database which registers a feature point group.

Here, as the feature point in the tree, at the time of registration, with respect to the feature points (2) to (5), the feature point (1) is the origin and the three-dimensional position, that is, each of the three-dimensional arrangement information, namely (X2, Y2). , Z2) to (X5, Y5, Z5) are also registered. As a method of detecting the three-dimensional position of such a feature point, there is an image detection method or a detection method using a measuring device.

According to the twelfth embodiment, as described above, the three-dimensional arrangement information of each feature point is simultaneously registered at the time of registration of the feature point, whereby the following effects can be obtained. That is, after a predetermined recognition is performed by comparison of feature points in matching or the like, re-verification can be further performed using the three-dimensional arrangement information. That is, according to the twelfth embodiment, it is possible to realize the reduction of misperception in matching.

In addition, various methods can be considered as a verification method for re-validation using the three-dimensional arrangement information. For example, based on the three-dimensional arrangement information, a comparison is performed between the position of each feature point when projecting each feature point on a two-dimensional plane with a predetermined feature point as the origin and the position of each feature point acquired and registered as usual. By calculating the deviation between the two, the accuracy of the recognition in the matching is obtained.

According to this method, for example, feature points in a group of feature points recognized as part of a tree in an input image are verified in order, and each point is the origin, and two-dimensional is based on the three-dimensional arrangement information of other feature points. It is verified whether the position of each feature point (1)-(5) at the time of projecting on a plane corresponds to the registered feature point (1)-(5). When the feature point 1 is the origin, when the feature point 2 is the origin,... This is done for all the feature points. And the thing at the time of verification in the case of the highest agreement is employ | adopted.

By the verification method described above, the positional relationship between the image input device (for example, a camera) and the object to be photographed can be obtained. Furthermore, by obtaining the lens parameters (viewing angle, focal length) of the image input device in advance, it is also possible to obtain the distance between each feature point on the object to be photographed and the image input device.

[Example 13]

In the thirteenth embodiment, in the sixth to twelfth embodiments described above, a plurality of feature point groups are extracted from the input image, and each feature point group is individually compared and compared in matching between the input image and the registered image. In addition to the recognition, the recognition is performed by adding the arrangement information which is the information on the positional relationship between the feature point groups.

A feature point group is extracted from the input image, and matching is performed by comparison with the feature point group for comparison and comparison. According to the thirteenth embodiment, it is shown in Fig. 24 from the arrangement relationship between the feature points (1) to (5) belonging to the feature point group in the tree and the feature points (6) to (11) belonging to the feature point group in the house. In the example, it can be recognized that the tree is arranged on the left side of the house within the screen.

According to the thirteenth embodiment, the following arrangement states can be determined by recognizing the arrangement relationship of the feature points as described above. For example, in the example shown in FIG. 24, the image where a tree is located to the left and a house to the right is an image picked up by the camera etc. which face the front of a house. In addition, the image where a tree and a house are located in substantially the same position on a screen is an image image | photographed by the camera etc. which are located in the direction in which a tree and a house are arrange | positioned in substantially straight line, ie, the inclination direction with respect to the front of a house.

Moreover, it is also possible to calculate the front-rear relationship between a tree and a house based on the expanded state of arrangement | positioning of a characteristic point group and a characteristic point. In addition, when these objects are in operation, it is possible to obtain the arrangement state at the present time.

If the arrangement relationship of the feature point group described above is registered in accordance with the scene, the object can be identified even in a case where the same object is arranged and positioned. Specifically, for example, in a case in which houses of the same design are arranged, the case in which the tree is on the left side and the house on the right side can be recognized as Mr. A's house, and vice versa. In addition, three or more feature point groups are included in the same screen, and the three-dimensional position information of the feature point groups is also recorded in advance, and according to the principle of triangulation, the relative positional relationship between the camera and the feature point group is determined. It can be obtained in detail.

[Example 14]

The fourteenth embodiment is an embodiment in which the matching processing unit, which is the processing unit that actually performs matching, is clustered in order to speed up the matching system or to realize a matching system corresponding to a large number of registered images.

In a conventional matching system, the method of parallelizing a matching processing unit as it is, and the method of parallelizing a database and a matching processing unit as it is, are common as long as the database which registers the image etc. used for matching is common. In this way, the effect of speeding up the process when receiving a large number of matching requests at the same time can be expected. However, it is not possible to speed up the matching process for a large number of registered images. In addition, by dividing and storing a large number of registered images into a plurality of parallelized databases, it is possible to reliably speed up the matching process for a large number of registered images. However, in the final evaluation of the matching result of the divided group of registered images, the image used for the matching is not necessarily the image which comes to a higher position after uniformly evaluating the entire group of divided images. Therefore, it is hard to say that it is a highly reliable matching process.

In the fourteenth embodiment, as to the group of images registered in a plurality of divided and parallelized databases (referred to as TP data information DBs), similar images are prevented from being registered in one TP data information DB. Adjust it together.

At the time of registering the image in the TP data information DB, the feature amount of the image to be registered is extracted. In addition, the shade may be a constant area shade or the feature point may be a feature point.

Next, a matching process is performed between the image to be registered and the pre-registered images of the plurality of divided and parallelized TP data information DBs to evaluate the similarity between the image to be registered and the pre-registered image. The matching processing unit may also be divided and parallelized in the same manner as in the TP data information DB.

Subsequently, with reference to the similarity evaluation, the image to be registered is registered in the TP data information DB which has been determined to have the lowest similarity with the image to be registered.

Further, as described above, the image to be registered may not be registered in the form of addition to the divided preregistered group of images, but of course, the entire group of previously registered images including the target of registration may be subdivided. In addition, the image to be registered may be registered once in the form of addition to the previously registered group of divided images, and thereafter, the whole image group may be periodically re-divided. In the re-division, similar images may be most dispersed and stored in each TP data information DB based on the similarity. For example, when a feature point is represented by a multidimensional vector, it is effective to consider the dispersion in the multidimensional vector space of the feature point in the said repartitioning.

In this manner, by selecting the TP data information DB for registering the target image to be registered appropriately, highly reliable matching can be realized.

A system configuration of the matching processing unit and the TP data information DB in the fourteenth embodiment will be described below with reference to FIG.

At the time of matching, first, the data to be matched is transmitted to the first matching processing unit 500. Next, the first matching processing unit 500 requests matching processing in parallel to the second matching processing units 521A to 521C corresponding to the parallelized TP data information DBs 520A to 520C.

Next, the second matching processing units 521A to 521C perform matching processing with the matching target data on the corresponding TP data information DBs 522A to 522C.

Next, the second matching processing units 521A to 521C transmit the matching result that is the result of the matching process to the first matching processing unit 500. Here, the matching result includes similarity information. The matching result may be only the highest matching result, a predetermined predetermined number of higher matching results, or a matching result having a similarity or higher than a predetermined rate. Here, the predetermined constant number is not limited to the number statically set in the system. For example, the number of data to be matched by the first matching processing unit 500 may be divided by the number of parallelized TP data information DBs. In this case, when the number of parallelizations of the TP data information DB is changed, the predetermined constant number is changed dynamically.

However, after the second matching processing units 521A to 521C transmit the matching result that is the result of the matching process to the first matching processing unit 500, the first matching processing unit 500 performs a second matching processing unit ( Matching results received from 521A to 521C are ordered in the order of similarity, and final results are created. In addition, the first matching processing unit 500 may of course perform matching processing again on the matching result received from the second matching processing units 521A to 521C. In this case, using a more detailed feature amount than the matching processing performed by the second matching processing units 521A to 521C is effective in improving reliability.

For example, in the second matching processing units 521A to 521C, the tone matching data having a small number of divisions of the area and the tone data having low gradation are used for the matching process, and the first matching processing unit 500 has a large number of divisions of the area. The tone data and the tone data with many gradations are used for the matching process. The second matching processing units 521A to 521C may use all the feature points in the first matching processing unit 500 by using some feature points held by the TP data information DBs 522A to 522C. In this case, the method for selecting some of the feature points may be selected based on the reliability and / or the validity of the feature points, or may be a certain number of times or more than a certain number of times. In addition, as a matching result by the 1st matching process part 500, the one determined as the most similar may be used as a matching result, and a plurality or more of arbitrary criteria may be used as a matching result. Here, the arbitrary criteria may be the number of results, for example, or may be a threshold of similarity.

In addition, in the configuration in which the TP data information DBs are divided and parallelized as the TP data information DBs 522A to 522C, the server cases themselves do not necessarily need to be parallelized.

By performing the matching process as described above, the divided and parallelized TP data information DB in the fourteenth embodiment can be effectively used.

[Example 15]

Further, for example, the following embodiments are also considered. In the fifteenth embodiment, a mobile telephone with a camera is assumed as the information presentation terminal. In the mobile phone with a camera, a part of feature amount processing such as extraction of a feature amount from an image is performed. To realize this, application software relating to the feature amount processing of the image is installed in advance in the mobile phone with camera. Hereinafter, the flow of image matching processing using the mobile phone with camera according to the fifteenth embodiment will be described with reference to the flowchart shown in FIG. 27.

First, the user starts the application software on a mobile phone with a camera, and photographs, for example, a predetermined image or a design on the ground using a camera function (step S51). Subsequently, some feature amount processing (in this embodiment, extraction of the feature amount from the image) is executed by the application in the camera-equipped cellular phone (step S52). The feature amount processing result (the feature amount extracted from the image in the present embodiment; the same applies hereinafter) is sent to the server in step S52.

The matching processing unit (not shown) in the server that has received the feature amount processing result is stored in the dictionary image database 50 in which the feature amounts of the images (comparative images) to be compared are registered based on the feature amount processing results. Inquiry is made to match the image (step S53). Then, the matching processing result is sent to the mobile phone with camera. On the server side, in addition to the dictionary image database 50, there are provided a multi-stage and parallel database 60 composed of a plurality of databases composed of multiple stages and arranged in parallel with each other. Therefore, when the comparison image required for the matching process is not stored in the dictionary image database 50, the matching processing unit (not shown) searches the multi-stage and parallel database 60.

Then, in the mobile phone with camera that has obtained the result of the matching processing in step S53, the operation instruction set in advance with respect to the matching processing result is executed (step S54). Here, for example, if the operation command is a URL display of a web site for a mobile phone, the matching processing unit (not shown) sends the URL to a web site browsing application of the mobile phone with a camera, and carries the camera with the camera. The phone executes display of the WEB site corresponding to the URL (step S54).

In addition, in the fifteenth embodiment, the feature amount extraction from the image is performed by the camera-equipped cellular phone. However, the server extracts the feature amount from the image by the computing capability of the camera-equipped mobile phone. Of course it is.

In addition, in order to indicate to the user that the image as an object to be photographed by the mobile phone with the camera is, for example, a marker image, it is carved (arranged near a specific chairman). May be arranged in the vicinity of the image. Here, FIG. 28 has shown the example which arrange | positioned one marking 72 which consists of an arrow which points to a marker image in the vicinity of the marker image 70. As shown in FIG. In addition, FIG. 29 has shown the example which arrange | positioned two marking 74 in the vicinity of the marker image 70 so that the said marker image 70 may be interposed.

Claims (20)

An imaging unit 11 for obtaining an image, A peripheral information acquisition unit 14 for acquiring peripheral information, A storage unit 15 that stores registration information, An analysis basic information building unit 16 for constructing analysis basic information from the registered information stored in the storage unit 15 based on the peripheral information acquired by the peripheral information acquisition unit 14; A related information acquisition unit 12 for acquiring information related to the image from an image obtained from the imaging unit 11 and analysis basic information obtained from the analysis basic information construction unit 16; Presenting unit 13 for presenting the relevant information acquired by the related information acquisition unit 12. And The analysis basic information construction unit 16 is based on at least one of time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. An information presentation system characterized by changing the analysis basic information. The method of claim 1, The imaging unit 11, the related information acquisition unit 12, and the presentation unit 13 are configured in an information presentation terminal 10A, The peripheral information acquisition unit 14, the storage unit 15, and the analysis basic information construction unit 16 are configured in the server 10B, The information presentation terminal (10A) is connected to the server (10B) by a communication line (20). The method of claim 2, The information presentation system (12) of the information presentation terminal (10A) obtains analysis basic information from a plurality of servers (10B). The method of claim 2, The analysis basic information construction unit 16 of the server 10B acquires information from at least one server 10B other than the server, combines the information, and then transmits the information to the information presentation terminal 10A. An information presentation system characterized by further constructing analysis basic information. The method of claim 1, The imaging unit 11, the related information acquisition unit 12, the peripheral information acquisition unit 14, the analysis basic information construction unit 16, and the presentation unit 13 are connected to an information presentation terminal 10A. Composed, The storage unit 15 is configured in the server 10B, The information presentation terminal (10A) is connected to the server (10B) by a communication line (20). The method of claim 1, The imaging section 11 and the presentation section 13 are configured in an information presentation terminal 10A, The related information acquisition unit 12, the peripheral information acquisition unit 14, the analysis basic information construction unit 16, and the storage unit 15 are configured in a server, The information presentation terminal (10A) is connected to the server (10B) by a communication line (20). The method of claim 1, The imaging unit 11, the related information acquisition unit 12, and the presentation unit 13 are configured in an information presentation terminal 10A, The peripheral information acquisition unit 14, the analysis basic information construction unit 16, and the storage unit 15 are configured in the server 10B, The information presentation terminal 10A is connected to the server 10B by a communication line 20, The server 10B further has a second related information acquiring section 17 having the same function as the related information acquiring section 12, The second related information acquiring unit 17 acquires the related information from more analysis basic information than the analysis basic information acquired by the related information acquiring unit 12, and acquires the related information from the related information acquiring unit 12. The presenting unit (13) presents the result of the second related information acquiring unit (17) even if it cannot. The method of claim 1, And the analysis basic information includes at least one of an image processing filter, a template size, a template gradation, a binarization threshold, and a matching threshold. The method of claim 1, The analysis basic information construction unit (16) sets an image processing parameter included in the analysis basic information based on the similarity between registration information called analysis basic information. An imaging unit 11 for obtaining an image, A peripheral information acquisition unit 14 for acquiring peripheral information related to the imaging unit 11; An analysis basic information constructing unit 16 for constructing analysis basic information based on the surrounding information acquired by the peripheral information obtaining unit 14 from the registered information downloaded from the server via the communication line 20; A related information acquisition unit 12 for acquiring information related to the image from an image obtained from the imaging unit 11 and analysis basic information obtained from the analysis basic information construction unit 16; Presenting unit 13 for presenting the relevant information acquired by the related information acquisition unit 12. And The analysis basic information construction unit 16 is based on at least one of time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. An information presentation terminal characterized by changing the analysis basic information. Peripheral information acquisition unit 14 for acquiring images obtained from imaging unit 11 transmitted from information presentation terminal 10A via communication line 20 and / or peripheral information related to imaging unit 11; , A storage unit 15 that stores registration information, An analysis basic information building unit 16 for constructing analysis basic information from the registered information stored in the storage unit 15 on the basis of the peripheral information acquired by the peripheral information acquisition unit 14; And The analysis basic information construction unit 16 is based on at least one of time, date, season, weather, temperature, humidity, place, ambient brightness, average color / color distribution of the acquired image, ambient sound, and model. Change the interpretation basic information, By downloading the analysis basic information constructed by the analysis basic information construction unit 16 to the information presentation terminal 10A, the analysis basic information obtained from the analysis basic information and the imaging unit 11 is obtained in the information presentation terminal 10A. A server characterized by obtaining information related to the image from an image and presenting it. The method of claim 11, A second related information acquiring section 17 having a function equivalent to that of the related information acquiring section 12, The second related information acquiring unit 17 acquires the related information from more analysis basic information than the analysis basic information acquired by the related information acquiring unit 12, and acquires the related information from the related information acquiring unit 12. The information presentation terminal (10A) presents the result of the second related information acquisition unit (17) even if it cannot. Step (step S12) of recording the feature points of the plurality of objects in the database in advance, step (step S52) of extracting the feature points from the input image desired to be registered, and the extracted feature points of the plurality of objects previously recorded in the database. The feature point in the input image and the object recorded in the database with reference to the comparison and contrast result of the step of comparing and comparing the feature point (Step S10; Step S23) and the comparison and matching step (Step S10; Step S23). An object having a step (step S24; step S53) for recognizing an object by identifying a feature point of the object, and a step (step S30) for displaying the identification result at the step (step S24; step S53) for performing the recognition. As a recognition method, In the step (step S121) of recording the feature points of the plurality of objects in advance in the database, the object is divided into a plurality of element objects, and the feature points of each element object are registered in the database and the recognition is performed (step S121). S24; in step S53, when the predetermined element object is not recognized, the object is not identified. The method of claim 13, In the step of performing the recognition (step S24; step S53), if the recognized combination of the plurality of element objects is a predetermined combination, the object recognition method is not performed. The method of claim 13, In the step of performing the recognition (step S24; step S53), the predetermined combination is not satisfied, but when at least one or more of the element objects are recognized, the display result (step S30) identifies the identification result as an indeterminate element. The object recognition method characterized by displaying to a user in the expression included. The method of claim 13, In the step (step S12) of recording the feature points of the plurality of objects in advance in the database, the image itself of the object is also recorded in the database. The method of claim 13, In step (Step S121) of recording the feature points of the plurality of objects in advance in the database, location information indicating a location where the image corresponding to the feature point is recorded is recorded in the database together with the feature points. Object recognition method. The method of claim 13, In the step of extracting the feature point (step S52), the feature point is extracted from the two-dimensional image of the object. The method of claim 13, In the step of extracting the feature point (step S52), the feature point is calculated based on three-dimensional arrangement information indicating a three-dimensional positional relationship of the object. The method of claim 13, When a feature point group consisting of a plurality of feature points is extracted in the step of extracting the feature points (step S52), in the step of comparing and comparing (step S10; step S23), not only the recognition of each feature point group but also the position of the feature point groups And performing the above comparison in consideration of feature point arrangement information which is relation.

Images