KR100915756B1 - Information presenting apparatus and information presenting terminal - Google Patents

Abstract

Recognition and identification unit 12 for recognizing and identifying individual markers within the shooting range from the image photographed by the imaging unit 11 having a predetermined shooting range, and identifying by the recognition and identification unit 12. An information designation unit 13 for designating predetermined information according to the information of the individual markers, and a time-delayed image in which the information designated by the information designation unit 13 is associated with the image photographed by the imaging unit 11 The information display device is configured by the image display unit 14 for generating the information, and the information on the individual markers for identification by the recognition and identification unit 12 is separate from the information display device 10. To be supplied as data set 21.

Description

Information timing device and information timing terminal {INFORMATION PRESENTING APPARATUS AND INFORMATION PRESENTING TERMINAL}

The present invention relates to an information presenting apparatus and an information presenting terminal for presenting information.

Background Art Conventionally, various systems have been proposed for recognizing a photographed target image (marker image) and thereby presenting information related to the target image.

For example, Japanese Patent Laid-Open No. 2003-244488 discloses an image information adding system including an image information adding device, a marking means, and an information storing means. The image information adding means includes optical information means and calculation means. The optical information means obtains an angle of view from the focal length at the time of photographing obtained from the information of the image input means and the image cropping size. The calculation means calculates the photographer's position and the shooting direction based on the obtained angle of view. In addition, the said marking means has a pair of pattern. The information storage means stores a background image in the same direction as the direction in which the marking means is picked up and the accompanying information accompanying it. In this image information adding system, first, a photographer's position and a photographing direction are calculated from a photographed image of a pair of patterns. Subsequently, the background image information obtained from the information storage means is processed based on these, and the background image information of the same visual field is acquired, and the incidental information is acquired. Then, the background image and the accompanying information of the same visual field are displayed.

Further, Japanese Laid-Open Patent Publication No. 2003-15980 discloses a technique for finding a corresponding URL from a server using an image of a logo taken. In other words, in this technique, first, a logo mark included in an advertisement printed on a magazine or the like on a mobile device is photographed. Next, the photographed logo mark is sent to the search server via the Internet. The search server searches for the logo mark closest to the logo mark sent over the Internet. The search server then sends the URL corresponding to the searched logo mark to the portable device via the Internet. In this way, the portable device can access the URL and browse the home page.

Both of the techniques disclosed in Japanese Patent Application Laid-Open No. 2003-244488 and Japanese Patent Application Laid-Open No. 2003-15980 have reference information of a target image in an information presenting apparatus. However, neither treats the reference information as a set. Therefore, it is not possible to respond extensively to the object of recognition of the information presenting apparatus. Also, the corresponding reference information cannot be easily exchanged.

<Start of invention>

SUMMARY OF THE INVENTION The present invention has been made in view of the above point, and it is an object of the present invention to provide an information presenting apparatus and an information presenting terminal that can cope with a wide range of recognition targets of an information presenting apparatus and that can easily exchange corresponding reference information. The purpose.

One aspect of the information presentation device of the present invention,

An identification unit for recognizing and identifying individual markers within the shooting range from an image picked up by the shooting unit having a predetermined shooting range;

A designation unit for designating predetermined information according to the information of the individual markers identified by the identification unit;

A regular image generator for generating a regular image in which information specified by the designation unit is associated with an image captured by the photographing unit

And

The information on the individual markers for identification by the identification section is supplied as a set of data files separate from the apparatus.

Moreover, one aspect of the information timing terminal of this invention is

As an information presenting terminal having an imaging unit having a predetermined shooting range, an information presenting device, a storage device, and a display unit,

The information timing device,

An identification unit for recognizing and identifying individual markers within the shooting range from the image picked up by the shooting unit;

A designation unit for designating predetermined information according to the information of the individual markers identified by the identification unit;

A regular image generation unit for generating a regular image in which information specified by the designation unit is associated with an image photographed by the image capturing unit and displaying the image on the display unit

And

Reference information on individual markers for identification by the identification section is supplied from the storage device as a data file set separate from the information presentation device.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing the configuration of an information presentation system to which the information presentation device according to the first embodiment of the present invention is applied.

Fig. 2 is a diagram showing an operation flowchart of the information presenting apparatus according to the first embodiment.

3 is a diagram illustrating a configuration of an information presentation system to which a modification of the information presentation device according to the first embodiment is applied.

Fig. 4 is a diagram showing the configuration of an information presenting system to which the information presenting apparatus according to the second embodiment of the present invention is applied.

5 is a view for explaining a modification of the information presenting apparatus according to the second embodiment.

6 is a diagram illustrating a configuration of an information presenting system to which another modification of the information presenting apparatus according to the second embodiment is applied.

FIG. 7 is a view showing an operation flowchart of the mobile telephone to which the configuration of FIG. 6 is applied.

FIG. 8 is a diagram showing a modification of the configuration of an information timing system to which the information timing apparatus according to the first embodiment of the present invention is applied. FIG.

The information presenting apparatus recognizes a predetermined marker, which is a target image, from an image acquired by a photographing unit such as a camera, and compares it with reference information (hereinafter, referred to as a template) for recognition matching introduced into a separate system in advance. By identifying, information (various forms such as text, graphics, and moving images) corresponding to the predetermined marker is obtained. The present invention aims to improve the convenience of such an information presenting device.

In such an information timing apparatus, until now, a template corresponding to a marker has been read in a predetermined position in the application software and then operated. However, this is an experiment that does not assume a scene in which many markers exist in the first place. Therefore, it is not practical. That is, it is thought that the information presenting device increases the usage value made by the user by holding a large number of templates that the device refers to at the time of marker recognition as in a so-called dictionary. On the other hand, when there are a large number of templates (for example, more than 10,000 levels) in the apparatus, the processing time for comparing and matching (matching) the marker and the template in the acquired image is greatly increased. Based on the experiments and reviews of the inventors of the present invention, it was concluded that suppressing the number of data to be referred to about 100 to 1000 optimizes the user's convenience and processing speed. Moreover, of course, the processing speed and the display speed depend on the platform on which the application software of the information presenting apparatus is installed. Here, the main platforms of the present invention are mainly studied as a mobile phone with a camera, a PDA (portable information terminal) with a camera, and a portable personal computer (notebook PC or tablet PC) as an application range. As described above, the inventors treated the template in the form of a so-called data set in which a certain number was bound, and invented the following system to improve the practical convenience of the user. That is, in addition to the application software of the information presenting apparatus, the data set is stored, and the data set is read in a batch during the startup or operation of the information presenting apparatus. By doing in this way, the data referenced at the time of marker recognition can be introduced.

EMBODIMENT OF THE INVENTION Hereinafter, the best form for implementing this invention is demonstrated with reference to drawings. <First Embodiment>

1 is a diagram showing the configuration of an information presenting system to which the information presenting apparatus according to the first embodiment of the present invention is applied.

This information presentation system comprises an information presenting device 10, which is an information presenting terminal according to the first embodiment of the present invention, a storage unit 20, a data set server 30, and an information server 40. As shown in FIG. The information presentation device 10 is composed of platform hardware. The storage unit 20 is provided in the platform. The data server 30 and the information server 40 are configured in a site accessible by the platform hardware.

Here, the information display device 10 includes a photographing unit 11, a recognition and identification unit 12, an information designation unit 13, a regular image generating unit 14, and an image display device 15. It consists of. The recognition and identification unit 12, the information designation unit 13, and the on-time image generating unit 14 are realized by application software of the information display apparatus set in the platform hardware.

In addition, the said imaging part 11 and the image display apparatus 15 may be equipped with platform hardware as a physical structure, or may be connected externally. Therefore, the recognition and identification unit 12, the information designation unit 13, and the on-time image generating unit 14 may be referred to as an information presenting device. However, in the present embodiment, it is taken as an apparatus for performing the shooting from the image to the final image on the final image, and the photographing portion 11, the recognition and identification portion 12, the information designation portion 13, the regular image generation portion 14 And the image display device 15 are referred to as an information presenting device.

Here, the photographing unit 11 is a camera or the like having a predetermined photographing range. The recognition and identification unit 12 recognizes and identifies individual markers within the shooting range from the image picked up by the shooting unit 11. The information designation unit 13 acquires predetermined information (display content) from the information server 40 in accordance with the information of the individual markers identified by the recognition and identification unit 12. The information designation unit 13 designates the predetermined information as related information. The on-time image generator 14 generates a on-time image in which the relevant information specified by the information designation unit 13 is associated with the image photographed by the photographing unit. The image display device 15 is a display such as a liquid crystal that displays a regular image generated by the regular image generation unit 14.

In the storage unit 20 in the platform, the data set 21 is stored from the data set server 30 via a communication unit or a storage medium (not shown). However, the introduction (downloading or media exchange) and the storage of the data set 21 can be performed regardless of whether the information presentation device 10 is started or before.

In this configuration, as shown in FIG. 2, the information presenting apparatus 10 first acquires an image by the photographing unit 11 (step S1). Next, the recognition and identification part 12 extracts predetermined markers, such as a square frame, with respect to the image acquired by said step S1 (step S2). Subsequently, the recognition and identification unit 12 reads in the data set 21 read from the storage unit 20 in the platform with respect to the image of the marker extracted in the step S2 (for example, an image in a rectangular frame). Comparison and identification are performed based on the reference data. In this way, the recognition and identification section 12 detects a matching marker image. And when the recognition and identification part 12 detects the matched marker image (step S3), the information to acquire from the corresponding data of the data set 21 again by the information designation part 13 which is the next step. The material and / or acquisition method is read out and executed (step S4). In general, information is obtained by accessing the information server 40 existing outside the network or the like through communication or the like from the platform. And the regular image generation part 14 processes the information acquired by the said information designation part 13 (not shown) so that it can be displayed on the image display apparatus 15 in the inside or the outside of a platform, and produces a regular image. . The image display device 15 displays information by sending the generated regular image to the image display device 15 from the image display unit 14 (step S6). Here, in some cases, it is also advantageous information presentation to generate a regular image which superimposes the information acquired above on the original image acquired by the imaging | photography part 11, and to transmit it to the image display apparatus 15. FIG. Therefore, the user can select how to perform the timing.

In addition, as shown in FIG. 3, a position and attitude calculation unit 16 may be provided between the recognition and identification unit 12 and the information designation unit 13. Here, the position and attitude calculation part 16 photographs by inverting from the inclination and the angle of view of the individual markers within the said photographing range identified by the said recognition and identification part 12, as shown by step S5 in FIG. The position 11 and the posture relationship between the unit 11 and the target marker are determined. The on-time image generating unit 14 is configured to calculate the position of the relevant information specified by the information specifying unit 13 in the information specifying unit 13 with respect to the image photographed by the photographing unit 11. On-time images which are superimposed and displayed in a posture are generated. In other words, by using the position and posture information, the on-time image generating unit 14 can generate the on-time image to properly display and superimpose the information image or the information texture on the photographed original image. In other words, the information presentation effect is improved.

In addition, although not shown in FIG. 1 and FIG. 3, when the storage capacity of a platform is large, the following is also possible. That is, when introducing the data set 21 from the data set server 30, the information server 40 communicates with the data set server 30, and the information (display content) corresponding to the introduced data set server 30 is introduced. ) Is introduced into the storage unit 20 in the platform in advance. In this way, the operation efficiency of the information presenting apparatus 10 can be improved.

Here, a description will be given of the case where the mobile phone with a camera is used as a platform in the first embodiment. A mobile phone is basically a device used personally. In recent years, most of the models can be introduced from the Internet site (hereinafter, abbreviated as "portable site") accessible by the mobile phone (installation by so-called download). This information presentation device 10 also presupposes the mobile telephone of such a model basically. In other words, the application software of the information presentation device 10 is installed in the storage unit 20 of the cellular phone. The data set 21 is also stored in the storage unit 20 of the mobile phone as appropriate through communication from the data set server 30 connected to a specific mobile site (not shown). In particular, as the usage range of this information presentation device 10 in a mobile telephone, the following usage methods are mentioned, for example. In other words, a photo existing in a magazine or newspaper is designated in advance in a marker, and a data set is prepared. The user reads a marker from the page of the publication into the mobile phone, and reads information related to the marker from the mobile site. In such a case, it is impossible to keep all the photographs, icons, illustrations, etc. of all publications as reference data. Therefore, it is practical to provide reference data in a specific use range and the like. That is, what is necessary is just to supply to a user by the arranging method called "the data set for referencing the photo published in the month-month issue of a specific magazine as a marker." In this way, the user side is also easy to use, and as described above, when one or more reference images are included in one data set, the storage unit 20 of the cellular phone is sufficiently collected and the recognition and identification processing time is also several seconds. Ends within In addition, a special device and processing are not necessary for the photograph, illustration, etc. of the printed matter side used for this information display device 10. However, in view of facilitating the recognition on the information presenting device 10 side, it is preferable to consider a border around a picture or a layout in which the contrast with the background becomes high.

According to the first embodiment of the present invention as described above, it is possible for the user to easily introduce a plurality of pieces of data of the range to be used into the information display device 10 and supply the data set. It is possible to realize services that are easy to prepare and easy to provide commercially.

In addition, when the function of calculating a position and a posture is also provided, the information acquired from the information server 40 can be displayed on an original image as an appropriate position and posture. In other words, it increases the user's information acquisition effect.

Second Embodiment

Next, a second embodiment of the present invention will be described.

4 is a diagram showing the configuration of an information presenting system to which the information presenting apparatus according to the second embodiment of the present invention is applied. The basic configuration and operation of this information timing system are similar to those of the first embodiment described above. The reference data can be handled in units of sets in the information presenting device 10, as described above, to increase the convenience of the user and to realistically supply the data set.

However, when the information providing apparatus 10 is widely used and the data set is supplied from a large variety of operators, it is preferable to perform the following. In other words, the high availability (hereinafter, referred to as basic data 17) for the user of the information display device 10 is not supplied separately as the data set 21, and a certain data set 21 is not supplied. It is preferable to make it available even if it selects. For example, the markers associated with the index information of the data set 21 itself, the markers most frequently used, and the like are excluded from the data set 21, and those points in the application software of the information display apparatus 10. Only resident reference data is valid. In other words, in the second embodiment, the data set 21 combines sets according to a user's purpose of use or a corresponding publication or object, and is supplied separately from the application software. However, reference data, etc., for markers that are particularly frequently used or have a high need, however, reside or be retained in the application software itself as basic data 17.

Here again, the case where the mobile phone is a platform will be described. For example, it is most practical to download the normal data set 21 through communication from a predetermined portable site. However, it is convenient for the mobile phone user to be able to guide and search on the index site (page on the mobile site) of the data set 21 at this time. In the case of accessing the site itself, the dedicated marker is read into the information display device 10 and made accessible by passing the URL to the site to the browsing software. No preparation is required. That is, the reference data corresponding to the marker in the application software is present as the basic data 17. In this case, a specific illustration, a logo, etc. may be used as a marker, and the plain square etc. which were obtained so that it can be obtained anywhere are set as a marker.

Alternatively, instead of residing or retaining the basic data 17 in the application software itself, as shown in FIG. 5, in the supplied data set 21, the basic data 17 is always included in any one of the data sets 21. At least one set of the same data file (in the drawing, template data "A") may be included.

That is, as described above, when the information presentation device 10 actually operates, the user introduces an arbitrary data set 21. At least one of the above-described basic data 17 is included in any one of the data sets 21, and it can always correspond to a marker having a high frequency of use or a high necessity. For example, as shown in Fig. 5, a plurality of data sets 21 (data sets 1 to n) are prepared, and one or more sets of data sets 21 are stored in the platform storage unit 20. Consider the case of introduction and preservation in the In this case, even if any data set 21 is selected, one or more kinds of basic data 17 are always included. Therefore, the user can image the basic marker and perform the basic operation without any special consideration. Although it is repeated, basic operation is "access to index page of data set" or "access to support window to the information timing device 10 supplier" or "access to weather information site" in predetermined area, Most other users want this behavior. That is, a basic operation is defined as an operation with a high frequency of use by a user.

In addition, as shown in FIG. 6, when the information timing device 10 is activated, the data set server 30 is connected, and the basic data 17 is always downloaded and put into another data set 21, or At the same time, reference may be made.

This configuration provides a method of introducing the basic data 17 which is effective when the data set 21 is separately supplied, in particular in the form of being downloaded from the data set server 30 via a network. That is, when the data set 21 is supplied to the information display device 10 through the network by the configuration shown in FIG. 6, the user selects the data set 21 and downloads it from the data set server 30. In addition to the data set 21, the basic data 17 can also be automatically downloaded simultaneously. In addition, when the basic data 17 is preserve | saved by the structure shown in FIG. 6 in the memory | storage part 20 of the platform which the information display device 10 which this information has already existed, the basic data 17 is updated. Can be planned.

As a result, the user can always use the basic data 17 in the information presenting device 10 without any special consideration.

For example, in recent years, it has become common to be able to use application software in mobile phones with cameras. It is assumed that the application software having functions except for the photographing unit 11 and the image display device 15 of the information display device 10 is installed on the platform. In this case, as shown in FIG. 7, in using the application software, the data set 21 is browsed through the communication of the cellular phone with a predetermined data set download site (step S11). Next, downloading is performed initially from the data set server 30 (step S12). Subsequently, the data set server 30 determines the necessity of updating the basic data 17 (step S13). That is, if the basic data 17 does not exist in the mobile telephone, it is determined that it is necessary. In addition, even if the basic data 17 already exists in the storage unit 20 in the mobile phone, the version of the basic data 17 is the same as that of the basic data 17 that the data set server 30 intends to provide. If it is older than the version, it is determined that an update is necessary. Subsequently, the basic data 17 is also downloaded in the same manner as in the data set 21 described above (step S14). The downloaded basic data 17 is then stored in the storage unit 20 in the mobile phone (step S15). The downloaded data set 21 is also stored in the storage unit 20 in the mobile phone (step S16). In this way, when the basic data 17 already exists in the storage unit 20 in the mobile phone, the basic data 17 is downloaded and saved after determining the necessity of updating by version comparison.

As described above, the necessity of the data set 21 is stored to the mobile telephone only to the data set 21 according to the needs of the user to some extent, so as not to inconvenience the securing of the marker identification processing speed and the necessity of the user. Make it compatible.

As the usage range of the information display device 10, a design such as a picture or an illustration on a publication such as a newspaper or a magazine is used as a marker, and the camera accesses or accesses the information from the mobile phone to the resulting information. There is an advance in information on-time by superimposing the acquired images. It is also possible to register not only printed matter but also objects and signs in the village as reference data as markers. In this case, it is possible to acquire additional information or the latest information by recognizing them as markers from the cellular phone.

Moreover, as a usage form using a mobile telephone, in the case of a product having a rectangular package such as CD, DV, and the like, the jacket design varies, and therefore, it can be used as a marker. For example, by distributing a data set relating to the jacket group to a user from a store or a separate record company, each jacket can be recognized as a marker by a mobile phone in a CD and / or DVD shop or rental shop. For this reason, for example, a URL is associated with the marker, and in this URL, for example, voice distribution of an important part of a piece of music as a piece of information associated with the marker can be delivered to the mobile phone. In addition, as this binding information, it is also possible to appropriately apply annotations (individual annotations in the jacket photograph) along the jacket surface.

That is, when using as a marker the jacket design of the goods which have a rectangular package, such as CD and DVD, as a use form using a mobile telephone, it is good to do as follows. First, (1) At least a part of the appearance image of the recording medium on which the music is fixed or its packaging is distributed in advance to the cellular phone as marker data. And (2) distributes predetermined music information (voice data or annotation information) related to the fixed music to the cellular phone that accesses the address derived by the marker.

By doing in this way, it is effective also as a promotion on the record company side, and the merit that the trouble of preparation of a city hall etc. can be omitted also at the store side.

In addition, the recognition and identification unit, the information designation unit, the on-time image generating unit, the position and the posture calculating unit described in each of the above embodiments are all realized by a CPU in which the information display apparatus is built and a program running on the CPU. Described. However, the present invention can also be realized in other aspects, such as providing a dedicated circuit.

Moreover, although the realization aspect of an in-platform storage part includes an external data pack, a removable storage medium (for example, flash memory), etc., it is not limited to this.

Although this invention was demonstrated based on the above Example, this invention is not limited to the Example mentioned above, Of course, various deformation | transformation and application are possible within the scope of the summary of this invention.

For example, also in the second embodiment, the position and attitude calculation unit 16 may be provided as in the first embodiment, and related information may be determined based on the calculated position and attitude.

1 and 3 to 6, a replaceable storage medium 50 may be used instead of the data set server 30 and / or the information server 40. As shown in FIG. In this case, the introduction of the data set 21 and the basic data 17 into the platform storage unit 20 means the development of data from the storage medium 50 into the internal memory.

As mentioned above, although this invention was demonstrated based on the 1st Example and the 2nd Example, this invention is not limited to the Example mentioned above, A various deformation | transformation and an application are possible within the scope of the summary of this invention. Of course.

Third Embodiment

For example, the configuration of the information presenting system to which the information presenting apparatus according to the first embodiment of the present invention shown in FIG. 1 is applied can be modified as shown in FIG. That is, FIG. 8 shows the recognition and identification unit 12 and the data set 21 installed in the storage unit 20 in the information presenting apparatus 10 in the first embodiment. Of course, it can be installed on the server side. In addition, when setting it as the information timing system of such a structure, the said storage medium 50 installed in the said storage part 20 does not need, since it does not need.

Claims (15)

An identification unit 12 for recognizing and identifying individual markers within the shooting range from an image picked up by the shooting unit 11 having a predetermined shooting range, A designation unit 13 for designating predetermined information according to the information of the individual markers identified by the identification unit 12, A regular image generating unit 14 for generating a regular image in which information specified by the designation unit 13 is associated with an image captured by the photographing unit 11 And As an information presenting device in which information about individual markers for identification by the identification unit 12 is supplied as a data file set 21 separate from the device 10, At least one pair of basic data 17 for designating the presence of information to be displayed by performing recognition and identification with a marker, independently of the data file set 21, When the recognition and identification with the marker are not possible by the basic data (17), the information presentation device is supplied with the data file set (21). delete delete The method of claim 1, And a data file set (21) according to a result of recognition and identification of the marker by the basic data (17). The method of claim 1, In the supplied data file set (21), at least one set of data files necessarily included in any one data file set (21) is contained. The method of claim 1, When the information presentation device 10 is activated, it is connected to a specific site and always downloads a predetermined data file, puts it into another data file set of the information presentation device 10, or makes reference possible. Information timing device. The method according to any one of claims 1 or 4 to 6, And a position and posture calculating section 16 for calculating a position and a posture of the photographing section 11 from individual marker images within the photographing range, The on-time image generator 14 calculates the information designated by the information designation unit 13 by the position and attitude calculation unit 16 with respect to the image photographed by the photographing unit 11. And an on-time image for generating the on-time image. As an information display terminal provided with the photographing section 11, the information presenting device, the storage device 20, and the display section 15 having a predetermined shooting range, The information timing device, An identification unit 12 for recognizing and identifying individual markers within the shooting range from the image photographed by the shooting unit 11; A designation unit 13 for designating predetermined information according to the information of the individual markers identified by the identification unit 12, The on-time image generation unit 14 which generates the on-time image associated with the image specified by the designation unit 13 and the image captured by the photographing unit 11 and displays it on the display unit 15. And Reference information about individual markers for identification by the identification section 12 is supplied from the storage device 20 as a data file set separate from the information presentation device, Retains at least one set of basic data 17 which designates the presence of information to be displayed by performing recognition and identification with a marker, independently of the data file set, When the recognition and identification with the marker by the basic data (17) is not possible, the information presentation terminal, characterized in that the supply of the data file set (21). The method of claim 8, The data file set (21) stored in the storage device (20) is downloaded from a specific site accessible to the information display terminal in advance or upon activation of the information display apparatus. delete delete The method of claim 8, And a data file set (21) according to a result of recognition and identification of the marker by the basic data (17). The method of claim 8, In the supplied data file set (21), at least one pair of data files necessarily included in any one data file set (21) is contained. The method of claim 8, An information-time terminal, which is connected to a specific site when the information-time device is activated, always downloads a predetermined data file, puts it into another data file set of the information-time device, or makes reference possible at the same time. The method according to any one of claims 8, 9 or 12 to 14, The information timing device, And a position and posture calculating section 16 for calculating a position and a posture of the photographing section 11 from individual marker images within the photographing range, The on-time image generator 14 calculates the information designated by the information designation unit 13 by the position and attitude calculation unit 16 with respect to the image photographed by the photographing unit 11. And an on-time image for generating an on-time image.