JP2007116211A - Method, system, and program for providing content associated information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method, system, and program that can provide associated information with many contents, correctly acquire associated information with contents causing less quality deterioration without a wasteful cost of useless processing such as unnecessary communication processing and produce contents cross-referenced with different associated information items from one and the same content. <P>SOLUTION: A client apparatus receives entry of contents, decides the presence/absence of electronic watermark information from the contents, detects the electronic watermark of the contents, extracts a feature quantity from the contents, transmits the extracted feature quantity to a server apparatus, and the server apparatus receives the feature quantity transmitted from the client apparatus, searches an information storage means on the basis of the feature quantity, and transmits associated information being a result of the searching to the client apparatus. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a content related information providing method, system, and program, and more particularly to a content related information providing method using digital watermark technology and feature quantity extraction technology in content related information acquisition technology for acquiring related information from content. And the program.

  A content related information providing system that provides related information corresponding to content such as images has been proposed.

  Specifically, those using a digital watermark technique or a feature quantity extraction technique are known.

  The digital watermark technology is a technology for embedding information so that it is difficult for humans to perceive content. An example of a content-related providing system using digital watermark technology is shown in FIG. When trying to acquire information related to the content at hand with a terminal such as a mobile phone, the watermark information embedded in the content is detected and sent to the related information providing server. A method is used in which content-related information corresponding to the digital watermark received from is retrieved and returned to the terminal.

  Regarding the digital watermark detection processing from content, a method that can be executed on a portable terminal with relatively low calculation processing capability has been proposed, and quick detection on the terminal is possible. In addition, with the digital watermark technology, not only the detected information but also the accuracy of the detected information, that is, the information related to the reliability, can be obtained at the same time. Can be avoided.

  In addition, it is necessary to change contents for embedding information, but information is embedded while suppressing deterioration of the contents due to embedding as much as possible.

  When digital watermarking is used, multiple pieces of information to be embedded in the content can be selected independently of the content, and it is possible to obtain a plurality of watermarked content associated with different related information based on one content. It is. Accordingly, for example, it is possible to embed information corresponding to a serial number and provide different related information for each number (for example, see Non-Patent Document 1).

On the other hand, the feature amount extraction technique is a technique for extracting unique information inherent in the content according to a specific algorithm without any modification to the target content. For example, in the case of image content, the calculation is based on information obtained from color space distribution and tone change. It is also conceivable to extract a specific object in the image and calculate its feature amount. FIG. 23 shows an example of a content-related information providing system using the feature quantity extraction technique. When trying to acquire information related to the content at hand with a terminal such as a mobile phone, the feature amount of the content is obtained on the terminal and sent to the related information providing server, or the content itself is sent to the related information providing server, A method is used in which a feature amount is obtained on the related information providing server, content related information corresponding to the feature amount is searched from the content related information DB and returned to the terminal on the related information providing server (for example, non-patent literature) 2).
"High-speed digital watermark detection method from analog images using a mobile phone with camera", Takao Nakamura, Atsushi Katayama, Masafumi Yamamuro, Noboru Sone, IEICE Transactions, D-II VOL. J87-D-II No.12 pp.2145-2155, December 2004 "Content identification technology with transformation tolerance" Tomonori Takada, Takehito Abe, Harumi Kawamura, Image Recognition and Understanding Symposium (MIRU2005) IS2-55 July 2005

  However, in the digital watermark technology, the amount of information to be embedded and the quality degradation of the content are in a trade-off relationship. Therefore, if the content quality is important, not much information can be embedded, and content that can provide related information The number was limited. Conversely, if the number of contents that can be handled, that is, the amount of information of watermark information is increased, there is a problem that the quality of the contents is greatly deteriorated.

In addition, when using feature extraction technology, it is not possible to determine on the terminal whether or not related information corresponding to the extracted feature is registered. The cost of communication and time for acquiring related information is high. More specifically,
1) The reliability of feature extraction cannot be obtained;
2) Unlike a digital watermark, even if a feature amount is extracted, it is not guaranteed that information related to the feature amount is registered in the DB;
There is a problem.

  Furthermore, since the same feature amount may be extracted in different images with similar patterns, there is a limit to the identifiable ability, so information that is different from the related information originally associated with the content is provided. There is a risk of end.

  In addition, when the feature amount extraction technique is used, since only one feature amount is essentially obtained from one content, it is impossible to associate different information with the same content.

  The present invention has been made in view of the above points, can provide related information for more content than before, has little quality deterioration, and there is no unnecessary processing cost such as unnecessary communication processing, A content related information providing method, system, and program capable of correctly acquiring related information associated with content, and further capable of generating content associated with different related information from the same content The purpose is to provide.

The present invention (Claim 1) is a content related information providing method in a system composed of a client device and a server device,
The client device
At least a watermark presence / absence determining step for receiving presence of content and determining presence / absence of digital watermark information from the content is performed,
Server device
At least, based on the input information, search the information storage means in which the feature quantity and the related information are stored in association with each other, perform an information search step of providing the related information as a search result to the client device,
On either the client device or the server device
At least one of a watermark detection step for detecting an electronic watermark of the input content and a feature amount extraction step for extracting a feature amount from the input content is performed.

  FIG. 1 is a diagram for explaining the principle of the present invention.

Further, the present invention (Claim 2) is provided in a client device,
A watermark presence / absence determination step (step 1) for accepting input of content and determining presence / absence of digital watermark information from the content;
A watermark detection step (step 2) for detecting a digital watermark of the content;
A feature quantity extraction step (step 3) for extracting feature quantities from the content;
Transmitting the extracted feature quantity to the server device (step 4),
In the server device,
An information receiving step (step 5) for receiving the feature amount transmitted from the client device;
An information storage unit is searched based on the feature amount (step 6), and an information search step (step 7) for transmitting related information as a search result to the client device is performed.
On the client device:
A display step (step 8) for displaying the related information received from the server device is performed.

Further, the present invention (Claim 3) is provided in a client device,
A watermark presence / absence determination step for accepting input of content and determining presence / absence of digital watermark information from the content;
A watermark detection step for detecting a digital watermark of the content;
Sending the content to the server device,
In the server device,
An information receiving step of receiving the previous content transmitted from the client device;
A feature extraction step for extracting a feature from the content;
The information storage means is searched based on the feature amount, and an information search step of transmitting related information as a search result to the client device is performed.
On the client device:
A display step of displaying related information received from the server device is performed.

The present invention (Claim 4) is a watermark presence / absence determination step for accepting input of content and determining presence / absence of digital watermark information from the content in the client device,
Transmitting the input content to the server device, and
In the server device,
An information receiving step for receiving content transmitted from the client device;
A watermark detection step for detecting a digital watermark of the content;
A feature extraction step for extracting a feature from the content;
The information storage means is searched based on the feature amount, and an information search step of transmitting related information as a search result to the client device is performed.
On the client device:
A display step of displaying related information received from the server device is performed.

Further, according to the present invention (Claim 5), in the watermark presence / absence determination step in the client device,
If the presence of the digital watermark cannot be confirmed, the subsequent steps are not executed assuming that the feature quantity corresponding to the content is not registered in the information storage unit in the server device.

Further, the present invention (Claim 6), in the search step,
The information storage means storing the feature quantity of the content and the related information corresponding to the set of the digital watermark embedded in the content, the digital watermark detected in the digital watermark detection step, and the feature extracted in the feature quantity extraction step Search based on information that combines quantity.

Further, according to the present invention (Claim 7), when there are a plurality of server devices,
In the client device, the server device to be accessed is selected based on the digital watermark information and feature amount detected from the content, or time information and position information.

The present invention (Claim 8) is a system content related information providing system including a client device and a server device,
The client device is at least
A watermark presence / absence determining unit that receives input of content and determines presence / absence of digital watermark information from the content;
Information transmitting means for transmitting information to the server;
Information display means for displaying related information received from the server device,
The server device is at least
Information storage means in which feature quantities and related information are stored in association with each other;
Search means for searching the information storage means based on the input information and acquiring related information;
Information return means for transmitting related information to the client device,
The client device and the server device are
Watermark detection means for detecting a digital watermark of content;
It has at least one of feature quantity extraction means for extracting feature quantities from the content.

  FIG. 2 is a principle configuration diagram of the present invention.

In the present invention (claim 9), the client device 200
A watermark presence / absence determination unit 220 that accepts input of content and determines presence / absence of digital watermark information from the content;
Watermark detection means 230 for detecting the digital watermark of the content;
Feature amount extraction means 240 for extracting feature amounts from content;
Information transmitting means 250 for transmitting the extracted feature quantity to the server device;
The server device 100
Information receiving means 110 for receiving the feature amount transmitted from the client device 200;
A search unit 120 that searches the information storage unit 130 based on the received feature quantity and acquires related information;
And information return means 140 for transmitting the related information to the client device.

In the present invention (claim 10), the client device
A watermark presence / absence determining unit that receives input of content and determines presence / absence of digital watermark information from the content;
Watermark detection means for detecting a digital watermark of content;
Information transmitting means for transmitting content to the server device,
Server device
Information receiving means for receiving the previous content transmitted from the client device;
Feature amount extraction means for extracting feature amounts from content;
An information search unit that searches the information storage unit based on the extracted feature amount and acquires related information;
Information return means for transmitting related information to the client device.

In the present invention (claim 11), the client device
A watermark presence / absence determining unit that receives input of content and determines presence / absence of digital watermark information from the content;
Information transmitting means for transmitting the input content to the server device,
Server device
Information receiving means for receiving content transmitted from the client device;
Watermark detection means for detecting a digital watermark of content;
Feature amount extraction means for extracting feature amounts from content;
An information search unit that searches the information storage unit based on the extracted feature amount and acquires related information;
Information return means for transmitting related information to the client device.

In the present invention (claim 12),
In the watermark presence / absence determining means 220 in the client device,
If the presence of the digital watermark cannot be confirmed, it is determined that the feature quantity corresponding to the content is not registered in the information storage unit in the server device, and the subsequent processing is not executed.

In the present invention (Claim 13), the server device 100
An information storage means for storing related information corresponding to a set of content features and a digital watermark embedded in the content;
Information retrieval means 120
Corresponds to a set of content features and a digital watermark embedded in the content based on information obtained by combining the digital watermark detected by the digital watermark detection unit and the feature amount extracted by the feature amount extraction unit Information storage means storing related information is searched.

Further, according to the present invention (Claim 14), when there are a plurality of server devices,
The client device includes means for selecting a server device to be accessed based on digital watermark information and feature amounts detected from content, or time information and position information.

The present invention (Claim 15) provides a computer with:
A watermark presence / absence determination step for accepting input of content and determining presence / absence of digital watermark information from the content;
A watermark detection step for detecting a digital watermark of the content;
A feature extraction step for extracting a feature from the content;
And a step of transmitting the extracted feature quantity to the server device.

The present invention (Claim 16) provides a computer having information storage means for storing feature quantities and related information in association with each other.
An information receiving step for receiving content transmitted from the client device;
A watermark detection step for detecting a digital watermark of the content;
A feature extraction step for extracting features from the input content;
An information retrieval step of retrieving information storage means based on the feature amount and transmitting related information as a retrieval result to the client device;
Is a related information providing program for executing

According to the present invention (Claim 17), a computer having information storage means for storing related information corresponding to a set of a feature amount of a content and a digital watermark embedded in the content is provided.
An information receiving step for receiving content transmitted from the client device;
A watermark detection step for detecting a digital watermark of the content;
A feature extraction step for extracting features from the input content;
An information retrieval step of retrieving information storage means based on the feature amount and the digital watermark, and transmitting related information as a retrieval result to the client device;
Is a related information providing program for executing

  As described above, according to the present invention, when a digital watermark can be detected from the input content, it indicates that the feature amount is already registered in the server device, and thus the related information acquisition always guarantees the registration of the related information. Processing is possible. This eliminates the need for the user to perform useless communication processing between server devices when shooting an image or the like in which no feature amount is registered or when a different feature amount is extracted due to a shooting error such as camera shake. , User convenience is improved.

  When related information is obtained by a combination of a feature amount and a watermark, the content is specified by the feature amount, information indicating whether or not the content has been registered, or switching between different related information for the same content Since the digital watermark is used, the information amount of the digital watermark can be reduced. Therefore, there are effects such as suppression of image degradation due to digital watermark embedding, saving of ID space for watermark information, improvement of digital watermark detection rate, and reduction of erroneous recognition rate when detecting digital watermark.

  There is a risk of misrecognition that the content that is similar in appearance with the feature amount alone has the same feature amount, but the correct related information can be selected by using digital watermark together, and the number of contents that can be used Can be increased.

  In addition, it is possible to generate content in which different pieces of related information are associated with each other from the same content, which is impossible with the feature amount alone. Accordingly, it is possible to provide appropriate related information according to the distribution medium (newspaper or magazine).

  In the embodiment described below, it is assumed that the feature amount of an image is registered in the related information storage unit of the server, and some information is embedded in the image with a digital watermark.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 3 is a system configuration diagram (part 1) according to the first embodiment of the present invention.

  The system shown in FIG. 1 includes a server device 100 and a client device 200.

  The server device 100 includes an information receiving unit 110, an information searching unit 120, a related information storage unit 130, and an information reply unit 140.

  The client device 200 includes a content input unit 210, a watermark presence / absence determination unit 220, a watermark detection unit 230, a feature amount extraction unit 240, an information transmission unit 250, and an information display unit 260.

  FIG. 4 shows a processing image of the first embodiment of the present invention.

  In the present embodiment, watermark information is embedded in advance in the content in which the feature amount of the image is registered in the related information storage unit 130 of the server device 100, and the watermark information is used as a feature amount registered flag. . If the watermark detection unit 230 of the client device 200 can confirm the presence of the watermark, it is determined that the feature amount has been registered in the server device 100, the feature amount extraction unit 240 extracts the feature amount, and the information transmission unit 250 sends the server to the server. Transmit to device 100. If the presence of the watermark cannot be confirmed, it is determined that the feature quantity is not registered in the server apparatus 100, and the process ends without performing a process such as feature quantity extraction or inquiry to the server apparatus 100.

  FIG. 5 is a flowchart of the operation in the first embodiment of the present invention.

  First, a target image is taken by a client device 200 such as a camera-equipped cellular phone, and the target image area is extracted and acquired by the content input unit 210 (step 110).

  Next, the watermark presence / absence determination unit 220 of the client device 200 determines whether the presence of the digital watermark can be confirmed from the target image area. If the presence of the watermark can be confirmed, it is determined that the feature amount has been registered in the server device 100 (step 111, Yes). If the presence of the watermark cannot be confirmed (No at step 111), the process is terminated without performing the subsequent processes.

  If the feature amount is registered in the server device 100, the watermark detection unit 230 detects the digital watermark (step 112), and the feature amount extraction unit 240 extracts the feature amount (step 113).

  The information transmission unit 250 transmits the feature amount extracted in step 113 to the server device 100 (step 114).

  The information reception unit 110 of the server device 100 receives the feature amount from the client device 200 and passes it to the information search unit 120. As a result, the information search unit 120 searches the related information storage unit 130 configured as shown in FIG. 6 based on the acquired feature amount, acquires the related information, and receives the client device 200 via the information return unit 140. (Step 115).

  As a result, the client device 100 that has acquired the related information displays the related information on a display device such as a display in the information display unit 260 (step 116).

  Here, as shown in FIG. 6, the related information storage unit 130 stores feature amounts and related information corresponding to the feature amounts. In addition, the said feature-value shall be previously extracted from the content and shall be matched with the relevant information for providing.

  In the present embodiment, the feature amount extraction unit 240 extracts the feature amount in step 113, but sufficient client processing capability may be required for feature amount extraction, and depending on the client, the processing capability is insufficient. Is also possible. Therefore, it is also conceivable to transmit the image as it is to the server device 100 after the presence / absence determination of the digital watermark and extract the feature amount from the image received on the server device 100 side. An example is shown below.

  FIG. 7 is a system configuration diagram (part 2) according to the first embodiment of this invention.

  In the figure, the same components as those in FIG.

  The system shown in FIG. 7 includes a server device 100 and a client device 200, similarly to the system shown in FIG.

  The client device 200 has a configuration in which the feature amount extraction unit 240 is removed from the configuration illustrated in FIG. 3. As illustrated in FIG. 8, the information transmission unit 250 transmits the input content to the server device 100. Transmit (step 204).

  The server device 100 has a configuration in which a feature amount extraction unit 150 is added to the configuration of FIG. 3, and the feature amount extraction unit 150 extracts a feature amount from the content transmitted from the client device 200 (step 206), and information It passes to the search unit 120. The information search unit 120 searches the related information storage unit 130 based on the feature amount information and acquires related information (step 207).

  Further, as shown in FIG. 9, similarly for the detection of a digital watermark, the server apparatus 100 may be provided with a watermark detection unit 160 and the server apparatus side 100 may detect the digital watermark. In that case, as shown in FIG. 10, after the client apparatus 200 confirms the presence / absence of the watermark (step 302), if the digital watermark exists (step 302, Yes), the content is sent from the information transmission unit 250. The server apparatus 100 transmits the contents (step 303), the server apparatus 100 receives the contents (step 304), and the watermark detection unit 160 detects the digital watermark (step 305). The subsequent processing is the same as the processing after step 205 described above.

  In addition, the client device 200 after the presence / absence determination of the presence of the digital watermark performs only simple preprocessing necessary for extracting the feature amount, compresses the information amount, and only the data of the preprocessing result is stored in the server device 100. And the server apparatus 100 may extract the final feature value.

  In this embodiment, the presence / absence determination of the watermark is used as a feature amount registered flag. However, the determination of the presence / absence of watermark embedding in step 111, step 202, and step 302 is performed in the aforementioned Non-Patent Document 1. This can be realized by using the methods described in “3.4.5 Error correction / detection decoding” and “3.5 Reliability of digital watermark detection”. At this time, the size of the embedded digital watermark information is not limited, and may be 1-bit information or may be embedded up to the upper limit of the embeddable information amount.

  In the present embodiment, the feature amount and the related information have a one-to-one relationship, and are stored in the related information storage unit 130 in the form shown in FIG. 6, for example. At this time, if the presence of the watermark is confirmed in step 111 (step 111, Yes), since the related information is registered in the server apparatus 100, if the feature information is inquired to the related information storage unit 130, The associated information associated with each other can be uniquely identified and acquired. For example, if the feature amount is “012345”, “http://abc.0000.co.jp” can be acquired as related information.

  In this embodiment, an example of a URL is shown as related information associated with a feature quantity. In addition to this, various related information such as URI, image data, text data, e-mail reply, and application download are available. Information can also be provided, and is not limited to the example of this embodiment.

[Second Embodiment]
FIG. 11 is a system configuration diagram according to the second embodiment of the present invention.

  In the figure, the same components as those in FIG.

  The system shown in FIG. 11 includes a server device 100 and a client device 200, similarly to the system shown in FIG.

  The client device 200 has the same configuration as that shown in FIG. The server device 100 is substantially the same as the configuration of FIG. 3, but the related information storage unit 135 has the configuration shown in FIG. 12, and includes feature amounts, watermark information, and related information to be provided. It is assumed that the feature amount stored in the related information storage unit 135 is previously extracted from the content. The watermark information is information on a watermark to be embedded in the content. The feature information and the related information providing watermark information are stored in the related information storage unit 135 in association with each other.

  FIG. 13 is a processing image according to the second embodiment of the present invention, and shows a case where related information is provided from a combination of watermark information and feature amounts. FIG. 14 is a flowchart of the operation in the second embodiment of the present invention.

  First, a target image is photographed by the client device 100 such as a mobile phone with a camera, and the target image area is extracted and acquired by the content input unit 210 (step 401).

  Next, the watermark presence / absence determination unit 220 determines whether or not a watermark exists in the image acquired in step 401. If the watermark exists (step 402, Yes), the watermark detection unit 230 detects a digital watermark (step 403). ) If there is no watermark (No in step 402), the process ends without performing any processing.

  Next, the feature amount extraction unit 240 extracts the feature amount (step 404), and transmits the digital watermark and the feature amount to the server apparatus 100 via the information transmission unit 250 (step 405).

  The information receiving unit 110 of the server device 100 receives the digital watermark and the feature amount transmitted from the client device 200, and the information search unit 120 uses the related information storage unit 135 based on the received watermark information and feature amount pair. And the related information is acquired and returned from the information return unit 140 to the client device 200 (step 406).

  The information display unit 260 of the client device 200 displays the related information transmitted from the server device 100 on the display device (step 407).

  In the above-described embodiment, the digital watermark detection and the feature amount extraction processing are performed on the client device 200 side, but it is not necessary to execute all the processing on the client device 200, and various variations are conceivable. For example, as shown in FIG. 15, the feature amount extraction unit 150 is provided in the server device 100, and as shown in FIG. 16, the digital watermark detection processing is performed by the client device 200, and the watermark information and the image (content) are sent to the server. It is also possible to transmit to the device 100 (step 504), and the server device 100 can perform the feature amount extraction processing (step 506) using the watermark information and the content.

  Further, both the digital watermark detection and the feature amount extraction processing may be performed by the server device 100.

  Moreover, the case where these processes are combined suitably is also considered. Further, whichever of the digital watermark detection process and the feature quantity extraction process may be performed first.

  An example to which the present embodiment is applied is shown below.

  As shown in FIG. 17, in the case of similar different patterns having the same feature amount, the pattern cannot be distinguished only by the feature amount, and optimal related information cannot be provided. Therefore, different watermark information is embedded in these images, and related information is registered in the related information storage unit 135 of the server apparatus 100 for each feature amount and watermark information set. By doing so, even when the same feature amount is extracted, it can be correctly distinguished by the watermark information.

  Furthermore, even if a feature quantity different from the actual one has been extracted due to factors such as camera shake, it is possible to determine that the feature quantity is incorrect by referring to the digital watermark information, It becomes possible to provide correct relevant information.

  In addition, when an image having the same pattern is distributed to different media (magazines and newspapers), there are cases where it is desired to change related information provided for each media. For example, this is the case when it is desired to distribute different coupons for an advertisement placed in magazine A and an advertisement placed in magazine B. At this time, since the same image is placed, different related information for each medium cannot be provided only by the feature amount. Therefore, as shown in FIG. 18, it is conceivable to embed digital watermark information that differs depending on the distribution medium. By extracting the same feature quantity from the image, it is shown that the advertisement image is the same, but since the digital watermark information is different, it turns out that the distribution medium is different, and it is possible to provide optimal related information Become.

  In the present embodiment, the related information is associated with the combination of the feature amount and the watermark information, and is stored in the related information storage unit 135 in the form shown in FIG. 12, for example. If the related information storage unit 135 is inquired by combining both the feature amount and the watermark information, the related information can be uniquely identified and acquired. For example, if the feature amount is “012345” and the watermark information is “00110011”, the related information is “http://abc.0000.co.jp”.

  In the present embodiment, an example is described in which related information provided for each medium is changed when the same image is distributed on different media. In this case, by embedding different watermark information for each medium, the related information to be provided can be changed even when the same feature amount is extracted. At this time, in the related information storage unit 135, for example, as shown in FIG. 12, it is assumed that feature amount information, watermark information, and related information are stored. For the combination of feature quantity “246802” and watermark “11000011”, the related information “http://def.0000.co.jp” is used, and for the combination of feature quantity “246802” and watermark “11001100” Related information “http://def.1111.co.jp” can be provided to provide different related information.

  In this embodiment, the URL is shown as related information associated with the combination of the feature quantity and the digital watermark. However, in addition to this, URI, image data, text data, e-mail reply, application Various related information such as download can also be provided, and the present invention is not limited to the example of this embodiment.

[Third Embodiment]
In the present embodiment, the inquiry destination server device from the client device is switched based on the watermark information detected on the client device, and the related information to be provided changes depending on the transmitted information and the inquiry destination server device.

  For example, in the example shown in FIG. 19, the digital watermark is detected before the client device makes an inquiry to the server device. In accordance with the detected electronic watermark information, the inquiry server devices are switched from a to d, and feature amount detection is performed on each server device. In the server device a, information A or information E is selected according to the feature amount. Information B is distributed in two ways: when server device b is selected and when a specific feature amount is extracted by server device c.

  According to this embodiment, even if the same feature amount is extracted, different information for each image such as a digital watermark is detected in advance on the client device side, and the inquiry server device is switched according to the information. As a result, it is possible to switch the provided information for each server device, and it is possible to provide detailed and diverse information rather than switching the transmission information by the feature amount alone or the digital watermark alone.

  Furthermore, information other than information detected from a captured image such as time information and position information can be used for switching the server device.

  As an example of using time information, prepare a large number of server devices for a time zone where many users are expected, and further load distribution, conversely reduce server devices when there are few users, reducing operating costs. Can be achieved. Further, by switching the transfer server device according to the timing of data transmission, it is possible to provide game characteristics.

  As an example of using position information, it is possible to obtain related information according to a region by accessing different server devices even with the same pattern depending on the location.

  In the present embodiment, the server apparatus that provides the related information is one stage, but relaying in a plurality of stages is also conceivable. In addition to detecting information on the client device and switching the server, there is only one server device that receives information from the client device. A server device that detects watermark information on the server device and provides related information is provided. A method of switching is also conceivable.

  According to the present embodiment, it is possible to prevent the server server device from being concentrated on one pole and to distribute the load on the server device.

[Fourth Embodiment]
In this embodiment, an example in which the above three embodiments are combined will be described.

  For example, if a digital watermark can be detected from an image, it indicates that the feature amount has been registered in the server device, and further switches the inquiry server device by reading the digital watermark information, and from the combination of the digital watermark and the feature amount Providing relevant information can be considered.

  For example, use for an affiliate program as shown in FIG. 20 is conceivable. Affiliate program is a mechanism in which an incentive (reward for compensation) is paid to a user from a company when a user posts an article introducing a product or a photograph, and the purchase of the product occurs through the article or photograph. It is. In this example, the book is a product, the cover of the book is an image for the affiliate, and the user advertises the book by pasting the image on some physical medium such as an advertisement or a letter.

  First, the user applies and registers his / her affiliate ID and a book to be registered for the affiliate server to the affiliate server. In response, the affiliate server returns a thumbnail image of the display of the book. At this time, a user ID and inquiry destination affiliate server information are embedded in the thumbnail image using a digital watermark. In addition, in the affiliate server, the feature amount of the book cover and the user ID are registered in association with each other. The user pastes this thumbnail on an advertisement or letter and distributes it.

  When a person who has received an advertisement or letter from a user takes a picture of a pasted thumbnail image with a client terminal such as a mobile phone and makes a purchase, the user registered with the affiliate server (Mr. A in FIG. 20) Incentives are given. The flow is shown below.

  (1) First, it is determined whether the image is an affiliate image by confirming the presence or absence of a digital watermark from the photographed image.

  (2) The inquiry affiliate server is found by reading the digital watermark.

  (3) The product can be specified by extracting the feature amount of the thumbnail image.

  (4) Furthermore, the affiliate registered user can be specified by inquiring the electronic watermark information to the affiliate server.

  From the combination of these information, the registered user is reduced by the affiliate.

  In the above-described embodiment, it is assumed that the feature amount does not change due to the digital watermark embedding. However, even if the feature amount changes after embedding the digital watermark, the feature amount after embedding is registered. If so, there is no problem.

  It is also clear that the respective effects can be obtained simultaneously by combining the first to fourth embodiments. As shown in FIG. 21, it is also conceivable to divide the digital watermark information to be embedded into the required number of bits and represent different information for each individual bit.

  Further, the feature amount used in the above-described embodiment may be a feature amount for a specific object or object in the image, in addition to the feature amount of the image content itself.

  The present invention is not limited to the above-described embodiment, but can be applied to moving image media such as video. For example, in the case of a CM in a TV program, it is conceivable to provide related information according to the broadcast program by specifying the CM with a feature amount and specifying a broadcast channel with a digital watermark.

  In addition, each operation in the above embodiment can be constructed as a program, installed in a computer used as a client device (mobile terminal) and a server device, executed, or distributed via a network. is there.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can also be applied to moving image media such as video. For example, in the case of a CM in a TV program, it can be applied to providing related information according to a broadcast program by specifying the CM with a feature amount and specifying a broadcast channel with a digital watermark.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. FIG. 2 is a system configuration diagram (part 1) according to the first embodiment of the present invention. 2 is a processing image according to the first embodiment of the present invention. 5 is a flowchart (No. 1) of an operation in the first embodiment of the present invention. 4 is a storage example of a related information storage unit in the first embodiment of the present invention. FIG. 2 is a system configuration diagram (part 2) according to the first embodiment of the present invention. 6 is a flowchart (No. 2) of an operation in the first embodiment of the present invention. FIG. 3 is a system configuration diagram (part 3) according to the first embodiment of the present invention. 6 is a flowchart (No. 3) of an operation in the first embodiment of the present invention. It is a system configuration figure (the 1) in a 2nd embodiment of the present invention. It is a memory | storage example of the related information storage part in the 2nd Embodiment of this invention. It is a processing image in the 2nd Embodiment of this invention. It is a flowchart (the 1) of the operation | movement in the 2nd Embodiment of this invention. It is a system configuration figure (the 2) in a 2nd embodiment of the present invention. It is a flowchart (the 2) of the operation | movement in the 2nd Embodiment of this invention. It is an application example (the 1) of the 2nd Embodiment of this invention. It is an application example (the 2) of the 2nd Embodiment of this invention. It is a figure for demonstrating the method to distribute a delivery server according to the watermark information in the 3rd Embodiment of this invention. It is the usage example to the affiliate in the 4th Embodiment of this invention. It is the example which combined the 1st-4th embodiment of this invention. It is an example of the related information provision system using a digital watermark. It is an example of the related information provision system using the feature-value.

Explanation of symbols

100 Server Device 110 Information Receiving Unit, Information Receiving Unit 120 Information Retrieval Unit, Information Retrieval Unit 130 Information Storage Unit, Related Information Storage Unit 135 Related Information Storage Unit 140 Information Reply Unit, Information Reply Unit 150 Feature Quantity Extraction Unit 160 Watermark Detection Unit 200 Client device 210 Content input unit 220 Watermark presence / absence determination unit, watermark presence / absence determination unit 230 Watermark detection unit, watermark detection unit 240 Feature amount extraction unit, feature amount extraction unit 250 Information transmission unit, information transmission unit 260 Information display unit, Information display section

Claims (17)

A content related information providing method in a system composed of a client device and a server device,
The client device is
At least a watermark presence / absence determining step for receiving presence of content and determining presence / absence of digital watermark information from the content is performed,
The server device
At least, based on the input information, search the information storage means in which the feature amount and the related information are stored in association with each other, and perform an information search step of providing related information as a search result to the client device,
In either the client device or the server device,
Performing at least one of a watermark detection step of detecting a digital watermark of the input content and a feature amount extraction step of extracting a feature amount from the input content;
Related information distribution method characterized by the above. In the client device,
The watermark presence / absence determination step;
The watermark detection step;
The feature extraction step;
Transmitting the extracted feature quantity to the server device, and
In the server device,
An information receiving step of receiving the feature amount transmitted from the client device;
Searching the information storage means based on the feature amount, and performing an information search step of transmitting related information as a search result to the client device,
In the client device,
A display step of displaying the related information received from the server device;
The related information providing method according to claim 1. In the client device,
The watermark presence / absence determination step;
The watermark detection step;
Transmitting the content to the server device; and
In the server device,
An information receiving step of receiving the previous content transmitted from the client device;
The feature extraction step;
Searching the information storage means based on the feature amount, and performing an information search step of transmitting related information as a search result to the client device,
In the client device,
A display step of displaying the related information received from the server device;
The related information providing method according to claim 1. In the client device, a watermark presence / absence determination step for receiving input of content and determining presence / absence of digital watermark information from the content;
Transmitting the input content to the server device, and
In the server device,
An information receiving step of receiving the content transmitted from the client device;
The watermark detection step;
The feature extraction step;
Searching the information storage means based on the feature amount, and performing an information search step of transmitting related information as a search result to the client device,
In the client device,
A display step of displaying the related information received from the server device;
The related information providing method according to claim 1. In the watermark presence / absence determining step in the client device,
5. The related information according to claim 1, wherein if the presence of the electronic watermark cannot be confirmed, the subsequent steps are not executed on the assumption that the feature quantity corresponding to the content is not registered in the information storage unit in the server device. How to provide. In the search step,
Information storage means storing related information corresponding to a feature amount of content and a set of digital watermark embedded in the content is extracted in the digital watermark detected in the digital watermark detection step and in the feature amount extraction step Search based on information that combines the feature amount
The related information providing method according to claim 1. When there are a plurality of the server devices,
5. The related information providing method according to claim 1, wherein the client device selects a server device to be accessed based on digital watermark information and feature amounts detected from content, or time information and position information. A system content related information providing system composed of a client device and a server device,
The client device is at least
A watermark presence / absence determining unit that receives input of content and determines presence / absence of digital watermark information from the content;
Information transmitting means for transmitting information to the server;
And information display means for displaying related information received from the server device,
The server device is at least
Information storage means in which feature quantities and related information are stored in association with each other;
Search means for searching the information storage means based on the input information and acquiring related information;
Information return means for transmitting the related information to the client device,
The client device and the server device are:
Watermark detection means for detecting a digital watermark of the content;
Having at least one of feature quantity extraction means for extracting feature quantities from the content;
Related information distribution system characterized by this. The client device is
The watermark presence / absence determining means;
The watermark detection means;
The feature extraction means;
The information transmission means for transmitting the extracted feature quantity to the server device;
The server device
Information receiving means for receiving the feature amount transmitted from the client device;
The search means for searching the information storage means based on the received feature quantity and acquiring related information;
And the information return means,
The related information provision system according to claim 8. The client device is
The watermark presence / absence determining means;
The watermark detection means;
The information transmitting means for transmitting the content to the server device,
The server device
Information receiving means for receiving the previous content transmitted from the client device;
The feature extraction means;
Information search means for searching the information storage means based on the extracted feature quantity, and acquiring related information;
The related information providing system according to claim 8, further comprising: the information return unit. The client device is
The watermark presence / absence determining means;
The information transmitting means for transmitting the input content to the server device,
The server device
Information receiving means for receiving the content transmitted from the client device;
The watermark detection means;
The feature extraction means;
Information search means for searching the information storage means based on the extracted feature quantity, and acquiring related information;
And the information return means,
The related information provision system according to claim 8. In the watermark presence / absence determining means in the client device,
12. If the presence of a digital watermark cannot be confirmed, it is determined that the feature quantity corresponding to the content is not registered in the information storage unit in the server device, and the subsequent processing is not executed. The related information provision system described. The server device
An information storage means for storing related information corresponding to a set of content features and a digital watermark embedded in the content;
The information search means includes
Based on information obtained by combining the digital watermark detected by the digital watermark detection unit and the feature amount extracted by the feature amount extraction unit, the feature amount of the content and the digital watermark embedded in the content Search for information storage means storing related information corresponding to the set of
The related information providing system according to claim 8. When there are a plurality of the server devices,
12. The related information providing system according to claim 8, wherein the client device includes means for selecting a server device to access based on digital watermark information and feature amounts detected from content, or time information and position information. On the computer,
A watermark presence / absence determination step for accepting input of content and determining presence / absence of digital watermark information from the content;
A watermark detection step of detecting a digital watermark of the content;
A feature amount extraction step of extracting a feature amount from the content;
Transmitting the extracted feature quantity to the server device; and executing the related information providing program. In a computer having information storage means for storing feature quantities and related information in association with each other,
An information receiving step of receiving the content transmitted from the client device;
A watermark detection step of detecting a digital watermark of the content;
A feature amount extraction step of extracting a feature amount from the input content;
An information search step of searching the information storage unit based on the feature amount and transmitting related information as a search result to the client device;
A related information providing program characterized in that A computer having information storage means for storing related information corresponding to a set of content features and a set of digital watermarks embedded in the content;
An information receiving step of receiving the content transmitted from the client device;
A watermark detection step of detecting a digital watermark of the content;
A feature amount extraction step of extracting a feature amount from the input content;
An information search step of searching the information storage unit based on the feature amount and the digital watermark, and transmitting related information as a search result to the client device;
A related information providing program characterized in that

Images