JP5013840B2 - Information providing apparatus, information providing method, and computer program - Google Patents

Description

  The present invention relates to an information providing apparatus, an information providing method, and a computer program that provide information in response to a request from a user terminal in a computer network.

On a computer network such as the Internet, a search service for searching a URL of a web page of content including the keyword and other related information for a keyword input by the user to the user terminal and transmitting it to the user terminal is practical. It has become. By the way, the user of a user terminal is various regardless of age and sex. For this reason, the document format that each user can easily view is also different. On the other hand, a technique for providing content in a document format desired by the user has been proposed (for example, Patent Document 1).
JP 2006-23936 A

  However, according to the technique of Patent Document 1, the content provided to the user is easy for the user to see. However, as in the case of a conventional general search service, the user desires if there is no input of an appropriate keyword. There is a problem that it is not possible to provide information.

  Accordingly, an object of the present invention is to provide an information providing apparatus, an information providing method, and a computer program that can provide information desired by a user without inputting an appropriate keyword.

(1) An information providing apparatus that provides information in response to a request from a user terminal, and includes a plurality of whole images (frames) constituting moving image content, a part of the whole image, and an identifiable object. , Feature information of the partial image, an in-scene position that is information indicating a temporal position of the partial image in the entire moving image content, an in-image position indicating coordinates in which the partial image is located in the entire image, An image search database for storing a motion vector, which is information indicating the motion of the partial image in the entire image, and a link to the moving image content including the entire image, and the partial image from the user terminal A partial image receiving unit that receives a search key as a search key, and a feature detection unit that detects features of the partial image received from the user terminal, The feature information that extracts the feature information correlated with the feature of the partial image received from the user terminal from the image search database by using the position in the scene, the position in the image, and the motion vector in combination. An extraction unit; a related information transmission unit that transmits the link corresponding to the feature information extracted by the feature information extraction unit to the user terminal; and the in- scene corresponding to the feature information extracted by the feature information extraction unit Based on the position, in the video content indicated by the link corresponding to the feature information, a scene in which the partial image is displayed on the user terminal is determined, and an advertisement corresponding to the partial image is displayed only while the partial image is displayed. Is displayed on the user terminal, and the position in the image and the motion vector corresponding to the feature information extracted by the feature information extraction unit are displayed. A scene to display an advertisement according to the motion of the partial image in the video content indicated by the link corresponding to the feature information, and based on the determination, the advertisement corresponding to the partial image is An information providing apparatus comprising an advertisement distribution unit to be displayed on a user terminal .

According to the invention of (1), the information providing apparatus uses the partial image as a key for searching for feature information, and transmits a link (URL or address) to content corresponding to the feature information to the user terminal. be able to. That is, it can be used not as a character string but as a key for searching a partial image. For this reason, even if an appropriate keyword is not input, information desired by the user can be provided. Furthermore, since the information providing apparatus can determine the time when the partial image input by the user as a search key exists in the entire moving image content, only the portion where the partial image input by the user exists is transmitted to the user terminal. be able to.
Further, the information providing apparatus stores the coordinates where the partial image is located in the entire image constituting the moving image content and the movement thereof in the image search database. Therefore, since the information providing apparatus can determine the position in the entire image where the partial image input by the user as a search key exists in the entire moving image together with the movement, only the portion where the partial image input by the user exists Can be transmitted to the user terminal. As a result, for example, if the position in the image is applied to advertisement distribution, a banner advertisement corresponding to the specific partial image is displayed on the user terminal only while the specific partial image is displayed in the moving image content. be able to. In addition, if the motion vector is further applied to advertisement distribution, it is assumed that not only while a specific partial image is displayed in the video content, but also during the time when the specific image is not displayed, When it is determined that the image is out of the image, a banner advertisement corresponding to the specific partial image can be continuously displayed on the user terminal.
Furthermore, the information providing apparatus can display an advertisement related to the partial image on the user terminal corresponding to the time when the partial image input by the user as a search key exists in the moving image content. Further, the information providing apparatus, on the basis of the motion vector of the partial image, for example, when the partial image deviates on a predetermined side of the moving image content being displayed, adds the partial image to the predetermined side of the moving image content. Related banner advertisements can be displayed. As a result, the information providing apparatus can provide an advertisement related to the partial image selected by the user who is likely to meet the user's interest without requiring the user to search for a banner advertisement or a sales page. .

( 2 ) In a plurality of whole images (frames) constituting the moving image content, feature information of the partial image that is a part of the whole image and includes an identifiable object, and temporal information of the partial image in the whole moving image content A position in the scene that is information indicating the position, a position in the image that indicates coordinates in which the partial image is located in the entire image, and a motion vector that is information indicating the movement of the partial image in the entire image. An information providing method for providing information in response to a request from a user terminal by an information providing apparatus having an image search database that stores the link to the moving image content including the whole image in association with the user terminal, A partial image receiving step of receiving the partial image as a search key from the user terminal, and the partial image received from the user terminal. Using the feature detection step for detecting the feature of the image and the position in the scene, the position in the image, and the motion vector in combination, the feature information correlated with the feature of the partial image received from the user terminal A feature information extraction step extracted from an image search database, a related information transmission step for transmitting the link corresponding to the feature information extracted in the feature information extraction step to the user terminal, and the feature information extraction step extracted Based on the position in the scene corresponding to the feature information, the scene in which the partial image is displayed on the user terminal in the moving image content indicated by the link corresponding to the feature information is determined and displayed. Only the advertisement corresponding to the partial image is displayed on the user terminal, and the feature information extraction step is performed. A scene that displays an advertisement according to the movement of the partial image in the moving image content indicated by the link corresponding to the feature information based on the position in the image corresponding to the feature information extracted by the group and the motion vector. An information providing method comprising: an advertisement distribution step of determining and displaying an advertisement corresponding to the partial image on the user terminal based on the determination .

According to the invention of ( 2 ), similarly to the invention described in (1), the information desired by the user can be provided without inputting an appropriate keyword. Furthermore, only the portion where the partial image input by the user exists in the entire moving image content can be transmitted to the user terminal.
Furthermore, in the moving image content, an advertisement related to the partial image can be displayed on the user terminal corresponding to the time when the partial image input by the user as a search key exists. In addition, based on the motion vector of the partial image, for example, when the partial image deviates on a predetermined side of the moving image content being displayed, a banner advertisement related to the partial image is displayed on the predetermined side of the moving image content. Can be displayed. As a result, it is possible to provide an advertisement related to the partial image selected by the user who is likely to meet the user's interest without requiring the user to search for a banner advertisement or a sales page.

( 3 ) In a plurality of whole images (frames) constituting the moving image content, the computer is a part of the whole image and includes characteristic information of the partial image including an identifiable object, and the whole moving image content of the partial image Information indicating the position in the scene, which is information indicating the temporal position in the image, the position in the image indicating the coordinates where the partial image is located in the entire image, and the information indicating the movement of the partial image in the entire image. A computer program for causing a motion vector to function as an information providing apparatus having an image search database for storing a motion vector and a link to the moving image content including the entire image, wherein the partial image is searched from a user terminal. A partial image receiving step for receiving the partial image received from the user terminal Feature information correlating with the feature of the partial image received from the user terminal by using the feature detection step of detecting a sign together with the position in the scene, the position in the image, and the motion vector. A feature information extraction step extracted from a search database, a link corresponding to the feature information extracted in the feature information extraction step, a related information transmission step transmitted to the user terminal, and the feature information extracted by the feature information extraction step Based on the position in the scene corresponding to the feature information, the scene in which the partial image is displayed on the user terminal in the moving image content indicated by the link corresponding to the feature information is determined, and the scene is displayed only while the partial image is displayed. An advertisement corresponding to the partial image is displayed on the user terminal, and the feature information extraction step includes Based on the in-image position and the motion vector corresponding to the feature information that has been issued, a scene for displaying an advertisement according to the motion of the partial image in the moving image content indicated by the link corresponding to the feature information is determined. A computer program for executing an advertisement distribution step for causing the user terminal to display an advertisement corresponding to the partial image based on the determination .

According to the invention of ( 3 ), as in the invention described in (1), the information desired by the user can be provided without inputting an appropriate keyword. Furthermore, only the portion where the partial image input by the user exists in the entire moving image content can be transmitted to the user terminal.
Furthermore, in the moving image content, an advertisement related to the partial image can be displayed on the user terminal corresponding to the time when the partial image input by the user as a search key exists. In addition, based on the motion vector of the partial image, for example, when the partial image deviates on a predetermined side of the moving image content being displayed, a banner advertisement related to the partial image is displayed on the predetermined side of the moving image content. Can be displayed. As a result, it is possible to provide an advertisement related to the partial image selected by the user who is likely to meet the user's interest without requiring the user to search for a banner advertisement or a sales page.

  According to the present invention, it is possible to provide an information providing apparatus, an information providing method, and a computer program that can provide information desired by a user without inputting an appropriate keyword.

[First Embodiment]
FIG. 1 is a diagram showing a schematic configuration and the like of an information providing apparatus 10 (hereinafter referred to as “apparatus 10”) according to the first embodiment of the present invention. FIG. 2 is a schematic diagram showing the user terminal 5 and the like. 3 and 4 are diagrams illustrating an example of a moving image displayed on the user terminal 5 or the like. As illustrated in FIG. 1, the device 10 includes an image dictionary generation device 20, a search device 40, a motion information addition device 60, an advertisement matching device 80, and a bookmark generation device 100. The device 10 can communicate with a plurality of user terminals 5, 5, user terminals 7, and the like via a communication line such as the Internet line 9.

  The image dictionary generation device 20 is a server that generates a search dictionary in which a partial image is associated with content such as a moving image including the partial image. The search key of the image dictionary generation device 20 is not a character string but a partial image. Hereinafter, the partial image input for searching is referred to as “search image”. Thus, the search image is not a whole image but a partial image. For example, an image including all of the A type automobile 200, the road 202, the sparrow 204, the tree 206, and the background 208 shown in FIG. 3C is referred to as an entire image (frame). Each image of the A type automobile 200 or the like is referred to as a partial image. A plurality of whole images (frames) constitute a content such as a moving image.

  The motion information adding device 60 is a server that determines the motion of a partial image, generates motion information, and adds the motion information to the partial image. The advertisement matching device 80 displays an advertisement such as an advertisement banner corresponding to the partial image on the display screen of the user terminal 5 or the like in response to the timing at which the specific partial image is displayed on the display screen of the user terminal 5 or the like. It is a server that performs processing for. The bookmark generation device 100 is a server that performs a process of making a partial image a bookmark that can be commonly used in a plurality of user terminals 5 and the like. Corresponding to the search image received from the user terminal 5 or the like, the search device 40 extracts a link (URL or address) to the content from the image search DB 34 (see FIG. 5) of the image dictionary generation device 20, and the user terminal 5 It is a server for sending to etc. Note that the search device 40 may transmit related information in addition to the link to the content.

  As shown in FIG. 2A, the user terminal 5 is, for example, a mobile phone, and includes a liquid crystal screen 5a, an operation button group 5b, an enter button 5c, and a move button 5d. In addition to the general function, the movement button 5d is also used to move the cursor to a specific partial image in the entire image displayed on the liquid crystal screen 5a. In addition to a general function, the determination button 5c is also used to determine a partial image on which the cursor is moved by the movement button 5d as a search image. As shown in FIG. 2B, the user terminal 7 is, for example, a laptop personal computer, and includes a liquid crystal screen 7a, an operation button group 7b, and a mouse 7c. Various information is displayed on the liquid crystal screens 5a and 7a. For example, the moving images shown in FIGS. 3A to 3C and FIGS. 4A to 4C are also displayed. The images of the scenes 1 to 6 shown in FIGS. 3 and 4 are continuously displayed to form a series of moving images.

[Image Dictionary Generation Device 20]
FIG. 5 is a schematic block diagram illustrating the configuration of the image dictionary generation device 20.
As illustrated in FIG. 5, the image dictionary generation device 20 includes a content acquisition unit 21. The content acquisition unit 21 is a configuration for acquiring (receiving) various contents from the outside of the device 10 via the Internet line 9 and functions as an image information acquisition unit. The content acquisition unit 21 includes a computer program and hardware such as a CPU (Central Processing Unit) that controls the apparatus 10 based on the program, and includes an input unit and an output unit. Each unit described below is similarly configured by a computer program and hardware, and has an input unit and an output unit.

  The content includes text data and image data. The image data includes moving image data. The image dictionary generation device 20 includes a content clustering unit 22. The content clustering unit 22 is configured to classify content acquired from the outside into clusters (subsets) and store them in a content DB (database) 30 in a state where the classification method is machine-learned by so-called supervised learning. That is, the content clustering unit 22 is an automatic classifier. Each content has a unique URL. The content clustering unit 22 described above functions as an image information storage unit. The cluster is, for example, “a set of automobile-related movies”, “a set of bicycle-related movies”, “a set of music-related movies”, or the like.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a partial image acquisition unit 23. The partial image acquisition unit 23 acquires, for example, an image of the A type automobile 200 in FIG. 3C (hereinafter also simply referred to as “A type automobile 200”. The same applies to other partial images). It is a configuration. For example, the partial image acquisition unit 23 automatically extracts a partial image from content stored in the content DB 30. The partial image acquisition unit 23 displays the image shown in FIG. 3C on the user terminal 7, and when the user of the user terminal 7 moves the cursor to the A-type automobile 200 with the mouse 7 c and clicks, the partial image acquisition unit 23 passes the Internet line 9. The entire image and the partial image (for example, the A type automobile 200) included in the entire image may be received.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a partial image clustering unit 24. The partial image clustering unit 24 is configured to perform machine learning of the classification method by so-called supervised learning, classify content acquired from the outside into clusters (subsets), and store the contents in the partial image DB 32. That is, the partial image clustering unit 24 is an automatic classifier and functions as a partial image storage unit. The cluster is, for example, “a set of cars”, “a set of bicycles”, “a set of ships”, or the like.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a feature adding unit 25. The feature adding unit 25 is a configuration for analyzing the features of the acquired partial image and generating feature information, and functions as a feature information generating unit. The feature adding unit 25 extracts, for example, the contour of the partial image and the number of used colors as features. The contour is extracted by edge processing. With these characteristics, even when the viewing angle of the same partial image (for example, Tokyo Tower) is different, it can be determined that the partial images are the same.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a tag adding unit 26. The tag adding unit 26 is configured to record the feature information extracted from the partial image as a tag (or index) in association with a link between the tag and the content stored in the content DB 30 in the image tag table. . The image tag table is stored in the image search DB 34.

  FIG. 6 is a diagram illustrating an example of an image tag table. As shown in FIG. 6, in the image tag table, tag ID, moving image ID (also called image ID or content ID), feature information, clustering ID, link to content, position in scene, position in image (frame) ( Coordinates) and motion vectors. “Movie ID” is, for example, identification information of the entire movie shown in FIGS. 3 and 4. “Tag ID” is identification information of a partial image. In the present embodiment, feature information is used as a search key, and “tag ID” is not used as a search key. “Feature information” is information indicating the features of the partial image. The feature information indicates, for example, the outline of the automobile and the number of colors used in the automobile (5 colors in the example of FIG. 6). “Clustering ID” is identification information of a cluster (subset) to which each partial image belongs. “Link to content” is, for example, the address or URL of the entire moving image shown in FIGS. “In-scene position” is, for example, information indicating the temporal position of each scene in the entire moving image in FIGS. “In-image position (coordinates)” is, for example, information indicating the coordinates where the A-type automobile 200 is located in the entire image of FIG. The “motion vector” is information indicating the motion of the partial image in the entire image. The position in the scene, the position in the image, and the motion vector will be described in detail later.

  FIG. 7 is a conceptual diagram illustrating an example of a partial image dictionary. The partial image dictionary is a part of the above-described image tag table. As shown in FIG. 7, the partial image dictionary is configured by associating at least one content address with one piece of feature information. The content may be a video. The image search DB 34 is commonly used by each device included in the device 10.

[Searching device 40]
The device 10 refers to the image search DB 34 based on the search image received from the user terminal 5 or the like by the search device 40 and returns the URL of the content to the user terminal 5 or the like. The search device 40 may transmit information related to the search image as text information in addition to the content URL.

  FIG. 8 is a block diagram illustrating a schematic configuration of the search device 40. As illustrated in FIG. 8A, the search device 40 includes a query reception unit 41, a search processing unit 42, a search result generation unit 43, a search result transmission unit 44, and a search log recording unit 45. The query receiving unit 41 is configured to receive a search image from the user terminal 5 or the like. The search processing unit 42 is configured to search the image search DB 34 for tag IDs of partial images having characteristics that correlate with the search image. The search result generation unit 43 is configured to generate a search result for transmission to the user terminal 5 or the like. The search result transmission unit 44 is configured to transmit the search result to the user terminal 5 and the like, and functions as a related information transmission unit. The search log recording unit 45 is configured to generate a record in which each partial image is searched.

  As illustrated in FIG. 8B, the search processing unit 42 includes an image acquisition unit 42a, a feature detection unit 42b, an image search unit 42c, and a related information extraction unit 42d. The image acquisition unit 42a functions as a partial image receiving unit that receives a partial image as a search key. The feature detection unit 42b has a configuration for detecting features such as the contour and the number of colors of the received partial image. The image search unit 42c functions as an identification information extraction unit that extracts a tag ID. The related information extraction unit 42d is configured to extract related information related to the partial image searched based on the search image from the content DB 30. For example, when the apparatus 10 receives a photo of a dish from the user terminal 5 as a search image, the related information extraction unit 42d extracts the recipe for the dish from the content DB 30. The related information extracted by the related information extraction unit 42d is transmitted to the user terminal 5 or the like by the search result transmission unit 44 described above.

  FIG. 9 is a diagram illustrating an example of a search result displayed on the liquid crystal screen 5a of the user terminal 5 based on the data transmitted by the search result transmission unit 44 (see FIG. 7A). As shown in FIG. 9, for example, if the search image is “A type car”, at least one address of the home page where the A type car is displayed is displayed together with the image of the A type car. Yes. At this time, related information such as the performance such as the displacement of the A type automobile may be displayed at the same time.

[About the motion information adding device 60]
FIG. 10 is a block diagram illustrating a schematic configuration and the like of the motion information adding device 60. FIG. 11 is a diagram illustrating an example of a moving image. 11, FIG. 3C and FIG. 4A are used among the moving images of FIG. 3 and FIG. As illustrated in FIG. 10A, the motion information addition device 60 includes an image acquisition unit 61, an in-image position acquisition unit 62, a motion extraction unit 63, and a motion evaluation unit 64. The image acquisition unit 61 is a configuration for acquiring moving image data including partial images from the content DB 30 of the image dictionary generation device 20. The in-image position acquisition unit 62 coordinates (for example, the coordinates of a partial image (for example, the A type automobile 200 in FIG. 11 (a)) in a specific scene (for example, the entire image illustrated in FIG. 11 (a)) in the moving image. This is a program for acquiring the coordinates X1) in FIG.

  The motion extraction unit 63 is a configuration for extracting a motion vector of a partial image. For example, the A type automobile 200 in the scene 3 in FIG. 11A moves to the coordinate X2 in the scene 4 in FIG. 11B. In this case, for example, the motion extraction unit 63 tracks the center coordinates of the contour of the A-type automobile 200 corresponding to the initial coordinates X1, and stores a vector (motion vector) a1 indicating the movement from the coordinates X1 to the coordinates X2 in advance. It is generated at a prescribed time interval, for example, at an interval of 1/30 second. The motion vector a1 and the like indicate the motion of the A type automobile 200. The motion extraction unit 63 includes a forward feeding unit 63a and a backward feeding unit (rewinding unit) 63b, and uses the partial image in one scene in the entire moving image as a base point (hereinafter referred to as “initial coordinates”). A motion vector a1 in the direction and a motion vector ar1 in the reverse feed direction are generated. For this reason, even if a partial image in one scene of the entire moving image is used as the initial coordinates, motion vectors a1 and ar1 indicating all the movements of the specific partial image in the entire moving image are generated as shown in FIG. 11B. Can do. The motion extraction unit 63 stores the extracted motion vector in the image search DB 34. As described above, the apparatus 10 can extract the motion vector for the entire moving image regardless of the temporal position of the partial image to be used first.

  The motion evaluation unit 64 is configured to determine the timing at which the partial image deviates from the entire image based on the initial coordinates and the motion vector. In other words, the motion evaluation unit 64 is a configuration for determining a time in which a specific partial image is continuously present in the entire image constituting the moving image. The motion evaluation unit 64 determines the timing at which a specific partial image deviates from the entire image based on the size of the entire image (for example, 1280 pixels × 1024 pixels), the initial coordinates, and the motion vector. In the example of FIG. 10B, if the initial coordinate is the base of the vector a1, the motion evaluation unit 64 refers to the vectors a1, a2, and a3 in the forward feed direction, and performs partial processing in the next scene where the vector a3 is generated. It is determined that the image is out of the whole image.

  Further, for example, as shown in FIG. 10C, the motion evaluation unit 64 once returns from the entire image in the scene in which the sparrow 204, which is a partial image, is generated in the scene where the vector b4 is generated. If it is, it is determined that a return to the entire image is assumed. The motion vector b1 or the like indicates the motion of the sparrow 204.

[Advertising matching device 80]
FIG. 12 is a block diagram illustrating a schematic configuration of the advertisement matching device 80. FIG. 13 is a schematic diagram showing the user terminal 5 and the like. As illustrated in FIG. 12, the advertisement matching device 80 includes a scene position acquisition unit 81 and an advertisement matching unit 82. For example, the scene position acquisition unit 81 determines a scene in which a specific partial image, for example, an A type automobile 200 is displayed, from the scenes 1 to 9 in FIG. In the example of FIG. 13B, it is determined that the A type automobile 200 is displayed in the scenes 2 to 4.

  The advertisement matching unit 82 supports the A type automobile 200 only while the specific partial image, for example, the A type automobile 200 is displayed based on the motion vector generated in advance for the entire moving image including the scenes 1 to 9. The banner advertisement B to be displayed (see FIG. 13A) is displayed on the liquid crystal screen 5a of the user terminal 5. As shown in FIG. 13A, scenes 1 to 9 are displayed as the moving image A on the liquid crystal screen 5a, but the banner advertisement B is displayed only during the scenes 2 to 4 in which the A type automobile 200 is displayed. It is displayed.

  On the other hand, as shown in FIG. 13C, the advertisement matching unit 82 generates a specific partial image (for example, sparrow 204) based on a motion vector generated in advance for the entire moving image including the scenes 1 to 9. If it is determined that the image has been removed from the entire image on the assumption that it will return, as shown in FIG. 13C, the banner advertisement C corresponding to the sparrow 204 is continuously displayed even in the scene 4 where the sparrow 204 is not displayed. Let

[About Bookmark Generation Device 100]
FIG. 14 is a block diagram showing a schematic configuration of the bookmark generating apparatus 100. FIG. 15 is an example of an image displayed on the liquid crystal screen 5 a of the user terminal 5. As illustrated in FIG. 14A, the bookmark generation device 100 includes a bookmark registration unit 101, a common image display unit 102, a common image count unit 103, and a bookmark DB 104.

  The bookmark registration unit 101 is configured to store the user ID, the image ID of the partial image, and the description page URL in association with each other in the bookmark DB 104 when receiving a partial image from the user terminal 5, for example. Functions as a bookmark registration unit. The partial image received from the user terminal 5 may be a moving image. As shown in FIG. 14B, the user ID, the image ID, and the explanation page URL are stored in the personal table. The personal table is set for each user in the bookmark DB 104.

  The bookmark registration unit 101 is also configured to store the bookmarks stored in the personal table in the common table shown in FIG. The common table is also set in the bookmark DB 104. In the common table, various types of information are stored in association with tag IDs. The operation of the bookmark registration unit 101 described above is an example of a bookmark registration step.

  The common image display unit 102 transmits the partial image data stored in the common table to the user terminal 5 or the like that has accessed a specific homepage (hereinafter referred to as “bookmark page”), as shown in FIG. Thus, it is the structure for displaying those partial images on the liquid crystal screen 5a, and functions as a bookmark page transmission unit. As described above, the partial images stored in the common table can be freely browsed by all the viewers of a specific homepage (hereinafter referred to as “bookmark page”), for example. As shown in FIG. 15, a plurality of partial images are displayed on the “bookmark page” as bookmarks. Some of the partial images may be moving images. The operation of the common image display unit 102 described above is an example of a bookmark page transmission step.

  The common image counting unit 103 counts the number of times that a user such as the user terminal 5 who has browsed the bookmark page clicks a specific bookmark, and the common count is “point” as shown in FIG. This is a configuration for recording in a table.

  Then, as shown in FIG. 15B, the search device 40 described above displays an explanatory text (including a link to the content) corresponding to the partial image (bookmark) with many points at the top of the search result page. However, it is displayed so that the user can easily see it. For example, when a user uses a partial image of a car as a search key and the search device 40 extracts a plurality of content URLs, the URL of a page including a bookmark with a large number of counts is preferentially displayed. Note that the search key image used by the user does not always match the bookmark (partial image) that determines the priority display order.

The above is the configuration of the apparatus 10. As described above, the device 10 can use a partial image as a key for searching for feature information, and transmit a link to content including the partial image corresponding to the feature information to the user terminal. That is, it can be used not as a character string but as a key for searching a partial image. For this reason, even if an appropriate keyword is not input, information desired by the user can be provided.
Further, since the apparatus 10 can determine the time during which the partial image input by the user as a search key exists in the entire moving image, only the portion where the partial image input by the user exists can be transmitted to the user terminal. it can.

  Further, the apparatus 10 can display, on the user terminal, an advertisement related to the partial image corresponding to the time when the partial image input by the user as a search key exists in the entire moving image. Since the partial image is an image selected by the user, the advertisement associated with the partial image should be of interest to the user. For this reason, it is possible to provide advertisement information suitable for the user's interest without requiring the user to search for a banner or sales page corresponding to the product desired to be purchased.

  Furthermore, when the partial image temporarily deviates from the entire image, the device 10 continuously displays an advertisement corresponding to the partial image on the user terminal. When a partial image temporarily deviates from the entire image, the user should concentrate on advertising information that suits that interest. For this reason, the advertising effect can be enhanced.

Moreover, the apparatus 10 can display a partial image as a bookmark on a plurality of user terminals. That is, it is possible to display a social bookmark that can be understood at a glance. Furthermore, the number of times each bookmark is selected by the user terminal can be counted, and bookmarks with a large number of counts can be preferentially displayed on the user terminal, so that popular bookmarks can be preferentially displayed. it can.
Furthermore, since a bookmark can be displayed as a moving image on a plurality of the user terminals, a bookmark that is easy to attract the user's interest as a moving image can be displayed.

  Hereinafter, an operation example of the apparatus 10 will be described with reference to FIGS. 16, 17, and 18. 16, 17, and 18 are schematic flowcharts illustrating an operation example of the apparatus 10. First, an operation example in which the apparatus 10 receives a partial image from the user terminal 5 or the like and transmits a search result to the user terminal 5 or the like will be described mainly using FIG.

  First, the device 10 receives a search image (partial image) from the user terminal 5 (step S1 in FIG. 16). This step S1 is an example of a partial image receiving step. Then, the feature of the search image is detected (step S2), and feature information correlated with the feature of the search image is extracted from the image tag table (see FIG. 6) (step S3). Step S2 described above is an example of a feature detection step, and step S3 is an example of a feature information extraction step.

  Subsequently, the device 10 extracts a content URL corresponding to the feature information extracted from the image tag table (step S4), and transmits the extracted content URL to the user terminal 5 (step S5). This step S5 is an example of a related information transmission step.

  Next, mainly using FIG. 17, an advertisement banner corresponding to the partial image is displayed on the liquid crystal screen 5 a of the user terminal 5 or the like corresponding to the timing at which the search image (partial image) is displayed. An example of the operation will be described.

  First, Steps S101 to S104 in FIG. 17 are the same as Steps S1 to S4 described above, and thus description thereof is omitted. Subsequent to step S104, the apparatus 10 evaluates the movement of the partial image corresponding to the extracted feature information, and determines the timing of deviating from the entire image (step S105). Then, an advertisement banner corresponding to the partial image is associated with the scene where the partial image is displayed (step S106).

  Subsequently, the apparatus 10 transmits the extracted content URL to the user terminal 5 (step S107). When the user terminal 5 accesses the content URL, a moving image is displayed on the liquid crystal screen 5a of the user terminal 5, and an advertisement banner corresponding to the partial image is displayed only while the partial image corresponding to the extracted feature information is displayed. Is displayed (step S108).

  Next, an operation example of the apparatus 10 when the user terminal 5 or the like selects a partial image from the bookmark page will be described mainly using FIG.

  First, the apparatus 10 receives a bookmark page browsing request from the user terminal 5 (step S201 in FIG. 18). Subsequently, a bookmark page is displayed on the user terminal 50 (step S202). When a specific bookmark (partial image) is designated from the user terminal 5 (step S203), the feature of the partial image is detected, and the feature and correlation of the partial image are detected from the image tag table (see FIG. 6). Certain feature information is extracted (step S204). Then, a link to the content including the partial image corresponding to the extracted feature information is extracted from the image tag table (see FIG. 6) (step S205). Then, the movement of the designated partial image is evaluated, and the timing for deviating from the entire image is determined (step S206).

  Subsequently, the apparatus 10 performs Step S207, Step S208, and Step S209. Steps S207 to S209 are the same as steps S106 to S108 described above, and thus description thereof is omitted.

  Subsequent to step S209, the apparatus 10 adds points (see FIG. 14C) of the designated partial image (step S210), and maintains or updates the display order of the search result page (step S211).

[Second Embodiment]
Next, a second embodiment will be described with reference to FIGS. FIG. 19 is a schematic block diagram showing an advertisement matching device 80A of the device 10A (see FIG. 1) of the second embodiment. FIG. 20 is a screen example of the user terminal 5 or the like. In addition to the function of the device 10 of the first embodiment, the device 10A (see FIG. 1) of the second embodiment can change the display position of the banner advertisement in accordance with the movement of the partial image.

  As illustrated in FIG. 20A, the device 10 </ b> A displays the moving image A on the liquid crystal screen 5 a of the user terminal 5 and displays banner advertisements B <b> 1 to B <b> 4 on any of the surroundings. As shown in FIG. 20B, the advertisement matching unit 82A of the device 10A displays the banner advertisement B2 on the left side of the video A when the motion vector of the specific partial image is S1 and deviates from the video A to the left. Display. On the other hand, as shown in FIG. 20C, the advertisement matching unit 82A displays a banner advertisement on the upper side of the video A when the motion vector of the specific partial image is S2 and deviates from the upper side of the video A. B3 is displayed.

[Programs and computer-readable recording media]
A computer program for causing a computer to execute the partial image reception step, the feature detection step, the feature information extraction step, the related information transmission step, and the like in the above-described operation example can be used. Moreover, it can also be set as the computer-readable recording medium which recorded such a computer program etc.

[Hardware Configuration of Information Providing Device 10]
FIG. 21 is a diagram illustrating a hardware configuration example of the devices 10 and 10A (hereinafter referred to as the information processing device 1000). The information processing apparatus 1000 includes a CPU (Central Processing Unit) 1010 (a plurality of CPUs such as the CPU 1012 may be added in a multiprocessor configuration), a bus line 1005, a communication I / F 1040, and a main memory 1050 that configure each control unit. , A BIOS (Basic Input Output System) 1060, a USB port 1090, an I / O controller 1070, an input means such as a keyboard and mouse 1100, and a display device 1022.

  Storage means such as a tape drive 1072, a hard disk 1074, an optical disk drive 1076, and a semiconductor memory 1078 can be connected to the I / O controller 1070.

  The BIOS 1060 stores a boot program executed by the CPU 1010 when the information processing apparatus 1000 is activated, a program depending on the hardware of the information processing apparatus 1000, and the like.

  The hard disk 1074 constituting the storage unit stores various programs for the information processing apparatus 1000 to function and programs for executing the functions of the present invention, and can configure various databases as necessary.

  As the optical disc drive 1076, for example, a DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive, or a CD-RAM drive can be used. In this case, the optical disk 1077 corresponding to each drive is used. A program or data can be read from the optical disk 1077 by the optical disk drive 1076 and provided to the main memory 1050 or the hard disk 1074 via the I / O controller 1070. Similarly, the tape medium 1071 corresponding to the tape drive 1072 can be used mainly for backup.

  The program provided to the information processing apparatus 1000 is provided by being stored in a recording medium such as the hard disk 1074, the optical disk 1077, or a memory card. The program may be installed in the information processing apparatus 1000 and executed by being read from the recording medium via the I / O controller 1070 or downloaded via the communication I / F 1040.

  The aforementioned program may be stored in an internal or external storage medium. Here, in addition to the hard disk 1074, the optical disk 1077, or the memory card, a magneto-optical recording medium such as an MD, or a tape medium 1071 can be used as a storage medium constituting the storage unit. Further, a storage device such as a hard disk 1074 or an optical disk library provided in a server system connected to a dedicated communication line or the Internet may be used as a recording medium, and the program may be provided to the information processing apparatus 1000 via the communication line.

  Here, the display device 1022 displays a screen for accepting data input to the server administrator, or displays a screen of a calculation processing result by the information processing device 1000. The cathode ray tube display device (CRT), liquid crystal A display device such as a display device (LCD) is included.

  Here, the input means accepts input by the server administrator, and may be constituted by a keyboard, a mouse 1100, and the like.

  The communication I / F 1040 is a network adapter that enables the information processing apparatus 1000 to be connected to a terminal via a dedicated network or a public network. The communication I / F 1040 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter.

  In the above example, the information processing apparatus 1000 has been mainly described. However, the functions described above can also be realized by installing a program in a computer and causing the computer to operate as the information processing apparatus 1000. Therefore, the functions realized by the information processing apparatus 1000 described as an embodiment in the present invention are executed by executing the above-described method by the computer, or by introducing the above-described program into the computer and executing it. Is also feasible.

  In the case where the apparatuses 10 and 10A according to the embodiments of the present invention or the method used for controlling these apparatuses are realized by a program on a computer, the storage medium for storing the program is electronic, magnetic, or optical. It can be a mechanical, electromagnetic, infrared or semiconductor system (or apparatus or equipment) or a propagation medium. Examples of this storage medium include semiconductor or solid state storage devices, magnetic tape, removable computer readable media examples include semiconductor or solid state storage devices, magnetic tape, removable flexible disks, random disks, Includes access memory (RAM), read only memory (ROM), rigid magnetic disk and optical disk. Examples of optical disks at the present time include compact disk read only memory (CD-ROM), compact disk read / write (CD-R / W) and DVD.

  As mentioned above, although embodiment of this invention was described, it only showed the specific example and does not specifically limit this invention. Further, the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to the effects described in the embodiments of the present invention.

It is a figure which shows schematic structure etc. of the information provision apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows a user terminal. It is a figure which shows an example of the moving image displayed on a user terminal. It is a figure which shows an example of the moving image displayed on a user terminal. It is a schematic block diagram which shows the structure of an image dictionary production | generation apparatus. It is a figure which shows an example of an image tag table. It is a conceptual diagram which shows an example of a partial image dictionary. It is a block diagram which shows schematic structure of a search device. It is a figure which shows an example of the search result displayed on the liquid crystal screen of a user terminal. It is a block diagram which shows schematic structure etc. of a motion information provision apparatus. It is a figure which shows an example of a moving image. It is a block diagram which shows schematic structure of an advertisement matching apparatus. It is the schematic which shows a user terminal. It is a block diagram etc. which show schematic structure of a bookmark production | generation apparatus. It is an example of an image displayed on a liquid crystal screen of a user terminal. It is a schematic flowchart which shows the operation example of an apparatus. It is a schematic flowchart which shows the operation example of an apparatus. It is a schematic flowchart which shows the operation example of an apparatus. It is a schematic block diagram which shows the advertisement matching apparatus of an apparatus. It is an example of a screen of a user terminal. It is a figure which shows the hardware structural example of an information provision apparatus.

DESCRIPTION OF SYMBOLS 10,10A Information provision apparatus 20 Image dictionary production | generation apparatus 21 Content acquisition part 22 Content clustering part 23 Partial image acquisition part 24 Partial image clustering part 25 Feature provision part 26 Tag provision part 40 Search apparatus 41 Query reception part 42 Search processing part 42a Image Acquisition unit 42b Feature detection unit 43c Image search unit 42d Related information extraction unit 43 Search result generation unit 44 Search result transmission unit 45 Search log recording unit 60 Motion information addition device 61 Image acquisition unit 62 In-image position acquisition unit 63 Motion extraction unit 63a Forward feed unit 63b Reverse feed unit 64 Motion evaluation unit 80, 80A Advertisement matching device 81 Scene position acquisition unit 82, 82A Advertisement matching unit 100 Bookmark generation device 101 Bookmark registration unit 102 Common image display unit 103 Common image count unit

Claims (3)

An information providing apparatus that provides information in response to a request from a user terminal,
In a plurality of whole images (frames) constituting the moving image content, the feature information of the partial image including a identifiable object that is a part of the whole image and the temporal position of the partial image in the whole moving image content are indicated. A position in the scene which is information, a position in the image indicating coordinates where the partial image is located in the whole image, a motion vector which is information indicating movement of the partial image in the whole image, and the whole An image search database for associating and storing a link to the video content including the image;
A partial image receiving unit for receiving the partial image as a search key from the user terminal;
A feature detector for detecting features of the partial image received from the user terminal;
Feature information extraction that extracts the feature information correlated with the feature of the partial image received from the user terminal from the image search database by using the position in the scene, the position in the image, and the motion vector in combination. And
A related information transmission unit that transmits the link corresponding to the feature information extracted by the feature information extraction unit to the user terminal;
Based on the position in the scene corresponding to the feature information extracted by the feature information extraction unit, a scene in which the partial image is displayed on the user terminal in the moving image content indicated by the link corresponding to the feature information. Determining and displaying an advertisement corresponding to the partial image on the user terminal only while the information is displayed;
Based on the position in the image corresponding to the feature information extracted by the feature information extraction unit and the motion vector, an advertisement is displayed according to the motion of the partial image in the moving image content indicated by the link corresponding to the feature information. An information providing apparatus comprising: an advertisement distribution unit that determines a scene to be displayed and displays an advertisement corresponding to the partial image on the user terminal based on the determination . In a plurality of whole images (frames) constituting the moving image content, the feature information of the partial image including a identifiable object that is a part of the whole image and the temporal position of the partial image in the whole moving image content are indicated. A position in the scene which is information, a position in the image indicating coordinates where the partial image is located in the whole image, a motion vector which is information indicating movement of the partial image in the whole image, and the whole An information providing method for providing information in response to a request from a user terminal by an information providing apparatus having an image search database that stores and associates a link to the moving image content including an image,
A partial image receiving step of receiving the partial image from the user terminal as a search key;
A feature detection step of detecting a feature of the partial image received from the user terminal;
Feature information extraction step for extracting feature information correlated with the feature of the partial image received from the user terminal from the image search database by using the position in the scene, the position in the image, and the motion vector in combination. When,
A related information transmission step of transmitting the link corresponding to the feature information extracted in the feature information extraction step to the user terminal;
A scene in which the partial image is displayed on the user terminal in the moving image content indicated by the link corresponding to the feature information based on the position in the scene corresponding to the feature information extracted by the feature information extraction step. Determining and displaying an advertisement corresponding to the partial image on the user terminal only while the information is displayed;
Based on the position in the image corresponding to the feature information extracted by the feature information extraction step and the motion vector, an advertisement is displayed according to the motion of the partial image in the moving image content indicated by the link corresponding to the feature information. An information providing method comprising: determining a scene to be displayed, and displaying an advertisement corresponding to the partial image on the user terminal based on the determination . In a plurality of whole images (frames) constituting the moving image content, the computer is characterized by the partial image feature information including a part of the whole image and an identifiable object, and the partial image temporal information in the whole moving image content. A position in the scene that is information indicating the position, a position in the image that indicates coordinates in which the partial image is located in the entire image, and a motion vector that is information indicating the movement of the partial image in the entire image. A computer program for causing an information providing apparatus having an image search database to store a link to the moving image content including the entire image in association with each other,
A partial image receiving step of receiving the partial image from the user terminal as a search key;
A feature detection step of detecting a feature of the partial image received from the user terminal;
Feature information extraction step for extracting feature information correlated with the feature of the partial image received from the user terminal from the image search database by using the position in the scene, the position in the image, and the motion vector in combination. When,
A related information transmitting step for transmitting to the user terminal the link corresponding to the feature information extracted in the feature information extracting step;
A scene in which the partial image is displayed on the user terminal in the moving image content indicated by the link corresponding to the feature information based on the position in the scene corresponding to the feature information extracted by the feature information extraction step. Determining and displaying an advertisement corresponding to the partial image on the user terminal only while the information is displayed;
Based on the position in the image corresponding to the feature information extracted by the feature information extraction step and the motion vector, an advertisement is displayed according to the motion of the partial image in the moving image content indicated by the link corresponding to the feature information. A computer program for determining a scene to be displayed and executing an advertisement distribution step for displaying an advertisement corresponding to the partial image on the user terminal based on the determination .

Images