JP2008146492A - Information providing device, information providing method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide advertisement information that matches the interests of a user, while eliminating the need for the user to make efforts to search for banners or store pages corresponding to a commodity that he wants to purchase. <P>SOLUTION: In an information providing device 10 for providing information in response to a request from a user terminal, an image dictionary creating device 20 has a content obtaining part for receiving animation composed of a plurality of overall images (frames), a partial image obtaining part for extracting partial images that constitute the overall images, and a feature imparting part for creating feature information by analyzing the partial images. A motion information imparting device 60 has a motion extracting part for extracting the motion vectors of the partial images, and a motion evaluating part for determining the time-based positions of the overall images in the animation that contain the partial images, and determining the time during which the partial images exist in succession, based on the motion vectors. An advertisement matching device 80 has an advertisement matching part or the like for displaying related advertisements on the user terminal in a way corresponding to the time during which the partial images received from the user terminal exist in succession. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information providing apparatus, an information providing method, and a computer program that provide information in response to a request from a user terminal in a computer network.

Sales promotion activities using so-called banner advertisements are widely performed on the Internet. For example, the banner advertisement is provided in a part of a web page, and when clicked by the user, another website linked to the banner advertisement is displayed. And the technique which enables it to order goods directly via such a banner advertisement is proposed (for example, patent document 1).
JP 2003-331189 A

  However, in the technique of Patent Document 1, if a banner for ordering a product that the user desires to purchase is not displayed on the display screen of the user's terminal device, the user eventually reaches the sales page desired to be purchased. There is a problem that it takes a long time to complete.

  Therefore, the present invention provides an information providing apparatus, an information providing method, and an information providing method capable of providing advertisement information that matches the user's interest without requiring the user to search for a banner or sales page corresponding to a product desired to be purchased. And to provide a computer program.

  (1) An information providing apparatus that provides information in response to a request from a user terminal, the image information acquiring unit receiving moving image information indicating a moving image composed of a plurality of whole images (frames), and the whole image A partial image acquisition unit that extracts a partial image constituting the entire image, a feature information generation unit that analyzes feature of the partial image and generates feature information, and a movement of the partial image in the entire image A motion extraction unit that extracts a motion vector; and a time position in the moving image of the whole image including the partial image is determined, and a time in which the partial image exists continuously in the moving image based on the motion vector And an advertisement related to the partial image is displayed on the user terminal corresponding to the time when the partial image received from the user terminal is continuously present. And ad matching unit which, with the characteristic information, and the motion vector, the information providing apparatus characterized by having an image search database that stores in association with links to the video information corresponding to the partial image.

  According to the invention of (1), the information providing apparatus can use the partial image as a search key and transmit a link to moving image information corresponding to the feature information of the partial image to the user terminal. That is, it can be used not as a character string but as a key for searching a partial image. And in the whole moving image, the advertisement related to the partial image can be displayed on the user terminal in correspondence with the time when the partial image input by the user as the search key exists. Since the partial image is an image selected by the user, the advertisement associated with the partial image should be of interest to the user. For this reason, it is possible to provide advertisement information suitable for the user's interest without requiring the user to search for a banner or sales page corresponding to the product desired to be purchased.

  (2) The advertisement matching unit continues to display the advertisement on the user terminal when it determines that the partial image will return within a predetermined time even when the partial image is out of the whole image. The information providing apparatus according to (1), wherein the information providing apparatus is displayed.

  According to the configuration described in (2), when a partial image temporarily deviates from the entire image, an advertisement corresponding to the partial image is continuously displayed on the user terminal. When a partial image temporarily deviates from the entire image, the user should concentrate on advertising information that suits that interest. For this reason, the advertising effect can be enhanced.

  (3) To the feature information indicating the feature of the partial image, the motion vector indicating the motion of the partial image in the moving image composed of a plurality of whole images (frames), and the moving image information indicating the moving image corresponding to the partial image An information providing method for providing information in response to a request from a user terminal by an information providing apparatus having an image search database for storing the link in association with an image information acquisition step of receiving the moving image information, A partial image acquisition step of extracting the partial image constituting the entire image from the entire image, a feature information generation step of analyzing features of the partial image to generate the feature information, and a movement of the partial image. A motion extraction step of extracting the motion vector; and determining a temporal position in the moving image of the entire image including the partial image. Based on the motion vector, the motion evaluation step of determining the time during which the partial images are continuously present in the moving image, and the time corresponding to the time during which the partial images received from the user terminal are continuously present, An information providing method comprising: an advertisement matching step for displaying an advertisement related to an image on the user terminal.

  According to the invention of (3), similar to the invention of (1), the advertisement information suitable for the user's interest is provided without requiring the user to search for a banner or a sales page corresponding to the product desired to be purchased. can do.

  (4) The computer indicates feature information indicating the characteristics of the partial image, a motion vector indicating the motion of the partial image in a moving image composed of a plurality of whole images (frames), and the moving image corresponding to the partial image. A computer program that functions as an information providing device that provides information in response to a request from a user terminal by an information providing device having an image search database that stores a link to moving image information in association with the video information. An image information acquisition step, a partial image acquisition step of extracting the partial image constituting the entire image from the entire image, and a feature information generation step of analyzing the characteristics of the partial image and generating the feature information A motion extraction step of extracting the motion vector, which is a motion of the partial image, and the partial image. A motion evaluation step of determining a time position of the whole image in the moving image and determining a time during which the partial image is continuously present in the moving image based on the motion vector; and the portion received from the user terminal An advertisement matching step of causing an advertisement related to the partial image to be displayed on the user terminal in response to a time when images are continuously present.

  According to the invention of (4), similar to the invention of (1), the advertisement information suitable for the user's interest is provided without requiring the user to search for a banner or a sales page corresponding to the product desired to be purchased. can do.

  According to the present invention, it is possible to provide an information providing apparatus, an information providing method, and a computer program that can provide information desired by a user without inputting an appropriate keyword.

[First Embodiment]
FIG. 1 is a diagram showing a schematic configuration and the like of an information providing apparatus 10 (hereinafter referred to as “apparatus 10”) according to the first embodiment of the present invention. FIG. 2 is a schematic diagram showing the user terminal 5 and the like. 3 and 4 are diagrams illustrating an example of a moving image displayed on the user terminal 5 or the like. As illustrated in FIG. 1, the device 10 includes an image dictionary generation device 20, a search device 40, a motion information addition device 60, an advertisement matching device 80, and a bookmark generation device 100. The device 10 can communicate with a plurality of user terminals 5, 5, user terminals 7, and the like via a communication line such as the Internet line 9.

  The image dictionary generation device 20 is a server that generates a search dictionary in which a partial image is associated with content such as a moving image including the partial image. The search key of the image dictionary generation device 20 is not a character string but a partial image. Hereinafter, the partial image input for searching is referred to as “search image”. Thus, the search image is not a whole image but a partial image. For example, an image including all of the A type automobile 200, the road 202, the sparrow 204, the tree 206, and the background 208 shown in FIG. 3C is referred to as an entire image (frame). Each image of the A type automobile 200 or the like is referred to as a partial image. A plurality of whole images (frames) constitute a content such as a moving image.

  The motion information adding device 60 is a server that determines the motion of a partial image, generates motion information, and adds the motion information to the partial image. The advertisement matching device 80 displays an advertisement such as an advertisement banner corresponding to the partial image on the display screen of the user terminal 5 or the like in response to the timing when the specific partial image is displayed on the display screen of the user terminal 5 or the like. It is a server that performs processing for. The bookmark generation device 100 is a server that performs a process of making a partial image a bookmark that can be commonly used in a plurality of user terminals 5 and the like. Corresponding to the search image received from the user terminal 5 or the like, the search device 40 extracts a link (URL or address) to the content from the image search DB 34 (see FIG. 5) of the image dictionary generation device 20, and the user terminal 5 It is a server for sending to etc. Note that the search device 40 may transmit related information in addition to the link to the content.

  As shown in FIG. 2A, the user terminal 5 is, for example, a mobile phone, and includes a liquid crystal screen 5a, an operation button group 5b, an enter button 5c, and a move button 5d. In addition to the general function, the movement button 5d is also used to move the cursor to a specific partial image in the entire image displayed on the liquid crystal screen 5a. In addition to a general function, the determination button 5c is also used to determine a partial image on which the cursor is moved by the movement button 5d as a search image. As shown in FIG. 2B, the user terminal 7 is, for example, a laptop personal computer, and includes a liquid crystal screen 7a, an operation button group 7b, and a mouse 7c. Various information is displayed on the liquid crystal screens 5a and 7a. For example, the moving images shown in FIGS. 3A to 3C and FIGS. 4A to 4C are also displayed. The images of the scenes 1 to 6 shown in FIGS. 3 and 4 are continuously displayed to form a series of moving images.

[Image Dictionary Generation Device 20]
FIG. 5 is a schematic block diagram illustrating the configuration of the image dictionary generation device 20.
As illustrated in FIG. 5, the image dictionary generation device 20 includes a content acquisition unit 21. The content acquisition unit 21 is a configuration for acquiring (receiving) various contents from the outside of the device 10 via the Internet line 9 and functions as an image information acquisition unit. The content acquisition unit 21 includes a computer program and hardware such as a CPU (Central Processing Unit) that controls the apparatus 10 based on the program, and includes an input unit and an output unit. Each unit described below is similarly configured by a computer program and hardware, and has an input unit and an output unit. The operation of the content acquisition unit 21 described above is an example of an image information acquisition step.

  The content includes text data and image data. The image data includes moving image data. The image dictionary generation device 20 includes a content clustering unit 22. The content clustering unit 22 is configured to classify content acquired from the outside into clusters (subsets) and store them in a content DB (database) 30 in a state where the classification method is machine-learned by so-called supervised learning. That is, the content clustering unit 22 is an automatic classifier. Each content has a unique URL. The content clustering unit 22 described above functions as an image information storage unit. The cluster is, for example, “a set of automobile-related movies”, “a set of bicycle-related movies”, “a set of music-related movies”, or the like.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a partial image acquisition unit 23. The partial image acquisition unit 23 acquires, for example, an image of the A type automobile 200 in FIG. 3C (hereinafter also simply referred to as “A type automobile 200”. The same applies to other partial images). It is a configuration. For example, the partial image acquisition unit 23 automatically extracts a partial image from content stored in the content DB 30. The partial image acquisition unit 23 displays the image shown in FIG. 3C on the user terminal 7, and when the user of the user terminal 7 moves the cursor to the A-type automobile 200 with the mouse 7 c and clicks, the partial image acquisition unit 23 passes the Internet line 9. The entire image and the partial image (for example, the A type automobile 200) included in the entire image may be received. The operation of the partial image acquisition unit 23 described above is an example of a partial image acquisition step.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a partial image clustering unit 24. The partial image clustering unit 24 is configured to perform machine learning of the classification method by so-called supervised learning, classify content acquired from the outside into clusters (subsets), and store the contents in the partial image DB 32. That is, the partial image clustering unit 24 is an automatic classifier and functions as a partial image storage unit. The cluster is, for example, “a set of cars”, “a set of bicycles”, “a set of ships”, or the like.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a feature adding unit 25. The feature adding unit 25 is a configuration for analyzing the features of the acquired partial image and generating feature information, and functions as a feature information generating unit. The feature adding unit 25 extracts, for example, the contour of the partial image and the number of used colors as features. The contour is extracted by edge processing. With these characteristics, even when the viewing angle of the same partial image (for example, Tokyo Tower) is different, it can be determined that the partial images are the same. The operation of the feature adding unit 25 described above is an example of a feature information generation step.

  As illustrated in FIG. 5, the image dictionary generation device 20 includes a tag adding unit 26. The tag adding unit 26 is configured to record the feature information extracted from the partial image as a tag (or index) in association with a link between the tag and the content stored in the content DB 30 in the image tag table. . The image tag table is stored in the image search DB 34. The operation of the tag adding unit 26 described above is an example of an image search basic information storing step.

  FIG. 6 is a diagram illustrating an example of an image tag table. As shown in FIG. 6, in the image tag table, tag ID, moving image ID (also referred to as image ID or content ID), feature information, clustering ID, link to content, position in scene, position in image (frame) ( Coordinates) and motion vectors. “Movie ID” is, for example, identification information of the entire movie shown in FIGS. 3 and 4. “Tag ID” is identification information of a partial image. In the present embodiment, feature information is used as a search key, and “tag ID” is not used as a search key. “Feature information” is information indicating the features of the partial image. The feature information indicates, for example, the outline of the automobile and the number of colors used in the automobile (5 colors in the example of FIG. 6). “Clustering ID” is identification information of a cluster (subset) to which each partial image belongs. “Link to content” is, for example, the address or URL of the entire moving image shown in FIGS. “In-scene position” is, for example, information indicating the temporal position of each scene in the entire moving image in FIGS. “In-image position (coordinates)” is, for example, information indicating the coordinates where the A-type automobile 200 is located in the entire image of FIG. The “motion vector” is information indicating the motion of the partial image in the entire image. The position in the scene, the position in the image, and the motion vector will be described in detail later.

  FIG. 7 is a conceptual diagram illustrating an example of a partial image dictionary. The partial image dictionary is a part of the above-described image tag table. As shown in FIG. 7, the partial image dictionary is configured by associating at least one content address with one piece of feature information. The content may be a video. The image search DB 34 is commonly used by each device included in the device 10.

[Searching device 40]
The device 10 refers to the image search DB 34 based on the search image received from the user terminal 5 or the like by the search device 40 and returns the URL of the content to the user terminal 5 or the like. The search device 40 may transmit information related to the search image as text information in addition to the content URL.

  FIG. 8 is a block diagram illustrating a schematic configuration of the search device 40. As illustrated in FIG. 8A, the search device 40 includes a query reception unit 41, a search processing unit 42, a search result generation unit 43, a search result transmission unit 44, and a search log recording unit 45. The query receiving unit 41 is configured to receive a search image from the user terminal 5 or the like. The search processing unit 42 is configured to search the image search DB 34 for tag IDs of partial images having characteristics that correlate with the search image. The search result generation unit 43 is configured to generate a search result for transmission to the user terminal 5 or the like. The search result transmission unit 44 is configured to transmit the search result to the user terminal 5 and the like, and functions as a related information transmission unit. The search log recording unit 45 is configured to generate a record in which each partial image is searched.

  As illustrated in FIG. 8B, the search processing unit 42 includes an image acquisition unit 42a, a feature detection unit 42b, an image search unit 42c, and a related information extraction unit 42d. The image acquisition unit 42a functions as a partial image receiving unit that receives a partial image as a search key. The feature detection unit 42b has a configuration for detecting features such as the contour and the number of colors of the received partial image. The image search unit 42c functions as an identification information extraction unit that extracts a tag ID. The related information extraction unit 42d is configured to extract related information related to the partial image searched based on the search image from the content DB 30. For example, when the apparatus 10 receives a photo of a dish from the user terminal 5 as a search image, the related information extraction unit 42d extracts the recipe for the dish from the content DB 30. The related information extracted by the related information extraction unit 42d is transmitted to the user terminal 5 or the like by the search result transmission unit 44 described above.

  FIG. 9 is a diagram illustrating an example of a search result displayed on the liquid crystal screen 5a of the user terminal 5 based on the data transmitted by the search result transmission unit 44 (see FIG. 7A). As shown in FIG. 9, for example, if the search image is “A type car”, at least one address of the home page where the A type car is displayed is displayed together with the image of the A type car. Yes. At this time, related information such as the performance such as the displacement of the A type automobile may be displayed at the same time.

[About the motion information adding device 60]
FIG. 10 is a block diagram illustrating a schematic configuration and the like of the motion information adding device 60. FIG. 11 is a diagram illustrating an example of a moving image. 11, FIG. 3C and FIG. 4A are used among the moving images of FIG. 3 and FIG. As illustrated in FIG. 10A, the motion information addition device 60 includes an image acquisition unit 61, an in-image position acquisition unit 62, a motion extraction unit 63, and a motion evaluation unit 64. The image acquisition unit 61 is a configuration for acquiring moving image data including partial images from the content DB 30 of the image dictionary generation device 20. The in-image position acquisition unit 62 coordinates (for example, the coordinates of a partial image (for example, the A type automobile 200 in FIG. 11 (a)) in a specific scene (for example, the entire image illustrated in FIG. 11 (a)) in the moving image. This is a program for acquiring the coordinates X1) in FIG.

  The motion extraction unit 63 is a configuration for extracting a motion vector of a partial image. For example, the A type automobile 200 in the scene 3 in FIG. 11A moves to the coordinate X2 in the scene 4 in FIG. 11B. In this case, for example, the motion extraction unit 63 tracks the center coordinates of the contour of the A-type automobile 200 corresponding to the initial coordinates X1, and stores a vector (motion vector) a1 indicating the movement from the coordinates X1 to the coordinates X2 in advance. It is generated at a prescribed time interval, for example, at an interval of 1/30 second. The motion vector a1 and the like indicate the motion of the A type automobile 200. The motion extraction unit 63 includes a forward feeding unit 63a and a backward feeding unit (rewinding unit) 63b, and uses the partial image in one scene in the entire moving image as a base point (hereinafter referred to as “initial coordinates”). A motion vector a1 in the direction and a motion vector ar1 in the reverse feed direction are generated. For this reason, even if a partial image in one scene of the entire moving image is used as the initial coordinates, motion vectors a1 and ar1 indicating all the movements of the specific partial image in the entire moving image are generated as shown in FIG. 11B. Can do. The motion extraction unit 63 stores the extracted motion vector in the image search DB 34. As described above, the apparatus 10 can extract the motion vector for the entire moving image regardless of the temporal position of the partial image to be used first.

  The motion evaluation unit 64 is configured to determine the timing at which the partial image deviates from the entire image based on the initial coordinates and the motion vector. In other words, the motion evaluation unit 64 is a configuration for determining a time in which a specific partial image is continuously present in the entire image constituting the moving image. The motion evaluation unit 64 determines the timing at which a specific partial image deviates from the entire image based on the size of the entire image (for example, 1280 pixels × 1024 pixels), the initial coordinates, and the motion vector. In the example of FIG. 10B, if the initial coordinate is the base of the vector a1, the motion evaluation unit 64 refers to the vectors a1, a2, and a3 in the forward feed direction, and performs partial processing in the next scene where the vector a3 is generated. It is determined that the image deviates from the entire image.

  Further, for example, as shown in FIG. 10C, the motion evaluation unit 64 once returns from the entire image in the scene in which the sparrow 204, which is a partial image, is generated in the scene where the vector b4 is generated. If it is, it is determined that a return to the entire image is assumed. The motion vector b1 or the like indicates the motion of the sparrow 204.

[Advertising matching device 80]
FIG. 12 is a block diagram illustrating a schematic configuration of the advertisement matching device 80. FIG. 13 is a schematic diagram showing the user terminal 5 and the like. As illustrated in FIG. 12, the advertisement matching device 80 includes a scene position acquisition unit 81 and an advertisement matching unit 82. For example, the scene position acquisition unit 81 determines a scene in which a specific partial image, for example, an A type automobile 200 is displayed, from the scenes 1 to 9 in FIG. In the example of FIG. 13B, it is determined that the A type automobile 200 is displayed in the scenes 2 to 4.

  The advertisement matching unit 82 supports the A type automobile 200 only while the specific partial image, for example, the A type automobile 200 is displayed based on the motion vector generated in advance for the entire moving image including the scenes 1 to 9. The banner advertisement B to be displayed (see FIG. 13A) is displayed on the liquid crystal screen 5a of the user terminal 5. As shown in FIG. 13A, scenes 1 to 9 are displayed as the moving image A on the liquid crystal screen 5a, but the banner advertisement B is displayed only during the scenes 2 to 4 in which the A type automobile 200 is displayed. It is displayed.

  On the other hand, as shown in FIG. 13C, the advertisement matching unit 82 generates a specific partial image (for example, sparrow 204) based on a motion vector generated in advance for the entire moving image including the scenes 1 to 9. If it is determined that the image has been removed from the entire image on the assumption that it will return, as shown in FIG. 13C, the banner advertisement C corresponding to the sparrow 204 is continuously displayed even in the scene 4 where the sparrow 204 is not displayed. Let

[About the bookmark generator 100]
FIG. 14 is a block diagram showing a schematic configuration of the bookmark generating apparatus 100. FIG. 15 is an example of an image displayed on the liquid crystal screen 5 a of the user terminal 5. As illustrated in FIG. 14A, the bookmark generation device 100 includes a bookmark registration unit 101, a common image display unit 102, a common image count unit 103, and a bookmark DB 104.

  The bookmark registration unit 101 is configured to store the user ID, the image ID of the partial image, and the description page URL in association with each other in the bookmark DB 104 when receiving a partial image from the user terminal 5, for example. Functions as a bookmark registration unit. The partial image received from the user terminal 5 may be a moving image. As shown in FIG. 14B, the user ID, the image ID, and the explanation page URL are stored in the personal table. The personal table is set for each user in the bookmark DB 104.

  The bookmark registration unit 101 is also configured to store the bookmarks stored in the personal table in the common table shown in FIG. The common table is also set in the bookmark DB 104. In the common table, various types of information are stored in association with tag IDs. The operation of the bookmark registration unit 101 described above is an example of a bookmark registration step.

  The common image display unit 102 transmits the partial image data stored in the common table to the user terminal 5 or the like that has accessed a specific homepage (hereinafter referred to as “bookmark page”), as shown in FIG. Thus, it is the structure for displaying those partial images on the liquid crystal screen 5a, and functions as a bookmark page transmission unit. As described above, the partial images stored in the common table can be freely browsed by all the viewers of a specific homepage (hereinafter referred to as “bookmark page”), for example. As shown in FIG. 15, a plurality of partial images are displayed on the “bookmark page” as bookmarks. Some of the partial images may be moving images. The operation of the common image display unit 102 described above is an example of a bookmark page transmission step.

  The common image counting unit 103 counts the number of times that a user such as the user terminal 5 who has browsed the bookmark page clicks a specific bookmark, and the common count is “point” as shown in FIG. This is a configuration for recording in a table.

  Then, as shown in FIG. 15B, the search device 40 described above displays an explanatory text (including a link to the content) corresponding to the partial image (bookmark) with many points at the top of the search result page. However, it is displayed so that the user can easily see it. For example, when a user uses a partial image of a car as a search key and the search device 40 extracts a plurality of content URLs, the URL of a page including a bookmark with a large number of counts is preferentially displayed. Note that the search key image used by the user does not always match the bookmark (partial image) that determines the priority display order.

The above is the configuration of the apparatus 10. As described above, the device 10 can use a partial image as a key for searching for feature information, and transmit a link to content including the partial image corresponding to the feature information to the user terminal. That is, it can be used not as a character string but as a key for searching a partial image. For this reason, even if an appropriate keyword is not input, information desired by the user can be provided.
Further, since the apparatus 10 can determine the time during which the partial image input by the user as a search key exists in the entire moving image, only the portion where the partial image input by the user exists can be transmitted to the user terminal. it can.

  Further, the apparatus 10 can display, on the user terminal, an advertisement related to the partial image corresponding to the time when the partial image input by the user as a search key exists in the entire moving image. Since the partial image is an image selected by the user, the advertisement associated with the partial image should be of interest to the user. For this reason, it is possible to provide advertisement information suitable for the user's interest without requiring the user to search for a banner or sales page corresponding to the product desired to be purchased.

  Furthermore, when the partial image temporarily deviates from the entire image, the device 10 continuously displays an advertisement corresponding to the partial image on the user terminal. When a partial image temporarily deviates from the entire image, the user should concentrate on advertising information that suits that interest. For this reason, the advertising effect can be enhanced.

Moreover, the apparatus 10 can display a partial image as a bookmark on a plurality of user terminals. That is, it is possible to display a social bookmark that can be understood at a glance. Furthermore, the number of times each bookmark is selected by the user terminal can be counted, and bookmarks with a large number of counts can be preferentially displayed on the user terminal, so that popular bookmarks can be preferentially displayed. it can.
Furthermore, since a bookmark can be displayed as a moving image on a plurality of the user terminals, a bookmark that is easy to attract the user's interest as a moving image can be displayed.

  Hereinafter, an operation example of the apparatus 10 will be described with reference to FIGS. 16, 17, and 18. 16, 17, and 18 are schematic flowcharts illustrating an operation example of the apparatus 10. First, an operation example in which the apparatus 10 receives a partial image from the user terminal 5 or the like and transmits a search result to the user terminal 5 or the like will be described mainly using FIG.

  First, the device 10 receives a search image (partial image) from the user terminal 5 (step S1 in FIG. 16). This step S1 is an example of a partial image receiving step. Then, the feature of the search image is detected (step S2), and feature information correlated with the feature of the search image is extracted from the image tag table (see FIG. 6) (step S3). Step S2 described above is an example of a feature detection step, and step S3 is an example of a feature information extraction step.

  Subsequently, the device 10 extracts a content URL corresponding to the feature information extracted from the image tag table (step S4), and transmits the extracted content URL to the user terminal 5 (step S5). This step S5 is an example of a related information transmission step.

  Next, mainly using FIG. 17, an advertisement banner corresponding to the partial image is displayed on the liquid crystal screen 5 a of the user terminal 5 or the like corresponding to the timing at which the search image (partial image) is displayed. An example of the operation will be described.

  First, Steps S101 to S104 in FIG. 17 are the same as Steps S1 to S4 described above, and thus description thereof is omitted. Subsequent to step S104, the apparatus 10 evaluates the movement of the partial image corresponding to the extracted feature information, and determines the timing of deviating from the entire image (step S105). Then, an advertisement banner corresponding to the partial image is associated with the scene where the partial image is displayed (step S106).

  Subsequently, the apparatus 10 transmits the extracted content URL to the user terminal 5 (step S107). When the user terminal 5 accesses the content URL, a moving image is displayed on the liquid crystal screen 5a of the user terminal 5, and an advertisement banner corresponding to the partial image is displayed only while the partial image corresponding to the extracted feature information is displayed. Is displayed (step S108).

  Next, an operation example of the apparatus 10 when the user terminal 5 or the like selects a partial image from the bookmark page will be described mainly using FIG.

  First, the apparatus 10 receives a bookmark page browsing request from the user terminal 5 (step S201 in FIG. 18). Subsequently, a bookmark page is displayed on the user terminal 50 (step S202). When a specific bookmark (partial image) is designated from the user terminal 5 (step S203), the feature of the partial image is detected, and the feature and correlation between the feature of the partial image are detected from the image tag table (see FIG. 6). Certain feature information is extracted (step S204). Then, a link to the content including the partial image corresponding to the extracted feature information is extracted from the image tag table (see FIG. 6) (step S205). Then, the movement of the designated partial image is evaluated, and the timing for deviating from the entire image is determined (step S206).

  Subsequently, the apparatus 10 performs Step S207, Step S208, and Step S209. Steps S207 to S209 are the same as steps S106 to S108 described above, and thus description thereof is omitted.

  Subsequent to step S209, the apparatus 10 adds points (see FIG. 14C) of the designated partial image (step S210), and maintains or updates the display order of the search result page (step S211).

[Second Embodiment]
Next, a second embodiment will be described with reference to FIGS. FIG. 19 is a schematic block diagram showing an advertisement matching device 80A of the device 10A (see FIG. 1) of the second embodiment. FIG. 20 is a screen example of the user terminal 5 or the like. In addition to the function of the device 10 of the first embodiment, the device 10A (see FIG. 1) of the second embodiment can change the display position of the banner advertisement in accordance with the movement of the partial image.

  As illustrated in FIG. 20A, the device 10 </ b> A displays the moving image A on the liquid crystal screen 5 a of the user terminal 5 and displays banner advertisements B <b> 1 to B <b> 4 on any of the surroundings. As shown in FIG. 20B, the advertisement matching unit 82A of the device 10A displays the banner advertisement B2 on the left side of the video A when the motion vector of the specific partial image is S1 and deviates from the video A to the left. Display. On the other hand, as shown in FIG. 20C, the advertisement matching unit 82A displays a banner advertisement on the upper side of the video A when the motion vector of the specific partial image is S2 and deviates from the upper side of the video A. B3 is displayed.

[Programs and computer-readable recording media]
A computer program for causing a computer to execute the partial image reception step, the feature detection step, the feature information extraction step, the related information transmission step, and the like in the above-described operation example can be used. Moreover, it can also be set as the computer-readable recording medium which recorded such a computer program etc.

[Hardware Configuration of Information Providing Device 10]
FIG. 21 is a diagram illustrating a hardware configuration example of the devices 10 and 10A (hereinafter referred to as the information processing device 1000). The information processing apparatus 1000 includes a CPU (Central Processing Unit) 1010 (a plurality of CPUs such as the CPU 1012 may be added in a multiprocessor configuration), a bus line 1005, a communication I / F 1040, and a main memory 1050 that configure each control unit. , A BIOS (Basic Input Output System) 1060, a USB port 1090, an I / O controller 1070, an input means such as a keyboard and mouse 1100, and a display device 1022.

  Storage means such as a tape drive 1072, a hard disk 1074, an optical disk drive 1076, and a semiconductor memory 1078 can be connected to the I / O controller 1070.

  The BIOS 1060 stores a boot program executed by the CPU 1010 when the information processing apparatus 1000 is activated, a program depending on the hardware of the information processing apparatus 1000, and the like.

  The hard disk 1074 constituting the storage unit stores various programs for the information processing apparatus 1000 to function and programs for executing the functions of the present invention, and can configure various databases as necessary.

  As the optical disc drive 1076, for example, a DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive, or a CD-RAM drive can be used. In this case, the optical disk 1077 corresponding to each drive is used. A program or data can be read from the optical disk 1077 by the optical disk drive 1076 and provided to the main memory 1050 or the hard disk 1074 via the I / O controller 1070. Similarly, the tape medium 1071 corresponding to the tape drive 1072 can be used mainly for backup.

  The program provided to the information processing apparatus 1000 is provided by being stored in a recording medium such as the hard disk 1074, the optical disk 1077, or a memory card. The program may be installed in the information processing apparatus 1000 and executed by being read from the recording medium via the I / O controller 1070 or downloaded via the communication I / F 1040.

  The aforementioned program may be stored in an internal or external storage medium. Here, in addition to the hard disk 1074, the optical disk 1077, or the memory card, a magneto-optical recording medium such as an MD, or a tape medium 1071 can be used as a storage medium constituting the storage unit. Further, a storage device such as a hard disk 1074 or an optical disk library provided in a server system connected to a dedicated communication line or the Internet may be used as a recording medium, and the program may be provided to the information processing apparatus 1000 via the communication line.

  Here, the display device 1022 displays a screen for accepting data input to the server administrator, or displays a screen of a calculation processing result by the information processing device 1000. The display device 1022 is a cathode ray tube display (CRT), liquid crystal display, or the like. A display device such as a display device (LCD) is included.

  Here, the input means accepts input by the server administrator, and may be constituted by a keyboard, a mouse 1100, and the like.

  The communication I / F 1040 is a network adapter that enables the information processing apparatus 1000 to be connected to a terminal via a dedicated network or a public network. The communication I / F 1040 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter.

  In the above example, the information processing apparatus 1000 has been mainly described. However, the functions described above can also be realized by installing a program in a computer and causing the computer to operate as the information processing apparatus 1000. Therefore, the functions realized by the information processing apparatus 1000 described as an embodiment in the present invention are executed by executing the above-described method by the computer, or by introducing the above-described program into the computer and executing it. Is also feasible.

  In the case where the apparatuses 10 and 10A according to the embodiments of the present invention or the method used for controlling these apparatuses are realized by a program on a computer, the storage medium for storing the program is electronic, magnetic, or optical. It can be a mechanical, electromagnetic, infrared or semiconductor system (or apparatus or equipment) or a propagation medium. Examples of this storage medium include semiconductor or solid state storage devices, magnetic tape, removable computer readable media examples include semiconductor or solid state storage devices, magnetic tape, removable flexible disks, random Includes access memory (RAM), read only memory (ROM), rigid magnetic disk and optical disk. Examples of optical disks at the present time include compact disk read only memory (CD-ROM), compact disk read / write (CD-R / W) and DVD.

  As mentioned above, although embodiment of this invention was described, it only showed the specific example and does not specifically limit this invention. Further, the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to the effects described in the embodiments of the present invention.

It is a figure which shows schematic structure etc. of the information provision apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows a user terminal. It is a figure which shows an example of the moving image displayed on a user terminal. It is a figure which shows an example of the moving image displayed on a user terminal. It is a schematic block diagram which shows the structure of an image dictionary production | generation apparatus. It is a figure which shows an example of an image tag table. It is a conceptual diagram which shows an example of a partial image dictionary. It is a block diagram which shows schematic structure of a search device. It is a figure which shows an example of the search result displayed on the liquid crystal screen of a user terminal. It is a block diagram which shows schematic structure etc. of a motion information provision apparatus. It is a figure which shows an example of a moving image. It is a block diagram which shows schematic structure of an advertisement matching apparatus. It is the schematic which shows a user terminal. It is a block diagram etc. which show schematic structure of a bookmark production | generation apparatus. It is an example of an image displayed on a liquid crystal screen of a user terminal. It is a schematic flowchart which shows the operation example of an apparatus. It is a schematic flowchart which shows the operation example of an apparatus. It is a schematic flowchart which shows the operation example of an apparatus. It is a schematic block diagram which shows the advertisement matching apparatus of an apparatus. It is an example of a screen of a user terminal. It is a figure which shows the hardware structural example of an information provision apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,10A Information provision apparatus 20 Image dictionary production | generation apparatus 21 Content acquisition part 22 Content clustering part 23 Partial image acquisition part 24 Partial image clustering part 25 Feature provision part 26 Tag provision part 40 Search apparatus 41 Query reception part 42 Search processing part 42a Image Acquisition unit 42b Feature detection unit 43c Image search unit 42d Related information extraction unit 43 Search result generation unit 44 Search result transmission unit 45 Search log recording unit 60 Motion information addition device 61 Image acquisition unit 62 In-image position acquisition unit 63 Motion extraction unit 63a Forward feed unit 63b Reverse feed unit 64 Motion evaluation unit 80, 80A Advertisement matching device 81 Scene position acquisition unit 82, 82A Advertisement matching unit 100 Bookmark generation device 101 Bookmark registration unit 102 Common image display unit 103 Common image count unit

Claims (4)

An information providing apparatus that provides information in response to a request from a user terminal,
An image information acquisition unit that receives moving image information indicating a moving image composed of a plurality of whole images (frames);
A partial image obtaining unit that extracts a partial image constituting the whole image from the whole image;
A feature information generating unit that analyzes the features of the partial image and generates feature information;
A motion extraction unit that extracts a motion vector that is a motion of the partial image in the entire image;
A motion evaluation unit that determines a temporal position in the moving image of the entire image including the partial image, and determines a time during which the partial image continuously exists in the moving image based on the motion vector;
An advertisement matching unit that displays an advertisement related to the partial image on the user terminal corresponding to a time in which the partial images received from the user terminal are continuously present;
An image search database for storing the feature information, the motion vector, and a link to the moving image information corresponding to the partial image;
An information providing apparatus comprising:   The advertisement matching unit causes the user terminal to continuously display the advertisement when it determines that the partial image returns from within the predetermined time even when the partial image is out of the whole image. The information providing apparatus according to claim 1. Feature information indicating the characteristics of the partial image, a motion vector indicating the motion of the partial image in a video composed of a plurality of whole images (frames), and a link to video information indicating the video corresponding to the partial image; An information providing method for providing information in response to a request from a user terminal by an information providing apparatus having an image search database that stores the information in association with each other,
An image information acquisition step for receiving the video information;
A partial image obtaining step of extracting the partial image constituting the whole image from the whole image;
A feature information generation step of analyzing the features of the partial image and generating the feature information;
A motion extraction step of extracting the motion vector which is a motion of the partial image;
A motion evaluation step of determining a temporal position in the moving image of the entire image including the partial image, and determining a time during which the partial image continuously exists in the moving image based on the motion vector;
An advertisement matching step for displaying an advertisement related to the partial image on the user terminal corresponding to a time when the partial images received from the user terminal are continuously present; and
An information providing method characterized by comprising: To the computer, the feature information indicating the characteristics of the partial image, the motion vector indicating the motion of the partial image in the moving image composed of a plurality of whole images (frames), and the moving image information indicating the moving image corresponding to the partial image. A computer program that functions as an information providing device that provides information in response to a request from a user terminal by an information providing device having an image search database that stores the link in association with
An image information acquisition step for receiving the video information;
A partial image obtaining step of extracting the partial image constituting the whole image from the whole image;
A feature information generation step of analyzing the features of the partial image and generating the feature information;
A motion extraction step of extracting the motion vector which is a motion of the partial image;
A motion evaluation step of determining a temporal position in the moving image of the entire image including the partial image, and determining a time during which the partial image continuously exists in the moving image based on the motion vector;
An advertisement matching step for displaying an advertisement related to the partial image on the user terminal corresponding to a time when the partial images received from the user terminal are continuously present; and
A computer program for executing

Images