JP2010200170A - Image information providing system, image information providing method, and image information providing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To promptly provide information related to a video object displayed on a screen in accordance with a request. <P>SOLUTION: An image information providing system includes: a television video receiving terminal 4 including an output screen for displaying a broadcasted program video image and a designation means for designating a video object included in a program video image and displayed on the output screen; a database server 1 for storing the broadcasted program video image; a database server 3 storing video object information preset correspondingly to the video object; and a video object specification device server 2 for specifying the designated video object by referring to the database server 1 based on time information at the time when the television video receiving terminal 4 designates the video object and designated position information indicating a designated position in the output screen, acquiring from the database server 3 the video object information related to the specified video object, and providing the video object information to the television video receiving terminal 4. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technology of a video receiving terminal device in a digital broadcasting service.

Product advertisements and advertisements have been made through stores, street posters, the Internet (Web advertisements, etc.), newspapers and magazines, TV commercials, TV shopping, and the like.
However, only users (customers) who are interested in the product originally visited the storefront, and it was not effective in the sense of a wide advertising effect to the public, and could not be said to be an effective advertising technique.

In addition, in product advertisements by street posters, newspapers, magazines, and TV commercials, even if the masses acquire information on products they are interested in, it is not possible to immediately purchase products, so that the user's interest can be sustained. was difficult.
Internet web advertisements allow users to purchase merchandise as is, but in this regard, only users who are originally interested in the merchandise often access web advertisements, which means a wide advertising effect to the public. The effect is considered to be low.

  In addition, regarding TV shopping using the TV program itself as an advertisement, it is possible to advertise a specific product, but the target of promotion is limited to the product set at the time of program production.

As a related technique for this, a method of providing object information corresponding to each video object (object) displayed in a television program is disclosed (Patent Document 1).
As a related technique, a receiving terminal device is disclosed that identifies a product displayed during a program broadcast and presents product information about the product at an arbitrary time after the program broadcast ends (Patent Document 2). ).

2005-244878 WO00-28737

However, in the related technique disclosed in Patent Document 2, it is necessary to link the video material information and the video material displayed in the program at the time of program production, and there is a disadvantage that the work man-hour becomes enormous.
For this reason, it is only possible to advertise specific objects, and it is difficult to efficiently advertise because it is not possible to acquire or purchase information on a single product (video object) that the user is interested in. It was.
Also, in the related technology disclosed in Patent Document 2, when providing arbitrary product information (object information), it is necessary to search for specified product information from an enormous amount of data. Inconveniently, it takes a long time to specify information, or it is not possible to narrow down products to be searched.

[Object of invention]
The present invention provides an image information providing system and an image information providing method capable of improving the inconveniences of the related technology and quickly providing information on a video object projected on a television program broadcast screen in response to a request from a television viewer. It is an object of the present invention to provide an image information providing program.

  To achieve the above object, an image information providing system according to the present invention comprises a video receiving terminal having an output screen for displaying a broadcast program video, and a designation means for designating a video object displayed on the output screen; An image information providing system comprising: a program video storage device for storing the broadcast program video; and a video product information storage device for storing video product information set in advance corresponding to each video product. Obtaining the time information indicating the time when the video receiving terminal has specified the video object and the specified position information indicating the specified position in the output screen, and the program video based on the time information and the specified position information A video object specifying device for specifying the specified video object with reference to a program video stored in a storage device, wherein the video object specifying device is identified from the video object information storage device; It has taken a configuration in which a video object information providing means for providing obtains the video object information according to the image object that is in the image receiving terminal.

  According to another aspect of the present invention, there is provided an image information providing method, a video receiving terminal having an output screen for displaying a broadcast program video and a designating means for designating a video object displayed on the output screen, and storing the program video. An image information providing system comprising a program video storage device and a video product information storage device storing video product information set in advance corresponding to each video product, in response to a request from the video receiving terminal According to another aspect of the present invention, there is provided an image information providing method for providing the video object information, the time information indicating the time when the video receiving terminal specifies the video object and the specified position information indicating the position specified in the output screen. And specifying the specified video object with reference to the program video stored in the program video storage device based on the time information and the specified position information, and specifying the specified video object from the video object information storage device. Acquiring the image object information according to the image object, and the image object information and transmits to the image receiving terminal.

  Furthermore, an image information providing program according to the present invention stores a video receiving terminal having an output screen for displaying a broadcast program video, a designation means for designating a video object displayed on the output screen, and the program video. An image information providing system comprising a program video storage device and a video product information storage device storing video product information set in advance corresponding to each video product, in response to a request from the video receiving terminal. An image information providing program for providing the video object information according to the time information indicating the time when the video receiving terminal specifies the video object and the specified position information indicating the position specified in the output screen. A specified video object specifying function for specifying the specified video object with reference to the program video storage device, and a video object related to the specified video object from the video object information storage device The video object information acquiring function of performing processing for obtaining distribution is characterized by causing a computer to execute provided in advance parallel in the video object information storage unit.

  Since the present invention is configured and functions as described above, according to this, the image specified based on the time information indicating the specified time of the image object in the output screen and the position information indicating the specified position on the output screen. Information related to the video object on the TV program broadcast screen arbitrarily designated by the video receiving terminal by means of a configuration comprising means for identifying the object and means for acquiring the video object information relating to the specified video object It is possible to provide an image information providing system, an image information providing method, and an image information providing program that can specify and provide the information accurately and quickly upon request.

It is a schematic block diagram which shows one Embodiment of the image information provision system by this invention. It is explanatory drawing which shows an example of the display content in the image information provision system disclosed in FIG. It is explanatory drawing which shows an example of the data structure of the database 3 in the image information provision system disclosed in FIG. It is explanatory drawing which shows an example of the method of the object extraction and selection in the image information provision system disclosed in FIG. It is explanatory drawing which shows an example of the method of the object extraction and selection in the image information provision system disclosed in FIG. It is a sequence chart which shows the whole operation | movement in the image information provision system disclosed in FIG.

[Embodiment]
Next, the basic configuration content of the embodiment of the present invention will be described.

  As shown in FIG. 1, an image information providing system according to the present embodiment includes a base station 5 that distributes TV program video, and a database server (program video storage device) that stores and holds TV video information transmitted from the base station 5. ) 1, a server (video object specifying device) 2 for acquiring image information (image frame) in response to a request sent via the network from the TV video information stored in the database server 1, and TV video information A database (video object information storage device) 3 for storing video object information stored corresponding to the included video object, and TV video information broadcast from the base station 5 are output and displayed, and the video is displayed on the server 2. It is configured to include a television image receiving terminal 4 that can be connected to the Internet for requesting physical information.

  As a result, the user can acquire information (object information) related to an arbitrary video object selected and designated during viewing of the program by using the television video receiving terminal 4.

  It is also possible to integrate the two database servers (database servers 1 and 3) shown in this embodiment into one. Further, the server may also be configured to include the server 2 that is an application and presentation server. Furthermore, it is good also as a structure which utilizes the home server provided in each household etc. as the database server 1 or the server 2. FIG.

  The database server 1, the server 2, the database server 3, and the television video receiving terminal 4 are respectively a large-capacity recording medium such as a hard disk drive (HD) and a DVD-RAM, a RAM and a ROM, and a CPU (Central Processing Unit). ) And the like, and the central processing unit performs various processing functions by performing processing based on a preset program.

  The server 2 includes user authentication based on user information sent from the TV video receiving terminal 4 (user authentication function) and an interface (IF) providing function for providing an interface to the TV video receiving terminal 4. Yes.

  However, at this time, actual data exchange is not performed between the television video receiving terminal and the server 2, and the television video receiving terminal 4 receives presentation information from the server 2.

The actual data information is not transmitted from the server 2 to the television video receiving terminal 4, and the television video receiving terminal 4 receives only presentation information from the server 2 as a thin client, and a screen based on this presentation information. Output and display information.
Thereby, the television image received by the television image receiving terminal 4 from the base station 5 is displayed on the interface provided by the server 2 (FIG. 2).

Further, the server 2 obtains a frame image (hereinafter referred to as “frame”) of the corresponding program from the database server 1 based on the date information, time information, and channel information received from the television video receiving terminal 4. A frame image specific acquisition function is provided.
Note that this frame indicates a frame image group constituting a video (moving image), and it is assumed that a moving image for one second is composed of 30 frames. In addition, the number of frames acquired for a single request from the server 2 to the database server 1 is, by default, 90 frames for a total of 3 seconds corresponding to time information ± 1.5 seconds.

Further, the server 2 has an image cutout function for trimming a frame acquired from the database server 1 based on the position time information received from the television video receiving terminal 4 and cutting out a part of the frame image.
Thereby, the range of the image to be compared can be limited to the range of the clipped image, the accuracy of image identification and object extraction can be improved, and the processing can be speeded up.

  In the server 2 of this embodiment, as shown in FIG. 4, ± 0.15 in the X direction and ± 0.1 in the Y direction based on the position information (X value, Y value) received from the television image receiving device 4. It is assumed that the image cutout function is performed in an area of 0.1. It is assumed that this trimming operation is executed for each acquired frame.

  Further, the server 2 is provided with an image comparison processing function for sequentially numbering 1 to 90 for each trimmed frame and comparing the images. Note that this image comparison processing is executed between frames adjacent to the numbered numbers as shown in FIG.

  Further, the server 2 acquires a frame before trimming of a frame stored as a candidate, determines a background and an object, and performs a trimming range (X ± 0.15 and Y ± 0.1 based on position information). A candidate object extracting function for extracting an object included in (image) as a candidate object.

Further, the server 2 has a color feature information calculation function for calculating color information and feature information of an object (candidate object) extracted as a candidate. This calculation is performed using an existing object identification algorithm provided in advance in the server 2. The color information and feature information calculated here are stored as numerical information.
Furthermore, by using an object identification method using the object identification algorithm of the server 2, it is possible to identify an object even when the shape of the object to be identified is deformed.

In this case, by registering data in the database server 3 in advance, it is possible to realize the service provided by the present invention for object information related to an object that is not provided by a company such as an animal or a plant.
When there are a plurality of different objects having color information and feature information approximated by a certain threshold value or more, they are regarded as the same object.

  In addition, the server 2 sends a program allocation request to the database server 3 by using the year / month / day information, time information (time information), and channel information specified by the object received from the TV video receiving terminal 4 as keys. It has an outgoing request transmission function.

  Further, the server 2 has a request result transmission function for transmitting the collation result (object candidate list) sent from the database server 3 to the television image receiving terminal 4 as a request result.

  As a result, the user viewing the program can select and specify an arbitrary object (video object) displayed on the TV screen using a TV image receiving terminal that can be connected to the Internet, and the object selected and specified on the server side can be selected. Such object information can be specified, and this object information can be output and displayed on the television screen.

    Based on this request, the database server 3 calculates a program including the object video designated by the TV video receiving terminal 4 (program indexing function).

  Here, in the database server 3, as shown in FIG. 5, information relating to the object providing company (1, 2) or sponsor company (1) corresponding to the determined program (here, program A) is provided. Is provided. For this reason, the database server 3 sets the object information of the provider company information stored corresponding to the program A as the object information collation target. Further, object information of the objects of the object provider companies 1 and 2 and the sponsor company 1 (hereinafter referred to as “corporate object”) stored corresponding to the program A is set as the object information collation target.

  The object information includes an image of the object stored in the database server 3, a product name, a manufacturer, a list price, a bite memo indicating the characteristics of the object, material information, and category information (FIG. 3). These pieces of information may be input to the database server 3 in advance by an object provider company, a sponsor company, or a third agency.

  Further, the database server 3 extracts a corporate object that approximates preset color information and feature information based on the color information and feature information of the candidate object extracted by the server 2, and preferentially performs a matching process. Is provided with a feature information matching function (FIG. 6).

The database server 3 limits the target corporate object to be subjected to the above collation processing to objects whose color information and feature information values are equal to or greater than a predetermined value, that is, the color information and feature information approximate. The setting to perform may be sufficient.
Thereby, for example, even in a server with a high processing load for object collation processing and low processing performance, collation processing according to the processing performance can be executed.

In addition, the database server 3 performs object matching using an existing algorithm such as a structure extraction method. At this time, the image of the candidate object extracted by the server 2 is collated with the image of the corporate object stored in advance in the database server 3.
Here, the collation result is stored as a score. The higher the score, the more similar the collated objects are, indicating that there is a high possibility that they are the same object. The highest score is 100, and the lowest score is 0.

  Further, the database server 3 has a candidate list transmission function for transmitting information indicating an object having a score equal to or higher than a predetermined value based on the collation result to the server 2 as an object candidate list together with the object information. ing.

The database server 3 may be configured to store object information related to objects such as animals and plants.
Thereby, the object information concerning the object which is not provided by the company can be provided to the television video receiving terminal.

  When the video object (hereinafter referred to as “object”) in the TV program video displayed on the interface is specified, the TV video receiving terminal 4 includes date information, time information, channel information, and channel information at the specified time. The position time information including the position information indicating the designated part on the screen is transmitted as an object request to the server 2 via the Internet.

Here, the position information is calculated with two values of X and Y, where X is the horizontal direction and Y is the vertical direction with respect to the screen. It is assumed that the X value and the Y value are calculated with the lower left end of the television image screen as the origin 0, the entire length in the horizontal direction as 1, and the length in the entire vertical direction as 1.
Thereby, for example, even when the horizontal width or vertical width of the TV video screen displayed on the TV video receiving terminal 4 is changed, the position information of the object touched by the user can be accurately determined without depending on the change. Can be transmitted to the server 2.

  In the present embodiment, the TV video receiving terminal 4 provided in advance with a touch panel method for instructing (specifying) an object by directly touching (touching) the screen of the TV video receiving device 4 with a finger. However, the television video receiving terminal 4 may be set to perform designation (object designation) using a dedicated touch pen or the like.

  The TV video receiving terminal 4 may be any TV video receiving terminal that can be connected to the Internet, and may be a TV that can be connected to the Internet, a desktop personal computer, a notebook computer, a mobile PC, a mobile phone, a game machine, a car navigation system, or the like. May be.

  In a network environment in which a broadcast network, an IP data network, a fixed telephone network, and a mobile phone network are integrated into one IP network by NGN (Next Generation Network), the present invention is a television video receiving terminal that can be connected to this network environment. Can be used.

  Although default values are set for the number of frames to be acquired and the trimming range, it is assumed that these values can be changed by learning by the apparatus or user designation.

In addition, the selected object can be purchased by adding a system for purchasing products to the above embodiment.
Specifically, the server 2 and the server of the credit company or the product handling company are linked to enable the ordering of products. In this case, it is also possible to acquire the inventory quantity from the server of the merchandise handling company and display the inventory quantity of the merchandise on the merchandise purchase screen of the TV video receiving terminal. The stock quantity of this product may be an approximate stock quantity or a description such as “stocked, few stocks, no stock”.

  Which object (product) information the user has accessed and which product has been purchased is linked to the user's anonymous account information (information that does not specify the user such as age, gender, and approximate location) and saved. It can also be provided to sponsor companies. From the viewpoint of protecting personal information, it is impossible to provide such information to the sponsor company without permission. Therefore, when providing such information to the sponsor company, the consent of the user is obtained in advance.

  Moreover, when providing such account information, the data itself is not provided to the sponsor company, but only the analysis result of what kind of user is interested in what product is sent to the sponsor company. provide. In this case, the server 2 may be configured to include a data analysis tool.

This makes it possible to identify and specify an object even when the shape or color of the object to be identified is deformed by utilizing an object identification method using an existing object extraction algorithm provided in the server 2 in advance. Can be done.
[Description of Operation of Embodiment]
Next, the contents of the overall operation of this embodiment will be outlined.
First, the user selects and designates a desired object displayed on the screen of the television video receiving terminal 4, and the server 2 displays time information indicating the time when the video receiving terminal designates the video object and the output screen. Specified position information indicating the position specified in the image, and specifying the specified video object with reference to the program video stored in the program video storage device based on the time information and the specified position information (specified video Product identification process).

As a result, the server 2 stores the image and information related to the object, and the database server 1 storing the broadcast TV video, based on the position on the TV screen that the user touches and the time (timing). With reference to the database server 3, the selected and designated object can be specified.
Next, the server 2 acquires the video object information related to the specified video object from the video object information storage device (database server 1) (video object information acquisition step), and transmits this video object information to the video receiving terminal. To do.

  As a result, the server 2 can provide the television video receiving terminal 4 with information (object information) related to the object selected by the user as a request result, and the user can obtain object information of the arbitrarily selected object. You can browse.

  In addition, about the said designation | designated image thing identification process and the image thing information acquisition process, it is good also as a structure which programs the execution content and makes a computer perform.

  Next, the operation of the image information acquisition system in this embodiment will be described in detail based on the sequence chart of FIG.

First, the user accesses and connects to the server 2 from the television image receiving terminal 4 via the Internet, and receives account authentication.
The server 2 authenticates the user account and provides a video output interface to the user's television video receiving terminal 4.

  Note that data is not actually transmitted from the server 2 to the television video receiving terminal 4, and the television video receiving terminal 4 receives only presentation information from the server 2 as a thin client.

  Thereby, the television image received by the television image receiving terminal 4 from the base station 5 is displayed in the interface provided by the server 2 (FIG. 1).

Here, while viewing the TV program, the user designates an object by directly touching (pointing) a video object (hereinafter referred to as “object”) in the displayed video on the screen of the TV video receiving terminal 4. (Step S1: FIG. 6).
Here, when the user touches the screen of the TV video receiving terminal 4, the date / time information, time information, channel information at the time of the touch, and position time information including position information indicating the touch part on the screen are used as the object request. And transmitted to the server 2 through the Internet.

Here, the position information is calculated with two values of X and Y, where X is the horizontal direction and Y is the vertical direction with respect to the screen. Further, the X value and the Y value are calculated by setting the lower left end of the television image screen as the origin 0, the length in the entire horizontal direction as 1, and the length in the entire vertical direction as 1.
Thereby, for example, even when the horizontal width or vertical width of the TV video screen displayed on the TV video receiving terminal 4 is changed, the position information of the object touched by the user can be accurately determined without depending on the change. Can be transmitted to the server 2.

  Next, an operation of extracting and selecting a designated video object (object) in the server 2 will be described with reference to FIGS. 4, 5, and 6.

First, based on the date / time information, time information, and channel information received from the TV video receiving terminal 4, the server 2 sets the corresponding time zone (for example, 2-3 seconds) of the corresponding program from the database server 1. A frame image (hereinafter referred to as “frame”) is acquired (steps S2 and S3: FIG. 6).
Note that this frame indicates a frame image group constituting a video (moving image), and it is assumed that a moving image for one second is composed of 30 frames. In addition, the number of frames acquired for a single request from the server 2 to the database server 1 is, by default, 90 frames for a total of 3 seconds corresponding to time information ± 1.5 seconds.

Here, the server 2 trims the frame acquired from the database server 1 based on the position time information received from the television video receiving terminal 4, and cuts out a part of the image (step S4).
Thereby, the range of the image to be compared can be limited to the range of the clipped image, the accuracy of image identification and object extraction can be improved, and the processing can be speeded up.

  Here, as shown in FIG. 4, the trimming of the frame is ± 0.15 in the X direction and ± 0 in the Y direction based on the position information (X value, Y value) received from the television video receiver 4. It shall be implemented in the area of .1. This trimming operation is executed for each acquired frame.

  Next, the server 2 sequentially numbers 1 to 90 for each trimmed frame and compares the images (step S4: FIG. 6).

As shown in FIG. 4, the image comparison is performed between frames adjacent to the numbered numbers.
This will be described in detail below.

First, the server 2 performs image comparison between the 45th frame and the 46th frame displayed at the time when the user points (designates) the television video receiving terminal 4.
Next, the 46th and 47th, 47th and 48th, and 48th and 49th frames are sequentially compared, and if the images differ by more than a preset threshold value, before the different images appear The first frame in the consecutive frames is stored as a candidate image.

  Specifically, as shown in FIG. 4, for example, image comparison is started from No. 46, and it is recognized that the frames No. 50 and No. 51 are almost identical until the No. 50 frame. Assume that it is determined in the comparison that the images are different by a certain threshold value or more.

  In this case, the first 46th frame is stored as a candidate image among the very first frames in consecutive frames regarded as identical, that is, the almost identical frames from 46th to 50th.

Thereafter, the frames are sequentially compared with the 51st and 52nd, and the 52nd and 53rd frames, and it is assumed that it is determined that both images are different from each other by a certain threshold or more in the comparison between the 59th and 60th frames.
In this case, the first frame in the consecutive frames regarded as identical, that is, the 51st frame is stored as a candidate image.
It should be noted that the comparison processing of the 46th to 90th frames and the comparison processing of the 45th to 1st frames are performed in parallel.

The same comparison processing is also performed for the 45th and 46th frames when the 46th to 90th frame comparison and the 45th to 1st frame comparison are completed. If both the 45th and 46th frames are identified as almost the same image, only the 45th image is stored as a candidate image (deleted if the 46th frame is stored as a candidate image), and both frames are different. When it is determined that the frame number 45 and 46 are both stored.
Here, the image comparison itself is performed by utilizing a structure extraction method and an existing image identification algorithm using color information / feature information.

As a result of comparing the images with 90 frames, if only one frame is stored as a candidate, that is, if it is determined that all 90 frames are the same image, No. 1 and 90 The image is compared with the numbered frame.
This is a process for correctly identifying even an image such as Pann that is photographed so as to flow around the center of the camera.

If the first and 90th frames are compared and it is determined that the images are almost the same, only the 45th frame is stored as a candidate. However, if it is determined that the images are different between the 1st and 90th frames, then the 1st and 45th frame images and the 45th and 90th frame images are compared.
The first and 45th images are almost the same image, but if it is determined that the 45th and 90th images are different, the image identification is further performed with the 45th and 68th frames and the 68th and 90th frames.
In this way, it is assumed that the frame number for image identification is narrowed, and the process is repeated until the image identification results are almost the same.

  Note that the number of frames acquired from the database server 1 and the trimming range tend to be included in which range in the display screen the object selected and specified by storing the time and position of the object selected and specified by the user. The number of frames acquired based on the learning contents and the trimming range may be automatically changed for each user.

  As a result, for example, when elderly people use, the timing and contact position intended by the person and the timing and contact position actually touching the screen may deviate greatly (than expected values). When such a user repeatedly specifies and selects an object, a frame is obtained that has more than 90 frames of time information ± 1.5 seconds as a default value, and the trimming range is larger than the default value. You may set so that it may be set to.

In addition, when a user with high reflectivity repeatedly selects and specifies an object, less than 90 frames of time information ± 1.5 seconds, which is the default value, are acquired, and the trimming range is also smaller than the default value. You may set so that it may be set to the value which shows a range.
As a result, the number of frames to be acquired and the trimming range are limited to narrower ones, so that it is possible to improve performance such as improvement of object identification accuracy and quick provision of request results.

Furthermore, information indicating the number of frames acquired for each user and the range (area) to be trimmed may be stored in association with each account information, and the trend may be analyzed at regular intervals.
Alternatively, a setting that allows the user to manually specify an arbitrary value from the television video receiving terminal 4 may be used. In this case, based on the specified value, the time required for presenting the object candidate list (the time from request request to reply of request result) may be calculated and presented to the user.

Further, it is assumed that the number of history information stored in association with the account information and the time limit (for example, a maximum of 100 objects and one month) can be changed by the user.
It is also assumed that the number of objects intentionally stored by the user (for example, a maximum of 50 objects) can be changed by the user.

Next, the server 2 extracts an object from the frame stored as a candidate.
Here, the server 2 acquires a frame before trimming of the frame stored as a candidate (step S5: FIG. 6).
Next, the server 2 determines the background and the object, and extracts objects included in the trimming range (X ± 0.15 and Y ± 0.1 images based on the position information) as candidate objects ( Step S5: FIG. 6).

  At this time, a plurality of objects may be extracted as candidates from one image, but when a plurality of objects are extracted as candidates, all of them are stored as candidate objects. Further, even when a plurality of frames are stored as candidates, it is conceivable that a plurality of objects are extracted as candidates. In this case as well, the extracted plurality of objects are stored as candidate objects.

  Next, color information and feature information of the object (candidate object) extracted as a candidate are calculated (step S6: FIG. 6). This calculation is performed using the existing object identification algorithm of the server 2. Here, the calculated color information and feature information are stored as numerical information.

  Here, by using the object identification method using the object identification algorithm of the server 2, it is assumed that the object can be identified even when the shape of the object to be identified is deformed.

  In this case, by registering data in the database server 3 in advance, it is possible to realize the service provided by the present invention for object information related to an object that is not provided by a company such as an animal or a plant.

  When there are a plurality of different objects having color information and feature information approximated at a certain threshold value or more, they are regarded as the same object.

Next, the operation of the object matching process for matching the extracted candidate object with the object information of the database server 3 will be described with reference to FIG.
Here, as shown in FIG. 5, it is assumed that provider company information indicating the object provider companies 1 and 2 and the sponsor company 1 is set corresponding to the program A.

First, the server 2 transmits to the database server 3 a program index request received from the TV video receiving terminal 4 using the date, time information (time information), and channel information for which object designation has been performed as keys.
Based on this request, the database server 3 calculates a program including the object video designated by the TV video receiving terminal 4 (program indexing function).

  Here, as shown in FIG. 5, the database server 3 stores object company information of an object provider company and a sponsor company corresponding to the determined program (here, program A). The object information of the providing company stored corresponding to the program A is the object information collation target. Here, the object information of the objects of the object provider companies 1 and 2 and the sponsor company 1 (hereinafter referred to as “corporate object”) stored corresponding to the program A is the object information collation target.

  Next, the database server 3 extracts corporate objects that approximate the preset color information and feature information based on the numerical data of the candidate object color information and feature information extracted by the server 2, and preferentially collates them. Perform (Step S6: FIG. 6).

The database server 3 limits the target corporate object to be collated to objects whose color information and feature information values are equal to or greater than a predetermined value, that is, color information and feature information approximate. It may be a setting for performing a collation process.
Thereby, for example, even in a server with a high processing load for object collation processing and low processing performance, collation processing according to the processing performance can be executed.

Here, the database server 3 performs object matching using an existing algorithm such as a structure extraction method. At this time, the image of the candidate object extracted by the server 2 is collated with the image of the corporate object stored in advance in the database server 3.
Here, the collation result is stored as a score. The higher the score, the more similar the collated objects are, indicating that there is a high possibility that they are the same object. The highest score is 100, and the lowest score is 0.

  Next, the database server 3 transmits information indicating an object having a score equal to or higher than a predetermined value based on the collation result to the server 2 together with the object information as an object candidate list (step S7: FIG. 6).

The server 2 that has acquired the object candidate list from the database server 3 transmits the comparison result as a request result to the user's television video receiving terminal 4 (step S8: FIG. 6).
The object information includes an image of the object stored in the database server 3, a product name, a manufacturer, a list price, a bite memo indicating object characteristics, material information, and category information (FIG. 3). These pieces of information are preliminarily input to the database server 3 by an object provider company, a sponsor company, or a third agency.

Hereinafter, the operation of outputting and displaying the request result (object candidate list) sent from the server 2 will be described based on the sequence chart of FIG.
In this case as well, the server 2 presents only the presentation to the television image receiving terminal 4 without actually transmitting data to the user's television image receiving terminal.
Thereby, the user can browse the data on the server 2 through the television image receiving terminal.

  By default, up to 20 collation results are displayed in order from the highest score. However, the data itself is managed on the server 2 side. Accordingly, the television video receiving terminal 4 performs only the output display of the object candidate list as in the thin client system, and the user can only browse the object candidate list.

In the TV video receiving terminal 4, the product name, manufacturer, image, price, bite memo, material information, and category information of each object candidate list sent as a request result are displayed for each object (step S9: FIG. 6).
In addition to the information related to each object, all the material information and category information of a plurality of objects in the object candidate list are displayed in a lump.

The user narrows down the selection based on the material information and category information that the object he / she wishes to select from among the material information and category information displayed at once (step S10). Next, the server 2 redisplays the candidates reflecting the selection narrowing (step S11).
Thereby, unintended candidate objects other than the selected candidate object can be excluded from the object candidate list, and the user can display a desired object candidate on the output screen.

  A plurality of material information and category information can be selected. The material information and category information are linked to each other.

A specific example will be described below.
Assume that “green tea in a plastic bottle”, “household detergent”, and “elongate cushion” are displayed on the television image receiving terminal as the request result (object candidate list). The category information is “beverage”, “household goods”, and “miscellaneous goods”, and the material information is “plastic” and “cloth”.
When the user selects “beverage” from the category information, “cloth” is automatically grayed out from the material information. This is because an object having category information “beverage” and material information “cloth” is not included in the list.
The user can narrow down the displayed candidate objects by specifying the material information and the category information in this way.

As described above, the user makes a request for narrowing down objects by designating material information and category information at the television video receiving terminal 4 (step S8: FIG. 6).
Next, the server 2 redisplays the object candidate list narrowed down in response to the object narrowing request from the television video receiving terminal 4 on the television video receiving terminal 4 (step S9: FIG. 6).

As a result, when the user wants to acquire more detailed information about the object displayed as the object candidate list, the user can browse the CM and related website set for the object.
Web searches can also be performed using product names as keys. Also in this case, the data is managed on the server side, and information can only be browsed from the television video receiving terminal.

Further, by adding a system for purchasing products to the above-described embodiment, the user can purchase an object selected and designated.
Specifically, it is possible to place an order for a product by linking the server 2 with a server of a credit company or a product handling company. In this case, information indicating the number of stocks may be acquired from the server of the product handling company so that the stock number of products can be displayed on the product purchase screen on the television image receiving terminal 4. In this case, the number of stocks of this product may be an approximate number of stocks, or may be expressed as being in stock, few stocks, no stock.

The information on the object candidate selected by the user from the object candidate list presented on the television video receiving terminal 4 may be stored in the server 2 as a selection history.
In this case, the selection history is stored in association with the account information for a preset period (for example, one month).

At this time, for example, when the selection history information exceeds 100, the history may be deleted in order from the oldest. Of the objects saved as history, object information that the user intentionally designates “save” is assumed to be semi-permanently saved, for example, up to 50 objects.
It is assumed that the number of the selection history information stored in association with the account information and the time limit (default is one month, maximum 100 objects) can be appropriately changed by the user. Also, the number of objects intentionally saved by the user (up to 50 objects by default) can be changed by the user.

  As described above, the image information acquisition system of this embodiment performs feature extraction of a selected and designated object when a user selects and designates a video object (object) in a broadcast program video displayed on a video screen. This is performed and collated with the image and information of the object registered in advance in the database, so that the selected and specified object can be specified and information (object information) related to the object can be provided to the user.

  Further, according to the present invention, it is possible to acquire object information not only from a TV program but also from any video information in an environment connected to the Internet. For example, the user can acquire object information included in a recorded program or a DVD or video image borrowed from a rental shop.

  Also, in an environment connected to the Internet according to the present invention, by applying the present invention, information of an object included in a still image displayed by a browser can be arbitrarily acquired by selecting and specifying.

Furthermore, object information can also be acquired from a moving image, a still image, or a photograph posted on a website taken by the user.
In this case, the user uploads a video, a still image, or a photograph posted on the website through the interface provided by the server 2 and uploads an image to the server 2 together with material information and category information. By transmitting, the server 2 performs collation from objects with similar color information and feature information values using the material information and category information transmitted from the TV video receiving terminal 4 as keys.

Also, object information can be acquired from a paper medium such as a magazine, a newspaper, or a book or an actual object by photographing the object and converting it into a digital image or video.
Specifically, the object information is obtained by photographing a target object using a device having a photography function such as a mobile phone or a digital camera, and transmitting the photographed photograph to the server 2 from the television image receiving terminal. Can get.

Furthermore, when the QR code is displayed, it is assumed that the object information can be acquired not from the photograph but also from the QR code.
In this case, the object code included in the object information table of the database server 3 is stored in the QR code, so that the user can quickly acquire the object information.

  The present invention is connectable to a wide area network environment when a wide area network environment in which a broadcast network, an IP data network, a fixed telephone network, and a mobile phone network are integrated into one IP network by NGN or the like is realized. The present invention can be applied to a network communication system including a video receiving terminal.

1 database server 2 server 3 database server 4 TV video receiving terminal 5 base station

Claims (8)

A video receiving terminal having an output screen for displaying the broadcast program video; a designation means for designating a video object displayed on the output screen; a program video storage device for storing the broadcast program video; An image information providing system comprising a video object information storage device storing video object information set in advance corresponding to a video object,
Obtaining time information indicating the time when the video receiving terminal specified the video object and specified position information indicating the specified position in the output screen, and storing the program video based on the time information and the specified position information A video object specifying device for specifying the specified video object with reference to a program video stored in the apparatus;
Image information provision characterized in that the image object specifying device includes image object information providing means for acquiring image object information relating to the specified image object from the image object information storage device and providing the acquired image object information to the image receiving terminal. system. The image information providing system according to claim 1,
The image information providing system according to claim 1, wherein the output screen of the video receiving terminal includes touch panel input means for designating a video object displayed on the output screen by touching the output screen. In the image information providing system according to claim 1 or 2,
The video object specifying device includes candidate video object output means for displaying a list of the specified video objects on the output screen when a plurality of different video objects are specified as candidates for the video object. A system for providing image information. The image information providing system according to claim 1,
The video object specifying device acquires a program image group which acquires a program image frame group displayed during a preset time including the designated program image frame from a program video stored in the program video storage device. And an image frame specifying unit for specifying a program image frame including the video object based on a color change amount between program image frames included in the program image frame group. . The image information providing system according to claim 1,
The video object specifying device specifies the specified program video based on the time information from the program video stored in the program video storage device, and based on the specified position information of a program image constituting the program video. An image information providing system comprising program image cutting means for trimming a preset image area. The image information providing system according to claim 4,
The video object specifying device includes image area specifying means for specifying a program area including the video object based on an amount of color change between image areas trimmed from the program images. Offer system. A video receiving terminal having an output screen for displaying the broadcast program video; a designation means for designating a video object displayed on the output screen; a program video storage device for storing the broadcast program video; An image information providing system comprising a video object information storage device storing video object information set in advance corresponding to a video object, and providing video object information according to a request to the video receiving terminal An image information providing method,
The video receiving terminal acquires time information indicating the time when the video object is specified and specified position information indicating a specified position in the output screen, and refers to the program video stored in the program video storage device The specified video object is specified based on the time information and the specified position information, and the video object information related to the specified video object is acquired from the specific video object information storage device and provided to the video receiving terminal. An image information providing method characterized by that. A video receiving terminal having an output screen for displaying the broadcast program video and a designation means for designating the video object displayed on the output screen, a program video storage device for storing the program video, and each video object Correspondingly, an image information providing system comprising a video object information storage device that stores preset video object information, and providing the video object information in response to a request from the video receiving terminal A program,
The specified video with reference to the program video storage device based on time information indicating a time when the video receiving terminal specifies the video object and specified position information indicating a position specified in the output screen A specified video object specifying function for specifying an object, and a video object information acquiring function for performing processing for acquiring the video object information related to the specified video object from the video object information storage device,
An image information providing program which is executed by a computer provided in advance in the video object information storage device.

Images