JP2003259336A - Data generating method, data generating apparatus, data transmission method, video program reproducing apparatus, video program reproducing method, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily and surely provide sub-information attached to a particular image object in a video program image to a viewer and to allow the viewer to easily and surely acquire the sub-information. <P>SOLUTION: A value capable of specifying an image object is set to each pixel in an index object area superimposed on the specified image object in a video program on a visual image superimposed on a video program image to produce index picture data, produces the link data for specifying access information to the sub-information attached to the image object in cross- reference with the value and transmits the data to a reproducing side. The reproducing side points the specified image object, acquires the access information to the sub-information attached to the image object and acquires the sub- information on the basis of the access information. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention provides information added to a specific image object (object) in a video program image to a viewer as sub information (sub content), and the viewer acquires the information. To a method and an apparatus therefor.

[0002]

2. Description of the Related Art In a video program such as digital broadcasting, information about objects such as people and products in the program image, for example, names of actors and singers who appear in drama and music programs, products introduced in the program image The conventional method for providing and providing viewers with information such as the specific contents of services and services, and detailed locations of facilities and tourist spots introduced in the program images has been the conventional method of displaying and selecting menus. The method by is considered.

[0003]

However, in the method of displaying and selecting the menu, (a) the viewer is
There is no means to recognize whether or not the desired information is added to the video program being viewed, so it is necessary to operate the menu screen unnecessarily to search whether the desired information is added. I have to.

Further, (b) when the program is provided by multi-frame display and the object information is displayed as a sub-frame, the display area of the main video program is reduced, so that the user wants to see the video with high image quality. I cannot satisfy the viewer's request.

Further, (c) when a video is reproduced and displayed by a viewing device (reception device) which does not have a video storage device such as a video server (home server), a video program or the like which is delivered on time by broadcast radio waves or CATV. Then
Since the story progresses while operating the menu, the viewer cannot not only see the video program while operating the menu, but also cannot obtain the target information as a result of the search by the menu operation. Dissatisfied with.

Further, (d) the number of menus that can be displayed in the limited display area of the video display device is limited, particularly when the menus are displayed simultaneously with the video program by multi-frame display. Therefore, in order to provide a lot of information by displaying the menu, it is necessary to make the menu structure multi-layered (hierarchical). But,
In that case, the menu operation becomes complicated and it takes time to acquire the target information.

[0007] (e) When selecting information by operating the menu, the viewer cannot read the characters displayed in the menu and recognize the contents, so that the viewer cannot make an accurate selection and is intuitive. Unable to make proper operation selection.

Further, (f) acquisition of information by menu operation requires the designation or name of the target object as a selection item, but if the viewer does not know the designation or name of the target object, There is no way to make up for that lack of knowledge. Therefore, it is difficult for the viewer to acquire information by motivating a fundamental question such as “who is this” and “what is this”.

(G) Since the moving image object in the video program moves on the screen or is replaced with another moving image object, it is difficult to present the information about the moving image object by the menu display, and There is a problem that information cannot be sequentially added, such as the object disappears while reading the characters displayed in the menu.

There is also a method of presenting information by means of the icon (h), but since the viewer pays attention to the video program and does not pay attention to the icon displayed in an inconspicuous position, the information is added. Sometimes you don't notice.

Further, in both (i) the menu system and the icon system, what can be presented is mainly information about the entire screen or the video program displayed at that time, or a display for switching additional services. Therefore, it is difficult to present information about the image object unit.

Further, according to the conventional method, it is not possible to perform image processing on an object-by-object basis, such as (j) a specific object of interest to the viewer in the video program image is highlighted, enlarged, or traced.

(K) In the video program recorded in a storage device such as an optical disk or a hard disk, which is installed in an information processing terminal such as a personal computer built in the viewing device or connected to the viewing device, It is not possible to add information that the viewer independently creates or collects about the image object as sub-contents so that the viewer can search for it.

Therefore, the present invention solves the above-mentioned problem and easily and surely provides the viewer with the sub-information which is the information added to the specific image object in the video program image. It is designed to be easily and surely acquired.

[0015]

A data generating method of the present invention is a method for generating a data on a virtual screen superimposed on a video program image,
For each pixel in the index object area, which is an area that overlaps the specified image object area in the video program image, a value that can specify the image object in the same video program is set. Then, the step of generating index picture data, which is data different from the video program data, and the information for accessing the sub information, which is the information added to the specified image object, are associated with the image object. And a step of generating, as link data, data defined in association with a value of the index object.

In the data transmission method of the present invention, the video program data and the index picture data and the link data generated as described above are multiplexed, and are multiplexed by the same transmission means or synchronously reproducible. It is transmitted to the reproduction side by the transmission means.

The video program reproducing apparatus of the present invention reproduces the transmitted video program data and displays the video program image on the display, and the video program image on the virtual screen superimposed on the video program image. For each pixel in the index object area, which is an area to be superimposed on the specified image object area in the image object, one value that can specify the image object in the same video program is set, A means for reproducing the index picture data, which is generated and transmitted as separate data from the program data, in synchronization with the video program data, and for accessing sub-information which is information added to the specified image object. Information is the index object corresponding to the image object. Means for reproducing the link data which is generated and transmitted by being defined in association with the value of the index in synchronization with the index picture data, and in the video program image displayed on the display based on the instruction operation. Pointing means for pointing the image object, and when there is an index object corresponding to the pointed image object, means for referencing the link data corresponding to the image object by the value of the index object, and its reference. It is provided with means for accessing the sub information by the access information defined in the obtained link data to obtain the sub information, and means for reproducing the obtained sub information.

According to the method or apparatus of the present invention as described above, the sub-information provider can easily and surely provide the sub-information about the specific image object in the video program image to the viewer. Therefore, the viewer can easily and surely acquire the sub information.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION [Image Object and Index Object: FIG. 1] The upper part of FIG. 1 shows an example of a video program image. The video program is configured to have an image of each of scenes 1, 2, 3, ... With digital video data in which luminance data and color difference data are multiplexed. Audio data and control information are multiplexed with the video data, but they are omitted here.

As a digital video system, NTS
Any of video systems of various formats defined as C, PAL or DTV (digital television) may be used.

In the example of FIG. 1, the video program VP1 of the scene 1 is assumed to have a person 1v, a terminal 2v, a chair 3v, a desk 5v, a plant 6v and a background, and can be developed from time ts1 to time ts2. It is composed of frames. The video program VP2 for scene 2 is configured as having a person 1v and a background, and scene 3
The video program VP3 is configured to have persons 1v and 4v and a background. Scenes 1, 2, 3
Person 1v is the same person.

As significant image objects in such a video program image, for example, in scene 1, person 1v, terminal 2v and chair 3v, in scene 2, person 1v, and in scene 3, persons 1v and 4v are used. , Specify each.

Significance means something that the viewer is interested in or interested in, or that the viewer seems to be interested in, and sub information (sub content) is added.

For the specified image object, the same position as the area of the image object on the image in which the image object appears on the virtual screen composed of invisible pixels superimposed on the image in which the image object appears. And the size-shaped area is set as an index object.

Specifically, in the example of FIG. 1, the index object 1a is set for the person 1v, and the terminal 2
For v, set index object 2a, and for chair 3v, index object 3
a is set, and the index object 4a is set for the person 4v.

Therefore, as shown as an index picture, the screen in which the index object is set is also composed of the screens of scenes and frames corresponding to the scenes and frames of the video program, and the index picture IP1 of scene 1 is , The index picture IP2 of the scene 2 has the index object 1a, and the index picture IP3 of the scene 3 has the index objects 1a, 4a.
Will have.

Further, for each pixel in each index object area, a value that can identify the image object corresponding to the index object is set to an index object value (value of pixels forming the index object). Set as.

For example, in the case of specifying a maximum of seven image objects in one frame of the video program image,
The index object value has a 3-bit length and is represented by a binary number. For example, 001 is assigned to each pixel in the area of the index object 1a corresponding to the person 1v, and the index object 2 corresponding to the terminal 2v is assigned.
010 is assigned to each pixel in the area of a, 011 is assigned to each pixel in the area of the index object 3a corresponding to the chair 3v, and the index object 4a corresponding to the person 4v is assigned. 100 is given to each pixel in the area, and 000 is given to each pixel in the area where the index object in the index picture of one frame is not set.

As will be described later, when the information of the reproduction start time and the reproduction end time of each scene is generated as a part of the link data and transmitted to the reproduction side, the reproduction start time information and the reproduction end time information. Since the index object of each scene and the image object corresponding to the scene can be identified by the index object value and the index object value, even the same index object corresponding to the same image object can have different index object values between different scenes. it can.

Practically, the data length of the index object value is made larger than 3 bits so that more image objects can be specified in one frame of the video program image. For example, if the index object value has a length of 8 bits, a maximum of 255 image objects can be specified in the video program image of one frame. In this case, the pixel value of the area other than the index object is represented by a hexadecimal number, for example, 00h.

As described above, a data stream in which an index object value is set for each pixel in the area of the index object and another pixel value is set for each pixel in the area other than the index object is the index. It is a picture.

[Generation of Index Picture: FIG. 2] FIGS. 2A and 2B show an example of the index picture generation device.

In the index picture generation device of the example of FIG. 2A, the video source 11 is a digital video camera,
A device that can obtain a digital video program, such as a digital video tape recorder, a telecine conversion device, or a video server. As the video program, ITU-R601 or ITU-R656 in a digital video system such as an NTSC video system or a PAL video system.
Luminance data (Y) and color difference data (Cb, C
r) is a multiplexed video data bitstream,
A bit stream of video data in a digital video system such as BTAS-002A of HDTV (High Definition Television) can be obtained.

However, the input video signal may be an analog video signal, which is converted into digital video data of a predetermined format by the output section of the video source 11 and output from the video source 11.

The video program stream VPs from the video source 11 is supplied to the object extraction unit 15 which constitutes the index picture generation unit 14, and the object extraction unit 15 is controlled by the control unit 19 based on the operation setting by the operation setting unit 18. , Outline enhancement, hue matching,
By performing processing such as spatial filtering, the area of the image object designated by the operation setting unit 18 in the video program image is changed from frame to frame as a signal OS indicating the area of the corresponding index object on the index picture. It is extracted continuously.

Further, the index object area signal OS is supplied to the index object value setting section 16, and the index object value setting section 16 designates each pixel in the index object area in the index picture according to the designation by the operation setting section 18. On the other hand, an index object value is set, a predetermined value is set for each pixel in the area other than the index object, and the index object value setting unit 16, that is, the index picture generation unit 14 sets a value of, for example, 8 bits. Index picture stream IP
s is obtained, and this index picture stream IP
s is MPEG2 or MPEG4 in the compression encoding unit 17.
It is encoded by a compression method such as.

Regarding the compression of index pictures,
In the same index object area, only a direct current component with a constant value exists, and an alternating current component occurs only in the contour portion of the index object, so that a high compression rate can be achieved.

Further, in sampling the index picture, the index picture can be similarly compressed at a high compression rate by reducing the number of pixels (the number of samples) to such an extent that there is no practical problem.

When the number of objects that can be extracted by the object extracting unit 15 at one time within one frame period is smaller than the maximum number of objects that can be specified in one frame of the video program image, for example, in one frame of the video program image. When it is made possible to specify a maximum of 255 image objects in the above, if the object extracting unit 15 can extract only seven objects at one time within one frame period, the index picture generating unit 14 determines the generated index pictures. Once it is recorded in the buffer, another image object area is extracted again as an index object area from the video program image of the same frame, the index object value is set for each pixel in that area, and the index picture is set. To generate After repeating the process, configured to multiplex the index picture of one frame a plurality of index pictures generated from the same frame of the video program image.

In the example of FIG. 2A, on the other hand, the video program stream VPs from the video source 11 is compression-encoded by the compression encoding unit 12 by a compression system such as MPEG2 or MPEG4, and the compressed video program. The stream is written in the frame buffer 13 and read from the frame buffer 13 so that the stream is output in synchronization with the compressed index picture stream IPS from the compression encoding unit 17.

However, in this way, the video program stream VPs from the video source 11 is compressed and encoded by the compression encoding unit 12.
After being compressed with, the data is written to the frame buffer 13 and read from the frame buffer 13. For example, MPEG is used between the video program image and the index picture.
In terms of synchronizing the generation timing of I, P, and B pictures and sharing motion vectors in the two-compression method,
Not necessarily appropriate.

Therefore, in the index picture generating apparatus of the example of FIG. 2B, the video program stream VPs from the video source 11 is written in the frame buffer 13, read out from the frame buffer 13, and then compressed and encoded. It is compression-encoded by the unit 12.

In this example, the compression coding unit 12 sends the control information CT to the compression coding unit 17, and the compression coding unit 17 receives the control information CT.
By controlling the compression encoding in the video program image and the index picture, for example, MP
It is possible to synchronize the generation timing of each picture of I, P and B in the EG2 compression method and share the motion vector. Also, by being able to share motion vectors,
The total amount of data in the index picture stream can be reduced.

In the example of FIG. 2A, the frame buffer 13
2 from the compression encoding unit 12 in the example of FIG.
The compressed video program stream VPS and the compressed index picture stream IPS from the compression encoding unit 17 are multiplexed and transmitted to the reproduction side by the same transmission means.

As a method of multiplexing the index picture stream IPS with the video program stream VPS, for example, the index picture stream IPS is used.
Is preferably defined as a subset in the transport stream of the MPEG2 system. As a result, the video program stream VPS and the index picture stream IPS can be transmitted without any change from the MPEG transmission method.

However, depending on the correspondence on the reproducing side, the index picture stream IPS may be multiplexed with a private stream specified by the MPEG2 system without being specified as a subset.

Also, the index picture stream I
Identification information such as a program ID (identification information) for ensuring the identity between the video program and the index picture in the PS, and reproduction reference time information for reproducing the index picture in synchronization with the video program on the reproduction side. In the case of multiplexing the video program stream VPS and the index picture stream IPS,
It can be transmitted to the reproducing side by another transmission means without being multiplexed.

[Sub-information and link data: FIG. 3 and FIG. 4] (Summary of sub-information) An image object for which an index object is set in a video program image is added as sub-information (sub-contents) to the image object. , Image data such as still images and moving images, audio data, or text format data, BML, XML, HTM for presenting by integrating these data
Generates Internet content described by a presentation tool such as L or VRML, content that can be transmitted on the Internet, or image data, sound data, or text format data that is compatible with a format that can be played on the playback side. Transmit to the playback side.

(Outline and data structure of link data)
Further, in association with the index object value, information relating to the image object corresponding to the index object is generated as link data and transmitted to the reproducing side.

Specifically, in the example of FIG. 1, for the scene 1, the link data for the index objects 1a, 2a and 3a corresponding to the person 1v, the terminal 2v and the chair 3v is generated as the link table data, and the scene is generated. For 2, the link data for the index object 1a corresponding to the person 1v is generated as link table data, and for the scene 3,
The link data for the index objects 1a and 4a corresponding to the persons 1v and 4v is generated as link table data.

Each link data has, as a minimum structure,
As shown in FIG. 3A, it is assumed that the start code PRE, the definition code DC, the index object value IOV, and the end code TRM are included.

The definition code DC is a control code, a program ID, a code indicating a category of a video program, a code indicating a sponsor (commercial provider) of the video program, an area code and the like.

By defining access information to the management center, which will be described later, in the definition code DC, the reproducing side can access the management center and acquire sub-information, as described later.

Further, as shown in FIG. 3 (B), link data having an attribute code added is made up of a start code PRE, a definition code DC, an attribute code AC, an index object value IOV, and an end code TRM. Can also be

The attribute code AC is, for example, (1)
Define the outline of the image object in text, (2) define the outline of sub-information in text, (3) define the category of image object in code, (4) define the category of sub-information in code, or A plurality of the above (1) to (4) are combined.

By adding the attribute code AC,
As will be described later, the playback side can perform image processing according to the attribute (category) of the image object or the sub-information, or according to the viewing tendency of the viewer. By analyzing the sub-information selection tendency of, the viewer-specific preference attribute can be obtained as a viewer profile.

Further, as link data, information for accessing the sub information added to the image object corresponding to the index object in association with the index object value is generated.

The link data in this case is at least
It is assumed to have an index object value, information indicating the start time and end time of the index object (image object), and access information (information for accessing the sub information).

For example, as shown in FIG. 3 (C), the link data includes start code PRE, definition code DC, index object value IOV, start time information BTC, end time information CTC, access information RLV, and end code TRM. 3D or with an attribute code added, as shown in FIG. 3D, start code PRE, definition code DC, attribute code AC, index object value IOV, start time information BTC, end time information CT.
C, access information RLV, and end code TRM.

Start time information BTC and end time information C
In the example of FIG. 1, TC indicates the reproduction start time ts1 and the reproduction end time ts2 of the scene 1 in each link data of the index objects 1a, 2a, and 3a of the scene 1, and indicates the index object 1a of the scene 2. In the link data, the reproduction start time ts2 and the reproduction end time ts3 of the scene 2 are shown. In the link data of the index objects 1a and 4a of the scene 3, the reproduction start time ts3 and the reproduction end time ts4 of the scene 3 are shown. is there.

However, since the index object 1a continuously appears from scene 1 to scene 3, as the link data for the index object 1a, the start time information BTC is the reproduction start time t of scene 1.
s1 and the end time information CTC may indicate the reproduction end time ts4 of the scene 3.

When the index object (image object) appears in the middle of the scene or disappears in the middle of the scene, the appearance time in the middle of the scene is used as the start time and the disappearance time in the middle of the scene is used as the end time. Good.

As shown in FIG. 3E, in the example of FIG. 3D, the definition code DC is composed of the control code DC (1) and the program ID (identification information) DC (2), and each code And assuming that the data length of each information is the number of bytes shown in the figure, it is assumed that 32 image objects are specified on average in one scene, and the average data amount per second when the scene duration is 3 seconds on average. , (128 Byte ×
32) / 3 seconds = 1365 bytes / second, that is, about 11
It becomes kbps.

(Access Information) The access information RLV is
The following information is used according to the resource (information source) of the sub-information.

(1) When sub information is acquired from a server device on the Internet, IP (Internet) is used.
URL (Uniform) specified by Protocol
mResource Locator).

(2) When sub-information is acquired from a server device on a communication network in a closed environment connected to the public network, the line number or call control information of the terminal device in which the server device is installed, and Hierarchical structure information and file name as file control information that identifies the sub information resource,
Alternatively, it may be a physical address or a logical address.

(3) DVD, video CD, CD-RO
When acquiring sub information from a package recording medium such as M or a digital video tape, the name of the recording medium, the product code, etc. are publicly organized as information for identifying the medium on which the sub information is recorded. Numbers, symbols, etc., and track number, sector number, hierarchical structure information, file name, or physical address, logical address, etc. as information indicating the sub information resource. However, when the sub information is also recorded on the medium on which the video program is recorded, the information for specifying the medium on which the sub information is recorded is not essential.

(4) When sub-information is acquired by satellite digital broadcasting, terrestrial digital broadcasting, or CATV scheduled program distribution, it is used as channel identification selection information, packet identification selection information, time information, or the like.

(5) When sub-information is obtained from a server device installed on the CATV network or a server device on the Internet via a bridge (IX: Internet Exchange) from the CATV network to the Internet, It is a URL for specifying an information resource, or drive information, hierarchical structure information or a file name, or a physical address or a logical address as information for specifying a resource of a server device in the CATV network.

As the access information, not only the sub information resource but also the CM (commercial) information resource of the sub information provider can be defined. Furthermore, the reproduction order of the sub information and the CM information, and the scrambled information can be defined. It is possible to define reproduction control information that defines a procedure for decoding the sub-information that has been read.

When access to the sub-information requires access to the management center as described later, the access information to the management center is defined together with the sub-information resource or the sub-information is defined. Do not define resources, only access information to the management center.

(Definition of Resource) Specifically, the following mode can be adopted as a method of defining a resource in link data.

(1) When the sub information resource is directly defined in the link data, the sub information provider (CM
The CM information of the provider can be added. In this case, the content in which the CM information of the sub information provider and the sub information selected by the viewer are integrated is provided, and the CM information can be displayed on the reproducing side by multi-window display or sequential reproduction. .

The target objects include works of art such as paintings and sculptures, and objects such as actors and athletes that the viewer is interested in and cannot sell in general.

In this case, EC (E
electronic commerce)
It is possible to add sales information or the like about a target object of a business operator that performs such as. In this case, it is possible to provide the viewer with detailed infomercial information. Target objects include cars, furniture, watches, accessories, clothes, and travel.

(2) When the sub information resource and the CM resource of the sub information provider are directly defined in the link data, or when the above reproduction control information is further defined directly, the sub information resource and the CM resource Can be specified separately, and the creator and provider of sub-information can be separated, which enables multi-use of sub-information.

When the reproduction control information is directly defined, as described later, the sub information is scrambled, and the descrambling key is embedded in the CM information in a dispersed manner. By collecting the distributed release keys and generating the scramble release key when the reproduction of the CM is completed, the system configuration is such that the sub information can be reproduced, so that the CM can be reliably reproduced.

If the playback-side viewing device is a digital broadcast receiving device having a compulsory storage area, the CM resource can be used as the compulsory storage area.

(3) When the sub information resource and the center access information are directly defined in the link data, on the reproducing side, the sub information is obtained by directly accessing the sub information resource, and the CM information is obtained by accessing the management center. To do. In this case as well, it is desirable to use a system that reliably reproduces the CM as described above.

In this system, it is possible to centrally manage the CM provision by the management center and selectively provide the CM.

(4) When only the center access information is defined in the link data, on the reproducing side, as will be described later, the program ID of the video program and the reproduction reference time information, and the index object value corresponding to the corresponding image object. Is notified to the management center, the corresponding sub-information resource, the CM providing method and the CM resource are acquired, and the CM resource is accessed and the CM information is acquired according to the CM providing method.

In this case, the management center can obtain a viewer-specific preference attribute from the viewer's sub-information selection tendency as a viewer profile, and selectively provide and manage the most suitable CM for each viewer. In addition, it is possible to centrally settle the payment for the sub information provider.

(Transmission of Link Data) The link data is
For example, the video program stream and the index picture stream are multiplexed so as to occupy a layer close to the bottom layer in the hierarchical structure of the multiplexed stream, and are transmitted to the reproduction side.

FIG. 4 shows the mode of multiplexing. Video program streams VPS1, VP for scenes 1, 2, 3
For S2 and VPS3, scenes 1, 2, and
3 index picture streams IPS1, IPS
2, IPS3 is multiplexed.

On the other hand, regarding the link data,
When the index object exists from the beginning of the scene, on the playback side, by the start of the scene, the image object that is the image processing target in the scene is determined, and the image processing method for the image object is determined.
Since it is necessary to define the sub information resource, it must be transmitted to the playback side before the start time of the scene.

Therefore, the link table data LTD1 for scene 1 is transmitted before scene 1, and the link table data LTD2 for scene 2 is multiplexed with the video program stream VPS1 and index picture stream IPS1 of scene 1, The link table data LTD3 for scene 3 is multiplexed into the video program stream VPS2 and index picture stream IPS2 of scene 2, and the link table data LTD4 for scene 4 is added to the video program stream VPS3 and index picture stream IPS3 of scene 3. Is multiplexed.

Depending on the system, the link data may be transmitted to the reproducing side by another transmission means without being multiplexed with the video program stream and the index picture stream.

[Interactive Viewing System: FIG. 5] FIG. 5 shows an example of the interactive viewing system. The system as a whole is composed of a generation system 10, a transmission system 30, and a reproduction system 40.

In the generation system 10, the video program stream VPs from the video source 11 is omitted in FIG. 5, but as shown in FIG.
Is written into the frame buffer 13 and read out from the frame buffer 13, and is then compression encoded by the compression encoding unit 12.

In the index picture generating unit 14, the index picture stream IPs is generated from the video program stream VPs from the video source 11, and the index picture stream IPs is compression encoded by the compression encoding unit 17.

In the sub-information generator 22, the sub-information data,
And the access information to the sub information is generated, and the sub information data of that is compression encoded by the compression encoding unit 23.

In the link data generation unit 21, the index object value, the start (appearance) time information and the end (disappearance) time information from the index picture generation unit 14, the access information from the sub information generation unit 22, or the content server 26. The primary link data for each index object is generated by the access information about the existing content from the.

Further, in the link data generator 21, the definition code (control code and program ID) and the attribute code from the content server 26 are added to the primary link data, and a diagram for each index object is shown. The link data as shown in 3 (E) is generated.

Then, the compressed video program stream from the compression encoding unit 12, the compressed index picture stream from the compression encoding unit 17, the link data from the link data generating unit 21, and the compression encoding unit 2
The compressed sub information data from 3 is multiplexed by the multiplexing unit 24, and the multiplexed stream is recorded on the satellite digital broadcasting network 31, the terrestrial digital broadcasting network 32, the CATV network 33, the package recording of the above-mentioned DVD or the like. It is transmitted to the interactive viewing device 50 of the reproduction system 40 by a transmission means such as the medium 34.

The multiplexed stream can also be transmitted to the reproduction side by the communication network 38 such as the Internet. In this case, the multiplexed stream is accumulated in the server device 25 connected to the communication network 38 and transmitted to the interactive viewing device 50 at the request of the reproducing side. The interactive viewing device 50 can reproduce this multiplexed stream.

However, the video program stream and the index picture stream are multiplexed on the multiplexed stream and transmitted to the reproducing side by the satellite digital broadcasting network 31 or the like, and the link data and the sub information data are separately reproduced by the communication network 38 on the reproducing side. The method of multiplexing and the means of transmission, such as transmission to a mobile station, can be selected according to the system.

In the reproduction system 40, the information processing terminal 41 such as a personal computer or an Internet connection terminal is connected to the interactive viewing device 50 via the terminal connection interface 73.

In the interactive viewing device 50, the video program stream, the index picture stream and the link data are synchronously reproduced as shown below, the video program image is displayed on the display 49, and the viewer by the remote commander 79 is displayed. The image object in the video program image is selected by the operation of
Sub-information about the selected image object is obtained.

[Interactive Viewing Device: FIG. 6] FIG. 6 shows an example of the interactive viewing device 50. In this example, the interactive viewing device 50 uses an IRD (Integrated Receive).
ver Decoder) or STB (Set-to)
p Box).

The interactive viewing device 50 includes a CPU 61, and a bus 62 of the interactive viewing device 50 includes a ROM in which a program to be executed by the CPU 61 and fixed data are written, and a C.
A memory 63, which is a RAM functioning as a work area of the PU 61, is connected to the receiving section (light receiving section) 78 for receiving an infrared remote control signal from a remote commander 79.
Are connected.

The multiplexed stream transmitted by the transmission means 36 such as the satellite digital broadcasting network 31 shown in FIG. 5 is at least a video program stream and an index picture stream which have been compression-encoded by the MPEG2 compression method. It is a thing.

The multiplexed stream is developed into a bit stream by the front end 51 and subjected to decoding processing such as error correction and descramble.

In the demultiplexer 52, the transport stream of the MPEG2 system is separated and the MPE
The G decoder 53 separates the audio / video stream and the control information from the transport stream.

The separated audio / video stream is decompressed and decoded by the audio / video decoder 54 into the video data and audio data of the video program. The decompressed video data is image-processed by the video processing unit 55 as described later, converted into an analog video signal by the video output unit 56, and supplied to the display 49 shown in FIG. The expanded voice data is converted into an analog voice signal by the voice output unit 57 and output to a voice output device such as a speaker.

The control information separated by the MPEG decoder 53 is sent to the bus 62 and used for controlling the lower decoders below the audio / video decoder 54.

On the other hand, when the index picture stream is multiplexed in the transport stream of the MPEG2 system without being defined as a subset, the demultiplexer 52 separates the index picture stream as shown in FIG. It is supplied to 59.

The index picture stream is MPE
When the transport stream of the G2 system is defined as a subset and multiplexed, the demultiplexer 52 or the MPEG decoder 53 separates the index picture stream and supplies it to the index picture decoder 59.

In the index picture decoder 59,
The index picture stream is expanded and decoded (reproduced) in synchronization with the expansion and decoding (reproduction) of the video data and audio data in the audio and video decoder 54. The decompressed index picture stream is supplied to the video processing unit 55.

When the link data is multiplexed in the multiplexed stream, the link data is separated by the demultiplexer 52, and the link data interface 65 is separated.
To the bus 62 and is decoded by the CPU 61.

When the sub information data is multiplexed in the multiplexed stream, the sub information data is separated by the demultiplexer 52, and the sub information data interface 66.
Sub-information data about an image object selected by the viewer as described later, such as compressed moving image information that is sent to the bus 62 via the MPEG decoder 53, is supplied to the MPEG decoder 53.
The sub information data to be decoded by the MPEG decoder 53 and the other sub information data to be decoded is decoded by the CPU 61.

The link data and the sub-information data are sent from the demultiplexer 52 via the common interface or directly from the demultiplexer 52 to the bus 6.
It is also possible to have a configuration in which the data is sent to the second.

When the link data or the sub information data is not multiplexed in the multiplexed stream and is transmitted from the server device 25 shown in FIG. 5 through the communication network 38, the link data or the sub information data is
It is taken into the bus 62 via the network interface 67 and processed as described above. If the sub information has a high data rate such as video information,
A dedicated data bus may be used instead of the bus 62 of the CPU 61.

A format converter 69 is connected to the bus 62. In the format conversion unit 69, as described below, the sub information data about the image object selected by the viewer is used as a presentation tool (XML, VRML, H) commonly used in a computer network.
If the data is described in TML, Java (registered trademark), or markup language to be specified in the future), the structure is analyzed and information such as moving image, still image, text, and audio is acquired as sub information. Then, it is decoded, the described effect is added, the display frame having the described structure is formed, the display frame suitable for the characteristics of the display 49 is reconfigured, and the link function indicated by the tag or the hotspot is displayed. The display frame is played back and output by executing a function such as placing it in the display frame.

The interactive viewing device 50 is provided with a hard disk device unit 71 as a storage device unit, and the data such as the video program or sub information received or obtained as described above is transferred to the hard disk device 72 via the hard disk interface 72. The data supplied to the unit 71, recorded on the hard disk, and reproduced from the hard disk are taken into the bus 62 via the hard disk interface 72.

Further, as shown in FIG. 5, the information processing terminal 41 such as a personal computer or an Internet connection terminal is connected to the bus 62 via the terminal connection interface 73, and the interactive information viewing apparatus 50 performs information processing. Various resources provided in the terminal 41 are used.

For example, (1) the access information in the link data about the image object selected by the viewer is transmitted to the information processing terminal 41, and the network access provided by the information processing terminal 41 or the access function such as the optical disk drive is used. Accessing the sub-information resource, (2) using the storage device of the information processing terminal 41 as a medium for recording data such as a video program or sub-information, (3) format conversion of sub-information data or sub-information For example, the software or system of the information processing terminal 41 is used for developing the character code font.

Regarding the above (1), the information processing terminal 41 is
The acquired sub information data is used as a presentation tool (XM
If the data is described in L, VRML, HTML, Java (registered trademark), or markup language to be specified in the future), the structure is analyzed and, as the sub information, a moving image, a still image, a text, a voice, etc. Information is obtained and decoded, the described effect is added, the display frame having the described structure is formed, the display frame suitable for the characteristics of the display 49 is reconstructed, and the display frame is indicated by tags, hot spots, or the like. By executing a function such as arranging the link function in the display frame, the data of the display frame is compressed by the MPEG2 compression method or the compression method used for compressing the video program stream, and the data is transmitted via the terminal connection interface 73. And transmits to the bus 62. The interactive viewing device 50 decodes the compressed display frame data and reproduces and outputs it.

Further, an external storage medium 75 such as a memory card or a rewritable optical disk is connected to the bus 62 via an external storage medium interface 76. The external storage medium interface 76 is an electrical interface when the external storage medium 75 is directly attached to the interactive viewing device 50, and is an external interface when the external storage medium 75 is attached to an external controller. The controller and the bus 62 are connected by wire, wireless or infrared.

The external storage medium 75 records sub-information data selected by the viewer, access information to the sub-information resource, data indicating the viewing tendency and viewing history of the viewer, personal information of the viewer, etc. Data required for various functions described below are stored.

Therefore, the external storage medium 75 can authenticate the viewer individually by the authentication means such as the password of the viewer or biometrics (fingerprint, voiceprint, iris, facial expression, etc.), and protect and record the recorded contents. It is supposed to have the function of canceling the protection of the contents.

When reproducing data such as a video program stream from the hard disk in the hard disk device 71 or the storage device of the information processing terminal 41, not only constant speed reproduction but also variable speed reproduction such as double speed reproduction is performed. The system is configured so that it can.

[Selection of Image Object and Image Processing: FIG. 7] As described above, in the state where the video program stream, the index picture stream and the link data are synchronously reproduced and the video program image is displayed on the display 49, the index is displayed. The picture is not displayed on the display 49 and cannot be visually recognized, but is present in a superimposed manner on the video program image.

In this state, the viewer operates the remote commander 79 to display a cursor on the display 49, and the cursor is placed within or in the area of the image object in which the index object in the video program image is set. When the pointing is performed in the vicinity, the CPU 61 of the interactive viewing device 50 detects the index object including the pointed pixel or closest to the pointed pixel from the index object value of the index picture superimposed on the video program image. Then, the image object on which the detected index object is superimposed is set as the selected image object.

FIG. 7 shows the image objects and index objects selected in this way. The “first selected index object” in the upper part is the index object 1a for the person 1v shown in FIG. The second row of "selected index objects" in the middle row indicates that only the terminal 2v shown in FIG. 1 is selected as the index object 2a. 3 ”indicates that the chair 3v and the person 4v shown in FIG. 1 are selected as the index objects 3a and 4a, respectively.

The CPU 61 performs the following image processing (a) or (b) in the video processing section 55 on the image object thus selected.

(A) A visual effect such as pseudo-stereoscopic visualization by framing or shadowing, or luminance or color tone modulation by highlighting or flicking is applied to the image object.

(B) An icon indicating that the sub-information is added to the image object is displayed at a position on the display 49 that does not hinder visual recognition of the video program image.

As a result, the viewer can know that the image object is the one to which the index object is set and the sub information is added.

The processing control signal sent from the CPU 61 to the video processing unit 55 basically needs only the index object value corresponding to the image object, the image processing method, and the time control information, which is very simple and CP
The burden on U61 is also light.

In addition, the interactive viewing device 50 is configured so that the CPU 61 performs the following image processing on the image object or the sub information.

(1) When the outline of the sub-information is transmitted as the attribute code of the link data, the outline of the sub-information is read from the corresponding link data by the index object value corresponding to the selected image object, On-screen display with characters and icons. This allows the viewer to know the outline of the sub-information added to the image object.

(2) When the category of the image object or the sub-information is transmitted as the attribute code of the link data, the viewer designates the category in advance so that the image object or the sub-information of the designated category can be transmitted. When appears, the image object of the specified category or the image object to which the sub-information of the specified category is added is subjected to the processing as described in (a) or (1) above, or another image. Perform image processing different from objects.

For example, when "furniture" is designated by the viewer and the category of the chair 3v shown in FIG. 1 is "furniture", the image object of the chair 3v is selected by the CPU 61. Therefore, the viewer can know that the object of interest or interest is displayed without any special operation at the time of displaying the image.

(3) The viewing tendency of the viewer is classified by classifying the selection tendency when the viewer selects an image object, using the systematic categories common to various video programs defined in the link data. The tendency is recorded in the hard disk device 71 or the like, and the history thereof is referred to. When an image object belonging to a category having a high selection tendency appears in the video program image, the image processing as described above is performed.

(4) Under the systematized categories common to various video programs as described above, the viewer designates the categories in advance, so that the image objects belonging to the designated categories are imaged. When it appears in the program image, the outline of the sub-information defined in the corresponding link data is fetched and recorded, and displayed on the display 49 by the display instruction operation by the viewer.

(5) The reproduction control information defined in the control code in the link data is used as depth information to stereoscopically visualize the selected image object by various proposed stereoscopic visualization algorithms. Acquires the parallax information and makes it stereoscopic.

(6) The selected image object including its neighboring area is enlarged and displayed at a fixed magnification or an arbitrary magnification.

(7) The moving image object is tracked and displayed while reconstructing the frame of the video program image by various proposed tracking algorithms.

The CPU 61 detects whether or not an index object is set in any of the image objects in the video program image displayed on the display 49 at that time by the viewer's detection instruction operation, When an index object is set for any of the image objects, the above process (a) is executed for all image objects for which the index object is set, or the above (b)
To notify the viewer that there is an image object to which the sub-information is added, and then by the pointing operation of the viewer, within or near the area of the specific image object as described above. It is also possible to configure the interactive viewing device 50 so as to execute the processes (1), (5), (6), (7) and the like on the image object when is pointed at.

Further, the video program provider forcibly designates the image object corresponding to the index object value defined in the link data as a processing target object by the control code of the link data, and the CPU 61 causes the CPU 61 to The system can also be configured to perform the above-described image processing on the image object forcibly specified.

[Acquisition of Sub Information by Center Access Method: FIGS. 8 to 11] In the interactive viewing system as shown in FIG. 5, for example, a video program stream, an index picture stream, and link data are multiplexed to obtain a satellite digital signal. When the sub-information data is transmitted to the reproduction side by the transmission means such as the broadcast network 31 and the sub-information data is transmitted to the reproduction side by the communication network 38 such as the Internet, the sub-information or the CM information is acquired and the EC described above is obtained.
For example, the following center access system can be constructed.

(System Configuration: FIG. 8) FIG. 8 shows an example of this system.

In this example, the broadcasters 81 and 82
Respectively transmit the above multiplexed streams to the interactive viewing device 50 via the satellite digital broadcasting network 31.

Under the broadcasters 81 and 82, a content server 85, a timetable server 86, and a sub information provider charging DB (database) 87 are provided and connected to the communication network 38. The interactive viewing device 50 is also connected to the communication network 38.

The system is provided with a management center 90 of any one of the following forms in order to manage the provision of sub-information and the charge for it.

In the first embodiment, a management center 90 is provided for each broadcaster. The operating expenses of the management center 90 are borne by the broadcasting company. Therefore, the management center 90
May be the same entity as the broadcaster.

In this case, in the interactive viewing device 50,
The center access information is extracted from the description information in the video program or the tuning information, so-called metadata, to access the management center 90. If the center access information is defined in the definition code of the link data, the management center 90 can be accessed by this.

In the second form, one management center 90 is provided for each of a plurality of broadcasters. For example, broadcasters in a certain area jointly operate the management center 90. Operating expenses will be shared by each broadcaster, or will be covered by acting as an EC or providing CM. In this case, the center access information is set in the interactive viewing device 50 in advance.

In the third form, the management center 90 is operated independently. The management center 90 receives the provision of the sub information resource from each broadcaster, and provides only the service to the viewers of the members. Management is based on membership fees from members.

This form has the advantages that personal information can be more surely preserved, EC settlement can be unified, and services such as delivery can be unified.

Under the management center 90, FIGS.
The EC server 91, the sub information resource DB 92, the CM resource DB 93, and the viewing history D according to each case as shown in FIG.
B94, viewing tendency DB95, personal information DB96, viewer billing DB97, CA (conditional access) customer information DB98, and credit settlement system server 99 are provided and connected to the communication network 38.

Further, in the communication network 38, the EC server 101 of the business operator 111, the CM server 102 of the business operator 112, and the sub information server 1 of the sub information producer 113.
Various server groups 100 such as 03 are connected. The business operator 111 is a sales company or maker that performs EC, and the business operator 112 is a sales company or maker that provides CM.

Further, in the communication network 38, the broadcast history DB 12 of the certification body 120 independent of each business
1 and the viewing history DB 122 are connected.

In the following, the case where the broadcaster 81 broadcasts the above-mentioned multiplexed stream will be described.

(First case: FIG. 9) FIG. 3 shows the first case.
In the case where, as access information as shown in (C) to (E), a sub data resource in which EC content (various information such as sales information about electronic commerce) or CM information is added to link data is directly defined. Is.

In this case, (1) the broadcaster 111 makes a contract to the broadcaster 81 for a fixed amount (paying a fixed price) or a pay-as-you-go (paying a price according to the number of viewers' sub-information requests). Under the contract, the product to be EC is used as a part of the image object of the video program, and the index object and the link data are set for the image object.

Alternatively, (2) an EC target product of the business operator 111 is placed under the control of the management center 90, and the management center 90 provides the EC content to the viewer, and the sales procedure and credit settlement of the product. In order to obtain the sales commission, the broadcaster 81 uses the EC target product of the provider 111 as a part of the image object of the video program, and the index object and the link data are added to the image object. To set.

In the first case, in the interactive viewing device 50, when the image object is selected as described above, the sub-information resource defined in the link data as described above, that is, in FIG. As shown,
A request for sub information is issued to the EC server 101 of No. 1, and as a response, the sub information is sent from the resource side to the interactive viewing device 50.

When the contract between the broadcasting company 81 and the company 111 is a fixed amount contract, it may be completed only by the above process.

On the other hand, the broadcasting company 81 and the company 1
If the contract with 11 is a pay-as-you-go contract,
It is necessary for the broadcaster 81 and the provider 111 to share the amount of sub-information provided.

Therefore, in this case, the interactive viewing device 50 is further connected to the management center 90 or the broadcaster 81.
Is reported that there was a request for side information.

When the broadcaster 81 and the operator of the management center 90 are the same, this request report is issued by the broadcaster 8
1 is accumulated by the sub information provider billing DB 87, and the charge corresponding to the accumulated result is transferred from the broadcasting company 81 to the company 111.
Will be charged to.

When the broadcaster 81 and the operator of the management center 90 are different, the broadcaster 81 grasps the number of requests for sub-information, so that the management center 90 sends the broadcaster 8
The request report is issued to No. 1 and the request report is accumulated by the sub information provider charging DB 87 of the broadcaster 81 as described above.

Furthermore, when it is necessary to ensure fairness, a sub information request report is also issued to the certification body 120.

When the requested sub information is EC content, the EC content as sub information is sent from the EC server 91 of the management center 90 to the interactive viewing device 50, not from the EC server 101 of the business operator 111. ,
The system can also be configured so that when the viewer purchases a product targeted for EC, the credit settlement system server 99 provided in the management center 90 completes the transaction.

(Second Case: FIG. 10) In the second case, the access information as shown in FIGS.
This is a case where the CM resource and the sub information resource are directly defined in the link data.

In this case, the broadcaster 112 is the broadcaster 81
In addition, under the fixed-rate or pay-as-you-go contract as described above, index objects and link data for image objects that are not directly commercial,
Ask the CM resource and sub information resource to be set.

In the second case, in the interactive viewing device 50, when the image object is selected as described above, the CM resource and the sub-information resource defined in the link data as described above, that is, As shown in FIG. 10, a CM request and a sub-information request are issued to the CM server 102 of the business operator 112 and the sub-information server 103 of the sub-information producer 113, and as a response, from each resource side. The CM information and the sub information are sent to the interactive viewing device 50.

In this case, in order to ensure that the CM information is transmitted to the viewer, it is necessary for the interactive viewing device 50 to adopt a method in which the sub information is reproduced after the reproduction of the CM is completed.

As an example, the sub information is scrambled, and the scramble release key is dispersed in the CM information, and the interactive viewing device 50 collects the dispersed release key during the reproduction of the CM, and When the playback of is completed,
The system is configured so that the descrambling key is generated and the sub information can be reproduced.

Alternatively, when the reproduction of the CM ends, as shown in FIG. 10, the interactive center device 90 requests the management center 90 for a scramble release key, so that the management center 90 scrambles the interactive center device 50. The release key may be sent.

Also in this second case, as in the first case shown in FIG. 9, request reports are issued and accumulated.

(Third Case: FIG. 11) In the third case, the procedure for acquiring the sub information by the center access is defined in the link data without directly defining the sub information resource, and the CM is sent from the management center 90. This is the case when the resource and the sub information resource are specified.

In this case, (1) the CM provided for a specific image object (sub-information) is fixed, and the sub-information provider (the business operator 112 and the sub-information producer 113) informs the management center 90 of the above. Under a flat-rate or pay-as-you-go contract such as the above, the relationship between the CM resource and the sub information resource and the index object is registered for the image object that is not directly commercial, and the sub information from the viewer is registered. As a response to the request, the CM resource and the sub information resource of the sub information provider (the business operator 112 and the sub information producer 113) are sent from the management center 90 to the interactive viewing device 50.

Alternatively, (2) a CM provided to a specific image object (sub-information) is not fixed, and a plurality of CM providers make a contract with the management center 90 for the above-mentioned meter-based contract. Based on this, the relationship between each CM resource and sub information resource and the index object is registered for the image object that is not directly commercialized, and the management center is used as a response to the sub information request from the viewer. 90, the CM resource and the sub-information resource of the CM provider selected at random or by a specific procedure are sent to the interactive viewing device 50.

Alternatively, (3) a CM provided to a specific image object (sub-information) is not fixed, and a plurality of CM providers make a contract with the management center 90 as described above for the pay-as-you-go contract. Based on the above, the relationship between each CM resource, CM attribute and sub information resource and the index object is registered for an image object that is not directly commercialized, and as a response to the sub information request from the viewer. The management center 90 sends the interactive viewing device 50 the CM resource and the sub-information resource of the CM provider selected as having the CM attribute that matches the preference attribute of the viewer.

In the third case, the interactive viewing device 50 accesses the management center 90 by the center access information in the above-described metadata or the center access information preset in the interactive viewing device 50, CM
Receives designation of resources and sub information resources.

In this case, when the CM provided for the specific image object (sub-information) is fixed as in (1) above, as shown in FIG. In response to the sub information request of, the management center 90 outputs the information of the CM resource and the sub information resource from the CM resource DB 93 and the sub information resource DB 92, and sends the information to the interactive viewing device 50.

In the interactive viewing device 50, according to the acquired resource information, as in the second case shown in FIG. 10,
A request for CM and a request for sub-information are issued to the CM resource and the sub-information resource, and in response thereto, the interactive viewing device 5 from each resource side.
0, CM information and sub information are sent.

When the contract between the broadcaster 81 and the CM provider or sub-information provider is a pay-as-you-go contract, as in the first case shown in FIG. 9, the interactive viewing device. CM from 50 to management center 90 or broadcaster 81
And that there was a request for side information.

CM resource DB provided in the management center 90
In the case where the CM attribute is recorded together with the CM resource in 93, C according to the preference of the viewer as described in (3) above.
The system can be configured so that M is provided to the viewer.

In this case, in the management center 90, the viewer's preference is analyzed by the viewing history DB 94, the viewing tendency DB 95, the personal information DB 96, etc., and the CM information and the sub information suitable for the assumed viewer's preference attribute are analyzed. An information resource is selected and the resource information is sent to the interactive viewing device 50.

According to this method, the viewer can mainly watch the CM that suits his / her taste, and the CM provider can expect an increase in the advertising effect.

To realize this function, the management center 90
In the request data to the user, identification information for identifying the individual viewer or the interactive viewing device 50 is required.
Generally, it is considered that conditional access control is performed in digital broadcasting. Therefore, providing the CA customer information DB 98 in the management center 90 is effective for collecting and analyzing the taste attributes of viewers.

[CM provision frame setting method: FIG. 12 and FIG. 13] According to the method of the present invention, since the CM publisher can provide the CM in association with the image object in the video program image, the broadcast can be performed. A video program provider such as a business operator can provide a CM offer frame to a CM publisher by an effective method different from the conventional method.

In the first method, as shown in FIG. 12, when the video program is composed of the periods Pa, Pb, Pc, ..., The head periods Pa1, Pb1, Pb1, Pb1, Pc. Pc1 ... is the compulsory provision frame C
a, Cb, Cc ... are CM publishers Xa, X, respectively.
b, Xc ... and the period Pa1, Pb1,
In the periods Pa2, Pb2, Pc2, ... Immediately after Pc1 ..., the CM publisher X enters the compulsory provision frames Ca, Cb, Cc, ...
a, Xb, Xc ...

For example, the submitter Xa can place a CM in the compulsory provision frame Ca in the period Pa1, and also in the period Pa2 immediately after that, the sub information added to the image objects shown as objects 1 to 5 appearing therein. Per C
It is a method that can issue M.

In the second method, as shown in FIG. 12, when the video program is composed of the periods Pa, Pb, Pc ..., The head periods Pa1, Pb1, Pb1, Pb1, Pc. Pc1 ... is the compulsory provision frame C
a, Cb, Cc ... are CM publishers Xa, X, respectively.
b, Xc ..., but period Pa1, Pb1, Pc1
Immediately after the period Pa2, Pb2, Pc2, ..., CM publishers Xa, X to the compulsory provision frames Ca, Cb, Cc, ...
b, Xc ... Provided to CM publishers different from those.

According to this method, more businesses can provide commercials within the limited time of the video program, and also as a video program provider such as a broadcast business, the profit structure is multi-layered. Can be realized.

In the third method, no compulsory provision frame is provided in the video program, and all CMs are provided when the sub information is acquired.

For example, as shown in FIG. 13, (1) the provision frame C1 is set in the period from time t1 to time t2,
The image objects appearing in this are provided to a CM advertiser, and (2) the offer frame C2 is set for the image objects excluding the specific image object during the period from time t2 to time t3 As shown by (3) offer frame C3, the offer frame is set for a specific image object for a certain period of time as shown by (3) offer frame C3, and provided to a certain CM publisher, and shown by (4) offer frame C4. like,
For example, a provision frame is set for a specific image object for a limited period and the image is provided to a CM publisher.
In the case of FIG. 13, (1) and (3) compete for the object 5, but in that case, one of them is selected.

For example, the clothes and accessories worn by the actors who appeared in the drama, the furniture used in the drama, the fixtures and dishes shown in the cooking program, etc. are provided free of charge by the sponsoring company, and in return the end rolls. In many cases, only the company name is announced, but this has a small advertising effect.

According to the third method, in this case, the sponsoring company can more directly advertise and sell the product or service appearing as an image object in the video program.

Since it is considered that the viewer is also interested in these objects, according to the third method, a system for directly connecting such a viewer and the CM provider is provided. Can be configured.

[0195]

As described above, according to the present invention, the sub information, which is the information added to the specific image object in the video program image, can be easily and surely provided to the viewer, and the viewer can easily perform the operation. And it can be acquired reliably.

Further, various kinds of information can be added as the sub information, and the sub information can be easily added or changed as needed.

Further, image processing such as visual effect addition and tracking processing can be easily performed on a specific image object, and it can be clearly indicated that sub information is added.

Further, by being combined with a computer network, a video program provider such as a broadcasting company, a business company that provides commercials as sub information, a business company that performs electronic commerce, etc., through an image object or sub information as a medium. And a viewer (consumer) are directly connected to each other, and a system for providing and acquiring a video program, sub-information, products and services can be easily constructed, and each individual's request can be satisfied. You can create a place.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a video program and an index picture.

FIG. 2 is a diagram illustrating an example of an index picture generation device.

FIG. 3 is a diagram showing an example of a data structure of link data.

FIG. 4 is a diagram for explaining a method of transmitting link data.

FIG. 5 is a diagram showing an example of an interactive viewing system.

FIG. 6 is a diagram showing an example of an interactive viewing device.

FIG. 7 is a diagram for explaining selection of an image object.

FIG. 8 is a diagram showing an example of a sub information acquisition system.

9 is a diagram showing data transmission in a first case in the system of FIG.

10 is a diagram showing data transmission in a second case in the system of FIG.

11 is a diagram showing data transmission in a third case in the system of FIG.

FIG. 12 is a diagram showing an example of a new method for a CM providing frame.

FIG. 13 is a diagram showing an example of a new method for a CM providing frame.

[Explanation of symbols]

Since the main parts are all described in the figure, they are omitted here.

Claims (17)

[Claims] 1. The same video program for each pixel in an index object area which is an area to be superimposed on a specified image object area in the video program image on the virtual screen to be superimposed on the video program image. In the step of setting a value capable of specifying the image object in the step of generating index picture data which is data different from the video program data, and the information added to the specified image object. A data generation method comprising: a step of generating, as link data, data that defines information for accessing certain sub information in association with a value of an index object corresponding to the image object. 2. A data transmission method for multiplexing video program data, index picture data and link data generated by the data generation method according to claim 1, and transmitting the multiplexed data to a reproducing side by the same transmission means. 3. Video program data, and index picture data and link data generated by the data generating method according to claim 1, are capable of being synchronously reproduced.
A data transmission method in which data is transmitted to the reproducing side by another transmission means. 4. The same video program for each pixel in an index object area, which is an area overlaid on a specified image object area in the video program image on the virtual screen overlaid on the video program image. And a means for generating an index picture data, which is a data different from the video program data, by setting a value capable of specifying the image object, and the information added to the specified image object. A data generating device comprising: a unit that generates, as link data, data that defines information for accessing certain sub-information in association with a value of an index object corresponding to the image object. 5. The data generation device according to claim 4, further comprising means for multiplexing the video program data, the index picture data and the link data. 6. A means for reproducing the transmitted video program data to display the video program image on a display, and a specified image object in the video program image on a virtual screen superimposed on the video program image. For each pixel in the index object area that is an area to be overlapped with the area, a value that can specify the image object in the same video program is set, and data different from the video program data is set. Means for reproducing the index picture data generated and transmitted as the above in synchronization with the video program data, and the information for accessing the sub information that is the information added to the specified image object, Specified in association with the value of the index object corresponding to Means for reproducing the link data generated and transmitted by synchronizing with the index picture data, and pointing means for pointing to an image object in the video program image displayed on the display based on an instruction operation. , When there is an index object corresponding to the designated image object, means for referring to the link data corresponding to the image object by the value of the index object, and access defined in the referred link data. A video program reproducing apparatus comprising: means for accessing sub information by information to acquire the sub information; and means for reproducing the acquired sub information. 7. The video program reproducing apparatus according to claim 6, further comprising means for notifying that an index object corresponding to the image object designated by the pointing means is present. 8. A video program reproducing apparatus according to claim 6, further comprising means for, when an index object corresponding to the image object designated by said pointing means is present, subjecting said image object to image processing different from other image objects. A video program playback device provided. 9. The video program reproducing apparatus according to claim 6, further comprising means for notifying that when an index object is superimposed on any image object in the video program image displayed on the display. A video program playback device provided. 10. The video program reproducing apparatus according to claim 6, wherein when an index object is superimposed on any image object in the video program image displayed on the display, the index object is superimposed. A video program reproducing apparatus comprising means for performing image processing on an existing image object different from that of an image object on which an index object is not superimposed. 11. The video program reproducing apparatus according to claim 6, further comprising a network interface as means for accessing the sub information and acquiring the sub information. 12. The video program reproducing apparatus according to claim 6, comprising a storage unit for recording the acquired sub-information, or an interface to which the storage unit is connected. 13. A step of reproducing the transmitted video program data to display the video program image on a display, and a specified image object in the video program image on a virtual screen superimposed on the video program image. For each pixel in the index object area that is an area to be overlapped with the area, a value that can specify the image object in the same video program is set, and data different from the video program data is set. And a step of playing back the index picture data generated and transmitted as the video program data, and the information for accessing the sub information that is the information added to the specified image object is the image object. Specified in association with the value of the index object corresponding to Reproducing the link data generated and transmitted by synchronizing with the index picture data, and instructing an image object in the video program image displayed on the display based on an instruction operation. , A step of referring to the link data corresponding to the image object by the value of the index object when the index object corresponding to the instructed image object exists, and the access specified in the referenced link data. A video program reproducing method comprising: a step of accessing sub information by information to acquire the sub information; and a step of reproducing the acquired sub information. 14. The video program reproducing method according to claim 13, comprising a step of recording the link data or the sub information. 15. The video program data and each pixel in an index object area which is an area to be superimposed on a specified image object area in the video program image on the virtual screen to be superimposed on the video program image. , A recording in which a single value that can identify the image object in the same video program is set, and index picture data generated as separate data from the video program data is recorded in a multiplexed manner. Medium. 16. The recording medium according to claim 15, wherein the information for accessing the sub information, which is information added to the specified image object by being multiplexed with the multiplexed data, is the image object. A recording medium in which link data generated by being defined in association with a value of an index object corresponding to is recorded. 17. The recording medium according to claim 15, wherein sub information, which is information added to the specified image object, is added to the multiplexed data or separately from the multiplexed data. Recorded recording medium.