JP4664173B2 - Transmitting apparatus, reproducing apparatus, and methods for subtitle concealment broadcasting - Google Patents

Description

  The present invention relates to a transmission device, a playback device, and methods for subtitle concealment broadcasting.

  News bulletins such as disaster information and election information are broadcast by superimposing a caption image on an original image constituting a normal broadcast. As a result, the viewer can obtain real-time information while watching the program. However, since there is a case where the broadcast is once recorded on the storage medium and viewed later, there is information that was useful at the time of broadcasting but is not useful at the time of actual viewing.

Patent Document 1 discloses a method for hiding unnecessary character information. FIG. 14 is an explanatory diagram showing a conventional character concealment method described in Patent Document 1. In FIG. In FIG. 14, the partial image before the insertion of the character corresponding to the character portion is transmitted as a part of the transmission bit string and synthesized on the receiving side. The code classification unit 1 separates the received signal into a main image on which subtitles are superimposed and a partial image before the subtitles are superimposed, and outputs them to the main image code decoding unit 3 and the sub image code decoding unit 5, respectively. The main image code decoding unit 3 and the partial image code decoding unit 5 decode the main image and the sub image, respectively. The operation unit 9 receives an operation from the viewer and outputs the operation to the control unit 7. The superimposing unit 11 outputs an image based on the main image and the sub image based on a command from the control unit 7. At this time, when the viewer performs an operation for outputting the main image, the main image is output. On the other hand, when an operation for outputting the original image is performed, the main image and the sub-image are combined and the subtitle portion is overwritten and the erased image is output.
JP 2000-41185 A

  In the configuration of Patent Document 1, considering the case where playback after recording is performed once, if digital recording is performed as it is, it is possible to select whether or not to display subtitles during playback. However, in this case, the viewer must manually select which one to select. In other words, when it is played back after it has been accumulated, it is necessary for the viewer to know the current time explicitly, compare the current time with the broadcast time of the content, and manually set whether or not to display captions appropriately. is there. In addition, since the viewer cannot know the time limit information such as campaign information for a limited period before reproduction, it is difficult to switch display / non-display of the limited period display as intended by the broadcaster. Further, a storage medium such as a recent DVD player can store a large amount of data, and switching the presence / absence of subtitle display with respect to the large amount of stored data is not easy for the viewer.

  SUMMARY OF THE INVENTION The present invention solves the above-described conventional problems, and an object of the present invention is to provide a transmission device, a playback device, and a method thereof related to closed caption concealment broadcasting that have good operability for viewers.

  In order to solve the above-described problem, the first invention of the present application is a main image generating means for generating a main image by superimposing first superimposition data on an original image, and an image of a portion superposed on the first superimposition data (hereinafter referred to as “superimposition data”). , Referred to as a partial image) from the original image, sub-image generation means for generating a first sub-image by superimposing second superimposition data on the partial image, the main image and the first sub-image And a transmission data generating means for generating transmission data by associating the main image with the broadcast time of the main image.

  The first superimposition data is insertion data such as subtitles directly connected to the broadcast time such as breaking news, earthquake information, and weather forecast. On the other hand, the second superimposition data is insertion data that can be reproduced at any time, for example, there is a range in a broadcastable period such as a commercial or there is no effective broadcast period. Also, the transmission device is, for example, a broadcasting station, and generates a main image and a first sub-image by superimposing the first and second superimposed data on the original image and the partial image, respectively, and transmits them to the playback device on the user side. The playback device plays back the main image and the first sub-image received from the transmission device, but temporarily stores them when the user does not view them immediately. When the time when the user of the playback device views (hereinafter referred to as the current time) is approximately the same as the broadcast time, the playback device plays back the main image. On the other hand, when the current time is not the same as the broadcast time, the playback device plays back an image in which the first sub-image is superimposed on the main image (hereinafter referred to as the first superimposed image). That is, the portion of the main image on which the first superimposed data is superimposed is covered with the first sub image. Thereby, the user can enjoy viewing without being bothered by obsolete information whose time has elapsed from the broadcast time. Further, since the playback device automatically switches between the first superimposed data and the second superimposed data, the user can view the video and music corresponding to the current time without performing an operation for viewing. .

  A second invention of the present application is the first invention, wherein the sub-image generating means further generates a second sub-image by superimposing third superimposition data on the partial image, and the transmission data generating means includes the main image and The second sub-image and the second sub-image are further associated with each other, the first sub-image is associated with first time limit information indicating a period during which the first sub-image can be reproduced. The transmission apparatus is characterized in that the transmission data is generated in association with the second deadline information indicating the period in which the data can be reproduced.

  The transmission device generates transmission data including a first sub image and a second sub image with time limit information in addition to the main image. The reproduction device compares the current time with the broadcast time of the main image, and reproduces the main image when the current time is the same as the broadcast time. On the other hand, when the current time is different from the broadcast time, the current time is compared with the first deadline information and the second deadline information. Then, when the current time is within the valid period indicated by the first time limit information, the playback device plays back the first sub-image superimposed on the main image. On the other hand, if the current time is within the valid period indicated by the second time limit information, the playback device plays back an image in which the second sub-image is superimposed on the main image (hereinafter referred to as a second superimposed image). As described above, since the image can be switched according to the current time, the user can enjoy viewing the image corresponding to the current time. In addition, the user himself / herself can view video and music corresponding to the current time without performing an operation for switching images.

  A third invention of the present application provides the transmission device according to the first invention, wherein the transmission data generation means generates transmission data by further associating the main image and the partial image.

  The transmission data includes a main image and its broadcast time, a first sub-image and its first deadline information, a second sub-image and its second deadline information, and a partial image. For example, when the current time by the playback device has elapsed since the broadcast time and is not within the first time limit information and the second time limit information, the playback device plays back the partial image superimposed on the main image. The partial image is an image obtained by cutting out the overlapping region with the first superimposed data from the original image, and superimposed data such as captions is not superimposed. Therefore, by superimposing the partial image on the main image, the user can view the original image without being bothered by the obsolete first superimposition data whose time has elapsed from the broadcast time.

  According to a fourth aspect of the present invention, in the first aspect, the transmission data generation means further generates transmission data by associating the first sub-image with coordinate information of the partial image in the original image. A transmitting device is provided.

  The coordinate information indicates the position of the partial image in the original image, and the first superimposed data of the main image can be completely hidden by superimposing the first sub-image on the main image based on the coordinate information. Thereby, the user can enjoy viewing without being bothered by the obsolete first superimposed data whose time has elapsed from the broadcast time.

  According to a fifth aspect of the present invention, in the first aspect, the transmission data generation means includes a packet that forms the main image and the first sub image so that the main image and the first sub image are associated with each other. Provided is a transmitter characterized by adding a time stamp or a packet ID.

  For example, the time stamp of the main image describes the broadcast time of the main image, and the time stamp of the first sub-image describes the broadcast time of the corresponding main image. In this case, the main image and the first sub image can be associated with each other according to the broadcast time. The association between the main image and the first sub-image using the time stamp is not limited to the broadcast time, and other time information may be used. Further, for example, each packet of the main image is assigned with a packet ID, and the packet of the first sub image is assigned with the packet ID of the corresponding main image, so that the main image corresponds to the first sub image. Put it on. In this way, by associating the main image and the related first sub-image, the playback device can accurately superimpose the main image and the first sub-image corresponding to each other.

  A sixth invention of the present application is the transmission according to the fifth invention, wherein the time stamp or the packet ID is a PTS (Presentation time stamp) or a PID (Packet Identifier) defined in the ISO / IEC13818-1 standard, respectively. Providing equipment. The main image and the first sub-image can be associated with each other using PTS or PID according to the existing ISO / IEC13818-1 standard.

  In a seventh aspect of the present invention, in the first aspect, the transmission data generation means transmits the first sub-image corresponding to the main image in a region where arbitrary data can be described accompanying the main image. Provided is a transmission device characterized by generating data. The transmission device describes the first sub-image corresponding to an area in which arbitrary data accompanying the main image can be described, and generates transmission data. Therefore, the playback apparatus can superimpose the first sub-image on the main image by playing back the first sub-image described in the area where arbitrary data accompanying the main image can be described together with the main image.

  According to an eighth aspect of the present invention, in the seventh aspect, an area that can be described with arbitrary data associated with the main image is a user data area of the ISO / IEC13818-2 standard, and the transmission data generation means includes the first sub-data generation means. Converting the image into a user data format defined in the ISO / IEC13818-2 standard, and generating transmission data by describing the first sub-image converted into the user data format in the user data area associated with the main image. A characteristic transmission apparatus is provided. The main image and the first sub-image can be associated with each other using existing ISO / IEC13818-2 standard user data.

  A ninth invention of the present application provides the transmission device according to the eighth invention, wherein the first sub-image converted into the user data format is arranged in a user data area in units of pictures of the corresponding main image. .

  Since the first sub image is arranged in the user data area for each main picture picture unit, that is, for each frame unit of the main image, the main image and the first sub image can be reliably associated with each other. In addition, when a main image is selected in the playback device, a first sub-image corresponding to the main image can be acquired at the same time. Therefore, when the main image is edited, it is possible to prevent the first sub image from being lost by editing the main image and the first sub image separately.

  The tenth invention of the present application further includes address acquisition means for acquiring an address on the World Wide Web (WWW) in which the second sub-image in which the third superimposed data is superimposed on the partial image is stored in the first invention, The transmission data generation means further associates the main image with the address of the second sub-image, and transmits the first sub-image and the first time limit information capable of reproducing the first sub-image in association with each other. Provided is a transmission device characterized by generating data.

  The second sub-image is stored on the WWW, and the transmission device transmits the address at which the second sub-image is stored, not the second sub-image itself, to the playback device. The playback device can acquire the second sub-image by accessing this address. Then, the playback device plays back the main image, the first superimposed image, or the second superimposed image based on the comparison result between the current time and the broadcast time. That is, the playback device outputs the main image if the current time is the same as the broadcast time, outputs the first superimposed image if the current time is within the first time limit information, and the first time limit even if the current time is the broadcast time. If it is not in the information, an image in which the second sub-image is superimposed on the main image is output.

  The eleventh invention of the present application includes a main image generating means for generating a main image by superimposing first superimposing data on an original image, and a partial image corresponding to a superimposing region of the first superimposing data extracted from the original image; Address acquisition means for acquiring an address on the WWW storing the first sub image on which the second superimposition data is superimposed, the main image and the address of the first sub image are associated with each other, and the main image And a transmission data generating means for generating transmission data in association with the broadcast time of the main image.

  The first sub-image is stored on the WWW, and the transmission device transmits the address at which the first sub-image is stored instead of the first sub-image itself to the playback device. The playback device can obtain the first sub-image by accessing this address. Then, the playback device plays back the main image or the first superimposed image based on the comparison result between the current time and the broadcast time as in the first invention of the present application.

  In a twelfth aspect of the present invention, a second superimposed data is applied to a main image in which first superimposed data is superimposed on an original image, and an image of a portion extracted from the original image and superimposed on the first superimposed data (hereinafter referred to as a partial image). Is a reproduction device that reproduces transmission data generated by associating the first sub-image on which the main image is superimposed with the main image and the broadcast time of the main image, wherein the main time and the main image are associated with each other. Comparing means for comparing the broadcast time of the image, and an image obtained by superimposing the first sub-image on the main image or the main image based on the comparison result of the comparing means (hereinafter referred to as first superimposed image) And an output means for outputting. The same effects as the first invention are achieved.

  In a thirteenth aspect of the present invention based on the twelfth aspect, the transmission data further includes a correspondence between the main image and a second sub-image in which third partial data is superimposed on the partial image, The first sub-image that can reproduce the first sub-image is associated with the second sub-image and the second sub-image that can reproduce the second sub-image, and the second sub-image is generated in association with each other. The comparison means compares the current time with the broadcast time, the first deadline information, and the second deadline information, and the output means, based on the comparison result of the comparison means, the main image, There is provided a playback device that outputs the first superimposed image or an image obtained by superimposing the second sub-image on the main image (hereinafter referred to as a second superimposed image). The same effects as the second invention are achieved.

  In a fourteenth aspect of the present invention based on the twelfth aspect, the transmission data is generated by further associating the main image and the partial image, and the output means is based on a comparison result of the comparison means. There is provided a playback apparatus that outputs the main image, the first superimposed image, or an image in which the partial image is superimposed on the main image (hereinafter referred to as a partially superimposed image). The same effects as the third invention are achieved.

  A fifteenth aspect of the present invention is the reproducing apparatus according to the twelfth aspect, wherein the transmission data is generated by associating the first sub-image with coordinate information of the partial image in the original image. provide. The same operation effect as the 4th invention is produced.

  According to a sixteenth aspect of the present invention, in the twelfth aspect, the transmission data includes time stamps in packets constituting the main image and the first sub image so that the main image and the first sub image are associated with each other. Alternatively, there is provided a playback device characterized in that a packet ID is added. The same effects as those of the fifth invention are achieved.

  A seventeenth invention of the present application is the reproducing device according to the sixteenth invention, wherein the time stamp or the packet ID is a PTS (Presentation time stamp) or a PID (Packet Identifier) defined in the ISO / IEC13818-1 standard, respectively. I will provide a. The same effect as that of the sixth invention is achieved.

  In an eighteenth aspect of the present invention, in the twelfth aspect, the transmission data includes a first sub-image corresponding to the main image described in an area accompanying the main image in which arbitrary data can be described. Is provided. The same effects as the seventh invention are achieved.

  According to a nineteenth aspect of the present invention, in the eighteenth aspect, an area that accompanies the main image and can describe arbitrary data is a user data area of ISO / IEC13818-2 standard, and the transmission data accompanies the main image. Provided is a playback device characterized in that a first sub-image converted into a user data format defined in the ISO / IEC13818-2 standard is described and generated in a user data area. The same operation effect as the 8th invention is produced.

  A twentieth aspect of the present invention provides a playback apparatus according to the nineteenth aspect, wherein the first sub-image converted into the user data format is arranged in a user data area in units of pictures of the corresponding main image. . The same effects as those of the ninth invention are achieved.

  In a twenty-first aspect of the present invention based on the twelfth aspect, the transmission data is associated with the main image and an address on the WWW where the second sub-image in which the third superimposed data is superimposed on the partial image is stored. In addition, the first sub-image and the first time-limit information capable of reproducing the first sub-image are generated in association with each other, and the comparing means includes the current time, the broadcast time, and the first time-limit. Image acquisition means for comparing the information and the second deadline information, and accessing the address of the second sub-image based on the comparison result of the comparison means to acquire the first sub-image, The output means provides a playback device characterized in that the main image or a first superimposed image obtained by superimposing the first sub-image acquired by the image acquisition means on the main image is output. The same operation effect as the 10th invention is produced.

  According to a twenty-second aspect of the present invention, a second superimposed data is applied to a main image in which first superimposed data is superimposed on an original image, and an image of a portion extracted from the original image and superimposed on the first superimposed data (hereinafter referred to as a partial image). And a reproduction apparatus that reproduces transmission data generated by associating the main image and the broadcast time of the main image with the address on the WWW in which the first sub-image on which is superimposed is stored And comparing the current time with the broadcast time of the main image, and obtaining the first sub-image by accessing the address of the first sub-image based on the comparison result of the comparison unit. Output means for outputting a first superimposed image obtained by superimposing the main image or the first sub-image obtained by the image obtaining means on the main image based on the comparison result obtained by the image obtaining means and the comparison means; Including and with That provides a playback device. The same operation effect as the 11th invention is produced.

  In the twenty-third invention of the present application, a main image generation step of generating a main image by superimposing first superimposition data on an original image, and an image of a portion to be superimposed on the first superimposition data (hereinafter referred to as a partial image) The main image and the first sub-image are associated with each other, the extraction step for extracting from the sub-image, the sub-image generation step for generating the first sub-image by superimposing the second superimposition data on the partial image, and the main image And a transmission data generation step for generating transmission data in association with the broadcast time of the main image. The same effects as the first invention are achieved.

  According to the twenty-fourth aspect of the present invention, the second superimposed data is applied to the main image in which the first superimposed data is superimposed on the original image, and the image of the portion extracted from the original image and superimposed on the first superimposed data (hereinafter referred to as the partial image). Is a reproduction method for reproducing transmission data generated by associating the main image with the broadcast time of the main image, wherein the current time and the main image are associated with each other. A comparison step of comparing the broadcast time of the image, and an output step of outputting a first superimposed image in which the first sub-image is superimposed on the main image or the main image based on the comparison result in the comparison step; The reproduction method characterized by including this is provided. Effects similar to those of the twelfth invention are achieved.

  According to the present invention, it is possible to provide a transmission device, a playback device, and a method thereof related to closed caption concealment broadcasting that are easy to operate for a viewer.

<Outline of the invention>
A transmission apparatus such as a broadcast station transmits transmission data including video, audio, and the like to a reproduction apparatus on the user side. The user can enjoy the program by the playback device playing back the transmission data. Note that the playback device temporarily stores transmission data when the user does not use the device immediately.

  FIG. 1 is an explanatory diagram showing transmission data generated by the transmission apparatus of the present invention. As shown in FIG. 1A, the transmission device generates a main image by superimposing the first superimposed data on the original image. Here, the original image refers to an image on which superimposition data such as subtitles is not superimposed. In addition, examples of the first superimposition data include insertion data such as subtitles directly connected to broadcast times such as breaking news, election information, earthquake information, and weather forecast. Further, as shown in FIG. 1B, the transmission device extracts a partial image corresponding to the first superimposed data from the original image, and superimposes the second superimposed data different from the first superimposed data on the partial image. A first sub-image is generated. As the second superimposition data, for example, insertion data that has a wide broadcastable period such as a commercial or has no effective broadcast period can be cited. The transmission device transmits transmission data including the generated main image, the broadcast time of the main image, and the first sub-image to the reproduction device. At this time, the transmission device associates the main image with the first sub-image, and further associates the main image with its broadcast time and transmits it to the playback device. Here, it is assumed that the broadcast time is 19:00 of 2005.6.10.

  FIG. 2 is an explanatory diagram showing reproduction data reproduced by the reproduction apparatus. The playback device acquires the current time and compares the current time with the broadcast time. Based on the comparison result, the main image is reproduced or an image obtained by superimposing the first sub-image on the main image (hereinafter referred to as a first superimposed image) is reproduced. Specifically, as shown in FIG. 2, the playback device first acquires 19:10 at the current time 2005.6.10 and compares it with 19:00 at the broadcast time 2005.6.10. If the playback device determines that the difference between the current time and the broadcast time is small, the playback device plays back the main image. On the other hand, for example, if the current time is 15:00 of 2005.6.12 and it is determined that the difference between the current time and the broadcast time is large, the first sub-image is superimposed on the main image and reproduced.

  Note that the transmission device may include the effective time during which the main image can be reproduced together with the broadcast time of the main image for which real-time viewing is required, and transmit the transmission data to the reproduction device. Alternatively, the playback device may have information about the valid period during which the main image can be played back in advance. Therefore, the playback device determines whether to play back the main image or the first superimposed image based on whether the current time is within this valid period.

  As described above, based on the comparison result between the current time and the broadcast time, the portion of the main image on which the first superimposed data is superimposed is covered with the first sub-image. Thereby, the user can enjoy viewing without being bothered by obsolete information whose time has elapsed from the broadcast time. Further, since the playback device automatically switches between the first superimposed data and the second superimposed data, the user can perform video or music corresponding to the current time without performing an operation for viewing according to the current time. Can be watched.

<First embodiment>
The closed caption concealment broadcasting system according to the first embodiment includes a transmission device shown in FIG. 3 and a reproduction device shown in FIG.

(1) Functional Configuration of Transmitting Device FIG. 3 is a block diagram showing a functional configuration of the transmitting device of the present invention. The main image generation unit 21 generates a main image in which the first superimposed data is superimposed on the original image based on the coordinate information. The coordinate information is information indicating a position in the original image on which the first superimposition data is superimposed, and is a coordinate of a partial image described later. The extraction unit 27 extracts an image of a portion on which the first superimposition data is superimposed based on the coordinate information (hereinafter referred to as a partial image) from the original image. The first sub-image generation unit 23 generates a first sub-image by superimposing the second superimposed data on the partial image extracted by the extraction unit 27. Similarly, the second sub-image generating unit 25 generates a second sub-image by superimposing the third superimposed data on the partial image. Here, the first to third superimposition data are insertion data such as captions as described above, and are superposed on the original image to notify the user of various information.

  The main image encoding unit 29 encodes the broadcast time of the main image and the main image in association with each other, and outputs them to the multiplexing unit 37. The main image encoding unit 29 compresses and encodes an image with high efficiency represented by, for example, the MPEG2 video standard (ISO / IEC13818-2). The partial image encoding unit 31 encodes the coordinate information and the partial image in association with each other, and outputs the encoded information to the multiplexing unit 37. Similarly, the first sub-image encoding unit 33 encodes the coordinate information, the first time limit information, and the first sub-image in association with each other, and outputs them to the multiplexing unit 37. The second sub-image encoding unit 35 encodes the coordinate information, the second time limit information, and the second sub-image in association with each other, and outputs the encoded information to the multiplexing unit 37.

  Here, the first time limit information is a period during which the first sub-image can be reproduced in the reproduction apparatus that reproduces the transmission data. Similarly, the second time limit information is a period during which the second sub-image can be reproduced in the reproduction apparatus. For example, it is assumed that the second superimposed data in the first sub-image is data related to a prize with a time limit, and there is a period during which the prize can be applied. The period during which the prize can be applied corresponds to the first deadline information. When the time at which the user wants to view the main image (hereinafter referred to as the current time) is within the valid period indicated in the first time limit information in the playback device, the first sub-image can be played back. On the other hand, when the current time has passed the valid period indicated in the first time limit information, the playback device does not play back the first sub-image. The validity period and the contents of the first to third superimposed data are set on the transmission device side during broadcasting. For example, the first superimposed data is set to “live broadcast”, the second superimposed data is set to “today's event”, the third superimposed data is set to “this week's event”, etc., and the first deadline information is 1 minute or more 1 Less than a day, the second deadline information is set to be 1 day or more and less than 7 days.

The multiplexing unit 37 receives the broadcast time of the main image and the encoded data of the main image from the main image encoding unit 29, the coordinate information and the encoded data of the partial image from the partial image encoding unit 31, and the first sub-image code. The coordinate information, the first time limit information, and the encoded data of the first sub image are received from the encoding unit 33, and the coordinate information, the second time limit information, and the encoded data of the second sub image are received from the second sub image encoding unit 35. . Then, the multiplexing unit 37 multiplexes the received encoded data in association with each other to generate transmission data. That is, the multiplexing unit 37 associates the partial image, the first sub-image, and the second sub-image with the main image, associates the main image with the broadcast time, and associates the first sub-image with the first time limit information and coordinates. The information is associated with each other, and the second sub-image, the second time limit information, and the coordinate information are associated with each other and multiplexed.
3, the broadcast time and coordinate information of the main image are directly input to the multiplexing unit 37, and the multiplexing unit 37 outputs the output from each encoding unit, the broadcast time and coordinate information of the main image. May be multiplexed to generate transmission data.

  The association of these pieces of information is performed, for example, by adding a time stamp or a packet ID to the packets constituting the main image, the partial image, the first sub image, and the second sub image. Specifically, the time stamp of the main image describes the broadcast time of the main image, and the time stamps of the partial image, the first sub-image, and the second sub-image describe the broadcast time of the corresponding main image. Thereby, a main image, a partial image, a 1st subimage, and a 2nd subimage can be matched by broadcast time. Note that the association between the main image and other partial images and sub-images using the time stamp is not limited to the broadcast time, and may be performed based on other time information. Also, for example, each packet of the main image is assigned a packet ID, and the packet of the corresponding main image is attached to the packet of the partial image, the first sub-image and the second sub-image, so that the main image is identified by the packet ID. The partial image, the first sub image, and the second sub image can be associated with each other. In this way, by associating the main image and the related first sub-image, the playback device can accurately superimpose the main image and the first sub-image corresponding to each other.

  As the time stamp or packet ID, a PTS (Presentation time stamp) or a PID (Packet Identifier) defined in the ISO / IEC13818-1 standard can be used, respectively. The transmission data generated by the multiplexing unit 37 is a transport stream specified by, for example, the MPEG2 system (ISO / IEC13818-1), and each encoded bit string of the transmission data has a PES packet configuration specified by the MPEG2 system. In this case, a PTS is assigned to each picture, and the display time of the main image and each image can be synchronized. As described above, the main image and the first sub-image can be associated with each other using PTS or PID according to the existing ISO / IEC13818-1 standard.

(2) Outline of Processing of Transmission Device FIG. 4 is an explanatory diagram showing an outline of processing of the transmission device. The main image generation unit 21 generates a main image by superimposing the first superimposed data “ABCD” on the original image based on the coordinate information. Based on the coordinate information, the extraction unit 27 extracts an original image of a portion on which the first superimposed data “ABCD” is superimposed. Further, the first sub-image generating unit 23 generates the first sub-image by superimposing the second superimposed data “A” on the partial image. Similarly, the second sub-image generating unit 25 generates the second sub-image by superimposing the third superimposed data “Iroha” on the partial image.

  Next, the main image encoding unit 29 encodes the main image on which the first superimposed data “ABCD” is superimposed in association with the broadcast time. The first sub-image encoding unit 33 encodes the first sub-image on which the second superimposition data “Aoi” is superimposed in association with the first time limit information “● month ○ day” and the coordinate information. Similarly, the second sub-image encoding unit 35 encodes the second sub-image on which the third superimposition data “Iroha” is superimposed in association with the second deadline information “▲ month Δ day” and the coordinate information. The multiplexing unit 37 multiplexes these encoded data and outputs it as transmission data.

(3) Functional Configuration of Playback Device FIG. 5 is a block diagram showing a functional configuration of the playback device of the present invention. The storage unit 41 is, for example, a hard disk or the like, and receives transmission data from the transmission device, and stores the transmission data until temporary storage or reproduction is started. When receiving the transmission data from the storage unit 41, the separation unit 43 converts the transmission data into a main image and its broadcast time, a partial image and coordinate information, a first sub-image, coordinate information and first time limit information, and a second sub-data. The image, coordinate information, and second time limit information are separated and output to each decoding unit.

  The main image decoding unit 45 receives and decodes the main image and its broadcast time, and outputs the main image to the image composition unit 57. The partial image decoding unit 47 decodes the partial image and the coordinate information and outputs them to the selection unit 55. The first sub-image decoding unit 49 decodes the first sub-image, the coordinate information, and the first time limit information, outputs the first sub-image and the coordinate information to the selection unit 55, and outputs the first time limit information to the comparison unit 53. To do. The second sub-image decoding unit 51 decodes the second sub-image, coordinate information, and second time limit information, outputs the second sub-image and coordinate information to the selection unit 55, and outputs the second time limit information to the comparison unit 53. To do.

  The comparison unit 53 acquires the broadcast time from the main image decoding unit 45, acquires the first time limit information from the first sub-image decoding unit 49, and acquires the second time-limit information from the second sub-image decoding unit 51. In addition, the time (hereinafter referred to as the current time) when the user of the playback device views is acquired. Then, the current time is compared with the broadcast time, the first deadline information, and the second deadline information, and the comparison result is output to the selection unit 55.

  The selection unit 55 receives the comparison result from the comparison unit 53 and outputs either the partial image, the first sub-image, or the second sub-image and the coordinate information to the image composition unit 57 based on the comparison result. That is, if the current time is within the valid period indicated by the first deadline information, the first sub-image is output to the image composition unit 57, and if the current time is within the valid period indicated by the second deadline information, the second sub-image is output. The image is output to the image composition unit 57. The selection unit 55 outputs the partial image to the image composition unit 57 when the current time is not the same as the broadcast time of the main image and the current time is not within the first deadline information and the second deadline information. . Furthermore, when the current time and the broadcast time of the main image are the same, the selection unit 55 does not output any of the first sub image, the second sub image, or the partial image to the image composition unit 57.

  The image composition unit 57 receives the partial image, the first sub-image or the second sub-image, and the coordinate information from the selection unit 55, and receives the main image from the main image decoding unit 45. The image composition unit 57 reproduces the main image when the current time is the same as the broadcast time. On the other hand, if the current time is within the effective period indicated by the first time limit information, the image composition unit 57 synthesizes and outputs an image in which the first sub-image is superimposed on the main image (hereinafter referred to as the first superimposed image). If the current time is within the effective period indicated by the second time limit information, an image in which the second sub-image is superimposed on the main image (hereinafter referred to as a second superimposed image) is synthesized and output. Furthermore, if the current time is not the broadcast time and does not fall within the validity period indicated by the first and second deadline information, an image in which the partial image is superimposed on the main image (hereinafter referred to as a partial superimposed image) is synthesized. And output. That is, when the current time and the broadcast time of the main image are different, the portion of the main image on which the first superimposed data is superimposed is output as the first sub-image, the second sub-image, or an image covered with the partial image. .

  For example, as described above, the first superimposed data is “live broadcast”, the second superimposed data is “today's event”, the third superimposed data is “this week's event”, and the first deadline information is 1 minute or more 1 Less than a day, the second deadline information is 1 day or more and less than 7 days. In this case, when the current time is the broadcast time, the playback device plays back the main image on which the characters “live broadcast” are superimposed. In addition, when the current time is between 1 minute and less than 1 day from the broadcast time, the first superimposed image on which characters of “today's event” are superimposed is reproduced, and the current time is from 1 day to less than 7 days from the broadcast time. If it is between, the second superimposed image on which the characters of “this week's event” are superimposed is reproduced, and if the current time does not apply to either, the partially superimposed image is reproduced.

  In addition, when the transmission time is generated by multiplexing the broadcast time and coordinate information of the main image in the multiplexing unit 37 as shown by the broken line part in FIG. 3, the separating part 43 is shown in the broken line part in FIG. Coordinate information and broadcast time are input to each function unit. That is, the separation unit 43 converts the transmission data from the transmission device into the main image, the broadcast time of the main image, the partial image, the first sub-image and the first deadline information, the second sub-image and the second deadline information. And coordinate information are output separately. Therefore, the comparison unit 53 acquires the broadcast time of the main image from the separation unit 43, and the image composition unit 57 acquires coordinate information from the separation unit 43.

(4) Effects As described above, the transmission device generates transmission data including the first sub image and the second sub image together with the main image. Then, the playback device displays an image corresponding to the current time based on a comparison result between the current time, the broadcast time of the main image, the first deadline information of the first sub-image, and the second deadline information of the second sub-image. Reproduce. Thereby, the user can enjoy viewing without being bothered by obsolete information whose time has elapsed from the broadcast time. Further, since the playback device automatically switches between the first superimposed data and the second superimposed data, the user can view the video and music corresponding to the current time without performing an operation for viewing. .

(5) Specific Example In the above, the main image, the partial image, the first sub-image, and the second sub-image are encoded by different encoders and then multiplexed by the multiplexing unit 37. However, the transmission data may be generated by describing the partial image, the first sub-image, and the second sub-image in an area that accompanies the main image and in which arbitrary data can be described. That is, the corresponding partial image, the first sub image, and the second sub image are embedded in an arbitrary area included in the bit string of the main image. Hereinafter, a user data area of the ISO / IEC13818-2 standard will be described as an example of an arbitrary area accompanying the main image. In the following, the partial image, the first sub-image, and the second sub-image corresponding to the main image are converted into the user data format defined by the ISO / IEC13818-2 standard, and the main image and each image converted into the user data format are converted into the user image format. The configuration embedded in the user data area will be specifically described.

  As described above, the multiplexing unit 37 of the transmission device arranges the encoded data of the main image in a part other than the user data area of the elementary stream, and broadcasts the main image, the coordinate information, the partial image, and the coordinate information. The first time limit information and the first sub-image, the coordinate information, the second time limit information, and the second sub-image are inserted into the user data area of the elementary stream to generate an elementary stream. Then, the multiplexing unit 37 divides the generated elementary stream into PES (Packetized Elementary Stream) variable-length packets and adds a header to generate a transport stream.

(5-1) User data area of each layer In particular, in the ISO / IEC13818-2 standard, MPEG2 has a hierarchical structure including a sequence layer, a GOP layer (Group of Pictures), and a picture layer (Picture Layer). Each layer has a user data area in which arbitrary data can be described. FIG. 6 is an explanatory diagram showing the syntax of SO / IEC13818-2 (MPEG Video). The sequence layer is defined by a function including sequence_header, sequence_extension, extension_and_user_data. sequence_header and sequence_extension define the header and extension data of the sequence layer, and extension_and_user_data defines the extension data and user data of the sequence layer. The GOP layer is defined by a function including GOP_header and user_data. GOP_header defines a GOP layer header, and user_data defines GOP layer user data. The picture layer is defined by a function including picture_header, picture_extension, extension_and_user_data, and picture_data. picture_header and picture_extension define the header and extension data of the picture layer, and extension_and_user_data defines the extension data and user data of the picture layer. Picture_data is a function that defines a slice layer and a macroblock layer. That is, arbitrary data can be described by extension_and_user_data and user_data of each layer. Therefore, the multiplexing unit 37 of the transmission device includes, in extension_and_user_data and user_data, the broadcast time of the main image other than the main image, the coordinate information and the partial image, the coordinate information, the first deadline information, the first sub-image, and the coordinate information. , Second deadline information and second sub-image are described. Based on the description of extension_and_user_data and user_data, the playback device superimposes and plays back the main image and the partial image, the first sub-image, or the second sub-image.

  As shown in FIG. 6, the extension_and_user_data function of the sequence layer is not necessarily defined, and can proceed to GOP_header next to sequence_extension. Also, GOP_header and user_data of the GOP layer are not necessarily defined, and can proceed to picture_header after sequence_extension. Similarly, the extension_and_user_data function of the picture layer does not necessarily need to be defined, and can proceed to picture_data next to picture_extension. When all the functions shown in FIG. 6 are defined, the process proceeds in the order of sequence_header → sequence_extension → extension_and_user_data → GOP_header → user_data → picture_header → picture_extension → extension_and_user_data → picture_data.

(5-2) Syntax Next, extension_and_user_data and user_data shown in FIG. 6 will be described with reference to FIGS. FIG. 7 is an explanatory diagram showing extension_and_user_data. For example, extension_and_user_data (0) is a sequence layer function, extension_and_user_data (1) is a GOP layer function, and extension_and_user_data (2) is a picture layer function. In FIG. 7, extension_and_user_data (i) is not defined by extension_and_user_data when i is other than 1, and only user_data is described. That is, only user_data is described in the GOP layer.

  FIG. 8 is an explanatory diagram showing user_data. user_data_start_code indicates the start of a user data area in which arbitrary data can be described. This indicates that user_data is executed until “0000 0000 0000 0000 0000 0001” appears in the bit stream of the elementary stream.

  FIG. 9 is a description example (1) of user_data. The start point at which the first sub-image, the second sub-image, or the partial image is started is described by start_corner_horizontal and start_corner_vertical, and the size of the image is described by size_of_horizontal and size_of_vertical. In addition, as described in the line “for (i = 0; i <size_of_horizontal * size_of_vertical; i ++) [”, the loop is rotated by the dot multiplied by size_of_horizontal and size_of_vertical, and the GBR data of each dot is described. To represent each image. With the user_data shown in FIG. 9, the image compositing unit 57 of the playback device allows the first sub-image, the second sub-image, or the part defined by the GBR data to be described in the coordinates described by start_corner_horizontal, start_corner_vertical, size_of_horizontal, and size_of_vertical of the main image. Images can be superimposed.

  FIG. 10 is a description example (2) of user_data. In user_data in FIG. 10, PTS (Presentation time stamp) defined in ISO / IEC13818-1 is further described. For example, “year” is described by PTS [32..30], “day” is described by PTS [29..15], and “hour and minute” are described by PTS [14..0]. The marker bit indicates a PTS bit delimiter and is always described as “1”, for example. Here, the broadcast time of the main image is described in PTS as shown in FIG. 10, and the same PTS as the corresponding main image is described for each of the first sub-image, the second sub-image, and the partial image. As a result, each picture of the main image can be associated with each picture of the first sub-image, the second sub-image, or the partial image. Therefore, the image composition unit 57 of the playback device can superimpose the first sub image, the second sub image, and the partial image corresponding to the main image for each picture.

  FIG. 11 is a description example (3) of user_data. 9 and FIG. 10 describe only one image to be superimposed on the main image, FIG. 11 describes a plurality of images to be superimposed on the main image and their respective time limit information. A first sub-image to be superimposed on the main image is described by a description starting from “time_frame — 1_start”, and a second sub-image to be superimposed on the main image is described by a description starting from “time_frame — 2_start”. Further, for example, first time limit information indicating a period during which the first sub-image can be reproduced is described by “time_frame — 1_start [23..0]”. For example, “time_frame_1_start [23..11]” describes “year” in 13 bits, “time_frame_1_start [9..6]” describes “month” in 4 bits, and “time_frame_1_start [4..0]” Then, “day” is described with 5 bits. Note that “time_frame — 1_start [10]” and “time_frame — 1_start [5]” indicate a delimiter of time limit information described in 24 bits, and are always described as “1”, for example. Similarly, “time_frame — 2_start [23..0]” describes second time limit information indicating a period during which the second sub-image can be reproduced.

  When the current time is within the first time limit information described by “time_frame — 1_start” and “time_frame — 1_end”, the selection unit 55 of the playback device outputs the first sub-image to the image composition unit 57. On the other hand, when the current time is within the second time limit information described by “time_frame_2_start” and “time_frame_2_end”, the selection unit 55 outputs the second sub-image to the image composition unit 57. If the broadcast time of the main image described in PTS is not the same as the current time and the current time is not in the first deadline information and the second deadline information, the selection unit 55 has the deadline information. The partial image that does not exist is output to the image composition unit 57. Furthermore, when the broadcast time of the main image described in PTS is the same as the current time, the selection unit 55 does not output any of the first sub image, the second sub image, or the partial image to the image composition unit 57. The image composition unit 57 receives the main image from the main image decoding unit 45 regardless of the current time. Therefore, the image composition unit 57 reproduces only the main image according to the current time, or superimposes and reproduces the main image and the first sub image, the second sub image, or the partial image output from the selection unit 55.

  As described above, the transmission device stores the partial image corresponding to the main image, the first sub-image, the second sub-image, the broadcast time of the main image, etc. in the user data area of the ISO / IEC13818-2 standard. Embed. Therefore, an elementary stream to which corresponding information is added can be generated for each picture of the main image, that is, for each frame. In addition, when a main image is selected in the playback device, a first sub-image corresponding to the main image can be acquired at the same time. Therefore, when the main image is edited, it is possible to prevent the first sub image from being lost by editing the main image and the first sub image separately.

  Note that the above description example is an example and is not limited to the above description.

<Second Embodiment>
In the first embodiment, the transmission data is generated by multiplexing the encoded data of the sub-image or partial image. However, in the second embodiment, the transmission apparatus accumulates the sub-images in a data accumulation unit 65 such as a database on the WWW (World wide web), and generates transmission data by multiplexing the accumulation destination addresses. . Further, the playback device accesses the storage destination address, acquires the sub-image, and outputs the image.

  The subtitle concealment broadcasting system according to the second embodiment will be described below. The closed caption concealment broadcasting system according to the second embodiment includes a transmission device shown in FIG. 12 and a reproduction device shown in FIG.

(1) Functional configuration of transmitting apparatus FIG. 12 is a block diagram showing a functional configuration of the transmitting apparatus of the present invention. The second sub-image encoding unit 35 of the transmission device in FIG. 12 generates encoded data obtained by encoding the second sub-image in which the third superimposed data is superimposed on the partial image together with the second time limit information and the coordinate information. The data is transmitted to the data storage unit 65. Then, the address acquisition unit 61 acquires the address where the second sub-image is stored from the data storage unit 65 and outputs the acquired address to the multiplexing unit 37. The multiplexing unit 37 receives the broadcast time of the main image and the encoded data of the main image from the main image encoding unit 29, the coordinate information and the encoded data of the partial image from the partial image encoding unit 31, and the first sub-image code. The coordinate information, the first time limit information, and the encoded data of the first sub-image are received from the conversion unit 33, and the address of the second sub-image is received from the address acquisition unit 61. Then, the multiplexing unit 37 multiplexes the received data in association with each other to generate transmission data. Other configurations are the same as those of the first embodiment, and will be briefly described below.

  The extraction unit 27 extracts a partial image from the original image based on the coordinate information. The first sub-image generation unit 23 generates a first sub-image by superimposing the second superimposed data on the partial image extracted by the extraction unit 27. Similarly, the second sub-image generating unit 25 generates a second sub-image by superimposing the third superimposed data on the partial image. The main image encoding unit 29 encodes the broadcast time of the main image and the main image in association with each other, and outputs them to the multiplexing unit 37. The partial image encoding unit 31 encodes the coordinate information and the partial image in association with each other. The first sub image encoding unit 33 encodes the coordinate information, the first time limit information, and the first sub image in association with each other. The second sub image encoding unit 35 encodes the coordinate information, the second time limit information, and the second sub image in association with each other.

(2) Functional Configuration of Playback Device FIG. 13 is a block diagram showing a functional configuration of the playback device of the present invention. In the reproducing apparatus of FIG. 13, when receiving the transmission data stored in the storage unit 41, the separation unit 43 converts the transmission data into the main image and its broadcast time, the partial image and coordinate information, the first sub-image, and the coordinate. The information and the first time limit information are separated from the address of the second sub-image. The image acquisition unit 71 receives the address of the second sub-image from the separation unit 43 and accesses the data storage unit 65 based on the address. The image acquisition unit 71 acquires the second sub-image, the second time limit information, and the coordinate information, and first outputs the second time limit information to the comparison unit 53.

  The comparison unit 53 acquires the broadcast time from the main image decoding unit 45, acquires the first time limit information from the first sub-image decoding unit 49, acquires the second time limit information from the image acquisition unit 71, and The time when the user of the playback device views (hereinafter referred to as the current time) is acquired. Then, the comparison unit 53 compares the current time with the broadcast time, the first deadline information, and the second deadline information, and outputs the comparison result to the selection unit 55 and the image acquisition unit 71. When receiving the comparison result that the current time is within the second time limit information, the image acquisition unit 71 outputs the acquired second sub-image and coordinate information to the selection unit 55. Other configurations are the same as those of the first embodiment, and will be briefly described below.

  The main image decoding unit 45 receives and decodes the main image and its broadcast time, the partial image decoding unit 47 decodes the partial image and the coordinate information, and the first sub image decoding unit 49 includes the first sub image, The coordinate information and the first time limit information are decoded and input to the comparison unit 53, the selection unit 55, and the image composition unit 57. Based on the comparison result from the comparison unit 53, the selection unit 55 outputs either the partial image, the first sub-image, or the second sub-image and the coordinate information to the image synthesis unit 57, or does not output any of them. The image composition unit 57 outputs the main image, the first superimposed image, the second superimposed image, or the partially superimposed image according to the current time.

  The transmitting apparatus and reproducing apparatus of the second embodiment as described above have the same operational effects as the first embodiment.

  Note that all the sub-images, partial images, etc. may be stored in the data storage unit 65 on the WWW, and only their addresses may be multiplexed.

<Other embodiment examples>
(1)
In the above-described embodiment, the caption concealment broadcasting system is described by taking as an example the superimposition of the main image and the sub-image. However, not only the superimposition of images, but also the sound accompanying the main image and the sound accompanying the sub-image may be superimposed. For example, it is assumed that the first superimposition data superimposed on the original image is a subtitle of earthquake information, and the sound superimposed on the sound of the original image is an alarm sound for notifying earthquake information. Therefore, a warning sound is further superimposed on the main image on which the earthquake information is superimposed. The second superimposed data superimposed on the partial image is a commercial, and the sound added to the partial image is a sound notifying the start of the commercial. Therefore, the commercial start sound is superimposed on the first sub-image in which the commercial is superimposed on the partial image. When receiving the transmission data including the main image and the first sub-image, the playback device first compares the current time with the broadcast time of the main image or the first time limit information of the first sub-image, as described above. . Based on the comparison result, the main image is output or the first sub-image is superimposed on the main image and output. At this time, the playback device outputs the main image if the current time is the same as the broadcast time, and outputs an alarm sound together with the earthquake information. On the other hand, when the current time is within the first deadline information, the first sub-image is superimposed on the main image, and the commercial start sound is superimposed on the alarm sound of the main image and output. Therefore, the user of the playback device can enjoy broadcasting without being bothered by sound related to obsolete information.

  As for the audio of the sub-image, arbitrary data can be described by using, for example, an ancillary data area of the ISO / IEC13818-3 standard. Since the voice can be described with the same configuration as described above, detailed description is omitted.

(2)
A computer program that causes a computer to execute the above-described wireless communication method in the wireless communication terminal and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, examples of the computer-readable recording medium include a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blue-ray Disc), and semiconductor memory. .

  The computer program is not limited to the one recorded on the recording medium, and may be transmitted via an electric communication line, a wireless or wired communication line, a network 63 represented by the Internet, and the like.

  The broadcasting system and the receiving system according to the present invention have an automatic subtitle concealment or automatic update function, and are useful as an automatic subtitle display control system in a storage-type broadcast receiving apparatus such as a digital recorder, and are applied to the use of electronic advertisements. it can. In addition, according to the present invention, even when it is desired to display the campaign information only for the period intended by the broadcaster, or even when the contact phone number displayed in the subtitle is changed unexpectedly after broadcasting, the viewer is made aware of anything. The display contents can be automatically followed to the latest contents without any problems.

(A) Explanatory drawing which shows the main image of this invention. (B) Explanatory drawing which shows the 1st subimage of this invention. Explanatory drawing which shows the reproduction data reproduced | regenerated by the reproducing | regenerating apparatus. The block diagram which shows the function structure of the transmitter of this invention. Explanatory drawing which shows the outline | summary of the process of a transmitter. The block diagram which shows the function structure of the reproducing | regenerating apparatus of this invention. Explanatory drawing which shows the syntax of SO / IEC13818-2 (MPEG Video). Explanatory drawing which shows extension_and_user_data. Explanatory drawing which shows user_data. Description example (1) of user_data. Description example of user_data (2) Description example of user_data (3) The block diagram which shows the function structure of the transmitter of this invention. The block diagram which shows the function structure of the reproducing | regenerating apparatus of this invention. Explanatory drawing which shows the conventional character concealment method described in the said patent document 1

Explanation of symbols

21: main image generation unit 23: first sub image generation unit 25: second sub image generation unit 27: extraction unit 29: main image encoding unit 31: partial image encoding unit 33: first sub image encoding unit 35 : Second sub image encoding unit 37: multiplexing unit 41: storage unit 43: separation unit 45: main image decoding unit 47: partial image decoding unit 49: first image decoding unit 51: second image decoding Unit 53: Comparison unit 55: Selection unit 57: Image composition unit 61: Address acquisition unit 65: Data storage unit 71: Image acquisition unit

Claims (24)

A first image in which second superimposed data is superimposed on a main image in which first superimposed data is superimposed on an original image and an image of a portion extracted from the original image and superimposed on the first superimposed data (hereinafter referred to as a partial image). A reproduction device that reproduces transmission data generated by associating a sub-image with the main image and a broadcasting time of the main image,
A comparison means for comparing the current time with the broadcast time of the main image;
Output means for outputting the main image or an image in which the first sub-image is superimposed on the main image (hereinafter referred to as a first superimposed image) based on the comparison result in the comparison means;
A playback apparatus comprising: In the transmission data, the first sub-image and the first sub-image that can reproduce the first sub-image and the first sub-image are further associated with the main image and the second sub-image in which the third superimposed data is superimposed on the partial image. Time limit information is associated with each other, and the second sub-image and the second time-limit information capable of reproducing the second sub-image are associated with each other and generated.
The comparison means compares the current time with the broadcast time, the first deadline information, and the second deadline information,
The output means, based on the comparison result of the comparison means, the main image, the first superimposed image, or an image obtained by superimposing the second sub-image on the main image (hereinafter referred to as a second superimposed image). The playback device according to claim 1, wherein the playback device outputs. The transmission data is generated by further associating the main image and the partial image,
The output means outputs the main image, the first superimposed image, or an image obtained by superimposing the partial image on the main image (hereinafter referred to as a partial superimposed image) based on the comparison result of the comparing means. The playback apparatus according to claim 1, wherein the playback apparatus is characterized.   The playback apparatus according to claim 1, wherein the transmission data is generated by associating the first sub-image with coordinate information of the partial image in the original image.   In the transmission data, a time stamp or a packet ID is added to a packet constituting each of the main image and the first sub image so that the main image and the first sub image are associated with each other. The playback apparatus according to claim 1.   6. The reproducing apparatus according to claim 5, wherein the time stamp or the packet ID is a PTS (Presentation time stamp) or a PID (Packet Identifier) defined in the ISO / IEC13818-1 standard, respectively.   2. The playback apparatus according to claim 1, wherein the transmission data includes a first sub-image corresponding to the main image described in an area where arbitrary data can be described accompanying the main image. . The area that can describe arbitrary data accompanying the main image is a user data area of the ISO / IEC13818-2 standard,
The transmission data is generated by describing a first sub-image converted into a user data format defined in the ISO / IEC13818-2 standard in a user data area associated with the main image, The playback apparatus according to claim 7.   9. The playback apparatus according to claim 8, wherein the first sub-image converted into the user data format is arranged in a user data area in units of pictures of the corresponding main image. In the transmission data, the main image and the address on the WWW where the second sub-image in which the third superimposition data is superimposed on the partial image are associated with each other, and the first sub-image and the first sub-image Generated in association with the first time limit information capable of reproducing the sub-image,
The comparison means compares the current time with the broadcast time, the first deadline information, and the second deadline information,
Further comprising image acquisition means for accessing the address of the second sub-image and acquiring the first sub-image based on the comparison result of the comparison means;
2. The playback apparatus according to claim 1, wherein the output unit outputs the main image or a first superimposed image obtained by superimposing the first sub-image acquired by the image acquisition unit on the main image. A first image in which second superimposed data is superimposed on a main image in which first superimposed data is superimposed on an original image and an image of a portion extracted from the original image and superimposed on the first superimposed data (hereinafter referred to as a partial image). A reproduction device that reproduces transmission data generated by associating an address on the WWW in which a sub-image is stored and associating the main image with a broadcast time of the main image,
A comparison means for comparing the current time with the broadcast time of the main image;
Image acquisition means for accessing the address of the first sub-image and acquiring the first sub-image based on the comparison result of the comparison means;
An output means for outputting a first superimposed image obtained by superimposing the main image or the first sub-image obtained by the image obtaining means on the main image based on the comparison result obtained by the comparing means;
A playback apparatus comprising: A transmission device for transmitting transmission data to be reproduced by a reproduction device to the reproduction device ,
Main image generation means for generating a main image by superimposing the first superimposed data on the original image;
Extraction means for extracting an image of a portion to be superimposed on the first superimposition data (hereinafter referred to as a partial image) from the original image;
Sub-image generating means for generating a first sub-image by superimposing second superimposition data on the partial image;
With associating the first sub-image and the main image, the transmission data generation means for generating the transmission data in association with the broadcast time of the main image and the main image,
Only including,
The playback device
A comparison means for comparing the current time with the broadcast time of the main image;
Output means for outputting the main image or an image in which the first sub-image is superimposed on the main image (hereinafter referred to as a first superimposed image) based on the comparison result in the comparison means,
A transmitting apparatus characterized by the above. The sub-image generating means further generates a second sub-image by superimposing third superimposition data on the partial image;
The transmission data generating means further associates the main image with the second sub-image, and associates the first sub-image with first time limit information indicating a period during which the first sub-image can be reproduced, 13. The transmission apparatus according to claim 12, wherein transmission data is generated by associating the second sub-image with second time limit information indicating a period during which the second sub-image can be reproduced.   The transmission apparatus according to claim 12, wherein the transmission data generation unit generates transmission data by further associating the main image and the partial image.   The transmission apparatus according to claim 12, wherein the transmission data generation unit further generates transmission data by associating the first sub-image with coordinate information of the partial image in the original image.   The transmission data generation unit adds a time stamp or a packet ID to packets constituting the main image and the first sub image so that the main image and the first sub image are associated with each other. The transmission device according to claim 12.   The transmission apparatus according to claim 16, wherein the time stamp or the packet ID is a PTS (Presentation time stamp) or a PID (Packet Identifier) defined in the ISO / IEC13818-1 standard, respectively.   The transmission data generation means generates transmission data by describing a first sub-image corresponding to the main image in an area where arbitrary data can be described accompanying the main image. Item 13. The transmission device according to Item 12. The area that can describe arbitrary data accompanying the main image is a user data area of the ISO / IEC13818-2 standard,
The transmission data generation means converts the first sub-image into a user data format defined by the ISO / IEC13818-2 standard, and converts the first sub-image into a user data area associated with the main image into the user data format. The transmission apparatus according to claim 18, wherein transmission data is generated by describing   The transmission apparatus according to claim 19, wherein the first sub-image converted into the user data format is arranged in a user data area in a picture unit of a corresponding main image. An address acquisition means for acquiring an address on a World Wide Web (WWW) in which a second sub-image in which the third superimposed data is superimposed on the partial image is stored;
The transmission data generation means further associates the main image with the address of the second sub-image, and transmits the first sub-image and the first time limit information capable of reproducing the first sub-image in association with each other. The transmission device according to claim 12, wherein the transmission device generates data. A transmission device for transmitting transmission data to be reproduced by a reproduction device to the reproduction device,
Main image generation means for generating a main image by superimposing the first superimposed data on the original image;
Address acquisition means for acquiring an address on the WWW storing a first sub-image in which a partial image corresponding to a superimposition area with the first superimposition data extracted from the original image and second superimposition data are stored. When,
With associating the address of said first sub image and the main image, the transmission data generation means for generating the transmission data in association with the broadcast time of the main image and the main image,
Only including,
The playback device
A comparison means for comparing the current time with the broadcast time of the main image;
Image acquisition means for accessing the address of the first sub-image and acquiring the first sub-image based on the comparison result of the comparison means;
An output means for outputting a first superimposed image obtained by superimposing the main image or the first sub-image obtained by the image obtaining means on the main image based on the comparison result obtained by the comparing means;
including,
A transmission apparatus characterized by the above. A first image in which second superimposed data is superimposed on a main image in which first superimposed data is superimposed on an original image and an image of a portion extracted from the original image and superimposed on the first superimposed data (hereinafter referred to as a partial image). A reproduction method for reproducing transmission data generated by associating a sub-image with the main image and a broadcast time of the main image,
A comparison step of comparing the current time with the broadcast time of the main image;
An output step of outputting a first superimposed image obtained by superimposing the first sub-image on the main image or the main image based on the comparison result in the comparison step;
A reproduction method comprising: A transmission method for transmitting transmission data to be reproduced in a predetermined reproduction method,
A main image generation step of generating a main image by superimposing the first superimposed data on the original image;
An extraction step of extracting, from the original image, an image of a portion to be superimposed on the first superimposed data (hereinafter referred to as a partial image);
A sub-image generating step of generating a first sub-image by superimposing second superimposition data on the partial image;
With associating the first sub-image and the main image, a transmission data generation step of generating the transmission data in association with the broadcast time of the main image and the main image,
Only including,
The predetermined reproduction method is:
A comparison step of comparing the current time with the broadcast time of the main image;
An output step of outputting a first superimposed image obtained by superimposing the first sub-image on the main image or the main image based on the comparison result in the comparison step;
The including,
A transmission method characterized by the above.

Images