JP2009290794A - Transport stream generating apparatus, transport stream generating method, content distribution server and control method of content distribution server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that repacketizing and re-multiplexing are required for a content distributed by VOD and the like after a Demux (packet separation) processing and a decoding processing are performed on TS data and caption data are added to data in the case of urgently inserting urgent disaster information by a caption, for example. <P>SOLUTION: In the transport stream generating apparatus, a caption packet is previously multiplexed/inserted into TS data and TS data are generated as NULL data, for example, even if character information to be displayed does not exist. A content distribution server device includes a function for overwriting new caption data and recording them in the case of urgently inserting and displaying new caption data into TS data generated by the transport stream generating device. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention provides a new subtitle for a video portion without subtitles without applying processing load to TS (transport stream) packetized audio and video and TS data generated by multiplexing subtitle data. The present invention relates to a technique for inserting (or replacing subtitles).

  At present, many video distribution services such as VOD (Video On Demand) using the Internet are provided due to an increase in the speed of network lines and the spread of receiving terminals (such as a television receiver with a network connection function). Incidentally, data distributed in such a moving image distribution service is held in a distribution server as data in a TS (transport stream) format suitable for transmission on a network. TS format data is composed by dividing video data and audio data encoded in a predetermined encoding format into packets called TS packets and multiplexing the TS packets in the order of playback timing. ing.

In addition, for example, caption data for displaying movie captions and other character information is similarly TS-packetized and multiplexed in the TS data.
JP 2004-215126 A

  By the way, in the case of normal television broadcasting, emergency disaster information such as earthquake bulletins and tsunami warnings, and other emergency subtitles such as other news bulletins are suddenly superimposed and displayed in the video. However, content distributed by VOD or the like is stored in a state (TS format data) in which TS packetization and multiplexing have been performed in advance as described above. Therefore, when new subtitle data is suddenly inserted, as shown in FIG. 22, (2) Demux (packet separation) processing or decoding processing is performed on TS data, and subtitle data is added. After that, there was a problem that (3) repacketization and remultiplexing had to be performed.

  In order to solve the above problems, the present invention has a configuration in which subtitle packets are always multiplexed and inserted in TS data, and when there is no character information to be displayed in the subtitle packets, for example, NULL data or transparent data Provided is a transport stream generation device having a function of configuring a transparent subtitle packet to be a layer. If new subtitle data needs to be inserted quickly, the TS data management data (PMT, etc.) is used in the content distribution server, which will be described later, without processing such as Demux or decoding. The position of the multiplexed subtitle packet is specified as it is. Then, a process for overwriting and inserting a new caption packet into the identified caption packet is performed. In this way, TS data generated by the transport stream generating apparatus of the present invention does not require rewriting management information such as PMT even when caption data is newly inserted, and thus greatly reduces processing load and processing time. can do.

  Specifically, the transport stream generating apparatus of the present invention acquires a video / audio packetizing unit that acquires video and / or audio content and converts it into a video / audio packet; Subtitle packetizing section, transparent subtitle packetizing section for generating a subtitle packet that is superimposed on content and displayed transparently, and management information acquisition section for acquiring management information of video and audio packets and subtitle packets in the transport stream And a management packet generation unit that generates a management packet indicating the acquired management information, and a transport stream generation unit that converts the packet generated according to the management information into a transport stream.

  In addition, by inserting transparent subtitle packets so that more transparent subtitles are superimposed over the entire playback time of the content, TS data that can additionally display new subtitles at any timing of the content is generated. A transport stream generating apparatus is also provided. Specifically, based on the above configuration, the management information acquisition unit acquires management information for transport stream so that transparent subtitle packets are superimposed over the entire playback time of the content to be configured with the transport stream This is a transport stream generation device having all acquisition means.

  Also, a content distribution server device having a function of overwriting and recording new subtitle data when the new subtitle data is suddenly inserted and displayed with respect to the TS data generated by the transport stream generation device. Also provide. Specifically, the content distribution server device stores the transport stream generated by the transport stream generation device and the transport stream stored in the storage unit in response to a playback request from the playback terminal. An acquisition unit that acquires the character information, a character information acquisition unit that acquires character information to be displayed superimposed on the content that is configured by the transport stream acquired by the acquisition unit, and the acquired character information in response to a reproduction request A recording unit that records as a subtitle packet arranged at a subtitle packet position included in advance in the transport stream being acquired; and a transmission unit that transmits the transport stream on which the recording has been performed to a playback terminal.

  In addition, a content distribution server device is also provided that forcibly displays highly urgent caption data such as disaster evacuation information at a content distribution destination. Specifically, in addition to the above configuration, forced character information that is character information associated with a forced identifier that identifies character information that the character information acquisition unit should forcibly display on the playback terminal as character information to be displayed. And a compulsory character information obtaining means for obtaining the content, and the recording unit records the subtitle packet including the compulsory identifier at the time of recording.

  The present invention also provides a transport stream generation method related to the transport stream generation device and a control method related to the content distribution server device.

  With the transport stream generating apparatus of the present invention having the above-described configuration, a transport stream configured to display a transparent subtitle layer in video can be generated. When it is necessary to insert new subtitle data quickly, the content distribution server of the present invention uses the management data (PMT etc.) of the TS data to perform the new subtitle without performing processing such as Demux or decoding. You can overwrite the packet and insert subtitle data.

  In this way, according to the present invention, it is not necessary to rewrite the PMT, which is management information, even when new caption data is inserted, and therefore the processing load and processing time can be greatly reduced.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to these embodiments, and can be implemented in various modes without departing from the spirit of the present invention.

  In the first embodiment, claims 1 and 7 will be mainly described. In the second embodiment, claims 2 and 8 will be mainly described. In the third embodiment, claims 3, 5, 9, and 11 will be mainly described. In the fourth embodiment, claims 4, 6, 10, and 12 will be mainly described.

  Example 1

    <Overview>

  FIG. 1 is a conceptual diagram for explaining an example of TS data generated by the transport stream generation device of the present embodiment. As shown in FIG. 1A, a video packet v, an audio packet a, and a caption packet t2 are multiplexed in this TS data. However, the transport stream generating apparatus according to the present embodiment is characterized in that a part (or all) of subtitle packets are generated and multiplexed as transparent subtitle packets. Therefore, when the generated TS data is reproduced and displayed, character data is not displayed even if subtitle packets are multiplexed, and normal content can be enjoyed.

  For example, when a telop of emergency disaster information is displayed due to, for example, an earthquake, as shown in FIG. 1B, a caption packet related to the telop of the emergency disaster information is located at the position of the transparent caption packet in the TS data. Is overwritten. In this manner, display character information such as emergency disaster information can be newly inserted without complicated processing such as Demux and decoding of TS data, repacketization, and remultiplexing.

    <Functional configuration>

  FIG. 2 is a diagram illustrating an example of functional blocks in the transport stream generation device according to the present embodiment. As shown in this figure, the “transport stream generating device” (0200) of this embodiment includes a “video / audio packetizing unit” (0201), a “caption packetizing unit” (0202), and a “transparent subtitle packet”. A “management unit” (0203), a “management information acquisition unit” (0204), a “management packet generation unit” (0205), and a “transport stream generation unit” (0206).

  Note that the functional blocks of the present apparatus and the content distribution server described below can be realized as hardware, software, or both hardware and software. Specifically, if a computer is used, a CPU, a main memory, a bus, or a secondary storage device (a hard disk, a non-volatile memory, a storage medium such as a CD or a DVD, a read drive for the medium, etc.), information Input devices used for input, printing equipment, display devices, other hardware components such as external peripheral devices, interfaces for external peripheral devices, communication interfaces, driver programs for controlling these hardware, Other examples include application programs and user interface applications.

  Then, through the arithmetic processing of the CPU according to the program developed on the main memory, the data input from the input device or other interface etc. and stored in the memory or hard disk is processed and stored, or each of the above hardware and software An instruction for controlling the above is generated.

  The present invention can be realized not only as an apparatus but also as a method. A part of the invention can be configured as software. Furthermore, a software product used for causing a computer to execute such software and a recording medium in which the product is fixed to a recording medium are naturally included in the technical scope of the present invention (the same applies throughout the present specification). Is).

  The “video / audio packetization unit” (0201) has a function of acquiring video and / or audio content and converting it into a video / audio packet. For example, an arithmetic device such as a CPU, a main memory, and a video / audio packetization program Can be realized. Specifically, the packetization here is binary data for reproducing and outputting video and audio, through intermediate packets such as PES packets, to TS packets of a predetermined length (188 bytes long) with various packet headers added. This is a process of dividing (in the case of a TTS packet in which a time stamp is added to a TS packet, the data length is 192 bytes).

  The “caption packetization unit” (0202) has a function of acquiring content composed of subtitles and packetizing the caption, and can be realized by an arithmetic device such as a CPU, a main memory, a video / audio packetization program, or the like. . The packetization here also specifically refers to processing of dividing binary data for reproducing and outputting subtitles on video and adding them to various packet headers and dividing them into TS packets and TTS packets of a predetermined length. The subtitle packet includes a subtitle management packet including information indicating a subtitle language, a display position, and the like in addition to a TS packet indicating character data of the subtitle itself. Hereinafter, the details of these caption packets will be described with reference to FIGS. 3 and 4.

  FIG. 3 is a conceptual diagram illustrating an example of a caption packet indicating character data itself. As shown in this figure, a caption packet is provided with a field indicating “data_group_id” at the beginning of its header. If, for example, identification information “1” is described in the field, character data to be displayed is the packet. It can be determined that it is described in the data part. In the data section, for example, text data for displaying characters such as “11th early morning, there was an earthquake in Fukuoka”, numeric symbols, etc. (it is packetized with a predetermined data length. (Some text data) is included. Further, the data portion includes a field for describing a control command, in which a command indicating display timing and the like is described.

  FIG. 4 is a conceptual diagram illustrating an example of a caption packet indicating caption management information. As shown in this figure, in the caption packet, identification information “0”, for example, is described in “data_group_id” at the head of the header. In the data portion, the number of subtitle languages displayed in the “num_languages” field is described, and “language code” such as “JPN (Japanese)” or “ENG (English)” is displayed in the subsequent field. Only as many languages are described. In the subsequent fields, information indicating the size of the subtitle display area, the display position of the display area, and the like are described as an “extended control command”. Then, according to the management information, the text data described in the data portion of the caption packet in FIG. 3 is displayed superimposed on the video.

  In addition, a subtitle packet indicating subtitle management information is provided with a field describing “DMF (caption display method)”. In this field, for example, a flag “0” indicating automatic display at the time of reception, a flag “1” indicating whether to select and display subtitles according to the setting at the reception terminal at the time of reception, and the like are described. Is done. Then, by using the flag of the “DMF” field, it is possible to realize a function for forcibly displaying an emergency insertion subtitle as described later in the fourth embodiment on the reception / playback apparatus side.

  This subtitle packet may be generated only when there is a character (number, symbol) string to be displayed superimposed on the video. Therefore, when there is no character string at all, the caption packetizing process in the caption packetizing unit is not performed. However, the characteristic functions of the transport stream generation device and the content distribution server of the present invention are that a subtitle packet or / and a transparent subtitle packet (to be described later) are previously added to the transport stream for the purpose of suddenly inserting a subtitle packet later. It is in the point of multiplexing. Therefore, when no caption packet is generated, that is, when there is no caption content to be displayed superimposed on the video, only the transparent caption packet described below is generated.

  The “transparent subtitle packetization unit” (0203) has a function of generating a subtitle packet that is superimposed on content and displayed transparently, and is realized by, for example, an arithmetic device such as a CPU, a main memory, a transparent subtitle packetization program, or the like. be able to. Specifically, in this transparent caption packetizing unit, the data portion of the caption packet shown in FIG. 3 is set as NULL data such as “fff...” And secured as a caption display area by the management information shown in FIG. There is a method of generating a transparent caption packet by setting the transparency of the region to the maximum. With such a configuration, a subtitle layer is superimposed and displayed on the content, but there is no character data to be displayed there, and the layer is also transparent. As shown in a), the content without subtitles is displayed as usual.

  Alternatively, in the management information “DMF” shown in FIG. 4, that is, a field that describes the display method of captions, a flag of “not necessarily displayed at reception” is set, or the transparent caption packet is immediately received at the reception / playback terminal. A method of generating a transparent caption packet by describing a control command to be discarded is also mentioned. In this case, the transparent subtitle packet does not form “subtitles are transparently superimposed on the content”, but the subtitles are not displayed when the content is played back. As a result, the subtitles are superimposed on the content and displayed transparently. The same effect can be obtained.

  As described above, the characteristic function of the present invention is that the subtitle packet or / and the transparent subtitle packet are multiplexed in advance in the transport stream for the purpose of suddenly inserting the subtitle packet later. Therefore, when there is already a caption packet to be multiplexed in the transport stream by the caption packetizing unit, the generation of the transparent caption packet is determined by making the following determination to determine whether to generate the transparent caption packet. You may do it.

  For example, from the viewpoint of notifying the content viewer, it is efficient to display subtitles at the beginning of the content. Therefore, for example, with reference to a subtitle packet control command (definition of subtitle display time) shown in FIG. 3, it is determined by a logical operation such as a CPU whether there is a subtitle packet to be displayed within a predetermined time from the start of content reproduction. If it is determined that there is no caption to be displayed, a transparent caption packet for transparently displaying the caption is generated at the predetermined time.

  Further, as will be described later in the second embodiment, subtitles or transparent subtitles may be configured to be displayed over the entire content so that a new subtitle can be inserted immediately when the content is played back at any timing. . Therefore, the display time of the caption packet to be displayed is obtained by referring to the control command for the caption packet shown in FIG. 3, and the total display time and the total playback time of the content are compared by a logical operation such as a CPU. If it is determined that the two do not match, a time zone during which no subtitle is displayed is specified from the control command of the subtitle packet, and a transparent subtitle packet is generated to display the subtitle transparently during that time zone. , And so on.

  As described above, the transport stream generating apparatus according to the present embodiment generates a transparent subtitle packet even if there is no subtitle to be displayed, and multiplexes this with TS data to add new subtitles such as emergency information on the later date. It is easy to handle.

  The “management information acquisition unit” (0204) has a function of acquiring management information of video and audio packets and subtitle packets in the transport stream. For example, an arithmetic device such as a CPU, a main memory, a management information acquisition program, etc. Can be realized. “Management information” refers to information described regarding the processing of each multiplexed packet for TS data, and includes information indicated by, for example, PAT (Program Association Table) or PMT (Program Map Table). It is done. “PAT” is a packet in which the PID (packet identifier) of the next PMT packet is described. Further, the PID of the PAT is constant as “0x00”. That is, the content reception / playback apparatus first checks each packet constituting the TS data and specifies the PAT indicated by the PID “0x00”. Then, the PID of the PMT indicated by the PAT packet is acquired, and the PMT packet in the TS data can be specified with reference to the PID.

  “PMT” is a packet indicating a PID such as a video packet, audio packet, subtitle packet, transparent subtitle packet, or an identifier indicating a video or audio encoding format. Therefore, PIDs of video, audio, and subtitle / transparent subtitle packets can be acquired from the PMT specified by referring to the PAT, and the type of each packet multiplexed in the TS data can be specified.

  In the management information acquisition unit, the PID of the PMT, the PID of video, audio, and subtitle / transparent subtitle packet is determined and acquired according to a preset rule.

  The “management packet generation unit” (0205) has a function of generating a management packet indicating the acquired management information, and can be realized by, for example, an arithmetic device such as a CPU, a main memory, a management packet generation program, or the like. Specifically, this management packet generation unit generates a “PAT packet” using the PID of the acquired PMT. In addition, a “PMT packet” is generated using the acquired video, audio, and PID of the subtitle / transparent subtitle packet.

  “Transport Streaming Unit” (0206) has a function of transporting generated packets in accordance with management information, and is realized by an arithmetic device such as a CPU, a main memory, a transport stream program, or the like. be able to.

  FIG. 5 is a diagram illustrating an example of transport stream processing in the transport stream unit. As shown in this figure, video data, audio data, subtitle / transparent subtitle data packets v, a, t1 / t2 are multiplexed, and packets including management information such as PAT and PMT are also multiplexed. One TS data is generated.

  When this TS data is distributed as it is, it is sequentially decoded by the reception / reproduction device, and video and audio are reproduced. Also, captions are superimposed and displayed in a scene including the caption packet (t1), and captions are not displayed because a transparent layer is superimposed in a scene including the transparent caption packet (t2).

  Further, as described above, the content generated by the transport stream generating apparatus of the present embodiment is made into a transport stream by inserting a transparent subtitle packet in advance even if there is no subtitle to be displayed. Therefore, for example, when it is desired to insert and display emergency earthquake information suddenly in this video content due to, for example, an earthquake, the content distribution server (to be described later) does not perform processing such as Demux or decoding. New subtitle data can be easily inserted by overwriting the subtitle packet in the transparent subtitle packet or the like.

    <Hardware configuration>

  FIG. 6 is a schematic diagram illustrating an example of a configuration of the transport stream generation device when the above functional components are realized as hardware. The operation of each hardware component in the TS data generation process including the transparent caption packet will be described with reference to FIG.

  As shown in this figure, the transport stream device includes a “CPU (central processing unit)” (0601) and “main memory” (0602) for executing various arithmetic processes. Also, an “HDD (Hard Disk Drive)” (0603) that holds the generated TS data, and an “I / O (Input / Output)” (0604) for acquiring each content data of video, audio, and subtitles. I have. They are connected to each other via a data communication path such as a “system bus” to transmit / receive information and process information.

  Further, a program for realizing each component shown in FIG. 2 is read out in the “main memory”, and the “CPU” executes various arithmetic processes by referring to the read out program. In addition, a plurality of addresses are assigned to each of the “main memory” and “HDD”, and in the arithmetic processing of the “CPU”, the address is specified and the stored data is accessed to access the data. It is possible to perform the arithmetic processing used.

  Here, for example, when a transport stream generation command is received from an operator via an input device or a GUI (not shown), the integrated interface device of the present embodiment stores the video / audio packetization program and subtitle packetization in the “main memory”. A program, a transparent caption packetization program, a management information acquisition program, a management packet generation program, a transport stream program, and the like are read out.

  Then, the “CPU” interprets the video / audio packetization program, and first, according to the program, the data of the video / audio content input from “I / O” is stored in the address 1 of the “main memory”. Then, the content data is divided into PES packets, and a packet header is added to each PES packet. Further, the video / audio packet generated by dividing the PES packet into TS packets of a predetermined length of 188 bytes (or 192 bytes) is stored at address 2 of “main memory”.

  Subsequently, the “CPU” interprets the subtitle packetizing program, and stores the subtitle content data input from “I / O” at address 3 of the “main memory” in the same way as video / audio packetization according to the program. To do. Then, the content data is divided into TS packets via PES packets and stored as subtitle packets at address 4 of “main memory”.

  Further, the “CPU” interprets the transparent caption packetizing program, and generates caption data using, for example, NULL data as main data of the content according to the program. Then, the NULL data is divided into TS packets via PES packets, and stored as transparent caption packets at address 5 of “main memory”.

  Subsequently, the “CPU” interprets the caption packetization program, and in accordance with the program, the ID and the like are obtained as management information from the packet header of the generated video / audio packet, caption packet, and transparent caption packet, for example. Is stored at address 6. Then, the “CPU” interprets the management packet generation program, and in accordance with the program, the acquired video / audio packet, subtitle packet, PMT packet indicating the identification ID of the transparent subtitle packet, PAT packet indicating the identification ID of the PMT packet, etc. It is generated as a management packet and stored at address 7 of “main memory”.

  Then, the “CPU” interprets the transport stream program, and in accordance with the program, the video / audio packet, subtitle packet, transparent subtitle packet, and management packet stored in the “main memory” are multiplexed to generate TS data. . The generated TS data is recorded and held at address A of “HDD”.

  In this way, the transport stream generating apparatus according to the present embodiment can convert the content into a transport stream with the transparent caption packet inserted in advance. Therefore, when it is desired to quickly insert and display subtitle data as described above, a subtitle packet for new insertion display is transmitted from the content distribution server (to be described later) to the subtitle packet of the TS data without performing processing such as Demux or decoding. And new subtitle data can be easily inserted by overwriting in a transparent subtitle packet.

    <Process flow>

  FIG. 7 is a flowchart illustrating an example of a processing flow in the transport stream generation device according to the present embodiment. The steps shown below may be steps executed by each hardware configuration of the computer as described above, or may be processing steps that constitute a program for controlling the computer recorded on a medium. I do not care.

  As shown in this figure, first, content consisting of video and / or audio is acquired and converted into video and audio packets (step S0701). Also, the content composed of subtitles is acquired and converted into subtitle packets (step S0702). In addition, a transparent caption packet is generated (step S0703). In addition, when there is no caption content as described above, only a transparent caption packet may be generated. Alternatively, it is determined whether subtitles are displayed at all playback timings of video content, and in the case of display, transparent subtitle packets need not be generated.

  Subsequently, the management information of the video / audio packet and the caption packet is acquired (step S0704), and a management packet indicating the acquired management information is generated (step S0705). Then, the packet generated according to the management information is converted into a transport stream (step S0706).

    <Brief description of effect>

  As described above, the transport stream generating apparatus according to the present embodiment can convert the content data into a transport stream in which a transparent caption packet is inserted in advance even if there is no caption to be displayed. Therefore, for example, when it is desired to insert and display emergency earthquake information suddenly in this video content due to, for example, an earthquake, the content distribution server (to be described later) does not perform processing such as Demux or decoding. New subtitle data can be easily inserted by overwriting the subtitle packet in the transparent subtitle packet or the like.

  << Example 2 >>

    <Overview>

  This embodiment is based on the first embodiment, and has a function of inserting a transparent subtitle packet so that a transparent subtitle is superimposed over the entire playback time of the content, so that a new subtitle can be inserted at any timing of the content. A transport stream generating device that generates TS data that can be additionally displayed.

    <Functional configuration>

  FIG. 8 is a diagram illustrating an example of functional blocks in the transport stream generation device according to the present embodiment. As shown in this figure, the “transport stream generating device” (0200) of the present embodiment is based on the first embodiment and includes a “video / audio packetization unit” (0801) and a “caption packetization unit” (0802). A “transparent subtitle packetization unit” (0803), a “management information acquisition unit” (0804), a “management packet generation unit” (0805), and a “transport stream conversion unit” (0806). . Since each of the above-described constituent elements has already been described in the first embodiment, the description thereof is omitted.

  A feature of the transport stream generation device according to the present embodiment is that the management information acquisition unit further includes an “all acquisition unit” (0807).

  “All acquisition means” (0807) has a function of acquiring management information for transport stream so that transparent subtitle packets are superimposed in the entire playback time of the content to be composed of the transport stream, For example, it can be realized by an arithmetic device such as a CPU, a main memory, an all acquisition program, or the like. Specifically, for example, the display time of the caption packet to be displayed is acquired by referring to the control command for the caption packet shown in FIG. Subsequently, the total playback time is acquired from the content data amount and the like. Then, both are compared by a logical operation such as CPU. If it is determined that the two do not match, a time zone during which no caption is displayed is calculated from the control command of the caption packet. Then, a control command for transparently displaying subtitles is acquired in that time zone.

  Then, the transparent caption packet to which the control command is added is multiplexed in the TS data, so that the transparent caption packet can be superimposed over the entire playback time of the content.

    <Process flow>

  FIG. 9 is a flowchart illustrating an example of a processing flow in the transport stream generation device according to the present embodiment. The steps shown below may be steps executed by each hardware configuration of the computer as described above, or may be processing steps that constitute a program for controlling the computer recorded on a medium. I do not care.

  As shown in this figure, first, content consisting of video and / or audio is acquired and converted into video and audio packets (step S0901). In addition, content consisting of subtitles is acquired and converted into subtitle packets (step S0902). In addition, a transparent caption packet is generated (step S0903).

  Subsequently, the management information of the video / audio packet and the caption packet in the transport stream is acquired so that the transparent caption is superimposed in the entire reproduction time of the content (step S0904), and the management packet indicating the acquired management information is acquired. Generate (step S0905). Then, the packet generated according to the management information is converted into a transport stream (step S0906).

    <Brief description of effect>

  As described above, with the transport stream generation apparatus of the present embodiment, it is possible to multiplex transparent subtitle packets in TS data so that transparent subtitles are superimposed over the entire playback time of the content. Therefore, TS data capable of additionally displaying a new caption can be generated at any timing of the content.

  Example 3

    <Overview>

  In the present embodiment, a content distribution server will be described that has a function of urgently inserting new caption data such as emergency disaster information in cooperation with the transport stream generating apparatus of the above embodiment. Specifically, the content distribution server according to the present embodiment sets the multiplexing position of the subtitle packet / transparent subtitle packet that is multiplexed and inserted in advance for the generated TS data stored and held for the distribution. It has a function of identifying and recording new caption data (emergency disaster information, etc.) at that position by overwriting it.

  In this way, for example, when an emergency earthquake information is to be suddenly inserted and displayed in this video content due to, for example, an earthquake, processing such as Demux or decoding is performed on the TS data stored and held for distribution. New subtitle data can be inserted easily without having to do so.

    <Functional configuration>

  FIG. 10 is a diagram illustrating an example of functional blocks in the content distribution server of the present embodiment. As shown in this figure, the “content distribution server” (1010) of this embodiment includes an “accumulation unit” (1011), an “acquisition unit” (1012), a “character information acquisition unit” (1013), It has a “recording unit” (1014) and a “sending unit” (1015). This content distribution server may be realized by a hardware device separate from the transport stream generation device of the above embodiment, or may be realized by the same hardware device, and both functions are realized separately by software. You may comprise so that it may be carried out.

  The “accumulation unit” (1011) has a function of accumulating the transport stream generated by the transport stream generation device (1000) described in the above embodiment. For example, an HDD, a flash memory, or an optical recording medium Or a storage device such as a reading device, an accumulation recording program, or the like. Specifically, for example, a transport stream acquired via I / O or the like is recorded and stored in a recording device such as an HDD.

When a delivery request is received from the client terminal device (playback terminal) on the network with respect to the transport stream accumulated in this way, the transport stream is delivered to the playback terminal in response to the request. . Further, in the content distribution server of this embodiment, subtitle packets / transparent subtitle packets are multiplexed and inserted in the transport stream that is stored and held, so that new subtitle data can be easily inserted with the following configuration. It is characterized by being able to.
Alternatively, the storage unit may be configured to store a transport stream generated by a device other than the transport stream generation device described in the above embodiment. However, even in such a case, in order to easily insert a new subtitle packet according to the configuration described later, the accumulated transport stream includes a video / audio packet, a subtitle packet superimposed on the content and displayed transparently, and a transport stream. This is generated by multiplexing the management packet indicating the management information of the video / audio packet and the caption packet in the port stream.

  The “acquisition unit” (1012) has a function of acquiring the transport stream stored in the storage unit in response to a playback request from the playback terminal. For example, an arithmetic device such as a CPU, a main memory, an I / O, It can be realized by an acquisition program. Specifically, when an HTTP request from a playback terminal on the network is received by I / O, the transport stream stored in the HDD or the like is read out to the main memory in accordance with the acquisition program.

  The “character information acquisition unit” (1013) has a function of acquiring character information to be displayed superimposed on the content constituted by the transport stream acquired by the acquisition unit. It can be realized by a memory, a character information acquisition program, or the like. The “character information” may be any subtitle that should be newly superimposed on the content, and includes, for example, emergency disaster information, other news information, weather forecast, stock information, advertisement information, and the like.

  FIG. 11 is a diagram illustrating an example of the character information acquired here. Since the basic data structure of the character information is the same as that shown in FIGS. 3 and 4, the description of “data_group_id” and “num_languages” that have already been described is omitted. Then, for example, in “SDF subtitle area size specifying command: aa” specified in the extended control command portion of the subtitle management information, the position where the subtitle data is to be displayed is specified as shown in FIG. .

  Then, subtitle data indicated by “b3f4bc...” Or the like is converted into a predetermined character code at the designated display position, and is displayed as a subtitle such as “I will notify you of stock information” as shown in FIG. It is that.

  The character information acquired by the character information acquisition unit does not necessarily exist. For example, in the case of emergency disaster information, there is no need to newly insert such subtitles if no disaster has occurred. Therefore, the content distribution server is configured to hold flag information indicating whether or not new character information to be multiplexed and inserted has been newly input and accumulated, and when receiving a content distribution request from a playback terminal, the flag It may be configured to refer to Then, if the flag information indicates that character information to be multiplexed and inserted is newly accumulated, the character information is acquired.

  Further, the character information may be stored and stored in association with the identification information of the content to be multiplexed and inserted, and the character information acquired for each content may be different.

  The “recording unit” (1014) has a function of recording the acquired character information as a caption packet arranged in a caption packet position included in advance in the transport stream being acquired in response to a reproduction request, such as a CPU. It can be realized by a computing device, a main memory, a storage device such as an HDD, and a recording program. Specifically, the PAT packet indicated by the ID “0x00” is specified with reference to the packet ID of the transport stream. Then, the PMT ID indicated by the PAT packet is acquired, and the PMT packet in the TS data is specified with reference to the ID.

  Then, the ID of the caption packet and the transparent caption packet indicated by the PMT packet is acquired, and the caption packet and the transparent caption packet in the TS data are specified with reference to the ID. Then, for the identified subtitle packet or transparent subtitle packet, as shown in FIG. 14A, the character information acquired by the character information acquisition unit is divided into TS packets and the new subtitle packet is generated. Processing such as rewriting is performed. Then, a transport stream in which a new caption packet is inserted as shown in FIG. 14B can be easily generated.

  The new subtitle packet is recorded only for the transparent subtitle packet arrangement position, for example, and is not recorded at the subtitle packet arrangement position where some subtitles are displayed (multiplexed in advance). Also good. In addition, recording of a new caption packet may be executed for both.

  The “sending unit” (1015) has a function of sending the transport stream on which the recording has been performed to the playback terminal, and is realized by an arithmetic unit such as a CPU, a main memory, I / O, and a sending program, for example. can do. Specifically, TS packets constituting the transport stream are sequentially transmitted with the IP address of the transmission source indicated by the HTTP request for content distribution as the destination.

  In this way, various text information such as emergency earthquake information, breaking news, stock information, and advertisement information can be easily obtained without processing such as Demux and decoding on TS data stored and held for distribution. Can be inserted.

    <Hardware configuration>

  FIG. 15 is a schematic diagram illustrating an example of a configuration in the content distribution server when the above functional components are realized as hardware. The operation of each hardware component in the recording process of a new subtitle packet that does not involve Demux and decoding processing for TS data will be described using this figure.

  As shown in this figure, the content distribution server includes a “CPU” (1501) for executing various arithmetic processes and a “main memory” (1502). Also, an “HDD” (1503) that holds the acquired TS data and an “I / O” (1504) for transmitting / receiving information to / from the playback terminal via the network are provided. They are connected to each other via a data communication path such as a “system bus” to transmit / receive information and process information.

  Further, a program for realizing each component shown in FIG. 2 is read out in the “main memory”, and the “CPU” executes various arithmetic processes by referring to the read out program. In addition, a plurality of addresses are assigned to each of the “main memory” and “HDD”, and in the arithmetic processing of the “CPU”, the address is specified and the stored data is accessed to access the data. It is possible to perform the arithmetic processing used.

  Then, the content distribution server according to the present embodiment, in accordance with the storage recording program read into the “main memory”, transmits the transport stream generated as described in the above embodiment by the transport stream generation device to “I / O”. ”And records and holds it in“ HDD ”.

  Thereafter, when a reproduction request such as an HTTP request is received from a reproduction terminal on the network via “I / O”, the reproduction request is stored at address 1 of “main memory”. Then, using the reproduction request stored in the main memory as a trigger, the “CPU” first refers to the presence / absence flag information of new caption data (not shown) according to the acquisition program. If the flag information indicates that there is no new caption data, the content distribution server transmits the transport stream stored and held in the “HDD” to the playback request transmission source “I / O”. Send from.

  On the other hand, if the flag information indicates that there is new subtitle data, the content distribution server reads the requested transport stream from “HDD” and stores it at address 2 of “main memory” in accordance with the acquisition program. .

  Subsequently, the character information acquisition program is interpreted by the “CPU”, and subtitle data (not shown) newly input and accumulated in, for example, “HDD” is read and stored at address 3 of the “main memory” according to the program. Then, a new caption packet generated by dividing the caption data into TS packets is stored at address 4 of “main memory”.

  Then, the “CPU” interprets the recording program, and according to the program, the PAT, PMT, and subtitle packet / transparent subtitle packet are determined from the ID of the packet constituting the transport stream stored in the address 2 of the “main memory”. Identify. Then, according to the recording program, a new subtitle packet stored at address 4 of the “main memory” or the like is recorded by, for example, overwriting processing at the position in the TS data of the identified subtitle packet / transparent subtitle packet. To go. Then, the transport stream in which the new caption packet is recorded is stored at the address 5 of the “main memory”, and is transmitted from “I / O” to the transmission source of the reproduction request.

  In this way, various text information such as emergency earthquake information, breaking news, stock information, and advertisement information can be easily obtained without processing such as Demux and decoding on TS data stored and held for distribution. Can be inserted.

    <Process flow>

  FIG. 16 is a flowchart illustrating an example of a process flow in the content distribution server of the present embodiment. The steps shown below may be steps executed by each hardware configuration of the computer as described above, or may be processing steps that constitute a program for controlling the computer recorded on a medium. I do not care.

  As shown in this figure, first, the transport stream generated by the transport stream generating method shown in FIG. 7 and the like is recorded for storage in the storage unit (step S1601). Thereafter, when a reproduction request is received from a reproduction terminal on the network (step S1602), the transport stream accumulated in the accumulation unit is acquired (step S1603).

Subsequently, for example, after determining whether there is new character information to be superimposed (not shown), if there is character information to be superimposed, the character information is acquired (step S1604). Then, the acquired character information is recorded as a caption packet placed in a transparent caption packet position included in advance in the transport stream (step S1605), and the transport stream is transmitted to the playback terminal that is the source of the playback request. It is sent out (step S1606).
Note that the transport stream recorded and accumulated in step S1601 indicates video / audio packets, subtitle packets superimposed on content and displayed transparently, and management information of video / audio packets and subtitle packets in the transport stream. As long as it is generated by multiplexing the management packet, it does not necessarily have to be generated by the method shown in FIG.

  Further, an example of processing for recording new character information in the transport stream in step S1605 will be described in more detail below.

  FIG. 17 is a flowchart showing an example of the recording process of the new character information in the transport stream. As shown in this figure, the next packet A is read from the transport stream stored in the storage unit (step S1701). Here, if the next packet A cannot be read out, the process is terminated assuming that the last packet of the transport stream has been checked.

  On the other hand, when the next packet A is successfully read, the CPU A refers to the ID of the packet A through arithmetic processing, and determines whether or not the ID matches the ID of the transparent caption data indicated by the PMT. Alternatively, it is determined whether or not the ID of the caption data is NULL data whose data body is “fff...”. Based on this determination, it is determined whether the read packet A is a transparent caption packet (step S1702). If the packet A is not a transparent caption packet, the read packet A is transmitted as it is.

  If the read packet A is a transparent subtitle packet, for example, flag information or the like is referred to by CPU calculation processing to check whether there is new character information (step S1703). Then, if a determination result that there is no new character information is output from the CPU, the read packet A is transmitted as it is.

  On the other hand, if a determination result indicating that there is new character information is output from the CPU, the caption packet B generated from the new character information is read (step S1704), and the time stamp value of the packet A is set to the time of the caption packet B. It is rewritten as a stamp value (step S1705). Then, the packet A is discarded, and a caption packet B indicating new character information is transmitted to the playback terminal (step S1706).

    <Brief description of effect>

  As described above, with the content distribution server of this embodiment, various character information such as emergency earthquake information, breaking news, stock information, and advertisement information are demultiplexed and decoded with respect to TS data stored and held for distribution. It can be easily inserted without the above process.

    <Other configuration of this embodiment>

  Further, for example, in the case of a transparent subtitle packet, the content distribution server according to the present embodiment does not have a subtitle to be displayed in the first place, and thus performs a process of replacing (or overwriting) with a subtitle packet of new character information as described above. good. On the other hand, in the case of a subtitle packet, it may be configured such that both subtitles are displayed by synthesizing a new subtitle packet and the subtitle packet that is present in advance as follows.

  FIG. 18 is a diagram illustrating an example of the synthesis process of both caption packets. As shown in this figure, there is a caption packet 1 such as emergency information to be newly displayed, and a caption packet 2 multiplexed in advance in a transport stream. When the caption packet 2 in the transport stream is specified as described above, the main part of the caption data excluding the packet header is extracted. Similarly, for the new caption packet 1, the main part of the caption data excluding the packet header is extracted. Then, a line feed code “0c0c” is sandwiched, the two caption data are combined, and a packet header is added to generate a composite caption packet. Then, the composite subtitle packet is transmitted to the playback terminal.

  When played back on the playback terminal in this way, existing caption data (for example, Japanese captions in English dialogue) and new caption data (such as earthquake emergency disaster information) are divided into two stages at the same time. Can be displayed in the content. In addition, the subtitle display area designation command is set to the full screen, and new subtitle data is displayed in the upper part of the content display screen by inserting NULL data such as “fff...” After the line feed code. You may perform a synthetic | combination process so that subtitle data previously existing in the lower stage may be displayed.

  FIG. 19 is a flowchart illustrating an example of a synthesis process of such a new caption packet and a caption packet that is present in advance. In addition, in this figure, not only the above synthesis process but also a process for exchanging a transparent subtitle packet and a new subtitle packet and then synthesizing the new subtitle packet and a pre-existing subtitle packet are described.

  As shown in this figure, the next packet is read from the transport stream stored in the storage unit (step S1901). Here, when the next packet cannot be read out, the process is terminated assuming that the last packet of the transport stream has been checked.

  On the other hand, if the next packet is successfully read, the CPU ID is used to refer to the packet ID to identify whether the read packet is a transparent caption packet (step S1902). The read packet A is sent as it is (step S1904).

  If the read packet is a transparent subtitle packet, for example, flag information or the like is subsequently referred to by CPU calculation processing to check whether there is new character information (step S1903). If the CPU determines that there is no new character information, the read packet is sent as it is.

  On the other hand, if the determination result that there is new character information is output from the CPU, the caption packet generated from the new character information and the transparent caption packet are exchanged (step S1905). Then, the new caption packet is stored in the buffer (step S1906), and the next packet is read from the storage unit (step S1907). In the same manner as described above, it is determined whether or not the read packet is a normal subtitle packet (step S1908). If it is not a subtitle packet, a new subtitle packet stored in the buffer is transmitted to the playback terminal (step S1908). S1909).

  If the read packet is a subtitle packet in advance, the character data of both is synthesized using a line feed code or the like as described in FIG. 18 (step S1910), and the synthesized subtitle packet is transmitted to the playback terminal. It is sent out (step S1911).

As described above, various character information such as emergency earthquake information, breaking news, stock information, and advertisement information can be easily inserted while leaving a predetermined caption.
The content distribution server can be applied to any of the multicast, broadcast, and unicast distributions. In the unicast method, it is possible to determine the contract content of subtitle information and distribute different subtitle content for each user. For example, stock information can be distributed to a certain user, and breaking news can be distributed to a certain user.

  Example 4

    <Overview>

  This example is based on the above example. For example, highly urgent subtitle data such as disaster evacuation information or subtitle data that a service provider or the like always wants to display on the playback terminal is compulsory at the content distribution destination. The content distribution server is characterized in that it is displayed.

    <Functional configuration>

  FIG. 20 is a diagram illustrating an example of functional blocks in the content distribution server according to the present embodiment. As shown in this figure, the “content distribution server” (2010) of the present embodiment is based on the third embodiment and is based on the “storage unit” (2011), “acquisition unit” (2012), and “character information acquisition”. Part "(2013)," recording part "(2014), and" sending part "(2015). Since each of the above-described constituent elements has already been described in the third embodiment, description thereof will be omitted.

  And the feature point of the content delivery server of a present Example is a point in which a character information acquisition part has a "forced character information acquisition means" (2016) further.

  “Forced character information acquisition means” (2016) has a function of acquiring forced character information. “Forced character information” refers to character information associated with a forced identifier that identifies character information that should be forcibly displayed on the playback terminal as character information to be displayed. For example, subtitle management information shown in FIG. In the DMF field, “0” is indicated, and character information of a subtitle packet that is controlled to be automatically displayed at the time of reception is included.

  The recording unit records the subtitle packet including the compulsory identifier, so that the subtitle can be forcibly displayed on the playback terminal.

    <Process flow>

  FIG. 21 is a flowchart illustrating an example of a processing flow in the content distribution server of the present embodiment. The steps shown below may be steps executed by each hardware configuration of the computer as described above, or may be processing steps that constitute a program for controlling the computer recorded on a medium. I do not care.

  As shown in this figure, first, the transport stream generated by the transport stream generating method shown in FIG. 7 and the like is recorded for storage in the storage unit (step S2101). Thereafter, when a reproduction request is received from a reproduction terminal on the network (step S2102), the transport stream accumulated in the accumulation unit is acquired (step S2103).

  Subsequently, for example, after the process of determining whether or not there is forced character information to be superimposed, which is not shown, if there is forced character information to be superimposed, the forced character information is acquired (step S2104). Then, the acquired forced character information is recorded as a caption packet arranged at a transparent caption packet position included in advance in the transport stream (step S2105), and the transport stream is transmitted to the playback terminal that is the source of the playback request. (Step S2106).

    <Brief description of effect>

  As described above, the content distribution server of this embodiment complies with the content distribution destination for highly urgent subtitle data such as disaster evacuation information and subtitle data that the service provider and the like always want to display on the playback terminal. Can be displayed automatically.

Conceptual diagram for describing an example of TS data generated by the transport stream generation device according to the first embodiment. The figure showing an example of the functional block in the transport stream production | generation apparatus of Example 1. FIG. Conceptual diagram showing an example of a caption packet indicating caption data itself Conceptual diagram showing an example of a caption packet indicating caption management information The figure showing an example of the transport stream production | generation process in the transport stream production | generation part of the transport stream production | generation apparatus of Example 1. FIG. 1 is a diagram illustrating an example of a hardware configuration in a transport stream generation device according to Embodiment 1. FIG. 7 is a flowchart illustrating an example of a process flow in the transport stream generation device according to the first embodiment. The figure showing an example of the functional block in the transport stream production | generation apparatus of Example 2. FIG. 10 is a flowchart illustrating an example of a process flow in the transport stream generation device according to the second embodiment. The figure showing an example of the functional block in the content delivery server of Example 3. The figure showing an example of the character information acquired in the character information acquisition part of the content delivery server of Example 3 The figure showing an example of the display area of the character information designated by character information The figure showing an example of the character information displayed by character information Schematic diagram showing an example of recording processing in the recording unit of the content distribution server of the embodiment 3 The figure showing an example of the hardware constitutions in the contents distribution server of Example 3. The flowchart showing an example of the flow of processing in the content delivery server of Example 3. 10 is a flowchart illustrating an example of a process for recording new character information in a transport stream in the content distribution server according to the third embodiment. The conceptual diagram showing an example of the synthetic | combination process of the new subtitle packet and the existing subtitle packet in the other content delivery server of Example 3 The flowchart showing an example of the synthesis | combination process of the new subtitle packet and the existing subtitle packet in the other content delivery server of Example 3. The figure showing an example of the functional block in the content delivery server of Example 4. The flowchart showing an example of the flow of a process in the content delivery server of Example 4. Conceptual diagram for explaining an example of a process for multiplexing and inserting a new caption packet into TS data in the prior art

Explanation of symbols

0200 Transport stream generation device 0201 Video / audio packetization unit 0202 Subtitle packetization unit 0203 Transparent subtitle packetization unit 0204 Management information acquisition unit 0205 Management packet generation unit 0206 Transport stream generation unit 1010 Content distribution server 1011 Storage unit 1012 Acquisition unit 1013 Character information acquisition unit 1014 Recording unit 1015 Sending unit

Claims (12)

A video / audio packetization unit for acquiring video and / or audio content and video / audio packetization;
A subtitle packetizing section that acquires subtitle content and subtitles the packet,
A transparent subtitle packetizing unit that generates a subtitle packet that is superimposed on content and displayed transparently;
A management information acquisition unit for acquiring management information of video and audio packets and subtitle packets in the transport stream;
A management packet generator for generating a management packet indicating the acquired management information;
A transport stream converting unit that converts the packet generated according to the management information into a transport stream;
A transport stream generating apparatus having   The management information acquisition unit has all acquisition means for acquiring management information for transport stream so that transparent subtitle packets are superimposed in all playback times of content to be composed of transport streams. The transport stream generation device described. A storage unit that stores the transport stream generated by the transport stream generation device according to claim 1 or 2,
An acquisition unit that acquires a transport stream stored in the storage unit in response to a playback request from a playback terminal;
A character information acquisition unit that acquires character information to be displayed in a superimposed manner on content configured by the transport stream acquired by the acquisition unit;
A recording unit for recording the acquired character information as a subtitle packet arranged in a subtitle packet position included in advance in a transport stream being acquired in response to a reproduction request;
A sending unit for sending the recorded transport stream to a playback terminal;
A content distribution server. The character information acquisition unit has compulsory character information acquisition means for acquiring compulsory character information that is character information associated with a compulsory identifier that identifies character information to be forcibly displayed on the playback terminal as character information to be displayed. And
The content distribution server according to claim 3, wherein the recording unit records the subtitle packet including the compulsory identifier at the time of recording. The transport stream generated by multiplexing the video / audio packet, the caption packet superimposed on the content and displayed transparently, and the management packet indicating the management information of the video / audio packet and the caption packet in the transport stream is stored. A storage unit;
An acquisition unit that acquires a transport stream stored in the storage unit in response to a playback request from a playback terminal;
A character information acquisition unit that acquires character information to be displayed in a superimposed manner on content configured by the transport stream acquired by the acquisition unit;
A recording unit for recording the acquired character information as a subtitle packet arranged in a subtitle packet position included in advance in a transport stream being acquired in response to a reproduction request;
A sending unit for sending the recorded transport stream to a playback terminal;
A content distribution server. The character information acquisition unit has compulsory character information acquisition means for acquiring compulsory character information that is character information associated with a compulsory identifier that identifies character information to be forcibly displayed on the playback terminal as character information to be displayed. And
The content distribution server according to claim 5, wherein the recording unit records as a caption packet including the compulsory identifier at the time of recording. A video / audio packetizing step of acquiring video and / or audio content and video / audio packetizing;
Subtitle packetizing step for acquiring subtitle content and subtitle packetizing,
Transparent subtitle packetizing step for generating a subtitle packet that is superimposed on content and displayed transparently;
Management information acquisition step of acquiring management information of video and audio packets and subtitle packets in the transport stream;
A management packet generation step for generating a management packet indicating the acquired management information;
A transport stream step of transporting a packet generated according to the management information into a transport stream;
A transport stream generation method for causing a computer to execute.   The management information acquisition step includes all acquisition steps for acquiring management information for transport stream so that transparent subtitle packets are superimposed in all playback times of content to be configured with a transport stream. The transport stream generation method described. An accumulation recording step of recording the transport stream generated by the transport stream generation method according to claim 7 or 8 for accumulation in the accumulation unit;
An acquisition step of acquiring a transport stream recorded and stored in the storage unit in response to a playback request from a playback terminal;
A character information acquisition step of acquiring character information to be displayed superimposed on the content constituted by the transport stream acquired in the acquisition step;
A recording step of recording the acquired character information as a subtitle packet arranged in a subtitle packet position included in advance in a transport stream being acquired in response to a reproduction request;
A sending step for sending the recorded transport stream to a playback terminal;
A method for controlling a content distribution server that causes a computer to execute the above. The character information acquisition step includes a compulsory character information acquisition step of acquiring compulsory character information that is character information associated with a compulsory identifier that identifies character information to be forcibly displayed on the playback terminal as character information to be displayed. ,
10. The content distribution server control method according to claim 9, wherein the recording step records the subtitle packet including the compulsory identifier at the time of recording. Storage unit for storing a transport stream generated by multiplexing video and audio packets, subtitle packets superimposed on content and displayed transparently, and management packets indicating management information of video and audio packets and subtitle packets in the transport stream An accumulation recording step for recording to accumulate in
An acquisition step of acquiring a transport stream recorded and stored in the storage unit in response to a playback request from a playback terminal;
A character information acquisition step of acquiring character information to be displayed superimposed on the content constituted by the transport stream acquired in the acquisition step;
A recording step of recording the acquired character information as a subtitle packet arranged in a subtitle packet position included in advance in a transport stream being acquired in response to a reproduction request;
A sending step for sending the recorded transport stream to a playback terminal;
A method for controlling a content distribution server that causes a computer to execute the above. The character information acquisition step includes a compulsory character information acquisition step of acquiring compulsory character information that is character information associated with a compulsory identifier that identifies character information to be forcibly displayed on the playback terminal as character information to be displayed. ,
12. The content distribution server control method according to claim 11, wherein the recording step records the subtitle packet including the compulsory identifier at the time of recording.

Images