JP3680072B2 - Image information encoding method, reproducing method, and reproducing apparatus - Google Patents

Description

  The present invention relates to an image information encoding method for encoding image information such as a sub-video that is reproduced simultaneously with a main video, and a reproducing apparatus and a reproducing method for decoding and reproducing the image information.

  2. Description of the Related Art In recent years, a moving image-compatible optical disk reproducing apparatus that reproduces an optical disk in which video and audio data are digitally recorded has been developed, and is widely used as a reproducing apparatus such as movie software and karaoke. Recently, a data compression system for moving images has been internationally standardized as a moving picture image coding expert group (MPEG) system. This MPEG system is a system for variable-length compression of video data.

  Currently, the MPEG2 system is being internationally standardized, and accordingly, a system format corresponding to the MPEG compression system is also defined as the MPEG2 system layer. In this MPEG2 system layer, it is defined that a transfer start time and a reproduction start time expressed using a reference time are set for each data so that moving images, sounds, and other data can be transferred and reproduced in synchronization. Yes. Normal reproduction is performed based on these pieces of information.

  One or more sub-pictures superimposed on the main picture, such as movie subtitles and images displayed as TV volume settings, are run-length compressed and recorded so as to be played back in synchronization with the main picture. The sub-picture is reproduced for each sub-picture unit composed of a plurality of packs based on a time stamp given to the sub-picture unit.

  In this case, the time stamp of the corresponding first sub-picture unit cannot be set to zero, and a value considering the transfer time of the pixel data has to be described.

  One sub-video unit is composed of a plurality of sequences, and is composed of a display control sequence including a time stamp as a display start time given to each sequence in order of each sequence, a command, and the like.

  An object of the present invention is to use the same display control sequence table even when displaying the same sub-picture data at a plurality of different times, and to make the display control sequence table relocatable.

In the present invention, sub-video information that can be reproduced simultaneously with main video information constitutes a sub-video data unit, the sub-video data unit is divided into packets of a certain data size, and the sub-video packet includes a packet header, The packet header includes a time stamp representing a display time, and the sub-picture data unit further includes a sub-picture unit header, run-length compressed sub-picture pixel data, and a display control sequence table. The video unit header includes start address information of the display control sequence table, the run length compression is performed by converting the same pixel data of the sub-video into data units, and the display control sequence table includes a plurality of display Including the control sequence, one display control sequence is Information indicating the start time for the display control of the sub picture, including the address information, and a region of the plurality of display control commands, said address information is configured to indicate the position of the next display control sequence, one As the display control command set in the display control command area, a command for setting a color code of the image data of the sub-picture is possible, and a pattern pixel, an emphasized pixel, and a background pixel are set by the command for setting the color code. The color code can be set for the sub-picture, and the pixel data of the sub-picture is encoded.

  As described above in detail, according to the present invention, the same display control sequence table can be used even when the same sub-picture data is displayed at a plurality of different times, and the display control sequence table can be made relocatable. .

  An image information encoding / decoding system according to an embodiment of the present invention will be described below with reference to the drawings. In addition, in order to avoid duplication description, the common referential mark is used for the part which is common in a function over several drawings.

  1 to 27 are diagrams for explaining an image information encoding / decoding system according to an embodiment of the present invention.

  FIG. 1 schematically shows a recording data structure of an optical disc OD as an example of an information holding medium to which the present invention can be applied.

  This optical disc OD is a double-sided bonded disc having a storage capacity of about 5 Gbytes on one side, for example, and a large number of recording tracks are arranged between the lead-in area on the inner circumference side of the disc and the lead-out area on the outer circumference side of the disc. Yes. Each track is composed of a large number of logical sectors, and various information (digital data compressed as appropriate) is stored in each sector.

  FIG. 2 shows an example of the data structure of a video (video) file recorded on the optical disc OD of FIG.

  As shown in FIG. 2, this video file includes file management information 1 and video data 2. The video data 2 records a video data unit (block), an audio data unit (block), a sub-video data unit (block), and information (DSI; Disk Search Information) necessary to control the data reproduction. DSI unit (block). Each unit is divided into packets having a fixed data size for each data type, for example. The video data unit, the audio data unit, and the sub-picture data unit are reproduced in synchronization with each other based on the DSI arranged immediately before these unit groups.

  That is, a system area for storing system data used in the disk OD, a volume management information area, and a plurality of file areas are formed in an aggregate of a plurality of logical sectors in FIG.

  Among the plurality of file areas, for example, file 1 is main video information (VIDEO in the figure), sub-picture information having sub-contents with respect to the main video (SUB-PICTURE in the figure), audio information (figure in the figure). AUDIO), playback information (PLAYBACK INFO. In the figure), and the like.

  FIG. 3 illustrates a logical structure of an encoded (run-length compressed) sub-picture information pack among the data structures illustrated in FIG.

  As shown in the upper part of FIG. 3, one pack of sub-picture information included in the video data is composed of 2048 bytes (2 kB), for example. One pack of sub-picture information includes one or more sub-picture packets after the header of the first pack. Each pack header is given a reference time (SCR; System Clock Reference) through reproduction of the entire file, and a sub-picture packet in a pack of sub-picture information to which the SCR of the same time is given will be described later. It is transferred to the decoder. The first sub-picture packet includes run-length compressed sub-picture data (SP DATA1) after the header of the packet. Similarly, the second sub-picture packet includes run-length-compressed sub-picture data (SP DATA2) after the header of the packet.

  A plurality of such sub-picture data (SP DATA1, SP DATA2,...) Are collected for one unit (one unit) of run length compression, that is, a sub-picture unit header 31 is given to the sub-picture data unit 30. . After this sub-picture unit header 31, pixel data 32 obtained by run-length compression of video data for one unit (for example, data for one horizontal line of a two-dimensional display screen), and display control sequence information of each sub-picture pack are stored. A table 33 containing follows.

  In other words, one unit of run-length compressed data 30 is formed by a collection of one or more sub-video data portions (SP DATA1, SP DATA2,...) Of one or more sub-video packets. The sub-picture data unit 30 includes a sub-picture unit header SPUH 31 in which various parameters for sub-picture display are recorded, display data (compressed pixel data) PXD 32 composed of run-length codes, and a display control sequence table DCSQT 33. It consists of

  FIG. 4 illustrates a part of the content of the sub-picture unit header 31 in the run-length compressed data 30 for one unit illustrated in FIG. 3 (see FIG. 19 for the other parts of the SPUH 31). Here, data of sub-video (for example, caption corresponding to a movie scene of the main video) recorded and transmitted (communication) together with the main video (for example, video main body of the movie) will be described.

  As shown in FIG. 4, the sub-picture unit header SPUH 31 includes a display size of the pixel data 32 on the TV screen, that is, a display start position and a display range (width and height) SPDSZ, and display control in the sub-picture data packet. The recording start position SPDCSQTA of the sequence table 33 is recorded.

More specifically, in the sub-picture unit header SPUH31, as shown in FIG. 4, various parameters (SPDDADR etc.) having the following contents are recorded:
(1) Information (SPDSZ) indicating the display start position and display range (width and height) of the display data on the monitor screen;
(2) Recording start position information of the display control sequence table 33 in the packet (sub-picture display control sequence table start address SPDCSQTA).

  The sub-picture pixel data (run length data) 32 shown in FIG. 3 or FIG. 4 has a unit data length (variable length) determined by the run length compression rules 1 to 6 shown in FIG. Then, encoding (run length compression) and decoding (run length expansion) are performed with a fixed data length.

  FIG. 5 shows a run-length compression rule 1 employed in the encoding method according to the embodiment when the sub-picture pixel data (run-length data) 32 portion illustrated in FIG. 4 is composed of 2-bit pixel data. -6 are demonstrated.

  FIG. 6 is a diagram for specifically explaining the compression rules 1 to 6 when the sub-picture pixel data (run-length data) 32 portion illustrated in FIG. 4 is composed of 2-bit pixel data. It is.

  According to Rule 1 shown in the first column of FIG. 5, when 1 to 3 identical pixels continue, one data unit of encoding (run-length compression) is constituted by 4-bit data. In this case, the first 2 bits represent the number of continuous pixels, and the subsequent 2 bits represent pixel data (pixel color information or the like).

  For example, the first compressed data unit CU01 of the uncompressed video data PXD shown in the upper part of FIG. 6 includes two 2-bit pixel data d0, d1 = (0000) b (b is binary) ). In this example, two identical 2-bit pixel data (00) b are continuous (continued).

  In this case, as shown in the lower part of FIG. 6, d0 and d1 = (1000) b obtained by connecting the 2-bit display (10) b of the continuation number “2” and the content (00) b of the pixel data are after compression. Is the data unit CU01 * of the video data PXD.

  In other words, according to rule 1, (0000) b of data unit CU01 is converted to (1000) b of data unit CU01 *. In this example, substantial bit length compression is not obtained. However, for example, if CU01 = (000000) b in which three identical pixels (00) b are consecutive, CU01 * = (1100) b after compression. Thus, a 2-bit compression effect can be obtained.

  According to rule 2 shown in the second column of FIG. 5, when 4 to 15 identical pixels continue, one encoded data unit is constituted by 8-bit data. In this case, the first 2 bits represent an encoding header indicating that the rule 2 is used, the subsequent 4 bits represent the number of continuing pixels, and the subsequent 2 bits represent pixel data.

  For example, the second compressed data unit CU02 of the uncompressed video data PXD shown at the top of FIG. 6 includes five pieces of 2-bit pixel data d2, d3, d4, d5, d6 = (0101010101) b. . In this example, five identical 2-bit pixel data (01) b are continuous (continue).

  In this case, as shown in the lower part of FIG. 6, d2 to d6 in which the encoded header (00) b, the 4-bit display (0101) b of the continuation number “5”, and the content (01) b of the pixel data are connected. = (00010101) b is the data unit CU02 * of the compressed video data PXD.

  In other words, according to rule 2, (0101010101) b (10-bit length) of data unit CU02 is converted to (00010101) b (8-bit length) of data unit CU02 *. In this example, the effective bit length compression is only 2 bits from 10 bits to 8 bits, but if the continuation number is 15 (30 bits length because 15 01 of CU02 continues, this is 8 bits) Compression data (CU02 * = 00111101), and a compression effect of 22 bits is obtained for 30 bits. That is, the bit compression effect based on rule 2 is greater than that of rule 1. However, rule 1 is also necessary to support run-length compression of fine images with high resolution.

  According to rule 3 shown in the third column of FIG. 5, when 16 to 63 identical pixels continue, one encoded data unit is composed of 12-bit data. In this case, the first 4 bits represent a coding header indicating that the rule 3 is used, the subsequent 6 bits represent the number of continuing pixels, and the subsequent 2 bits represent pixel data.

  For example, the third compressed data unit CU03 of the uncompressed video data PXD shown in the upper part of FIG. 6 includes 16 pieces of 2-bit pixel data d7 to d22 = (101010... 1010) b. In this example, 16 pieces of the same 2-bit pixel data (10) b are continuous (continued).

  In this case, as shown in the lower part of FIG. 6, d7 to d22 in which the encoded header (0000) b, the 6-bit display (010000) b of the continuation number “16”, and the content (10) b of the pixel data are connected. = (000001000010) b is the data unit CU03 * of the compressed video data PXD.

  In other words, according to the rule 3, (101010... 1010) b (32-bit length) of the data unit CU03 is converted into (0000010010) b (12-bit length) of the data unit CU03 *. In this example, the effective bit length compression is 20 bits from 32 bits to 12 bits, but if the continuation number is 63 (126 bits because 63 of 10 CU03s continue), this is 12 bits. Compression data (CU03 * = 0000011111110), and a compression effect of 114 bits is obtained for 126 bits. That is, the bit compression effect based on rule 3 is greater than that of rule 2.

  According to Rule 4 shown in the fourth column of FIG. 5, when 64 to 255 identical pixels continue, one encoded data unit is composed of 16-bit data. In this case, the first 6 bits represent an encoding header indicating that the rule 4 is used, the subsequent 8 bits represent the number of continuing pixels, and the subsequent 2 bits represent pixel data.

  For example, the fourth compressed data unit CU04 of the uncompressed video data PXD shown in the upper part of FIG. 6 includes 69 pieces of 2-bit pixel data d23 to d91 = (111111... 1111) b. In this example, 69 identical 2-bit pixel data (11) b are continuous (continue).

  In this case, as shown in the lower part of FIG. 6, d23 to d91 are obtained by connecting the encoded header (000000) b, the 8-bit display (00100101) b of the continuation number “69”, and the content (11) b of the pixel data. = (00000000010010111) b is the data unit CU04 * of the compressed video data PXD.

  In other words, according to rule 4, (111111... 1111) b (138 bit length) of the data unit CU04 is converted to (0000000010010111) b (16 bit length) of the data unit CU04 *. In this example, the effective bit length compression is 122 bits from 138 bits to 16 bits. However, if the continuation number is, for example, 255 (510 bits length because 255 of 11 CU01s continue), this is 16 bits. Compression data (CU04 * = 00000011111111111), and a compression effect of 494 bits is obtained with respect to 510 bits. That is, the bit compression effect based on rule 4 is greater than that of rule 3.

  According to rule 5 shown in the fifth column of FIG. 5, when the same pixel continues from the switching point of the encoded data unit to the end of the line, one encoded data unit is composed of 16-bit data. In this case, the first 14 bits represent an encoding header indicating that the rule 5 is used, and the subsequent 2 bits represent pixel data.

  For example, the fifth compressed data unit CU05 of the uncompressed video data PXD shown in the upper part of FIG. 6 includes one or more pieces of 2-bit pixel data d92 to dn = (000000... 0000) b. In this example, the same 2-bit pixel data (00) b continues (continues) in a finite number. However, in rule 5, the number of continuing pixels may be any number of 1 or more.

  In this case, as shown in the lower part of FIG. 6, d92 to dn = (0000000000000) b obtained by connecting the encoded header (00000000000000) b and the content (00) b of the pixel data is the compressed video data PXD. The data unit is CU05 *.

  In other words, according to rule 5, (000000... 0000) b (unspecified bit length) of the data unit CU05 is converted to (0000000000000) b (16 bit length) of the data unit CU05 *. According to Rule 5, if the same pixel continuation number up to the line end is 16 bits or more, a compression effect can be obtained.

  According to rule 6 shown in the sixth column of FIG. 5, the length of the compressed data PXD for one line is not an integer multiple of 8 bits when the pixel line on which the encoding target data is arranged ends (that is, not byte aligned). ), 4-bit dummy data is added so that one line of compressed data PXD is in byte units (that is, byte-aligned).

  For example, the total bit length of the data units CU01 * to CU05 * of the compressed video data PXD shown in the lower part of FIG. 6 is always an integer multiple of 4 bits, but is not necessarily an integer multiple of 8 bits. Not necessarily.

  For example, if the total bit length of the data units CU01 * to CU05 * is 1020 bits and 4 bits are insufficient for byte alignment, as shown in the lower part of FIG. 6, 4-bit dummy data CU06 * = (0000 ) B is added to the end of 1020 bits, and byte-aligned 1024-bit data units CU01 * to CU06 * are output.

  The 2-bit pixel data is not necessarily limited to displaying four types of pixel colors. For example, pixel data (00) b represents a background pixel of a sub-picture, pixel data (01) b represents a sub-picture pattern pixel, pixel data (10) b represents a first emphasized pixel of a sub-picture, and pixel Data (11) b may represent the second emphasized pixel of the sub-picture.

  If the number of constituent bits of the pixel data is larger, other types of sub-picture pixels can be specified. For example, when the pixel data is composed of 3 bits (000) b to (111) b, in the sub-picture data to be run-length encoded / decoded, a maximum of 8 types of pixel colors + pixel types (emphasis effect) are selected. It becomes possible to specify.

  FIG. 7 illustrates the flow from mass production to user-side playback of a high density optical disc with encoded image information (31 + 32 + 33 in FIG. 3); from broadcast / cable distribution of encoded image information to user / It is a block diagram explaining the flow to reception / reproduction in a subscriber.

  For example, when the pre-compression run length data is input to the encoder 200 of FIG. 7, the encoder 200 performs the run length compression (encoding) of the input data by, for example, software processing based on the compression rules 1 to 6 of FIG. The

  When data having a logical configuration as shown in FIG. 2 is recorded on the optical disc OD as shown in FIG. 1, the run-length compression process (encoding process) by the encoder 200 of FIG. 7 is performed on the sub-picture data of FIG. Implemented.

  Various data necessary for completing the optical disc OD is also input to the encoder 200 in FIG. These data are compressed based on, for example, the MPEG (Mortion Picture Expert Group) standard, and the compressed digital data is sent to the laser cutting machine 202 or the modulator / transmitter 210.

  In the laser cutting machine 202, the MPEG compressed data from the encoder 200 is cut on a mother disk (not shown), and the optical disk master 204 is manufactured.

  In the mass production facility 206 for bonding two high-density optical disks, the master information is transferred to a laser light reflecting film on a polycarbonate substrate having a thickness of 0.6 mm, for example, using the master 204 as a model. A large number of two polycarbonate substrates onto which different master information has been transferred are bonded together to form a double-sided optical disc (or a single-sided reading type double-sided disc) having a thickness of 1.2 mm.

  The bonded high-density optical disc OD mass-produced at the facility 206 is distributed in various markets and reaches the user.

  The distributed disc OD is played back by the playback device 300 of the user. The apparatus 300 includes a decoder 101 that restores the data encoded by the encoder 200 to the original information. The information decoded by the decoder 101 is sent to a user's monitor TV, for example, and visualized. Thus, the end user can view the original video information from the mass-distributed disc OD.

  On the other hand, the compressed information sent from the encoder 200 to the modulator / transmitter 210 is modulated in accordance with a predetermined standard and transmitted. For example, the compressed video information from the encoder 200 is satellite broadcast (212) together with the corresponding audio information. Alternatively, the compressed video information from the encoder 200 is cable-transmitted (212) together with the corresponding audio information.

  The compressed video / audio information transmitted by broadcast or cable is received by the receiver / demodulator 400 of the user or subscriber. The receiver / demodulator 400 includes a decoder 101 that restores the data encoded by the encoder 200 to the original information. The information decoded by the decoder 101 is sent to a user's monitor TV, for example, and visualized. In this way, the end user can appreciate the original video information from the compressed video information transmitted by broadcast or cable.

  FIG. 8 is a block diagram showing an embodiment (non-interlaced specification) of decoder hardware that performs image decoding (run-length expansion) according to the present invention. The decoder 101 (see FIG. 7) for decoding the run-length compressed sub-picture data SPD (corresponding to the data 32 in FIG. 3) can be configured as shown in FIG. Sub-picture packets in the pack of sub-picture information to which the SCR of the same time is given are sequentially transferred to the decoder 101, and the SCR is also supplied. The main STC 120a starts counting from a value that becomes zero after a predetermined time. This predetermined time corresponds to the transfer time of one sub-video unit (maximum size) corresponding to the SCR. For example, when the maximum size of one sub-picture unit is 65535 bytes and the transfer rate is 740 nsec / 1 byte, 65535 × 740 / 1000000≈48.5 msec, and the value starts from a value that becomes zero after about 48.5 msec. ing.

  Hereinafter, a sub-picture data decoder that performs run-length expansion on a signal including pixel data subjected to run-length compression in the format shown in FIG. 4 will be described with reference to FIG.

  As shown in FIG. 8, the sub-picture decoder 101 includes a data I / O 102 to which the sub-picture data SPD is input; a memory 108 that stores the sub-picture data SPD; and a memory control that controls a read / write operation of the memory 108 Unit 105; and detecting the continuation code length (encoding header) of one unit (one unit) from the run information of the code data (run-length compressed pixel data) read out from the memory 108, and the continuation code length A continuation code length detection unit 106 that outputs segmentation information; a code data segmentation unit 103 that extracts code data for one unit in accordance with information from the continuation code length detection unit 106; and an output from the code data segmentation unit 103 A signal indicating run information of one compression unit, and a signal output from the continuation code length detection unit 106. Receives a signal (period signal) indicating the number of consecutive “0” bits indicating how many consecutive “0” data bits from the beginning of the code data for one unit, and calculates the number of continuous pixels of one unit from these signals A pixel length output unit 104 (Fast) which receives the pixel color information from the code data segmentation unit 103 and the period signal output from the run length setting unit 107 and outputs the color information only during that period. -in / Fast-out type); a microcomputer 112 that reads header data (see FIG. 4) in the sub-picture data SPD read from the memory 108 and performs various processing settings and control based on the read data; An address control unit 109 for controlling the read / write address of the memory 108; color information for a line for which no run information exists; Insufficient pixel color setter 111 set by 2; in a display valid authorization unit 110 for determining the display area when displaying a sub-picture such as TV screens, is constructed.

  24 to 27, the timer 120 and the buffer 121 are connected to the MPU 112 of the decoder 101. The timer 120 includes a main STC (system timer) 120a and a sub STC (sub timer) 120b.

  The above description will be described again in another way as follows. That is, as shown in FIG. 8, the run-length compressed sub-picture data SPD is sent to the bus inside the decoder 101 via the data I / O 102. The data SPD sent to the bus is sent to the memory 108 via the memory control unit 105 and stored therein. The internal bus of the decoder 101 is connected to the code data segmentation unit 103, the continuation code length detection unit 106, and the microcomputer (MPU or CPU) 112.

  The sub video unit header 31 of the sub video data read from the memory 108 is read by the microcomputer 112. The microcomputer 112 sets a decoding start address (SPDDADR) in the address control unit 109 based on the various parameters shown in FIG. 4 from the read header 31, and displays the display start position and display of the sub-picture in the display valid permission unit 110. Information on the width and display height (SPDSZ) is set, and the display width of the sub-picture (number of dots on the line) is set in the code data segmentation unit 103. Various set information is stored in the internal register of each unit (109, 110, 103). Thereafter, various information stored in the register can be accessed by the microcomputer 112.

  The address control unit 109 accesses the memory 108 via the memory control unit 105 based on the decoding start address (SPDDADR) set in the register, and starts reading sub-picture data to be decoded. The sub-picture data read from the memory 108 in this manner is given to the code data cutout unit 103 and the continuation code length detection unit 106.

  The encoding header (2 to 14 bits in the rules 2 to 5 in FIG. 5) of the run-length compressed sub-picture data SPD is detected by the continuation code length detection unit 106, and the continuation pixel number of the same pixel data in the data SPD is determined. Based on the signal from the continuation code length detection unit 106, the run length setting unit 107 detects it.

  That is, the continuation code length detection unit 106 detects the encoded header (see FIG. 5) by counting the number of “0” bits of data read from the memory 108. The detection unit 106 transmits the segmentation information SEP. To the code data segmentation unit 103 according to the detected value of the encoded header. INFO. give.

  The code data segmentation unit 103 receives the given segmentation information SEP. INFO. Accordingly, the number of continuous pixels (run information) is set in the run length setting unit 107, and the pixel data (SEPARATED DATA; here, the pixel color) is set in the FIFO type pixel color output unit 104. At that time, the code data segmentation unit 103 counts the number of pixels of the sub-picture data, and compares the pixel count value with the display width of the sub-picture (the number of pixels in one line).

  If the byte alignment is not performed when decoding for one line is completed (that is, the data bit length for one line is not a multiple of 8), the code data segmentation unit 103 converts the last 4-bit data on the line. It is considered as dummy data added at the time of encoding and is discarded.

Based on the number of continuous pixels (run information), the pixel dot clock (DOTCLK), and the horizontal / vertical synchronization signal (H-SYNC / V-SYNC), the run length setting unit 107 sends a pixel to the pixel color output unit 104. A signal (PERIOD SIGNAL) for outputting data is given. Then, while the pixel data output signal (PERIOD SIGNAL) is active (that is, during the period during which the same pixel color is output), the pixel color output unit 104 displays the decoded pixel data from the code data segmentation unit 103 as a decoded display. Output as data.

  At this time, if the decoding start line is changed by an instruction from the microcomputer 112, there may be a line without run information. In that case, the deficient pixel color setting unit 111 supplies the deficient pixel color data (COLOR INFO.) Set in advance to the pixel color output unit 104. Then, the pixel color output unit 104 outputs the insufficient pixel color data (COLOR INFO.) From the insufficient pixel color setting unit 111 while the line data without the run information is provided to the code data segmentation unit 103.

  That is, in the case of the decoder 101 of FIG. 8, if there is no image data in the input sub-picture data SPD, the microcomputer 112 sets the insufficient pixel color information in the insufficient pixel color setting unit 111 accordingly. Yes.

  A display enable signal for determining which position on the monitor screen (not shown) to display the decoded sub-video is synchronized with the horizontal / vertical synchronization signal of the sub-video image. Then, it is given from the display activation permission unit (Display Activator) 110. A color switching signal is sent from the permission unit 110 to the output unit 104 based on the color information instruction from the microcomputer 112.

  The address control unit 109 sends address data and various timing signals to the memory control unit 105, the continuation code length detection unit 106, the code data segmentation unit 103, and the run length setting unit 107 after setting the processing by the microcomputer 112. To do.

  When a pack of sub-picture data SPD is taken in via the data I / O unit 102 and stored in the memory 108, the contents of the pack header of this data SPD (decode start address, decode end address, display start position, The display width, display height, etc.) are read by the microcomputer 112. The microcomputer 112 sets a decode start address, a decode end address, a display start position, a display width, a display height, and the like in the display valid permission unit 110 based on the read content.

  Hereinafter, the operation of the decoder 101 in FIG. 8 will be described in the case where the compressed pixel data has a 2-bit configuration (the usage rule is the rules 1 to 6 in FIG. 5).

  When the decoding start address is set by the microcomputer 112, the address control unit 109 sends address data corresponding to the memory control unit 105 and sends a read start signal to the continuation code length detection unit 106.

  The continuation code length detection unit 106 sends a read signal to the memory control unit 105 in response to the read start signal sent to read the encoded data (compressed sub-picture data 32). Then, the detection unit 106 checks whether all the upper 2 bits of the read data are “0”.

  If they are not “0”, it is determined that the block length of the compression unit is 4 bits (see Rule 1 in FIG. 5).

  If they (higher 2 bits) are “0”, the subsequent 2 bits (higher 4 bits) are checked. If they are not “0”, it is determined that the block length of the compression unit is 8 bits (see Rule 2 in FIG. 5).

  If these (upper 4 bits) are “0”, the subsequent 2 bits (upper 6 bits) are checked. If they are not “0”, it is determined that the block length of the compression unit is 12 bits (see Rule 3 in FIG. 5).

  If they (upper 6 bits) are “0”, the following 8 bits (upper 14 bits) are checked. If they are not “0”, it is determined that the block length of the compression unit is 16 bits (see Rule 4 in FIG. 5).

  If these (upper 14 bits) are “0”, it is determined that the block length of the compression unit is 16 bits and that the same pixel data continues to the end of the line (see Rule 5 in FIG. 5).

  If the number of bits of the pixel data read up to the line end is an integer multiple of 8, it is left as it is, and if it is not an integer multiple of 8, 4-bit dummy data is added at the end of the read data in order to realize byte alignment. Is determined to be necessary (see rule 6 in FIG. 5).

  The code data segmentation unit 103 extracts one block (one compression unit) of the sub-picture data 32 from the memory 108 based on the determination result by the continuation code length detection unit 106. Then, in the cutout unit 103, the extracted data for one block is cut into the number of continuous pixels and pixel data (pixel color information and the like). The data (RUN INFO.) Of the separated continuous pixel number is sent to the run length setting unit 107, and the separated pixel data (SEPARATED DATA) is sent to the pixel color output unit 104.

  On the other hand, the display enable permission unit 110 follows the display start position information, the display width information, and the display height information received from the microcomputer 112, the pixel dot clock (PIXEL-DOT CLK), the horizontal synchronization signal ( A display permission signal (enable signal) for designating a sub-picture display period is generated in synchronization with H-SYNC) and the vertical synchronization signal (V-SYNC). This display permission signal is output to the run length setting unit 107.

  The run length setting unit 107 outputs a signal that is output from the continuation code length detection unit 106 and indicates whether or not the current block data continues to the line end, and the continuation pixel data from the code data segmentation unit 103. (RUN INFO.) Is sent. The run length setting unit 107 determines the number of pixel dots that the block being decoded is responsible for based on the signal from the detection unit 106 and the data from the segmentation unit 103, and outputs the pixel color during the period corresponding to the number of dots. The display permission signal (output enable signal) is output to the unit 104.

  The pixel color output unit 104 is enabled during reception of a period signal from the run length setting unit 107, and during that period, the pixel color information received from the code data segmentation unit 103 is synchronized with the pixel dot clock (PIXEL-DOT CLK). Then, the decoded display data is sent to a display device (not shown). That is, the same display data as the number of consecutive pixel patterns of the block being decoded is output from the pixel color output unit 104.

  When the continuation code length detection unit 106 determines that the encoded data is the same pixel color data until the line end, the continuation code length detection unit 106 outputs a signal for the continuation code length 16 bits to the code data segmentation unit 103, and a run length setting unit A signal indicating that the pixel color data is the same up to the line end is output to 107.

  When the run length setting unit 107 receives the signal from the detection unit 106, the pixel color output unit 104 keeps the color information of the encoded data in the enabled state until the horizontal synchronization signal H-SYNC becomes inactive. Output enable signal (period signal).

  When the microcomputer 112 changes the decoding start line in order to scroll the display content of the sub-picture, there is no data line to be used for decoding in the preset display area (that is, the decoding line is insufficient). )there is a possibility.

  In order to cope with such a case, the decoder 101 of FIG. 8 prepares in advance pixel color data for filling in the deficient line. When a line shortage is actually detected, the display mode is switched to a display mode for insufficient pixel color data. More specifically, when the data end signal is given from the address control unit 109 to the display valid permission unit 110, the permission unit 110 sends a color switching signal (COLOR SW SIGNAL) to the pixel color output unit 104. In response to this switching signal, the pixel color output unit 104 switches the decoded output of the pixel color data from the code data to the decoded output of the color information (COLOR INFO.) From the insufficient pixel color setting unit 110. This switching state is maintained during the shortage line display period (DISPLAY ENABLE = active).

  When the line shortage occurs, the decoding processing operation can be stopped during that time instead of using the insufficient pixel color data.

  Specifically, for example, when a data end signal is input from the address control unit 109 to the display valid permission unit 110, a color switching signal designating display stop may be output from the permission unit 110 to the pixel color output unit 104. . Then, the pixel color output unit 104 stops displaying the sub-picture while the display stop designation color switching signal is active.

  FIG. 9 illustrates two examples (non-interlaced display and interlaced display) of how the character pattern “A” is decoded in the encoded pixel data (sub-picture data).

  The decoder 101 in FIG. 8 can be used when decoding compressed data as shown in the upper part of FIG. 9 into non-interlaced display data as shown in the lower left part of FIG.

  On the other hand, when the compressed data as shown in the upper part of FIG. 9 is decoded into the interlaced display data as shown in the lower right part of FIG. 9, a line doubler that scans the same pixel line twice (for example, line # of the odd field) Line # 10 having the same content as 1 is rescanned in the even field; V-SYNC unit switching is required.

  When non-interlaced display of an image display amount equivalent to interlaced display is performed, another indoubler (for example, line # 10 having the same contents as line # 1 at the lower right of FIG. 9 is continued to line # 1; SYNC unit switching) is required.

FIG. 10 is a flowchart for explaining the software executed by the microcomputer 112 of FIG. 11 or FIG. 12, for example, for executing image decoding (run length expansion) according to an embodiment of the present invention. (Here, the decoding of the display control sequence table DCSQT33 of FIG. 3 is not described. Decoding of the DCSQT33 portion will be described later with reference to FIGS. 25 to 27.)
FIG. 11 is a flowchart for explaining an example of the contents of the decoding step (ST105) used in the software of FIG.

  That is, the microcomputer 112 reads the first header 31 portion of the run-length compressed sub-picture data (pixel data is composed of 2 bits) and analyzes the contents (see FIG. 4). Then, the number of lines and the number of dots of the image data to be decoded are designated based on the analyzed header contents. When the number of lines and the number of dots are designated (step ST101), the line count number and the dot count number are initialized to “0” (step ST102 to step ST103).

  The microcomputer 112 sequentially takes in the data bit string that follows the sub-picture unit header 31 and counts the number of dots and the number of dot counts. Then, the number of continuous pixels is calculated by subtracting the number of dots from the number of dots (step ST104).

  When the number of continuing pixels is calculated in this way, the microcomputer 112 executes a decoding process according to the value of the number of continuing pixels (step ST105).

  After the decoding process in step ST105, the microcomputer 112 adds the dot count number and the continuation pixel number to obtain a new dot count number (step ST106).

  Then, the microcomputer 112 sequentially fetches the data and executes the decoding process of step ST105. When the accumulated dot count number matches the initially set line end number (line end position), the data for one line is obtained. The decoding process ends (YES in step ST107).

  Next, if the decoded data is byte-aligned (YES in step ST108A), the dummy data is removed (step ST108B). Then, the line count is incremented by +1 (step ST109), and the processes of step ST102 to step ST109 are repeated until the final line is reached (NO in step ST1010). If the final line is reached (YES in step ST1010), the decoding ends.

  The processing content of the decoding processing step ST105 in FIG. 10 is as shown in FIG. 11, for example.

  In this process, 2 bits are acquired from the beginning, and weaving for determining whether or not the bits are “0” is repeated (steps ST111 to ST119). Thereby, the number of continuous pixels corresponding to the run length compression rules 1 to 6 in FIG. 5, that is, the number of continuous runs is determined (step ST120 to step ST123).

  After the number of consecutive runs is determined, the 2 bits read subsequently are used as a pixel pattern (pixel data; pixel color information) (step ST124).

  When the pixel data (pixel color information) is determined, the index parameter “i” is set to 0 (step ST125), and the 2-bit pixel pattern is output until the parameter “i” matches the number of consecutive runs (step ST126). (Step ST127), the parameter “i” is incremented by +1 (step ST128), the output of one unit of the same pixel data is finished, and the decoding process is finished.

  As described above, according to the sub-picture data decoding method, the sub-picture data can be decoded by simple processes such as a determination process of several bits, a data block separation process, and a data bit counting process. This eliminates the need for a large code table used in the conventional MH encoding method, and simplifies the processing and configuration for decoding the encoded bit data into the original pixel information.

  In the above embodiment, the code bit length for one unit of the same pixel can be determined by reading bit data of up to 16 bits at the time of data decoding. However, the code bit length is not limited to this. For example, the code bit length may be 32 bits or 64 bits. However, if the bit length increases, a data buffer with a larger capacity is required.

  In the above embodiment, the pixel data (pixel color information) is, for example, color information of three colors selected from a 16-color palette, but in addition to this, the three primary colors (red component R, green) The amplitude information of the component G, the blue component B; or the luminance signal component Y, the chroma red signal component Cr, the chroma blue signal component Cb, etc.) can also be expressed by 2-bit pixel data. That is, the pixel data is not limited to a specific type of color information.

  FIG. 12 is a block diagram illustrating an outline of an optical disc recording / reproducing apparatus in which encoding (SPUH + PXD + DCSQT encoding in FIG. 3) and decoding (SPUH + PXD + DCSQT decoding) are performed.

  In FIG. 12, an optical disc player 300 basically has the same configuration as a conventional optical disc playback apparatus (compact disc player or laser disc player). However, the optical disc player 300 uses a digital signal (encode) before decoding the run-length compressed image information from the inserted optical disc OD (recording information containing run-length compressed sub-picture data). The digital signal can be output as is. Since the encoded digital signal is compressed, the required transmission bandwidth may be less than that required when transmitting uncompressed data.

  The compressed digital signal from the optical disc player 300 is turned on air via the modulator / transmitter 210 or sent to a communication cable.

  On-air compressed digital signals or cable transmitted compressed digital signals are received by the receiver / demodulator 400 of the receiver or cable subscriber. The receiver 400 includes a decoder 101 configured as shown in FIG. 8, for example. The decoder 101 of the receiver 400 decodes the received and demodulated compressed digital signal, and outputs image information including original sub-picture data before being encoded.

  In the configuration of FIG. 12, if the transmission / reception transmission system has an average bit rate of about 5 Mbit / s or more, high-quality multimedia video / audio information can be broadcast.

  FIG. 13 is a block diagram illustrating a case where image information encoded according to the present invention is transmitted and received between any two computer users via a communication network (such as the Internet).

  A user # 1 having self-information # 1 managed by a host computer (not shown) has a personal computer 5001, and various input / output devices 5011 and various external storage devices 5021 are connected to the personal computer 5001. Yes. An internal slot (not shown) of the personal computer 5001 is equipped with a modem card 5031 having an encoder and decoder according to the present invention and having a function necessary for communication.

  Similarly, a user #N having another self-information #N owns a personal computer 500N, and various input / output devices 501N and various external storage devices 502N are connected to the personal computer 500N. In addition, an internal slot (not shown) of the personal computer 500N is equipped with a modem card 503N in which an encoder and a decoder based on the present invention are incorporated and functions necessary for communication.

  Assume that a certain user # 1 operates a computer 5001 to communicate with a computer 500N of another user #N via a line 600 such as the Internet. In this case, both the user # 1 and the user #N have modem cards 5031 and 503N in which an encoder and a decoder are incorporated, so that the image data compressed efficiently according to the present invention can be exchanged in a short time.

  FIG. 24 shows an outline of a recording / reproducing apparatus that records encoded image information (SPUH + PXD + DCSQT in FIG. 3) on the optical disc OD and decodes the recorded information (SPUH + PXD + DCSQT) based on the present invention.

  The encoder 200 of FIG. 24 performs the same encoding process (the process corresponding to FIGS. 13 to 14) as the encoder 200 of FIG. 7 by software or hardware (including firmware or a wired logic circuit). It is configured.

  The recording signal including the sub-picture data encoded by the encoder 200 is subjected to, for example, (2, 7) RLL modulation in the modulator / laser driver 702. The modulated recording signal is sent from the laser driver 702 to the high power laser diode of the optical head 704. A pattern corresponding to the recording signal is written on the magneto-optical recording disk or the phase change optical disk OD by the recording laser from the optical head 704.

  Information written to the disk OD is read by the laser pickup of the optical head 706, demodulated by the demodulator / error correction unit 708, and subjected to error correction processing as necessary. The demodulated and error-corrected signal is subjected to various data processing in the audio / video data processing unit 710, and information before recording is reproduced.

  The data processing unit 710 includes a decoding processing unit corresponding to the decoder 101 in FIG. By this decoding processing unit, decoding processing (decompression of compressed sub-video data) corresponding to FIGS. 10 to 11 is executed.

  Here, the sub-picture data shown in FIG. 2 or 3 is composed of a plurality of channels as shown in FIG. The sub video data unit is composed of a plurality of sub video data packets of channels arbitrarily selected from the plurality of channels. The sub-picture here has information such as characters or figures, is reproduced simultaneously with the video data and the audio data, and is superimposed on the video data reproduction screen.

  FIG. 16 shows the data structure of the sub-video packet. As shown in FIG. 16, the sub-picture packet data includes a packet header 3, a sub-picture header 31, a sub-picture data 32, and a display control sequence table 33.

  In the packet header 3, a time at which the playback system should start display control of the sub-picture data unit is recorded as a presentation time stamp (PTS). However, this PTS is recorded only in the header 3 of the head sub-picture data packet in each sub-picture data unit (Y, W) as shown in FIG. In this PTS, values in the sub-picture data units are described in a plurality of sub-picture data units reproduced at a predetermined reproduction time SCR.

  FIG. 18 shows the serial arrangement state (n, n + 1) of sub-video units (see 30 in FIG. 3) composed of one or more sub-video packets and the time described in the packet header of one unit (n + 1) of them. The stamp PTS and the display control state of the unit (n + 1) corresponding to the PTS (display clear of the previous sub-picture and designation of the display control sequence of the sub-picture to be displayed) are illustrated.

  In the sub-picture header 31, the size of the sub-picture data packet (2-byte SPCSZ) and the recording start position (2-byte SPDCSQTA) of the display control sequence table 33 in the packet are recorded.

  The display control sequence table 33 includes a sub-picture display control time stamp (SPDCTS) indicating the display start time / display end time of video data, and sub-picture data (PXD) 32 to be displayed. One or more pieces of display control sequence information (DCSQT; Display Control Sequence Table) having a recording position (SPNDCSQA; Sub-Picture Next Display Control Sequence Address) and a sub-picture data display control command (COMMAND) as one group are recorded. .

  Here, the time stamp PTS in the packet header 3 is defined by a relative time from a reference time (SCR: System Clock Reference) through reproduction of the entire file, such as the reproduction start time at the beginning of the file (FIG. 2). Has been. This SCR is described in a pack header given before the packet header 3. On the other hand, each time stamp SPDCTS in the display control sequence table 33 is defined by a relative time from the PTS.

  “0” is described in the sub-picture display control time stamp SPDCTS of the sequence to be displayed first.

  Next, the time stamp PTS process of the sub video data packet in the reproduction system will be described. Here, it is assumed that this PTS process is executed in a sub-picture processor (for example, MPU 112 in FIG. 8 and its peripheral circuit) in the reproduction system.

  FIG. 15 is a diagram for explaining how the buffering state of a sub-video data unit changes depending on a sub-video channel having a time stamp PTS when sub-video data is decoded.

  (1) A sub-video processor (FIG. 8 and others) decodes a sub-video data packet of a channel selected in advance from sub-video data packets sent from the outside (such as an optical disk or a broadcasting station), and the packet Check if there is a PTS in the box.

  For example, when a PTS exists as indicated by channel * 4f in FIG. 15, the PTS is separated from the packet header 3. Thereafter, as shown in FIG. 17, for example, PTS is added to the head of the sub-picture data, and the sub-picture data with the PTS header is buffered (stored) in the sub-picture buffer (for example, the buffer 121 in FIG. 8).

  The graph of FIG. 15 illustrates a state in which the buffering amount to the sub video buffer 121 is accumulated as the sub video data packet of the channel with PTS * 4f is buffered.

  (2) After the system reset, the sub-picture processor is in the vertical blanking period immediately after receiving the first packet including the PTS (during the switching period from one display screen frame / field to the next display screen frame / field) ), And the acquired PTS is compared with the count value of the main STC 120a (a counter in the sub-picture processor, for example, a part of the timer 120 in FIG. 8) as a reference time counter. The main STC 120a measures the elapsed time from the reference time SCR through the reproduction of the entire file, such as the reproduction start time of the beginning of the file, but starts from a value that becomes zero after the predetermined time described above. ing. Thus, “0” can be described in the PTS (attached to the first packet of the sub-video unit) without considering the transfer time of the sub-video unit.

  (3) If the count value of the STC 120a is larger than the PTS as a result of the comparison between the PTS and the count value of the STC 120a, the sub-picture data is immediately displayed. On the other hand, when the count value of the STC 120a is smaller than the PTS, no processing is performed. This comparison is performed again during the next vertical blanking period.

  (4) When sub-picture data processing is started, the first sub-picture display control time stamp SPDCTS recorded in the display control sequence table 33 in the sub-picture data packet during the same vertical blanking period is It is compared with the count value of the sub reference time counter (sub STC) in the processor. This sub-STC is composed of a sub-standard time counter (for example, the other part of the timer 120 in FIG. 8) sub-STC 120b in the sub-video processor that measures the elapsed time from the reproduction start time of the sub-video data unit. Accordingly, every time the display is switched to the next (subsequent) sub-picture data unit, all the bits are cleared to “0”, and then the sub STC 120b starts incrementing (time counting) again.

  (5) As a result of comparison between the count value of the sub STC 120b and the sub-picture display control time stamp SPDCTS, if the count value of the sub STC 120b is larger than SPDCTS, the control data of the first display control sequence in the display control sequence table 33 ( DCSQT; for example, DCSQT0 in FIG. 16) is immediately executed, and the sub-picture display process is started.

  (6) Once the display process is started, the PTS added to the first packet of the sub-picture data unit next to the currently displayed sub-picture data unit is read every vertical blanking period. The read PTS is compared with the count value of the main STC 120a.

  As a result of this comparison, if the count value of the main STC 120a is larger than the PTS, the channel pointer in FIG. 16 is set to the address value of the PTS of the next sub-picture data unit, and the sub-picture data unit to be processed becomes the next one. Can be switched. For example, taking FIG. 17 as an example, the sub-picture data unit Y is switched to the next sub-picture data unit W by changing the setting of the channel pointer. At this time, since the data of the sub video data unit Y is no longer necessary, an empty area of the size of the sub video data unit Y is generated in the sub video buffer (for example, memory 108 in FIG. 8). Therefore, a new sub-video data packet can be transferred to this empty area.

  As a result, the buffering state of the sub video data packet (see FIG. 15) is determined from the size of the sub video data unit (for example, unit W in FIG. 17) and the switching time (the switching time from unit Y to unit W). , At the time of encoding the sub-picture data (of unit W), it can be uniquely defined in advance. Therefore, when video / audio / sub-video packets are serially transferred, a bit stream that does not cause overflow or underflow in the buffer of each decoder section (in the case of a sub-video decoder, other memory 108 in FIG. 8). Generation is possible.

  Further, as a result of the comparison between the PTS and the count value of the main STC 120a, if the count value of the main STC 120a is not larger than the PTS, the sub-picture data unit is not switched, and the display control sequence table pointer (DCSQT pointer in FIG. ) Is set to the address value of the next display control sequence table DCSQT. Then, the sub-picture display control time stamp SPDCTS of the next DCSQT in the current sub-picture data packet is compared with the count value of the sub STC 120b. Whether or not to execute the next DCSQT is determined based on the comparison result. This operation will be described in detail later.

  Since the last DCSQT in the sub-picture data packet indicates itself as the next display control sequence table DCSQT, the DCSQT process (5) is basically the same.

  (7) In normal playback, the processes (4), (5), and (6) are repeated.

  In the process of (6), when reading the PTS of the next sub-picture data unit, the value of the channel pointer (see FIG. 16) indicating the PTS is the packet size (SPCSZ) in the current sub-picture data unit. It is calculated by using.

  Similarly, the value of the DCSQT pointer indicating the sub-picture display control time stamp SPDCTS of the next DCSQT in the display control sequence table 33 is the size information of the DCSQT described in the table 33 (the next sub-picture display control sequence). Address SPNDCSQTA).

  Next, details of each of the sub-picture header 31, the sub-picture data 32, and the display control sequence table 33 will be described.

  FIG. 19 shows the structure of the sub-picture unit header (SPUH) 31. The sub-picture unit header SPUH includes the size (SPDSZ) of the sub-picture data packet and the recording start position information of the display control sequence table 33 in the packet (sub-picture display control sequence table start address SPDCSQTA; DCSQ relative address pointer). It is out.

  The contents of the sub-picture display control sequence table SPDCSQT indicated by the address SPDCSQTA are composed of a plurality of display control sequences DCSQ1 to DCSQn as shown in FIG.

  Further, as shown in FIG. 21, each display control sequence DCSQ (1 to n) includes a sub-picture display control time stamp SPDCTS indicating the sub-picture display control start time and an address SPNDCSQA indicating the position of the next display control sequence. And one or more sub-picture display control commands SPDCCMD.

  The sub-picture data 32 is composed of a collection of data areas (PXD areas) that correspond one-to-one with individual sub-picture data packets.

  Here, the sub-picture pixel data PXD at an arbitrary address in the same data area can be read out until the sub-picture data unit is switched. As a result, any sub-picture display (for example, sub-picture scroll display) that is not fixed to one sub-picture display image can be performed. This arbitrary address is set by a command (SETDSPXA in the command table of FIG. 22) for setting the display start address of the sub-picture data (pixel data PXD).

  FIG. 23 shows a specific example of the display control sequence table 33. As described above, one display control sequence information (DCSQT) in the display control sequence table 33 includes a plurality of display control commands after the sub-picture display control time stamp (SPDCTS) and the sub-picture data recording position (SPNDCSQA). (COMMAND3, COMMAND4, etc.) and various parameter data set by the command are arranged. An end command (end code) indicating the end of display control is added at the end.

  Next, the processing procedure of the display control sequence table 33 will be described.

  (1) First, the time stamp (SPDCTS) recorded in the first DCSQT (DCSQT0 in FIG. 16) of the display control sequence table 33 is the sub-STC 120b (for example, one function of the timer 120 in FIG. 8) of the sub-picture processor. Compared with the count value.

  (2) If the count value of the sub STC 120b is larger than the time stamp SPDCTS as a result of the comparison, all display control commands COMMAND in the display control sequence table 33 are displayed until the display control end command CMDEND (FIG. 22) appears. Executed.

  (3) After the display control is started, the sub-picture display control time stamp SPDCTS and the count value of the sub STC 120b recorded in the next display control sequence table DCSQT at regular time intervals (for example, every vertical blanking period). To determine whether to update to the next DCSQT (that is, whether to move the DCSQT pointer in FIG. 16 to the next DCSQT).

Here, the time stamp SPDCTS in the display control sequence table 33
Is recorded in the relative time after the PTS is updated (that is, after the sub-picture data unit is updated), so that it is not necessary to rewrite the SPDCTS even if the PTS of the sub-picture data unit changes. Therefore, the same display control sequence table DCSQT can be used even when the same sub-picture data 32 is displayed at a plurality of different times. That is, the display control sequence table DCSQT can be made relocatable.

  Next, details of the sub-video display control command will be described. FIG. 22 shows a list of sub-picture display control commands SPDCCMD. The main sub-picture display control commands are as follows.

(1) Command STADSP that sets the display start timing of sub-picture pixel data
This is a command for executing display start control of the sub-video data 32. That is, when switching from a certain DCSQT to a DCSQT including this command STADSP, display of the sub-picture data 32 is started from the time indicated by the time stamp SPDCTS of the DCSQT including this command.

  When the sub-picture processor (for example, the MPU 112 in FIG. 8) decodes this command, the sub-picture processor immediately starts (because the time indicated by the SPDCTS of the DCSQT to which this command belongs has passed). The enable bit of the display control system is activated.

(2) Command STPDSP for setting display end timing of sub-picture pixel data
This is a command for executing display end control of the sub-video data 32. When the sub-picture processor decodes this command, the enable bit of the display control system inside the sub-picture processor immediately (because the time indicated by the SPDCTS of the DCSQT to which this command belongs when the command is accessed). To the active state.

(3) Command SETCOLOR for setting the color code of sub-picture pixel data
This is a command for setting the color code of the sub-picture pixel data. By this command, the sub-picture becomes a pattern pixel such as a character or a pattern, an emphasis pixel such as a trimming of the pattern pixel, and a background pixel that is a pixel other than the pattern pixel and the emphasis pixel in the area where the sub-picture is displayed Color information can be set separately.

(4) Command SETCONTR for setting the contrast of sub-picture pixel data with respect to the main picture
Similar to the command SETCOLOR, this is a command for setting contrast data instead of color code data for the four types of pixels exemplified in FIG.

(5) Command SETDAREA for setting the display area of sub-picture pixel data on the main picture
This is a command for designating the position where the sub-picture pixel data 32 is displayed.

(6) Command SETDSPXA for setting display start address of sub-picture pixel data
This is a command for setting the display start address of the sub-picture pixel data 32.

(7) Command CHGCOLLON for setting the color code of sub-picture pixel data and the contrast of sub-picture pixel data with respect to the main picture
This is a command for changing the color code of the sub-picture pixel data 32 and the contrast of the sub-picture pixel data 32 with respect to the main picture during display.

  Note that the command table of FIG. 22 includes, in addition to the above-described commands, a command FSTADSP for forcibly setting the display start timing of sub-picture pixel data, and a command CMDEND for ending sub-picture display control.

  FIG. 24 is a flowchart for explaining an example of a method for generating the sub-picture unit 30 as shown in FIG.

  For example, when subtitles and / or images corresponding to the lines of video (main video) are used as sub-pictures, the line subtitles / images are converted into bitmap data (step ST10). When creating this bitmap data, it is necessary to determine in which region and in which position of the video screen the subtitle portion is to be displayed. For this purpose, the parameter of the display control command SETDAREA (see FIG. 22) is determined (step ST12).

  When the display position (spatial parameter) of the sub video is determined, the process proceeds to encoding of pixel data PXD constituting the sub video (not encoding the entire main video). At that time, the color of the subtitle (sub-video), the background color of the subtitle area, and the mixing ratio of the subtitle color / background color to the video main video are determined. For this purpose, parameters of display control commands SETCOLOR and SETCONTR (see FIG. 22) are determined (step ST14).

  Next, the timing at which the created bitmap data should be displayed in accordance with the video dialogue is determined. This timing is determined by the sub-picture time stamp PTS. At that time, the maximum time of the time stamp PTS and the parameters (temporal parameters) of the display control commands STADSP, STPDSP, and CHGCOLCON (see FIG. 22) are determined (step ST16).

  Here, the sub-video time stamp PTS is finally determined from the consumption model of the target decoder buffer of the MPEG2 system layer. Here, the time at which subtitle display is started is determined as the maximum time of the sub-picture time stamp PTS.

  The display control commands STADSP and STPDSP are recorded as relative times from the sub-picture time stamp PTS. Therefore, the commands STADSP and STPDSP cannot be determined until the PTS is determined. Therefore, in this embodiment, the absolute time is determined, and the relative value is determined after the absolute time of the PTS is determined.

  Further, when it is desired to change the display color and display area spatially and temporally with respect to the created caption, the parameter of the command CHGCOLLON based on the change is determined.

  When the display position (spatial parameter) and display timing (temporal parameter) of the sub-picture are determined (tentatively), the contents (DCSQ) of the sub-picture display control sequence table DCSQT are created (step ST18). Specifically, the value of the display control start time SPDCTS (see FIG. 21) of the display control sequence table DCSQ is the effective time of the display control command STADSP (display start timing) and the effective time of the display control command STPDSP (display end timing). Determined in accordance with

  When the created pixel data PXD32 and the display control sequence table DCSQT33 are combined, the size of the sub-picture data unit 30 (see FIG. 3) can be determined. Therefore, the sub-picture unit header SPUH31 is determined by determining the parameters SPDSZ (sub-picture size; see FIG. 19) and SPDCSQTA (start address of the display control sequence table; see FIG. 19) of the sub-picture unit header SPUH31. To do. Thereafter, by combining SPUH 31, PXD 32, and DCSQT 33, a sub-picture unit for one caption is created (step ST20).

  When the size of the created sub-picture unit 30 exceeds a predetermined value (2048 bytes or 2 kbytes) (Yes in step ST22), the sub-picture unit 30 is divided into a plurality of packets in units of 2 kbytes (step ST24). In this case, the time stamp PTS is recorded only in the packet at the head of the sub-video unit 30 (step ST26).

  When the size of the created sub-video unit 30 is within a predetermined value (2 kbytes) (No in step ST22), only one packet is generated (step ST23), and the time stamp PTS is recorded at the head of the packet. (Step ST26).

  One or more packets thus completed are packed and combined with video and other packs to complete one data stream (step ST28). At this time, the order of the packs is determined based on the sequence recording code SRC and the sub-picture time stamp PTS from the consumption model of the target decoder buffer of the MPEG2 system layer. Here, the PTS is determined for the first time, and thereby each parameter (such as SPDCTS) in FIG. 21 is finally determined.

  FIG. 25 is a flowchart for explaining an example of a procedure for performing parallel processing on pack decomposition and decoding of the sub-picture data stream generated according to the processing procedure of FIG.

  First, the decoding system reads the ID of the transferred stream, and transfers only the selected sub-picture pack (separated from the data stream) to the sub-picture decoder (for example, the sub-picture decoder 101 in FIG. 8). (Step ST40).

  When the first pack transfer is performed, the index parameter “i” is set to “1” (step ST42), and the first sub-picture pack disassembly process (step ST44; described later with reference to FIG. 26) is executed. Is done.

  The disassembled pack (including compressed sub-picture data PXD as shown in the lower part of FIG. 6) is temporarily stored in the sub-picture buffer (memory 108 in FIG. 8) (step ST46), and the index parameter “i” is 1. Is incremented by one (step ST50).

  If there is an incremented i-th pack, that is, if the pack decomposed in step ST44 is not the final pack (NO in step ST52), the decomposition process for the incremented i-th sub-picture pack (step ST44) is executed. Is done.

  The disassembled i-th sub-picture pack (here, the second pack) is temporarily stored in the sub-picture buffer (memory 108) in the same manner as the first disassembled pack (step ST46), and the index parameter “i”. Is further incremented by one (step ST50).

  As described above, a plurality of sub-picture packs are successively decomposed while incrementing the index parameter “i” (step ST44) and stored in the sub-picture buffer (memory 108) (step ST46).

  If the continuously incremented i-th pack does not exist, that is, if the pack decomposed in step ST44 is the final pack (YES in step ST52), the sub-picture pack decomposition processing of the stream to be decoded is completed. .

  While the sub-picture pack disassembly process (steps ST44 to ST52) is being executed continuously, the sub-picture buffer temporarily stored in the sub-picture buffer (memory 108) is executed independently and in parallel with the sub-picture pack disassembly process. The video pack is decoded.

  That is, when the index parameter “j” is set to “1” (step ST60), an operation of reading the first sub-picture pack from the sub-picture buffer (memory 108) is started (step ST62). At this time, if the first sub-picture pack is not yet stored in the memory 108 (NO in step ST63; the process in step ST46 is not yet performed), the pack data to be read is stored in the memory 108. Until the decoding process, the empty loop (steps ST62 to ST63) of the pack reading operation is executed.

  If the first sub-picture pack is stored in memory 108 (YES in step ST63), the sub-picture pack is read and decoded (step ST64; specific examples of the decoding process are shown in FIGS. 24 to 27). Reference later).

  The result of this decoding process (for example, including the sub-picture data PXD before compression as shown in the upper part of FIG. 6) is sent from the sub-picture decoder 101 of FIG. 8 to the display system (not shown) during the decoding process. A sub-picture corresponding to the data is displayed.

  If the display control end command (CMDEND in FIG. 22) is not executed in the decoding process (NO in step ST66), the index parameter “j” is incremented by one (step ST67).

  If the incremented j-th pack (here, the second pack) exists in the memory 108, the pack is read from the memory 108 and decoded (step ST64). The decoded jth sub-picture pack (here, the second pack) is sent to the display system in the same manner as the first decoded pack, and the index parameter “j” is further incremented by one (step ST67). ).

  As described above, while incrementing the index parameter “j” (step ST67), one or more sub-picture packs stored in the memory 108 are successively decoded (step ST64), and the sub-picture data thus decoded is decoded. Sub-picture image display corresponding to (PXD) is executed.

  If the display control end command (CMDEND in FIG. 22) is executed in the decoding process (Yes in step ST66), the decoding process of the sub-picture data in the sub-picture buffer (memory 108) is finished.

  The above decoding process (steps ST62 to ST64) is repeated unless the end command CMDEND is executed (NO in step ST66). In this embodiment, the decoding process is ended when the end command CMDEND is executed (YES in step ST66).

  FIG. 26 is a flowchart for explaining an example of the pack disassembly process of FIG. The sub-picture decoder 101 skips the pack header (see FIG. 3) from the transferred pack and obtains a packet (step ST72). If this packet does not have a time stamp PTS (NO in step ST74), the packet header (PH) is deleted, and only the sub-picture unit data (PXD) is stored in the buffer (eg 121) of the sub-picture decoder (step 121). ST76).

  If the packet has a time stamp PTS (YES in step ST74), only the PTS is extracted from the packet header (PH), and the extracted PTS is connected to the sub-picture unit data (30). It is stored in the buffer 121 (step ST78).

  27 and 28 are flowcharts for explaining an example of the sub-picture decoding process of FIG. Among the decoding processes, for example, the process of returning the compressed data PXD shown in the lower part of FIG. 6 to the uncompressed data PXD shown in the upper part of FIG. 6 has been described with reference to FIGS.

  That is, sub-video packets in a pack of sub-video information corresponding to a predetermined playback time, that is, to which the SCR of the same time is given, are sequentially transferred to the sub-video decoder 101 for one sub-video unit, and according to the SCR The sub video decoder 101 starts from a value at which the count of the main STC 120a becomes zero after a predetermined time, and checks whether or not the PTS added to the head packet of the sub video data unit is transferred (step ST80). As a result, when the PTS is transferred, the sub-picture decoder 101 compares the PTS with the count value of the main STC 120a (step ST81), and when the PTS is not transferred, waits until the PTS is transferred. ing.

  As a result of the comparison, when the count value of the main STC 120a is larger than the PTS, the sub-picture decoder 101 checks whether or not the PTS added to the first packet of the next sub-picture data unit is transferred (step ST82). ) When the count value of the main STC 120a is smaller than PTS, it waits until it becomes the same.

  As a result, when the PTS added to the head packet of the next sub-picture data unit is transferred, the sub-picture decoder 101 compares the PTS with the count value of the main STC 120a (step ST83).

  If the PTS of the next sub-picture data unit is not transferred in step 82 or if the count value of the main STC 120a is smaller than the PTS of the next sub-picture data unit in step 83, the sub-picture decoder 101 decodes the sub-picture decoder 101 to decode. The video data unit is determined to be current (step ST84), and the process proceeds to step 86.

  That is, since the PTS of the first sub-picture data unit is “0”, the sub-picture data unit to be decoded is determined when the count value of the main STC 120 a becomes “0”, and the process proceeds to step 86.

  If the count value of the main STC 120a is larger than the PTS of the next sub video data unit in step 83, the sub video decoder 101 determines the next sub video data unit to be decoded (step ST85), and proceeds to step 86. .

  Next, the sub-picture decoder 101 checks whether or not the time stamp SPDCTS (“0”) recorded in the first DCSQT (DCSQT0 in FIG. 16) of the display control sequence table 33 has been transferred (step ST86). . When the time stamp SPDCTS is transferred, the sub-picture decoder 101 compares the time stamp SPDCTS with the count value of the sub STC 120b (step ST87), and when not transferred, returns to step 81.

  If the count value of the sub STC 120b is larger than the time stamp SPDCTS as a result of the comparison in step 87, the sub-picture decoder 101 checks whether or not the time stamp SPDCTS of the next display control sequence DCSQ is transferred (step). ST88). As a result, when the time stamp SPDCTS is transferred, the sub-picture decoder 101 compares the time stamp SPDCTS with the count value of the sub STC 120b (step ST89).

  If the time stamp SPDCTS of the next display control sequence DCSQ is not transferred in step 88, or if the count value of the sub STC 120b is smaller than the time stamp SPDCTS of the next display control sequence DCSQ in step 89, the sub-picture decoder 101 The display control sequence DCSQ to be decoded is determined to be the current one (step ST90), and the process proceeds to step 92.

  If the count value of the sub STC 120b is larger than the time stamp SPDCTS of the next display control sequence DCSQ at step 89, the sub-picture decoder 101 determines the display control sequence DCSQ to be decoded as the next one (step ST91), and step 92. Proceed to

  Next, sub-picture decoder 101 starts decoding the pixel data corresponding to the display control sequence DCSQ to be decoded using each command (step ST92).

  For example, the display position and display area of the sub video are set by the command SETDAREA, the display color of the sub video is set by the command SETCOLOR, and the contrast of the sub video with respect to the video main video is set by the command SETCONTR.

  The pixel data subjected to run-length compression while performing display control in accordance with the switching command CHGCOLCON until the display end timing command STPDSP is executed in another display control sequence DCSQ after the display start timing command STADSP is executed. PXD (32) is decoded.

  Further, sub-picture decoder 101 checks whether or not all display control commands COMMAND corresponding to display control sequence DCSQ have been transferred until display control end command CMDEND (step ST93).

  As a result, the sub-picture decoder 101 clears the count value of the sub STC 120b when it is not transferred, returns to step 81, and again performs decoding for the first display control sequence DCSQ. Proceed to 94.

  In step 94, the sub-picture decoder 101 checks whether or not there is a next display control sequence DCSQ. If there is a next display control sequence DCSQ, the sub-picture decoder 101 returns to step 88 to perform decoding for the next display control sequence DCSQ. If there is no next display control sequence DCSQ, the process returns to step 82 and the process for the next sub-picture data unit is performed.

  As described above, since the initial value of the main timer is “0” after a predetermined time in consideration of the transfer time of the pixel data, the presentation time stamp PTS in the first sub-picture data unit to be reproduced is set. Zero can be described without considering the transfer time of pixel data.

  In addition, since the display of the sub-picture is prohibited until the main timer first changes to zero, the function of distinguishing between the negative value and the positive value of the main timer can be omitted.

  Further, when the decoding process is interrupted in the middle of the process for the desired display control sequence DCSQ, the decoding process is performed again from the first display control sequence DCSQ0, so that the display is lost due to the decoding process being interrupted. Can be prevented from being disturbed.

  In the above example, the initial value of the main timer is set to “0” after a predetermined time in consideration of the transfer time of the pixel data. However, the present invention is not limited to this, and the initial value of the sub timer is set to the transfer time of the pixel data. In consideration, the value may be “0” after a predetermined time. In this case, the same effect as described above can be achieved.

The figure which shows schematically the recording data structure of the optical disk as an example of the information holding medium which can apply this invention. The figure which illustrates the logical structure of the data recorded on the optical disk of FIG. FIG. 3 is a diagram illustrating a logical structure of a sub-picture pack to be encoded (run length compression, addition of a display control sequence table, etc.) in the data structure illustrated in FIG. 2. The figure which illustrates the content of the sub video data part to which the encoding method based on one Embodiment of this invention is applied among the sub video packs illustrated in FIG. When the pixel data constituting the sub-picture data portion illustrated in FIG. 4 is composed of a plurality of bits (here, 2 bits), compression rules 1 to 6 employed in the encoding method according to the embodiment of the present invention. FIG. The compression rules 1 to 6 employed in the encoding method according to the embodiment of the present invention when the pixel data constituting the sub-picture data portion illustrated in FIG. Figure. Explains the flow from mass production to user-side playback of high-density optical discs with encoded image information; shows the flow of encoded image information from broadcast / cable distribution to reception / playback at the user / subscriber FIG. The block diagram explaining one Embodiment (non-interlace specification) of the decoder hardware which performs the image decoding (run length expansion | extension etc.) based on this invention. FIG. 4 is a diagram for explaining two examples (non-interlaced display and interlaced display) of how the character pattern “A” is decoded in encoded pixel data (sub-picture data). The flowchart figure explaining the software which performs the image decoding (run length expansion | extension part) which concerns on one embodiment of this invention, for example, is performed by MPU (112) of FIG. The flowchart figure explaining an example of the content of the decoding step (ST105) used with the software of FIG. The block diagram explaining the case where the compressed data reproduced | regenerated from the high-density optical disk with the encoded image information is broadcast or cable-distributed as it is, and the compressed data distributed by broadcast or cable is decoded by the user or subscriber side. The block diagram explaining the case where the encoded image information is transmitted / received between arbitrary two computer users via a communication network (Internet etc.). The block diagram explaining the outline | summary of the optical disk recording / reproducing apparatus with which encoding and decoding are performed. The figure explaining how the buffering state of a sub-picture data unit changes with sub-picture channels with a time stamp (PTS) when sub-picture data is decoded according to the present invention. The figure explaining the data structure of a subpicture packet. The figure explaining the position of the time stamp (PTS) in a subpicture data unit. The figure which illustrates the correspondence of the sub-picture unit arranged in series, the time stamp (PTS) described in the packet header of one unit, and the display control sequence (DCSQ). FIG. 5 is a diagram for explaining a sub-picture size and a display control sequence table start address (DCSQ relative address pointer) among parameters included in the sub-picture unit header (SPUH) of FIG. 3 or FIG. 4. The figure explaining the structure of a sub-picture display control sequence table (SPDCSQT). The figure explaining the contents of each parameter (DCSQ) which comprises the table (SPDCSQT) of FIG. The figure explaining the content of the display control command (SPDCCMD) of a subpicture. The figure which shows the specific example of the display control sequence table in the packet shown in FIG. The flowchart figure explaining an example of the sub-picture encoding process sequence of this invention centering on the process of a display control sequence (DCSQ). FIG. 25 is a flowchart for explaining an example of a procedure for performing parallel processing of pack decomposition and decoding of a sub-picture data stream encoded by the processing procedure of FIG. The flowchart figure explaining an example of the pack decomposition | disassembly process of FIG. FIG. 26 is a flowchart for explaining an example of the sub-picture decoding process of FIG. FIG. 26 is a flowchart for explaining an example of the sub-picture decoding process of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... File management information 2 ... Video data PH ... Packet header 30 ... Sub video unit 31 ... Sub video unit header SPUH 32 ... Sub video pixel data PXD 33 ... Display control sequence table DCSQT 101 ... Decoder 102 ... Data I / O DESCRIPTION OF SYMBOLS 103 ... Code data segmentation part 104 ... Pixel color output part (FIFO type) 105 ... Memory control part 106 ... Continuation code length detection part 107 ... Run length setting part 108 ... Memory 109 ... Address control part 110 ... Display effective permission part 111 ... Insufficient pixel color setting unit 112 ... Microcomputer (MPU or CPU) 113 ... Header cutting unit 114 ... Line memory 115 ... Selector 118 ... Select signal generation unit 120 ... System timer 120a ... Main STC 120b ... Sub STC 121 ... Buffer memory

Claims (5)

Sub-picture information that can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a certain data size,
The sub-picture packet includes a packet header, and the packet header includes a time stamp representing a display time;
The sub-picture data unit further includes a sub-picture unit header, run-length compressed sub-picture pixel data, and a display control sequence table.
The sub-picture unit header includes start address information of the display control sequence table,
The run length compression is performed by converting the same pixel data of the sub-picture into a data unit,
The display control sequence table includes a plurality of display control sequences,
One display control sequence includes information indicating a start time related to sub-picture display control, address information, and a plurality of display control command areas.
The address information is configured to indicate the position of the next display control sequence,
As the display control command set in one display control command area, a command for setting the color code of the image data of the sub-picture is possible, and the pattern pixel and the emphasized pixel are set by the command for setting the color code. And color code can be set for background pixels,
A sub-picture encoding method, wherein the sub-picture pixel data is encoded. In the display control sequence, address information indicating the position of the next display control sequence is arranged after the information indicating the start time related to the display control of the sub-picture, and the address information indicating the position of the next display control sequence. The sub-picture encoding method according to claim 1, wherein the one or more display control commands are arranged after the video. In the sub-picture data unit, pixel data of the run-length compressed sub-picture is arranged after the sub-picture unit header, and the display control sequence table is placed after the pixel data of the run-length compressed sub-picture. encoding method of the sub-picture of the arrangement has been that the claims preceding claim. Sub-picture information that can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a certain data size,
The sub-picture packet includes a packet header, and the packet header includes a time stamp representing a display time;
The sub-picture data unit further includes a sub-picture unit header, run-length compressed sub-picture pixel data, and a display control sequence table.
The sub-picture unit header includes start address information of the display control sequence table,
The run length compression is performed by converting the same pixel data of the sub-picture into a data unit,
The display control sequence table includes a plurality of display control sequences,
One display control sequence includes information indicating a start time related to sub-picture display control, address information, and a plurality of display control command areas.
The address information is configured to indicate the position of the next display control sequence,
As the display control command set in one display control command area, a command for setting the color code of the image data of the sub-picture is possible, and the pattern pixel and the emphasized pixel are set by the command for setting the color code. And reproducing the sub-picture from an information recording medium configured so that a color code can be set for a background pixel. Sub-picture information that can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a certain data size,
The sub-picture packet includes a packet header, and the packet header includes a time stamp representing a display time;
The sub-picture data unit further includes a sub-picture unit header, run-length compressed sub-picture pixel data, and a display control sequence table.
The sub-picture unit header includes start address information of the display control sequence table,
The run length compression is performed by converting the same pixel data of the sub-picture into a data unit,
The display control sequence table includes a plurality of display control sequences,
One display control sequence includes information indicating a start time related to sub-picture display control, address information, and a plurality of display control command areas.
The address information is configured to indicate the position of the next display control sequence,
As the display control command set in one display control command area, a command for setting the color code of the image data of the sub-picture is possible, and the pattern pixel and the emphasized pixel are set by the command for setting the color code. And a decoder for the sub-picture information configured so that a color code can be set for the background pixels.

Images