JP2004328780A - Encoding method, decoding method and decoding device of image information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable using completely the same display control sequence table DCSQT, i.e. making the display control sequence table DCSQT relocatable, even when the same sub-video data 32 are displayed at a plurality of different times. <P>SOLUTION: Since a time stamp SPDCTS in the display control sequence table 33 is recorded at a relative time after PTS is updated (namely, after the sub-video data unit is updated), the SPDCTS is not required to be rewritten even if the PTS of the sub-video data unit changes. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image information encoding method for encoding image information such as sub-images that are reproduced simultaneously with a main image, a reproducing apparatus for decoding and reproducing the image information, and a reproducing method.

  2. Description of the Related Art In recent years, an optical disk playback device for moving images, which plays back an optical disk on which video and audio data are digitally recorded, has been developed, and is widely used as a playback device for movie software, karaoke, and the like. Recently, a data compression method for moving images has been internationally standardized as a Moving Picture Image Coding Expert Group (MPEG) method. The MPEG method is a method of performing variable length compression on video data.

  At present, the MPEG2 system is being internationally standardized, and accordingly, a system format corresponding to the MPEG compression system is also defined as an MPEG2 system layer. In the MPEG2 system layer, it is specified that a transfer start time and a reproduction start time expressed by using a reference time are set for each data so that moving images, audios, and other data can be transferred and reproduced synchronously. I have. Normal reproduction is performed based on these information.

  One or more sub-pictures superimposed on the main picture, such as a movie subtitle or an image displayed as a TV volume setting value, are recorded after being run-length compressed so that they are reproduced in synchronization with the main picture. The sub-picture is reproduced for each sub-picture unit composed of a plurality of packs based on a time stamp given to the sub-picture unit.

  In this case, the time stamp of the corresponding first sub-picture unit cannot be set to zero, and a value in consideration of the transfer time of the pixel data has to be described.

  Further, one sub-picture unit is composed of a plurality of sequences, and is composed of a display control sequence composed of a time stamp as a display start time, a command, and the like, which are given to each sequence in the order of each sequence.

  SUMMARY OF THE INVENTION It is an object of the present invention to provide a display control sequence table that can use the same display control sequence table even when the same sub-picture data is displayed at a plurality of different times, and that the display control sequence table can be made relocatable.

In the method for encoding a sub-picture according to the present invention, the sub-picture information which can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a fixed data size, the sub-picture packet includes a packet header,
The packet header includes a time stamp representing a display time, and the sub-picture data unit further includes a display control sequence table including a display control sequence, pixel data of a run-length compressed sub-picture, and a sub-picture unit header,
The sub-picture unit header includes start address information of the display control sequence table,
The run-length compression is performed by converting a continuation of the same pixel data of the sub-picture into data units,
The display control sequence includes information indicating a start time for display control of a sub-picture, address information of a next display control sequence, one or more sub-picture display control commands, and parameter data set by the sub-picture display control command. ,
As the sub-picture display control command, a command to set a display start timing of the sub-picture pixel data or a command to set a display end timing of the sub-picture pixel data can be set,
The configuration is such that the number of extension fields corresponding to the parameter data set by the command to set the display start timing and the command to set the display end timing is set to 0 bytes in advance,
The pixel data of the sub-picture is encoded.

  As described in detail above, according to the present invention, the same display control sequence table can be used even when the same sub-picture data is displayed at a plurality of different times, and the display control sequence table can be made relocatable. .

  Hereinafter, an image information encoding / decoding system according to an embodiment of the present invention will be described with reference to the drawings. In order to avoid redundant description, common reference numerals are used for functionally common parts in a plurality of drawings.

  FIGS. 1 to 27 are diagrams for explaining an image information encoding / decoding system according to an embodiment of the present invention.

  FIG. 1 schematically shows a recording data structure of an optical disk OD as an example of an information storage medium to which the present invention can be applied.

  The optical disk OD is a double-sided bonded disk having a storage capacity of about 5 GB on one side, for example, and has a large number of recording tracks arranged between a lead-in area on the inner peripheral side of the disk and a lead-out area on the outer peripheral side of the disk. I have. Each track is composed of a number of logical sectors, and various information (digital data appropriately compressed) is stored in each sector.

  FIG. 2 illustrates a data structure of a video file recorded on the optical disc OD in FIG.

  As shown in FIG. 2, the video file includes file management information 1 and video data 2. The video data 2 records a video data unit (block), an audio data unit (block), a sub-picture data unit (block), and information (DSI; Disk Search Information) necessary for controlling the reproduction of these data. DSI units (blocks). Each unit is divided into packets of a fixed data size for each data type, for example. The video data unit, audio data unit and sub-picture data unit are reproduced in synchronization with each other based on the DSI arranged immediately before these unit groups.

  That is, a system area for storing system data used in the disk OD, a volume management information area, and a plurality of file areas are formed in the aggregate of a plurality of logical sectors in FIG.

  Of the plurality of file areas, for example, file 1 includes main video information (VIDEO in the figure), sub-video information (SUB-PICTURE in the figure) having auxiliary contents for the main video, and audio information (FIG. AUDIO), playback information (PLAYBACK INFO. In the figure), and the like.

  FIG. 3 illustrates the logical structure of a pack of encoded (run-length compressed) sub-picture information in the data structure illustrated in FIG.

  As shown in the upper part of FIG. 3, one pack of the sub-picture information included in the video data is composed of, for example, 2048 bytes (2 kB). One pack of the sub-picture information includes one or more sub-picture packets after the header of the first pack. The pack header is provided with a reference time (SCR; System Clock Reference) through reproduction of the entire file. Sub-picture packets in a pack of sub-picture information to which the SCR of the same time is added will be described later. The data is transferred to the decoder. The first sub-picture packet includes run-length compressed sub-picture data (SP DATA1) after the header of the packet. Similarly, the second sub-picture packet includes run-length compressed sub-picture data (SP DATA2) after the header of the packet.

  A plurality of such sub-picture data (SP DATA1, SP DATA2,...) Are collected for one unit (one unit) of run-length compression, that is, a sub-picture data unit 30 is provided with a sub-picture unit header 31. . After the sub-picture unit header 31, pixel data 32 obtained by run-length-compressing video data of one unit (for example, data of one horizontal line of a two-dimensional display screen) and display control sequence information of each sub-picture pack are stored. A table 33 including the following follows.

  In other words, the run-length compressed data 30 for one unit is formed by a set of sub-picture data portions (SP DATA1, SP DATA2,...) Of one or more sub-picture packets. The sub-picture data unit 30 includes a sub-picture unit header SPUH31 in which various parameters for displaying a sub-picture are recorded, display data (compressed pixel data) PXD32 composed of a run-length code, a display control sequence table DCSQT33, It is composed of

  FIG. 4 illustrates a part of the content of the sub-picture unit header 31 in the run-length compressed data 30 for one unit illustrated in FIG. 3 (see FIG. 19 for the other part of the SPUH 31). Here, data of sub-pictures (for example, subtitles corresponding to movie scenes of the main picture) that are recorded and transmitted (communicated) together with the main picture (for example, a movie picture main body) will be described.

  As shown in FIG. 4, the sub-picture unit header SPUH31 includes a display size of the pixel data 32 on the TV screen, that is, a display start position and a display range (width and height) SPDSZ, and a display control in the sub-picture data packet. The recording start position SPDCSQTA of the sequence table 33 is recorded.

More specifically, the sub-picture unit header SPUH31 records various parameters (such as SPDDADR) having the following contents, as shown in FIG. 4:
(1) information (SPDSZ) indicating a display start position and a display range (width and height) of the display data on the monitor screen;
(2) Recording start position information (display control sequence table start address SPDCSQTA of sub-picture) of the display control sequence table 33 in the packet.

  The data length (variable length) of one unit of the pixel data (run-length data) 32 of the sub-picture shown in FIG. 3 or FIG. 4 is determined by the run-length compression rules 1 to 6 shown in FIG. Then, encoding (run-length compression) and decoding (run-length decompression) are performed with the determined data length.

  FIG. 5 shows run-length compression rule 1 used in the encoding method according to one embodiment when the sub-picture pixel data (run-length data) 32 illustrated in FIG. 4 is composed of 2-bit pixel data. 6 are explained.

  FIG. 6 is a diagram for specifically explaining the compression rules 1 to 6 in a case where the sub-picture pixel data (run-length data) 32 illustrated in FIG. 4 is composed of 2-bit pixel data. It is.

  According to rule 1 shown in the first column of FIG. 5, when one to three identical pixels continue, one unit of encoded (run-length compressed) data is constituted by 4-bit data. In this case, the first two bits represent the number of continuous pixels, and the subsequent two bits represent pixel data (such as pixel color information).

  For example, the first compressed data unit CU01 of the video data PXD before compression shown in the upper part of FIG. 6 includes two 2-bit pixel data d0, d1 = (0000) b (b is binary) ). In this example, two identical 2-bit pixel data (00) b are continuous (continuous).

  In this case, as shown in the lower part of FIG. 6, d0 and d1 = (1000) b connecting the 2-bit display (10) b of the continuation number "2" and the content (00) b of the pixel data are obtained after compression. Is a data unit CU01 * of the video data PXD.

  In other words, (0000) b of data unit CU01 is converted to (1000) b of data unit CU01 * according to rule 1. In this example, substantial compression of the bit length is not obtained. For example, if CU01 = (000000) b in which three identical pixels (00) b are continuous, CU01 * = (1100) b after compression. Thus, a 2-bit compression effect can be obtained.

  According to rule 2 shown in the second column of FIG. 5, when 4 to 15 identical pixels continue, 8-bit data constitutes one unit of encoded data. In this case, the first two bits represent an encoded header indicating that the rule is based on Rule 2, the subsequent four bits represent the number of continuous pixels, and the subsequent two bits represent pixel data.

  For example, the second compressed data unit CU02 of the video data PXD before compression shown in the upper part of FIG. 6 includes five 2-bit pixel data d2, d3, d4, d5, d6 = (01010101101) b. . In this example, five identical 2-bit pixel data (01) b are continuous (continuous).

  In this case, as shown in the lower part of FIG. 6, d2 to d6 in which the encoded header (00) b, the 4-bit display (0101) b of the continuation number "5", and the content (01) b of the pixel data are connected. = (00010101) b is the data unit CU02 * of the compressed video data PXD.

  In other words, (0101010101) b (10-bit length) of the data unit CU02 is converted into (00010101) b (8-bit length) of the data unit CU02 * according to Rule 2. In this example, the effective bit length compression amount is only 2 bits from 10 bits to 8 bits, but if the continuation number is, for example, 15 (30 bits long since 15 CU02 01s are continuous), this is 8 bits (CU02 * = 00111101), and a compression effect of 22 bits can be obtained for 30 bits. That is, the bit compression effect based on Rule 2 is greater than that of Rule 1. However, Rule 1 is also required to cope with run-length compression of fine images with high resolution.

  According to rule 3 shown in the third column of FIG. 5, when 16 to 63 identical pixels continue, one unit of encoded data is constituted by 12-bit data. In this case, the first 4 bits represent an encoded header indicating that the data is based on Rule 3, the next 6 bits represent the number of continuous pixels, and the subsequent 2 bits represent pixel data.

  For example, the third compressed data unit CU03 of the video data PXD before compression shown in the upper part of FIG. 6 includes 16 2-bit pixel data d7 to d22 = (101010... 1010) b. In this example, 16 pieces of the same 2-bit pixel data (10) b continue (continue).

  In this case, as shown in the lower part of FIG. 6, d7 to d22 in which the encoded header (0000) b, the 6-bit display (010000) b of the continuation number "16", and the content (10) b of the pixel data are connected. = (000000010010) b is the data unit CU03 * of the compressed video data PXD.

  In other words, (101010... 1010) b (32-bit length) of the data unit CU03 is converted to (0000000100010) b (12-bit length) of the data unit CU03 * according to Rule 3. In this example, the substantial bit length compression amount is 20 bits from 32 bits to 12 bits. However, if the number of continuations is, for example, 63 (126 bits long because 63 of CU03 are continuous, this is 12 bits) (CU03 * = 0000011111110), and a compression effect of 114 bits can be obtained for 126 bits. That is, the bit compression effect based on Rule 3 is greater than that of Rule 2.

  According to rule 4 shown in the fourth column of FIG. 5, when 64 to 255 identical pixels continue, one unit of encoded data is constituted by 16-bit data. In this case, the first 6 bits represent an encoded header indicating that the data is based on Rule 4, the subsequent 8 bits represent the number of continuous pixels, and the subsequent 2 bits represent pixel data.

  For example, the fourth compressed data unit CU04 of the video data PXD before compression shown in the upper part of FIG. 6 includes 69 2-bit pixel data d23 to d91 = (111111... 1111) b. In this example, 69 pieces of the same 2-bit pixel data (11) b are continuous (continuous).

  In this case, as shown in the lower part of FIG. 6, d23 to d91 in which the encoded header (000000) b, the 8-bit display (00100101) b of the continuation number “69”, and the content (11) b of the pixel data are connected. = (0000000010010111) b is the data unit CU04 * of the compressed video data PXD.

  In other words, (111111... 1111) b (138-bit length) of the data unit CU04 is converted into (0000000010010111) b (16-bit length) of the data unit CU04 * according to Rule 4. In this example, the substantial bit length compression amount is 122 bits from 138 bits to 16 bits, but if the number of continuations is, for example, 255 (510 bits long since 255 CU01s are 11 continuous), this is 16 bits. (CU04 * = 000001111111111), and a compression effect of 494 bits is obtained for 510 bits. That is, the bit compression effect based on Rule 4 is greater than that of Rule 3.

  According to rule 5 shown in the fifth column of FIG. 5, when the same pixel continues from the switching point of the encoded data unit to the end of the line, one unit of encoded data is constituted by 16-bit data. In this case, the first 14 bits represent an encoded header indicating that the data is based on Rule 5, and the following 2 bits represent pixel data.

  For example, the fifth compressed data unit CU05 of the uncompressed video data PXD shown in the upper part of FIG. 6 includes one or more 2-bit pixel data d92 to dn = (000000... 0000) b. In this example, a finite number of the same 2-bit pixel data (00) b are continuous (continuous), but according to rule 5, the number of continuous pixels may be one or more.

  In this case, as shown in the lower part of FIG. 6, d92 to dn = (000000000000000000) b, which connects the encoded header (0000000000000) b and the content (00) b of the pixel data, are converted into the compressed video data PXD. The data unit is CU05 *.

  In other words, (000000... 0000) b (unspecified bit length) of the data unit CU05 is converted into (000000000000000000) b (16-bit length) of the data unit CU05 * according to Rule 5. According to Rule 5, if the number of continuations of the same pixel up to the line end is 16 bits or more, a compression effect is obtained.

  According to rule 6 shown in the sixth column of FIG. 5, when one pixel line in which data to be encoded is arranged ends, the length of one line of compressed data PXD is not an integral multiple of 8 bits (that is, it is not byte-aligned). ), 4-bit dummy data is added so that the compressed data PXD for one line is in a byte unit (that is, byte-aligned).

  For example, the total bit length of the data units CU01 * to CU05 * of the compressed video data PXD shown in the lower part of FIG. 6 is always an integral multiple of 4 bits, but is always an integral multiple of 8 bits. Not necessarily.

  For example, if the total bit length of the data units CU01 * to CU05 * is 1020 bits and 4 bits are insufficient for byte alignment, as shown at the bottom of FIG. 6, 4-bit dummy data CU06 * = (0000) B) is added to the end of the 1,020 bits, and 1024-bit byte-aligned data units CU01 * to CU06 * are output.

  Note that the 2-bit pixel data is not necessarily limited to one that displays four types of pixel colors. For example, pixel data (00) b represents a background pixel of the sub-picture, pixel data (01) b represents a pattern pixel of the sub-picture, pixel data (10) b represents a first emphasized pixel of the sub-picture, The data (11) b may represent the second emphasized pixel of the sub-picture.

  If the number of constituent bits of the pixel data is larger, another type of sub-picture pixel can be specified. For example, when the pixel data is composed of (000) b to (111) b of 3 bits, up to eight types of pixel colors + pixel types (enhancement effect) can be set in the sub-picture data to be run-length encoded / decoded. Can be specified.

  FIG. 7 explains the flow from mass production to reproduction on the user side of a high-density optical disk having encoded image information (31 + 32 + 33 in FIG. 3); It is a block diagram explaining the flow until reception / reproduction in a subscriber.

  For example, when run-length data before compression is input to the encoder 200 of FIG. 7, the encoder 200 performs run-length compression (encoding) of the input data by software processing based on compression rules 1 to 6 of FIG. You.

  When data having a logical configuration as shown in FIG. 2 is recorded on the optical disc OD as shown in FIG. 1, the run-length compression processing (encoding processing) by the encoder 200 in FIG. Implemented.

  Various data necessary for completing the optical disc OD are also input to the encoder 200 of FIG. These data are compressed based on, for example, the MPEG (Mortion Picture Expert Group) standard, and the compressed digital data is sent to the laser cutting machine 202 or the modulator / transmitter 210.

  In the laser cutting machine 202, the MPEG compressed data from the encoder 200 is cut on a mother disk (not shown) to manufacture an optical disk master 204.

  In the mass production equipment 206 for two-layered high-density optical discs, the master 204 is used as a template, and the information of the master is transferred to a laser light reflecting film on a polycarbonate substrate having a thickness of 0.6 mm, for example. A large amount of two polycarbonate substrates onto which different pieces of master information have been transferred are bonded together to form a 1.2 mm thick double-sided optical disk (or single-sided reading double-sided disk).

  The bonded high-density optical disk OD mass-produced in the facility 206 is distributed to various markets and reaches the user.

  The distributed disk OD is reproduced by the reproduction device 300 of the user. The apparatus 300 includes a decoder 101 for restoring data encoded by the encoder 200 to original information. The information decoded by the decoder 101 is sent to, for example, a user's monitor TV, where it is visualized. Thus, the end user can view the original video information from the mass-distributed disc OD.

  On the other hand, the compression information sent from the encoder 200 to the modulator / transmitter 210 is modulated according to a predetermined standard and transmitted. For example, the compressed video information from the encoder 200 is satellite broadcast (212) together with the corresponding audio information. Alternatively, the compressed video information from the encoder 200 is transmitted by cable (212) together with the corresponding audio information.

  The broadcast or cable transmitted compressed video / audio information is received by a user / subscriber receiver / demodulator 400. The receiver / demodulator 400 includes a decoder 101 for restoring data encoded by the encoder 200 to original information. The information decoded by the decoder 101 is sent to, for example, a user's monitor TV, where it is visualized. Thus, the end user can enjoy the original video information from the compressed video information broadcast or transmitted by cable.

  FIG. 8 is a block diagram showing an embodiment (non-interlace specification) of hardware of a decoder for executing image decoding (run-length expansion) based on the present invention. The decoder 101 (see FIG. 7) for decoding the run-length compressed sub-picture data SPD (corresponding to the data 32 in FIG. 3) can be configured as shown in FIG. The sub-picture packets in the sub-picture information pack to which the SCR at the same time is added are sequentially transferred to the decoder 101, and the above-mentioned SCR is also supplied. The main STC 120a starts counting from a value that becomes zero after a predetermined time. The predetermined time corresponds to the transfer time of one sub-picture unit (maximum size) corresponding to the SCR. For example, when the maximum size of one sub-picture unit is 65535 bytes and the transfer rate is 740 nsec / 1 byte, 65535 × 740/100000 ≒ 48.5 msec, and starts from a value that becomes zero after about 48.5 msec. ing.

  Hereinafter, the sub-picture data decoder that extends the length of a signal including the run-length-compressed pixel data in the format shown in FIG. 4 will be described with reference to FIG.

  As shown in FIG. 8, the sub-picture decoder 101 includes a data I / O 102 to which the sub-picture data SPD is input; a memory 108 for storing the sub-picture data SPD; and a memory control for controlling a read / write operation of the memory 108. A unit 105 for detecting a continuous code length (encoded header) of one unit (one unit) from the run information of the code data (run-length compressed pixel data) read out from the memory 108; A continuation code length detection unit 106 that outputs separation information; a code data separation unit 103 that extracts code data for one unit according to the information from the continuation code length detection unit 106; an output from the code data separation unit 103 A signal indicating run information of one compression unit, and a signal output from the continuation code length detection unit 106. Receive a signal (period signal) indicating the number of consecutive "0" bits from the beginning of the code data of one unit from the beginning of the code data of one unit (period signal), and calculate the number of continuous pixels of one unit from these signals. And a pixel color output unit 104 (Fast) that receives the pixel color information from the code data separation unit 103 and the period signal output from the run length setting unit 107 and outputs the color information only during that period. a microcomputer 112 for reading header data (see FIG. 4) in the sub-picture data SPD read from the memory 108, and performing various processing settings and control based on the read data; An address control unit 109 for controlling a read / write address of the memory 108; Insufficient pixel color setter 111 set by 2; in a display valid authorization unit 110 for determining the display area when displaying a sub-picture such as TV screens, is constructed.

  24 to 27, the timer 120 and the buffer 121 are connected to the MPU 112 of the decoder 101. The timer 120 includes a main STC (system timer) 120a and a sub STC (sub timer) 120b.

  The above description will be described in another way as follows. That is, as shown in FIG. 8, the run-length compressed sub-picture data SPD is sent to the bus inside the decoder 101 via the data I / O 102. The data SPD sent to the bus is sent to the memory 108 via the memory control unit 105 and stored therein. The internal bus of the decoder 101 is connected to the code data separating unit 103, the continuous code length detecting unit 106, and the microcomputer (MPU or CPU) 112.

  The sub-picture unit header 31 of the sub-picture data read from the memory 108 is read by the microcomputer 112. The microcomputer 112 sets the decoding start address (SPDDDR) in the address control unit 109 from the read header 31 based on the various parameters shown in FIG. The information (SPDSZ) of the width and the display height is set, and the display width (the number of dots on a line) of the sub-picture is set in the code data separating unit 103. The set various information is stored in an internal register of each unit (109, 110, 103). Thereafter, the various information stored in the register can be accessed by the microcomputer 112.

  The address control unit 109 accesses the memory 108 via the memory control unit 105 based on the decoding start address (SPDDDR) set in the register, and starts reading sub-video data to be decoded. The sub-picture data thus read from the memory 108 is supplied to the code data cutout unit 103 and the continuous code length detection unit 106.

  The encoded header of the run-length-compressed sub-picture data SPD (2 to 14 bits in rules 2 to 5 in FIG. 5) is detected by the continuation code length detection unit 106, and the number of continuation pixels of the same pixel data in the data SPD is It is detected by the run length setting unit 107 based on the signal from the continuous code length detection unit 106.

  That is, the continuation code length detection unit 106 counts the number of “0” bits of the data read from the memory 108 and detects the encoded header (see FIG. 5). The detection unit 106 supplies the code data separation unit 103 with the separation information SEP. INFO. give.

  The code data separation unit 103 receives the provided separation information SEP. INFO. , The number of continuous pixels (run information) is set in the run length setting unit 107, and the pixel data (SEPARATED DATA; here, the pixel color) is set in the FIFO type pixel color output unit 104. At that time, the code data separating unit 103 counts the number of pixels of the sub-picture data, and compares the pixel number count value with the display width of the sub-picture (the number of pixels in one line).

  If the byte is not aligned at the time when the decoding of one line is completed (that is, the data bit length of one line is not a multiple of 8), the code data separating unit 103 extracts the last 4 bits of data on that line. It is regarded as dummy data added at the time of encoding, and is discarded.

The run length setting unit 107 supplies a pixel color output unit 104 with a pixel based on the number of continuous pixels (run information), a pixel dot clock (DOTCLK), and a horizontal / vertical synchronization signal (H-SYNC / V-SYNC). A signal (PERIOD SIGNAL) for outputting data is given. Then, while the pixel data output signal (PERIOD SIGNAL) is active (that is, during the period of outputting the same pixel color), the pixel color output unit 104 decodes the pixel data from the code data separation unit 103 into a decoded display. Output as data.

  At this time, if the decoding start line has been changed by an instruction from the microcomputer 112, there may be a line without run information. In this case, the missing pixel color setting unit 111 supplies the preset missing pixel color data (COLOR INFO.) To the pixel color output unit 104. Then, the pixel color output unit 104 outputs the insufficient pixel color data (COLOR INFO.) From the insufficient pixel color setting unit 111 while the line data without the run information is given to the code data separating unit 103.

  That is, in the case of the decoder 101 of FIG. 8, if there is no image data in the input sub-picture data SPD, the microcomputer 112 sets the insufficient pixel color information to the insufficient pixel color setting unit 111 accordingly. I have.

  The pixel color output unit 104 is synchronized with a horizontal / vertical synchronization signal of the sub-picture image by a display enable signal for determining a position on the monitor screen (not shown) where the decoded sub-picture is to be displayed. Then, it is provided from the display activation permitting section (Display Activator) 110. Further, based on a color information instruction from the microcomputer 112, a color switching signal is sent from the permission unit 110 to the output unit 104.

  The address control unit 109 sends address data and various timing signals to the memory control unit 105, the continuation code length detection unit 106, the code data separation unit 103, and the run length setting unit 107 after the processing setting by the microcomputer 112. I do.

  When a pack of the sub-picture data SPD is fetched via the data I / O unit 102 and stored in the memory 108, the contents of the pack header (decoding start address, decoding end address, display start position, Display width, display height, etc.) are read by the microcomputer 112. The microcomputer 112 sets a decode start address, a decode end address, a display start position, a display width, a display height, and the like in the display validity permitting unit 110 based on the read content.

  Hereinafter, the operation of the decoder 101 in FIG. 8 will be described in the case where the compressed pixel data has a 2-bit configuration (the usage rules are rules 1 to 6 in FIG. 5).

  When the microcomputer 112 sets a decode start address, the address control unit 109 sends the corresponding address data to the memory control unit 105 and sends a read start signal to the continuous code length detection unit 106.

  The continuation code length detection unit 106 sends a read signal to the memory control unit 105 in response to the received read start signal to read the encoded data (compressed sub-video data 32). Then, the detection unit 106 checks whether or not all the upper two bits of the read data are “0”.

  If they are not “0”, it is determined that the block length of the compression unit is 4 bits (see rule 1 in FIG. 5).

  If those (upper 2 bits) are “0”, the subsequent 2 bits (upper 4 bits) are checked. If they are not “0”, it is determined that the block length of the compression unit is 8 bits (see rule 2 in FIG. 5).

  If those (upper 4 bits) are “0”, the subsequent 2 bits (upper 6 bits) are checked. If they are not “0”, it is determined that the block length of the compression unit is 12 bits (see rule 3 in FIG. 5).

  If those (upper 6 bits) are “0”, further 8 bits (upper 14 bits) are checked. If they are not “0”, it is determined that the block length of the compression unit is 16 bits (see rule 4 in FIG. 5).

  If these (upper 14 bits) are “0”, it is determined that the block length of the compression unit is 16 bits and the same pixel data continues until the line end (see rule 5 in FIG. 5).

  If the number of bits of the pixel data read up to the line end is an integer multiple of 8, the bit data is left as it is, and if it is not an integer multiple of 8, 4-bit dummy data is added to the end of the read data to realize byte alignment. Is determined to be necessary (see rule 6 in FIG. 5).

  The code data separating unit 103 extracts one block (one compression unit) of the sub-picture data 32 from the memory 108 based on the determination result by the continuous code length detecting unit 106. Then, in the separation unit 103, the extracted data for one block is separated into the number of continuous pixels and pixel data (such as pixel color information). The data of the divided continuous pixel number (RUN INFO.) Is sent to the run length setting unit 107, and the separated pixel data (SEPARATED DATA) is sent to the pixel color output unit 104.

  On the other hand, according to the display start position information, the display width information, and the display height information received from the microcomputer 112, the display validity permitting section 110 supplies a pixel dot clock (PIXEL-DOT CLK) and a horizontal synchronization signal (PIXEL-DOT CLK) supplied from outside the device. H-SYNC) and a vertical synchronizing signal (V-SYNC) to generate a display permission signal (enable signal) for specifying a sub-image display period. This display permission signal is output to run length setting section 107.

  The run length setting unit 107 includes a signal output from the continuation code length detection unit 106 and indicating whether or not the current block data continues to the line end, and a continuation pixel data from the code data separation unit 103. (RUN INFO.) Is sent. The run length setting unit 107 determines the number of pixel dots covered by the block being decoded based on the signal from the detection unit 106 and the data from the separation unit 103, and during the period corresponding to the number of dots, outputs the pixel color. It is configured to output a display permission signal (output enable signal) to the unit 104.

  The pixel color output unit 104 is enabled while receiving the period signal from the run length setting unit 107, and synchronizes the pixel color information received from the code data separation unit 103 with the pixel dot clock (PIXEL-DOT CLK) during that period. Then, the decoded display data is transmitted to a display device (not shown) or the like. That is, the same display data as the number of continuous dots of the pixel pattern of the block being decoded is output from the pixel color output unit 104.

  If the continuation code length detection unit 106 determines that the encoded data is the same pixel color data until the line end, the continuation code length detection unit 106 outputs a signal for a continuation code length of 16 bits to the code data separation unit 103, and the run length setting unit A signal 107 indicates that the pixel color data is the same until the line end.

  Upon receiving the above signal from the detection unit 106, the run length setting unit 107 sets the pixel color output unit 104 such that the color information of the encoded data keeps the enabled state until the horizontal synchronization signal H-SYNC becomes inactive. To output an output enable signal (period signal).

  When the microcomputer 112 changes the decoding start line in order to scroll the display content of the sub-picture, the data line to be used for decoding does not exist in the preset display area (that is, the decoding line is insufficient). )there is a possibility.

  In order to cope with such a case, the decoder 101 in FIG. 8 prepares pixel color data for filling the missing lines in advance. Then, when the line shortage is actually detected, the display mode is switched to the display mode of the insufficient pixel color data. More specifically, when a data end signal is given from the address control unit 109 to the display validity permission unit 110, the permission unit 110 sends a color switching signal (COL SW SW SIGNAL) to the pixel color output unit 104. In response to the switching signal, the pixel color output unit 104 switches the decoded output of the pixel color data from the code data to the decoded output of the color information (COLOR INFO.) From the insufficient pixel color setting unit 110. This switching state is maintained during the display period of the insufficient line (DISPLAY ENABLE = active).

  When the line shortage occurs, instead of using the insufficient pixel color data, the decoding operation can be stopped during that time.

  Specifically, for example, when a data end signal is input from the address control unit 109 to the display validity permission unit 110, the permission unit 110 may output a color switching signal designating display suspension to the pixel color output unit 104. . Then, the pixel color output unit 104 stops displaying the sub-picture while the display stop designation color switching signal is active.

  FIG. 9 illustrates two examples (non-interlaced display and interlaced display) of how the character pattern “A” is decoded in the encoded pixel data (sub-picture data).

  The decoder 101 of FIG. 8 can be used when decoding compressed data as shown in the upper part of FIG. 9 into non-interlaced display data as shown in the lower left part of FIG.

  On the other hand, when decoding compressed data as shown in the upper part of FIG. 9 into interlaced display data as shown in the lower right part of FIG. 9, a line doubler that scans the same pixel line twice (for example, line # of an odd field) Line # 10 having the same content as 1 is re-scanned in an even field; switching between V-SYNC units is required.

  When non-interlaced display is performed with the same image display amount as interlaced display, another indubbler (for example, line # 10 having the same content as line # 1 at the lower right of FIG. 9 is made continuous with line # 1; SYNC unit).

FIG. 10 is a flowchart for executing image decoding (run-length decompression) according to an embodiment of the present invention and explaining software executed by the microcomputer 112 shown in FIG. 11 or FIG. 12, for example. (Decoding of the display control sequence table DCSQT33 in FIG. 3 is not described here. Decoding of the DCSQT33 portion will be described later with reference to FIGS. 25 to 27.)
FIG. 11 is a flowchart illustrating an example of the contents of the decoding step (ST105) used in the software of FIG.

  That is, the microcomputer 112 reads the first header 31 portion of the run-length compressed sub-picture data (pixel data is composed of 2 bits) and analyzes the contents (see FIG. 4). Then, the number of lines and the number of dots of the image data to be decoded are specified based on the content of the analyzed header. When the number of lines and the number of dots are designated (step ST101), the number of lines and the number of dots are initialized to “0” (steps ST102 to ST103).

  The microcomputer 112 sequentially takes in the data bit string following the sub-picture unit header 31 and counts the number of dots and the number of dots. Then, the number of continuous pixels is calculated by subtracting the number of dots from the number of dots (step ST104).

  When the number of continuous pixels is calculated in this way, the microcomputer 112 performs a decoding process according to the value of the number of continuous pixels (step ST105).

  After the decoding process in step ST105, the microcomputer 112 adds the dot count number and the continuation pixel number, and sets this as a new dot count number (step ST106).

  Then, the microcomputer 112 sequentially fetches the data and executes the decoding process in step ST105. When the accumulated dot count matches the line end number (line end position) set first, the microcomputer 112 performs the data processing for one line. Is completed (Yes in step ST107).

  Next, if the decoded data is byte-aligned (Yes in step ST108A), the dummy data is removed (step ST108B). Then, the line count number is incremented by +1 (step ST109), and the processing of steps ST102 to ST109 is repeated until the last line is reached (No in step ST1010). If the last line has been reached (step ST1010: YES), the decoding ends.

  The processing content of the decoding processing step ST105 in FIG. 10 is, for example, as shown in FIG.

  In this process, after acquiring two bits from the beginning, weaving to determine whether the bit is “0” is repeated (step ST111 to step ST119). As a result, the number of continuous pixels corresponding to the run length compression rules 1 to 6 in FIG. 5, that is, the number of consecutive runs is determined (steps ST120 to ST123).

  Then, after the number of consecutive runs is determined, the two bits read subsequently are used as a pixel pattern (pixel data; pixel color information) (step ST124).

  When the pixel data (pixel color information) is determined, the index parameter “i” is set to 0 (step ST125), and a 2-bit pixel pattern is output until the parameter “i” matches the number of consecutive runs (step ST126). (Step ST127), increments the parameter “i” by +1 (Step ST128), ends the output of one unit of the same pixel data, and ends the decoding process.

  As described above, according to the decoding method of the sub-picture data, the decoding process of the sub-picture data can be a simple process of determining a few bits, separating a data block, and counting data bits. This eliminates the need for a large code table used in the conventional MH encoding method and the like, and simplifies the processing and configuration for decoding the encoded bit data into the original pixel information.

  In the above-described embodiment, the code bit length for one unit of the same pixel can be determined by reading bit data of up to 16 bits at the time of data decoding, but the code bit length is not limited to this. For example, the code bit length may be 32 bits or 64 bits. However, as the bit length increases, a data buffer having a larger capacity is required.

  Further, in the above embodiment, the pixel data (color information of the pixel) is the color information of three colors selected from, for example, a color palette of 16 colors. However, in addition to this, the three primary colors (red component R, green Component G, blue component B; or luminance signal component Y, chroma red signal component Cr, chroma blue signal component Cb, etc.) can also be represented by 2-bit pixel data. That is, the pixel data is not limited to the specific type of color information.

  FIG. 12 is a block diagram illustrating an outline of an optical disc recording / reproducing apparatus in which encoding (encoding of SPUH + PXD + DCSQT in FIG. 3) and decoding (decoding of SPUH + PXD + DCSQT) are executed.

  In FIG. 12, an optical disk player 300 has basically the same configuration as a conventional optical disk reproducing device (compact disk player or laser disk player). However, the optical disc player 300 uses a digital signal (encoding) before decoding the run-length-compressed image information from the inserted optical disc OD (where the image information including the run-length-compressed sub-picture data is recorded). Digital signal as it is). Since this encoded digital signal is compressed, the required transmission bandwidth may be smaller than in the case of transmitting uncompressed data.

  The compressed digital signal from the optical disc player 300 is sent on-air via the modulator / transmitter 210 or sent out to a communication cable.

  The compressed digital signal that has been aired or transmitted by cable is received by the receiver / demodulator 400 of the receiver or cable subscriber. The receiver 400 includes the decoder 101 having a configuration as shown in FIG. 8, for example. The decoder 101 of the receiver 400 decodes the received and demodulated compressed digital signal, and outputs image information including original sub-video data before being encoded.

  In the configuration of FIG. 12, if the transmission / reception transmission system has an average bit rate of about 5 Mbit / sec or more, high-quality multimedia video / audio information can be broadcast.

  FIG. 13 is a block diagram illustrating a case where image information encoded according to the present invention is transmitted / received between any two computer users via a communication network (such as the Internet).

  User # 1 having self-information # 1 managed by a host computer (not shown) owns personal computer 5001, and various input / output devices 5011 and various external storage devices 5021 are connected to personal computer 5001. I have. An internal slot (not shown) of the personal computer 5001 is provided with a modem card 5031 having a built-in encoder and decoder according to the present invention and having functions necessary for communication.

  Similarly, a user #N having another self information #N owns a personal computer 500N, and various input / output devices 501N and various external storage devices 502N are connected to the personal computer 500N. An encoder and a decoder according to the present invention are incorporated in an internal slot (not shown) of the personal computer 500N, and a modem card 503N having functions necessary for communication is mounted.

  Now, it is assumed that a user # 1 operates the computer 5001 and communicates with a computer 500N of another user #N via a line 600 such as the Internet. In this case, since both user # 1 and user #N have modem cards 5031 and 503N in which an encoder and a decoder are incorporated, the present invention can efficiently exchange compressed image data in a short time.

  FIG. 24 shows an outline of a recording / reproducing apparatus which records encoded image information (SPUH + PXD + DCSQT in FIG. 3) on an optical disk OD and decodes the recorded information (SPUH + PXD + DCSQT) according to the present invention.

  The encoder 200 of FIG. 24 executes the same encoding processing (processing corresponding to FIGS. 13 to 14) as the encoder 200 of FIG. 7 by software or hardware (including firmware or a wired logic circuit). It is configured.

  A recording signal including sub-picture data and the like encoded by the encoder 200 is subjected to, for example, (2, 7) RLL modulation in the modulator / laser driver 702. The modulated recording signal is sent from the laser driver 702 to the high-power laser diode of the optical head 704. By the recording laser from the optical head 704, a pattern corresponding to the recording signal is written on the magneto-optical recording disk or the phase-change optical disk OD.

  The information written on the disk OD is read by the laser pickup of the optical head 706, demodulated by the demodulator / error correction unit 708, and subjected to error correction processing as needed. The demodulated and error-corrected signal undergoes various data processing in the audio / video data processing unit 710, and the information before recording is reproduced.

  This data processing unit 710 includes a decoding processing unit corresponding to the decoder 101 in FIG. The decoding process (decompression of the compressed sub-picture data) corresponding to FIGS.

  Here, the sub-picture data shown in FIG. 2 or 3 is composed of a plurality of channels as shown in FIG. The sub-picture data unit is composed of a plurality of sub-picture data packets of a channel arbitrarily selected from the plurality of channels. The sub-picture has information such as characters or figures, is reproduced at the same time as video data and audio data, and is superimposed and displayed on a video data reproduction screen.

  FIG. 16 shows the data structure of a sub-picture packet. As shown in FIG. 16, the sub-picture packet data includes a packet header 3, a sub-picture header 31, sub-picture data 32, and a display control sequence table 33.

  In the packet header 3, the time at which the playback system should start the display control of the sub-picture data unit is recorded as a presentation time stamp (PTS). However, as shown in FIG. 17, the PTS is recorded only in the header 3 of the first sub-picture data packet in each sub-picture data unit (Y, W). In the PTS, in a plurality of sub-video data units reproduced at a predetermined reproduction time SCR, values along the reproduction order are described for each sub-video data unit.

  FIG. 18 shows the serial arrangement state (n, n + 1) of sub-picture units (see 30 in FIG. 3) composed of one or more sub-picture packets, and the time described in the packet header of one unit (n + 1) of them. The stamp PTS and the state of display control of the unit (n + 1) corresponding to the PTS (clearing of the display of the previous sub-picture and designation of the display control sequence of the sub-picture to be displayed from now on) are illustrated.

  In the sub-picture header 31, the size of the sub-picture data packet (SPCSZ of 2 bytes) and the recording start position of the display control sequence table 33 (SPDCSQTA of 2 bytes) in the packet are recorded.

  The display control sequence table 33 includes a sub-picture display control time stamp (SPDCTS) indicating the display start time / display end time of the video data and the sub-picture data (PXD) 32 to be displayed. One or more display control sequence information (DCSQT; Display Control Sequence Table) in which a recording position (SPNDSQQA; Sub-Picture Next Display Control Sequence Address) and a sub video data display control command (COMMAND) are grouped together are recorded. .

  Here, the time stamp PTS in the packet header 3 is defined as a relative time from a reference time (SCR; System Clock Reference) throughout reproduction of the entire file, such as a reproduction start time at the beginning of the file (FIG. 2). Have been. This SCR is described in a pack header provided before the packet header 3. On the other hand, each time stamp SPDCTS in the display control sequence table 33 is defined by a relative time from the PTS.

  “0” is described in the sub-video display control time stamp SPDCTS of the sequence to be displayed first.

  Next, the time stamp PTS processing of the sub-picture data packet in the playback system will be described. Here, it is assumed that the PTS processing is executed in the sub-picture processor (for example, the MPU 112 in FIG. 8 and its peripheral circuits) in the playback system.

  FIG. 15 is a diagram for explaining how the buffering state of the sub-picture data unit changes depending on the sub-picture channel having the time stamp PTS when decoding the sub-picture data.

  (1) The sub-picture processor (FIG. 8 and others) decodes a sub-picture data packet of a preselected channel from among sub-picture data packets sent from the outside (such as an optical disc or a broadcasting station), and decodes the packet. To see if there is a PTS inside.

  For example, when a PTS exists as shown by the channel * 4f in FIG. 15, the PTS is separated from the packet header 3. Thereafter, as shown in FIG. 17, for example, a PTS is added to the head of the sub-picture data, and the sub-picture data with a PTS header is buffered (stored) in a sub-picture buffer (for example, buffer 121 in FIG. 8).

  Note that the graph of FIG. 15 illustrates a state in which the buffering amount to the sub-video buffer 121 is accumulated as the sub-video data packets of the channel * 4f with PTS are buffered.

  (2) After the system reset, the sub-picture processor performs the vertical blanking period immediately after receiving the first packet including the PTS (during the switching period from one display screen frame / field to the next display screen frame / field). ), And compares the fetched PTS with a count value of a main STC 120a (a counter in the sub-picture processor, for example, a part of the timer 120 in FIG. 8) as a reference time counter. The main STC 120a measures the elapsed time from the reference time SCR through the reproduction of the entire file such as the reproduction start time at the beginning of the file. The main STC 120a starts from a value that becomes zero after the above-described predetermined time. ing. Accordingly, “0” can be described in the PTS (attached to the first packet of the sub-picture unit) without considering the transfer time of the sub-picture unit.

  (3) As a result of the comparison between the PTS and the count value of the STC 120a, if the count value of the STC 120a is larger than the PTS, the sub-picture data is immediately displayed. On the other hand, if the count value of STC 120a is smaller than PTS, no processing is performed. This comparison is performed again during the next vertical blanking period.

  (4) When the processing of the sub-picture data is started, during the same vertical blanking period, the first sub-picture display control time stamp SPDCTS recorded in the display control sequence table 33 in the sub-picture data packet is changed to the sub-picture. It is compared with the count value of a sub reference time counter (sub STC) in the processor. The sub-STC is configured by a sub-reference time counter (for example, the other part of the timer 120 in FIG. 8) sub-STC 120b in the sub-image processor that measures the elapsed time from the reproduction start time of the sub-image data unit. Therefore, every time the display is switched to the next (subsequent) sub-picture data unit, all bits of this sub-STC 120b are cleared to "0", and then the increment (time count) is started again.

  (5) As a result of comparing the count value of the sub STC 120b with the sub-video display control time stamp SPDCTS, if the count value of the sub STC 120b is larger than SPDCTS, the control data of the first display control sequence of the display control sequence table 33 ( DCSQT (for example, DCSQT0 in FIG. 16) is immediately executed, and the display processing of the sub-picture is started.

  (6) Once the display processing is started, the PTS added to the first packet of the sub-picture data unit next to the currently displayed sub-picture data unit is read every vertical blanking period, and this reading is performed. The obtained PTS is compared with the count value of main STC 120a.

  As a result of this comparison, if the count value of main STC 120a is larger than the PTS, the channel pointer in FIG. 16 is set to the address value of the PTS of the next sub-picture data unit, and the next sub-picture data unit to be processed is set to the next. Can be switched. For example, taking FIG. 17 as an example, switching from the sub-picture data unit Y to the next sub-picture data unit W is performed by changing the setting of the channel pointer. At this point, since the data of the sub-picture data unit Y is no longer needed, a free area of the size of the sub-picture data unit Y is generated in the sub-picture buffer (for example, the memory 108 in FIG. 8). Therefore, a new sub-picture data packet can be newly transferred to this empty area.

  Thus, the buffering state of the sub-picture data packet (see FIG. 15) is determined based on the size of the sub-picture data unit (for example, unit W in FIG. 17) and its switching time (the switching time from unit Y to unit W). ,.. (At the time of encoding the sub-video data). Therefore, when a video / audio / sub-picture packet is serially transferred, a buffer of each decoder unit (in the case of a sub-picture decoder, the other memory 108 in FIG. 8) does not generate an overflow or underflow in a bit stream. Generation becomes possible.

  When the PTS is compared with the count value of the main STC 120a, if the count value of the main STC 120a is not larger than the PTS, the sub-video data unit is not switched, and the display control sequence table pointer (the DCSQT pointer in FIG. ) Is set to the address value of the next display control sequence table DCSQT. Then, the sub video display control time stamp SPDCTS of the next DCSQT in the current sub video data packet is compared with the count value of the sub STC 120b. Based on the comparison result, it is determined whether or not to execute the next DCSQT. This operation will be described later in detail.

  Since the last DCSQT in the sub-picture data packet indicates itself as the next display control sequence table DCSQT, the DCSQT processing of (5) is basically unchanged.

  (7) In normal reproduction, the processing of (4), (5), and (6) is repeated.

  In the process (6), when reading the PTS of the next sub-picture data unit, the value of the channel pointer (see FIG. 16) indicating the PTS is set to the packet size (SPCSZ) in the current sub-picture data unit. Is obtained by using.

  Similarly, the value of the DCSQT pointer indicating the next DCSQT sub-picture display control time stamp SPDCTS in the display control sequence table 33 is the size information of the DCSQT described in this table 33 (the next sub-picture display control sequence). Address SPNDSQTA).

  Next, details of the sub-picture header 31, the sub-picture data 32, and the display control sequence table 33 will be described.

  FIG. 19 shows the structure of the sub-picture unit header (SPUH) 31. The sub-picture unit header SPUH includes the size (SPDSZ) of the sub-picture data packet and the recording start position information of the display control sequence table 33 in the packet (sub-picture display control sequence table start address SPDCSQTA; relative address pointer of DCSQ). In.

  The contents of the sub-video display control sequence table SPDCSQT indicated by the address SPDCSQTA are composed of a plurality of display control sequences DCSQ1 to DCSQn as shown in FIG.

  As shown in FIG. 21, each display control sequence DCSQ (1 to n) includes a sub-picture display control time stamp SPDCTS indicating a sub-picture display control start time and an address SPNDSQSQA indicating the position of the next display control sequence. And one or more sub-picture display control commands SPDCCMD.

  The sub-picture data 32 is composed of a group of data areas (PXD areas) corresponding one-to-one with individual sub-picture data packets.

  Here, until the sub-picture data unit is switched, the sub-picture pixel data PXD at an arbitrary address in the same data area can be read. As a result, an arbitrary sub-video display (for example, scroll display of a sub-video) that is not fixed to one sub-video display image can be performed. This arbitrary address is set by a command (SETDSPXA in the command table of FIG. 22) for setting the display start address of the sub-picture data (pixel data PXD).

  FIG. 23 shows a specific example of the display control sequence table 33. As described above, one display control sequence information (DCSQT) in the display control sequence table 33 includes a plurality of display control commands after the sub-video display control time stamp (SPDCTS) and the sub-video data recording position (SPNDSQSQA). (COMMAND3, COMMAND4, etc.) and various parameter data set by the command are arranged. An end command (end code) indicating the end of the display control is added at the end.

  Next, a processing procedure of the display control sequence table 33 will be described.

  (1) First, the time stamp (SPDCTS) recorded in the first DCSQT (DCSQT0 in FIG. 16) of the display control sequence table 33 is the time stamp (SPDCTS) of the sub video processor sub STC 120b (for example, one function of the timer 120 in FIG. 8). It is compared with the count value.

  (2) As a result of the comparison, when the count value of the sub STC 120b is larger than the time stamp SPDCTS, all the display control commands COMMAND in the display control sequence table 33 are processed until the display control end command CMDEND (FIG. 22) appears. Be executed.

  (3) After the display control is started, the sub-video display control time stamp SPDCTS and the count value of the sub STC 120b recorded in the next display control sequence table DCSQT at fixed time intervals (for example, every vertical blanking period). Is compared with the next DCSQT (that is, whether the DCSQT pointer in FIG. 16 is moved to the next DCSQT).

Here, the time stamp SPDCTS in the display control sequence table 33
Is recorded at a relative time after the PTS is updated (that is, after the sub-picture data unit is updated), so that it is not necessary to rewrite the SPDCTS even if the PTS of the sub-picture data unit changes. Therefore, the same display control sequence table DCSQT can be used even when the same sub-picture data 32 is displayed at a plurality of different times. That is, the display control sequence table DCSQT can be made relocatable.

  Next, details of the sub-image display control command will be described. FIG. 22 shows a list of sub-picture display control commands SPDCCMD. The main sub-image display control commands are as follows.

(1) Command STADSP for setting display start timing of sub-picture pixel data
This is a command for executing the display start control of the sub-picture data 32. That is, when switching from a certain DCSQT to a DCSQT including the command STASPD, the display of the sub-picture data 32 is started from the time indicated by the time stamp SPDCTS of the DCSQT including the command.

  When the sub-picture processor (for example, the MPU 112 in FIG. 8) decodes this command, the sub-picture processor immediately decodes this command (since the time indicated by the DCDCT SPDCTS to which this command belongs has passed at the time of accessing this command). Activate the enable bit of the display control system.

(2) Command STPDSP for setting display end timing of sub-picture pixel data
This is a command for executing the display end control of the sub-picture data 32. When the sub-picture processor decodes this command, it immediately transmits the enable bit of the display control system inside the sub-picture processor (since the time indicated by SPDCTS of the DCSQT to which this command belongs has passed at the time of accessing this command). To the active state.

(3) Command SETCOLOR for setting color code of sub-picture pixel data
This is a command for setting the color code of the sub-picture pixel data. By this command, the sub-picture is composed of a pattern pixel such as a character or a pattern, an emphasized pixel such as a trimming of the pattern pixel, and a background pixel which is a pixel in a region other than the pattern pixel and the emphasized pixel in a region where the sub-image is displayed. And color information can be set.

(4) Command SETCONTR for setting contrast of sub-picture pixel data with respect to main picture
This is a command for setting contrast data instead of color code data for the four types of pixels illustrated in FIG. 40, similarly to the command SETCOLOR.

(5) Command SETDAREA for setting display area of sub-picture pixel data on main picture
This is a command for specifying a position at which the sub-picture pixel data 32 is displayed.

(6) Command SETDSPXA for setting the display start address of sub-picture pixel data
This is a command for setting the display start address of the sub-picture pixel data 32.

(7) Command CHGCOLCON to set switching of color code of sub-picture pixel data and contrast of sub-picture pixel data with respect to main picture
This is a command for changing the color code of the sub-picture pixel data 32 and the contrast of the sub-picture pixel data 32 with respect to the main picture during display.

  The command table in FIG. 22 includes, in addition to the above-described commands, a command FSTADSP for forcibly setting the display start timing of the sub-picture pixel data, and a command CMDEND for ending the display control of the sub-picture.

  FIG. 24 is a flowchart illustrating an example of a method of generating the sub-picture unit 30 as illustrated in FIG.

  When, for example, subtitles and / or images corresponding to dialogue of a video (main video) are used as sub-images, the dialogue subtitles / images are converted into bitmap data (step ST10). When creating this bitmap data, it is necessary to determine in which position and in which region on the video screen the subtitle portion is to be displayed. Therefore, the parameters of the display control command SETDAREA (see FIG. 22) are determined (step ST12).

  When the display position (spatial parameter) of the sub-picture is determined, the process proceeds to encoding of the pixel data PXD constituting the sub-picture (the whole main picture is not encoded). At this time, the color of the caption (sub-picture), the background color of the caption area, and the mixture ratio of the caption color / background color to the video main picture are determined. Therefore, the parameters of the display control commands SETCOLOR and SETCONTR (see FIG. 22) are determined (step ST14).

  Next, the timing at which the created bitmap data is to be displayed in accordance with the speech of the video is determined. This timing is determined by the sub-picture time stamp PTS. At this time, the maximum time of the time stamp PTS and each parameter (temporal parameter) of the display control commands STADSP, STPDSP and CHGCOLCON (see FIG. 22) are determined (step ST16).

  Here, the sub-picture time stamp PTS is finally determined from the consumption model of the target decoder buffer of the MPEG2 system layer. Here, the time at which the display of subtitles is started is determined as the maximum time of the sub-picture time stamp PTS.

  The display control commands STADSP and STPDSP are recorded as relative times from the sub-picture time stamp PTS. Therefore, the commands STADSP and STPDSP cannot be determined until the PTS is determined. Thus, in this embodiment, the absolute time is determined, and the relative value is determined after the absolute time of the PTS is determined.

  When it is desired to change the display color or display area spatially and temporally with respect to the created caption, the parameters of the command CHGCOLCON are determined based on the change.

  When the display position (spatial parameter) and the display timing (temporal parameter) of the sub-picture are (tentatively) determined, the contents (DCSQ) of the sub-picture display control sequence table DCSQT are created (step ST18). Specifically, the value of the display control start time SPDCTS (see FIG. 21) of the display control sequence table DCSQ is determined by the effective time of the display control command STADSP (display start timing) and the effective time of the display control command STPDSP (display end timing). Determined in accordance with

  By combining the created pixel data PXD32 and the display control sequence table DCSQT33, it is possible to determine the size of the sub-picture data unit 30 (see FIG. 3). Therefore, the sub-picture unit header SPUH31 is created by determining the parameters SPDSZ (sub-picture size; see FIG. 19) and SPDCSQTA (start address of the display control sequence table; see FIG. 19) of the sub-picture unit header SPUH31 based on the size. I do. Thereafter, by combining SPUH31, PXD32, and DCSQT33, a sub-picture unit for one subtitle is created (step ST20).

  If the size of the created sub-picture unit 30 exceeds a predetermined value (2048 bytes or 2 kbytes) (YES in step ST22), it is divided into a plurality of packets in units of 2k bytes (step ST24). In this case, the time stamp PTS is recorded only in the packet at the head of the sub-picture unit 30 (step ST26).

  If the size of the created sub-picture unit 30 is within a predetermined value (2 kbytes) (No in step ST22), only one packet is generated (step ST23), and the time stamp PTS is recorded at the beginning of the packet. (Step ST26).

  One or more packets thus completed are packed and combined with video and other packs to form one data stream (step ST28). At this time, the order of the packs is determined based on the sequence recording code SRC and the sub-picture time stamp PTS from the consumption model of the target decoder buffer of the MPEG2 system layer. Here, the PTS is determined for the first time, whereby the parameters (SPDCTS and the like) in FIG. 21 are finally determined.

  FIG. 25 is a flowchart illustrating an example of a procedure for performing parallel processing of pack decomposition and decoding of the sub-picture data stream generated according to the processing procedure of FIG.

  First, the decoding system reads the ID of the transferred stream and transfers only the selected sub-picture pack (separated from the data stream) to the sub-picture decoder (for example, the sub-picture decoder 101 in FIG. 8). (Step ST40).

  When the first pack transfer is performed, the index parameter "i" is set to "1" (step ST42), and the first sub-picture pack is decomposed (step ST44; described later with reference to FIG. 26). Is done.

  The decomposed pack (including the compressed sub-picture data PXD as shown in the lower part of FIG. 6) is temporarily stored in the sub-picture buffer (the memory 108 in FIG. 8) (step ST46), and the index parameter “i” is set to 1 Is incremented by one (step ST50).

  If the incremented i-th pack exists, that is, if the pack decomposed in step ST44 is not the final pack (No in step ST52), the decomposing process (step ST44) for the incremented i-th sub-picture pack is executed. Is done.

  The decomposed i-th sub-picture pack (here, the second pack) is temporarily stored in the sub-picture buffer (memory 108) in the same manner as the first decomposed pack (step ST46), and the index parameter "i" Is further incremented by one (step ST50).

  As described above, a plurality of sub-picture packs are successively decomposed while incrementing the index parameter "i" (step ST44) and stored in the sub-picture buffer (memory 108) (step ST46).

  If the continuously incremented i-th pack does not exist, that is, if the pack decomposed in step ST44 is the last pack (step ST52: YES), the sub-picture pack decomposing process of the stream to be decoded ends. .

  While the sub-picture pack disassembly processing (steps ST44 to ST52) is continuously performed, the sub-picture pack temporarily stored in the sub-picture buffer (memory 108) is executed independently and in parallel with the sub-picture pack disassembly processing. The video pack is decoded.

  That is, when the index parameter "j" is set to "1" (step ST60), an operation of reading the first sub-picture pack from the sub-picture buffer (memory 108) is started (step ST62). At this point, if the first sub-picture pack has not yet been stored in the memory 108 (No in step ST63; when the processing in step ST46 has not been performed), the pack data to be read is stored in the memory 108. Until the decoding process, the empty loop of the pack read operation (steps ST62 to ST63) is executed.

  If the first sub-picture pack is stored in the memory 108 (Yes in step ST63), the sub-picture pack is read and decoded (step ST64; see FIGS. 24 to 27 for a specific example of the decoding process). See below).

  The result of this decoding process (including, for example, uncompressed sub-picture data PXD as shown in the upper part of FIG. 6) is sent from the sub-picture decoder 101 of FIG. 8 to a display system (not shown) during the decoding process, and is decoded. A sub-picture corresponding to the data is displayed.

  If the display control end command (CMDEND in FIG. 22) has not been executed in the decoding process (No in step ST66), the index parameter “j” is incremented by one (step ST67).

  If the incremented j-th pack (here, the second pack) exists in the memory 108, the pack is read from the memory 108 and decoded (step ST64). The decoded j-th sub-picture pack (here, the second pack) is sent to the display system in the same manner as the first decoded pack, and the index parameter “j” is further incremented by one (step ST67). ).

  As described above, while incrementing the index parameter "j" (step ST67), one or more sub-picture packs stored in the memory 108 are continuously decoded (step ST64), and the decoded sub-picture data is decoded. The image display of the sub-video corresponding to (PXD) is executed.

  If a display control end command (CMDEND in FIG. 22) is executed in the decoding process (YES in step ST66), the decoding process of the sub-picture data in the sub-picture buffer (memory 108) ends.

  The above decoding process (steps ST62 to ST64) is repeated unless the end command CMDEND is executed (No in step ST66). In this embodiment, the decoding process ends when the end command CMDEND is executed (Yes in step ST66).

  FIG. 26 is a flowchart illustrating an example of the pack disassembly process of FIG. The sub-picture decoder 101 skips the pack header (see FIG. 3) from the transferred pack to obtain a packet (step ST72). If the packet does not have a time stamp PTS (No in step ST74), the packet header (PH) is deleted, and only the sub-picture unit data (PXD) is stored in the buffer (eg, 121) of the sub-picture decoder (step ST74). ST76).

  When the packet has a time stamp PTS (Yes in step ST74), only the PTS is extracted from the packet header (PH), and the extracted PTS is connected to the sub-picture unit data (30). The data is stored in the buffer 121 (step ST78).

  FIGS. 27 and 28 are flowcharts illustrating an example of the sub-picture decoding process of FIG. Among the decoding processes, for example, the process of returning the compressed data PXD shown in the lower part of FIG. 6 to the non-compressed data PXD shown in the upper part of FIG. 6 has been described with reference to FIGS.

  That is, sub-picture packets in a pack of sub-picture information corresponding to a predetermined reproduction time, that is, to which the SCR of the same time is added, are sequentially transferred to the sub-picture decoder 101 for one sub-picture unit, and according to the SCR. The sub-picture decoder 101 starts from a value at which the count of the main STC 120a becomes zero after a predetermined time, and checks whether or not the PTS added to the head packet of the sub-picture data unit has been transferred (step ST80). As a result, if the PTS has been transferred, the sub-picture decoder 101 compares the PTS with the count value of the main STC 120a (step ST81), and if the PTS has not been transferred, stands by until the PTS is transferred. ing.

  As a result of this comparison, when the count value of main STC 120a is larger than the PTS, sub-picture decoder 101 checks whether or not the PTS added to the first packet of the next sub-picture data unit has been transferred (step ST82). If the count value of the main STC 120a is smaller than the PTS, the main STC 120a waits until it becomes the same.

  As a result, if the PTS added to the first packet of the next sub-picture data unit has been transferred, the sub-picture decoder 101 compares the PTS with the count value of the main STC 120a (step ST83).

  If the PTS of the next sub-picture data unit is not transferred in step 82, or if the count value of the main STC 120a is smaller than the PTS of the next sub-picture data unit in step 83, the sub-picture decoder 101 The video data unit is determined to be the current one (step ST84), and the process proceeds to step 86.

  That is, since the PTS of the first sub-picture data unit is "0", when the count value of the main STC 120a becomes "0", the sub-picture data unit to be decoded is determined, and the routine proceeds to step 86.

  If the count value of the main STC 120a is larger than the PTS of the next sub-picture data unit in step 83, the sub-picture decoder 101 determines the next sub-picture data unit to be decoded (step ST85), and proceeds to step 86. .

  Next, the sub-picture decoder 101 checks whether or not the time stamp SPDCTS (“0”) recorded in the first DCSQT (DCSQT0 in FIG. 16) of the display control sequence table 33 has been transferred (step ST86). . If the time stamp SPDCTS has been transferred, the sub-picture decoder 101 compares the time stamp SPDCTS with the count value of the sub STC 120b (step ST87). If not transferred, the process returns to step 81.

  Then, as a result of the comparison in step 87, if the count value of the sub STC 120b is larger than the time stamp SPDCTS, the sub-picture decoder 101 checks whether or not the time stamp SPDCTS of the next display control sequence DCSQ has been transferred (step S87). ST88). As a result, when the time stamp SPDCTS is transferred, the sub-video decoder 101 compares the time stamp SPDCTS with the count value of the sub STC 120b (step ST89).

  If the time stamp SPDCTS of the next display control sequence DCSQ is not transferred in step 88, or if the count value of the sub STC 120b is smaller than the time stamp SPDCTS of the next display control sequence DCSQ in step 89, the sub-picture decoder 101 , The display control sequence DCSQ to be decoded is determined to be the current one (step ST90), and the process proceeds to step 92.

  If the count value of the sub STC 120b is larger than the time stamp SPDCTS of the next display control sequence DCSQ in step 89, the sub-picture decoder 101 determines the next display control sequence DCSQ to be decoded (step ST91), and step 92. Proceed to.

  Next, the sub-video decoder 101 starts decoding the pixel data corresponding to the display control sequence DCSQ to be decoded by using the respective commands (step ST92).

  For example, the display position and display area of the sub-picture are set by the command SETDAREA, the display color of the sub-picture is set by the command SETCOLOR, and the contrast of the sub-picture with respect to the video main picture is set by the command SETCONTR.

  Then, from the execution of the display start timing command STADSP until the execution of the display end timing command STPDSP in another display control sequence DCSQ, the display control in accordance with the switching command CHGCOLCON is performed, and the run-length compressed pixel data is executed. PXD (32) is decoded.

  Further, the sub-video decoder 101 checks whether or not all the display control commands COMMAND corresponding to the display control sequence DCSQ have been transferred up to the display control end command CMDEND (step ST93).

  As a result, if not transferred, the sub-picture decoder 101 clears the count value of the sub STC 120b, returns to step 81, decodes the first display control sequence DCSQ again, and if transferred, returns to step 81. Go to 94.

  In this step 94, the sub-picture decoder 101 checks whether or not there is the next display control sequence DCSQ, and if there is the next display control sequence DCSQ, the process returns to step 88 to decode the next display control sequence DCSQ. If there is no next display control sequence DCSQ, the process returns to step 82 and the process for the next sub-picture data unit is performed.

  As described above, the initial value of the main timer becomes “0” after a predetermined time in consideration of the transfer time of the pixel data. Zero can be described without considering the transfer time of the pixel data.

  Further, since the display of the sub-picture is prohibited until the main timer first changes to zero, the function of distinguishing between the negative value and the positive value of the main timer can be omitted.

  Further, if the decoding process is interrupted in the middle of the process for the desired display control sequence DCSQ, the decoding process is performed again from the first display control sequence DCSQ0, thereby causing a display omission due to the interruption of the decoding process. Display collapse can be prevented.

  In the above example, the initial value of the main timer is set to “0” after a predetermined time in consideration of the transfer time of the pixel data. However, the present invention is not limited to this. Considering this, the value may become “0” after a predetermined time. In this case, the same effect as above can be achieved.

FIG. 1 is a diagram schematically showing a recording data structure of an optical disk as an example of an information storage medium to which the present invention can be applied. FIG. 2 is a view exemplifying a logical structure of data recorded on the optical disc of FIG. 1. FIG. 3 is a diagram illustrating a logical structure of a sub-picture pack to be encoded (run-length compression, addition of a display control sequence table, and the like) in the data structure illustrated in FIG. 2. FIG. 4 is a diagram exemplifying the contents of a sub-picture data portion of the sub-picture pack illustrated in FIG. 3 to which the encoding method according to the embodiment of the present invention is applied; In the case where the pixel data constituting the sub-picture data portion illustrated in FIG. 4 is composed of a plurality of bits (here, 2 bits), compression rules 1 to 6 employed in the encoding method according to the embodiment of the present invention. FIG. In the case where the pixel data constituting the sub-picture data portion illustrated in FIG. 4 is composed of 2 bits, the compression rules 1 to 6 employed in the encoding method according to one embodiment of the present invention will be specifically described. FIG. The flow from mass production to reproduction on the user side of a high-density optical disk having encoded image information will be described; and the flow of encoded image information from broadcast / cable distribution to reception / reproduction by a user / subscriber will be described. FIG. FIG. 2 is a block diagram illustrating an embodiment (non-interlaced specification) of decoder hardware that executes image decoding (such as run-length expansion) based on the present invention. FIG. 7 is a view for explaining two examples (non-interlaced display and interlaced display) of how a character pattern “A” is decoded among encoded pixel data (sub-picture data). FIG. 9 is a flowchart for explaining software executed by the MPU (112) in FIG. 8 for performing image decoding (run-length decompression) according to the embodiment of the present invention. FIG. 11 is a flowchart illustrating an example of the contents of a decoding step (ST105) used in the software of FIG. 10. FIG. 4 is a block diagram illustrating a case where compressed data reproduced from a high-density optical disk having encoded image information is broadcast or distributed as it is, and the broadcast or cable distributed compressed data is decoded by a user or a subscriber. FIG. 11 is a block diagram illustrating a case where encoded image information is transmitted and received between any two computer users via a communication network (such as the Internet). FIG. 2 is a block diagram illustrating an outline of an optical disc recording / reproducing apparatus on which encoding and decoding are performed. FIG. 7 is a diagram illustrating how the buffering state of a sub-picture data unit changes depending on a sub-picture channel having a time stamp (PTS) when decoding sub-picture data according to the present invention. FIG. 4 is a view for explaining the data structure of a sub-picture packet. FIG. 4 is a diagram for explaining the position of a time stamp (PTS) in a sub-picture data unit. The figure which illustrates the correspondence relationship with the time stamp (PTS) and display control sequence (DCSQ) described in the packet header of one of the sub-video units arranged in series. FIG. 5 is a view for explaining a sub-picture size and a start address (a relative address pointer of DCSQ) of a display control sequence table among parameters included in a sub-picture unit header (SPUH) of FIG. 3 or FIG. 4. The figure explaining the structure of a sub-picture display control sequence table (SPDCSQT). The figure explaining the content of each parameter (DCSQ) which comprises the table (SPDCSQT) of FIG. The figure explaining the content of the display control command (SPDCCMD) of a sub-picture. FIG. 17 is a view showing a specific example of a display control sequence table in the packet shown in FIG. 16. FIG. 9 is a flowchart illustrating an example of a sub-picture encoding processing procedure of the present invention, centering on processing of a display control sequence (DCSQ). FIG. 25 is a flowchart illustrating an example of a procedure for performing parallel processing of pack decomposition and decoding of the sub-picture data stream encoded by the processing procedure of FIG. 24. FIG. 26 is a flowchart illustrating an example of the pack disassembly process of FIG. 25. FIG. 26 is a flowchart for explaining an example of the sub-picture decoding process in FIG. 25. FIG. 26 is a flowchart for explaining an example of the sub-picture decoding process in FIG. 25.

Explanation of reference numerals

  DESCRIPTION OF SYMBOLS 1 ... File management information 2 ... Video data PH ... Packet header 30 ... Sub-picture unit 31 ... Sub-picture unit header SPUH 32 ... Sub-picture pixel data PXD 33 ... Display control sequence table DCSQT 101 ... Decoder 102 ... Data I / O 103: Code data separation unit 104: Pixel color output unit (FIFO type) 105: Memory control unit 106: Continuous code length detection unit 107: Run length setting unit 108: Memory 109: Address control unit 110: Display validity permission unit 111 ... Insufficient pixel color setting section 112... Microcomputer (MPU or CPU) 113... Header separating section 114... Line memory 115... Selector 118... Select signal generating section 120. Buffer memory

Claims (3)

Sub-picture information that can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a fixed data size, the sub-picture packet includes a packet header,
The packet header includes a time stamp representing a display time, and the sub-picture data unit further includes a display control sequence table including a display control sequence, pixel data of a run-length compressed sub-picture, and a sub-picture unit header,
The sub-picture unit header includes start address information of the display control sequence table,
The run-length compression is performed by converting a continuation of the same pixel data of the sub-picture into data units,
The display control sequence includes information indicating a start time for display control of a sub-picture, address information of a next display control sequence, one or more sub-picture display control commands, and parameter data set by the sub-picture display control command. ,
As the sub-picture display control command, a command to set a display start timing of the sub-picture pixel data or a command to set a display end timing of the sub-picture pixel data can be set,
The configuration is such that the number of extension fields corresponding to the parameter data set by the command to set the display start timing and the command to set the display end timing is set to 0 bytes in advance,
A sub-picture encoding method characterized by encoding pixel data of the sub-picture. Sub-picture information that can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a fixed data size, the sub-picture packet includes a packet header,
The packet header includes a time stamp representing a display time, and the sub-picture data unit further includes a display control sequence table including a display control sequence, pixel data of a run-length compressed sub-picture, and a sub-picture unit header,
The sub-picture unit header includes start address information of the display control sequence table,
The run-length compression is performed by converting a continuation of the same pixel data of the sub-picture into data units,
The display control sequence includes information indicating a start time for display control of a sub-picture, address information of a next display control sequence, one or more sub-picture display control commands, and parameter data set by the sub-picture display control command. ,
As the sub-picture display control command, a command to set a display start timing of the sub-picture pixel data or a command to set a display end timing of the sub-picture pixel data can be set,
An information storage medium recorded and configured so that the number of extension fields corresponding to the parameter data set by the command for setting the display start timing and the command for setting the display end timing is set to 0 bytes in advance. And reproducing the sub-picture information from the sub-picture information. Sub-picture information that can be reproduced simultaneously with the main picture information constitutes a sub-picture data unit,
The sub-picture data unit is divided into packets of a fixed data size, the sub-picture packet includes a packet header,
The packet header includes a time stamp representing a display time, and the sub-picture data unit further includes a display control sequence table including a display control sequence, pixel data of a run-length compressed sub-picture, and a sub-picture unit header,
The sub-picture unit header includes start address information of the display control sequence table,
The run-length compression is performed by converting a continuation of the same pixel data of the sub-picture into data units,
The display control sequence includes information indicating a start time for display control of a sub-picture, address information of a next display control sequence, one or more sub-picture display control commands, and parameter data set by the sub-picture display control command. ,
As the sub-picture display control command, a command to set a display start timing of the sub-picture pixel data or a command to set a display end timing of the sub-picture pixel data can be set,
A decoder for the sub-picture information, wherein the number of extension fields corresponding to the parameter data set by the command for setting the display start timing and the command for setting the display end timing is set to 0 bytes in advance. A playback device comprising:

Images