KR100215620B1 - System for transmitting and receiving video information - Google Patents

Abstract

It reduces display spatial and display time waste of sub-picture data.

Sub-picture information 32 including packet header information (PH) including a time stamp representing the reproduction start time of the sub-picture packet, and pixel data (P × D) compressed by a predetermined method as a sub-picture. And display control order information 33 including one or more display control order DCSQTs for controlling the order in which the sub-pictures are displayed using the sub-picture information 32, the size of the sub-picture packet and the display control order information ( The sub-picture packet is encoded using the sub-picture header information SPUH including the position of 33).

The content of the encoded sub-picture packet is decoded at the reproduction time and displayed based on the display control sequence DCSQT.

Description

Image information transmission and reception system

1 is a diagram showing a recording data structure of an optical disc as an example of an information holding medium to which the present invention can be applied.

FIG. 2 is a diagram illustrating a logical structure of data recorded on the optical disc of FIG.

FIG. 3 is a diagram illustrating a logic structure of a sub-picture 되는 that is encoded (execution length compression, addition of a display control order table, etc.) among the data structures illustrated in FIG.

4 is a diagram illustrating the contents of a sub-picture data portion to which an encoding method according to an embodiment of the present invention is applied, of the sub-picture pack illustrated in FIG.

5 is a compression rule employed as an encoding method according to an embodiment of the present invention in the case where pixel data constituting the sub-picture data portion illustrated in FIG. 4 is composed of a plurality of bits (here 2 bits). Drawings explaining 1-6.

6 illustrates compression rules 11 to 15 employed in the encoding method according to another embodiment of the present invention when the pixel data constituting the sub-picture data portion illustrated in FIG. 4 is composed of 1 bit. drawing.

FIG. 7 shows pixel data constituting the sub-picture data portion illustrated in FIG. 4, for example, comprising first to ninth lines, and two-bit pixels (up to four types) are arranged on each line. In the case where the character patterns "A" and "B" are represented by two-bit pixels on each line, a diagram specifically explaining how pixel data of each line is encoded (execution length compression).

FIG. 8 is a diagram illustrating two examples (non-interlaced display and interlaced display) of how the character pattern "A" is decoded among pixel data (sub-picture data) encoded in the example of FIG.

FIG. 9 specifically shows compression rules 1 to 6 employed as an encoding method according to an embodiment of the present invention when the pixel data constituting the sub-picture data portion illustrated in FIG. 4 is composed of 2 bits. Drawing to explain.

10 illustrates the flow from mass production of high density optical discs with encoded image information to reproduction on the user side, based on the present invention; Block diagram illustrating a flow from transmission / cable distribution of encoded image information to reception / reproduction by a user / subscriber based on the present invention.

Fig. 11 is a block diagram for explaining an embodiment (noninterlacing rule) of decoder hardware that executes image decoding (execution length extension, etc.) based on the present invention.

12 is a block diagram for explaining another embodiment (interlacing specification) of decoder hardware for executing picture decoding (execution length decompression portion) based on the present invention.

FIG. 13 is a flowchart for explaining software executed by the encoder 200 of FIG. 10 as executing the image encoding (the execution length compression portion) according to the embodiment of the present invention.

FIG. 14 is a flowchart for explaining an example of the contents of the encoding step (ST806) used with the software of FIG.

FIG. 15 is a flowchart for explaining software executed by the MPU 112 of FIG. 11 or 12 as executing image decoding (execution length extension portion) according to an embodiment of the present invention.

FIG. 16 is a flowchart for explaining an example of the contents of the decoding step (ST10O5) used in the software of FIG.

FIG. 17 is a block diagram for explaining another embodiment of decoder hardware that executes image decoding (execution length extension, etc.) based on the present invention. FIG.

18 is a flowchart for explaining the first half of an image decoding (execution length decompression portion) process according to another embodiment of the present invention.

19 is a flowchart for explaining the second half of the image decode (execution length decompression portion) process according to another embodiment of the present invention.

20 is a flowchart for explaining an example of the contents of the encoding header detection step (ST205) in FIG.

Fig. 2 is a flowchart for explaining how the image decoding process of the present invention is carried out when the decoded image is shifted the screen.

Fig. 22 is a block for explaining the case where compressed data reproduced from a high density optical disc having image information encoded based on the present invention is broadcast or cable-distributed as it is and broadcast or cable-distributed compressed data is decoded at the user or subscriber side. Degree.

Fig. 23 is a block diagram for explaining the case where the encoded wave information based on the present invention is transmitted and received between any two computer users via a communication network (internet, etc.).

Fig. 24 is a block diagram illustrating an outline of an optical disc recording / reproducing apparatus on which encoding and decoding are executed according to the present invention.

25 is a diagram illustrating a state in which an encoder based on the present invention is ICized.

26 illustrates a state in which a decoder based on the present invention is ICized.

27 is a diagram illustrating a state where an encoder and a decoder based on the present invention are ICized.

FIG. 28 is a diagram for explaining the position of a time stamp (PTS) in a subpicture data block. FIG.

29 is a diagram for explaining the data structure of a sub-picture packet.

30 is a diagram illustrating a correspondence relationship between a serially arranged sub-picture unit and a time stamp (PTS) and a display control order (DCSQ) described in a packet header of one unit therein;

31 is a diagram for explaining a sub picture size and a start address (relative address pointer of DCSQ) of the display control order table among parameters included in the sub picture unit header (SPUH) in FIG. 3 or FIG.

32 is a diagram for explaining the configuration of a sub-picture display control procedure table (SPDCSQT).

33 is a view for explaining the contents of each parameter DCSQ constituting the table SPDCSQT in FIG. 32;

34 is a diagram for explaining the contents of a display control command (SPDCCMD) of a sub-picture.

35 is a view for explaining the contents of the pixel control data (PCD).

FIG. 36 is a view for explaining the bit structure of a command FSTADSP forcibly setting the display start timing of pixel data of a sub-picture among the command sets illustrated in FIG.

FIG. 37 is a view for explaining the bit structure of an instruction STADSP for setting the display start timing of pixel data of a sub-picture among the instruction sets illustrated in FIG.

FIG. 38 is a view for explaining the bit structure of the command STPDSP for setting the display end timing of the pixel data of the sub-picture among the instruction sets illustrated in FIG.

FIG. 39 is a view for explaining the bit structure of the command SETCOLOR for setting the color code of the pixel data of the sub-picture among the command sets illustrated in FIG.

FIG. 40 is a diagram for explaining an example of processing of color data in a sub-picture data processor (for example, the decoder 101 of FIG. 11).

FIG. 41 is a view for explaining the bit structure of the command SETDAREA for setting the contrast between the sub picture and the main picture among the command sets illustrated in FIG.

FIG. 42 is a view for explaining the bit structure of the command SETm_ [lambda] for setting the display area of sub-picture pixel data among the instruction sets illustrated in FIG.

FIG. 43 is a view for explaining the bit structure of the command SETDSPXA for setting the display start address of sub-picture pixel data in the command set illustrated in FIG.

FIG. 44 is a view for explaining the bit structure of the command CHGCOLCON for switching the color and contrast of sub-picture pixel data in the command set illustrated in FIG.

45 is a view for explaining the bit structure of the command CMDEND for ending the display control of the sub-picture among the set of command names illustrated in FIG. 34;

FIG. 46 is a view for explaining the bit structure of the line control information LCINF of the pixel line among the parameters of the pixel control data (PCD) illustrated in FIG.

FIG. 47 is a view for explaining the bit structure of the pixel control information PCINF among the parameters of the pixel control data (POD) illustrated in FIG. 35;

48 is a diagram for explaining a specific example of a sub-picture display frame.

FIG. 49 is a diagram for explaining in detail what is the content of each parameter of the pixel control data (PCD) in FIG. 35 when the sub-picture display frame is as shown in FIG.

FIG. 50 is a diagram for explaining a problem in the case of processing bitmap data of a sub-picture without using the present invention. FIG.

FIG. 51 is a diagram for describing the problem when the sub-image is processed without using the present invention. FIG.

FIG. 52 is a view for explaining how the buffering state of a sub-picture data block changes with a sub-picture channel having a time stamp (PTS) in the case of decoding the sub-picture data according to the present invention.

53 is a flowchart for explaining an example of the sub-picture encoding processing procedure of the present invention centering on the processing of the display control procedure (DCSQ).

FIG. 54 is a flowchart for explaining an example of a procedure of parallel processing a pack decomposition and decoding of a sub-picture data stream encoded in the processing sequence of FIG. 53; FIG.

55 is a flowchart for explaining an example of the pack decomposition process in FIG. 54;

FIG. 56 is a flowchart for explaining an example of the sub-picture decoding process of FIG. 54; FIG.

FIG. 57 is a flowchart for explaining another example of a procedure of parallel processing a pack decomposition and a decoding of a sub-picture data stream encoded in the processing sequence of FIG. 53;

Fig. 58 is a diagram explaining a recording method of sub-picture data (PXD) when the display mode of the sub-picture is the interlace mode.

59 is a diagram showing a specific example of the display control procedure table in the packet shown in FIG. 29;

Explanation of symbols for main parts of the drawings

1: file management information 2: video data

PH: packet header 30: sub-picture unit

31: Sub picture unit header SPUH 32: Pixel data PXD of sub picture

33: Display control procedure table DCSQT 101: Decoder

102: data I / 0 103: encoded data segment

104: pixel color output unit (FIFO type) 105: memory control unit

106: code length detection unit 107: execution length setting unit

108: memory 109: address control unit

110: display valid permission section 111: insufficient pixel color setting section

112: microcomputer (MPU or CPU) 113: header slice

1l4: Line memory 115: Selector

118: selection signal generator 120: system timer

121: buffer memory 1210: color register

1220 change color register 200

202: laser cutting device 204: optical disk master

206: Two-piece high density optical disk mass production facility

202 to 206: recording device 210: modulator / transmitter

212: broadcasting unit / cable output unit 300: disc player (playback device)

400: Receiver / Regenerator (Playback Device) 5001 (50ON): Personal Computer

5011 (501N): I / O devices 5021 (502N): External memory devices

5031 (503N): Encoder / Decoder & Mortem 702: Modulator / Laser Driver

704: optical head (recording laser) 706: optical head (read laser / laser mixup)

708: Rejuvenation of the original phase / error administration

710: audio / video data processing unit (including a decoding processing unit of sub-picture data)

OD: 2 sheets high density optical disc (recording medium)

The present invention relates to an improvement of a system for encoding and decoding image information such as sub-pictures which are reproduced simultaneously with a main image.

The present invention also relates to a data recording method, a recording medium, and the like, of a sub-image data supplied together with image data and simultaneously reproduced on a recording medium such as an optical disc.

Superimpose sub-pictures, such as subtitles of movies and images displayed as set values of television volume, are largely realized by two methods, a character code method and a bitmap data method.

In the character code method, a character, such as a character or a shape registered or prepared in advance, is kept in the character recording area of the character generator, and a code assigned to the character is given to the character generator to express a desired character.

In this method, dedicated hardware such as a character generator is required. However, since the character is displayed by assigning a code, the amount of data to be sent to the display system is smaller than that of sending the character's bitmap data to the display system as it is and displaying a sub-picture. . However, since only the characters prepared in advance can be displayed, the display use of the sub-picture by this system is limited.

On the other hand, in the bitmap data system, since the bitmap data of the subpicture is sent to the display system as it is, no dedicated hardware for generating the subpicture from the code is required. Since there is no limitation on the shape of the sub-image that can be displayed, the display usage of the sub-image is expanded.

In this system, however, the color data of the sub-picture, the sub-data contour color data necessary for superimposing the sub-pictures, and the superimposition blend ratio data of the main and sub-pictures must be sent to the display system for each pixel. The amount of data to do is huge.

In general, in the bitmap data system, since data about all pixels of the display screen (hereinafter referred to as a frame) is sent to the display system regardless of the size of the sub-picture, there is a lot of useless data in the display space (see FIG. 50). ).

In addition, even if either of the character code method or the bitmap data method does not change 0 in the type of the subpicture displayed, the subpicture data must be continuously provided at every display frame period, which is useful for display time. More data is missing (see Figure 51).

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and can greatly reduce the display space waste and display time waste of sub-picture data, and has excellent freedom of expression of sub-pictures. An object of the present invention is to provide an encode / decode system for securing image information.

In order to achieve the above object, in the first aspect of the present invention, there is provided a data recording method for packetizing a sub-picture data that can be simultaneously reproduced together with main video data in a predetermined unit and recording the data on a recording medium. A packet header information (PH) including a reproduction start time of a video data packet, a sub-picture data (DPX) which is a display content of a sub-picture, and a control procedure for displaying the sub-picture using this sub-picture data. The sub-picture packet is constituted by the above display control procedure information DCSQT and the sub-picture header information SPUH including the size of the sub-picture data packet and the recording position of the display control procedure information.

In a second invention, a data recording method for packetizing sub-picture data that can be simultaneously reproduced together with main video data in predetermined units and recording them on a recording medium, wherein at least a reference time is used to reproduce playback time of the sub-picture data packet. One or more pieces of display control procedure information indicating a packet header information (PH) including the?, Sub-picture data (PXD) that is the display content of the sub-picture, and a control procedure for displaying the sub-picture using the sub-picture data. DCSQT) and sub-picture header information (SPUH) including a size of the sub-picture data packet and a recording position of the display control procedure information, to configure the sub-picture packet; The reproduction time PTS of the sub-picture data packet in the packet header information is recorded only for the first sub-picture data packet in the sub-picture data block.

In the third invention, in the first or second invention, at least the display start time and the display end time of the sub-picture, the recording position of the sub-picture data to be displayed, and the display control for the sub-picture data recorded at this recording position. The information group constitutes the above display control procedure information (DCSQ in FIG. 33).

In the fourth invention, in the third invention, the display start time and the display end time of the sub-picture are defined as relative time from the playback start time of the sub-picture data packet.

In the fifth aspect of the invention, display start control information (the thirty-fourth) is used as control of starting display of the sub-picture data based on the display start time included in the display control procedure information as the display control information of the third invention. STADSP) is recorded.

In the sixth invention, as the display control information of the third invention, display end control information for performing control to end the display of the sub-picture data based on the display end time included in the display control procedure information (Thirty-fourth) STPDSP) is recorded.

In the seventh invention, color setting information (SETCOLOR in FIG. 34) for setting the color for each pixel type of the sub-picture data is recorded as the display control information of the third invention.

In the eighth invention, as the display control information of the third invention, mixing ratio setting information (SETCONTR in FIG. 34) for setting the mixing ratio for each pixel type of the sub-picture data for the main image is recorded.

In the ninth invention, display area setting information (SETDAREA in FIG. 34) for setting the display area of the sub-picture data on the main image is recorded as the display control information of the third invention.

In the tenth invention, use range setting information (SETDSPXA in FIG. 34) for setting a range to be used for display in the sub-picture data is recorded as the display control information of the third invention.

In the eleventh invention, as the display control information of the third invention, a change in color for each pixel type of the sub-picture data and a change in mixing ratio for each pixel type of the sub-picture data for the main image are set in units of pixels. Color / mixing ratio change setting information (CHGCOLCON in FIG. 34) for recording is recorded.

In the twelfth invention, display control end setting information (CMDEND in FIG. 34) for setting the end of display control of the sub-picture data is recorded as the display control information of the third invention.

In a thirteenth aspect of the present invention, in a recording medium in which sub-picture data that can be simultaneously reproduced together with main video data is packetized and recorded in predetermined units, at least one of the sub-picture data packets expressed by using a reference time is recorded. One or more display control procedures indicating a packet header information (PH) including a reproduction start time, a sub-picture data (PAD) which is the display content of the sub-picture, and a control procedure for displaying the sub-picture using this sub-picture data. Information DCSQT and sub-picture header information SPUH including the size of the sub-picture data packet and the recording position of the display control procedure information.

In a fourteenth aspect of the present invention, in a recording medium in which sub-picture data that can be simultaneously reproduced together with main video data is packetized and recorded in predetermined units, at least one of the sub-picture data packets expressed using a reference time is recorded. One or more display control procedures indicating a packet header information (PH) including a reproduction start time, a sub-picture data (PXD) that is a display content of a sub-picture, and a control procedure for displaying the sub-picture using this sub-picture data. Information (DCSQT), sub-picture header information (SPUH) including the size of the sub-picture data packet and the recording position of the display control procedure information, and further reproduce the sub-picture data packet in the packet header information. The time PTS is recorded only for the first subpicture data packet in the subpicture data block.

In the present invention, by using one or more display control sequence information (DCSQT), the display space waste and display time waste of the sub-picture data can be greatly reduced, and the freedom of expression of the sub-picture in the bitmap data system can be obtained. It can secure a wide range of sub-pictures.

That is, in the present invention, use range setting information (SETDSPXA) for setting a range to be used for display among sub-picture data is provided so as not to display data outside the use range. It can greatly reduce the display spatial waste of the amount of data generated when sending.

That is, in the present invention, the color setting information SETCOL0R and the mixing ratio setting information SETCONTR for each pixel type, such as pattern pixels, subsidiary lines, and background of the sub-picture data, are provided to form the sub-picture image as the sub-picture display data. As compared with the conventional method of having only the information and forgetting the mixing ratio information of the color information for each pixel, it is possible to guarantee the same degree of sub-image shape representation with a smaller data amount.

Further, in the present invention, color / mixing ratio change setting information (CHGCOLCON) is set for setting the color change of each subtype of the sub-picture data and the mixing ratio of the sub-picture data for each pixel type of the main image in pixel units. The dynamic display of the image can be realized with the same precision as that of the conventional bitmap data method, and with a smaller data amount than the bitmap data method.

In addition, the sub-picture is rarely changed in color information for each pixel, and there is no fear of excessive data amount of the color / mixing ratio change setting information itself.

In the present invention, even if the color of the sub-image is changed, the sub-image can be displayed over the display time of a plurality of frames using the same sub-image data as long as the shape does not change. Therefore, the display time waste of the sub-picture data can be greatly reduced as compared with the conventional method in which the sub-picture data must be continuously given to the display system at the frame period regardless of whether the color is changed.

EXAMPLE

EMBODIMENT OF THE INVENTION Hereinafter, with reference to drawings, the encoding / decoding system of image information which concerns on 1st Embodiment of this invention is demonstrated. In addition, in order to avoid description of the north-south, common reference numerals are used for functionally common parts across a plurality of drawings.

1 through 59 are diagrams for explaining an encoding / decoding system of image information according to an embodiment of the present invention.

1 shows a recording data structure of an optical disc OD as an example of an information holding medium to which the present invention can be applied.

The optical disk OD is, for example, a double-sided bonded disk having a storage weight of about 5G bytes on one side, and a plurality of recording tracks are arranged between the lead-in area on the inner circumferential side of the disk and the lead-out area on the outer circumferential side of the disk. It is arranged. Each track is composed of a plurality of logical colors, and various pieces of information (appropriate compressed digital data) are stored in each color.

FIG. 2 illustrates a data structure of a file for video (video) recorded on the optical disc OD of FIG.

As shown in FIG. 2, this video file includes file management information 1 and video data 2. The video data 2 is composed of a video data block, an audio data block, a sub-picture data block, and a DSI block in which information (DSI: Disk Search Information) necessary for controlling their data reproduction is recorded. Each block is divided into, for example, packets of one data size for each type of data. The video data blocks, audio data blocks and sub-picture data blocks are reproduced in synchronization with each other based on the DSI placed immediately before these block groups.

That is, a system area for storing system data used in the disk OD, a volume management information area and a plurality of file areas are formed in the aggregate of the plurality of logical color vectors in FIG.

Among the plurality of file areas, for example, the file 1 includes main video information (VIDEO in the drawing), sub-picture information (SUB-PICTURE in the drawing), and audio information (in the drawing) having contents supplementary to the main video. AUDIO), playback information (PLAYBACK INFO. In the drawing), and the like.

FIG. 3 illustrates a logical structure of a pack of sub-picture information encoded (execution length compression) of the data structure illustrated in FIG.

As shown in the upper part of Fig. 3, one pack of sub-picture information included in the video data is composed of, for example, 2048 bytes (2kB). One pack of this sub-picture information includes one or more sub-picture packets after the header of the first pack. The first sub-picture packet includes the execution length-compressed sub-picture data SP DATA1 after the header of the packet. Similarly, the second sub-picture packet includes the sub-length video data SP DATA2 compressed with execution length after the header of the packet.

The plurality of sub-picture data (SP DATA1, SP DATA2, ...) is collected for one unit (1 unit) of execution length compression, that is, the sub-picture unit header 31 is stored in the sub-picture data unit 30. Is given. After this sub-picture unit header 31, pixel data 32 of execution length-compressed image data (e.g., data for one horizontal line of a two-dimensional display screen) and display control procedure information of each sub-picture pack The table 33 containing continues.

In other words, the execution length compressed data 30 for one unit is formed of a set of sub-picture data portions (SP DATA1, SP DATA2, ...) of one or more sub-picture packets. The sub-picture data unit 3o includes a sub-picture unit header SPUH31 in which various parameters for sub-picture display are recorded, display data (compressed pixel data) PXD 32 composed of execution length codes, and a display control procedure table DCSQT. It consists of 33.

4 shows a part of the contents of the sub-picture unit header 31 among the execution length compressed data 30 for one unit shown in FIG. 3 (see FIG. 31 for other parts of SPUH31). ). Here, the data of the sub-picture (e.g., the subtitle corresponding to the scene of the movie of the main picture) which is recorded and transmitted (communicated) simultaneously with the main picture (e.g. the video main body of the movie) will be described.

As shown in FIG. 4, the sub-picture unit header SPUH 31 includes the start address SPDDADR of the pixel data (display data) 32 of the sub-picture, the end address SPEDADR of the pixel data 32, and the pixel data 32. The display size on the TV screen, i.e., the display start position and display range (width and height) SPDSZ, and the recording start position SPDCSQTA of the display control procedure table 33 in the sub-picture data packet are recorded.

In some cases, the SPUH 31 has a background color (SPCHI) designated by the system, a sub-picture color (SPCINFlO) designated by the system, a pallet color number (SPADJINFO) of an emphasis color designated by the system, and sub-picture pixel data ( 32), the mixing information SPCONT of the sub-picture SP with respect to the main video MP, the start timing of the sub-picture (corresponding to the frame number of the main video) (SPDST), and each line interval. The start addresses (SPLine1 to SPlineN) of the decode data may be recorded.

The start addresses (SPLine1 to SPlineN) of the decode data for each line interval are not included in the SPUH 31 in the preferred embodiment of the present invention, but are provided for a plurality of sub-picture fields.

More specifically, in the sub-picture unit header SPUH 31, various parameters (such as SPDDADR) having the following contents are recorded as shown in FIG. 4: (1) Display data (sub-zero) following this header Start address information (SPDDADR: relative address from the beginning of the header) of the pixel data on the image;

(2) end address information (SPEDADR: relative address from the head of the header) of this display data;

(3) information SPDSZ indicating a display start position and a display range (width and height) of the display data on the monitor screen;

(4) Recording start position information (display control procedure table start address (SPDCSQTA) of the sub-picture) of the display control procedure table 33 in the packet.

In addition, in one of various conceivable embodiments of the present invention, the sub-picture unit header SPUH 31 may include the followings:

(5) information SPCHI indicating a background color designated by the system (number of 16 color pallets set in the story information table or the display control procedure table);

(6) information (SPCINFO) indicating a sub-picture color designated by the system (number of the 16-color color pallet set in the story information table or the display control procedure table);

(7) information (SPAJDNFO) indicating the sub-image emphasis color (number of the color pallet set in the story information table or the display control procedure table) designated by the system;

(8) Sub-picture image mode information (SPMOD) designated by the system and indicating whether the non-interlaced field mode is an interlaced frame mode or the like, and (when pixel data to be compressed consists of various bits, pixel data The number of bits may be specified by the content of this mode information.);

(9) information (SPCONT) indicating a mixing ratio of a sub-image and a main image designated by the system;

(10) information SPDST indicating the display start timing of the sub-picture by the frame number (e.g., one picture plane number in MPEG) of the main video;

(11) Information (SPlin1 to SPlinN) indicating the start address (relative address from the head of the sub-picture unit header) of the encoded data on the first to the N-th lines of the sub-picture.

Further, the information SPCONT indicating the mixing ratio of the sub-image and the main image indicates the mixing ratio of the sub-image, for example, by (system set value) / 16 or (system set value) / 255, and (16-set value) / 16 or (255-set value). ) / 255 to indicate the mixing ratio of the sub-pictures.

In this sub-picture unit header 31 (or each sub-picture field), start addresses SPLine1 to SPlineN of decode data of each line interval exist. For this reason, by changing the designation of the decoding start line to an instruction from the microcomputer (MPU or CPU) on the decoder side, it is possible to realize a scroll of only the sub-picture on the display screen. (This screen movement will be described later with reference to FIG. 21.)

By the way, according to the embodiment of the present invention, the sub-picture unit header 31 can record the field / frame mode (SPMOD) indicating how the sub-picture corresponds to the NTSC-type TV field / frame.

Normally, bit 0 is recorded in this field / frame mode recording unit (SPMOD).

On the decoder side receiving such sub-picture data unit 30, it is determined by this bit 0 that the frame mode (non-interlace mode) is received, and the received code data is decoded for each line. By doing so, an image in a decoded state as illustrated at the bottom left of FIG. 8 is output from the decoder, which is displayed on a display screen such as a monitor or television (TV).

On the other hand, when bit 1 is recorded in the field / frame mode recording unit SPMOD, the decoder determines that the field mode (interlace mode) is present. In this case, after the sign data is decoded line by line, the same data is continuously output for two lines as illustrated in the lower right of FIG. By doing so, a screen corresponding to the interlace mode of the TV can be obtained. As a result, the image quality is rougher than that of the frame mode (non-interlaced mode), but it is possible to display an image of twice that amount with the same data amount as the frame mode.

The pixel data (execution length data) 32 of the sub-picture shown in FIG. 3 or 4 is any of the execution length compression rules 1 to 6 or the execution length compression rules 11 to 15 shown in FIG. As a rule is applied, the data length (variable length) of that unit is determined. Then, encoding (execution length compression) and decoding (execution length extension) are performed at the determined data length.

Rules 1 to 6 of FIG. 5 are used when the pixel data to be compressed have a multi-bit configuration (2 bits here), and rules 11 to 15 of FIG. 6 are used when the pixel data to be compressed are 1-bit configuration. .

Which of the execution length compression rules 1 to 6 or the execution length compression rules 11 to 15 is used depends on the contents (bit width flags, etc.) of the parameter SPMOD (see near the center of the table at the bottom of FIG. 4) in the sub-picture unit header 31. It can be decided by. For example, when the bit width flag of the parameter SPMOD is 1, the pixel data of the execution length compression target is 2-bit data, and the rules 1 to 6 in Fig. 5 are used. On the other hand, when the bit width flag of the parameter SPMOD is 0, the pixel data of the execution length compression target is 1 bit data, and the rules 11 to 15 in Fig. 6 are used.

Now, in the case where the pixel data can take 1, 2, 3 or 4 bit configurations, it is assumed that four types of compression rule groups A, B, C, and D are prepared corresponding to these bit configuration values. In this case, the parameter SPMOD is a 2-bit flag, the flag 0 specifies 1-bit pixel data using the rule group A, the flag 1 specifies 2-bit pixel data using the rule group B, and the flag 10 is ruled. 3-bit pixel data using the group C is specified, and 4-bit pixel data using the regular group D can be specified with the flag 11. Here, the rules 11-15 of FIG. 6 can be used for the compression rule group A, and the rules 1-6 of FIG. 5 can be used for the compression rule B. FIG. The compression rule groups C and D are obtained by appropriately changing the configuration header values and the number of rules of the encoding header, the number of continuous pixels, and the pixel data in FIG.

FIG. 5 is a diagram illustrating an embodiment of the present invention in the case where the portion of the sub-picture pixel data (execution length data) 32 illustrated in FIG. 4 is composed of a plurality of bits of pixel data (here 2 bits). The execution length compression rules 1 to 6 employed in the code method are explained.

9 is a diagram for specifically explaining the compression rules 1 to 6 in the case where the portion of the sub-picture pixel data (execution length data) 32 illustrated in FIG. 4 is composed of 2-bit pixel data. .

In Rule 1 shown in the first column of Fig. 5, when 1 to 3 identical pixels are continued, one unit of data of encoding (execution length compression) is composed of 4-bit data. In this case, the number of pixels continues to be represented by the first two bits, and the pixel data (such as color information of a pixel) is represented by two consecutive bits.

For example, the first compressed data unit CU01 of the video data PMXD before compression shown in the upper part of FIG. 9 includes two 2-bit pixel data d0, d1 = (0000) b (b indicates that it is binary). In this example, two identical two-bit pixel data 00b are consecutive.

In this case, as shown in the lower part of FIG. 9, d0 and d1 = (1000) b which can connect the 2-bit display 10b of the continuation number "2" and the content (00) b of the pixel data are compressed. It becomes the data unit CUO1 * of subsequent video data PXD.

In other words, the rule 1 converts (0000) b of the data unit CU01 to (1000) b of the data unit CU01 *. In this example, no substantial bit length compression is obtained. For example, if the same pixel (00) b is three consecutive CU01 = (000000) b, after compression, CU01 * = (1100) b, and two-bit compression is performed. The effect can be obtained.

In Rule 2 shown in the second column of FIG. 5, when 4 to 15 identical pixels are continued, 8-bit data constitutes one unit of encoded data. In this case, the first two bits represent an encoding header indicating the basis of Rule 2, the subsequent four bits represent the number of pixels, and the subsequent two bits represent pixel data.

For example, the second compressed data unit CU02 of the video data PXD before compression shown in the upper part of FIG. 9 includes five two-bit pixel data d2, d3, d4, d5, and d6 = (0101010101) b. In this example, five identical two-bit pixel data (01) are contiguous.

In this case, as shown in the lower part of Fig. 9, d2 to d6 = the concatenation of the encoding header (00b), the 4-bit display (0101) b of the continuation number "5", and the content (01) b of the pixel data = B is the data unit CU02 * of the compressed video data PXD.

In other words, according to rule 2, (0101010101) b (10-bit length) of the data unit CU02 is converted into (00010101) b (8-bit length) of the data unit CU02 *. In this example, the actual bit length compression is only 2 bits from 10 bits to 8 bits, but if the continuation number is 15 (30 bits long since 15 01 of CU02 is continuous), this is 8 bits of compressed data (CU02 * = 00111101), and a compression effect of 22 bits can be obtained for 30 bits. That is, the bit compression effect based on rule 2 is larger than that of rule 1. However, Rule 1 is also required to cope with the execution length compression of fine images having high resolution.

In Rule 3 shown in the third column of FIG. 5, when 16 to 63 identical pixels are continued, 12-bit data constitutes one unit of encoded data. In this case, the first 4 bits represent an encoding header indicating the basis of the rule 3, the subsequent 6 bits represent the number of pixels, and the subsequent 2 bits represent pixel data.

For example, the third compressed data unit CU03 of the video data PXD before compression shown in FIG. 9 includes 16 two-bit pixel data d7 to d22 = (101010 ... 1010) b. . In this example, 16 identical 2-bit pixel data 10b are contiguous.

In this case, as shown in the lower part of Fig. 9, d7 to d22 which can connect the encoding header (0000) b, the 6-bit display (010000) b of the continuation number "16", and the content (10) b of the pixel data. = (000001000010) b becomes the data unit CU03 * of the compressed video data PXD.

In other words, according to rule 3, (101010 ...) 10101) b (32-bit length) of the data unit CU03 is converted into (000001000010) b (12-bit length) of the data unit CU03 *. In this example, the actual bit length compression is 20 bits from 32 bits to 12 bits, but if the continuation number is 63 (126 bits long because 63 of CU03 is 63 consecutive), this is 12 bits of compressed data (CU03 * = 000011111110). ), A compression effect of 114 bits can be obtained for 126 bits. That is, the bit compression effect based on rule 3 is larger than that of rule 2.

In Rule 4 shown in the fourth column of Fig. 5, when 64 to 255 identical pixels are continued, 16-bit data constitutes one unit of encoded data. In this case, the first 6 bits indicate an encoding header indicating the basis of rule 4, the subsequent 8 bits indicate the number of pixels, and the subsequent 2 bits indicate pixel data.

For example, the fourth compressed data unit CU04 of the video data PXD before compression shown in the upper part of FIG. 9 includes 69 two-bit pixel data d23 to d91 = (111111 ... 1111) b. have. In this example, 69 identical 2-bit pixel data 11b are contiguous.

In this case, as shown in the lower part of FIG. 9, d23 to d91 which can connect the encoding header (000000) b, the 8-bit display (00100101) b of the continuation number "69", and the content 11b of the pixel data. = (0000000010010111) b becomes the data unit CU04 * of the compressed video data PXD.

In other words, according to rule 4, b (138 bit length) of the data unit CU04 (111111 ... 1111) is converted into (0000000010010111) b (16 bit length) of the data unit CU04 *. In this example, the actual bit length compression is 122 bits from 138 bits to 16 bits, but in the case of continuity numbers, for example, 255 (510 bits long since 11 of CU01 are 255 consecutive), this is 16 bits of compressed data (CU04 * = 0000001111111111 ), And a compression effect of 494 bits can be obtained for 510 bits. That is, the bit compression effect based on rule 4 is larger than that of rule 3.

In Rule 5 shown in the fifth column of Fig. 5, when the same pixel continues from the switching point of the encoded data unit to the end of the line, 16-bit data constitutes one unit of encoded data. In this case, the first 14 bits represent an encoding header indicating the basis of Rule 5, and the subsequent two bits represent pixel data.

For example, the fifth compressed data unit CU05 of the video data PXD before compression shown in the upper part of FIG. 9 includes one or more two-bit pixel data d92 to dn = (000000 ·········· 0000) b Doing. In this example, the same two-bit pixel data (00) is continuous in a fixed number, but in rule 5, the number of pixels may be one or more.

In this case, as shown in the lower part of FIG. 9, the encoding header (00000000000000) b and d92 to dn = (0000000000000000) b which can connect the content (00) b of the pixel data are the data of the compressed video data PXD. Unit CU05 *.

In other words, the rule 5 converts (000000 ... b) (unspecified bit length) of the data unit CU05 into (0000000000000000) b (16 bit length) of the data unit CU05 *. In Rule 5, the compression effect can be obtained when the same number of pixels continuing up to the line increment is longer than 16 bits.

In Rule 6 shown in the sixth column of FIG. 5, the length of the compressed data PXD for one line is not an integer multiple of 8 bits (i.e., no byte alignment) when the pixel line on which the encoding target data is arranged ends one line. In this case, 4 bits of dummy data are added so that one line of compressed data PXD is in bytes (that is, byte aligned).

For example, the total bit length of the data units CU of * -CU05 * of the compressed video data PXD shown in the lower part of FIG. 9 is necessarily an integer multiple of 4 bits, but is not necessarily limited to an integer multiple of 8 bits. .

For example, if the total bit length of the data units CU01 * to CU05 * is 1020 bits and four bits are insufficient for the byte alignment, the four-bit dummy data CU06 * = (0000) b is replaced as shown in the lower part of FIG. In addition to the end of 1020 bits, the byte-aligned data units CU01 * to CU06 * of 1024 bits are output.

In addition, 2-bit pixel data is not necessarily limited to displaying four types of pixel colors. For example, the pixel data 00b represents the background pixel of the sub-image, the pixel data 01b represents the pattern pixel of the sub-image, and the pixel data 10b represents the first emphasis pixel of the sub-image, and the pixel data 11 b) may represent the second emphasis pixel of the sub-picture.

If the number of constituent bits of the pixel data is larger, different types of sub-picture pixels can be designated. For example, when the pixel data is composed of three bits (000) b to (111) b, up to eight types of pixel colors + pixel types (emphasis effect) can be specified for the execution length encoded / decoded sub-picture data. Will be.

FIG. 6 is used as an encoding method according to another embodiment of the present invention when the sub-picture pixel data (execution length data) 32 illustrated in FIG. 4 is composed of 1-bit pixel data. The execution length compression rules 11 to 15 are described.

In Rule 11 shown in the first column of FIG. 6, when 1 to 7 identical pixels are continued, one unit of data (encoding of execution length compression) is encoded into 4-bit data. In this case, the number of pixels continues to be represented by the first three bits, and the pixel data (such as pixel type information) is represented by one subsequent bit. For example, if 1-bit pixel data is 0, it represents the background pixel of the sub-image, and if it is 1, it represents the pattern pixel of the sub-image.

In Rule 12 shown in the second column of FIG. 6, when 8 to 15 identical pixels are continued, 8-bit data constitutes one unit of encoded data. In this case, the first three bits represent a code and a header (for example, 000) indicating the basis of the rule 12, the number of pixels continues to be represented by four consecutive bits, and the pixel data is represented by one subsequent bit.

In Rule 13 shown in the third column of Fig. 6, when 16 to 127 identical pixels are continued, one unit of encoded data is composed of 12-bit data. In this case, the first 4 bits represent an encoding header (e.g., 0000) indicating that the rule 13 is based on Rule 13, the number of pixels continues in 7 bits, and the pixel data in 1 bit thereafter.

In Rule 14 shown in the fourth column of Fig. 6, when the same pixel continues from the switching point of the encoded data unit to the end of the line, 8-bit data constitutes one unit of encoded data. In this case, an encoding header (e.g., 0000000) indicating that the first 7 bits are based on the rule 14 is shown, and pixel data is represented by one subsequent bit.

In Rule 15 shown in the fifth column of FIG. 6, the length of the compressed data PXD for one line is not an integer multiple of 8 bits (i.e., no byte alignment) when the pixel line on which the encoding target data is arranged ends one line. In this case, 4-bit dummy data is added so that the compressed data PXD for one line is in bytes (that is, byte aligned).

Next, an image encoding method (encoding method using execution length compression encoding) will be described in detail with reference to FIG.

7 shows pixel data constituting the sub-picture pixel data (execution length data) 32 illustrated in FIG. 4, for example, comprising first to ninth lines, each pixel having a 2-bit configuration (maximum). It has a case where four types of contents) are arranged, and the character patterns "A" and "B" are represented by 2-bit pixels on each line. In this case, how pixel data of each line is encoded (execution length compression) will be specifically described.

As illustrated at the top of Fig. 7, the image serving as the source is composed of three types (up to four kinds) of pixel data. That is, the pixel color of the background of the sub-picture is shown in the 2-bit image data (00), and the pixel color of the characters "A" and "B" in the sub-picture is shown in the 2-bit image data (01), and 2 In the bit image data 10b, the emphasis pixel colors for the sub-picture characters "A" and "B" are shown.

When the original images including the letters A " and " B " are scanned by a scanner or the like, these character patterns are read in units of one pixel from left to right for each scanning line.

The video data read in this way is input to an encoder (200 in the embodiment of FIG. 10 described later) which performs execution length compression based on the present invention.

The encoder can be configured as a microcomputer (MPU or CPU) on which software for executing execution length compression based on the rules 1 to 6 described in FIG. 5 is operated. This encoder software will be described later with reference to the flowcharts of FIGS. 13 and 14.

An encoding process of execution length compression of the sequential bit strings of the character patterns "A" and "B" read in units of one pixel will be described below.

In the example of FIG. 7, it is assumed that there are three pixel colors of the source image, so that the image data (sequential bit strings of the character patterns "A" and "B") to be encoded has a background pixel color of "0". The bit pixel data (00) b is represented, the character pixel color "#" is represented by two-bit pixel data (01) b, and the emphasis pixel color "o" is represented by the two-bit pixel data 10b. The number of bits (= 2) of this pixel data (00,01, etc.) may be referred to as pixel width.

For the sake of simplicity, in the example of FIG. 7, the display width of the encoded video data (sub-picture data) is 16 pixels, and the number of scanning lines (the height of the display) is 9 lines.

First, the pixel data (sub picture data) obtained from the scanner is first converted into an execution length value before compression by a microcomputer.

That is, taking the first line of the upper part of FIG. 7 as an example, three associations "..." are converted into (* 3), and one subsequent "0" is (0 * 1). Is converted to (# * 1), the subsequent one "o" is converted to (0 * 1), and the subsequent three consecutive "‥" are (** 3). ), The subsequent one "o" is converted to (0 * 1), the subsequent four consecutive "####" is converted to (# * 4), and the next one "o" is Is converted to (0 * 1), and the last one "·" is converted to (** 1).

As a result, as shown in the center of Fig. 7, the pre-compression execution length data of the first line is " * 3/0 * 1 / # · 1/0 * 1 / * 3/0 * 1 / # * 4/0 * 1 / * 1 ". This data is constituted by a combination of image information such as a character pixel color and the number of continuous pixels indicating the number of consecutive pixels.

Similarly, the pixel data strings in the second to ninth lines of the upper part of FIG. 7 become the precompression execution length data strings as shown in the second to ninth lines of the middle part of FIG.

If attention is paid to the data of the first line, the compression rule 1 of Fig. 5 is applied since the line start rotor background pixel colors " 3 " are continued. As a result, the first "..." of the first line, i.e., (* 3), is set to (1100) in which two bits 11 representing "3" and (00) representing a background pixel color "..." are combined. Is coded.

The next data on the first line is also rule 1 because there is one "o". As a result, the next "o" of the first line, i.e., (0 * 1), is encoded by combining (0110) a combination of two bits (01) representing "1" and (10) representing the emphasis pixel color "o". .

In addition, rule 1 is also applied to the following data because there is one "#". As a result, the next "#" of the first line, that is, (# * 1), is encoded as (0101) in which two bits (01) representing "1" and (01) representing the character pixel color "#" are combined. (The part about this # is shown in the middle and lower part of Fig. 7, surrounded by broken lines.)

Likewise below, (0 * 1) is encoded as (0110), (** 3) is encoded as (1100), and (0 * 1) is encoded as (0110).

Since the data after the first line has four "#", the compression rule 2 of FIG. 5 is applied. As a result, this "# "(# * 4) of the first line is a 2-bit header (00) indicating that rule 2 is applied, a 4-bit (0100) indicating the number of pixels "4", and a character pixel color. It is encoded as (00010001), which combines (01) representing "#". (The part about # is shown surrounded by broken lines.)

Rule 1 is applied to the subsequent data on the first line because "o" is one. As a result, this "o", i.e., (0 * 1), is encoded with (0110) in which two bits (01) representing "1" and (10) representing the emphasis pixel color "o" are combined.

As the last data of the first line, "1" is one, rule 1 is applied. As a result, this "" (** 1) is a two-bit (01) representing "1" di and a background pixel color "." It is encoded as (0100) combining (00) representing.

As described above, the execution length data of the first line before compression "* 3/0 * 1 / # * 1/0 * 1 / * 3/0 * 1 / # * 4/0 * 1 / * 1" Execution length is compressed as in (1100) (0110) (0101) (0110) (1100) (0110) (00010001) (0110) (0100), and the encoding of the first line ends.

In the same manner below, encoding proceeds to the eighth line. In the ninth line, all one line is occupied by the same background pixel color ". &Quot; In this case, compression rule 5 of FIG. 5 is applied. As a result, the precompression execution length data "* 16" of the 9th line shows the 14-bit header (00000000000000) which shows that the same background pixel color "..." continues to the end of a line, and the background pixel color "*". It is encoded with 16 bits (0000000000000000) which combines two bits of pixel data (00).

The encoding based on Rule 5 also applies to the case where the data to be compressed continues from the middle of the line to the end of the start line.

FIG. 10 explains the flow from mass production to reproduction on the user side of a high-density optical disc having encoded image information (31 + 32 + 33 in FIG. 3) based on the present invention; It is a block diagram explaining the flow from broadcast / cable distribution to reception / reproduction by a user / subscriber of encoded image information based on the present invention.

For example, if the pre-compression execution length data as shown in the middle part of FIG. 7 is input to the encoder 200 of FIG. 10, the encoder 200 is input by software processing based on the compression rules 1 to 6 of FIG. Data is run-length compressed (encoded).

When data having a logical configuration as shown in FIG. 2 is recorded on the optical disc OD as shown in FIG. 1, the execution length compression process (encoding process) by the encoder 200 of FIG. 3 degrees of sub-image data are performed.

The encoder 200 of FIG. 10 also receives various data necessary for completing the optical disc OD. These data are compressed, for example, based on the specification of the Motion Picture Expert Group (MPEG), and the digital data after compression is sent to the laser cutting machine 202 or the modulator / transmitter 210.

In the laser cutting machine 202, MPEG compressed data from the encoder 200 is cut into a mother disk, not shown, to produce an optical disk master 204.

In the two-pack high-density optical disk mass production facility 200, the master 204 is released, and the master information is transferred to a laser light reflecting film on a polycarbonate substrate having a thickness of 0.6 mm, for example. A large quantity of two polycarbonate substrates on which separate master information has been transferred are bonded together to form a double-sided optical disc (or single-sided double-sided disc) of 1.2 millimeters in thickness.

The bonded high-density optical disc OD produced at the facility 206 is distributed to various markets and is near the user.

The distributed disk OD is reproduced by the playback device 300 of the user. The apparatus 300 includes a decoder 101 for restoring data encoded by the encoder 200 to original information. The decoded information by the decoder 101 is sent to the user's monitor TV and imaged, for example. In this way, the last user can see the original video information from the mass distributed disc OD.

On the other hand, the compressed information sent from the encoder 200 to the modulator / transmitter 210 is modulated and transmitted according to a predetermined standard. For example, the compressed image information from the encoder 200 is satellite broadcasted 212 simultaneously with the corresponding audio information. Alternatively, the compressed image information from the encoder 200 is cabled 212 simultaneously with the corresponding audio information.

Broadcast or cabled compressed video / audio information is received by the receiver / recovery unit 400 of the user or subscriber. The receiver / circular norther 400 has a decoder 101 for restoring the data encoded by the encoder 200 to the original information. The decoded information by the decoder 101 is sent to the user's monitor TV and imaged, for example. In this way, the last user can view the original video information from the compressed video information transmitted by the broadcast or cable transmission.

FIG. 11 is a block diagram showing one embodiment (noninterlacing rule) of decoder hardware for executing image decoding (execution length extension) based on the present invention. The decoder 101 (see FIG. 10) which decodes the execution length compressed sub-picture data SPD (equivalent to the data 32 of FIG. 3) can be configured as shown in FIG.

Hereinafter, with reference to FIG. 11, a sub-picture data decoder for extending the execution length of a signal including the execution length compressed pixel data in the format as shown in FIG. 4 will be described.

As shown in Fig. 11, this sub-picture decoder 101 includes data I / 0 102 to which sub-picture data SPD is inputted; A memory 108 for storing sub-picture data SPD; A memory control section 105 which controls the read / write operation of the memory 108; From the execution information of the code data (execution length compressed pixel data) read out from the memory 108, a continuous code length (encoding header) of one unit (one block) is detected, and the continuation of outputting the truncated information of the code length is continued. A code length detection unit 106; A code data cutting unit 103 for extracting code data for one block in accordance with the information from the code length detecting unit 106; A signal showing execution information in units of one compression unit as output from the code data splitting section 103 and a "0" of the data bit as the output from the code length detection section 106 are the heads of the code data for one block. An execution length setting unit 107 which receives a signal (period signal) showing the number of consecutive "0" bits from the number of continuous lines, and calculates the number of continuous pixels of one block from these signals; A pixel color output unit 104 (Fast-in / Fast) that receives pixel color information from the code data cutting unit 103 and a period signal output from the execution length setting unit 107, and outputs color information for that period. -out type); A microcomputer 112 that reads the header data (see FIG. 4) of the sub-picture data SPD data read out from the memory 108, and performs various processing settings and control based on the read data; An address control unit 109 for controlling the read / write address of the memory 108; An insufficient pixel color setting unit 111 in which color information on a line where execution information does not exist is set by the microcomputer 112; And a display valid permission unit 110 for determining a display area when displaying a sub-picture on a TV screen or the like.

In addition, although mentioned in the description of FIGS. 53-57, the system timer 120 and the buffer 121 are connected to the MPU 112 of the decoder 101. As shown in FIG.

The above description will be described again as a separate description as follows. That is, as shown in FIG. 11, the execution length compressed sub-picture data SPD is sent to the bus inside the decoder 101 through the data I / 0 102. As shown in FIG. The data SPD sent to the bus is sent to the memory 108 through the memory control section 105 and stored there.

In addition, the internal bus of the decoder 101 is connected to the code data cutting unit 103, the code length detection unit 106, and the microcomputer (MPU or CPU) 112.

The sub picture unit header 31 of the sub picture data read out from the memory 108 is read by the microcomputer 112. The microcomputer 112 sets the decoding start address SPDDADR in the address control unit 109 based on the various parameters shown in FIG. 4 from the read header 31 and in the display valid permission unit 110. The display start position of the sub-picture, the display width and the information (SPDSIZE) of the display height are set, and the display width (the number of dots on a line) of the sub-picture is set in the code data interceptor 103. The set various pieces of information are stored in the internal registers of the units 109, 110, and 103. After that, the various information stored in the register can be accessed by the microcomputer 112.

The address control unit 109 accesses the memory 108 via the memory control unit 105 based on the decoding start address SPDDADR set in the register, and starts reading the sub-picture data to be decoded. In this way, the sub-picture data read out from the memory 108 is provided to the code data cutting section 103 and the continuation code length detecting section 106.

The encoding header (2 to 14 bits in the rules 2 to 5 in FIG. 5) of the execution length compressed sub-picture data SPD is continuously detected by the code length detection unit 106, and the number of continuous pixels of the same pixel data in the data SPD is determined. The execution length setting unit 107 is detected based on the signal from the continuation code length detection unit 106,

In other words, the code length detection unit 106 counts the number of zero bits of data read from the memory 108 and detects the code and the header (see FIG. 5). The detecting unit 106 sends the segment information SEP. To the code data cutting unit 103 according to the detected code and the value of the header. INFO. To give.

The code data cutoff section 103 is assigned cutoff information SEP.INFO. According to this, the number of pixels (execution information) is continuously set in the execution length setting unit 107, and the pixel data (SERARATED DATA (here, pixel color)) is set in the pixel color output unit 104 of the FIFO type.

At that time, the code data cutting unit 103 counts the number of pixels of the sub-picture data, and compares the pixel count count value with the display width (number of pixels on one line) of the sub-picture.

If the decode of one line is not byte aligned (that is, the data bit length of one line is not a multiple of 8), the sign data interceptor 103 reads the last four bits of data on that line. When you code, it is regarded as appended dummy data and truncated.

The execution length setting unit 107 transmits to the pixel color output unit 104 on the basis of the number of continuous pixels (execution information), the pixel dot clock DOTCLK, and the horizontal / vertical synchronization signal H-SYNC / V-SYNC. And a signal PERIODSIGNAL for outputting pixel data. Then, the pixel color output unit 104 decodes the pixel data from the code data cutting unit 103 while the pixel data output signal PERIODSIGNAL is active (that is, during the period of outputting the same pixel color). Output as.

At that time, when the decoding start line is changed by the instruction from the microcomputer 112, a line without execution information may exist. In that case, the insufficient pixel color setting unit 111 gives the pixel color output unit 104 data of the preset insufficient pixel color to the pixel color output unit 104. Then, the line data without the execution information becomes the code data intercepting unit. While being given to 103, the pixel color output unit 104 outputs the insufficient pixel color data (COLOR INFO.) From the insufficient pixel color setting unit 111.

That is, in the case of the decoder 101 of FIG. 11, if there is no image data in the inputted sub-picture data SPD, the microcomputer 112 sets the insufficient pixel color information in the auxiliary pixel color setting section 111.

The pixel color output unit 104 displays a display enable signal for determining which position on the monitor screen, not shown, to display the decoded sub-picture in synchronization with the horizontal / vertical synchronization signal of the sub-picture. It is provided from the display activator 110. In addition, a color conversion signal is sent from the permission unit 110 to the output unit 104 based on the color information instruction from the microcomputer 112.

The address control unit 109 is configured to process the memory control unit 105, the code length detection unit 106, the code data cutting unit 103, and the execution length setting unit 107 after the processing is set by the microcomputer 112. Address data and various timing signals are sent.

When the pack of sub-picture data SPD is input via the data I / O section 110, and it is stored in the memory 108, the contents (decode start address, decode end address, display start position) of the pack header of this data SPD , Display width, display height, etc.) are read by the microcomputer 112. The microcomputer 112 sets the decode start address, the decode end address, the display start position, the display width, the display height, and the like in the display valid permission unit 110 based on the read contents. At this time, the number of bits of compressed pixel data (in this case, 2 bits of pixel data) can be configured to form the contents of the sub-picture unit header 31 of FIG.

The operation of the decoder 101 of FIG. 11 will be described below in the case where the compressed pixel data has a 2-bit configuration (the usage rule is the rules 1 to 6 in FIG. 5).

When the decoding start address is set by the microcomputer 112, the address control unit 109 sends address data corresponding to the memory control unit 1o5, and continuously sends a read start signal to the code length detection unit 106. FIG.

The code length detection unit 106 then sends a read signal to the memory control unit 105 in response to the read start signal that has been sent to read the encoded data (compressed sub-picture data 32). In the detection unit 106, it is checked whether all of the upper two bits of the read data are "0".

If it is not "0", it is determined that the block length of the compression unit is 4 bits (see Rule 1 in Fig. 5).

If it (the upper two bits) is "0", the next two bits (the upper four bits) are checked.

If it is not "0", the block length of the compression unit is determined to be 8 bits (see Rule 2 in Fig. 5).

If it (upper 4 bits) is "uh", then the following 2 bits (upper 6 bits) are checked.

If it is not "0", the block length of the compression unit is determined to be l2 bits (see Rule 3 in Fig. 5).

If it (the upper six bits) is "0", the next eight bits (upper 14 bits) are checked.

If it is not "0", the block length of the compression unit is determined to be 16 bits (see Rule 4 in Fig. 5).

If it (the upper 14 bits) is "0", it is determined that the block length of the compression unit is 16 bits and the same pixel data continues to the end of the line (see Rule 5 in Fig. 5).

If the number of bits of the pixel data read to the end of the line is an integer multiple of 8, the state is left as it is, and if it is not an integer multiple of 8, it is determined that 4 bits of dummy data are required at the end of the read data in order to realize byte alignment. (See Rule 6 in Figure 5).

The code data cutting unit 103 continuously extracts one block (one compression unit) of the sub-picture data 32 from the memory 108 on the basis of the determination result by the code length detection unit 106. In the dividing section 03, the extracted one-block data is further divided into the number of pixels and the pixel data (such as pixel color information). The data of the number of segmented successive pixels (RUNINFO.) Is sent to the execution length setting section 107, and the segmented pixel data (SEPARATEDDATA) is sent to the pixel color output section 104.

On the other hand, the display valid permission unit 110, in accordance with the display start position information, the display width information and the display height information received from the microcomputer 112, the pixel dot clock (PIXEL-DOT CLK) and horizontal synchronization supplied from the outside of the apparatus. In synchronism with the signal H-SYNC and the vertical synchronizing signal V-SYNC, a display enable signal (enable signal) specifying a sub-picture display period is generated. This display permission signal is output to the execution length setting unit 107.

The execution length setting unit 107 outputs a signal indicating whether or not the current block data continues to the end of the line as output from the code length detection unit 106, and the continuation pixel data (RUN INFO) from the code data cutting unit 103. .) Is sent. The execution length setting unit 107 determines the number of pixel dots that the block in decoding is responsible for based on the signal from the detecting unit 106 and the data from the cutting unit 103, and during the period corresponding to the number of dots. And the display permission signal (output enable signal) to the pixel color output unit 104.

The pixel color output unit 104 is enabled during the reception of the period signal from the execution length setting unit 107, and during this period, the pixel color information received from the code data cutting unit 103 is converted into the pixel dot clock PIXEL-. In synchronization with the DOT CLK), the decoded display data is sent to a display device or the like not shown. That is, display data such as the pixel pattern continuous dot number of blocks during decoding is output from the pixel color output unit 104.

In addition, when the code length detection unit 106 determines that the encoded data is the same pixel color data until the end of the line, the code length detection unit 106 continuously outputs a signal for 16-bit code length to the code data cutting unit 103, and executes the execution length setting unit ( 107) outputs a signal indicating the same pixel color data until the end of the line.

When the execution length setting unit 107 receives the signal from the detection unit 106, it outputs the pixel color such that the color information of the encoded data continues to be enabled until the horizontal synchronizing signal H-SYNC becomes inactive. The output 104 outputs an output enable signal (period signal).

In addition, when the microcomputer 112 changes the decoding start line to shift the display contents of the sub-picture, there is a possibility that no data line for decoding use exists in the preset display area (that is, the decode line is insufficient). There is this.

In order to cope with such a case, the decoder 101 of Fig. 11 prepares in advance pixel color data which fills up an enumerated line. When the line shortage is actually detected, the display mode of the insufficient pixel color data is switched. Specifically, when the data end signal is provided from the address control unit 109 to the display valid permission unit 110, the permission unit 110 sends a color switching signal COLOR SW SIGNAL to the pixel color output unit 104. In response to this switching signal, the pixel color output unit 104 switches the decode output of the pixel color data from the code data to the decode output of the color information (COLOR INF0.) From the insufficient pixel color setting unit 110. . This switching state is maintained during the display period of the under line (DISPLAY ENABLE = active).

When the line shortage occurs, instead of using the insufficient pixel color data, the decoding processing operation can be stopped during that time.

Specifically, for example, when a data end signal is input from the address control unit 109 to the display valid permission unit 110, a color conversion signal for designating a stop of display from the permission unit 110 to the pixel color output unit 104 is output. Just do it. In such a case, the pixel color output unit 104 stops displaying the sub-image during the active display period designation color change signal.

8 illustrates two examples (non-interlaced display and interlaced display) of how the character pattern "A" is decoded among pixel data (sub-picture data) encoded as an example of FIG.

The decoder 101 of FIG. 11 can be used in the case of decoding compressed data as shown in the upper part of FIG. 8 into non-interlaced display data as shown in the lower left of FIG.

On the other hand, in the case of decoding the compressed data as shown in the upper part of FIG. 8 into the interlace display data as shown in the lower right of FIG. 8, a line doubler (for example, scanning the same pixel line twice) (for example, Line # 10 having the same content as line # 1 of the odd field is rescanned in the even field; V-SYNC unit switching) is required.

In the case of non-interlaced display of an image display amount equivalent to interlace display, another line doubler (e.g., line # 10 having the same content as line # 1 in the lower right of FIG. 8 is continued to line # 1; H- SYNC unit switching).

Fig. 12 is a block diagram illustrating an embodiment (interlacing rule) of decoder hardware having the function of the above line doubler. The decoder 101 of FIG. 10 can also be configured as a decoder of FIG.

In the structure of FIG. 12, the microcomputer 112 detects the generation timing of the odd field and even field of the interlace display based on the horizontal / vertical synchronization signal of the sub-picture.

When the odd field is detected, the microcomputer 112 gives the selection signal generation unit 118 a mode signal indicating "current odd field". By doing so, a signal for selecting the decoded data from the decoder 101 is output from the selection signal generator 118 to the selector 115. Then, pixel data (see lower right in FIG. 8) of the odd field lines # 1 to # 9 is sent out from the decoder 101 through the selector 115 to the outside as a video output. At this time, the pixel data of the lines # 1 to # 9 of these odd fields are stored in the line memory 114 once.

Upon detecting the shift to the even field, the microcomputer 112 gives the selection signal generation unit 118 a mode signal indicating "is the current even field". By doing so, a signal for selecting the one stored in the line memory 114 is output from the selection signal generator 118 to the selector 115. Then, pixel data (see lower right in FIG. 8) of lines # 10 to # 18 of even fields is sent out from the line memory 114 to the outside as a video output via the selector 115.

In this way, the sub-picture image of the lines # 1 to # 9 of the odd field (character "A" in the example of FIG. 8) and the sub-picture image of the lines # 10-# 18 of the even field (character "A" in FIG. 8). ) Is combined to realize interlace display.

By the way, in the sub-picture unit header 31 of the sub-picture data shown in FIG. 4, parameter bits SPMOD indicating the frame display mode / field display mode of the TV screen are provided.

A noninterlaced display of an image display amount equivalent to interlaced display is, for example, as follows.

When the microcomputer 11 of FIG. 12 reads the sub-picture unit header 31, it is determined whether the interlaced mode (active "1") is set from the set value (active = "1"; inactive = "0") of the parameter SPMOD. Whether it is in interlace mode (non-active "0" can be determined.

In the configuration of FIG. 12, when the parameter SPMOD is active = " 1 ", the microcomputer 112 detects that it is in the interlaced mode, and sends the mode signal indicating the interlaced mode to the selection signal generator 118. The generation unit 118 receiving this mode signal gives the selector 115 a switching signal for each occurrence of the horizontal synchronizing signal H-SYNC. The selector 115 then alternates between the decode output (DECODEDMTA) of the current field from the sub-picture decoder 101 and the decode output of the current field temporarily stored in the line memory 114 for each occurrence of the horizontal synchronization signal H-SYNC. Switch and send the video output to an external TV.

As described above, when the current decoded data and the decoded data in the line memory 114 are switched for each H-SYNC and outputted, twice the density (horizontal scanning line is twice the original image (decoded data)) on the TV screen. The image with) is displayed in interlaced mode.

In the sub-picture decoder 101 having such a configuration, data is not decoded after being read for one line, but the bit data inputted sequentially is read out from 2 to 16 bits while being counted one bit from the beginning of the decode data unit block to decode. do. In this case, the bit length (4 bits, 8 bits, 12 bits, 16 bits, etc.) of one unit of decode data is detected immediately before decoding. Then, the pixel data compressed in the detected data length unit is reconstructed (reproduced) in real time into three types of pixels ("*", "o", "#" in the example of FIG. 7, for example).

For example, in decoding pixel data encoded according to the rules 1 to 6 of FIG. 5, the sub-picture decoder 101 may include a bit counter and a relatively small data buffer (line memory 114, etc.). In other words, the circuit configuration of the sub-picture decoder 101 can be made relatively simple, and the whole apparatus including this encoder can be miniaturized.

That is, the encoder of the present invention does not require a large code table in the decoder as in the conventional MH coding method, and also eliminates the need to read data twice at the encoding time as in the arithmetic encoding method. In addition, the decoder of the present invention does not require relatively complicated hardware such as a multiplier, and a counter and a small buffer can be implemented by addition of a simple circuit.

According to the present invention, the execution length compression / encoding and the execution length extension / decoding of various kinds of pixel data (up to four types in a 2-bit configuration) can be realized with a relatively simple configuration.

FIG. 13 is a flowchart for explaining the software executed by the encoder 200 of FIG. 10 as executing the image encoding (the execution length compression) according to the embodiment of the present invention.

A series of encoding processes in which the execution of FIG. 5 is based on the compression rules 1 to 6 is executed as software processing by a microcomputer of the encoder 200 shown in FIG. The entire encoding process by the encoder 200 can be performed in accordance with the flow of FIG. 13, and the execution length compression of the pixel data in the sub-picture data can be performed in accordance with the flow of FIG. 14. (In this case, the display of FIG. The encoding of the control sequence table DCSQT 33 is not referred to. The encoding of the ∞S 33 portion will be described later with reference to FIG. 53.)

In this case, the computer inside the encoder 200 first prepares the header area of the sub-picture data when the line number and the dot number of the image data are designated by key input or the like (step ST801), and sets the line count number to "0". Initialize (step ST802).

When the pixel patterns are input one by one, the computer inside the encoder 200 acquires the pixel data of the first one pixel (two bits in this case), saves the pixel data, and sets the pixel count to "1". At the same time, the dot count number is set to "1" (step ST803).

Subsequently, the internal computer of the encoder 200 acquires the pixel data (two bits) of the next pixel pattern and compares it with the pixel data in storage which has been input before one (step ST804).

As a result of this comparison, if the pixel data is not the same (NO in step ST805), the encoding conversion process 1 is performed (step ST806), and the current pixel data is stored (step ST807). The pixel count number is increased by +1, and the dot count number is also increased by +1 in correspondence with this (step ST808).

If the pixel data is the same as a result of the comparison in step ST804 (YES in step ST805), the encoding conversion processing 1 in step ST8O6 is skipped and the process proceeds to step ST808.

After the increase in the pixel count number and the dot count number (step ST808), the internal computer of the encoder 200 checks whether the pixel line currently being encoded is the end (step ST809). If the line is terminated (YES in step ST809), encoding conversion processing 2 is performed (step ST810). If the line is not terminated (NO in step ST809), the process returns to step ST804, and the processing of steps ST804 to ST808 is repeated.

After the encoding conversion processing 2 of step ST810 is finished, the computer inside the encoder 200 checks whether the bit string after encoding is an integer multiple of 8 bits (byte aligned) (step ST81lA). If the bytes are not aligned (NO in step ST811A), 4-bit dummy data 0000 is added to the end of the bit string after encoding (step ST811B). After the dummy addition process or after the encoded bit string is byte aligned (YES in step ST811A), the line counter of the computer in the encoder (such as a general register in the microcomputer) is increased by +1 (step ST812).

If the last line is not reached after the line counter is incremented (NO in step ST813), the process returns to step ST803, and the processes of steps ST803 to ST812 are repeated.

If the last line is reached after the line counter is incremented (YES in step ST813), the encoding process (here, execution length compression of the bit string of the 2-bit pixel data) ends.

14 is a flowchart for explaining an example of the contents of the encode conversion process 1 in FIG.

In the encoding conversion process 1 (step ST806) in FIG. 13, since the encoding target pixel data is 2 bits wide, the execution length compression rules 1 to 6 in FIG. 5 are applied.

Corresponding to these rules 1 to 6, the pixel count number is 0 (step ST901), the pixel count number is 1 to 3 (step ST902), the pixel count number is 4 to 15 (step 1 ST903), and the pixel count number is 16 to Determination is made by computer software to determine whether it is 63 (step ST904), the pixel count number is 64 to 255 (step ST905), the pixel count value indicates the end of the line (step ST906), or the pixel count is 256 or more (step ST907). All.

The internal computer of the encoder 200 determines the number of bits (one unit length of pixel data of the same type) of the execution field based on the determination result (step ST908 to ST913), after the sub-picture unit header 31. The number of bits of this execution field bit is reserved. The number of pixels continues to be output to the execution field secured in this way, the pixel data is output to the pixel field, and recorded in the storage device (not shown) inside the encoder 200 (step ST914).

FIG. 15 is a flowchart for explaining software executed by the microcomputer 112 of FIG. 11 or FIG. 12, which executes image decoding (mounting length extension) according to an embodiment of the present invention. The decoding of the display control procedure table DCSQT 33 in Fig. 3 is not mentioned. The decoding of the DCSQT 33 portion will be described later with reference to Figs. 54 to 57.)

FIG. 16 is a flowchart for explaining an example of the contents of the decode step STl005 used in the software of FIG.

In other words, the microcomputer 112 reads the first header 31 portion of the execution length-compressed sub-picture data (pixel data has a 2-bit configuration), and interprets the contents (see FIG. 4). And based on the contents of the parsed header, the data is decoded based on the number of lines and dots. If the number of lines and the number of dots are specified (step ST1001), the line count number and dot count number are initialized to "0" (step ST1002 to step ST1003).

The microcomputer 112 sequentially executes a data bit string subsequent to the sub-picture unit header 31, and counts the number of dots and the number of dot counts. Then, the number of dots is subtracted from the number of dots, and the number of pixels is continuously calculated (step ST1004).

When the number of pixels continues to be calculated in this way, the microcomputer 112 performs decoding processing in accordance with the value of the number of pixels continued (step ST1005).

After the decoding process of step ST1005, the microcomputer 112 adds the dot count number and the continuing pixel number, and makes this the new dot count number (step ST1006).

Then, the microcomputer 112 executes the decoding process of step ST1005 by sequentially inputting data, and when the accumulated dot count number coincides with the line end number (position of end of line) initially set, one line is used. The decoding process is terminated with respect to the data of (step ST1007 example).

Next, if the decoded data is byte aligned (YES in step ST1008A), the dummy data portion is removed (step ST1008B). The number of line counts is increased by +1 (step ST1009), and the processes of steps ST1002 to ST1009 are repeated until the final line is reached (NO in step ST1010). When the last line is reached (YES in step ST1010), the decoding ends.

The processing contents of the decoding processing step ST1005 of FIG. 15 are shown in FIG. 16, for example.

In this process, two bits are acquired from the beginning, and the determination as to whether or not the bit is "0" is repeated (steps ST1101 to ST1109). As a result, the number of continuous pixels corresponding to the execution length compression rules 1 to 6 of FIG. 5, that is, the number of continuous executions is formed (steps ST1110 to ST1113).

After the execution continuation number is determined, two bits subsequently read into the pixel pattern (pixel data; color information of the pixel) (step ST1114).

When the pixel data (color information of the pixel) is determined, the index parameter "i" is set to 0 (step ST1115), and the 2-bit pixel pattern is output until the parameter "i" matches the number of execution consecutive numbers (step ST1116). (Step ST1117), the parameter " i " is increased by +1 (step ST1118) to stop the output of one unit of the same pixel data and end the decoding process.

As described above, according to the decoding method of the sub-picture data, the decoding processing of the sub-picture data ends with simple processing of determination processing of several bits, segmentation processing of data blocks, and counting processing of data bits. For this reason, a large code table used in the conventional MH encoding method or the like is not necessary, and the processing and configuration for decoding the encoded bit data into the original pixel information are simplified.

In the above embodiment, the code bit length for one unit of the same pixel can be determined by reading up to 16 bits of bit data during data decoding, but this code bit length is not limited to this. For example, this code bit length may be 32 bits or 64 bits. However, as the bit length increases, a larger data buffer is required.

Further, in the above embodiment, the pixel data (pixel color information) is set to three color information selected from, for example, sixteen color palettes. In addition, three primary colors (red component R, green component G, and blue component) B; or luminance signal component Y, chroma red signal component Cr, chroma blue signal component Cb, and the like, respectively, amplitude information may be represented by 2-bit pixel data. That is, the pixel data is not limited to a specific kind of color information.

FIG. 17 shows a modification of FIG. In FIG. 11, the microcomputer 112 cuts out the header in software. In FIG. 17, the header cutting out is performed in hardware inside the decoder 101. In FIG.

That is, as shown in FIG. 17, the execution length compressed sub-picture data SPD is sent to the bus inside the decoder 101 via the data I / OC102. The data SPD sent to the bus is sent to and stored in the memory 108 through the memory control section 105. The majority bus of the decoder 101 is connected to a code data cutting unit 103, a continuation code length detecting unit 106, and a header cutting unit 113 connected to a microcomputer (MPU or CPU) 112. .

The sub-picture unit header 31 of the sub-picture data read out from the memory 108 is read by the header cutting section 113. The cutout part 1313 sets the decoding start address SPDDADR in the address control part 109 based on the various parameters shown in FIG. 4 from the read header 31, and adds it to the display valid permission part 110. The display start position of the image, the information (SPDSZ) of the display width and the display height are set, and the display width (number of dots on a line) of the sub-picture is set in the code data interceptor 103. The set various pieces of information are stored in the internal registers of the respective parts l09, l10, 103. After that, the various information stored in the register can be accessed by the microcomputer 112.

The address control unit 109 accesses the memory 108 via the memory control unit 105 based on the decoding start address SPDDADR set in the register and starts reading the sub-picture data to be decoded. In this way, the sub-picture data read out from the memory 108 is provided to the code data cutting section 103 and the continuation code length detecting section 106.

The encoding header (2-14 bits in the rules 2-5 of FIG. 5) of the execution length-compressed sub-picture data SPD is continuously detected by the code length detection unit 106, and the number of continuous pixels of the same pixel data in the data SPD is obtained. The execution length setting unit 107 is detected based on the signal from the continuation code length detection unit 106.

A decoding method other than the decoding method described with reference to FIGS. 15 and 16 will now be described with reference to FIGS. 17 to 21.

18 is a flowchart for explaining the first half of an image decode (execution length extension) process according to another embodiment of the present invention.

When starting decoding, each block inside the decoder 101 of FIG. 17 is initialized (clearing the register, resetting the counter, etc.). Thereafter, the sub-picture unit header 31 is read, and the contents (various parameters in FIG. 4) are set in an internal register of the header segmentation section 113 (step ST1200).

When various parameters of the header 31 are set in the register of the header segmentation section 113, the microcomputer 112 is notified of the state that the reading of the header 31 is complete (step ST1201).

When the microcomputer 112 receives the header read end state, it designates a decoding start line (for example, SPLine1 in FIG. 4), and notifies the header segmentation section 113 of the start line (step ST1202).

When receiving the notification of the designated decode start line, the header segmentation section 113, based on various parameters of the header 31 set in its own register, the address (SPDDADR in FIG. 4) and the decode end address of the designated decode start line. (SPEDADR in FIG. 4; an address shifted by one line relative to the starting line address) is set in the address control section 109, and the display start position, display width, display height and (SPDSIZE in FIG. 4) of the decoded sub-picture are Set in the display valid permission unit 110, the value of the display width CLNEPIX; Although not shown in FIG. 4, the number of pixels for one line included in the SPDSIZE) is set in the code data splitting section 103 (step ST1203).

The address control unit 109 sends the decode address to the memory control unit 105.

In doing so, the data to be decoded (compressed sub-picture data SPD) is read out from the memory 108 to the encoded data segment 103 and the code length detection section 106 via the memory control section 105. At that time, the read data is set in each of the loan registers of the cutting section 103 and the detecting section 106 in units of bytes (step ST1204).

The code length detection unit 106 continues to count the number of 0 bits of data read out from the memory 108, and detects an encoding header corresponding to any one of the rules 1 to 5 in FIG. ST1205). Details of this encoding header detection will be described later with reference to FIG.

Then, the code length detection unit 106 generates the segment information SEP.INF0. Corresponding to any one of the rules 1 to 5 in FIG. 5 according to the detected code and the value of the header (step ST1206).

For example, if the count value of the 0 bit of the data read out from the memory 108 is zero, the slice information SEP.INFO. Is generated, and this count value is 2, the interstitial information SER. INF0. Is generated, and if this count is 4, the slice information indicating rule 3 is SEP.INFO. Is generated, and if this count is 6, the cutoff information SEP.INF0. Is generated, and if this count is 14, the cutoff information indicating Rule 5 is SEP.INFO. Is generated. The cut information generated in this way SEP.INFO. Is transmitted to the encoded data slicer 103.

The coded data cut-out section 03 is divided into pieces of cut-off information SEP.INFO. In accordance with the contents of the above, the number of pixels PIXCNT (execution information) is set in the execution length setting unit 107, and the 2-bit pixel data (pixel data; Data) is set in the pixel color output unit 100. At this time, inside the cutting section 103, the current count value NOWPIX of the pixel counter (not shown) is continuously increased by the number of pixels PIXCNT (step ST1207).

19 is a flowchart for explaining the second half (after node A in FIG. 18) of the image decode (execution length extension) process according to another embodiment of the present invention.

In the preceding step ST1203, the encoded data segmentation section 103 is informed of the number of pixel data (dots) LNEPIX for one line corresponding to the display width of the sub-picture from the header segmentation section 1113. The encoded data splitting section 103 checks whether the value NOWPIX of the internal pixel counter exceeds the number LNEPIX for the notified one-line pixel data (step ST1208).

In this step, when the pixel counter value NOWPIX is equal to or larger than the number of pixel data LNEPIX for one line (NO in step ST1208), the internal register of the segmentation section 103 in which one-byte data is set is cleared, and the pixel is clear. The counter value NOWPIX becomes zero (step ST1209). At this time, in the case of byte alignment, the 4-bit data is truncated. When the pixel counter value NOWPIX is smaller than the number of pixel data LNEPIX for one line (YES in step ST1208), the internal register of the segmentation section 103 is not cleared and remains as it is.

The execution length setting unit 107 includes the number of continuous pixels PIXCNT (execution information) set in the preceding step ST1207, the dot clock DOTCLK for determining the transfer speed of the pixel dots, and the horizontal and vertical for synchronizing the sub-image to the display screen of the main image. From the synchronization signals H-SYNC and V-SYNC, a display period signal PERIOD SIGNAL for outputting the pixel data set in the pixel color output unit 104 for a necessary period is generated. The generated display period signal is provided to the pixel color output unit 104 (step ST1210).

The pixel color output unit 104 displays the decoded data (e.g., pixel data representing the pixel color) set in the preceding step ST1207 while the display period signal is given from the execution length setting unit 107. Output as data (step ST121l).

The sub-picture display data output in this way is later synthesized with the image of the main video appropriate for the circuit portion, not shown, and displayed on the TV monitor, not shown.

If the decoded data has not finished after the pixel data output processing in step ST1211, the process returns to step ST1204 in FIG. 18 (NO in step ST1212). Whether or not the decoded data is finished is determined whether or not the data up to the end address (SPEDADR) of the sub-picture display data set by the header segmentation section 113 has been processed and finished in the encoding data segmentation section 103. can do.

If the decoding of the data has ended (YES in step ST1212), it is checked whether or not the display permission signal DISPLAYENABLE from the display valid permission unit 110 is active. The display valid permission unit 110 generates a display permission signal in an active state (for example, a high level) until the data end signal DATA END SIGNAL is sent from the address control unit 109.

If the display permission signal is active, it is determined that the data decode has ended and is still in the display period (YES in step ST1213). In this case, the display valid permission unit 110 sends a color conversion signal to the execution length setting unit 107 and the pixel color output unit 104 (step ST1214).

At this time, the pixel color output unit 104 receives the insufficient pixel color data from the insufficient pixel color setting unit 111. Upon receiving the color conversion signal from the display valid permission unit 110, the parsing color power unit 104 converts the output pixel color data into the under pixel color data from the under pixel color setting unit 111 (step ST1215). Then, while the display permission signal is active (loop of steps ST1213 to ST1215), the display area of the sub-picture is displayed in the under-pixel color provided by the under-pixel color setting unit 1111 during the display period of the sub-picture without decode data. Is filled.

If the display permission signal is inactive, it is determined that the display period of the decoded sub-picture has ended (NO in step ST12l3). If so, the display valid permission unit 110 transmits to the microcomputer 112 an end state indicating that the subpicture decode for one frame has finished (step ST1216). In this way, sub-picture decoding processing for one screen (one frame) is completed.

20 is a flowchart for explaining an example of the contents of the encoding header detection step ST1205 in FIG. This coded header detection process is executed by the continuation code length detection unit 106 of FIG. 17 (or FIG. 11).

First, the code length detection unit 106 is initialized and the state counter STSCNT of most of them is set to zero (step ST1301). After that, the contents of the second two bits of the data being read from the memory 108 to the detection unit 106 in byte units are checked. The contents of these two bits are zero daemon (YES in step ST1302), and the counter STSCNT is increased by one (step ST1303). If the checked two bits do not reach the end of one byte being read by the detection unit 106 (NO in step ST13), the contents of the subsequent two bits are checked again. If the contents of these two bits are 0 (YES in step ST1302), the counter STSCNT is incremented by one again (step ST1303).

As a result of the repeated loops of steps ST1302 to ST1304, when the next two bits checked in step ST1302 reach the end of one byte read out by the detection unit 106 (YES in step ST13), the encoding header of FIG. It is larger than 6 bits. In this case, the next data byte is read from the memory 108 to the detection unit 106 (step ST13∞), and the status counter STSCNT is set to 4 (step ST1307). At the same time, the same data is also read one byte into the encoded data fragment section 03.

After the status counter STSCNT is set to 4 or if the contents of the two bits checked in the preceding step ST13O2 are not 0O (NO in step ST1302), the contents of the status counter STSCNT are confirmed, and the content is the encoding header of FIG. Is outputted as the content of (step ST1307).

That is, if the status counter STSCMlr = ∝, the encoded header that exits Rule 1 of FIG. 5 is detected. If the status counter STSCNT = 1, the code and header representing Rule 2 of FIG. 5 are detected, and if the status counter STSCNT = 2, The encoding header showing the rule 3 of FIG. 3 is detected, and if the state counter STSCNT = 3, the encoding header showing the rule 4 of FIG. 5 is detected, and if the state counter STSCNT = 4, the rule 5 of FIG. Coded header) is detected.

21 is a flowchart for explaining how the image decoding process of the present invention is carried out when the decoded image is moved to the screen.

First, each block of the decoder 101 of Fig. 11 or 17 is initialized, and the line counter LINCNT, not shown, is cleared to zero (step ST1401). Next, the microcomputer 11 (FIG. 11) or the header cutting part 113 (FIG. 17) receives the header read end state sent to step ST1201 in FIG. 18 (step ST1402).

The contents (firstly zero) of the line counter LINCNT are transmitted to the microcomputer 112 (FIG. 1L) or the header cutout 113 (FIG. 17) (step STl4O3). The microcomputer 112 or the header slicer 113 checks whether the received state is the end state (step ST1206) of one frame (one screen) (step ST1404).

If the received state is not the end state of one frame (NO in step ST1405), it waits until this end state is reached. If the received state is the end state of one frame (YES in step ST1405), the line counter LINCNT is increased by one (step ST1406).

If the contents of the incremented line counter LINCNT do not reach the end of the line (NO in step ST1407), the decoding processing of FIGS. 15 to 16, or the decoding processing of FIGS. 18 to 19 is resumed (step ST1408). Return to ST14∞.

By repeating the decoding loop (steps ST1403 to ST1408), the sub-length compressed video having the execution length is decoded and the screen is shifted.

On the other hand, when the content of the incremented line counter LINCNT reaches the end of the line (YES in step ST1407), the decoding processing of the sub-picture data accompanying the screen shift ends.

FIG. 22 is a block diagram illustrating an outline of an optical disc recording / reproducing apparatus on which encoding (SPUH + PXD + DCSQT encoding in FIG. 3) and decoding (SPUH + Dm + SQT decoding) based on the present invention is executed.

In FIG. 22, the optical disc player 3∞ has the same configuration as the conventional optical disc player (compact disc player or laser disc player). The optical disc player 300, however, is adapted to decode the execution length compressed image information from the inserted optical disc OD (the image information including the execution length compressed sub-picture data based on the present invention). A digital signal (digital signal while encoded) can be output. Since the digital signal while being encoded is compressed, the required transmission bandwidth may be less than that of the case of transmitting uncompressed data.

The compressed digital signal from the optical disc player 300 is aired through the modulator / transmitter 210 or is sent out to the communication cable.

The compressed digital signal, which is aired, or the compressed digital signal transmitted by the cable, is received by the receiver / recovery receiver 4o0 of the receiver or cable subscriber. This receiver 400 is provided with a decoder 101 of the configuration as shown, for example, in FIGS. 11 or 17. The decoder 101 of the receiver 400 decodes the received and demodulated compressed digital signal, and outputs image information including original sub-picture data before being encoded.

In the configuration of Fig. 22, if the transmission / reception system of transmission / reception usually has an average bit rate of 5M ㅂ 1t / sec or more, it is possible to broadcast high-quality multimedia video and audio information.

23 is a block diagram illustrating a case where encoded image information based on the present invention is transmitted and received between any two computer users via a communication network (internet, etc.).

User # 1 having magnetic information # 1 managed by a host computer, not shown, owns a personal computer 5∞1, and this personal computer 5001 includes various input / output devices 5011 and various external storage devices. 5o21 is connected. In addition, an internal slot (not shown) of this personal computer 5001 incorporates an encoder and decoder based on the present invention, and is equipped with a mother card ∞31 having a function necessary for communication.

Similarly, user #N having a separate magnetic information #N group is owned by a personal computer 50ON, and various personal input / output devices 501N and various external storage devices 502N are connected to the personal computer 50ON. It is. In addition, a large slot (not shown) of the personal computer 50ON incorporates an encoder and a decoder based on the present invention, and is equipped with a modem card 503N having a function necessary for communication.

Now, suppose a user # 1 operates a computer 5∞1 to communicate with another computer 50ON of user #N via a line 6∞ such as the Internet.

In this case, since user # 1 and user #N both have modem cards 5031 and (∞3N) in which an encoder and a decoder are built in together, image data compressed by the present invention can be exchanged efficiently in a short time. have.

FIG. 24 records recorded image information (SPtm + PXD + ∞SQT of FIG. 3) on the optical disc ∞, and decodes the recorded information (SPUH + Dm + ∞SQT) on the basis of the present invention. The outline of the recording / playback apparatus is shown.

Encoder 2∞ in FIG. 24 is an encoding process (process corresponding to FIGS. 13-14), such as encoder 2∞ in FIG. 10, or software or hardware (including firmware or wired logic circuitry). Is configured to run.

The recording signal including the sub-picture data encoded by the encoder 2∞ is modulated by, for example, (2,7) RLL in the frequency converter / laser driver 702. The modulated recording signal is sent from the laser driver 702 to the high output taper diode of the optical head 7 field.

By the recording laser from the optical head 704, a pattern corresponding to the recording signal is recorded on the magneto-optical recording disk or the phase change optical disk OD.

The information recorded on the disc OD is read by laser mixing up of the optical head 706, demodulated in the original reconstructor / error correction unit 708, and subjected to error generation processing as necessary. The demodulated error corrected signal is subjected to various data processing in the audio / video data processing unit 710 to reproduce the information before recording.

This data processing unit 710 includes a decoding processing unit corresponding to the decoder 101 of FIG. By this decoding processing section, decoding processing (decompression of the compressed sub-picture data) corresponding to FIGS. 15 to 16 is executed.

Fig. 25 illustrates a state in which an encoder based on the present invention is IC confirmed simultaneously with its peripheral circuit.

Fig. 26 illustrates a state in which a decoder based on the present invention is ICized simultaneously with its peripheral circuit.

Figure 27 illustrates a state in which an encoder and a decoder based on the present invention are ICized together with their peripheral circuits.

That is, the encoder or decoder based on the present invention can be integrated with the necessary peripheral circuits, and this IC can be implemented in various devices to implement the present invention.

In addition, the data line on which the bit string of the compressed data PXD as shown in FIG. 9 is loaded is usually configured to include image information of one horizontal scanning line of the TV display screen. However, this data line is configured to contain a plurality of pieces of image information of the horizontal scanning line on the rear surface of the TV, or one horizontal line on the TV expansion surface.

It may be configured to include image information of all the scanning lines (that is, one frame).

The object of data encoding based on the compression rule of the present invention is not limited to the sub-picture data (color information of 3 to 4 colors) used in the specification. The pixel data portion constituting the sub-picture data is multi-bitted, so that various kinds of information can be filled. For example, if the pixel data has an 8-bit configuration per dot, the 256-color color image can be transmitted only to the sub-image.

Here, the sub-picture data shown in FIG. 2 or FIG. 3 is composed of a plurality of channels as shown in FIG. The sub-picture data block is composed of a plurality of sub-picture data packets of a channel arbitrarily selected from these multiple channels.

The sub-picture here has information such as text or graphics, and is reproduced simultaneously with video data or audio data, and superimposed on the reproduction screen of video data.

29 shows a data structure of a subpicture packet. As shown in FIG. 29, the packet data of a sub-picture is composed of a packet header 3, a sub-picture header 31, a sub-picture data 32, and a display control procedure table 33. As shown in FIG.

In the packet header 3, the time at which the playback system should start display control of the sub-picture data block is recorded as a presentation time stamp (PTS). However, this PTS is recorded only in the header 3 of the head subpicture data packet in each subpicture data block Y and W, as shown in FIG.

FIG. 30 shows the serial arrangement state (n, n + 1) of a sub-picture unit (see 30 in FIG. 3) composed of one or more sub-picture packets, and the time described in the packet header of one unit (n + 1) therein. The stamp PTS and the state of display control of the unit (n + 1) corresponding to this PTS (clear display of the previous subpicture and designation of the display control procedure of the subpicture to be displayed) are illustrated.

The sub-picture header 31 records the size of the sub-picture data packet (2 bytes of SPCsZ) and the recording start position (2-byte SPDCSQTA) of the display control procedure table 33 in the packet.

In the display control procedure table 33, a sub-picture display control time stamp (SPDCTS) indicating the start time / end time of display of the existence data and the sub-picture data (PXD) Display control sequence information (DCSQT) in which the recording position (SPNDCS) (32) of Sub-Picture Next Display Control Sequence Address (32) and the display control command (C0m smm) of sub-picture data are used as one group. ) Is written more than l.

Here, the time stamp PTS in the packet header 3 is defined as the relative time from the reference time (SCR; System C1ock Reference), which is the reference for the whole playback sound, such as the playback start time at the beginning of the file (FIG. 2). It is. On the other hand, each time stamp SPDCTS in the display control procedure table 33 is defined by the relative time from the PTS.

Next, time stamp PTS processing of sub-picture data packets in the reproduction system will be described. It is assumed here that this PTS process is executed in the sub-picture processor (for example, the MPU 112 and its peripheral circuit of FIG. 11) in the reproduction system.

FIG. 52 is a diagram for explaining how a buffering state of a sub-picture data block changes with a sub-picture channel having a time stamp PTS when decoding sub-picture data.

(1) The sub-picture processor (FIGS. 11, 17, etc.) decodes a sub-video data packet of a preselected channel among sub-video data packets sent from the outside (optical disk or broadcasting station, etc.), and the PTS in the packet decodes. Check if there is any.

If a PTS is present, for example, as shown in channel < RTI ID = 0.0 > 4F < / RTI > Then, for example, as shown in FIG. 28, the PTS is attached to the head of the sub-picture data, and the sub-picture data with the PTS header is buffered (stored) in the sub-picture buffer (e.g., buffer 121 in FIG. 11). .

The graph of FIG. 52 exemplifies how the buffering amount to the sub-picture buffer 121 accumulates as the sub-picture data packet of the PTS attached channel ◆ 4f is buffered.

(2) After the system reset, the sub-picture processor performs this PTS during the vertical blanking period (during the transition from one display screen frame / field to the next display screen frame / field) immediately after receiving the first packet including the PTS. Input and compare the input PTS with the count value of the reference time counter STC. This reference time counter STC is used by a counter in the sub-picture processor (for example, part of timer 120 in FIG. 11) that predicts the elapsed time from the reference time SCR, for example, by playing the entire file such as the start time of playback of the file head. It is composed.

(3) As a result of the comparison between the PTS and STC, when the STC is larger than the PTS, the sub-picture data is immediately displayed. On the other hand, when STC is smaller than 玎 S, no processing is performed. This comparison is run again during the next vertical misfortune period.

(4) Entering the processing of the sub-picture data, during the same vertical blanking period, the first sub-picture display control time stamp SPDCTS recorded in the display control procedure table 33 in the sub-picture data packet is the sub-reference in the sub-picture processor. The count value of the time counter (sub STC) is compared. This sub-STC predicts the elapsed time from the reproduction start time of the sub-picture data block. It consists of a sub-reference time counter (e.g., outside the timer 120 in FIG. 11) in the sub-picture processor. Therefore, when the display changes to the next sub-picture data block, all the bits are cleared to ,, After that, the increment (time count) is started again.

(5) As a result of comparing the sub-STC with the sub-picture display control time stamp SPDCTS, when the sub-STC is larger than the SPDCTS, control data (DCSQT) of the head of the display control procedure of the display control procedure table 33 (for example, DCSQTO in FIG. Is executed immediately, and the display processing of the sub-picture is started.

(6) Once the display processing is started, the PTS added at the beginning of the next sub-picture data block of the sub-picture data block currently displayed for each vertical bleaching period is read out, and the read-out PTS and reference time counter STC are read. The count value of is compared.

As a result of this comparison, if the STC is larger than the PTS, the channel pointer of FIG. 29 is set to the address value of the PTS of the next sub-picture data block, and the sub-picture data block to be processed is switched to the next. For example, referring to FIG. 28, by changing the setting of the channel pointer, the sub-picture data block Y is switched to the next sub-picture data block W. FIG. At this point, since the data of the sub-picture data block Y is no longer necessary, an empty area of the size of the data block Y is created in the sub-picture buffer (for example, the memory 108 in FIG. 11). For this reason, the sub-picture data packet can be newly transferred to this empty area.

As a result, the buffering state (see FIG. 52) of the sub-picture data packet is determined from the size of the sub-picture data block (e.g., block W in FIG. 28) and its switching time (block Y to block W). At the time of encoding the sub-picture data, it is possible to uniformly define in advance. Therefore, when serially transmitting packets of video, audio, and sub-pictures, generation of a bit stream in which the overflow of each decoder unit (in the case of the sub-picture decoder, memory 11 of FIG. 11) does not occur or overflow occurs. This is possible.

Further, as a result of the comparison between the PTS and STC, if the STC is not larger than the PTS, the sub-picture data block is not switched, and the display control procedure table pointer (DCSQT pointer of FIG. 29) is the address of the next display control procedure table DCSQT. Value is set. The sub-picture display control time stamp SPDCTS of the next DCSQT in the current sub-picture data packet is compared with the sub-STC. Based on this comparison result, it is determined whether or not the next DCS word is executed. This operation will be described later in detail.

Also, since the last DCSQT in the sub-picture data packet instructs itself as the next display control procedure table DCS word, the ∞ SQT process of (5) is basically unchanged.

(7) In the normal reproduction, the processes of (4), (5) and (6) are repeated.

In the process of (6), when reading I 玎 S of the next sub-picture data block, the value of the channel pointer (see FIG. 29) indicating the PTS group is determined by the packet size (the size of the sub-picture data block of the face). SPCSZ).

Similarly, the value of the DCSQT pointer indicating the sub-picture display control time stamp SPDCTS of the next DCSQT in the display control procedure table 33 is determined by the size information of ∞ SQT described in this table 33 (the next sub-picture display control procedure). Address is obtained using the address SPNDCSQTA).

Next, details of each of the sub-picture header 31, the sub-picture data 32, and the display control procedure table 33 will be described.

31 shows the structure of the sub-picture unit header (SPUH) 31. As shown in FIG. The sub-picture unit header SPUH includes the size (SPDSZ) of the sub-picture data packet and recording start position information (display control procedure table start address SPDCSQT of the sub-picture; relative address pointer of the DCSQ) of the display control procedure table 33 in the packet. have.

The contents of the sub-picture display control procedure table SPDCSQT indicated by the address SPDCSQTA are composed of a plurality of display control procedures DCSQ1 to DCSQn, as shown in FIG.

In addition, each display control sequence DCSl to n) is a sub-picture display control time stamp SPDCTS indicating the display control start time of the sub-picture and an address SPNDCS indicating the position of the next display control sequence, as shown in FIG. And one or more sub-picture display control commands SPDCCMD.

The sub picture data 32 is composed of a set of data areas (PXD areas) corresponding to individual sub picture data packets one-to-one.

Here, the sub picture pixel data PXD of any address in the same data area can be read out until the sub picture data block is switched. Thereby, arbitrary sub-picture display (for example, screen movement display of a sub-picture) which is not fixed to one sub-picture display image is possible. This arbitrary address is set by an instruction (SETDSP in the instruction table of FIG. 34) that sets the display start address of the sub-picture data (pixel data PXD).

FIG. 43 shows the bit structure of the instruction SET DSPV furnace for setting the display start address of the sub-picture pixel data among the instruction sets illustrated in FIG. The meaning of the structure of this command is explained below.

If the line data size of the sub-picture line included in the sub-picture data 32 is different, the head address of the next line is inevitably determined after the previous line data is decoded. Therefore, when the image data is arranged in line number order as in the prior art, it becomes very difficult to read the sub-pixel pixel data PXD from the buffer (memory 08) while skipping for one line in the interlace mode. .

Thus, as shown in FIG. ∞, for each data area corresponding to each sub-picture data packet, the sub-picture data 32 is divided into the first field area 61 and the last field area 62 for recording. Doing. In order to be able to set two head addresses of the first field and the last field in the interlace mode, the start address area 63 for the first field and the start address area 64 for the last (bottom) field are set in the instruction SETDSDn. I install it.

In the non-interlaced mode, only one field of sub-picture data may be recorded, and the same address may be recorded in two areas of the start address area ∞ for the first field and the start address area 64 for the last field.

59 shows a specific example of the display control procedure table 33. As shown in FIG. As described above, one display control sequence information DCSQT in the display control sequence table 33 includes a plurality of display control instructions COmlAND3 after the sub-image display control time stamp SPDCTS and the sub-image data recording position SPNDCSQA. , COMMAND4, etc.) and various parameter data set by the instruction are arranged. Then, an end instruction (end code) indicating the end of display control is added last.

Next, the processing procedure of the display control procedure table 33 will be described.

(1) First, the time stamp SPDCTS recorded in the first DCS of the display control procedure table 33 (DCSQTO in Fig. 29) is the sub-STC of the sub-picture / processor (e.g., timer 120 in Fig. 11). Is one function).

(2) As a result of the comparison, when the sub-STC is larger than the time stamp SPDCTS, all the display control commands C mmD in the display control procedure table 33 are executed until the display control termination command CMDEblI (degree diagram) appears. .

(3) After the display control starts, whether or not to update the next DCSQT by comparing the sub-image display control time stamp SPDCTS recorded in the next display control procedure table DCSQT with the sub-STC at a predetermined time (for example, every vertical blanking period). (I.e., move the DCSQT pointer of FIG. 29 to the next DCS) is determined.

Here, since the time stamp SPDCTS in the display control procedure table 33 is recorded at a relative time after the PTS is updated (that is, after the sub-picture data block is updated), the SPDCTS is corrected even if the PTS of the sub-picture data packet changes. There is no need to write. Therefore, even when the same sub-picture data 32 is displayed at a plurality of different times, the same display control procedure table DCS can be used. That is, the display control procedure table DCSQT can be rearranged.

Next, the details of the display control command of the sub-picture will be described. 34 shows a list of sub-image display control commands SPDCCMD. The main sub-image display control commands include the following.

(1) Command STADSP for Setting Display Start Timing of Sub-Video Pixel Data FIG. 37 shows the configuration of this command STADSP. This is a command to execute display start control of the sub-picture data 32. In other words, when the ∞SQTNI that includes this command STADSP is switched in any ∞S, the display of the sub-picture data 32 is started from the time indicated by the time stamp SPDCTS of the ∞S word including this command.

The sub-picture processor (e.g., MPU 112 in FIG. 1) decodes this command, immediately after the command exits (since the time passed by SPDCTS of the DCSQT to which this command belongs) passes. Enable bit in the display control system.

(2) Command STPDSP for Setting Display End Timing of Sub-Video Pixel Data FIG. 38 shows the configuration of this command STPDSP. This is a command for executing display end control of the sub-picture data 32. When the sub-picture processor decodes this command, it immediately activates the enable bit of the display control system inside the sub-picture processor (since it has passed the time indicated by SPDCTS of the ∞SQT to which it belongs). Shall be.

(3) Command SETCOLOR for setting the color code of sub-picture pixel data FIG. 39 shows the configuration of this command SETCOLOR. This is a command for setting the color code of the sub-picture pixel data. With this command, the sub-image is a background pixel which is a pixel of a pattern pixel such as a character or a shape, an emphasis pixel such as the provision of wires of the pattern pixel, and a pixel of a region other than the pattern pixel and the emphasis pixel in the region where the sub-image is displayed. Color information can be set separately.

As shown in FIG. 40, the sub-picture processor has a color register 1210 capable of setting a color code by this command SETCOUm. Once the color code is set, the register 1210 holds this color code data until a reset is made with the same command. Color data corresponding to the pixel type (e.g., the type characterized by the 2-bit pixel data in FIG. 5) represented by the sub-picture data 32 is selected from the color register 1210.

The sub-picture processor also includes a change color data register 1220 set by a command C_GCO_ON to set the color change and contrast change of the sub-picture pixel data. When the data output selected from the register 1220 is active, the selection output from the register 1220 is given priority over the selection output in the register 121D (SEL1), and the result of the selection is colored. It is output as data.

(4) Command SETCONTR for setting the contrast of sub-picture pixel data with respect to the main video [0058] FIG. 4 shows a configuration of this command SETCONTR. This is the same as the command SETCOlOR, which is a comfort command for setting contrast data for the four types of pixels illustrated in FIG.

(5) The model type SETDAREA, which sets the display area of the sub-picture pixel data on the main image, shows the structure of this command SETDAREA. This is an instruction for identifying the position at which the sub-picture pixel data 32 is displayed.

(6) Command SETDSPXA for setting the display cache address of sub-picture pixel data FIG. 43 shows the configuration of this command SETDSPM. This is a command for setting the display start address of the sub-picture pixel data 32.

(7) Command CHGCOLCON 44 for setting the color code of the sub-picture pixel data and the contrast switching of the sub-picture pixel data with respect to the main video. FIG. 44 shows the configuration of this command CHGCOLCON. This is a command for changing the contrast of the sub-image pixel data 32 for the sub-image and the main image during display.

This command CHGCOLCON, as shown in FIG. 44, contains the size (extended field size) of the pixel control data and the pixel control data (PCD). In addition, the instruction table of FIG. 34 includes a command FSTADSP for forcibly setting the display start dimming sound of the sub-picture pixel data (see FIG. 36), and a command CMDEND for terminating display control of the sub-picture. See also).

35, 46, and 47 are views for explaining the configuration of the pixel control data PCD. As shown in Fig. 35, the pixel control data PCD is composed of the line control information LCINF, the pixel control information PCI ', and an end hod indicating the end of the pixel control data.

Here, the line control information LCINF is composed of a change (change) start line number, a change number (change score), and a change (change) end line number (or the number of continuous lines) as shown in FIG. That is, the contrast control of the outline correction color, the sub picture color, and the sub picture with respect to the main picture is started from any line on the display frame, and the outline correction color, the sub picture color, the contrast is changed (or changes) on the line, In addition, it is shown by the line control information LCINF whether a change (change) common to it continues to any line.

The pixel control information PCINF is a pixel position for changing (changing) the contour correction color, the sub-image color, and the contrast on the line indicated by the line control information LCINF, the contour correction color after the change (the toilet seat), the sub-picture color, and the like. The contents of variations such as contrast are shown.

The pixel control data PCD consisting of this line control information LCI 'and the pixel control information PCINF is set to the required number of sub-picture display frames.

For example, the pixel display data PCD set for the image of the sub-picture display frame as shown in FIG. 48 is as shown in FIG.

That is, in this specific example, since the line where the change (change) is started is "line 4", the change (change) start line number is "4", and the position where the pixel is changed (change) is "position". A "," Position B ", and" Position C "are located at three positions, and the pixel change number (pixel change score) becomes" 3 ", and the common change state of this pixel continues to" Line 11 ". The number of lines becomes "7".

In addition, "line 12" is different from the pixel change state until then, and since "line 13" has no pixel change, the change start line number is "l2" and the change score rule is "2". A separate line control information LCINF is set that sets the number of lines.

In addition, since "line 14" includes four pixel change groups, the next "line 15" has no pixel change, and the change start line number is "14", and the change score is "4". In addition, another line control information l_INF is set in which the number of lines continues to be "1".

Finally, an end code is set.

Next, the procedure of display control using the line control information LCINF and the pixel control information PCINF will be described.

(1) The display control of the sub-picture consists of repeatedly executing the control commands COMMAND1 to the sub-picture display pulse included in the display control procedure table 33 (DCSQT1 to DCSQnl in FIG. 29). The contents of this control command are shown in a table of the sub-image display control command SPDCCMD of FIG.

Which display control order DCSQT1 to DCSQTN is executed by the ∞ SQT pointer in FIG. 29 is executed.

(2) The various parameters implemented by each display control command (STADSP, STPDSP, SETCOLOR, SETCONTR, SETDARE / NO, SETDSPXA, CHGCOLCON, etc.) shown in FIG. 34 are not rewritten by the same command. In the decoding of the sub-picture information, the sub-picture processor is held in an internal register of the sub-picture processor (for example, the MPU 112 of FIG. 1L). However, the various parameters held in this internal register are all cleared except for the sub-picture data block switching (for example, when switching from block Y to block W in FIG. 28) and some parameters (LCINl # PCINF).

Incidentally, the parameters LCINF and PCINF of the pixel control data PCD of FIG. 35 are held in the internal register of the MPU 112 until every instruction CHGC10_ON of FIG. 34 is re-executed.

(3) In the highlight mode, display control is performed by the parameters LCINF and PCINF set by the system MPU 112, and LCINF and Pcn " of sub-picture channel data are all ignored. These set parameters are reset by the system MPU 112 again during the highlight mode, or are maintained in the MPU until the LCINF and PCI 중 in the sub-picture data are reset to the normal mode, and the sub-picture by the parameters thereof. The display continues.

(4) The display area is set by lines and dots of the number designated by start and end in both the horizontal direction and the vertical direction. Therefore, when only one line is displayed, the numbers of the display start line and the display end line become the same. In addition, when it does not display, display is stopped by the display end instruction.

53 is a flowchart for explaining an example of a method of generating a sub-picture unit 30 as shown in FIG.

For example, when a subtitle and / or image corresponding to the dialogue of the video (main video) is used as the sub-picture, the dialogue caption / image is bitmap data expanded (step ST10).

When creating this bitmap data, it is necessary to determine in which area of the video screen the subtitle part is displayed. Therefore, the parameter of the display control command SETDAREA (see FIG. 34) is determined (step ST12).

When the display position (spatial parameter) of the sub-picture is formed, it is transferred to the encoding of the pixel data PXD constituting the sub-picture (not the entire main video is encoded; details of the PXD encoding are shown in FIGS. 5 to 14). It is described elsewhere with reference).

For each of them, a blending ratio for the video main picture of the color of the subtitle (sub picture), the background color of the subtitle area, and the subtitle color and background color is formed. Therefore, the parameters of the display control commands SETCOLOR and SETCONTR (see FIG. 34) are determined (step ST14).

Next, the displayed bit timing data is formed in accordance with video dialogue. This timing formation is cooled by the sub-picture time stamp PTS. At that time, the maximum time of the time stamp H 'and the parameters (temporal parameters) of the display control commands STADSP, STPDSP and CHGCOLCON (see FIG. 34) are determined (step ST16).

Here, the sub-picture time stamp PTS is finally determined from the consumption model of the target decoder buffer of the MPEG2 system layer. Here, the time at which the display of the subtitles is started is determined as the maximum time of the sub-picture time stamp PTS.

The display control commands STADSP and STPDSP are recorded as relative time from the sub picture time stamp PTS. Therefore, the commands STADSP and STPDSP cannot be determined until the PTS is determined. Therefore, in this embodiment, the absolute time is determined and the relative value is formed after the absolute time of the PTS is formed.

When the display color and display area are to be changed spatially and temporally with respect to the created subtitle, a parameter of the command CHGCOLCON based on the change is formed.

When the display position (spatial parameter) and display dimming (temporal parameter) of the sub-picture are determined, the content (DCSQ) of the sub-picture display control procedure table DCSQT is created (step ST18). Specifically, the value of the display control start time SPDCTS (see FIG. 33) of the display control procedure table ∞ SQ is the fermentation time of the display control command STADSP (display start timing) and the fermentation of the display control command sn_sP (posh end timing). It is decided based on time.

By matching the created pixel data PM) 32 and the display control procedure table DCSQT 33, the size of the sub-picture data unit 30 (see FIG. 3) can be determined. Thus, based on the size, the parameters SPDSZ (see FIG. 31) and SPDCSQTA (start address of the display control procedure table; see FIG. 31) of the sub-picture unit header SPUH # 1 are determined to determine the sub-picture unit. The header SPUH 31 is created. Thereafter, by subtracting the SPUH 31, PM x 32, and ∞ S language 33, a sub-picture unit for one subtitle is created (step ST20).

If the size of the created sub-picture unit 30 exceeds a predetermined value (2048 bytes or 2k bytes) (YES in step ST22), it is divided into a plurality of packets in units of 2k bytes (step ST24). In this case, the time stamp PTS is Only the packet leading to the sub-picture unit 30 is recorded (step ST26). If the size of the created sub-picture unit 30 is equal to or larger than a predetermined value (2k bytes) (NO in step ST22), only one packet is generated (step ST23), and the time stamp PTS is recorded at the head of the packet (step ST26).

One or more packets thus completed are packetized and combined with the video pack to complete one data stream (step ST28).

At this time, the arrangement order of each pack is determined based on the sequence record code SRC and the sub picture time stamp PTS from the consumption model of the target decoder buffer of the MPEG2 system layer.

Here, the PTS is finally determined, thereby finally determining each parameter (SPDCTS, etc.) in FIG. 33. FIG. 18 is a flowchart for explaining an example of a procedure of parallel processing a pack decomposition and decoding of a sub-picture data stream generated in accordance with the processing sequence of FIG. 53. FIG.

First, the decoding system reads the ID of the stream to be transmitted, and transmits only the selected subpicture pack (one separated from the data stream) to the subpicture decoder (e.g., the subpicture decoder 101 of FIG. 11 or 17). (Step ST40).

When the first Z-transfer is performed, the Inmex filter r is set to "1" (step ST #), and the decomposition process of the first sub-picture mac (step ST44; described with reference to 55th group) is executed. .

The decomposed pack (including the compressed subpicture data PXD as shown below in FIG. 9) is temporarily stored in the subpicture buffer (memory 108 in FIG. 11 or 17) (step ST46), The index parameter r is increased by one (step ST50).

If there is an increased i-th pack, that is, if the pack decomposed in step ST44 is not the last pack (NO in step ST52), a decomposition process (simple ST44) is performed for the increased i-th sub-picture pack.

The disassembled i-th sub-picture pack (here, the second pack) is temporarily stored in the sub-picture buffer (memory 108) like the disassembled pack at the first (step ST46), and the index parameter r is further increased by one. (Only "1 ST50").

As described above, the plurality of sub-pictures are successively decomposed (step ST44) while the index parameter i is increased (step ST44) and stored in the sub-picture buoy (memory 108) (step ST46).

If there is no successively increased i-th shock, i.e., when the shock processed in step ST44 is the final pack (YES in step ST52), the sub-picture mech decomposition processing of the stream to be decoded ends.

Decoding of the sub-picture pack temporarily stored in the sub-picture buffer (memory 108) while executing the sub-picture pack decomposition processing (steps ST44 to ST52) continuously and independently of the sub-picture pack decomposition processing. The process is performed.

That is, when the index parameter r is set to "D" (step ST60), the operation enters the operation of reading the first sub-picture pack from the sub-picture buffer (memory 108) (step ST62). At this point, if the first sub-picture 아직 has not yet been stored in the memory 108 (NO in step ST63; when the process of step ST46 has not yet been performed), the pack data to be read is stored in the memory 108. Until that time, the decoding process executes an empty loop (steps ST62 to ST63) of the pack read operation.

If the first sub-picture 격 is stored in the memory 108 (step ST63), the sub-picture 땍 is read out and decoded (step ST64; see FIGS. 53 to 57 for a specific example of the decoding process). To be described later).

The decoding process result (e.g., including the sub-picture data PXD before compression as shown in FIG. 9 above) is sent from the FIG. 11 or FIG. 17 sub-picture decoder 101 to the display system (not shown) during the decoding process. Then, the sub-picture corresponding to the decode data is displayed.

If the display control end instruction (CMDEbD in FIG. 34) is not executed in the decoding processing (NO in step ST66), one index parameter r is increased (step ST67).

If the incremented i-th pack (here 2) is present in the memory 108, the pack is read from the memory 108 and decoded (step ST). The decoded j-th sub-picture pack (here, the second pack) is sent to the display system like the decoded pack, and the index parameter r is further increased by one (step ST67).

As described above, while increasing the index parameter j (step ST67), one or more sub-pictures stored in the memory 108 are successively decoded (step 64), and the decoded sub-picture data P # is decoded. Image display of the corresponding sub-image is executed.

When the display control end command (CMDEbm in FIG. 34) is executed in the decoding process (YES in step ST66), the decoding process of the sub-picture data in the sub-picture buffer (memory 108) is finished.

The above decode values (steps ST62 to ST64) are repeated unless the end instruction CMDE 'is executed (NO in step ST66). In this embodiment, the decoding process ends by executing the end instruction CMDEND (YES in step ST66).

55 is a flowchart for explaining an example of the pack decomposition process of FIG. The sub-picture decoder 101 skips the header (see FIG. 3) from the transmitted packet and obtains a packet (step ST442). If there is no time stamp PTS in this packet (step ST444), the packet header PH is deleted, and only the sub-picture unit data P 'is stored in the sub-picture decoder's buffer (e.g., 121). ).

When the packet has a time stamp PTS (only ST444), only the PTS is extracted from the packet header PH, and the extracted? B is connected to the sub-picture unit data 3O, and the buffer of the sub-picture decoder 101 is used. It is stored in 121 (step ST448).

56 is a flowchart for explaining an example of the sub-picture decoding process of FIG. 54. FIG.

The sub-picture decoder 101 compares the time SCR of the system dimer 12O with the time stamp PTS stored in buffy 121 (step ST640). When it matches (only ST1 642), the decoding process of the sub-picture unit 30 is started. During this decode process, for example, the process of returning the compressed data PXD shown in the lower part of FIG. 9 to the uncompressed data PXD shown in the upper part of FIG. 9 will be described with reference to FIGS. 15, 16 and others.

In this decoding process, each instruction of the display control procedure DCSQ is executed.

That is, the display position and display area of the subpicture are set by the command SETDAREA, the display color of the subpicture is set by the command SETCOLOR, and the contrast of the subpicture with respect to the video main picture is set by the command SETCONTR (step STM4).

After execution of the display start timing command STADSP, the display data in accordance with the switching command CHGCOLCON is changed while the display end timing command STPDSP is executed in a separate display control procedure DCSQ. ) Is quenched (step ST646).

The processing steps ST644 and ST646 are skipped when the time SCR of the system timer l20 and the time stamp PTS stored in the buffy 121 do not match (NO in step ST642).

FIG. 57 is a flowchart for explaining an example of a method for decoding a limited data stream in the order of FIG. 53; While the processing of FIG. 54 is parallel processing in which the decomposition of the sub-picture mac and the sub-picture decoding are independent in time, the processing of FIG. 57 is the parallel processing in which the decomposition of the sub-picture pack and the sub-picture decoding are linked in time. That is, FIG. 57 assumes a case where the decomposition processing and the sub-picture decoding processing of the sub-picture pack simultaneously co-treat at the same pace.

In the processing of FIG. 57, the decoding system first reads the ID of the stream to be transmitted, so that only the selected subpicture pack (one separated from the data stream) is a subpicture decoder (Fig. 11 or 17). (101) (step ST40).

When the first pack transfer is slow, the index parameter r is set to "1" (step ST42), and the decomposition process of the first sub-picture 떽 (step ST44) is executed.

The disassembled mac is temporarily stored in the sub-video buffy (memory 108) (step ST46). After that, the index filter r is set in the index parameter f (step ST48), and the index filter i is increased by one (step ST50).

If there is an increased i-th pack, i.e., the shot decomposed in step ST44 is not the final mac (NO in step ST52), an alternative decomposition process (step ST44) is performed on the increased i-th sub-image mec.

The disassembled i-th subpicture 땍 (here, the second pack) is temporarily stored in the subpicture buffer (memory 108) like the first disassembled pack (step ST46), and the index parameter i is further increased by one. (Step ST50).

As described above, the plurality of sub-picture packs are successively decomposed (step ST44), and stored in the sub-picture buffer (memory 108) while increasing the index parameter i (step ST46).

If the i-th pack that has been continuously increased does not exist, that is, if the pack processed in step ST44 is the final mac (YES in step ST52), the sub-picture pack decomposition processing of the stream to be decoded ends.

While the sub-picture pulse decomposition processing (steps ST44 to # 52) is continuously executed, the sub-image decoding processing is executed in parallel with the sub-image shock decomposition processing and temporarily stored in the sub-picture buffer (memory 108). Is done.

That is, the index filter indicator i = 1 is set in the index filter editor r (step ST48), j = the sub-image mech of the tooth is read out from the memory 108 (step ST62), and j = the negative part of the first row. The decoding process of the video Mac is slowed (only ST64).

During this j = decoding process of the first sub-picture pack (step ST64), the decomposition processing of the i = second sub-picture mac (step ST44) increased by one in step ST50 is performed in parallel.

The above decoding process (steps ST62 to ST64) is repeated only when the end instructions CMD and END are not executed (NO in step SrI166). The decoding process ends with the thread of the end instruction CMDEbm (step ST66).

As described above, according to the present invention, it is possible to drastically reduce display time waste and display time waste of sub-picture data, and achieve freedom of sub-picture clarity such as a bitmap data method. The use of sub-pictures can be secured.

That is, in the present invention, use range setting information for setting a range to be used for display in the sub-picture data is provided so that data other than the use range is not displayed, and all data for one frame is sent to the display system. It is possible to significantly reduce the display spatial waste of the amount of data generated in the case.

In addition, in the present invention, color setting information for each pixel type, such as pattern pixels of sub-picture data, line installation, bugs, etc., and mixing ratio setting information of sub-pictures are provided, so that only sub-picture image shape information is provided as sub-picture display data. The sub-picture of the same degree as the conventional method of subtracting the color information and the mixing ratio information for each pixel.

Shape representation can be guaranteed with a smaller data amount.

In the present invention, color / mixing ratio change setting information is provided to set the change in the color of each subpixel of the subpicture data and the blend ratio of each pixel type of the subpicture data for the main video in pixel units. Dynamic display can be realized with the same accuracy as that of the conventional bitmap data system and with a smaller data amount than the bit-bit data system.

In addition, in the sub-image, color information rarely changes from one pixel to another, and there is no fear that the data amount of the color / mixing ratio change setting information itself becomes excessive.

In addition, in the present invention, even if the color of the sub-image is changed, the shape does not change, and the sub-image can be displayed over the number of frames using the same sub-image data. Therefore, the display time waste of the sub-picture data can be greatly reduced as compared with the conventional method in which the sub-picture data must be continued to be given to the display system at the frame period even though the color is not changed.

Claims (4)

Means for packetizing a sub-picture reproducible with the main picture in predetermined units, and then communicating the packetized information; means for bitmap data content of the sub-picture; Means for determining a spatial parameter (SET-A) for setting a display area of the sub-image on the display area of the main image; Means for determining data (PXD) obtained by compressing the contents of the bitmap data subpicture by a predetermined method; Means for creating temporal parameters (STADSP / STPDSP) for specifying display start timing and display end timing of the sub-picture; Means for determining one or more display control sequence tables (DCSQTs) for controlling the order of displaying the sub-pictures; A sub-picture unit header (SPUH) including a size (SPDSZ) of the sub-picture packet and a position (SPDCSQTA) of the display control sequence table (DCSQT) is created, and the compressed data is stored in this sub-picture unit header (SPUH). Means for determining a sub-picture unit by adding (RXD) and the display control sequence table (DCSQT); Means for packetizing the created sub-picture unit if the size of the created sub-picture unit is equal to or larger than a predetermined value; Means for recording a time stamp (PTS) representing a playback start time of the sub-picture in a packet of the sub-picture unit having a size greater than or equal to the predetermined value or a sub-picture unit having a size within the predetermined value; Means for packing a packet of a sub-picture unit in which the time stamp (PTS) is recorded or a sub-picture unit in which the time stamp (PTS) is recorded, and streaming data of the packed sub-picture unit together with other data; Transmitting means for transmitting the streamed data; Receiving means for receiving the streamed data transmitted by the transmitting means; Means for separating data of said packed sub-picture unit from said stream and data received by said receiving means; Means for decomposing the zoom of the separated sub-picture unit; Means for extracting this time stamp when the timing stamp PTS is recorded in the decomposed pack; Compare the time indicated by the predetermined system timer with the contents of the extracted time stamp to expand the compressed data PXD based on one or more contents of the display control sequence table DCSQT when both match. And means for generating the sub-video before being encoded by encoding the image information. A system for packetizing sub information reproducible together with main information in predetermined units, and then communicating the packetized information, comprising: means for determining data Dm in which the contents of the sub information are compressed by a predetermined method; ; Means for determining a temporal parameter (STADSP / STPDSP) group for determining display start dieing and display end dimming of the sub information; Means for devising one or more display control sequence tables (DCSQTs) to control the order of displaying the sub information; A sub information unit header (SPUH) including a size (SPDSZ) of the sub information packet and a position (SPDCSQTA) of the display control sequence table (DCSQT) is created, and the compressed data is stored in this sub information unit header (SPUH). Means for determining a sub information unit by adding (PXD) and the display control sequence table (DCSQT); Means for packetizing the created sub information unit if the size of the created sub information unit is equal to or larger than a predetermined value; Means for recording a time stamp (PTS) representing a reproduction start time of the sub information in a packet of a sub information unit having a size greater than or equal to the predetermined value or a sub information unit having a size within the predetermined value; Means for packing a packet of a sub information unit in which the time stamp (PTS) is recorded, or a sub information unit in which the time stamp (PTS) is recorded, and string-packing data of the packed sub information unit; A transmission means wave for transmitting the streamed data; Receiving means for receiving the streamed data transmitted by the transmitting means; Means for separating data of the encoded sub information unit from the stream and data received by the receiving means; Means for disassembling the pack of the separated sub information unit; Means for extracting the time stamp (PTS) when the time stamp (PTS) is recorded in the decomposed pack; Based on the substitution of one or more of the display control sequence tables (DCSQTs) when the two systems match, the time indicated by a predetermined system timer is compared with the contents of the extracted time stamp. And means for decompressing compressed data PXD and generating the sub information before being encoded. CLAIMS 1. A system for packetizing sub information usable with main information in predetermined units and then communicating packet-determined information, comprising: means for determining data (PXD) in which the contents of the sub information are compressed by a predetermined method; Means for determining a temporal header (STADSP / STPDSP) group for characterizing the start time and end time of use of the sub information; Means for determining one or more control sequence tables (DCSQTs) for controlling the order of using the sub information; A sub information unit header (SPUH) including the size (SPDSZ) of the sub information packet and the position (SPDCSQTA) of the control sequence take (DCSQT) is determined, and the compressed data is stored in the sub information unit header (SPUH). Means for determining a sub information unit by adding (PXD) and the control sequence table (DCS language); Means for packetizing the created sub information unit if the size of the created sub information unit is equal to or larger than a predetermined value; Means for recording a time stamp indicative of the start time of use of the sub information in a packet of a sub information unit having a size greater than or equal to the predetermined value or a sub information unit having a size within the predetermined value; Means for streaming a packet of a sub information unit in which the time stamp (PTS) is recorded or a sub information unit in which the time stamp (PTS) is recorded, and streaming data of the packed sub information unit; Transmitting means for transmitting the streamed data; Receiving means for receiving stream confirmation data transmitted by the transmitting means; Means for separating data of the minified sub information unit from the stream and data received by the receiving means; Means for disassembling the pack of the separated sub information unit; Extracting the time stamp (PTS) when the time stamp (PTS) is recorded in the decomposed pack; Compare the time indicated by a predetermined system timer with the contents of the extracted time stamp, and when both match, expand the compressed data Dm based on the contents of one or more of the control sequence table, and incode And means for generating the sub information before the information is received. Means for packetizing sub information usable with the main information in predetermined units and then communicating the packet-determined information, for determining a data (PXD) group of contents of the sub information encoded in a predetermined method. and; Means for determining a usage start timing and an end use dimming op of the sub-information temporal parameter; Means for determining one or more control sequence tables (∞ SQTs) for controlling the order of using the sub information; A sub information unit header SPUH including a size SPDSZ of the sub information packet and a position SPDCSQTA of the control sequence table DCSQT is created, and the compressed data (SPUH) is written into the sub information unit header SPUH. Dm) and means (ST20) for designating a sub information unit by adding the control sequence table (∞S); Means for packetizing the created sub information unit if the size of the created sub information unit is equal to or larger than a predetermined value; Means for recording a time stamp (PTS) representing a start time of use of the sub information in a packet of a sub information unit having a size equal to or larger than the predetermined value or a sub information unit within a predetermined value; Means for passing the packet of the sub information unit in which the time stamp (PTS) is recorded or the sub information unit in which the time stamp (PTS) is recorded, and streaming the data rule of the packed sub information unit; Transmitting means for accepting the streamed data; Receiving means for receiving the stream and data transmitted by the transmitting means; Means for separating data of the packed sub information unit from the streamed data received by the receiving means; Means for disassembling the pack of the separated sub information unit; Means for extracting the time stamp (PTS) when the time stamp (PTS) is recorded in the decomposed pack; The time indicated by a predetermined system timer is compared with the contents of the extracted time stamp, and based on the contents of one or more of the control sequence table ∞SQT when both match, the encoded data PXD And means for generating the sub information before being encoded.