JP3437191B2 - Terminal device and control method of terminal device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a terminal device which is connected to a plurality of lines and which transmits and receives images and the like.

[0002]

2. Description of the Related Art When using a conventionally known analog telephone line, digital data can be transmitted only at a low speed.

However, in recent years, with the advancement of communication technology, semiconductor technology, and optical technology, digital lines have been established and high-speed and large-capacity data transmission has become possible.

In particular, a characteristic of digital transmission is that the same level of quality is maintained without deterioration of quality due to transmission.
Since it is possible to integrate the media without requiring a transmission path according to the characteristics of the transmission data media, it has become possible to transmit data between the composite media terminals. Therefore, a telephone terminal that simultaneously transmits an image from a conventional voice-only telephone has appeared.

Under these circumstances, international standardization by CCITT and the like is in progress to enable mutual communication between a plurality of different terminals, and AV (Au) such as a videophone and a videoconference system using digital lines is being developed.
CCITT Recommendation (or Draft) for AV service, including service specification, protocol specification and multimedia multiplexing frame configuration specification for AV service.
H. 320, H.M. 242, H.M. It has been announced as 221 etc.

The above H. 221 is 64 kbps to 1
Exchange of frame structure and terminal capability in AV services up to 920 kbps and FAS (Fra of communication mode)
meAlignment Signal), BAS (B
It Allocation Signal) code allocation is defined. In addition, H. H.242 defines protocols such as capability exchange and communication mode switching between AV terminals using BAS. In 320, the system aspect of the entire AV service is defined.

[0007] In the above-mentioned recommendation or draft recommendation, after setting the end-to-end physical connection and establishing the synchronization by the FAS in the in-channel, the BA is in-channel.
A method for performing multimedia communication of images, voice, data, etc. between terminals is defined by procedures such as a sequence of exchanging terminal capabilities using S and a mode switching sequence by specifying a communication mode.

[0008] However, it is outside the stipulated range in which each terminal changes its own terminal capacity according to the situation and which communication mode is used within the range of the exchanged capacity.

The information transfer rate of each medium in multimedia communication is determined by designating the voice encoding method for voice information, and the data information is designated by the presence or absence of its use and the transfer rate when it is used. The transfer rate of the image information is obtained by subtracting the transfer rate of the voice information and the transfer rate of the data information from the determined and set information transfer rate of the entire communication path.

An example of such a composite media terminal is shown in FIG.
3 shows. In the figure, 1 is a handset which is one of voice input / output means, 2 is a microphone which is one of voice input means, and 3 is a speaker which is one of voice output means. Four
Is a handset 1, a microphone 2, a speaker 3 as a voice input / output unit according to an instruction from a system control unit 14 described later
Switch function, gain adjustment function for adjusting the gain to adjust the volume level, on / off detection function for detecting whether the handset 1 is in an on-hook state or an off-hook state, and when the microphone 2 and the speaker 3 are used as voice input / output means. The voice interface unit has an echo canceling function for canceling echoes and a tone generating function for generating tones such as dial tone, ringing tone, busy tone and incoming tone.

Reference numeral 5 indicates 64 according to an instruction from the system control unit 14.
kbps PCM (A-law), 64 kbps PC
M (μ-law), 64 kbps / 56 kbps / 48
kbps (SB-ADPCM), 32 kbps ADP
CM, 16 kbps (for example, APC-AB), 8 kpb
Voice coding / decoding having a function of A / D (analog / digital) converting and coding a transmission voice signal according to a voice coding / decoding algorithm such as s, and a function of decoding and D / A converting a reception voice signal It is a chemical department. The voice encoding / decoding unit 5 has a voice encoding unit 5a and a voice decoding unit 5b.

Reference numeral 6 is one of the image inputting means, which is a camera for inputting a self-portrait, and 7 is one of the image inputting means, which is a document camera for inputting a picture or a drawing. Reference numeral 8 denotes a display unit (display means) for displaying an input image from the camera 6 or the document camera 7, a received image after being vacant, or an image from the system control unit 14. Reference numeral 9 denotes a switching function of the image input means, which is generated by an instruction from the system control unit 14,
The video interface unit has a display switching function and a display dividing function for each image described above, and a signal conversion function for electrically / physically matching the video signal of each image input / output. The video interface unit is an image input unit 9a.
And an image output unit 9b.

Reference numeral 10 denotes an image encoding / decoding unit having a function of A / D converting a transmitting image to encode and a function of decoding a receiving image to D / A convert. This image encoding / decoding unit 10 Is capable of band-compressing raw data of large-capacity image by various methods such as motion compensation, frame dropping, inter-frame compensation and intra-frame compensation, DCT transform, vector quantization transform, etc. Image data is traditionally enabled. The image encoding / decoding unit 10
It has an image encoding unit 10a and an image decoding unit 10b. At present, the basic interface of ISDN (Digital Integrated Services Network) line is 64 kbps, but CCIT is used as an image encoding method that can be transmitted even at this transmission speed.
H.T Recommendation Draft There is 261.

Reference numeral 11 is a data terminal for performing data communication. 12 is a data terminal 11 and a system control unit 1
4 is a data interface unit for notifying the demultiplexing unit 15 of the transmission data from 4 and notifying the data terminal 11 or the system control unit 14 of the reception data. Reference numeral 13 is an operation unit such as a keyboard or a touch panel used for inputting control information. 14 is a CPU, ROM, RAM,
It is equipped with an auxiliary storage device, a character generator, an image signal generation circuit, etc., monitors the state of each part, controls the entire device,
It is a system control unit that creates an operation / display screen and executes an application program according to the state.

Reference numeral 15 is audio data from the audio encoding / decoding unit 5, image data from the video encoding / decoding unit 10, data from the data interface unit 12, and BAS (Bit Allocation) from the system control unit 14.
Signal) is multiplexed in transmission frame units, and a reception frame is demultiplexed into each medium of a constituent unit and notified to each unit. H.264 of CCITT Recommendation.
221. A line interface unit 16 controls the line according to the ISDN user network interface.

[0016]

Problem 1 However, according to the above-mentioned conventional multimedia terminal, only mutual communication with one terminal is possible. When considering mutual communication with multiple points, it is necessary to have a plurality of line units. Further, when communicating images and sounds, since the communication capacity of the communication path is limited, encoding / decoding for compressing / decompressing data is performed. For this reason, it is necessary to have as many image decoding units as there are terminals that communicate with each other at the same time.

Among these, since the amount of data of audio compression is smaller than that of an image, it is possible to realize a plurality of decoding units without much cost increase.

However, in the case of image compression, since the algorithm is complicated and the data amount is large, the circuit scale becomes enormous and high-speed arithmetic processing is required. Further, since the storage capacity used becomes very large, it is possible to have a plurality of the same image decoding units, but it is not very realistic in view of economy.

Therefore, a first object of the present invention is to provide a terminal device capable of simultaneous communication with multimedia terminals at multiple points without having a plurality of image decoding units.

Problem 2 In this way, when considering mutual communication with multiple points, it is natural that the line section and the audio / image decoding section are required for the number of terminals simultaneously communicating. Furthermore, since the transmission capacity increases at multiple points, it is necessary to efficiently transmit each medium.

However, there is no means for notifying the transmitting terminal that the multipoint connection is made as a standard function. If such a function is added, the function can be used only between special terminals.

Further, when the transmitting terminal does not know that the partner terminal is connected at multiple points, useless data will be transferred. For example, when the screen is divided into four and the images at four points are monitored at the same time, the data transmitted as a full-size image becomes 1/4 when displayed on the screen, so much useless data is transferred. Will be done.

A second object of the present invention is to provide a terminal device capable of communicating with ordinary terminals at multiple points.

Problem 3 Further, when receiving image data from a plurality of terminals, usually, the image data is processed and combined in frame units.
If the frame transfer rate differs depending on each terminal,
The image data of each frame may be insufficient, and the decoding unit may malfunction.

A third object of the present invention is to easily compose a screen even when the transfer routes of frames are different between terminals.
Another object of the present invention is to provide a terminal device in which a malfunction does not occur even in the decoding unit.

Problem 4 When receiving image data from a plurality of terminals, if the number of terminals communicating at the same time is smaller than the number of terminals equal to the number of lines, the image data is processed and combined in frame units to obtain the image data. Shortage occurs, and a malfunction occurs in the decoding unit.

A fourth object of the present invention is to provide a terminal device in which even when the number of terminals communicating at the same time is smaller than the number of lines, the shortage is easily complemented to prevent the decoding section from malfunctioning. To do.

Problem 5 In the multipoint control device, as described above, in order to decode the compressed image once, and to encode after combining, this encoding / decoding unit is required for the number of terminals communicating at the same time. In addition, since the synthesizing / editing section and the large capacity also need to process the raw image data at high speed, the apparatus becomes very large-scaled and very expensive.

Further, when displaying images of terminals at multiple points on one screen, it is necessary to reduce and combine the images of the terminals, which further complicates the processing and reduces the size of the terminals. Only useless data will be sent.

Further, when the images of a plurality of points are displayed on one screen, it is difficult to determine which image is the image of the terminal at which point.

Therefore, a fifth object of the present invention is to provide a recording apparatus configured so that the relationship between the display image and the terminal point can be clearly identified.

Problem 6 Further, considering that the screen is enlarged, the movement is improved, and the image quality is improved at the point of particular interest among the images of a plurality of points, the high-speed image for all the decoding units is taken into consideration. Since data processing capability is required, there is a problem that the structure is extremely wasteful.

Therefore, in view of the above-mentioned point, the sixth object of the present invention is to realize a sufficient display mode required by dividing the displayed screen into a main screen and a sub screen. To provide a terminal device to obtain.

[0034]

In order to achieve such an object, a terminal device according to the present invention is a terminal capable of simultaneous communication with a maximum of N terminals (N: an integer of 2 or more) via a plurality of lines. A device, connected to each of said lines, 1 / N
A plurality of separating means for respectively extracting the encoded image data of the reduced screen format, and the 1 / N reduced screens of the encoded image data so that a maximum of N of the 1 / N reduced screens can be stored in one screen of a prescribed size. Display position control means for changing the position designation information indicating the allocation position on the screen of the encoded image data included in the header part, and extracted by the plurality of separating means and processed by the display position control means. Multiplexing means for multiplexing a plurality of coded image data of the 1 / N reduced screen format to generate coded data of the one-screen format of the specified size; and the specified processing processed by the multiplexing means. And a decoding means for decoding encoded data of one screen format of size.

In addition, the control method of the terminal device according to the present invention has a maximum of N units (N: an integer of 2 or more) via a plurality of lines.
A method for controlling a terminal device capable of simultaneous communication with a terminal, comprising: a plurality of separating steps, each of which is connected to each of the lines and extracts encoded image data of a 1 / N reduced screen format; / N Reduced screen size 1
A display position control step of changing the position designation information indicating the allocation position on the screen of the encoded image data included in the header part of the encoded image data so that a maximum of N pieces can be stored in the screen. , The plurality of encoded image data of the 1 / N reduced screen format extracted in the plurality of separation steps and processed in the display position control step are multiplexed to encode one screen format of the specified size. The method is characterized by including a multiplexing step of generating data and a decoding step of decoding the coded data of the one-screen format of the specified size processed in the multiplexing step.

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

Embodiment 1 FIG. 1 is a block diagram of a multimedia terminal according to an embodiment of the present invention. In the figure, 1 is a handset which is one of the audio input / output means of the present apparatus, 2 is a microphone which is one of the audio input / output means of the present apparatus, and 3 is one of the audio output means of the present apparatus Is a speaker. Reference numeral 4 designates a function of switching the handset 1, the microphone 2 and the speaker 3 as voice input / output means, a gain adjusting function for volume level adjustment, and whether the handset 1 is in an on-hook state or an off-hook state according to an instruction from the system control unit 14. An on / off detection function to detect, an echo cancellation function to eliminate echo when using the microphone 2 and speaker 3 as voice input / output means, and a tone generation function such as dial tone, ring tone, busy tone, and ring tone. It is a voice interface section.

In accordance with an instruction from the system control unit 14, 5 is
64 kbps PCM (A-law), 64 kbps
PCM (μ-law), 7 kHz audio (SB-A
DPCM), 32 kbps ADPCM, 16 kbps
(For example, APC-AB) A transmission voice signal is transmitted according to a voice encoding / decoding algorithm such as 8 kbps.
A voice encoding / decoding unit having a function of performing D / A conversion and encoding and a function of decoding a received voice signal and performing D / A conversion.

Reference numeral 6 is a camera for inputting a self-portrait, 7 is one of the image inputting means of the present apparatus and is a document camera for inputting a picture or drawing, and 8 is an input image from the camera 6 or the document camera 7. Image received from the other party and system control unit 1
4 is a display unit for displaying the image from FIG. Reference numeral 9 designates a switching function of the image input means according to an instruction from the system control unit,
Display switching and display division function of each image described above,
This is an image input / output unit having a signal conversion function for electrical / signal matching with the video signal of each image input / output unit.

Reference numeral 10 denotes an image coding / decoding unit having a function of A / D converting a transmission image to be encoded and a function of decoding a reception image to perform D / A conversion. In addition, band compression is performed by various methods such as motion compensation, frame dropping, interframe compensation and interframe compensation, DCT conversion, vector quantization conversion, etc., and the capacity is reduced to enable transmission on a digital line.

At present, the basic interface of the ISDN line is 64 kbps, but as an image coding method that can be transmitted even at this transmission speed, CCITT Recommendation Draft H.264. 26
There is one.

Reference numeral 11 is a data terminal for performing data communication. A data interface unit 12 notifies the demultiplexing unit 15 of the transmission data from the data terminal 11 and the system control unit 14 and also notifies the data terminal 11 or the system control unit of the reception data. Thirteen
Is an operation unit such as a keyboard or a touch panel used for inputting control information to control the apparatus. 14
Is provided with a CPU, a ROM, a RAM, an auxiliary storage device, a character generator, an image signal generation circuit, etc., monitors the state of each part, controls the entire device, creates an operation / display screen according to the state, and executes an application program. It is a system control unit for performing.

Reference numeral 15 is audio data from the audio encoding / decoding unit 5, image data from the video encoding / decoding unit 10,
Data from the data interface unit 12, BAS (Bit Allocatio) from the system control unit 14
n Signal) is multiplexed in a transmission frame unit, and a reception frame is separated into each medium of a constituent unit and notified to each unit. CCITT recommendation is H.264. There is 221.

The demultiplexing unit 15 corresponds to the line interface unit 16, and 16 is a line interface unit for controlling the line in accordance with the ISDN user network interface. The line interface unit 16 is composed of four units so that it can simultaneously communicate with four terminals. Demultiplexing unit 1
The numeral 5 corresponds to the line interface unit 16 and is composed of four pieces.

Reference numeral 17 denotes an image control section peculiar to this embodiment.

Next, a method of negotiating the terminal capability and changing the terminal capability will be described. First, ISDN
When communicating via a line, an outband signal (that is, Dch) is used to make a call, as shown in FIG.

As shown in FIG. 2, the call setup from the terminal A to the terminal B and the response from the terminal B to the terminal A enable Bch communication. Other communication channels include Dch, H0,
There is also H1 and the like, but only Bch will be described and the following will be omitted.

Next, according to the H. In accordance with 242, as shown in FIG. 3, an in-band signal procedure of Bch (that is, the inside of Bch is allocated to the data section and the control section and controlled by the control section) is performed. Since this control is called an in-channel control procedure, this name will be used hereinafter. To execute this in-channel control, a control bit is required in Bch, so that the frame structure is H.264. 221.

This frame structure is shown in FIG. The multi-frame structure of FIG. 4 is for Bch (64 kbps).

First, the multi-frame structure is based on 1 octet / 125 μsec as shown in FIG.
Frame = 80 octets, as shown in (b), 1 sub-timal frame = 2 frames, as shown in (c), 1 multi-frame = 8 sub-multi-frame structure, and sub-channel of 8 kbps in bit direction Are defined from # 1 to # 8.

However, only the # 8 sub-channel has a transfer rate of 6.4 kbps, and the control bit is F.
AS (Frame Alignment Signa)
l) and BAS (Bitrate Allocation)
Signal) is inserted.

The FAS and BAS enable in-channel control of Bch. FAS is used for frame and multi-frame synchronization. The BAS is used for exchanging information on terminal capabilities necessary for determining a multiplexing method such as subchannels or for setting capabilities. In particular, BAS
Indicates sub-multiframe (20
Switching is possible every msec).

Next, the procedure will be described with reference to FIG.

First, when Bch becomes communicable, both terminals A and B transmit FAS. The terminal capability at this time is mode 0 (audio and FAS, BAS
(Only mode). This FAS is searched by the partner terminal, and H.S. When the condition for establishing frame synchronization defined by 242 is satisfied, A in the bit configuration in the FAS shown in FIG. 5 is set to "0" and transmission is performed. When the terminal receives A = 0, it is confirmed that the other terminal has established frame synchronization.

Next, the transmission capability of the own terminal is transmitted to the partner terminal by BAS, and the terminal capabilities of the partner terminal are recognized by each other. This is so-called exchange of communication ability.

If mutual communication is possible at this point, data communication is started. If the capability needs to be changed, similarly, the desired terminal capability is transmitted as a command using BAS, and after the partner terminal completes setting the capability, data communication is started.

Data communication is independent for transmission and reception, and synchronization is established and terminal capability is set separately. Therefore,
In some cases, synchronization may be lost only in one direction, or the type of data may differ between transmission and reception.

When the data communication is completed and the call is disconnected, the disconnecting side (terminal A in FIG. 3) first sets the mode 0 using BAS. As a result, the in-channel control of Bch returns to the initial state.

Next, as shown in FIG. 2, disconnection and release are performed in the Dch out-band procedure to complete all communications.

FIG. 5 shows the bit structure in the FAS.

The A bit is an indication of out of frame synchronization. The E bit is an indication of whether a CRC error has occurred. C ₁ , C ₂ , C ₃ , and C ₄ are bits of CRC4. N1 to N5 are for multi-frame numbering.

R1 to R4 are channel numbers.

TEA is a terminal device alarm and is set to "1" when an input signal cannot be received or responded to due to a failure inside the terminal.

FIG. 6 shows the bit structure in the BAS.

As shown in FIG. 6A, the upper 3 bits represent the attribute, and the remaining 5 bits represent the attribute value of the attribute.

FIG. 6B shows the content of the attribute.
The attribute value is, for example, a transfer rate value, a codec type,
Parameter values specific to each media are defined.

In this embodiment, a plurality of terminals can be connected to the line and can communicate with each other at the same time. However, it is necessary to limit the image transmission capability to all terminals connected at that time.

Specifically, at the time of transmitting the capability BAS shown in FIG. 3, the image format (QCIF or CIF) included in the terminal capability 2 of FIG. 6B is set to QCIF, and all terminals are set. From, only QCIF of image data can be received. A detailed description of CIF and QCIF will be given later, but in brief, the number of pixels of QCIF is CIF.
It is defined as 1/4 of. However, when the number of connected terminals is one, it is as usual.

Further, the number of connected terminals is changed from a plurality of terminals to one terminal.
When there is a change , the image format is changed from QCIF to CIF and the capability BAS is transmitted to the terminal, so that the transmission capability can be limited .

[0084] On the contrary, the connection terminal is strange to multiple terminal from the first terminal
If it becomes more, relative to all terminals, the image format by sending a capability BAS set to QCIF,
Limitations can also be placed on the transmission capacity.

In this way, by controlling the image receiving terminal to limit the image format received from the partner terminal, desired image data can be received from the transmitting terminal.

FIG. 7 shows a block configuration of the image controller 17 and its peripheral portion. In the figure, reference numeral 18 denotes a BCH / image separation unit that performs BCH error correction processing on the data received from the demultiplexing unit 15, removes the error correction frame, and forms image data. 19 is a GN (GOB Number) in the GOB header in the image data.
It is a GN conversion unit that converts r). Reference numeral 20 is a data buffer, 21 is a gate circuit, 22 is a BCH / image multiplexing unit for adding BCH correction parity to form an error correction frame, 2
A control unit 3 controls each of the above units in response to an instruction from the system control unit 14.

The data from the demultiplexing unit 15 is an error correction frame having a format as shown in FIG.

In one frame, the error correction frame bit is 1 bit, the fill identifier is 1 bit, and the image data is 49 bits.
2 bits and error correction parity are composed of 18 bits and 512 bits. Further, this frame is composed of 8 frames to form one multi-frame.

The BCH / image separation unit 18 synchronizes multi-frames with error correction frame bits, performs error correction processing on the image data with the error correction parity, and validates / invalidates the image data with the fill identifier. The image data is checked, and if valid, the image data is transferred to the GN converter 19.

The GN converter 19 decodes the contents of the received image data. The structure of this image data is shown in FIG.
It has a multiplexed frame structure as shown in. As shown in (a) of this figure, F is added to the beginning of one frame of data on the screen.
H (frame header) is added, and one block obtained by dividing the screen into 12 is sequentially transmitted as GOB from GOB1 to GOB12.

FIG. 11 shows this GOB division method.

H. In the H.261 Recommendation Draft, a plurality of different standards such as NTSC, PAL, and digital television standards exist for the video signal to be handled, and therefore, a video signal format common throughout the world is adopted so that they can communicate with each other. This format is called the CIF format, and the number of samples is 352 pixels × 288 lines with luminance Y.
The color differences Cr and Cb are defined by 176 pixels × 144 lines. Further, 1/4 of CIF is called QCIF format, and the number of samples is defined by luminance Y of 176 pixels × 144 lines and color differences Cr and Cb of 88 pixels × 72 lines.

In GOB, the number of samples is luminance 176 pixels × 48.
The lines and the color differences Cr and Cb are defined as 88 pixels × 24 lines and correspond to 1/12 of CIF and 1/3 of QCIF.

Here, as shown in FIG. 11, in CIF as GOB numbers, GOB1 to GOB12, QC
GOB1, GOB3, and GOB5 are assigned in the IF.

FIG. 10B shows the contents of FH (frame header) and GOB of FIG. 10A. This FH
The (frame header) is composed of PSC, TR, and PTYPE. PSC is the frame start code and is 20
Bit "0000 0000 0000 0001"
TR is a frame number and uses a 5-bit value from "1" to "30".
PE is type information (6 bits), split screen instruction information, document camera instruction information, screen freeze release, information source format instruction information (CIF, QCIF)
It is included.

The GOB header includes GBSC, GN, and GQU.
It is composed of ANT. GBSC is a GOB start code and is 16-bit "0000 0000 0000."
0001 ”. GN is a GOB number of 4 bits and is used from“ 1 ”to“ 12. ”When GN is“ 0 ”, it is used by PSC.
Both GBSC + GN of OB can be regarded as a continuous value of 20 bits. GQUANT is quantization characteristic information, and includes 5 bits of quantization step size information.

The MB header is a macro block (hereinafter referred to as MB
(Referred to as)) is a header of a pixel block. MB
Is composed of 33 MBs and 1 GOB.
Is 8 pixels x 8 lines of 4 luminance signals (Y) and 8 pixels x 8
It is composed of two line color difference signals (Cb, Cr).

Here, as the number of each block, numbers 1 to 4 are assigned to Y, numbers 5 to Cb, and numbers 6 to Cr. This MB header contains MBA, MTYPE, and M
It is composed of QUANT, MVD and CBP.

MBA is a macroblock address representing the position of the MB, only the first MB is an absolute value, and thereafter, it is a differential variable length code. MTYPE is MB type information, INTRA (intra-frame coding), IN
A processing type such as TER (inter-frame differential coding), MC (motion-compensation inter-frame differential coding), FIL (filter) that has processed the data of the MB is inserted. MQUANT is the same as GQUANT.

MVD is motion vector information. CBP
Is a significant block pattern, and 4 in Y of MB
And the number of the effective pixel block of Cr and Cb is included as information. After this MB header, the compression-encoded pixel data is as described above, that is, Y4.
Pixel blocks that have become significant blocks among the individual Cr and Cb are in numerical order.

Here, PSC of FH and GB of GOB
The SC and GN have a unique data pattern that does not exist in other data so that the header of FH or GOB can be searched on the decoder side.

In the GN conversion section 19 shown in FIG.
It has a function of searching the GBSC of the image data multiplexed frame shown in (4), decoding the GN after the GBSC, and replacing the value of this GN with another value.

The buffer 20 buffers the image data from the GN conversion unit 19 to make the transfer timing asynchronous, and the image data is transferred to the BCH / image multiplexing unit 22 at the timing of the gate circuit 21. BCH / image multiplex unit 2
2 returns this image data to the error correction frame shown in FIG. 9 and transfers it to the image decoding unit 10 (see FIG. 1).

FIG. 8 shows a configuration in which a plurality of image control units 17 described above are used to perform simultaneous reception from four terminals. In this figure, for the image data received from the terminal A, the line interface section 16 (a), the demultiplexing section 15 (a), and the image control section 17 (a) have the same configuration as in FIG. , The above process is performed,
The same processing is performed on the received image data of the terminals B, C, and D. Therefore, the received data from each terminal can be simultaneously processed in parallel.

At the time of exchanging the transmission capability from each terminal, the image reception transmission capability of the own terminal is transmitted to the partner terminal as QCIF, and the image data is received from the partner terminal in the QCIF format. ), The four QCIFs received from the four terminals are reconfigured so as to fit in the CIF.

Therefore, the system control unit 1 shown in FIG.
4 instructs each image control unit 17 (a) to 17 (d) at which position on the CIF screen the image from each terminal is to be displayed. In each image control unit, the GN conversion unit 19 converts the GN value as instructed.

For example, to display in QCIF (A) of FIG. 12 (a), the GN value is not converted as shown in FIG. 12 (b). In the case of QCIF (B), the GN value is replaced with a value obtained by adding +1 to the current GN value. Similarly, QCIF
In case of (C), the value obtained by adding 6 to the current GN value, QCI
In the case of F (D), the current GN value is replaced with a value that is +7.

Each QCI whose GN value is converted in this way
The image data of F is buffered by each buffer unit, time-division multiplexed by each gate circuit, and then CIF.
Image data multiplexed frame.

The operation of this gate circuit is shown in FIG. Here, the signals a, b, c, d are image data from the BCH / image separation unit 18 included in each image control unit 17. Since the received image data is QCIF, F
It is composed of H and 3 GOBs. The image data is GN converted and buffered as instructed.

In the example of FIG. 13, the terminal A is connected to Q of FIG.
This is an example for displaying CIF (A), terminal B on QCIF (B), terminal C on QCIF (C), and terminal D on QCIF (D).

Further, the signals f, g, h, i shown in FIG.
Are ON / OFF signals of the gate circuits included in the image control units A, B, C and D (see FIG. 8). That is, the gate is opened at the time of "H", and the gate is closed at the time of "L". By controlling this signal, GOB of QCIF of each terminal is multiplexed as shown by signal e, and 12 GOBs of CIF are transferred to the BCH / image multiplexing unit 22.

The image decoding unit 10 decodes the image data in the CIF format, and the decoded image is displayed on the screen of the display unit 8 via the image output unit 9. On this screen, the images of four terminals are simultaneously displayed by dividing the screen into four. In this way, the four terminals can simultaneously receive the images and simultaneously display the images on the screen.

Embodiment 2 FIG. 14 is a block diagram showing details of an image control unit according to another embodiment of the present invention. In this figure, 30 is
This is a BCH / image separation unit for performing BCH error correction processing on the data received from the demultiplexing unit 15 and removing the error correction frame to obtain image data. An FH decoding unit 31 has a function of decoding the frame header (FH) in the image data. Reference numeral 32 is an FH deletion unit that removes the FH from the image data. 33 is a GN (GOB Number) in the GOB header in the image data.
It is a GN conversion unit that converts r). 34 is a data buffer, 35 is a GOB generator for generating GOB data, 36
Is a gate circuit, 37 is an FH insertion unit for inserting FH into image data without FH, 38 is a BCH / image multiplexing unit for adding an error correction frame with BCH correction parity, 39 is a changeover switch, 40 is a system control unit The control unit controls each of the above units in response to an instruction from 14.

The data from the demultiplexer 15 is an error correction frame having a format as shown in FIG.

In one frame, the error correction frame bit is 1 bit, the fill identifier is 1 bit, and the image data is 49 bits.
2 bits and error correction parity are composed of 18 bits and 512 bits. Further, this frame is composed of 8 frames to form one multi-frame.

The BCH / image separation unit 30 synchronizes multi-frames with error correction frame bits, performs error correction processing on the image data with the error correction parity, and checks the validity / invalidity of the image data with the fill identifier. If valid, it is transferred to the FH deletion unit 32. At the same time, the data is also transferred to the FH decoding unit 31.

The FH decoding section 31 starts the decoding when it enters the FH search and finds the FH, and at the same time notifies the FH deleting section 32, and in response to this, the FH deleting section 32 deletes the FH. The FH decoding unit 31 notifies the control unit 40 of the decoding result.

Next, the image data from which the FH has been deleted by the FH deleting section 32 is transferred to the GN converting section 33. The GN converter 33 starts decoding the contents of this image data. The structure of this image data is a multiplexed frame structure as shown in FIG. However, for convenience of description, the description will be made with FH added.

FIG. 10A shows the structure of GOB blocks. In this way, FH is added to the head of data of one frame of the screen, and one block obtained by dividing the screen into 12 is sequentially transmitted as GOB from GOB1 to GOB12. The GOB dividing method is as shown in FIG. 11, and thus the description thereof is omitted.

FIG. 10B shows the details of the FH (frame header) and GOB of FIG. 10A. This FH is composed of PSC, TR and PTYPE. PSC is a frame start code and is 20-bit "0000 0000 00000001 0000".
Consists of. TR is a frame number and uses a 5-bit value from "1" to "30". PTYPE is type information (6 bits), and includes split screen instruction information, document camera instruction information, screen freeze release, and information source format instruction information (CIF, QCIF). That is, the FH decoding unit 31 notifies the control unit 40 of the decoding result of the above content.

The GOB header is composed of GBSC, GN and GQU.
It is composed of ANT. GBSC is a GOB start code and is 16-bit "0000 0000 0000."
0001 ”. GN is a GOB number of 4 bits and is used from“ 1 ”to“ 12. ”When GN is“ 0 ”, it is used by PSC.
Both GBSC + GN of OB can be regarded as a continuous value of 20 bits.

GQUANT is 5 bits of quantization characteristic information and includes information on the quantization step size.

The MB header is a header of a pixel block called a macro block (MB). MB is 33 MB
One GOB is composed of 4 luminance signals (Y) of 8 pixels × 8 lines and 2 color difference signals (Cb, Cr) of 8 pixels × 8 lamps.

Here, as the numbers of the respective blocks, numbers 1 to 4 are assigned to Y, 5 to Cb, and 6 to Cr. This MB header contains MBA, MTYPE, and M
It is composed of QUANT, MVD and CBP.

MBA is a macroblock address indicating the position of the MB, only the first MB is an absolute value, and thereafter, it is a differential variable length code. MTYPE is MB type information, INTRA (intra-frame coding), IN
A processing type such as TER (inter-frame differential encoding), MC (motion-compensated inter-frame differential encoding), FIL (filter) that has been processed on the MB data is inserted. MQUANT is the same as GQUANT.

MVD is motion vector information. CBP
Is a significant block pattern, and 4 in Y of MB
And the number of the effective pixel block of Cr and Cb is included as information. After this MB header, the compression-encoded image data is as described above, that is, Y4.
Pixel blocks that have become significant blocks among the individual Cr and Cb are in numerical order.

Here, PSC of FH and GB of GOB
The SC and GN have a unique data pattern that does not exist in other data so that the FH or GOB header can be searched by the decoder side.

The GN conversion unit 33 shown in FIG. 14 decodes the contents of the above data, searches the GBSC of the image data multiplexed frame of FIG. 10, decodes the GN after the GBSC, and the value of this GN. It has the function of substituting other values.

The buffer unit 34 buffers the image data from the GN conversion unit 33, makes the transfer timing asynchronous, and transfers the image data to the FH insertion unit 37 at the timing of the gate unit 36.

The changeover switch 39 is controlled by the control unit 40, and the control unit 40 monitors the capacity condition of the buffer unit 34, and if there is not enough transferable data in the buffer unit 34, the GOB generation unit 35 side is selected. It switches and transfers the data of the GOB generation unit 35 to the gate unit 36. Conversely, if there is sufficient data that can be transferred to the buffer unit 34, the data is stored in the buffer unit 34 and transferred to the gate unit 36 by switching to the buffer side.

The GOB generator 35 receives the GO from the controller 40.
The GN value of B is designated, and GOB corresponding to the GN value is generated. Then, when the changeover switch 39 is switched to the GOB generation unit side, it is output to the gate unit 36.

In the FH inserting section 37, the FH deleting section 32 is applied to the image data without FH output from the gate section 36.
The FH generated by the control unit 40 is inserted at the beginning of the image data in place of the FH deleted in. The FH inserting section 37 is not necessary when the terminals are facing each other, but is effective when connecting at multiple points. This will be described later.

After returning to the original image data in the FH inserting section 35, the BCH / image multiplexing section 38 converts this image data into the image data shown in FIG.
The image decoding unit 10 returns to the error correction frame shown in FIG.
(See FIG. 1).

FIG. 15 shows an example of a configuration in which a plurality of image control units 17 described above are used to simultaneously receive from four terminals. In this figure, the image data received from the terminal A is the same as the portion enclosed by the dotted line in FIG. 14 by the line interface unit 16 (a), the demultiplexing unit 15 (a), and the image control basic unit A50 (a). Therefore, the above-described processing is performed, and the same processing is performed on the received image data of the terminals B, C, and D.

That is, in this case, the overall control unit 52 for controlling the four image control basic units 50 (a) to 50 (d) is added. By this overall control unit 52, the image control basic unit A is added. Image data without FH from 4 to D is sent to each gate unit to perform time division multiplexing of the image data, and further, the FH insertion unit 37 is instructed to add FH to the head of the multiplexed image data. . The image data thus generated is regenerated into an error correction frame by the BCH / image multiplexing unit 38 and transferred to the image decoding unit.

As described above, the received data from each terminal can be processed in parallel at the same time.

At the time of exchanging the transmission capability from each terminal, the image reception transmission capability of the terminal itself is transmitted to the partner terminal as QCIF, and the image data is received from the partner terminal in the QCIF format. ), The four QCIFs received from the four terminals are reconfigured so as to fit in the CIF.

Therefore, the system control unit 14 instructs the overall control unit 52 and each of the image control basic units 50 (a) to 50 (d) at which position on the CIF screen the image from each terminal should be displayed. . Each image control basic unit 50 (a) -50
In (d), the GN conversion unit 33 converts the GN value as instructed.

For example, in order to display in QCIF (A) of FIG. 12 (a), the GN value is not converted as shown in FIG. 12 (b). In the case of QCIF (B), the GN value is replaced with a value obtained by adding +1 to the current GN value. Similarly, QCIF
In case of (C), the value obtained by adding 6 to the current GN value, QCI
In the case of F (D), the current GN value is replaced with a value that is +7.

Each QCI whose GN value is converted in this way
The F image data is buffered by each buffer unit 34 and time-division multiplexed by each gate unit 36 to form a CIF image data multiplexed frame.

FIG. 16 is a timing chart showing the operation of the image controller. In the figure, signals a, b, c, and d are BC in each of the image control basic units 50 (a) to 50 (d).
This is an image data signal transferred from the H / image separation unit 30 to the GN conversion unit 33 via the FH deletion unit 32. For example, the A ₁ block of the signal a indicates a block of image data of one frame received from the A terminal. The numbers are serial numbers. In other words, the content of this image data block is 3 GOBs because the received image data is QCIF.
It is composed of.

For these image data blocks, the GN converter 32 is instructed by the system controller 14 as instructed.
The GN value is converted by and is buffered by the buffer unit 34.

The signals e, f, g and h indicate the image data stored in the buffer section 34. For example, as for the A ₁ block of the signal e, this A ₁ block is the buffer unit 3
4 shows the time from the completion of the input to 4 to the start of the transfer to the gate unit 36.

The signal i is sent from the overall control unit 52 to the FH insertion unit 3
7 to the image control basic units 50 (a) to 50
The timing for inserting FH in the image data frame obtained by multiplexing the image data of QCIF from (d) and reconstructing it in CIF is shown. The FH inserting unit 37 receives the F signal including the FH information instructed by the overall control unit 52 by the i signal.
Insert H.

The signals k, m, o, and q are supplied to each image control basic unit 5
It is a timing signal of the gate unit 36 included in 0 (a) to 50 (d). When these signals are "H", the data from the buffer section 34 or GOB generation section 35 is FH.
Transfer to the insertion unit 37.

The signals j, l, n, p are the signals k,
Image data corresponding to m, o, and q is transferred from the gate unit 36 to the FH insertion unit 37. For example,
A ₁ of the signal j indicates an image data block from the buffer unit 34 of the image control basic unit A. The block marked with X means that the insufficient GOB block is generated from the GOB generation unit 35 to compensate for the insufficient amount when the data is not available in the buffer unit 34 or when the timing needs to be shifted. ing.

In FIG. 1, terminal A has QCIF (D) shown in FIG. 12A, terminal B has QCIF (C), and terminal C has Q.
CIF (B) and terminal D become QCIF (A) and are displayed on the screen.

The image data block generated by this GOB generation unit 35 is a GOB block having the same GN value as the GN value replaced by the GN conversion unit 33 as the GN value of GOB. As for the contents of the GOB block, MB does not need to be added unless there is coefficient data, so GO
Although the B header alone may be used, “0000 0001 11” may be provided immediately after the GOB header or immediately after the MB block.
It is possible to insert a special code having no meaning, such as "1", for the next timing. This code is discarded in the image decoding unit.

Furthermore, when the number of terminals that are communicating simultaneously is three or two, since there is always a shortage on the screen, the image control basic unit corresponding to the terminal that is not communicating has the changeover switch 39. Always switch to the GOB generation unit 35 side, and
The GOB block from the OB generator 35 is always transferred.

Here, the contents of the GOB block at this time may be a meaningless special code as described above, but in this case, since the decoding section 10 discards the screen and the screen is not updated, the screen is not updated. In order to clear, all the MBs are provided with coefficient data such that they become white or black data and prepared in the INTRA mode so that they are transferred periodically. Alternatively, data that is encoded in advance so that a message of some kind is displayed is prepared and transferred.

The signal r in FIG. 16 is an image data signal output from the FH inserting section 37. Here, the blackened portions indicate the FH of each frame. As shown in the signal r, three GOCs of QCIF of each terminal are multiplexed and 12 GOBs of CIF are transferred to the BCH / image multiplexing unit 38.

The image encoding unit 10 decodes the image data in the CIF format, and the decoded image is displayed on the screen of the display unit 8 via the image output unit 9. On this screen, the images of four terminals are simultaneously displayed by dividing the screen into four. In this way, the four terminals can simultaneously receive the images and simultaneously display the images on the screen.

Embodiment 3 FIG. 17 is a block diagram showing the configuration of a multimedia terminal according to an embodiment of the present invention. In the figure, 1 is a handset which is one of the voice input / output means of the present apparatus, and 2 is
The microphone 3 which is one of the audio input / output means of this device is a speaker which is one of the audio output means of this device. Four
Is a function of switching between the handset 1, the microphone 2 and the speaker 3 as an audio input / output unit according to an instruction from the system control unit 14, and a gain adjusting function for adjusting a volume level.
An on / off detection function for detecting whether the handset 1 is in an on-hook state or an off-hook state, an echo cancel function for canceling an echo when the microphone 2 and the speaker 3 are used as voice input / output means, a dial tone, a ring tone, It is a voice interface unit that has a tone generation function for busy tones and ring tones.

In accordance with an instruction from the system control unit 14, 5 is
64 kbps PCM (A-law), 64 kbps
PCM (μ-law), 7 kHz audio (SB-A
DPCM), 32 kbps ADPCM, 16 kbps
(For example, APC-AB) A transmission voice signal is transmitted according to a voice encoding / decoding algorithm such as 8 kbps.
A voice encoding / decoding unit having a function of performing D / A conversion and encoding and a function of decoding a received voice signal and performing D / A conversion.

Reference numeral 6 is a camera for inputting a self-portrait, 7 is one of the image inputting means of the present apparatus and is a document camera for inputting a picture or drawing, and 8 is an input image from the camera 6 or the document camera 7. Image received from the other party and system control unit 1
4 is a display unit for displaying the image from FIG. Reference numeral 9 designates a switching function of the image input means according to an instruction from the system control unit,
Display switching and display division function of each image described above,
This is an image input / output unit having a signal conversion function for electrical / signal matching with the video signal of each image input / output unit.

Reference numeral 10 denotes an image encoding / decoding unit having a function of A / D converting a transmission image to be encoded and a function of decoding a reception image to perform D / A conversion. In addition, band compression is performed by various methods such as motion compensation, frame dropping, interframe compensation and interframe compensation, DCT conversion, vector quantization conversion, etc., and the capacity is reduced to enable transmission on a digital line.

At present, the basic interface of the ISDN line is 64 kbps, but as an image coding method which can be transmitted even at this transmission speed, CCITT Recommendation Draft H.264. 26
There is one.

Reference numeral 11 is a data terminal for performing data communication. A data interface unit 12 notifies the demultiplexing unit 15 of the transmission data from the data terminal 11 and the system control unit 14 and also notifies the data terminal 11 or the system control unit of the reception data. Thirteen
Is an operation unit such as a keyboard or a touch panel used for inputting control information to control the apparatus. 14
Is provided with a CPU, a ROM, a RAM, an auxiliary storage device, a character generator, an image signal generation circuit, etc., monitors the state of each part, controls the entire device, creates an operation / display screen according to the state, and executes an application program. It is a system control unit for performing.

Reference numeral 15 denotes audio data from the audio encoding / decoding unit 5, image data from the video encoding / decoding unit 10,
Data from the data interface unit 12, BAS (Bit Allocatio) from the system control unit 14
n Signal) is multiplexed in a transmission frame unit, and a reception frame is separated into each medium of a constituent unit and notified to each unit. CCITT recommendation is H.264. There is 221.

The demultiplexing unit 15 corresponds to the line interface unit 16 and is a line interface unit for controlling the line according to the ISDN user network interface. The line interface unit 16 is composed of four units so that it can simultaneously communicate with four terminals. Demultiplexing unit 1
The numeral 5 corresponds to the line interface unit 16 and is composed of four pieces.

Reference numeral 17 denotes an image control section peculiar to this embodiment.

Reference numeral 61 is a superimposing section for embedding character characters from the system control section 14 into the image-decoded image data of this embodiment.

Next, a method of negotiating the terminal capability and changing the terminal capability will be described. First, ISDN
When communicating via a line, an outband signal (that is, Dch) is used to make a call, as shown in FIG.

As shown in FIG. 2, Bch communication becomes possible by call setup from terminal A to terminal B and response from terminal B to terminal A. Other communication channels include Dch, H0,
There is also H1 and the like, but only Bch will be described and the following will be omitted.

[0165] Next, using this Bch that has become communicable, Recommendation H.264 is established. In accordance with 242, as shown in FIG. 3, an in-band signal procedure of Bch (that is, the inside of Bch is allocated to the data section and the control section and controlled by the control section) is performed. Since this control is called an in-channel control procedure, this name will be used hereinafter. To execute this in-channel control, a control bit is required in Bch, so that the frame structure is H.264. 221.

This frame structure is shown in FIG. The multi-frame structure of FIG. 4 is for Bch (64 kbps).

First, the multi-frame structure is based on 1 octet / 125 μsec as shown in FIG.
Frame = 80 octets, as shown in (b), 1 sub-timal frame = 2 frames, as shown in (c), 1 multi-frame = 8 sub-multi-frame structure, and sub-channel of 8 kbps in bit direction Are defined from # 1 to # 8.

However, only the # 8 sub-channel has a transfer rate of 6.4 kbps, and the control bit is F.
AS (Frame Alignment Signa)
l) and BAS (Bitrate Allocation)
Signal) is inserted.

The FAS and BAS enable in-channel control of Bch. FAS is used for frame and multi-frame synchronization. The BAS is used for exchanging information on terminal capabilities necessary for determining a multiplexing method such as subchannels or for setting capabilities. In particular, BAS
Indicates sub-multiframe (20
Switching is possible every msec).

Next, the procedure will be described with reference to FIG.

First, when Bch becomes communicable, both terminals A and B transmit FAS. The terminal capability at this time is mode 0 (audio and FAS, BAS
(Only mode). This FAS is searched by the partner terminal, and H.S. When the condition for establishing frame synchronization defined by 242 is satisfied, A in the bit configuration in the FAS shown in FIG. 5 is set to "0" and transmission is performed. When the terminal receives A = 0, it is confirmed that the other terminal has established frame synchronization.

Next, the transmission capability of the own terminal is transmitted to the partner terminal by BAS, and the terminal capabilities of the partner terminal are recognized by each other. This is so-called exchange of communication ability.

If the two can communicate with each other at this point, data communication is started. If the capability needs to be changed, similarly, the desired terminal capability is transmitted as a command using BAS, and after the partner terminal completes setting the capability, data communication is started.

Data communication is independent for transmission and reception, and synchronization is established and terminal capability is set separately. Therefore,
In some cases, synchronization may be lost only in one direction, or the type of data may differ between transmission and reception.

When the data communication is completed and the call is disconnected, the disconnecting side (terminal A in FIG. 3) first sets the mode 0 using BAS. As a result, the in-channel control of Bch returns to the initial state.

Next, as shown in FIG. 2, disconnection and release are performed in the Dch out-band procedure to complete all communications.

FIG. 5 shows the bit structure in the FAS.

The A bit is an indication of out of frame synchronization. The E bit is an indication of whether a CRC error has occurred. C ₁ , C ₂ , C ₃ , and C ₄ are bits of CRC4. N1 to N5 are for multi-frame numbering.

R1 to R4 are channel numbers.

TEA is a terminal device alarm and is set to "1" when an input signal cannot be received or responded to due to a failure inside the terminal.

FIG. 6 shows the bit structure in the BAS.

As shown in FIG. 6A, the upper 3 bits represent the attribute, and the remaining 5 bits represent the attribute value of the attribute.

FIG. 6B shows the content of the attribute.
The attribute value is, for example, a transfer rate value, a codec type,
Parameter values specific to each media are defined.

In the present embodiment, it is possible to connect to a plurality of terminals via a line and simultaneously communicate with each other, but it is necessary to limit the image transmission capability to all terminals connected at that time.

Specifically, at the time of transmitting the capability BAS shown in FIG. 3, the image format (QCIF or CIF) included in the terminal capability 2 of FIG. 6B is set to QCIF, and all the terminals are set. From, only QCIF of image data can be received. A detailed description of CIF and QCIF will be given later, but in brief, the number of pixels of QCIF is CIF.
It is defined as 1/4 of. However, when the number of connected terminals is one, it is as usual.

In addition, the number of connected terminals is changed from a plurality of terminals to one terminal.
When there is a change , it is also possible to change the image format from QCIF to CIF and change the transmission capability by transmitting the capability BAS to the terminal.

[0187] On the contrary, the connection terminal is strange to multiple terminal from the first terminal
If it becomes more, relative to all terminals, the image format by sending a capability BAS set to QCIF,
Limitations can also be placed on the transmission capacity.

In this way, by controlling the image receiving terminal to limit the image format received from the partner terminal, it becomes possible to receive desired image data from the transmitting terminal.

FIG. 7 is a block diagram showing details of the image controller. In the figure, reference numeral 30 performs BCH error correction processing on the data received from the demultiplexing unit 15,
BC for removing error correction frames and converting to image data
H / image separation unit. An FH decoding unit 31 has a function of decoding the frame header (FH) in the image data. 32 is an F that removes the FH from the image data.
This is an H deletion unit. 33 is a GN for converting GN (GOB Number) in the GOB header in the image data
It is a conversion unit. 34 is a data buffer, 35 is a GOB generator for generating GOB data, 36 is a gate circuit, and 37 is a gate circuit.
Is an FH insertion section for inserting FH into image data without FH,
Reference numeral 38 is a BCH / image multiplexing unit for adding an error correction frame with BCH correction parity, 39 is a changeover switch, 40
Is a control unit that controls each of the above units in response to an instruction from the system control unit 14.

The data from the demultiplexing unit 15 is an error correction frame having a format as shown in FIG.

In one frame, the error correction frame bit is 1 bit, the fill identifier is 1 bit, and the image data is 49 bits.
2 bits and error correction parity are composed of 18 bits and 512 bits. Further, this frame is composed of 8 frames to form one multi-frame.

The BCH / image separation unit 30 synchronizes multi-frames with error correction frame bits, performs error correction processing on the image data with the error correction parity, and checks the validity / invalidity of the image data with the fill identifier. If valid, it is transferred to the FH deletion unit 32. At the same time, the data is also transferred to the FH decoding unit 31.

The FH decoding unit 31 starts the decoding when it enters the FH search and finds the FH, and at the same time notifies the FH deleting unit 32, and in response to this, the FH deleting unit 32 deletes the FH. The FH decoding unit 31 notifies the control unit 40 of the decoding result.

Next, the image data from which the FH has been deleted by the FH deleting section 32 is transferred to the GN converting section 33. The GN converter 33 starts decoding the contents of this image data. The structure of this image data is a multiplexed frame structure as shown in FIG. However, for convenience of description, the description will be made with FH added.

FIG. 10A shows the structure of GOB blocks. In this way, FH is added to the head of data of one frame of the screen, and one block obtained by dividing the screen into 12 is sequentially transmitted as GOB from GOB1 to GOB12. The GOB dividing method is as shown in FIG. 11, and thus the description thereof is omitted.

H. In the H.261 Recommendation Draft, a plurality of different standards such as NTSC, PAL, and digital television standards exist for the video signal to be handled, and therefore, a video signal format common throughout the world is adopted so that they can communicate with each other.

This format is called CIF format, and the number of samples is 352 pixels × 288 lines with luminance Y.
The dyes Cr and Cb are defined by 176 pixels × 144 lines.

Further, 1/4 of CIF is called QCIF format, and the number of samples is defined by luminance Y of 176 pixels × 144 lines and dyes Cr and Cb of 88 pixels × 72 lines.

In GOB, the number of samples is luminance 176 pixels × 48.
The line is defined by 88 pixels of pigment Cr and Cb × 24 lines, and corresponds to 1/3 of CIFno1 / 12 and QCIF. Here, as shown in FIG. 11, the GOB number is CI.
F is GOB1 to GOB12, QCIF is GOB
1, GOB3, GOB5 are assigned.

FIG. 10B shows the details of the FH (frame header) and GOB of FIG. 10A. This FH is composed of PSC, TR and PTYPE. PSC is a frame start code and is 20-bit "0000 0000 00000001 0000".
Consists of. TR is a frame number and uses a 5-bit value from "1" to "30". PTYPE is type information (6 bits), and includes split screen instruction information, document camera instruction information, screen freeze release, and information source format instruction information (CIF, QCIF). That is, the FH decoding unit 31 notifies the control unit 40 of the decoding result of the above content.

The GOB header includes GBSC, GN, and GQU.
It is composed of ANT. GBSC is a GOB start code and is 16-bit "0000 0000 0000."
0001 ”. GN is a GOB number of 4 bits and is used from“ 1 ”to“ 12. ”When GN is“ 0 ”, it is used by PSC.
Both GBSC + GN of OB can be regarded as a continuous value of 20 bits.

GQUANT is 5 bits of quantization characteristic information, and includes information on the quantization step size.

The MB header is a header of a pixel block called a macro block (MB). MB is 33 MB
One GOB is composed of 4 luminance signals (Y) of 8 pixels × 8 lines and 2 color difference signals (Cb, Cr) of 8 pixels × 8 lamps.

Here, as the numbers of the respective blocks, numbers 1 to 4 are assigned to Y, numbers 5 to Cb, and numbers 6 to Cr. This MB header contains MBA, MTYPE, and M
It is composed of QUANT, MVD and CBP.

MBA is a macroblock address indicating the position of the MB, only the first MB is an absolute value, and thereafter, it is a variable length code of the difference. MTYPE is MB type information, INTRA (intra-frame coding), IN
A processing type such as TER (inter-frame differential encoding), MC (motion-compensated inter-frame differential encoding), FIL (filter) that has been processed on the MB data is inserted. MQUANT is the same as GQUANT.

MVD is motion vector information. CBP
Is a significant block pattern, and 4 in Y of MB
And the number of the effective pixel block of Cr and Cb is included as information. After this MB header, the compression-encoded image data is as described above, that is, Y4.
Pixel blocks that have become significant blocks among the individual Cr and Cb are in numerical order.

Here, PSC of FH and GB of GOB
The SC and GN have a unique data pattern that does not exist in other data so that the FH or GOB header can be searched by the decoder side.

The GN conversion unit 33 shown in FIG. 7 decodes the contents of the above-mentioned data, searches the GBSC of the image data multiplexed frame of FIG. 10, decodes the GN after the GBSC, and the value of this GN. It has the function of substituting other values.

The buffer unit 34 buffers the image data from the GN conversion unit 33, makes the transfer timing asynchronous, and transfers the image data to the FH insertion unit 37 at the timing of the gate unit 36.

The change-over switch 39 is controlled by the control unit 40, and the control unit 40 monitors the capacity condition of the buffer unit 34. If there is not enough transferable data in the buffer unit 34, the changeover switch 39 is set to the GOB generation unit 35 side. It switches and transfers the data of the GOB generation unit 35 to the gate unit 36. Conversely, if there is sufficient data that can be transferred to the buffer unit 34, the data is stored in the buffer unit 34 and transferred to the gate unit 36 by switching to the buffer side.

The GOB generation unit 35 receives the GO from the control unit 40.
The GN value of B is designated, and GOB corresponding to the GN value is generated. Then, when the changeover switch 39 is switched to the GOB generation unit side, it is output to the gate unit 36.

In the FH inserting section 37, the FH deleting section 32 is applied to the image data without FH output from the gate section 36.
The FH generated by the control unit 40 is inserted at the beginning of the image data in place of the FH deleted in. The FH inserting section 37 is not necessary when the terminals are facing each other, but is effective when connecting at multiple points. This will be described later.

After returning to the original image data in the FH inserting section 35, the BCH / image multiplexing section 38 converts the image data into the original image data shown in FIG.
The image decoding unit 10 returns to the error correction frame shown in FIG.
(See FIG. 1).

FIG. 8 shows a configuration example in the case where a plurality of image control units 17 described above are used to simultaneously receive from four terminals. In this figure, the image data received from the terminal A is the same as the portion enclosed by the dotted line in FIG. 14 by the line interface unit 16 (a), the demultiplexing unit 15 (a), and the image control basic unit A50 (a). Therefore, the above-described processing is performed, and the same processing is performed on the received image data of the terminals B, C, and D.

That is, in this case, the overall control unit 52 for controlling the four image control basic units 50 (a) to 50 (d) is added. By this overall control unit 52, the image control basic unit A is added. Image data without FH from 4 to D is sent to each gate unit to perform time division multiplexing of the image data, and further, the FH insertion unit 37 is instructed to add FH to the head of the multiplexed image data. . The image data thus generated is regenerated into an error correction frame by the BCH / image multiplexing unit 38 and transferred to the image decoding unit.

As described above, the received data from each terminal can be simultaneously processed in parallel.

At the time of exchanging the transmission capability from each terminal, the image reception transmission capability of the terminal itself is transmitted to the other terminal as QCIF, and the image data is received from the other terminal in the QCIF format. ), The four QCIFs received from the four terminals are reconfigured so as to fit in the CIF.

Therefore, the system control unit 14 instructs the overall control unit 52 and each of the image control basic units 50 (a) to 50 (d) at which position on the CIF screen the image from each terminal should be displayed. . Each image control basic unit 50 (a) -50
In (d), the GN conversion unit 33 converts the GN value as instructed.

For example, to display in QCIF (A) of FIG. 12 (a), the GN value is not converted as shown in FIG. 12 (b). In the case of QCIF (B), the GN value is replaced with a value obtained by adding +1 to the current GN value. Similarly, QCIF
In case of (C), the value obtained by adding 6 to the current GN value, QCI
In the case of F (D), the current GN value is replaced with a value that is +7.

Here, conversion information of the GN value of the GN conversion unit 33 of each image control unit (or position information of QCIF).
Are stored in the storage unit 62 (see FIG. 17). that time,
Correspondence is made between the line number from each line i / f and the user information included in the call control information, or the information registered by the operator together with the line number in advance, and the correspondence relationship shown in FIG. 19 is registered. .

Each QCI whose GN value is converted in this way
The F image data is buffered by each buffer unit 34 and time-division multiplexed by each gate unit 36 to form a CIF image data multiplexed frame.

FIG. 16 is a timing chart showing the operation of the image controller. In the figure, signals a, b, c, and d are BC in each of the image control basic units 50 (a) to 50 (d).
This is an image data signal transferred from the H / image separation unit 30 to the GN conversion unit 33 via the FH deletion unit 32. For example, the A ₁ block of the signal a indicates a block of image data of one frame received from the A terminal. The numbers are serial numbers. In other words, the content of this image data block is 3 GOBs because the received image data is QCIF.
It is composed of.

For these image data blocks, the GN converter 32 is instructed by the system controller 14 as instructed.
The GN value is converted by and is buffered by the buffer unit 34.

Signals e, f, g and h represent image data stored in the buffer section 34. For example, as for the A ₁ block of the signal e, this A ₁ block is the buffer unit 3
4 shows the time from the completion of the input to 4 to the start of the transfer to the gate unit 36.

The signal i is sent from the overall control unit 52 to the FH insertion unit 3
7 to the image control basic units 50 (a) to 50
The timing for inserting FH in the image data frame obtained by multiplexing the image data of QCIF from (d) and reconstructing it in CIF is shown. The FH inserting unit 37 receives the F signal including the FH information instructed by the overall control unit 52 by the i signal.
Insert H.

The signals k, m, o, and q are the respective image control basic units 5
It is a timing signal of the gate unit 36 included in 0 (a) to 50 (d). When these signals are "H", the data from the buffer section 34 or GOB generation section 35 is FH.
Transfer to the insertion unit 37.

The signals j, l, n, p are the signals k,
Image data corresponding to m, o, and q is transferred from the gate unit 36 to the FH insertion unit 37. For example,
A ₁ of the signal j indicates an image data block from the buffer unit 34 of the image control basic unit A. The block marked with X means that the insufficient GOB block is generated from the GOB generation unit 35 to compensate for the insufficient amount when the data is not available in the buffer unit 34 or when the timing needs to be shifted. ing.

In FIG. 1, terminal A has QCIF (D) shown in FIG. 12A, terminal B has QCIF (C), and terminal C has Q.
CIF (B) and terminal D become QCIF (A) and are displayed on the screen.

The image data block generated by the GOB generation unit 35 is a GOB block having the same GN value as the GN value of the GOB replaced by the GN conversion unit 33. As for the contents of the GOB block, MB does not need to be added unless there is coefficient data, so GO
Although the B header alone may be used, “0000 0001 11” may be provided immediately after the GOB header or immediately after the MB block.
It is possible to insert a special code having no meaning, such as "1", for the next timing. This code is discarded in the image decoding unit.

Furthermore, when the number of terminals that are communicating simultaneously is three or two, there is always a shortage on the screen, so the image control basic unit corresponding to the terminal that is not communicating has the changeover switch 39. Always switch to the GOB generation unit 35 side, and
The GOB block from the OB generator 35 is always transferred.

Here, the contents of the GOB block at this time may be a meaningless special code as described above, but in this case, the screen is not updated because it is discarded by the decoding unit 10, and the screen is not updated. In order to clear, all the MBs are provided with coefficient data such that they become white or black data and prepared in the INTRA mode so that they are transferred periodically. Alternatively, data that is encoded in advance so that a message of some kind is displayed is prepared and transferred.

The signal r in FIG. 16 is an image data signal output from the FH inserting section 37. Here, the blackened portions indicate the FH of each frame. As shown in the signal r, three GOCs of QCIF of each terminal are multiplexed and 12 GOBs of CIF are transferred to the BCH / image multiplexing unit 38.

Since the image decoding unit 10 decodes the image data in the CIF format, the image data at four points as shown in FIG. 12 is accumulated and processed in the frame memory in the image decoding unit 10. Become.

Next, when the operator displays images of multiple points on the same screen, the system control unit 14 can superimpose the character on the image.

FIG. 18 shows an internal configuration diagram of the superimposing section 61. The image data transferred from the image decoding unit detects horizontal synchronization and vertical synchronization by the synchronization detection unit 63 and is sent to the switching timing generation unit 65. The switching timing generation unit 65 receives the position information for superimposing the character from the control unit 64, generates a timing signal of the position based on the synchronization signal, and switches the switch 67.

The switch 67 is a switch for switching between the image data (input A) from the image decoding section and the character image (input B) output from the character generator 66. In response to a switching instruction from the switching timing generation section 65, The switch 67 is switched from the input A to the input B, and the character image is embedded in the image data.

The character generator 66 includes a control unit 6
The character code to be output is received from 4, and the character image corresponding to the character code is output to the switch 67.

In this way, the superimposing unit 61 can display the designated character at the designated position of the image data according to the instruction from the system control unit 14.

As another method of superimposing,
It is also possible to once expand the image data on the image memory and overwrite the character image on the image memory.

Next, FIG. 19 registered at the time of GN conversion.
The information corresponding to each image number is read out according to the operator's instruction based on the correspondence table of No. 3, and the character string of the selected information is transferred to the superimposing unit 61.

At that time, position information corresponding to the image number is also transferred. As a result, the character information corresponding to each image number is superimposed on the screen.

Embodiment 4 FIG. 20 is a block diagram showing a fourth embodiment of the present invention. In this embodiment, an image scaling unit 71 is used instead of the superimposing unit 61 shown in FIG. I have inserted it. That is, the reference numeral 71 is an image scaling unit for scaling the image-decoded image data.

Hereinafter, this embodiment and the third embodiment (FIG. 1) will be described.
7)).

The signal r shown in FIG. 16 corresponds to the FH inserting section 37.
Is an image data signal output from. Here, the blackened portions indicate the FH of each frame. As shown by the signal r, three GOBs of QCIF of each terminal are multiplexed and 12 GOBs of CIF are transferred to the BCH / image multiplexing unit 38 (see FIG. 14). Since the image decoding unit 10 decodes the image data in the CIF format, images at four points as shown in FIG. 12 are accumulated and processed in the frame memory in the image decoding unit 10.

Here, for example, when it is desired to display the image of the point of the central person of the conversation with respect to the images of the four points of the frame memory, with emphasis on the images of the other points,
Main screen (or main screen) with the image with that emphasis
By displaying (or not displaying) other than that as a sub-screen (or a sub-screen), the convenience of the operator can be improved.

[0246] First, for the terminal to the point selected on the main screen, the transmission capacity of reception is set to Bch of 2ch (for example,
Image data about 112kbps, audio data 16kbps
s), the capability BAS is transmitted, and to the terminal selected on the sub-screen, the capability BAS is transmitted as Bch of 1ch (for example, image data about 48 kbps, audio data 16 kbps), and the point not selected. For the terminal of the above, there is no video in the capability BAS, or B of 1ch
Audio capacity of ch is 64 kbps and only audio is available BA
By transmitting S or by replacing the image data received by the image control unit 17 with invalid image data, the processing capacity of the decoding unit is allocated to the processing of the image data of the main screen as much as possible.

The image data at four points thus accumulated by the decoding process is transferred to the image scaling unit 71 (see FIG. 20).

The image scaling section 71 has a function of scaling each of the images at the four points as described above. The block diagram is shown in FIG.

In FIG. 21, 72 is the image decoding unit 10.
A scaling processing unit that performs scaling processing on the image data output from the system control unit 14 according to an instruction from the system control unit 73, a processing memory of the scaling processing unit 72, and a scaling processing image data 74. It is an image memory.

An example of scaling shown in FIG. 22 is image data QC.
IF (A) as the main screen, image data QCIF
(B) and QCIF (D) are displayed as sub-screens.

This will be described with reference to FIG. When the image data QCIF (A) is input to the scaling processing unit 72, the system control unit 14 issues an instruction for processing the main screen. Therefore, in the vertical direction, an image of several lines is stored in the processing memory 73. The data is stored and interpolation of line insertion is performed while performing filtering processing such as weighted addition with several lines before and after. In the horizontal direction, while performing filter processing such as weighted addition of several pixels before and after the line pixels,
Interpolation for pixel insertion is performed.

The image thus enlarged and generated is transferred to the image memory 74.

Next, when the image data QCIF (B) is input, the system control unit 14 has instructed the processing of the sub-screen. Therefore, in the vertical direction, several lines of images are stored in the processing memory 73. Data is stored, and line thinning-out correction is performed while performing filter processing such as weighted addition with several lines before and after. Further, in the horizontal direction, pixel thinning-out correction is performed while performing filter processing such as weighted addition with several pixels before and after the line pixels.

The image thus reduced and generated is transferred to the image memory 74.

Since QCIF (C) has been instructed by the system control unit 14 not to be displayed, no image data is input and the image data is passed.

For QCIF (D), QCIF
Similar to (B), reduction processing is performed and transferred to the image memory 74. Also, the sub screen is overwritten on the main screen and becomes a window display.

The image data thus edited in the image memory 74 is transferred to the image output section 9 and displayed on the display section 8.

As described above, according to the above-mentioned embodiment, when images are mutually communicated simultaneously among multiple points,
In the conventional case, it can be realized by preparing only one image decoding unit, which had to be provided for the number of terminals that perform simultaneous communication.

Therefore, since the number of terminals to which the decoding unit is added can be reduced and the composition of a plurality of screens to be transmitted can be performed only by controlling the change of the GOB header, it is particularly suitable for screen composition. No editing processor is required.

[0260] Also, the image is 1/1 of full size at the transmitting terminal.
Since N (N is the maximum number of communicable terminals) is sent before being sent, there is no need to perform a reduction editing process of images when synthesizing screens.

Furthermore, in the case of synthesizing an image with a plurality of decoding units, the image is deteriorated because it is decoded, synthesized, and then encoded again. However, in the above embodiment, the screen synthesis is performed without decoding. Therefore, there is an advantage that the image is not deteriorated.

As described above, the particular effect that the system configuration of the terminal can be significantly reduced without degrading the function is obtained.

Further, according to the above-described embodiment, when images are mutually communicated between multiple points at the same time, the partner terminal does not have to be aware of the fact that its own terminal is connected at multiple points, and the partner terminal sends the information to the sending terminal. On the other hand, it is possible to control the image format and perform simultaneous communication by a normal procedure.

Therefore, communication is possible as long as the terminal conforms to the standard, and the transmitting terminal side does not need a special procedure for multipoint connection.

As described above, the cost performance of the system is significantly improved by adding the function of intercommunication between multiple points to the system without increasing the cost by changing the system configuration or the function of other transmitting terminals. is there.

Furthermore, the transmission capacity is not wasted, which is effective in reducing the communication cost.

Further, even when the transfer rates of the frames of the image data from a plurality of terminals are different, it is possible to make up for the deficiency, so that simultaneous reception is possible without malfunction in the decoding section, and it does not depend on the sending terminal. Communication is possible without restrictions when connecting between multiple points. Therefore, there is an effect that the multipoint communication can be constructed at a low cost for the terminal and the system with the normal configuration.

Further, according to the above-mentioned embodiment, even when the number of terminals communicating at the same time does not reach the maximum number N of terminals, the shortage can be compensated for, so that the simultaneous reception can be performed without malfunction in the decoding section. It becomes possible, and communication is possible without restriction conditions when connecting between multiple points without depending on the transmitting terminal. Therefore, there is an effect that the multipoint communication can be constructed at a low cost while keeping the terminal and system configuration as usual.

Further, for example, it is possible to prepare an image for clearing the screen and clear the insufficient screen, so that the previously input image is not displayed as it is without being updated. In addition, it is possible to notify the operator that the message has not been received by preparing an image of a standard message in advance and displaying it on the insufficient screen, which is effective in improving the convenience of the man-machine interface. is there.

[0270]

In addition, according to the above-described embodiment, not only the location of the terminal can be confirmed but also auxiliary information of the communication partner can be displayed, so that the communication partner can be more deeply understood. There is an effect that it can measure the support of.

[0272]

Further, according to the above-mentioned embodiment, by improving the image quality of only the important images and displaying the other images with normal or lower than normal image quality, it is possible to optimize the reception of the image data at multiple points without waste. There is an effect that it can be achieved.

As described above, according to the present invention,
When images are mutually communicated simultaneously between multiple points, in the conventional case, one screen of a prescribed size has to be provided as the number of terminals for simultaneous communication, which is an image decoding unit.
One decoding unit for decoding encoded data of the format
Just have it, and also have a single-screen format of the specified size.
One decoding step of decoding the encoded data of
Reproduce images from multiple terminals at the same time and re-create on the same screen.
It is possible to present. Further, since the image is reduced in size to 1 / N (N is the maximum number of terminals that can be communicated) of the full size at the transmitting terminal and sent, it is not necessary to reduce and edit the image when combining the screens . What slave, since no large-scale and complicated the configuration of the apparatus, the effect is obtained that attained a significant reduction in cost.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a diagram showing a D channel communication procedure.

FIG. 3 is a diagram showing in-channel control of B channel.

FIG. 4 CCITT H. FIG. 221 is a frame configuration diagram of 221.

FIG. 5 is a diagram showing a bit configuration of FAS.

FIG. 6 is a diagram showing a bit configuration of BAS.

FIG. 7 is a detailed configuration diagram of an image control unit.

FIG. 8 is a unit configuration diagram of an image control unit.

FIG. 9 is a diagram showing a BCH error correction frame.

FIG. 10 is a diagram showing an image data multiplexed frame.

FIG. 11 is a diagram showing CIF and QCIF formats.

FIG. 12 is a diagram showing a GN (GOB Number) conversion method.

FIG. 13 is a timing chart showing the operation of the image control unit.

FIG. 14 is a block diagram showing an image control unit according to another embodiment of the present invention.

FIG. 15 is a unit diagram of the image controller shown in FIG.

16 is a timing chart showing the operation of each image control unit shown in FIG.

FIG. 17 is a block diagram showing another embodiment of the present invention.

FIG. 18 is an internal configuration diagram of a superimposing unit.

FIG. 19 is a diagram showing incidental information of each image.

FIG. 20 is a block diagram showing another embodiment of the present invention.

FIG. 21 is an internal configuration diagram of an image scaling unit.

[Fig. 22] Fig. 22 is a diagram illustrating the correspondence between the screen display and the image configuration during decoding.

FIG. 23 is an explanatory diagram of a conventional technique.

[Explanation of symbols]

1 handset 2 microphone 3 speakers 4 Voice interface section 5 Speech coding / decoding section 6 cameras 7 Document camera 8 Display 9 Image input / output section 10 image encoding / decoding unit 11 Data terminal 12 Data interface section 13 Operation part 14 System control unit 15 Demultiplexer 16 line interface 17 Image control unit 61 Superimpose department 71 Image scaling section

Claims (8)

(57) [Claims] 1. A terminal device capable of simultaneous communication with a maximum of N terminals (N: an integer of 2 or more) via a plurality of lines, each of which is connected to each of the lines and has a 1 / N reduced screen format.
A plurality of separating means for respectively extracting the encoded image data, and the 1 / N reduced screens of the encoded image data so that a maximum of N of the 1 / N reduced screens can be stored in one screen of a prescribed size .
Display position control means for changing the position specifying information indicating the allocation position on the screen of the encoded image data included in the header portion, is extracted by the plurality of separating means, the display position control
A plurality of the 1 / N reduced screen pictures processed by the means.
Format, multiplex the encoded image data,
Generates encoded data in a single-screen format with a specified size
Multiplexing means and 1 of the specified size processed by the multiplexing means
Decoding hand that decodes coded data in screen format
A terminal device comprising a step . 2. The position specifying information according to claim 1, wherein when the position specifying information is changed, new position specifying information is assigned to display a target 1 / N reduced image at a position specified by an operator's instruction. Terminal device characterized by. 3. The method of claim 1, in the case of a further 1 terminal and intercommunication images, by one stroke Menfu <br/> formats of the specified size, or the 1 / N reduced screens Four
Notifies be received by either of the mat to the other terminal, when the intercommunication of multiple terminals and images, before
Note that the terminal device is provided with means for notifying all the partner terminals that the reception is possible only in the 1 / N reduced screen format . 4. The method of claim 1, may further the 1 / N transfer frame rate of the reduced screen data is different for each terminal, that includes the transfer rate compensation <br/> means to generate image data missing Terminal device characterized by. 5. In claim 1, when performing simultaneous communication with K (K <N) transmitting terminals, a shortage of (N−K) transmitting image data with N transmitting terminals. terminal device being characterized in that comprises a missing data interpolation means to generate. 6. The terminal device according to claim 5, wherein a fixed message or a white screen or a black screen which is accumulated in advance is generated for the (N−K) units of insufficient image data. . 7. The registration device according to claim 1, further comprising: a registration unit that registers incidental information of the image data in association with the image data; and a registration unit that displays the decoded N-point image data. A terminal device comprising: additional display means for displaying the information on or near the image based on the registered information. 8. A method of controlling a terminal device capable of simultaneously communicating with a maximum of N terminals (N: an integer of 2 or more) via a plurality of lines, wherein 1 / N is connected to each of the lines. Reduced screen format
A plurality of separating steps for extracting the respective encoded image data, and to store up to N 1 / N reduced screens in one screen of a prescribed size .
Screen of the encoded image data included in the header section
A display position control step of changing the position designation information indicating the upper allocation position, and the display position control step extracted in the plurality of separation steps.
Of the plurality of 1 / N reduced screen formats processed in
The coded image data is multiplexed and the specified size
To generate encoded data in one screen format
Process and one screen screen of the specified size processed in the multiplexing process.
And a decoding step of decoding the encoded data of the format .