JPH07336461A - Picture communication equipment - Google Patents

Abstract

PURPOSE:To decode signals in the time-division processing and to prevent the reception of data beyond the reception capacity of a decoding part. CONSTITUTION:A reception means 1, a decoding means 2, an adjusting means 3, a transfer means 4, and a transmission means 5 are provided, and encoded picture signals from plural terminals are received by the reception means 1 and are decoded by the decoding means 2, and the adjusting means 3 performs such adjustment that the sum total of encoded picture signals received by the reception means 1 doesn't exceed the capacity of the picture decoding means 2. The reception capacity corresponding to the processing capacity is assigned to each of plural communication terminals and is transmitted to plural communication terminals by the transfer means 4, thereby preventing the trouble that data beyond the capacity of the decoding means 2 is received at the time of decoding the encoded picture signals by the time-division processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image communication device,
For example, it is suitable for use in an image communication device that transmits and receives video and audio via a communication line.

[0002]

2. Description of the Related Art In the case of a conventional analog telephone line, voice can be transmitted at a normal speed, but data can be transmitted only at a low speed.

However, in recent years, along with the progress of communication technology, semiconductor technology, optical technology, etc., digital lines have been established and it has become possible to transmit a large amount of data at high speed. In particular, in the case of digital transmission, there is no deterioration in quality due to transmission, and the characteristic is that the same level of quality is maintained even after transmission. In addition, there is a feature that the media can be integrated without requiring a transmission path according to the characteristics of the medium of the transmission data, and these features enable the transmission between the composite media terminals. Therefore, recently, a telephone terminal capable of simultaneously transmitting an image from a conventional voice-only telephone has appeared.

Under these circumstances, international standards such as ITU are being advanced so that mutual communication can be made between different compound terminals, and AV (such as videophones and videoconference systems using digital lines) is being developed. Audi
o Visual) service specifications, protocol specifications, multimedia multiplexing frame configuration specifications, etc. for AV services are defined in ITU Recommendation H.264. 320, H.M. 24
2, H.H. It has been announced as 221 etc.

The above H. 221, 64 Kbps to 1
Exchange of frame structure and terminal capability in AV services up to 920 Kbps, FAS (Fr
amAlignment Signal), BAS (B
It Allocation Signal) code allocation and the like are defined.

Further, the above H. 242 defines a protocol such as capability exchange between AV terminals using BAS and communication mode switching. In 320, the system aspect of the entire AV service is defined.

Further, in the above-mentioned recommendation, after setting the end-to-end physical connection and establishing the synchronization by FAS in the in-channel, the exchange sequence of the terminal capability using the BAS in the in-channel and the mode switching by the designation of the communication mode. A method for multimedia communication of images, voice, data, etc. between terminals is defined by a procedure such as a sequence. However, it is out of the specified range which communication mode is used within the range of the capability of changing or exchanging its own terminal capability in each terminal.

By the way, the information transfer rate of each medium in multimedia communication is determined by designating a voice coding method for voice information. In addition, the data information is determined by specifying whether or not it is used and the transfer rate when it is used, and subtracts the transfer rate of voice information and the transfer rate of data information from the set information transfer rate of the entire communication path. The rest is the transfer rate of image information.

In addition, multimedia is being developed also in personal computers and workstations, and communication of voice and video is becoming possible from conventional data communication. The communication medium in this case is mainly a local area network, and packetizes data for communication.

The local area network is usually a network that is closed on the premises. However, recently, the communication protocol is also specified in the physical layer (7), which is one of the layers that hierarchically divides the functions of the nodes in the network, and is used in a fairly wide area network through gateways and servers. Mutual communication is also active.

[0011]

However, according to the above-mentioned conventional TV telephone apparatus, only mutual communication with one terminal can be performed. Therefore, when mutual communication with multiple points is considered. Had to have multiple lines.

Further, since the communication capacity of the communication path is limited when performing image and voice communication, encoding / decoding for compressing / expanding data is performed. I needed to have a department.

By the way, in the case of audio compression, since the data amount is smaller than that of an image, it does not matter so much to have a plurality of encoding / decoding units. However, in the case of image compression, the algorithm is complicated and the amount of data is large, resulting in an enormous circuit scale. Further, since high-speed arithmetic processing is required and the storage capacity used is very large, it is difficult to have a plurality of storages.

Therefore, a method of decoding by time division processing can be considered. In that case, the data received from each terminal is received, buffered temporarily, and decoded according to the buffered accumulated amount. I have to go and change it. However, in this case, since image signals are received asynchronously and completely independently from a plurality of partner terminals, there is a problem that data exceeding the processing capability of the decoding unit is received. .

The present invention has been made in view of the above problems, and an object of the present invention is to perform decoding by time-division processing and not receive data that exceeds the receiving capability of the decoding unit.

[0016]

An image communication apparatus of the present invention is an image communication apparatus for performing image communication with a plurality of communication terminals, and a receiving means for receiving encoded image signals sent from the plurality of terminals, and the receiving means. Decoding means for decoding the encoded image signal received by the means, and adjusting means for adjusting the amount of the encoded image signal received by the receiving means so as not to exceed the processing capacity of the image decoding means. It is equipped with.

Another feature of the present invention is that
And a transmitting unit for transmitting to the plurality of communication terminals reception capabilities according to the processing capacities respectively assigned to the plurality of communication terminals by the adjusting unit.

Another feature of the present invention is that the adjusting means allocates different processing capabilities to each terminal.

[0019] Further, another feature of the present invention is that the transmitting means transmits the encoding control information for encoding by the plurality of terminals to the plurality of communication terminals.

Another feature of the present invention is that, in an image communication apparatus for performing image communication with a plurality of communication terminals, receiving means for receiving coded image signals sent from the plurality of terminals, and the receiving means. Decoding means for decoding the coded image signal received by the image decoding means, and the number of pixels for processing the coded image signals from all terminals received by the receiving means within a given time Adjusting means for adjusting so as not to exceed the number of pixels to be processed within an arbitrary time, and receiving capability according to the number of pixels to be processed within the arbitrary time, which is assigned to each terminal based on the adjustment result. Is transmitted to the plurality of communication terminals.

Another feature of the present invention is that, in an image communication apparatus for performing image communication with a plurality of communication terminals, a receiving means for receiving coded image signals sent from the plurality of terminals, and the receiving means. Decoding means for decoding the coded image signal received by the image decoding means, and the number of pixels for processing the coded image signals from all terminals received by the receiving means within a given time The first adjusting means for adjusting so as not to exceed the number of pixels to be processed within an arbitrary time of the means, and the total of the encoded image signals received by the receiving means exceeds the processing capacity of the image decoding means. Second adjusting means for adjusting so as not to exist, and a receiving capacity according to the number of pixels to be processed within the arbitrary time assigned to each terminal based on the adjustment result and the number of pixels per frame. It is and a transmission means for transmitting to each remote terminal.

Another feature of the present invention is that the adjusting means allocates different processing capabilities to each terminal.

Another feature of the present invention is that the adjusting means allocates a high processing capacity allocation ratio to a terminal having a high priority and a low allocation ratio to a terminal having a low priority. There is.

Another feature of the present invention is that the adjusting means assigns a large processing capacity to a terminal with a large screen size for displaying an image, and processes the terminal with a small screen size for displaying a screen. I try to allocate less capacity.

[0025]

Since the present invention has the above technical means, when the coded image signals received by the receiving means from a plurality of terminals are decoded by the decoding means, the total of the coded image signals is the above image decoding. Since the adjusting capability is adjusted so as not to exceed the processing capability of the encrypting means, the receiving capability corresponding to the processing capability is assigned to each of the plurality of communication terminals, and the communicating capability is transmitted to the plurality of communication terminals. ,
When decoding the coded image signal by time division processing, it is possible to prevent the inconvenience of receiving data that exceeds the processing capability of the decoding means.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the image communication apparatus of the present invention will be described below with reference to the drawings. FIG. 1 is a functional configuration diagram for explaining main functions of the image communication apparatus according to the present embodiment. Figure 1
In, a receiving means, a decoding means, an adjusting means, a transmitting means, a transmitting means, 9 an image output section, 8
Is a display unit.

The receiving means is provided for receiving coded image signals transmitted from a plurality of terminals in communication. The decoding means is provided to decode the coded image signal transmitted from the plurality of terminals and received by the receiving means.

The adjusting means is provided in order to adjust the total ability of decoding the coded image signals received from all terminals so as not to exceed the processing ability of the image decoding means. The transmitting means is for transmitting the receiving capability corresponding to the processing capability assigned to each terminal to each partner terminal based on the adjustment result.

The transmitting means is for transmitting the receiving capability information output from the transmitting means to the other terminal. The image output unit 9 is for converting a decoded image signal output from the decoding unit into a television signal, as will be described later in detail, and 8 is output from the image output unit 9. It is for displaying a television signal on the screen.

FIG. 2 is a block diagram showing the structure of a TV telephone apparatus according to an embodiment of the present invention. In particular, it is assumed that the number of lines is 4 and the simultaneous communication is assumed to be 4 terminals. It is a figure.

In the case of the image communication apparatus of the present invention, the number of lines is not limited, but for convenience sake, the case of four lines will be described. In FIG. 2, 1 is a handset which is one of the voice input / output means, 2 is a microphone which is one of the voice input / output means, and 3 is a speaker which is one of the voice output means.

Reference numeral 4 is a voice interface unit. The voice interface unit 4 includes a system control unit 12 described later.
It has a function of switching the handset 1, the microphone 2 and the speaker 3 according to the identification information. Also, the gain adjustment function for adjusting the volume level and the handset 1
Is an on / off detection function that detects whether it is an on-hook state or an off-hook state, an echo cancellation function that cancels echo when using the microphone 2 and speaker 3, and dial tone, ring tone, busy tone, ring tone, etc. It also has a tone generation function.

Reference numeral 5 is a voice encoding / decoding unit (× 4).
The voice encoding / decoding unit 5 is instructed by the system control unit 12 to generate 64 kbps PCM (A-law), 64 k
bpsPCM (μ-law), 7kHz audio (S
B-ADPCM), 32 kbps ADPCM, 16 kb
The transmitted voice signal is transmitted according to a voice encoding / decoding algorithm such as ps (for example, APC-AB) 8 kbps.
It has a function of performing D / D conversion and encoding, and a function of decoding a received voice signal and performing D / A conversion.

Reference numeral 6 is a standard-equipped image pickup camera for picking up an image of a person, 7 is one of image input means, and a document camera for inputting a picture or a drawing, and 8 is an image pickup camera 6 or a document camera 7. The display unit displays an input image signal, a received image signal sent from a communication partner, and an image signal input from the system control unit 12.

Reference numeral 9 denotes an image input / output unit, which has a function of switching the image input means in accordance with an instruction from the system control unit 12, a function of switching display of each image signal and a display dividing function, and a function of each image input / output unit. It has a signal converter for electrical / signal matching with the video signal.

An image coding / decoding unit 10 has a function of A / D converting and coding a transmission image signal and a function of decoding and D / A converting a reception image signal. The image encoding / decoding unit 10 performs band compression on raw data of a large-capacity image signal by various methods such as motion compensation, frame dropping, inter-frame compensation and inter-frame compensation, DCT conversion, and vector quantization conversion. The capacity is reduced to enable transmission via digital lines. Note that the current ISDN line has a basic interface of 64 kbps, but as the encoding method of the image signal that can be transmitted even at this transmission speed,
ITU Recommendation H. There is 261.

Reference numeral 11 denotes an operation unit for controlling the image communication device, which has a keyboard for inputting control information, a touch panel, and the like. A system control unit 12 includes a CPU, a ROM, a RAM, an auxiliary storage device, a character generator, an image signal generation circuit, and the like, monitors the state of each unit, controls the entire device, and creates an operation / display screen according to the state. It executes application programs. Reference numeral 13 denotes a demultiplexing unit (× 4), which is audio data from the audio encoding / decoding unit 5, image data from the video encoding / decoding unit 10, and BA from the system control unit 12.
It has a function of multiplexing S in transmission frame units and separating a reception frame into each medium of a constituent unit and notifying each unit. The ITU recommendation is H.264. There is 221.

Reference numeral 14 is a line interface (× 4) for controlling the line in accordance with the ISDN user network interface. Further, 15 is an image decoding processing capability control unit,
This is for performing control for allocating processing capacity to each terminal according to the decoding processing capacity of the image decoding unit. Note that four terminals are four line units 14 and four multiplexing units 1.
It becomes possible to communicate at the same time via 3. Speech decoding unit 5
, The audio coded data received from the four terminals are simultaneously decoded by the four audio decoding units, and the image decoding unit 10 decodes the four image coded data by one decoding unit by time division processing. .

Next, the internal structure of the image decoding unit will be described with reference to FIG. In FIG. 3, 16 to 19 are BCH decoding units for removing the BCH frame of the coded image data. Further, reference numerals 20 to 23 are reception buffers in which the BCH frame is removed and the image data multiplexed frame is temporarily stored and synchronized with the decoding unit.

Reference numeral 24 is a decoding selection circuit for selecting one piece of image coded data to be decoded from the image coded data of each of the reception buffers 20 to 23, and 25 is a frame header and a GOB header of the image data multiplexed frame. An image frame separation unit 26 is a variable length decoding (VLD) unit that performs variable length decoding of image data.

Reference numeral 27 is an inverse quantizer for inverse quantizing the coefficient data of each macroblock, and 28 is an inverse DC based on the coefficient data.
An inverse DCT unit for T, 29 is an FH decoding unit for decoding the contents of the frame header, 30 is a GOBH decoding unit for decoding the contents of the GOB header, and 31 is an MBH decoding unit for decoding the contents of the macroblock.

Reference numeral 32 is a quantization step size setting unit for setting the quantization step size of the header information from the GOB header and the macroblock header. 33 and 3
Reference numeral 4 denotes a switch for switching between writing the difference between the image data received in the INTER mode and the image data of the previous frame in the frame memory or writing the image data received in the INTRA mode as it is in the frame memory. Reference numerals 35 and 36 are switches for switching whether or not to perform filter processing. Reference numeral 37 denotes a filter, which filters the previous frame in the motion compensation mode.

Reference numeral 38 denotes a first frame control unit, which outputs a memory address to be read out from the frame memory storing the image data of the previous frame at the time of decoding. A second frame control unit 39 outputs a memory address for writing the decoded image data in the frame memory at the time of decoding. A third frame control unit 40 outputs a memory address to be read from the frame memory in order to transfer the image data in the frame memory to the image input / output unit.

41 and 42 are frame memories (F
M). Reference numeral 43 is a switch for selecting from the first to third frame control units 38, 39 and 40 and outputting a memory address to the frame memory 42.
Reference numeral 44 denotes the first to third frame control units 38, 39, 4
It is a switch that selects from 0 and outputs a memory address to the frame memory 42.

Reference numeral 45 is a switch for selecting which frame memory is written when writing to the frame memory. Reference numeral 46 denotes a switch, which refers to the previous frame in the INTER mode and selects which frame memory to read from at that time.

Reference numeral 47 is a decoding control unit for controlling the entire image decoding unit, 48 is a storage unit for storing a management table and the like, and 49 is a switch for selecting a frame memory of image data to be output to the image output unit. is there. With the above configuration,
It is possible to process a plurality of image data in a time-sharing manner and make it appear as if they are being processed simultaneously.

The internal structure of the image output unit 9 is shown in FIG. In FIG. 4, reference numeral 50 is an image memory control unit for designating a memory space for writing and reading to and from the image memory, and 51 is a storage unit for registering a memory space area for storing image data from each terminal.

Further, 52 is a color look-up table, 53 is a switch for switching between graphic image data and received image data, 54 is a video synchronization signal generator,
55 is an A / D converter.

Reference numeral 56 is a switch control section for controlling to switch to image data when the graphic data is a specific data. Reference numeral 57 is an image memory for separately storing image data from each terminal, and 58 is a pixel density conversion unit for performing pixel density conversion on the image data from the decoding unit.

The image output unit 9 thus configured can display the graphic data from the color lookup table 52 on the entire surface of the screen, and display a plurality of received image data on the screen as a window. Further, it is possible to display graphic data from the color lookup table 52 on the received image data like superimpose display. Further, when a plurality of received image data are overlapped with each other, one of the received images can be selected to display the overlapped portion.

Next, the operation of the above configuration diagram will be described, but before that, the encoding of image data will be briefly described.
The structure of the image to be encoded is shown in (a) and (b) of FIGS. 5 and 6. The above H. According to the H.261 standard, a plurality of different standards such as NTSC, PAL, and digital television standards exist for the video signals to be handled, so that a video signal format common to the world is adopted so that they can communicate with each other.

This format is called CIF format, and the number of samples is 352 pixels × 288 lines with luminance Y.
The color differences Cr and Cb are defined by 176 pixels × 144 lines. Also, regarding the sampling points (sampling points),
The color difference (Cr, Cb) is the luminance 4 points (Y1, Y2, Y
3, Y4) are defined as equidistant points.

Further, 1/4 of CIF is called QCIF format, and the number of samples is defined by luminance Y of 176 pixels × 144 lines and color differences Cr and Cb of 88 pixels × 72 lines.

Here, the format is a plurality of G
The GOB format is composed of 33 MB formats. Furthermore,
The MB format has a luminance block of 8 pixels × 8 lines, four blocks of luminance Y1, luminance Y2, luminance Y3, and luminance Y4, and a color difference block of 8 pixels × 8 lines of Cr,
It is composed of two blocks of Cb and has a hierarchical structure.

With this hierarchical structure, it is possible to perform encoding in MB format units. The GOB format has a luminance of 176 pixels x 48 lines and a color difference of C
r, Cb 88 pixels × 24 lines, which corresponds to 1/12 of the CIF format and 1/3 of the QCIF format. GOB format numbers are assigned from GOB1 to GOB12 in the CIF format.
GOB1, GOB3, and GOB5 are assigned to the F format.

The frame structure of the coded image data is a multiplexed frame structure as shown in FIG. However, for convenience of explanation, here, the frame header F
The description will be made with H added.

FIG. 7 shows a configuration of GOB blocks. In this way, the frame header FH is added to the head of the data of one frame of the screen, and one block obtained by dividing the screen into 12 is sequentially transmitted as GOB1 to GOB12.

The frame header FH includes PSC, TR and P
It is composed of TYPE. Frame header PSC
Is a frame start code of 20 bits "0000 00".
000000 0001 0000 ".

The frame header TR has a frame number of 5
Values of bits "1" to "30" are used. PT
The YPE is 6 bits of type information and includes split screen instruction information, document camera instruction information, screen freeze release, and information source format instruction information (CIF, QCIF). That is, the FH decoding unit notifies the control unit 28 of the result of decoding the above contents.

The GOB header includes GBSC, GN and GQU.
It is composed of ANT. The header GBSC is a GOB start code of 16 bits "0000 0000 0000".
0001 ".

The header GN is a GOB number of 4 bits.
Use from "1" to "12". Header GN is "0"
In this case, since it is used in the frame header PSC, PSC in the frame header FH and GBSC + in GOB +
Both GN can be regarded as a continuous value of 20 bits.

The header GQUANT is 5 bits of quantization characteristic information and contains information on the quantization step size. The MB header is a header of a pixel block called a macro block (MB).

The macroblock MB is, as described above,
One GOB header is composed of 33 macro blocks MB, and 1 MB includes 4 luminance signals (Y) of 8 pixels × 8 lines and color difference signals (Cb, C of 8 pixels × 8 lines).
r) It is composed of two pieces.

Here, as the number of each block, the luminance Y
Are assigned numbers 1 to 4, Cb is 5 and Cr is 6. This MB header is MBA and MTYP
It is composed of E, MQUANT, MVD and CBP.

MBA is a macroblock address indicating the position of the macroblock MB, and the first macroblock MB.
Only the absolute value, and the variable length code after that is the absolute value. M
TYPE is the type information of the macro block MB, IN
The macro block M such as TRA (intra-frame coding), INTER (inter-frame differential coding), MC (motion compensation inter-frame differential coding), FIL (filter), etc.
The processing type in which the processing is performed on the data of B is inserted. MQUANT is the same as GQUANT.

MVD is motion vector information. CB
P is a significant block pattern, and the macroblock M
The four brightness Y of B and the number of the effective pixel block of Cr and Cb are included as information. After the MB header, the compression-encoded image data is as described above.
Pixel blocks that have become significant blocks out of four luminance Y Cr and Cb are arranged in numerical order.

The data output to the line is an error correction frame having a format as shown in FIG. As shown in FIG. 8, in one frame, the error correction frame bit is 1 bit, the fill identifier is 1 bit, and the image data is 49 bits.
2 bits, error correction parity is composed of 18 bits and 512 bits. Further, this frame is composed of 8 frames to form one multi-frame.

The image data compression method and image format are ITU-T. H.261 has already been made into a recommendation, and if it complies with the recommendation, mutual communication is possible with a TV phone that complies with other recommendations.

Next, the encoding method will be briefly described. First, the two-dimensional DCT conversion is performed on the data in the frame as a block of 8 pixels × 8 pixels by utilizing the fact that the correlation between the pixels is strong in the image in the natural world and the frequency components are concentrated in the low frequency and the high frequency is small. There is intraframe coding.

Then, in the image blocks at the same position in the previous frame and the current frame, when the correlation between the two is strong, the difference between the frames is taken and the block of 8 pixels × 8 pixels is subjected to the two-dimensional DCT conversion with respect to the difference value. There is encoding.

Then, when a similar image block relatively moves from the previous frame to the current frame, the image data itself is detected by only detecting this and sending the information on the moving amount and moving direction of the image block. There is motion compensation that reduces the amount of generated data by not sending it.

Next, there is a zero run length coding which utilizes that the coefficient value for each frequency after DCT conversion has a value in a low frequency region, but a value does not easily occur in a high frequency region and continues to have a zero value.

Next, there is quantization in which the data generation amount is adjusted by changing the data quantization step width in accordance with the data generation amount.

Next, by assigning a short code value to a data pattern having a high frequency of occurrence and a long code value to a data pattern having a low frequency of occurrence, the total amount of data that is less than the amount of generated data is There is variable length coding that converts to quantities.

In addition, a plurality of compression techniques such as frame dropping that skips frames and drops the image data itself can be hybridized to enable moving image communication even in low rate communication.

Next, the image decoding processing capability control unit will be described with reference to FIGS. 9 and 10. FIG. 9 is an operation flow of allocation control of the decoding processing capability, and FIG.
Is an operation flow of allocation control of a memory space that can be processed by the decoding unit. The image processing capability is, for example, the number of pixels that can be processed per unit time.

For example, when the CIF is 60 fps,
The number of pixels to be processed per second is 352 pixels × 288 lines × 60 frames = 6,082,560 pixels.
In the case of QCIF, it is 1/4 of that.

Here, in order to simplify the description in terms of the number of pixels processed per second when describing the processing capability, the number of pixels to be processed per second will be hereafter referred to as an image format (for example, CI).
F / QCIF) and frame rate (eg frame /
Seconds) will be explained. However, the image communication device of the present invention is not limited to this combination.

When the operation of the image decoding processing capacity control section is started, first, after initialization processing is performed in step S1, the decoding processing capacity X of the image decoding section is set in step S2. Next, in step S3, the number of terminals for simultaneous communication is set, and in the next step S3, a number n is assigned to the terminals in order of priority.

Then, in step S5, the number n and the brightness Y0 are set to "0", and the number n is incremented by 1 in step S6. Next, in step S7, the decoding processing capacity of the image decoding unit is divided by the number of terminals, the average decoding processing capacity to be allocated to each terminal is obtained, and the processing coefficient Yn to be allocated to the terminal is multiplied by the priority coefficient αn. To calculate. Then, in the next step S8, the calculated processing capacity Y
Set n as the processing capability of the nth terminal.

Next, in step S9, the processing capacity is accumulated. Then, in the next step S10, it is determined whether or not the accumulated processing capacity exceeds the processing capacity of the image decoding unit. If not, step S1
The process proceeds to step 1 to check whether or not the processing capacity has been assigned to all terminals. Then, if not finished, the process returns to step S6.

Here, the number n is incremented by 1, and the processing capacity of the next terminal is allocated. At that time, the number of terminals for simultaneous image decoding simultaneous communication is calculated from the processing capacity of the installation unit and is already accumulated. The processing capacity obtained by subtracting the processing capacity is divided by the number of terminals whose allocation is not completed, and the average is calculated. And
In step S7, the decoding processing capability of the terminal is calculated by multiplying it by the priority coefficient of the terminal.

Next, in step S8, the calculated processing capacity is set as the processing capacity of the terminal, and the same processing is performed thereafter. In this way, when the allocation of the processing capability to all the terminals is completed, the process ends.

Further, as a result of the judgment in step S10,
If the accumulated processing capacity exceeds the processing capacity of the image decoding unit, the process proceeds to step S12, error processing is performed,
The process ends. As described above, the image decoding processing control unit transmits the terminal receiving capacity according to the assigned processing capacity to each terminal based on the calculated processing capacity of each terminal.

Next, the flowchart of FIG. 10 will be described. Here, the image memory space of the image decoding unit needs a reference memory in which a previous frame is stored and a memory in which the current memory is stored in order to obtain a difference from the current frame.

However, the allowable memory space allocated to both of them is determined by the memory capacity of the decoding section. Therefore, it is important to control how to allocate the memory capacity to each terminal within the allowable memory capacity range.

As shown in FIG. 10, first, step S
After performing the initialization process at 21, the process proceeds to step S2 to set the number V of pixels that can be processed by the image decoding unit. Next, in step S23, after setting the number of terminals for simultaneous communication, in step S24, a number n is assigned to the terminals in order of priority.

After that, the number n and the number of pixels W0 are set to "0" in step S25, and the number n is incremented by 1 in step S26. Next, proceeding to step S27, the number of pixels that can be processed by the image decoding unit is divided by the number of terminals to obtain the average number of pixels that can be processed and is assigned to each terminal. Number Wn
To calculate.

Next, in step S28, the calculated pixel number Wn is assigned to the memory area which is allowed as the pixel number of the nth terminal. Next, step S2
In 9, the number of pixels is accumulated, and in step S30, it is checked whether the accumulated number of pixels exceeds the number of pixels that can be processed by the image decoding unit.

If the number of pixels that can be processed does not exceed the number of pixels that can be processed as a result of the determination in step S30, it is checked in step S31 whether the number of pixels that can be processed has been assigned to all terminals. If not, the process returns to step S6.

Here, after the number n is incremented by 1, the number of pixels that can be processed by the next terminal is assigned in the next step S27. At that time, the number of pixels obtained by subtracting the cumulative total number of pixels that have already been allocated from the number of pixels that can be processed by the image decoding unit is divided by the number of unallocated terminals, and the average is calculated. The decoding processing capability of the terminal is calculated by multiplying by the priority coefficient of the terminal. In the next step S28, the calculated number of processable pixels is set as the number of pixels of the terminal. The same processing is performed thereafter. Thus
When the allocation of the number of pixels that can be processed to all terminals is completed, the process ends.

The image decoding processing control unit, based on the number of pixels of each terminal calculated as described above,
The terminal receiving capability corresponding to the assigned number of pixels is transmitted. In the above description, the control information to be transmitted to the partner terminal is transmitted after being replaced with the terminal reception capability according to the processing capacity and the number of pixels that can be processed, but the information is transmitted as it is to the partner terminal. You may make it perform the encoding control based on information.

Further, in addition to the above information, any control information related to encoding (eg, data transfer rate,
In the same manner as the operation flow of the allocation control shown in FIG. 9 and FIG. 10, each of the terminals is allocated and the allocation information is transmitted to the partner terminal. It is possible to control based on the information, or to transmit the terminal receiving capability according to the information.

Here, the H. 221 and H.221. 26
Although it is obvious that the present invention is not limited to the recommendation such as 1, etc., for the sake of convenience, the description will be given along with the above recommendation. For example, the decoding processing capability of the image decoding unit is CIF format 3
When it is possible to process at 0 fps, and when communicating with four terminals, the processing capacity assigned to each terminal is CIF format if the priority order is not set for each terminal.
It has a processing capacity of 5 fps or a processing capacity of 30 fps in QCIF.

When one of the four terminals has a priority set and the priority coefficient is 2, the capacity twice as much as the average processing capacity is allocated, so that two normal terminals are allocated. As a result, the remaining 2 terminals are averaged over 3 terminals, and as a result, the ratio of the processing capacity of each terminal is
It becomes 3: 1: 1: 1. In this case, one terminal is CIF
Format is 15 fps, 3 terminals is CIF 5 fps
Or, if one terminal is in CIF format, 15 fps,
The three terminals have QCIF of 20 fps.

Further, by transmitting the calculated processing capabilities to the partner terminals, each partner terminal encodes and transmits the image signal according to the capabilities, so that the decoding provided in the image decoding unit Since it is possible to use the maximum capacity and allocate the optimum processing capacity to each terminal, it is possible to realize highly efficient and excellent multipoint image communication.

Next, the operation of the entire decoding section will be described. The coded image data received from each terminal by the above method is BCH-decoded separately and accumulated in each reception buffer. Although a plurality of BCH decoding units are used here, it is possible to combine them into one by time division processing.

The decoding control unit 47 searches the storage status of the receiving buffer before decoding and detects the frame header. Alternatively, it seems that one frame is accumulated by a method such as calculating the storage capacity of the reception buffer (in H.261, the maximum is 64 kbit per frame in the QCIF format, and the maximum 256 kbit per frame in CIF). If so, the image coded data is read out by selecting from one of the reception buffers accumulated for one frame or more.

Then, the data is subjected to decoding such as VLD, inverse quantization and inverse DCT to obtain image data. Then, in the INTER mode, the memory area of the frame memory that stores all the image data of the image data of the selected reception buffer is searched from the frame memory management table registered in the storage unit 48. Then, the memory area of the frame memory is selected by the switch 46, all the image data is read out by the address control of the frame memory control 1, and the filter processing is performed by the filter 37 if a filter is necessary.

Next, the adder is selected by the switches 33 and 34, the decoded image data and the entire image data are added, and the empty memory area of the frame memory is searched from the frame memory management table. . Then, the switch 45 selects the memory area of the frame memory, and the added image data is written by the address control of the second frame memory control unit 39.

In the INTRA mode, the decoded image data is not read from the memory area of the frame memory selected by the switch 46, and the decoded image data is switched by the switch 33.
The switch 34 selects the adder OFF, and the image data is written in the memory area of the frame memory previously selected by the switch 45.

When transferring the image data to the image output unit, the memory area of the frame memory of the image data to be read is selected by the switch 49, and the image data is transferred by the address control from the third frame control unit 40. read out. When this image data is transferred, the image decoding unit 10 also transfers to the image output unit 9 information for identifying which reception buffer the image data is from.

Next, examples of frame memory allocation of received images are shown in FIGS. 11 and 12. FIG. 11A shows C
This is the use of the frame memory when received in the 1: 1 communication in the IF format.

Further, FIG. 11B shows the use of the frame memory in the case of 1: 1 communication in the QCIF format. That is, when communicating by QCIF, there are many unused areas of the memory, which is quite useless.

FIG. 11C shows a case where four terminals in the present embodiment are assigned the same decoding processing capability and simultaneously communicate with each other, and the terminals are received in the QCIF format. Further, the memory area of the frame memory is divided into four for A, B, C and D, and QCIF (A) and QCIF
(B), QCIF (C), and QCIF (D).

FIG. 12D shows the case of allocation by CIF at the time of FIG. 11C. In this case, more memory is required than when receiving by QCIF. FIG. 12E shows a case where one of the four terminals is prioritized and the processing capacity ratio is 3: 1: 1: 1 in the present embodiment, where one terminal is CIF and three terminals are QCIF. is there.

Since the reference frame required for the decoding process is also required, the frame memory needs to be twice as large as the above frame memory. Thus, the decoding of the image data from each terminal is performed using the above-mentioned allocated memory area. Further, the transfer of image data to the image input / output unit is also transferred for each memory area assigned to each terminal.

FIG. 13 shows the frame control timing. In FIG. 13, the image data indicates the image data currently decoded. The frame cycle is set so that the decoding processing time for one frame of image data is shorter than the frame processing cycle.

The block size indicates the processing time. The numbers in the block are the frame identification number, which is represented by the reception buffer and the frame number. For example,
A1 is the first frame number of the reception buffer A.

The write frame memory is a frame memory number in which the above image data is written. The read frame memory is a frame memory number from which all image data to be added with the image data is read.

Therefore, when the image data is A2, the write frame memory is FM2 (A) and the read frame memory is FM1 (A), the image coded data read from the reception buffer A is decoded. Image data
In the INTER mode, FM2 is obtained by adding the previous image data A1 in the reception buffer A stored in FM1 (A).
Write to (A) and use FM as it is in INTRA mode
(A) is written.

If one frame of image coded data to be decoded is not stored in the buffer, the frame processing cycle is not performed. If an error occurs during the decoding of the encoded image data, the decoding process is interrupted and the written frame memory is not updated immediately but is continuously registered as an empty frame memory. Further, the frame memory that has been read out is not updated but is continuously held and carried over to the next decoding process. Therefore, it is necessary to manage the usage status of the frame memory for each frame processing cycle.

FIG. 14 shows a frame memory management table used for this purpose. This management table is stored in the storage unit 47, and refers to the empty frame memory at the beginning of writing image data for each frame period. Then, when the writing is completed, the image data of the frame memory read for addition becomes old,
Since this old image data is unnecessary, this frame memory is registered as empty, and the written image data in the frame memory is replaced with the old image data. Therefore, this frame memory is newly registered.

In the case of FIG. 14, the registration status at the position of the arrow in FIG. 13 is shown. Image data B2 to FM
The image data B1 is read from 1 (B), added, and FM
Since writing to (B) has just finished, FM1
(B) is empty and FM2 (B) is B2.

When reading the decoded image data from the frame memory to transfer it to the image output unit, the image data decoded one cycle before the frame processing cycle is read at the next frame processing cycle. Easy control is sufficient.

However, the image data may be read from any frame memory other than the frame memory in the writing process for decoding. In addition to the read image data, the frame identification number is also transferred to the image output unit in FIG.

The transferred image data is subjected to pixel density conversion and stored in the image memory. At that time, the image memory control unit 50 refers to the memory space of the image memory allocated to each of the other terminals stored in the storage unit 51, and refers to the image memory space corresponding to the frame identification number. Write the data. When displaying the written image data, the memory space is output to the display unit through the read switch 54 and the A / D converter.

In this way, by decoding a plurality of image data from a plurality of terminals by time division processing, it is possible to use only one without having a plurality of decoding units. Also, since the number of frame memories of the decoding unit need only be increased by one to the number of terminals that need to communicate at the same time, economical decoding is possible even when communicating simultaneously with multiple terminals. You will be able to build a chemical department.

FIG. 15 shows the frame memories FM1 and FM2.
It shows an example of memory allocation in the case of fully utilizing. In this case, the frame memories FM1 and FM2 are
It can be divided into eight in the QCIF format and can communicate with up to seven terminals simultaneously.

In this case, the memory area of each QCIF is not fixed for each terminal, and the decoded image data of each terminal, in the INTRA mode, has the smallest vacant number from the frame memory management table of FIG. A memory area is detected and written in the corresponding memory area.

In the INTER mode, all the image data to be added are detected from the frame memory management table in FIG. 16, the image data in the corresponding memory area is read, and the image data is added to the decoded image data and added. Will be written to the memory area.

As described above, the decoding processing capacity of the image decoding unit is used to the maximum, the large-capacity image memory is also used to the maximum, and the processing capacity is optimized so that the transfer capacity of each terminal is optimized. Since allocation is possible, it is possible to perform optimal multipoint image communication without any waste.

[0123]

As described above, the present invention decodes a plurality of image data transmitted from a plurality of terminals by time division processing,
Moreover, since the data that exceeds the receiving capacity of the decoding unit is not received, it is not necessary to have multiple image decoding units, and the number of frame memories in the image decoding unit is minimized. Therefore, the decryption unit can be economically constructed. As a result, each terminal can be provided with a multipoint communication function at low cost, the service can be easily received, and the convenience of the user can be dramatically improved.

Further, since the processing capability of the image decoding unit can be utilized to the maximum, and the processing capability can be optimally distributed to each terminal, it is possible to communicate with a plurality of terminals simultaneously.
It is possible to optimize multipoint image communication and improve efficiency.

[Brief description of drawings]

FIG. 1 is a functional configuration diagram of an image communication apparatus showing an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a TV phone device showing an embodiment of the present invention.

FIG. 3 is a block diagram showing an internal configuration of an image decoding unit.

FIG. 4 is a block diagram showing an internal configuration of an image output unit.

FIG. 5 is a diagram showing an image format.

FIG. 6 is a diagram showing an image format and sampling points.

FIG. 7 is a diagram showing an image data multiplex frame.

FIG. 8 is a diagram showing an error correction frame.

FIG. 9 is a flowchart showing an operation of an image decoding processing capacity control unit.

FIG. 10 is a flowchart showing an operation of an image decoding processing capacity control unit.

FIG. 11 is a diagram showing allocation of a received image frame memory.

FIG. 12 is a diagram showing allocation of a reception image frame memory.

FIG. 13 is a diagram showing frame control timing.

FIG. 14 is a diagram showing an example of a frame memory management table.

FIG. 15 is a diagram showing an example of allocation of a frame memory of a received image.

FIG. 16 is a diagram showing an example of a frame memory management table.

[Description of Codes] Receiving means Decoding means Adjusting means Transmitting means Sending means 1 Handset 2 Microphone 3 Speaker 4 Audio interface section 5 Audio coding section 6 Imaging camera 7 Document camera 8 Display section 9 Image input / output section 10 Image coding section 11 Operation Unit 12 System Control Unit 13 Demultiplexing Unit 14 Line Interface Unit 15 Image Decoding Processing Capacity Control Unit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI technical display location H04N 7/18 A

Claims (9)

[Claims] 1. An image communication apparatus for performing image communication with a plurality of communication terminals, receiving means for receiving coded image signals sent from the plurality of terminals, and decoding the coded image signals received by the receiving means. An image communication apparatus, comprising: a decoding unit for adjusting the amount of the coded image signal received by the receiving unit so as not to exceed the processing capacity of the image decoding unit. 2. The transmission means for transmitting to the plurality of communication terminals the reception capacity according to the processing capacity assigned to the plurality of communication terminals by the adjusting means, respectively. Image communication device. 3. The image communication apparatus according to claim 1, wherein the adjusting means allocates different processing capabilities to each terminal. 4. The image communication apparatus according to claim 2, wherein the transmission means transmits the encoding control information for encoding by the plurality of terminals to the plurality of communication terminals. 5. An image communication apparatus for performing image communication with a plurality of communication terminals, receiving means for receiving coded image signals sent from the plurality of terminals, and decoding the coded image signals received by the receiving means. The number of pixels for processing the coded image signals from all terminals received by the receiving means by the receiving means is within a certain arbitrary time, the number of pixels for processing within the certain time by the image decoding means. Adjusting means for adjusting so as not to exceed the number, and transmitting means for transmitting, to the plurality of communication terminals, the receiving ability according to the number of pixels to be processed within the arbitrary time allocated to each terminal based on the adjustment result. An image communication device comprising: 6. An image communication apparatus for performing image communication with a plurality of communication terminals, receiving means for receiving an encoded image signal sent from the plurality of terminals, and decoding the encoded image signal received by the receiving means. The number of pixels for processing the coded image signals from all terminals received by the receiving means by the receiving means is within a certain arbitrary time, the number of pixels for processing within the certain time by the image decoding means. First adjusting means for adjusting so that the number does not exceed the number, and second adjusting means for adjusting so that the total of the encoded image signals received by the receiving means does not exceed the processing capacity of the image decoding means. A transmission means for transmitting, to each partner terminal, the number of pixels to be processed within the arbitrary time assigned to each terminal based on the adjustment result and the reception capability according to the number of pixels per frame. Image communication apparatus, characterized in that. 7. The image communication apparatus according to claim 6, wherein the adjusting means allocates different processing capabilities to each terminal. 8. The image communication apparatus according to claim 6, wherein the adjusting unit allocates a processing capacity allocation ratio to a terminal having a high priority to a high ratio and to a terminal having a low priority to a low ratio. . 9. The adjusting means allocates a large processing capacity to a terminal having a large screen size for displaying an image and a small processing capacity to a terminal having a small screen size for displaying a screen. 6. The image communication device according to item 6.