JPH0723377A - Picture transmitter - Google Patents

Abstract

PURPOSE:To prevent a picture from being deteriorated at the time of changing a picture transmission rate. CONSTITUTION:A picture transfer rate calculating circuit 50 calculates a transfer rate allocated to picture transmission. In the case of changing the picture transfer rate, an overall INTRA control circuit 52 instructs an image encoding circuit 22 to encode all blocks in the fixed number of frames by an intra-frame encoding system. Consequently the fixed number of frames after changing the picture transfer rate are encoded by intra-frame encoding and transmitted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image transmitting device,
More specifically, the present invention relates to an image transmission device in an image transmission system such as a videophone and a video conference.

[0002]

2. Description of the Related Art Digital public communication network (so-called IS
With the spread of DN), simultaneous communication of image, voice and data has become possible, and a videophone and a video conference system have become practical. Service specifications, protocol specifications, and multimedia multiplexing frame structure specifications for audio / video services using digital lines are recommended by the International Telegraph and Telephone Consultative Committee (CCITT) Recommendation H.264. 320, H.M. 242
And H .; It has been announced as 221 mag.

H. 320 defines general system aspects of the audiovisual service. H. 221 is 64
FAS (Frame) used for exchange of frame structure and terminal capability, designation of communication mode, etc. in audio / video transmission on Kbps to 1,920 Kbps channel.
Alignment Signal) and BAS (Bi
t-rate Allocation Signal)
Specifies the coding assignment of. H. 242 defines a protocol for capability exchange between terminals and communication mode switching by BAS.

In the above-mentioned recommendation, after the end-to-end physical connection is set up and the synchronization is established by FAS in the in-channel, the exchange sequence of the terminal capability by the BAS in the in-channel and the mode switching sequence by the designation of the communication mode are performed. Image between terminals by the procedure such as,
A method for performing complex information communication such as voice and data is specified.

Each terminal is capable of changing or selecting its own terminal capacity within a predetermined range according to the situation, and which communication mode is used within the range of the exchanged capacity is within a prescribed range. Outside.

Regarding the transmission speed of each information in the case of simultaneously transmitting image, voice and data, the voice is determined by the encoding system of the voice code, the data is set to a specified value, and the image is transmitted through the communication line. The remaining transmission capacity in speed is allocated.

As a compression method of image information, there has been proposed a coding method in which intra-frame coding and motion-compensated inter-frame coding are mixed so as to increase the compression rate and to suppress the propagation of transmission errors. .

The digital public network has already been put to practical use as ISDN. The interface provided to the user is the basic interface (2B (64 Kbp
s) + D (16 Kbps)) and primary group interfaces (H0 (384 Kbps), H1 (1.5 Mbps) and 23B + D). The basic interface replaces the current analog public network. The D channel, which is an out-band channel, is used for call connection with the partner terminal, and the B channel is used for transmission of information data such as voice, image and data. Since there are two B channels,
By synchronizing these two channels, the maximum is 12
8 Kbps data transfer (bulk transfer) becomes possible.
Bulk transfer is very effective because a large amount of data transmission capacity is required for a TV phone or the like.

[0009]

In a videophone and a video conference, images, voice and data, and mutual control information are multiplexed and transmitted as necessary. Therefore, when the information to be transmitted or the encoding method thereof is changed, the multiplexing configuration is changed each time.

Since the change of the transmission rate is transmitted to the receiving terminal by the in-channel control command of B channel, it is very difficult to synchronize with the transmission information. If the time of multiplexing change of the transmitting terminal and the time of multiplexing changing of the receiving terminal are different, a decoding error occurs in the receiving terminal.

In particular, in the case of image data, as described above, one screen is divided into certain arbitrary blocks, and inter-frame coding (INTER) for coding and transmitting the difference value of the previous frame for each block. Since it is used together with intra-frame coding (INTRA) for coding within one frame,
On the receiving side, even if the frame synchronization is lost after the frame synchronization is once disturbed, the original image cannot be restored until a new intra-coded screen is received for all the blocks. That is, if the image transmission rate is changed, there is a drawback that it takes a considerable time for the receiving side to reproduce and display a normal image.

It is an object of the present invention to present an image transmitting apparatus that does not cause such inconvenience.

[0013]

An image transmitting apparatus according to the present invention is an image transmitting apparatus for compressing and transmitting image information by an encoding method having intra-picture encoding and inter-picture encoding, and an image to be transmitted. Encoding means for compressing and encoding information, transmission rate detecting means for detecting a change in image transmission rate, and in accordance with the detection result of the transmission rate detecting means, the encoding means encodes the entire screen in the screen. And a control means for encoding a predetermined number of screens after the transmission rate is changed by the encoding method.

[0014]

By the above means, when the image transmission rate is changed, a predetermined number of screens are compressed and transmitted by the coding method for intra-coding the entire screen. As a result, the receiving terminal can decode and display the received image immediately after changing the image transmission rate without referring to the previous screen. As a result, the image disturbance due to the change of the image transmission rate hardly occurs, and even if it occurs, the image is recovered in an extremely short time.

[0015]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a schematic block diagram of a terminal device in an embodiment of the present invention.

In FIG. 1, 10 is a camera for photographing conference participants, 14 is a document camera for photographing conference materials such as drawings, 16 is an image display device such as a CRT or liquid crystal display device, 18 is a camera 10, An image input / output circuit that selects the output image of 14 for transmission, selectively combines the output images of the cameras 10 and 14 and the received image, and supplies them to the image display device 16.

Reference numeral 22 is an image encoding circuit for encoding the image signal to be transmitted, and 24 is an image decoding circuit for decoding the received encoded image signal. Here, the image coding circuit 22 uses intra-frame and inter-frame coding using discrete cosine transform (DCT), vector quantization, and motion compensation.
In addition, the image data is band-compressed by dropping frames. I
CCITT Recommendation H.264 is an encoding method that can be transmitted at 64 Kbps of the basic interface of the SDN line. 261
There is.

Reference numeral 26 is a handset consisting of a microphone and a speaker, 28 is a microphone, 30 is a speaker, and 32 is a voice input / output interface for the handset 26, the microphone 28 and the speaker 30. The voice input / output interface 32 not only switches the voice input / output of the handset 26, the microphone 28, and the speaker 30, but also performs echo / cancel processing, dial tone, ring tone,
Tones such as busy tones and ring tones are generated.

Reference numeral 34 is a voice input / output interface 32.
Is a voice encoding circuit for encoding the voice signal to be transmitted from the device, and 36 is a voice decoding circuit for decoding the received encoded voice signal and outputting it to the voice input / output interface 32. The transmission rate and the encoding method are 64 Kbps PCM (A-law) and 64 Kbps PCM (μ-l).
aw), 7 KHz audio (SB-ADPCM), 3
2 kbps ADPCM, 16 kbps (eg LD
-CELP) and 8 Kbps.

Reference numeral 38 is a data terminal device such as a personal computer, and 40 is a data interface for connecting the data terminal device 38.

Reference numeral 42 is a system control circuit for controlling the entire system,
Reference numeral 44 denotes an operating device for inputting various instructions to the system control circuit 42, such as a keyboard, a touch panel,
It consists of a digitizer and a pointing device such as a mouse.

Reference numeral 46 is a communication line (for example, ISDN line).
Line interface, 48 is the image coding circuit 2
2. Information to be transmitted from the voice encoding circuit 34 and the data interface 40 and the system control circuit 42
Control information from H.264. 221 format is multiplexed and supplied to the line interface 46, and the image, audio, data and control signals are separated from the received information supplied from the line interface 46, and the image decoding circuit 24, the audio decoding circuit 36 and the data are respectively separated. A demultiplexing circuit that supplies the interface 40 and the system control circuit 42.

In this embodiment, the line interface 46
Is a gate circuit 46a for one channel B1 of the two B channels and a gate circuit 46 for the other channel B2.
b and a communication control circuit 46c for controlling communication through the D channel.

Further, the system control circuit 42 particularly includes an image transfer rate calculation circuit 50 for calculating the transfer rate assigned to the image data and an all-INTRA control circuit 52 for forcing intra-frame coding of all blocks of one frame. To have. Image transfer rate calculation circuit 50 and all INTs
Details of the operation of the RA control circuit 52 will be described later.

The flow of image signals and audio signals in the embodiment shown in FIG. 1 will be briefly described. The input image from the camera 10 and the document camera 14 is selected by the image input / output circuit 18 and applied to the image encoding circuit 22. The image encoding circuit 22 encodes the input image signal in the encoding mode according to the control signal from the system control circuit 42 and the internal determination, and outputs it to the demultiplexing / multiplexing circuit 48.

On the other hand, the input voice signal from the microphone or the microphone 28 of the handset 26 is input to the voice encoding circuit 34 via the voice input / output interface 32, where it is encoded and input to the demultiplexing circuit 48.

Data to be transmitted from the data terminal 38 is sent via the data interface 40 to the demultiplexing circuit 4
Enter in 8. Further, the data to be transmitted, which is input from the operation device 44, is also input to the demultiplexing / multiplexing circuit 48 via the data interface 40.

The demultiplexing circuit 48 is the image coding circuit 2.
2 and the encoded signal and data from the voice encoding circuit 34.
The data from the interface 40 and the control command from the system control circuit 42 are multiplexed and output to the line interface 46. Line interface 46
Outputs the signal from the demultiplexing / multiplexing circuit 48 to the connected communication line in a predetermined format.

The signal received from the communication line is supplied from the line interface 46 to the demultiplexing / multiplexing circuit 48. The demultiplexing circuit 48 separates the coded image signal, the coded audio signal, the data and the control command from the received signal, and the image decoding circuit 24, the audio decoding circuit 36, the data interface 40 and the system control circuit 42, respectively. To enter.

The image decoding circuit 24 is a demultiplexing circuit 4
The image input / output circuit 18 decodes the encoded image signal from
To enter. The image input / output circuit 18 includes the cameras 10, 14
And the received image from the image decoding circuit 24 are selectively combined and input to the image display device 16. The image input / output circuit 18 uses, for example, picture-in
It fits in the corresponding window in the picture or window display system. As a result, the input image and / or the received image is displayed on the screen of the image display device 16.

The received voice signal decoded by the voice encoding circuit 36 is applied to the speaker of the handset 26 and / or the speaker 30 via the voice input / output interface 32. As a result, the voice from the communication partner can be heard.

The received data separated by the demultiplexing / multiplexing circuit 48 is transferred from the data interface 40 to the data terminal 38.
Entered in.

H. 261 recommendation, NTSC system, PA
A common video format is defined to enable communication between multiple standards such as L-system and digital television signals. CIF format and QC
IF format. In the CIF format, the number of samples is 352 pixels × 288 lines for the luminance signal Y and 176 pixels × 144 lines for the color difference signals Cr and Cb. QCI
The F format has a quarter of the information amount of the CIF format, and the number of samples is 176 pixels × 144 lines for the luminance signal Y and 88 pixels × 72 lines for the color difference signals Cr and Cb.

As the elemental technique of the compression method, an image in a frame is divided into blocks of 8 pixels × 8 pixels, and two-dimensional discrete cosine transform (DCT transform) is performed on the blocks, ie, intraframe coding and the previous frame. Two-dimensional DC for the difference between frames for the block at the same position in the current frame
Inter-frame coding for T conversion, motion compensation that reduces the amount of generated code by compensating for motion of images between frames, DCT
Zero-run-length coding that uses zero values that generally continue in the high-frequency region in the transform coefficient, quantization that changes the quantization step size according to the amount of data generated, short codes for frequently occurring data patterns Variable-length coding that assigns long code values to data patterns that occur infrequently, and frame skipping that skips frames are used to achieve high compression ratios and low-rate communication channels. It enables video transmission over.

The inter-frame coding (INTER) mode can achieve a high compression rate when the correlation between frames is high, so that when there is a certain correlation or more between frames, I
The NTER mode is adopted, and when the correlation is less than a certain value, the intra-frame coding (INTRA) mode is adopted.

In the INTER mode, the quantization error, which is the difference between the quantization on the transmitting side and the quantization on the receiving side, is accumulated at the receiving side terminal, and becomes prominent when the quantization is coarse, so that Generally, the INTRA mode is arranged. In order to prevent transmission of a transmission error, the INTRA mode is periodically arranged for each block. In particular,
Only when there is no reference image for the difference, such as when starting image communication, or when there is a scene change (when it is determined that there is no inter-frame correlation on the entire screen due to scene switching) , All the blocks in the frame are intra-coded. This is called all-INTRA processing. All IN
By the TRA processing, the decoding error and the quantization error can be eliminated and the screen can be refreshed.

Next, a method of negotiating the terminal capability and changing it will be described by taking an ISDN line as an example. ISDN
In the line, as shown in FIG. 2, an outband signal (that is, D channel) is used to make a call. As shown in FIG. 2, the call setup from the terminal A to the terminal B and the call setup from the terminal B to the terminal A are performed.
In response to the request, communication on the B channel becomes possible. There are other communication channels such as D channel, H0 and H1, but only the B channel will be described.

Recommendation H.264 is used by using the B channel that has become communicable in this way. According to 242, an in-band signaling procedure is carried out on the B channel, as shown in FIG. 3, thereby allocating the inside of the B channel to the data part and the control part controlling the communication. The control performed by the in-channel control unit is called in-channel control. A control bit is required in the B channel for in-channel control, and its frame structure is recommended by H.264. 221.

FIG. 4 shows a frame structure in the B channel for executing the in-channel control. FIG. 4 shows a multiframe structure for the B channel (64 Kbps). This multi-frame structure is 1 octet / 125
Based on μs, one frame has 80 octets as shown in FIG. 4A, one sub-multiframe has two frames as shown in FIG. 4B, and one multiframe has 8 sub-frames as shown in FIG. 4C. It becomes multi-frame. Eight subchannels # 1 to # 8 of 8 Kbps are defined in the bit direction.

However, only the sub-channel # 8 has a transfer rate of 6.4 Kbps and FAS as a control bit.
(Frame Alignment Signal) and BAS (Bit-rate Allocation)
Signal) is inserted. The FAS and BAS enable in-channel control of the B channel.

FAS is used for frame and multi-frame synchronization. The BAS is used for exchanging information on terminal capabilities or setting capabilities necessary for determining a multiplexing method such as subchannels. In particular, the BAS can be switched every sub-multiframe (20 ms) even during data communication.

The in-band signal procedure shown in FIG. 3 will be briefly described. When the B channel becomes available for communication, terminal A,
Both Bs send FAS. The terminal capability at this time is
It is the mode 0 in the initial state (a mode only for voice and FAS and BAS). This FAS is searched by the other terminal,
H. When the condition for establishing frame synchronization defined by H.242 is satisfied, the bit structure A in the FAS shown in FIG.
0 "is transmitted. When the terminal receives A = 0,
It is confirmed that the partner terminal has established frame synchronization.

Next, the capability information of the own terminal is transmitted to the partner terminal by BAS, and the capabilities of the partner terminals are mutually confirmed. If they can communicate with each other at this point, data communication is started. When it is necessary to change the capability, similarly, the BAS is used to transmit the terminal capability as a command, and after the partner terminal completes the setting of the capability, data communication is started.

In data communication, transmission and reception are independent, and synchronization is established and terminal capability is set separately.
Therefore, synchronization may be lost in only one direction, or the type of data may differ between transmission and reception.

When the data communication is completed and the call is disconnected, first, the disconnecting terminal (terminal A in FIG. 4) is set to B.
Set to mode 0 using AS. As a result, the in-channel control of the B channel returns to the initial state. Next, as shown in FIG. 2, disconnection and release are performed in the D-channel out-band procedure, and all communication is completed.

FIG. 5 shows the bit structure in the FAS. Bit A indicates whether frame synchronization is lost, and E bit is C
Indicates whether an RC error has occurred. C1, C2, C3, C4
Are bits of CRC4. N1 to N5 are for multiframe numbering, and R1 to R4 are channel numbers. T
EA is a terminal device alarm, and is set to "1" when the terminal cannot respond to the received signal due to a failure inside the terminal.

FIG. 6 shows the bit structure in the BAS. As shown in FIG. 6A, the upper 3 bits represent an attribute,
The remaining 5 bits indicate the attribute value of that attribute. FIG. 6B shows the content of the attribute. The attribute value is, for example, a transfer rate value,
There are codec types, parameter values specific to each medium or information, and the like.

FIG. 7, FIG. 8, FIG. 9, FIG. 10 and FIG. 11 show flow charts of the characteristic operation of this embodiment as a whole. Here, X is the transfer rate secured for information transmission, and Y
Is a transfer rate obtained by subtracting the transfer rate secured for audio data transfer from the transfer rate X, Z is a transfer rate obtained by subtracting the transfer rate of data (LSD, HSD) from the transfer rate Y, and Zp is a transfer rate Z before the transmission capability is changed. Is.

First, as communication start processing, X, Y, Z,
Zp is initialized to 0, and the number of frames of all INTRA frames (frames in which all blocks are intra-coded) to be transmitted when the transmission capability is changed is set (S1).

It is checked whether or not it is a compound information terminal that handles images, voice and data (S2). If it is not a compound information terminal, it is checked whether or not it is a normal voice-only telephone (S3). In the case of communication as a telephone (S3), connection is established with the other terminal by call control on the D channel (S4) and connection is made on the B channel (S).
5). The voice section is activated (S6), and the line is maintained until the disconnection operation (S7). The disconnection operation disconnects the line (S7) (S8). If you are not on the phone (S
3), other terminal processing is executed (S9).

In the case of the complex information terminal (S2), the call is controlled by the D channel to connect with the partner terminal (S10), and the B channel is connected (S11). In-channel control of B channel is started (S12), and FAS synchronization is detected (S
13). When FAS synchronization is established (S13), BAS
The capability of the other terminal is confirmed by negotiation of the capability by (S14).

If the partner terminal has voice capability (S15),
In response to the voice transmission request from the operator (S16), the voice unit is activated (S17, 18). The transfer rate A of the audio data is subtracted from the secured transfer rate X of the B channel (S
19) It is checked whether or not the subtraction result Y is positive, that is, whether or not there is room to transfer an image or data (S).
20). If Y is not positive (S20), the data part and the image coding part are stopped (S21), and the process proceeds to S49. If Y is positive (S20), it is determined whether or not the partner terminal has the data capability (S25).

When the partner terminal does not have voice capability (S1
5) or when there is no voice transmission request by the operator (S1)
6), the voice part is stopped (S22, 23), and X is substituted for Y (S24). That is, the secured transfer rate X of the B channel is assigned for the transfer of images and data. Then, it is determined whether or not the partner terminal has the data capability (S25).

If the partner terminal has data capability (S2
5) In response to the data transmission request from the operator (S2
6) The data section is activated (S27, 25). Subtract the data transfer rate B from the available transfer rate Y (S
29), it is checked whether or not the subtraction result Z is positive, that is, whether or not there is room to transfer the image (S30). Z
Is not positive (S30), the image coding unit is stopped (S30).
31), and proceeds to S49. If Z is positive (S30), the presence or absence of the image capability of the partner terminal is determined (S35).

When the partner terminal does not have the data capability (S2
5) or when there is no data transmission request from the operator (S
26), the data part is stopped (S32, 33), and Y is substituted for Z (S34). That is, the available transfer rate Y is assigned for image transfer. After this, it is determined whether or not the image quality of the other terminal is available (S35).

If the partner terminal has image capability (S35),
In response to the image transmission request from the operator (S36), the image encoding unit is activated (S37, 38). By comparing Z and Zp, it is checked whether or not the image transfer rate is changed (S3
9), Z ≠ Zp, that is, if there is a change (S39), all I
Send the specified number of NTRA frames (S40,
41), Zp is updated with Z (S42), and thereafter, the image information is encoded as usual (S43). Even when the image transfer rate is not changed (S39), the image information is encoded as usual (S43). It progresses to S46 after S43.

When there is no image transmission request from the operator (S
36), the image coding unit is stopped (S44, 45).

When the partner terminal does not have image capability (S3
5) When there is a request from the operator to change the transmission ability (S4)
6) The capacity change is notified to the partner terminal (S47), if the B channel is added to the contents of the change (S53), the B channel connection is added (S54), and if the B channel is reduced (S55). ), Reducing the B channel connection (S56),
After that, S14 and subsequent steps are repeated.

If there is a disconnection request (S48) and it is from the partner terminal (S49), the BA from the partner terminal is sent.
According to the notification of the transition to the mode 0 by the S command, the mode transitions to the mode 0 in which only voice is handled (S50). If the disconnection request is from the own terminal (S49), the BAS command notifies the partner terminal of the transition to the mode 0, and the own terminal is set to the mode 0 (S51). After S50,51,
Line disconnection processing is executed by D channel control (S5
2).

The receiving side terminal does not need to be particularly aware of the above operation of the transmitting side terminal and does not need to add a specific function to the receiving side terminal.

In this embodiment, S39 to S42 prevent image distortion even when the image transfer rate is changed.
Even if it happens, it will be recovered in a very short time.

[0063]

As can be easily understood from the above description, according to the present invention, even if the image transmission rate is changed, the disturbance of the transmitted image can be greatly suppressed, and the transmitted image can be restored early. Further, since the transmitting side terminal is only controlled within the range of the recommendation, there is an advantage that it can communicate with all the receiving terminals complying with the recommendation.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing a procedure from call setting to disconnection.

FIG. 3 is a diagram showing an in-channel control procedure.

FIG. FIG. 221 is an explanatory diagram of a frame configuration of 221.

FIG. 5 is a bit configuration diagram of FAS.

FIG. 6 is a bit configuration diagram of BAS.

FIG. 7 is a part of an operation flowchart of the present embodiment.

FIG. 8 is a part of an operation flowchart of the present embodiment.

FIG. 9 is a part of an operation flowchart of the present embodiment.

FIG. 10 is a part of an operation flowchart of the present embodiment.

FIG. 11 is a part of an operation flowchart of the present embodiment.

[Explanation of symbols]

10: Camera 14: Document camera 16: Image display device 18: Image input / output circuit 22: Image coding circuit 24: Image decoding circuit 26:
Handset 28: Microphone 30: Speaker 32:
Voice input / output interface 34: Voice coding circuit
36: voice decoding circuit 38: data terminal 40: data interface 42: system control circuit 4
4: Operating device 46: Line interface 46a,
46b: Gate circuit 46c: Communication control circuit 48: Separation / multiplexing circuit 50: Image transfer rate calculation circuit 52:
All INTRA control circuit

Claims (2)

[Claims] 1. An image transmitting apparatus for compressing and transmitting image information by an encoding method having intra-picture encoding and inter-picture encoding, and encoding means for compressing and encoding image information to be transmitted, and image transmission. A transmission rate detecting unit that detects a change in the rate and a predetermined number after the transmission rate is changed by an encoding method that encodes the entire screen in the screen according to the detection result of the transmission rate detecting unit. An image transmitting apparatus comprising: a control unit that encodes a screen. 2. The image transmitting apparatus according to claim 1, wherein the predetermined number is plural.