JP3263265B2 - Image communication method and image communication device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image communication apparatus for performing communication using an image such as a videophone or a video conference apparatus.

[0002]

2. Description of the Related Art Conventionally, a video telephone and a video conference apparatus conforming to international standards have been known as image communication apparatuses. In these conventional image communication devices, ITU-T
Recommendation H. In accordance with H.261 (recommended number), the transmitting side first encodes and transmits the entire captured image as an intra-frame signal, then captures the next screen, encodes difference information with respect to the previous full-screen information, and encodes the inter-frame signal. Is transmitted as. On the receiving side, similarly, the operation of decoding and displaying the transmitted intra-frame signal and displaying the subsequently received inter-frame signal after decoding is performed.

[0003]

However, in a videophone or a video conference apparatus according to the above-mentioned conventional international standard, encoding, transmission and decoding of full-screen information having a large amount of information as compared with difference information are difficult. There is a problem that it takes a lot of time, and it takes a considerable amount of time before an image is displayed on the receiving side after the line connection.

[0004] In addition, full-screen information depends on an image to be captured.
Since the amount of information differs, the time required for encoding / decoding also differs. For this reason, there is a problem that a considerable difference occurs in the time until the image is displayed on the receiving side between the simple image of a single color and the complicated image with various changes.

[0005] Further, since the communication time varies due to the difference in the amount of information of the image to be captured, if the time until the reception of the intra-frame signal exceeds the timer for detecting the intra-frame signal in the terminal, the receiving side is not changed. In this case, there is a problem that a signal in a frame is lost.

[0006] In performing image communication, communication using only difference information is not established unless all screen information is first transmitted and received. However, if the time required for the image to be displayed differs depending on the screen information captured as described above, or if it takes an enormous amount of time, psychological problems of the user, anxiety until the picture of the other party appears, etc. There is a major problem in promoting image communication terminals.

[0007] An object of the present invention is to solve the above-mentioned problems, so that an image can be viewed immediately when a line is connected, regardless of the type of image to be captured, such as complexity or simplicity, and the frame on the receiving side can be displayed. It is an object of the present invention to provide an image communication method and an image communication apparatus that can prevent a signal from being lost.

[0008]

In order to achieve the above object, according to one aspect of the present invention, an image communication terminal using an encoding method using an inter-frame difference is connected to an image communication terminal. Prior to transmitting and receiving information, an image communication method for notifying capabilities between terminals is performed, and it is determined whether or not the same fixed image signal is stored at the time of the capability notification between the image communication terminals, and stored. If it is determined that the image information is transmitted, the input side of the image information is compared with the standard image signal, only the difference signal between them is encoded and transmitted to the other terminal, the image information receiving side In the method, the stored fixed image signal is displayed until the difference signal is received, and when the difference signal is received, the image signal obtained by decoding the difference signal and the stored fixed image signal are superimposed. display The image communication method configured to means that.

In the above image communication method, the fixed image signal may be a single color image, or
It is preferable that the fixed image signal is an image in which an object is arranged on a screen.In another aspect of the present invention, the fixed image signal is connected to a transmission path,
An image communication device that includes an encoding unit and a decoding unit using an encoding method using an inter-frame difference, and in an image communication device that transmits and receives image information, a storage unit that stores a standard image signal,
When the transmission path is connected, the own terminal notifies the other terminal that the fixed image signal is stored in the terminal, or stores the same fixed image signal based on the notification from the other terminal. Negotiation means for detecting that the input image signal has been detected, and when image information is to be transmitted, the input image signal is compared with the standard image signal, and only the difference signal between them is encoded. When the image information is received when the detection is performed and the detection is performed and the image information is received, the standard image signal stored in the storage unit is displayed until the difference signal is received, An image communication apparatus having a configuration including, upon receiving the difference signal, control means for superimposing and displaying an image signal obtained by decoding the difference signal and the stored fixed image signal.

In the above image communication apparatus, the fixed image signal stored in the storage unit may be a single color image, or the fixed image signal stored in the storage unit may include an object arranged in a screen. Preferably, it is an image

[0011]

According to the first and fourth aspects of the present invention, the image communication terminal at each point connected by the transmission line stores the same image signal in the storage unit, and after the transmission line is connected, In, negotiate whether each terminal stores the same image signal, if both are storing the same image signal, the stored image signal,
It compares the captured image with the actually captured image, encodes only the difference information, and transmits it to the terminal at the other point.The terminal at the other point transmits the image signal stored therein until the difference signal is transmitted. When the difference signal is transmitted, the difference information is decoded, the difference information is decoded, and the same image signal as that of the partner terminal stored therein is superimposed to restore the image actually captured by the partner. Reducing the amount of information, shortening the communication time, regardless of the type of image to be captured, such as complexity or simplicity, the image can be viewed immediately when the line is connected, and the receiving side does not lose the signal in the frame. Realize image communication.

According to the second and fifth aspects of the present invention, in addition to the functions of the first and fifth aspects of the present invention, the fixed image signal stored in the terminal is converted into a simple image as a single color image. Thereby, the storage capacity is reduced.

According to the third and sixth aspects of the present invention, in addition to the functions of the first and fifth aspects, the fixed image signal stored in the terminal is defined as an image in which an object is arranged on a screen. As compared with the invention of claim 5, the difference signal between the fixed image signal stored in the terminal and the actually captured image signal is reduced, the communication time is reduced, and higher speed image communication is realized. I do.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

In the embodiment described below, connection is made to the basic interface of ISDN, and the ITU-T Recommendation H.264 is used. An image communication terminal conforming to H.320 (recommendation number) will be described as a configuration example. It should be noted that other signals such as audio are not described because they are not directly related to the present invention.

FIG. 1 is a block diagram showing an embodiment of an image communication apparatus according to the present invention.

In FIG. 1, 1 is an operation unit, 2 is a control unit,
3 is an image input unit, 4 is an image encoding unit, 5 is a demultiplexing unit,
Reference numeral 6 denotes a network interface unit, 7 denotes an image display unit, 8 denotes an image decoding unit, and 9 denotes a fixed image signal storage unit.

The operation unit 1 transmits to the control unit 2 a designation of a partner terminal performed by the operator, a call activation request, and a call response request.

The control unit 2 receives a request from the operation unit 1 for starting a call or the like, connects a line, and instructs each unit from the start to the end of communication and manages the operation of each unit.

The image input unit 3 captures an image from the outside, and transfers the captured image to the image encoding unit 4.

At the start of communication, the image coding unit 4 reads out a standard image signal from a standard image signal storage unit 9 which stores a standard image signal in advance and compares the input image with the standard image signal according to its ability. By doing so, the differential signal is extracted and the ITU-T Recommendation H. H.261 (recommended number), and the frame is transferred to the demultiplexer 5, or the input image is directly used as per ITU-T Recommendation H.264. 2
The entire screen information is coded according to 61 and transferred to the demultiplexing unit 5. At times other than the start of communication, a difference signal is extracted by comparing the currently captured image with the next newly captured image, and the ITU-T Recommendation H.264. 2
61, and the demultiplexing unit 5
To the ITU-T Recommendation H.
261 is encoded, and transferred to the demultiplexing unit 5.

The demultiplexing unit 5 converts the coded image signal transferred from the image coding unit 4 and other coded signals such as audio into ITU-T Recommendation H.264. Multiplexing is performed using a frame configuration conforming to H.221 (recommended number), or conversely, a multiplexed signal received from a partner via the network interface unit 6 is separated into an image signal and other signals. The demultiplexing unit 5
Follows ITU-T Recommendation H.264 after line connection. 242 (recommendation number), etc., the capability notification between the terminals is performed with the partner terminal to determine the common capability with the partner terminal.

The network interface unit 6 receives a line connection instruction from the control unit 2, and connects a line to a partner terminal via the ISDN network.

The image display section 7 displays an image from the image decoding section 8.

The image decoding unit 8 decodes the image signal transferred from the demultiplexing unit 5 and separated from other signals such as audio, according to ITU-T Recommendation H.264. 261 (recommended number), and at the start of communication, at the start of communication, the fixed image signal read from the fixed image signal storage unit 9 is superimposed on the decoded image signal and transferred to the image display unit 7 or the full-screen encoded information Is received, the decoded image signal is transmitted without being superimposed and transferred to the display unit 7. At the time other than the start of communication, the image previously transferred to the image display unit 7 includes:
Next, after transferring from the demultiplexing unit 5, the decoded difference signal is superimposed and transferred to the image display unit 7.

In the above, when there is no image signal defined in advance, the control unit 2 instructs the image encoding unit 4 to encode the full screen information with respect to the image taken from the image input unit 3. To perform

The image encoding unit 4 encodes the full screen information,
The data is transferred to the demultiplexing unit 5. The demultiplexing unit 5 performs the ITU-T Recommendation H.264 as in the case of having a fixed image signal. 221
After multiplexing with other signals according to the (recommended number), the image signal is transmitted from the network interface unit 6 to the partner terminal via the ISDN network. Since the image signal is transmitted from the partner terminal in the same manner, after reception, the image signal and the other signals are separated by the demultiplexing unit 5 and only the image signal is transferred to the image decoding unit 8.

The image decoding section 8 decodes the transferred image signal and displays it on the image display section 7.

When the transmission / reception of the image signal is completed, the operation of comparing the first captured image signal with the next captured image signal, extracting the difference signal, encoding, and transmitting the signal is performed.

Here, the operation of the control section 2 will be described in detail.

The control unit 2 receives a request from the operation unit 1 for starting a call and starting a call, and instructs the network interface unit 6 to perform a line connection. When the line is connected, the multiplexing / demultiplexing unit 5 sends an ITU message to the other terminal.
-T Recommendation H. 242 (recommended number).

In response to the result of the capability exchange, an instruction such as an encoding system and a decoding system is given to necessary units (the image encoding unit 4, etc.).

First, an operation example of the control unit 2 in image transmission will be described.

The image input unit 3 is instructed to capture an image and to transfer the captured image to the image encoding unit 4.

Next, for the image encoding unit 4, the encoding method and operation mode (the standard mode as recommended or the original mode using the standard image signal according to the present invention) as a result of the capability exchange are set. Therefore, it instructs to encode and transfer to the demultiplexing unit 5.

Next, the image signal transferred from the image encoding unit 4 is transmitted to the demultiplexing unit 5 by combining the data other than the image with the data other than the image. Multiplexing into a frame configuration conforming to H.221 (recommendation number), and an instruction to transfer to the network interface unit 6.

Next, the network interface unit 6 is instructed to transmit the multiplexed data transferred from the demultiplexing unit 5 to the partner terminal at a transmission rate according to the result of the capability exchange.

The operation example of the control section 2 in the image transmission is as described above.

Next, an operation example of the control unit 2 in image reception will be described below.

The network interface unit 6 receives the multiplexed signal transmitted from the partner terminal via the ISDN network, and instructs the network interface unit 6 to transfer the multiplexed signal to the demultiplexing unit 5.

Next, the demultiplexing unit 5 is instructed to separate the multiplexed data transferred from the network interface unit 6 into an image and data other than an image, and to transfer it to the image decoding unit 8. .

Next, for the image decoding unit 8, the encoding method and the operation mode (the standard mode as recommended or the original mode using the standard image signal according to the present invention) as a result of the capability exchange are set. Therefore, it instructs to decode and transfer to the image display unit 7.

Next, the image display unit 7 is instructed to display the signal transferred from the image decoding unit 8.

The above is an operation example of the control unit 2 in image reception.

Next, the relationship between input and output of image signals will be described with reference to FIG.

When a line connection is made between terminals, the ITU-T Recommendation H.264 is used between the demultiplexing units 5 of each terminal via the network interface unit 6 and the ISDN network. 242 (advisory number) compliant terminals are notified of each other's capabilities. When it is determined from the capability notification that an image signal (described later) of a predetermined fixed format is stored, the control unit 2 instructs the image encoding unit 4 to output the fixed image signal storage unit. 9 is read in advance and compared with the input image transferred from the image input unit 3 to extract a difference signal and issue an instruction to perform encoding.

The image coding unit 4 transfers the coded image signals extracted from the two image signals to the demultiplexing unit 5.

The demultiplexing unit 5 multiplexes the transferred difference signal with a signal other than an image such as audio and transfers the multiplexed signal to the network interface unit 6.

The network interface unit 6 transmits the transferred multiplexed signal to the partner terminal via the ISDN.

Similarly, if the reception of the capability information between the terminals indicates that both of them have stored an image signal of a predetermined fixed format, the control unit 2 sends the image signal to the image decoding unit 8. First, the standard image signal stored in the standard image signal storage unit 9 is read out, transferred to the image display unit 7, and an instruction to display the signal is issued. Then, the pre-stored standard image signal is read out again from the standard image signal storage unit 9, the read standard image signal is superimposed on the received signal transmitted from the image decoding unit 8, and transferred to the image display unit 7. Give instructions.

In the image decoding unit 8, the network interface unit 6 decodes the image signal received through the ISDN network and separated from other signals such as audio by the demultiplexing unit 5, and then decodes the decoded differential signal. Is superimposed on the standard image signal and transferred to the image display unit 7.

The image display section 7 displays the superimposed signal transferred from the image decoding section 8.

On the other hand, when the mutual capability notification performed between the network interface units 6 of the respective terminals reveals that they do not have image signals defined in advance by each other, the control unit 2
Instructs the image encoding unit 4 to encode the full screen information for the screen fetched from the image input unit 3.

The image encoding unit 4 encodes the full screen information,
The data is transferred to the demultiplexing unit 5. The demultiplexing unit 5 performs the ITU-T Recommendation H.264 as in the case of having a fixed image signal. 221
After multiplexing with other signals according to the (recommended number), the image signal is transmitted from the network interface unit 6 to the partner terminal via the ISDN network. Since the image signal is transmitted from the partner terminal in the same manner, after reception, the image signal is separated into the image signal and the other signals by the demultiplexing unit 5, and only the image signal is transferred to the image decoding unit 8.

The image decoding section 8 decodes the transferred image signal and displays it on the image display section 7.

When the transmission and reception of the above image signals are completed, the transmission and reception of the inter-frame difference signal are performed next. As a result of the notification of the partner capability, when both terminals have the fixed image signal, the image coding unit 4 compares the fixed image signal with the input image signal, but when the two terminals do not have the fixed image signal, The first and second image signals are compared, the difference signal is extracted, encoded, and transmitted.

The above is the relationship between input and output of image signals.

Next, the fixed format image signal will be described with reference to FIG.

As the image signal stored in the fixed image signal storage 9, a single color signal such as blue or black as shown in FIG. The advantage of using a single color is that the time required for encoding and decoding is reduced.

In addition to the above, a method of arranging a person image of the upper body at the center of the image assuming a meeting as shown in FIG. 6B is also conceivable. It is also conceivable to change it. Also, assuming monitoring, etc.,
It is also conceivable to arrange a specific object on the screen or to store only the background. Alternatively, storing a typical one-shot is also effective for monitoring a place with little movement.

In these cases, there are advantages that the difference signal is smaller than that of the single color, the transmission efficiency is improved, the communication delay is reduced, and an initial screen closer to the actual image can be obtained.

Next, the capability notification between terminals will be described with reference to FIGS.
This will be described with reference to FIG. FIG. 2 is a flowchart summarizing an operation example of the present embodiment, FIG. 3 is a diagram showing a communication procedure related to a capability notification, FIG. 4 is a diagram showing notification contents in a capability notification, and FIG. It is a figure showing the contents of an extension byte.

First, after the line is connected, the ITU-T
Recommendation H. 242 (recommendation number) and the like, the capability notification between terminals is performed. The procedure of this capability notification is as shown in FIG. 3, and the content is as shown in FIG. That is, when a video mode, a transfer rate, a non-standard mode indicating possession of a fixed image signal, and the like are transmitted from the receiving terminal to the transmitting terminal (1), the transmitting terminal determines a common maximum capacity (2), and And notifies the receiving terminal.

The "fixed image signal storage" to be notified here is not included in FIG. 4, but uses a non-standard command as shown in FIG. As a possible notification method of “regular image signal storage”, the provider code coincidence, that is, ByteNo. e and f are completely matched with b _{0 to} b ₇ , and a code is assigned to “regular image signal storage” using a unique mode identification code. A method may be considered in which all of b _{0 to} b _{7 of} g are set to “1” so that the terminal is recognized as a “standard image signal storage” terminal.

FIG. 2 summarizes the operation of this embodiment. When it is determined that an image signal of a certain fixed format, which is defined in advance, is stored, the fixed image signal is compared with the input image.

Next, a difference signal is generated from the signal after the superposition. Then, the difference signal is transmitted to another point.

After transmission, a difference signal with respect to the previous input signal is encoded and transmitted to another point. Hereinafter, in the same manner, after encoding the difference information, operations such as transmission to another point and insertion of full-screen information at certain intervals are repeated.

Similarly, for the reception, the operation of decoding the received difference information, superimposing it on the previously displayed signal, displaying it, decoding the full screen information received at a certain interval, and then displaying it is repeated.

On the other hand, if the fixed form image signal is not stored in the capability notification, the full screen signal is encoded for the captured image in the same manner as in the conventional image communication apparatus, and transmitted to another point.

After transmission, the difference signal for the previous full screen signal is encoded and transmitted to another point. Hereinafter, in the same manner, after encoding the difference signal, the operation of transmitting the signal to another point and inserting the image signal at certain fixed intervals is repeated.

The image communication method and the image communication apparatus according to the present invention can be applied not only to a video conference system but also to any system for transmitting and receiving images. There can be many different things.

[0072]

As described above, according to the image communication method and the image communication apparatus of the present invention, at the start of communication, regardless of the type such as complexity or easiness of the initially captured image, it is stored in the storage unit in each terminal. Memorize the same fixed image signal,
After connecting the transmission line, between the terminals, a negotiation is performed as to whether each terminal stores the same fixed image signal, and when both terminals store the same fixed image signal, the fixed image signal and Since the differential signal is transmitted, the amount of information can be reduced and the communication time can be reduced, and the receiving side can immediately display the stored image signal and receive the intra-frame signal. It is possible to eliminate loss of the intra-frame signal, which occurs on the receiving side when the time until the timer for detecting the intra-frame signal in the terminal exceeds the time.

Further, since the receiving side can display its own fixed image from the start of communication to the reception of the difference signal,
The effect of eliminating psychological anxiety and the like can be obtained. Therefore, it is considered that the effect of the present invention will be further enhanced if an image is prepared according to the application as a standard image.
For example, a person is placed on the screen at the time of a meeting, and an object to be monitored is placed on the screen at the time of monitoring.

Further, in the above, when a monochromatic screen is used as the standard image, the storage capacity of the storage unit can be reduced, and when an image in which objects are arranged in the screen is used, the communication time is reduced. It can be further reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of an image communication apparatus according to the present invention.

FIG. 2 is a diagram showing the operation of an embodiment of the image communication device according to the present invention.

FIG. 3 is a diagram showing a communication procedure regarding a capability notification in the embodiment.

FIG. 4 is a diagram showing notification contents in a capability notification in the embodiment.

FIG. 5 is a diagram showing contents of an extension byte in a capability notification in the embodiment.

FIGS. 6A and 6B are diagrams illustrating an example of contents stored in a fixed image signal storage unit.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 operation unit 2 control unit 3 image input unit 4 image encoding unit 5 demultiplexing unit 6 network interface unit 7 image display unit 8 image decoding unit 9 fixed image signal storage unit

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-6-90466 (JP, A) JP-A-6-319130 (JP, A) JP-A 4-186986 (JP, A) JP-A-6-186 233009 (JP, A) JP-A-4-343558 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N ^7/ 14-7/15 H04N ^7/ 24-7/68

Claims (6)

(57) [Claims] An image communication method for mutually notifying capabilities of image terminals prior to transmitting and receiving image information after image communication terminals using an encoding method using an inter-frame difference are connected to each other, It is determined whether or not the same fixed image signal is stored at the time of the capability notification between the image communication terminals.If it is determined that the same fixed image signal is stored, on the transmission side of the image information, the input image signal and the A comparison with the standard image signal is performed, only the difference signal between them is encoded and transmitted to the partner terminal. On the receiving side of the image information, the stored standard image signal is displayed until the difference signal is received. Receiving the difference signal, superimposing and displaying an image signal obtained by decoding the difference signal and the stored fixed image signal. 2. The image communication method according to claim 1, wherein said standard image signal is a single color image. 3. The image communication method according to claim 1, wherein said standard image signal is an image in which an object is arranged in a screen. 4. An image communication device connected to a transmission path, comprising an encoding unit and a decoding unit using an encoding method using an inter-frame difference, and transmitting and receiving image information. A storage unit that, when a transmission path is connected, notifies the other terminal that the own terminal stores the fixed image signal in the terminal, or the same based on the notification from the other terminal. Negotiation means for detecting that a fixed image signal is stored, and when the detection is performed and image information is transmitted, an input image signal is compared with the fixed image signal, and When only the difference signal is encoded and transmitted to the partner terminal, or when the image signal is received when the detection is performed, the fixed form stored in the storage unit until the difference signal is received. image Control means for displaying a signal, receiving the difference signal, and superimposing and displaying an image signal obtained by decoding the difference signal and the stored fixed image signal. 5. The image communication apparatus according to claim 4, wherein the fixed image signal stored in the storage unit is a single color image. 6. The image communication device according to claim 4, wherein the fixed image signal stored in the storage unit is an image in which an object is arranged on a screen.