JPH04129466A - Multi-medium communication equipment - Google Patents

Abstract

PURPOSE:To prevent interruption of transmission of unexpected video information by revising the coding and decoding system of video information when the transmission speed of the video information is lower than a specified value. CONSTITUTION:A video coding decoding section 10 is provided with a CODEC selection switch selecting a CODEC for a video signal inputted from a video interface section 9, an output video signal to a multiplexer/demultiplexer section 15, a video signal inputted from the multiplexer/demultiplexer section 15 an output video signal to the video interface section 9 and revises the coding decoding system of the video information when the transfer speed able to be assigned to the video information is lowered than the specified value when the video signal transmission is set. For example, the transfer speed is automatically selected in response to the transmission speed able to be assigned to the system other than the current video image CODEC. Thus, the interruption of the unexpected video information transmission is prevented.

Description

Detailed Description of the Invention (Industrial Field of Application) The present invention provides a video telephone device/TV device capable of multiplexing multimedia information such as audio information, video information, code data information, etc. and mutually communicating with each other. AV for conference systems, etc.
The present invention relates to multimedia communication devices, typified by (Audio Visual) communication devices.

(Prior Art) In recent years, communication services using 15DN lines have been put into practical use, and AV (Audi.

Visual) services are attracting attention. AV Sasa-
Service specifications, protocol specifications, multimedia multiplexing frame structure specifications, video information encoding methods, etc. for
CITT Recommendation or Draft Recommendation) (320, H, 242,
It has been announced as H, 221, H261, etc. H,2
21 defines the frame structure for AV services on 64 kbps to 1920 kbps channels. Figure 3 shows 8.22 on 64kbps x 1 channel.
1, numbers 1 to 8 on the horizontal axis are bit numbers, numbers 1 to 80 on the vertical axis are octet numbers, and 80 octets constitute one frame. The FAS (frame synchronization signal) in the figure controls frame synchronization and multiframe synchronization, monitors communication quality, and controls alarm information notification. The table shows that FAS 1 multiframe = 8 submultiframes (1 submultiframe = 2
This shows the bit assignment between frames. Also BAS
(Bit rate allocation signal) specifies terminal capability, bit rate allocation for each media within an actual frame, and performs various other controls and notifications. BAs as shown in the table
are transmitted in even frames, and corresponding error correction bits are transmitted in odd frames. H.242 defines communication procedures such as a capability information exchange sequence and a mode switching sequence using in-channel BAS between AV terminals. 1L320 defines the overall system aspects of AV services. p in I-1,261
An encoding/decoding method for video information at a speed of x64 kbls (p=1 to 30) is defined.

According to the above recommendation or draft recommendation, when performing multimedia communication such as video, audio, data (all user information other than video and audio), the information transmission speed of each media, and the audio operation mode for audio information must be specified. determined by
Data information is basically determined by specifying a data mode.

The transmission speed of video information is the remainder after subtracting the transmission speed of audio information, data information, etc. from the capacity of the communication line effective for mutual communication. Therefore, the transmission speed of video information cannot be specified clearly, but is variable depending on the specification of other media. Video information according to 8.261 is basically not taken into account when only a very low speed is allocated as the video information transmission speed, so it is recommended to stop the transmission of video information, but no clear regulations have been made. Not yet. Coding systems for video information other than H.261 are currently outside the scope of the CCITT recommendations or draft recommendations. Apart from this, the video still image encoding method is CC
Standardization work is being carried out by the ITT/ISO joint group JPEG, but the treatment of this method is outside the scope of the above recommendations.

(Problem to be Solved by the Invention) However, in the above conventional example, the transmission speed of video information is only automatically allocated by subtracting the total transmission speed of media capable of audio information, data information, etc. From the user's perspective, does the transmission rate assigned to the video information based on the user's specified other media selection instructions end up being too low for the video codec capacity of the device in question? Operations must be carried out with this in mind, but this is extremely difficult. Furthermore, it is also conceivable that the transmission speed allocated to video information becomes insufficient when the number of connections in use is reduced due to the occurrence of an arbitrary connection failure when multiple connections are used.

Although the 8.261 video code can send and receive video information even at such a very low transmission speed, the monitored video will be extremely unnatural. In addition, the above ccIT
As recommended in the T Recommendation draft, a method has been considered to simply turn off the video information when the value falls below a certain predetermined value. The drawback was that it felt extremely strange because the video information was lost. Moreover, since the transmission video off that occurs when the transmission mode is changed does not affect the local side, it has the disadvantage that it is difficult to recognize unless the reception mode changes such as turning off the reception video from the other side. there were. When turning off transmitted video, it is recommended to first send a screen freeze request using a BAS command, then send a BAS command to turn off the video, and then turn off video transmission. Therefore, the image that is frozen on the receiving side is not necessarily clear, and the sending side has the disadvantage that it is not aware of what kind of image will be frozen.

Furthermore, there are no particular regulations regarding means for restarting video information that has been turned off.

The present invention has been made in view of the above circumstances, and its purpose is to eliminate the problem of sudden loss of video information, and to avoid blurry screen freezing as in H.261 video images. Another object of the present invention is to obtain a multimedia communication device capable of restarting video information that has been turned off.

(Means for Solving the Problems) In order to achieve this object, the present invention multiplexes and mutually communicates multimedia information such as audio information, video information, code data information, etc. via a communication line, and specifies audio information. The code data information is encoded and decoded according to the specified transmission speed, and the code data information is mutually communicated according to the specified transmission speed, and the transmission speed allocated to voice information, code data information, other control information, etc. from the total transmission speed of the communication line used. In a multimedia communication device that encodes and decodes video information according to the remaining transmission speed after subtracting the above-mentioned video information, when the transmission speed of the video information becomes lower than a certain specified value, the encoding/decoding method of the video information is changed. It has a changing means.

In addition to the above invention, there is also provided a means for storing that a change in the encoding/decoding method of video information has been made due to a decrease in the allocated transmission speed, and a means for storing that the transmission speed that can be allocated to the video information is higher than a certain specified value again. The present invention has means for returning the video information to the original encoding/decoding method when the video information is encoded/decoded.

(Operation) In the apparatus according to the present invention, when the transmission rate that can be assigned to video information becomes lower than a certain specified value when video transmission is on, the encoding/decoding method for video information is changed. For example, the current video codec is automatically switched in accordance with the allocatable transmission rate to a different system. This prevents inadvertent interruption of video information transmission and enables automatic provision of video information transmission appropriate to the transmission speed.

In addition, when switching to a still image, the user can select the image to be sent, and in this case, it is possible to select the image to be sent, such as a frozen screen of an H3261 video image, and from the perspective of the sending side, which moment the image is frozen by the other party. It is also possible to avoid methods that you will not know if you have messed up.

It also remembers that a change in the encoding/decoding method for video information was made due to a decrease in the allocated transmission speed, and when the transmission speed that can be allocated to video information becomes higher than a certain specified value, the video information is changed back to the original code. Return to the encryption/decoding method. This results in
Transmission of moving image information using the optical encoding/decoding method can be restarted without requiring any special operation by the user.

(Example) An example of the present invention will be described below with reference to the drawings. 1st
FIG. 1 is a block diagram showing the configuration of a multimedia communication device according to an embodiment of the present invention. In the figure, 1 is a handset which is the audio input/output means of this device, 2 is a microphone which is one of the audio input means of this device, 3 is a speaker which is one of the audio output means of this device, 4 is a switching process for switching the handset 1, microphone 2, and speaker 3 as audio input/output means according to instructions from the system control unit 14, which will be described later; and on/off hook detection for detecting whether the handset is in an on-hook state or an off-hook state. an audio interface unit that performs echo canceling processing to eliminate echo when the microphone 2 and speaker 3 are used as audio input/output means, processing for generating tones such as dial tone, ring tone, busy tone, ring tone, etc.;
5 is a 64 kbps PC according to instructions from the system control unit 14.
MA-1aw, 64kbps PCMμmlaw, 64
bps/56kbPs/48kbpsSB-AD
PCM, 32khpsADPCM, 16kbps
This is an audio encoding/decoding unit that encodes a transmitted audio signal and decodes a received audio signal according to an audio signal encoding/decoding algorithm such as APC-AB (for example, APC-AB), 8 kbps, or the like. In addition, 6 is one of the video input means of this device, and is a camera for inputting self-images, etc., and 7 is one of the video input means of this device, and 5 is a document camera for inputting drawings, etc. , 8 is a display unit for displaying incoming images from the camera 6 or document camera 7, images received from the other party, operation faces, etc.;
Reference numeral 9 denotes a video unit that performs, in response to instructions from the system control unit 14, processing for switching video input means, display switching processing for input video, received video, and operation screen, and video signal synthesis processing for displaying them separately on the display unit 8. Interface part, 10
The encoding of transmitted video and the decoding of received video signals in accordance with CCI TT Recommendation Draft H, 261, the encoding and decoding of video in a proprietary mode, and the standardization of joint group JPEG of CCITT/ISO. This is a video encoding/decoding unit (video codec unit) as a changing unit that performs encoding/decoding of video still images. Furthermore, 11
2 is a data terminal for performing data communication, and 2 is a demultiplexing unit 15/\ notifies transmission data from the data terminal 11 and system control unit 14, and also notifies received data to the data terminal 11 or system control unit 14. 13 is an operation section such as a keyboard used for inputting control information for controlling the device in general; 14 is an operation section such as a keyboard;
is equipped with a CPU, ROM, RAM, auxiliary storage device, etc., monitors the status of each part, controls the entire device, calculates the transmission speed to be assigned to video information based on input control information, usage line status, etc.1, and It is a system control unit that performs final mode judgment and control, operation/display screen creation according to the state, execution of application programs, etc., and the system control unit 14 includes the storing means and restoring means of claim 2. have Furthermore, 15 is an audio signal from the audio encoding/decoding unit 5 according to the CCTTT recommendation draft H1221, a video signal from the video encoding/decoding unit 0, data from the data interface unit 12, data from the system control unit 14, and A demultiplexing unit multiplexes control information such as CCITT Recommendation Draft H, 221, H, 242, etc. in units of transmission frames, separates received frames into constituent media, and notifies each unit; 16 is a l5DN user/network This is a line interface unit that controls lines according to the interface.

FIG. 2 is an internal block diagram of the video encoding/decoding section 10 shown in FIG. In the figure, reference numeral 101 denotes a transmission codec selection switch for selecting which video codec the video signal input from the video interface unit 9 is input to.

102 is an H.261 video codec that controls the encoding and decoding of moving images in accordance with CCI TT Recommendation Draft H.261; 103 is a JPEG video codec that controls the encoding and decoding of still images in accordance with JPEG; 104 is a unique mode video codec that controls encoding and decoding of simple moving images using a unique mode.

A transmission codec selection switch 105 selects which video codec is used to output the video signal to the demultiplexer 15. 106 is a reception codec selection switch that selects and switches to which video codec the video signal inputted from the demultiplexer 15 is input; 107 selects which video codec to output the video signal to the video interface unit 9; This is a receive codec selection switch. 108 is a video codec section (102
~104) A video codec control unit that controls the entire system.

Table 3 is shown in CCITT Recommendation H, 221.
2 is a diagram showing a frame structure in a kbps channel, and the table shows hint assignments of FAS and RAS within one multiframe. FIG.

FIGS. 4(a)-(f) and FIGS. 5(a)-(b) are frame structures showing examples of multiplexing multimedia into 8.221 frames.

The process will be explained in detail below based on the flowchart shown in FIG.

First, as shown in FIG. 4(a), it is assumed that mutual communication is performed using 64 kbps x 1 channel, 21 audio information at 16 kbps, and 22 video information at 46.4 kbps. In other words, it is assumed that the video switch indicating whether or not to perform video transmission is in an on state. Here, in the multimedia communication device, if when the video switch is on, if the transmission speed that can be allocated to video information is X (= 40 Kbps) or more, the transmission speed is 8.261 as shown in FIG. 4(b). Enables transmission of video information. In Figure 4(b), 21 is the fourth
Same as figure (a), 23 is 40Kbps video information, 24
indicates data at 6.4 Kbps. The transmission speed that can be allocated to video information is less than X/Y (= 24Kbps)
If it is above, it is possible to send simple video information in the unique mode, and if it is less than 7, the initial setting is that video can only be sent by sending JPEG-compliant still image information.

First, in step 5101, the process monitors and waits until there is a mode switching request to change the current communication mode.

If a mode switching request is detected, in step S102, the multiplexing format for each medium is determined according to the switching contents, and the transmission rate that can be assigned to the video information is calculated. The content of the mode switching request here is data information 14.4.
If the request is for an additional kbps, the transmission rate that can be allocated to video information at this time is 32 kbps. next,
Proceeding to step 5103, it is checked whether the video information transmission is currently on (the video switch is on). If video transmission is on, the process proceeds to step 5104, where the transmission rate is compared with a predefined specified value X. This specified value may be approximately 40 kbps as recommended in the CCITT draft recommendation, or may be determined based on the capability of the applicable H1261 video codec. If the transmission speed that can be assigned to the video is lower than the specified value X, step 510
Proceed to step 5. Furthermore, in step 5105, the transmission rate that can be assigned to the video is compared with a predefined specified value Y. In the above example, the allocatable transmission rate is lower than X and higher than Y, so as a result, it is decided to select video information transmission using simple video information using a unique mode video codec. At this time, the process advances to step 8106, and the unique mode video codec 104 (see FIG. 2) is activated to start operating at a transmission rate of 32 Kbps. Next, the process advances to step 5107, and based on the above results, <RAS
Along with the transmission of commands, in this case 14.4 Kbps data transmission and 16 Kbps audio information transmission using the IB, transmission using the new mode for each medium is started. At this time, if 8.261 video transmission is on, a BAS command to turn it off is also sent at the same time. Above simple video information 3
Regarding the allocation to the 2Kbps H,221 frame,
Since it is not specified in Recommendation Draft H.221/H.242, extended BAS commands, such as BAS attribute value "11"
]” Multi-byte extension command assigned to “C”
This can be done using various expansion commands such as CITT non-standard capability messages/command messages, or 14.
8.2 4Kbps data and 32Kbps simple video
ccIT as a protocol for LSD (low speed data) and MLP (data transmission using multi-layer protocol = audiographic conference [AGC]) referred to in 21.
14.4Kbps or use the BAS commands assigned to the T recommendation draft AV, 270, etc.
, 32Kbps, total 46.4Kbps, variable LSD/
14.4Kbps and 32Kbps by combining as a single data by variable MLP RAS command and by upper protocol
Various methods can be considered, such as dividing the data into Kbps and using it. At this time, as a result, the media multiplex format of each H, 221 frame becomes as shown in FIG. 4(C)/FIG. 4(e). In Fig. 4(c) and (e), 21 is shown in Fig. 4(c) and (e).
Same as a), 25 is 32Kbps simple video information, 26
indicates 14.4 Kbps of data, and 27 indicates 46.4 kbps of simple video information and data. For example, if the content of the switching request in step 5101 is a request to add data +4.4 Kbps and change audio information to 32 Kbps mode, the allocatable transmission speed for video information is 10 Kbps.
It becomes s. At this time, since the transmission rate is lower than the specified value X-Y, video information transmission using a still image is selected, and the process advances to step 5108. Step 5108
Then, it displays to the user whether preparations for importing the still image information to be transmitted are OK or not, and waits for an instruction. When transmitting a still image of a self-portrait, the user confirms it on the self-monitor and then instructs to capture the transmitted image. Next step S109
Proceed to step 5107 above, and data 1
RAS command and audio 32Kbps to allocate 4.4Kbps and still image 16Kbps to H, 221 frames
At the same time as sending the B'A S command to set the mode to s mode, transmission of each medium in the new mode is started.

At this time, if H.261 video transmission is on, it is turned off. At this time, as a result, the formats of multiplexing each media into 8.221 frames are as shown in FIGS. 4(d) and 4(f).

In Figures 4(d) and (f), 26 is the same as in Figure 4(c),
28 is 32Kbps audio information, 29 is 16Kbps still image information, 30 is 30.4K of still image information and data.
Indicates bps.

Next, in step 5101, it is assumed that a request is made to switch to the audio information l 6 Kbps mode and the data 6, 4 Kbps mode. At step 5102, it is found that the allocatable transmission rate for video information is 40Kbp5. In step 5104, this transmission rate is greater than or equal to X, so the process advances to step 5110. In step 5110, the H.261 video codec 102 (see FIG. 2) is configured with a transmission rate of 40 Kb.
Instructs to start transmission by ps and starts inputting video signals. Next, in step 5111, B
Upon transmission of the AS command, transmission of each media in the new mode is started. If H,261 video transmission is turned off at this time, H,261 video transmission is automatically performed in step 5112.
61 Send the RAS command to turn on video transmission, and
, 261 video transmission also resumes at the same time. In other words, even in cases where simple video information, still image information, etc. are multiplexed using an upper layer protocol such as LSD/MLP, it is assigned to video information when the 8.261 video transmission switch is turned on as described above. Keep in mind that video information transmission was switched to another video codec because the possible transmission speed became temporarily unsuitable, and the above LSD/MLP
By separately storing the assigned transmission speeds for video information and data that are multiplexed using upper-level protocols, etc., it is possible to accurately assign the actual video information when a mode switching request is made again. By calculating the transmission speed, you can appropriately determine which video codec to use.

Furthermore, if the mode switching request in step 5101 explicitly instructs to turn off the video, the video information is turned off in step 5103, so the process advances to step 5113, where the RAS command is sent and updated in response to the mode switching request. By simply starting each media transmission according to the mode, the above-described determination is not made even if there remains a transmission rate that can be allocated to video information.

In the above embodiment, the explanation was given only for the case of a single channel, but it can of course be applied to the case of multiple channels, and it can be applied not only when changing the mode due to a request from a user, but also when a line failure occurs when using multiple channels. It is also effective for A simple example is shown in FIG. 5(a) and FIG. 5(b). First, as shown in Figure 5(a), 2 of CHI and CH2
Using channels, 31 audio information 48kbps, 3
It is assumed that mutual communication was performed at 76.8 kbps for video information of No. 2. Here, if the second channel CH2 side is disconnected for some reason, only one channel of CHI can be used. Audio information is the same 48Kbp
s, only 14.4 kbps can be allocated to video information. This is the multiplexing example shown in FIG. 5(b). At this time, if the above explanation is applied, the video will be 14.
Still image information will be transmitted at 4 kbps. As a result, media is allocated to 8.221 frames as shown in FIG. 5(b). In Fig. 5(b), 31 is shown in Fig. 5(a).
), and 33 indicates 14.4 Kbps still image information.

In addition, in this embodiment, in accordance with the CCITT Recommendation/Recommendation Draft, the transmission speed is allocated to video information by subtracting the total transmission speed of the other two days from the total transmission speed of the available communications. We have been thinking about a method for automatically changing the encoding/decoding method for dream image information when priority is given to specifying the mode of other media, but conversely, we have considered a method that allows the user to specify the encoding/reprinting method for video information. It is also conceivable to use a means to control the designation of the media by setting the

(Effects of the Invention) As explained above, the present invention enables the encoding and decoding method of video information to be changed when the transmission rate that can be assigned to video information becomes lower than a certain specified value when video transmission is on. For example, by automatically switching according to the transmission speed that can be assigned to a method different from the current video codec,
It becomes possible to prevent inadvertent interruption of video information transmission, and it also becomes possible to automatically provide video information transmission appropriate to the transmission speed. In addition, if the user is allowed to select the image to be sent when switching to a still image, it would be possible to prevent blurred images like the frozen screen of the HO261 video image, and to prevent the sender from knowing at what moment the image was frozen by the other party. It is also possible to avoid this method.

Furthermore, the above state is automatically stored on the multimedia communication device side, and after the transition to the above state, when the transmission rate that can be assigned to the video information becomes higher than a certain specified value, the video information is encoded back to the original encoding. By returning to the decoding method, video information transmission using the optical encoding/decoding method can be resumed without requiring any special operation by the user.

[Brief explanation of the drawing]

FIG. 1 is a schematic block diagram of a multimedia communication device showing an embodiment of the present invention, FIG. 2 is an internal block diagram of a video encoding/decoding unit, and FIG. 3 is a CCITT Recommendation Draft H, 22.
The frame structure diagram shown in FIG. 1, FIGS. 4(a) to (f),
FIGS. 5(a) and 5(b) are structural diagrams showing an example of multimedia multiplexing within an 8.221 frame, and FIG. 6 is a flowchart for explaining the multiplexing operation. hand cent, 2... microphone, 3... speaker 4...
...Audio interface unit, 5...Audio encoding/decoding unit, 6...Camera, 7...Document camera, 8...Display unit 9 Video interface unit, 10...Video encoding/decoding unit, DESCRIPTION OF SYMBOLS 11... Data terminal, 12 Data interface section, 13... Operation section, 14... System control section, 15... Demultiplexing section, 16... Line interface section, 101... Transmission codec selection switch,
102.=-H, 261 video codec, 103...
JPEG video codec, 104... Original mode video codec, 105... Transmission codec selection switch,
106... Reception codec selection switch, 107...
- Reception codec selection switch, 108... video codec control unit.

Claims (1)

[Claims] 1. Multimedia information such as audio information, video information, code data information, etc. is multiplexed and communicated with each other via a communication line, and the audio information is encoded and decoded according to a specified transmission rate; Code data information is mutually communicated according to the specified transmission speed, and voice information and
In a multimedia communication device that encodes and decodes video information according to the remaining transmission speed after subtracting the transmission speed allocated to code data information, other control information, etc., the transmission speed of the video information is lower than a certain specified value. 1. A multimedia communication device characterized by having a changing means for changing an encoding/decoding method of video information when the video information becomes low. 2. The multimedia communication device according to claim 1,
A means for storing that a change in the encoding/decoding method of video information was made due to a decrease in the allocated transmission speed, and a means for storing the change in the video information as the original when the transmission speed that can be allocated to the video information again becomes higher than a predetermined value. What is claimed is: 1. A multimedia communication device characterized by comprising means for returning to an encoding/decoding method.