JPH04223755A - Multi-media terminal - Google Patents

Abstract

PURPOSE:To prevent a meaningless picture or a disturbed picture from being displayed in a display part. CONSTITUTION:A picture switching part 17 selecting either a received picture signal from an opposite terminal or a control picture signal from a control part 14 controlling a self terminal and outputting it to a picture output part 9a is given. When the control part 14 recognizes that picture data exists in communication data, the picture switching part 17 is caused to select the received picture signal and the display part 8 is caused to display it. When the control part recognizes that there is no picture data in communication data, the picture switching part 17 is caused to select the control picture signal and to display it in the display part 8.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia terminal that transmits and receives images and audio.

0002

BACKGROUND OF THE INVENTION In recent years, with advances in communication technology, semiconductor technology, and optical technology, digital lines have been developed and it has become possible to transmit large amounts of data at high speed.

[0003] In particular, the characteristics of digital transmission are that there is no quality deterioration due to transmission and the same level of quality is maintained;
There is no need for a transmission path that corresponds to the characteristics of the media used to transmit data, and media can be integrated, making it possible to transmit data between complex media terminals. Therefore, telephone terminals that can simultaneously transmit images from conventional voice-only telephones have appeared.

[0004] Under these circumstances, international standardization by CCITT and others is progressing to enable mutual communication even between different composite terminals.
The service specifications, protocol specifications, and multimedia multiplex frame configuration specifications for Visual) services are C
CITT Recommendation (or Draft) H. 320, H. 242,
H. It is announced as 221 etc.

[0005]H. 221, 64Kbps to 192
Frame structure and terminal capability exchange in AV services up to 0 Kbps, communication mode FAS (Frame A)
lignment Signal), BAS (Bit
The code allocation of AllocationSignal) is defined. H. H.242 defines protocols such as capability exchange and communication mode switching between AV terminals using BAS, and H.242 defines protocols such as capability exchange and communication mode switching between AV terminals using BAS. In 320, system aspects of the AV service in general are defined.

[0006] In the above recommendation or draft recommendation, after setting up an end-to-end physical connection and establishing synchronization by FAS in the in-channel,
A method for performing multimedia communication such as images, audio, and data between terminals is specified using procedures such as a sequence for exchanging terminal capabilities using , and a mode switching sequence by specifying a communication mode. However, how each terminal changes its own terminal capabilities depending on the situation and which communication mode to use within the range of exchanged capabilities is outside the scope of regulations. The information transfer rate of each media in multimedia communication is determined by specifying the audio encoding method for audio information, and by specifying whether or not to use it and the transfer rate if used. The transfer speed of image information is the remainder obtained by subtracting the transfer speed of audio information and the transfer speed of data information from the information transfer speed of the entire set communication channel.

[0007]

[Problems to be Solved by the Invention] However, according to the conventional multimedia terminals described above, when image data is communicated between terminals, normally when the terminals are connected to the other terminal,
The display screen will switch from the multimedia terminal operation screen display to the received image, but after that, unless the communication ends or the operator intentionally switches the screen,
Continue to display the received image. However, even if the other party's terminal stops sending image data during communication or switches to other media information, the screen cannot be changed, so the display screen continues to show meaningless images or Alternatively, there is a problem in that a distorted image is displayed.

The present invention has been made in view of the above circumstances, and its purpose is to provide a multimedia terminal that does not display meaningless or distorted images.

[0009]

[Means for Solving the Problems] In order to achieve the above object, the present invention provides a multimedia terminal equipped with an audio input/output means and an image input/output means, which controls a received image signal received from a partner terminal and the own terminal. an image switching section that selects one of the control image signals from the control section and outputs it to the image output section; means for causing the received image signal to be selected and displayed on the display section; and means for causing the image switching section to select the control image signal and display it on the display section when it is recognized that there is no image data in the communication data. This is how it was done.

0010

[Operation] In the multimedia terminal according to the present invention,
If it is recognized that there is image data in the communication data, the image switching section selects the received image signal and displays it on the display section.
If it is recognized that there is no image data in the communication data, the image switching unit selects the control image signal and displays it on the display unit. This prevents the display of meaningless or distorted images.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a multimedia terminal according to an embodiment of the present invention.

In the figure, 1 is a handset which is one of the voice input/output means, 2 is a microphone which is one of the voice input means, 3 is a speaker which is one of the voice output means, and 4 is a system control which will be described later. In response to instructions from the unit 14, the function of switching the handset 1, microphone 2, and speaker 3 as audio input/output means, the gain adjustment function of adjusting the gain to adjust the volume level, and detecting whether the handset 1 is in an on-hook state or an off-hook state. On/off detection function, echo cancellation function that eliminates echo when microphone 2 and speaker 3 are used as audio input/output means, dial tone/
a voice interface unit having a tone generation function for generating tones such as a ringing tone, a busy tone, and an incoming tone;
5 is 64Kbps according to instructions from the system control unit 14.
PCM (A-law), 64Kbps PCM (μ-
raw), 64Kbps/56Kbps/48Kbps
(SB-ADPCM), 32Kbps ADPCM,
Audio encoding that has the function of AD converting and encoding the transmitted audio signal according to audio encoding/decoding algorithms such as 16Kbps (for example, APC-AB) and 8Kbps, and the function of decoding and D/A converting the received audio signal. This is a decoding section. The voice encoding/decoding section 5 includes a voice encoding section 5a and a voice decoding section 5b.

6 is one of the image input means, and is a camera for inputting a self-portrait, etc.; 7 is one of the image input means, and is a document camera for inputting pictures, drawings, etc.; 8 is the camera 6 or A display section (display means) for displaying an input image from the document camera 7, an image received from the other party, and an image from the system control section 14; 9 is a switching function of the image input means caused by an instruction from the system control section 14, as described above; A video interface section 10 has display switching and display division functions for each image, and a signal conversion function for electrically/physically matching the image signals of each image input/output section;
D conversion and encoding function, decoding the received image and converting it to D/
The image encoding/decoding unit 10 is an image encoding/decoding unit having a function of performing A conversion, and the image encoding/decoding unit 10 performs motion compensation, frame dropping, interframe compensation and intraframe compensation, DCT transformation, Bandwidth compression is performed using various methods such as vector quantization conversion, reducing capacity and making it possible to transmit image data over digital lines. The video interface section 9 has an image input section 9a and an image output section 9b, and the image encoding/decoding section 10 has an image encoding section 10a and an image decoding section 10b. Currently, the basic interface of the ISDN line is 64Kbps, but the only image encoding method that can be transmitted even at this transmission speed is CCITT.
Draft Recommendation H. There are 261.

Reference numeral 11 denotes a data terminal for performing data communication; 12 notifies transmission data from the data terminal 11 and the system control unit 14 to the demultiplexing unit 15; and transmits received data to the data terminal 11 or the system control unit 14. 13 is a keyboard used for inputting control information to control the apparatus shown in FIG. 8;
Operation unit such as touch panel, 14 is CPU, ROM, RA
M, a system control unit that includes an auxiliary storage device, a character generator, an image signal generation circuit, etc., and monitors the status of each part, controls the entire device, creates operations/display screens according to the status, executes application programs, etc. , 15
are audio data from the audio encoding/decoding section 5, image data from the video encoding/decoding section 10, data from the data interface section 12, and BA from the system control section 14.
This is a demultiplexing unit that multiplexes the H. 221. Reference numeral 16 denotes a line interface section that controls the line according to the ISDN user network interface. 17 is an image switching circuit that selects the received image signal from the image decoding section 10b and the control image signal from the system control section 14 and transmits it to the image output section 9b; 18 is an image switching circuit that selects the received image signal from the image decoding section 10b and the control image signal from the system control section 14; This is a synchronization detection unit that detects the synchronization signal of and notifies the system control unit 14 of the synchronization signal. The synchronization detection unit 18 monitors the horizontal/vertical synchronization signal in the received image signal, and if a synchronization signal is generated, it is regarded as synchronization, and if the synchronization signal is not generated, it is regarded as out of synchronization, and the signal is sent to the system control unit 14. This is a notification to that effect.

If an error occurs during image decoding in the image decoding section 10b, the synchronization detection section 18 is activated due to the disturbance of the synchronization signal. Furthermore, when the image decoding section 10b detects an undecodable error that greatly affects the image during image decoding, it considers the error to be out of synchronization, and the image decoding section 10b detects this error in the synchronization detection section 18. There is also a method in which the synchronization detection section 18 determines that the synchronization is out of synchronization by notifying the synchronization as described above, and the synchronization detection section 18 communicates this to the system control section 14 .

Next, a method for negotiating terminal capabilities and changing terminal capabilities will be described. First, I.S.D.
When communicating over N lines, as shown in FIG. 2, a call is made using an outband signal (that is, D channel). As shown in FIG. 2, communication on the B channel becomes possible by setting up a call from terminal A to terminal B and responding from terminal B to terminal A. Although there are other communication channels such as D channel, H0, H1, etc., only the B channel will be explained and the following will be omitted.

Next, the recommendation H. According to H.242, the in-band signaling procedure of the B channel (that is, the B channel is allocated to a data section and a control section and controlled by the control section) is performed as shown in FIG. This control is called an in-channel control procedure, and this name will be used hereafter. To execute this in-channel control, a control bit is required in the B channel, so the frame structure is H. 221.

FIG. 4 shows this frame structure. The multiframe structure in FIG. 4 is for B channel (64 Kbps). First, the multi-frame structure is based on 1 octet/125 μsec, as shown in (a).
Frame = 80 octets, as shown in (b), 1 sub-multiframe = 2 frames, as shown in (c), 1 multiframe = 8 sub-multiframes, and in the bit direction, 8Kbps subchannel are defined from #1 to #8. However, only the #8 subchannel has a transfer rate of 6.4 Kbps and uses FAS (Frame Alignment Si) as a control bit.
gnal) and BAS (Bitrate Allocati
On Signal) is inserted. The FAS and BAS enable in-channel control of the B channel. FAS is used for frame and multiframe synchronization. The BAS is used for exchanging terminal capability information or capability setting necessary to determine a multiplexing method for subchannels, etc. In particular, BAS can be switched every sub-multiframe (20 msec) even during data communication. Octet data is transmitted in order from the lowest octet number.

Next, the procedure will be explained with reference to FIG. First, when the B channel becomes communicable, both terminals A and B transmit FAS. The terminal capability at this time is the initial mode 0 (audio, FAS, and BAS only mode). This FAS is searched by the partner terminal and H.
When the conditions for establishing frame synchronization specified in H.242 are met, A in the bit configuration in the FAS shown in FIG.
Set it to “0” and send. When the terminal receives A=0, it is confirmed that frame synchronization with the other terminal has been established. Next, the capability information of the own terminal is transmitted to the other terminal using the BAS (dotted line in FIG. 3), and both terminals recognize the terminal capabilities of the other terminal. If they can communicate with each other at this point, data communication will begin. If a capability change is required, similarly, the terminal capability is sent as a command using BAS, and after the other terminal completes setting the capability, data communication is started. Data communication is independent for sending and receiving.
Establishment of synchronization and setting of terminal capabilities are done separately. Therefore, synchronization may occur in only one direction, or the types of data may differ between sending and receiving. When the data communication is completed and the call is disconnected, the terminal on the disconnecting side (terminal A in FIG. 3) first uses the BAS to change to mode 0. As a result, B
In-channel control of the channel has returned to its initial state.

Next, as shown in FIG. 2, the D channel is disconnected and released in an out-of-band procedure to complete all communications.

FIGS. 5(a) to 5(c) show the bit configuration within the FAS. The A bit is an indication of out-of-frame synchronization. The E bit is an indication of whether a CRC error has occurred. C1, C2, C3, C4 are CRC 4
It's a bit. N1 to N5 are for multiframe numbering. R1 to R4 are channel numbers. T.E.
A is a terminal device alarm, which is set to "1" when the terminal is unable to receive or react to an input signal due to an internal failure. Note that the service channel in FIG. 5(a) is the eighth subchannel shown in FIG. 4(a).

FIGS. 6(a) and 6(b) show the bit configuration within the BAS. As shown in (a), the upper 3 bits represent an attribute, and the remaining 5 bits represent the attribute value of that attribute. (b) shows the contents of the attributes. For example, the attribute values are defined as transfer rate values, codec types, parameter values unique to each medium, and the like.

Next, the operational flow of one embodiment of the multimedia terminal according to the present invention is shown in FIGS. 7 to 10. First, select communication using a multimedia terminal or telephone (step S1), and in the case of telephone communication, connect the image switching unit (SW) 17 between C and B (steps S2, S3). , a telephone screen created by the system control unit 14 is displayed on the display unit 8 (step S4). For example, when making a call, the numeric keypad, phone book, or key-entered phone number will be displayed; when receiving a call, if the other party is known, the other party's phone number will be displayed; ,
Call time etc. will be displayed. The voice section (voice interface section 4 and voice encoding/decoding section 5) is activated and maintained until the call is disconnected (steps S5 to S7).
).

Next, in the case of communication using a multimedia terminal, the image switching section 17 is connected between C and B (step S8), and the display section 8 is provided with the image for the multimedia terminal created by the system control section 14. A screen is displayed (step S9). For example, when making a call, the transmission capacity settings used for communication and the numeric keypad, terminal number book, and keyed-in terminal number are displayed, just like a telephone call, and when a call is received, the capacity of the other party's terminal and the information being transmitted are displayed. If the capability display or the other party's terminal number is known, that terminal number is displayed. During ability negotiation by BAS (step S10), if audio is available as an ability, the audio section is activated (steps S11, S12),
Subtract the transfer rate A of the selected audio codec from the transfer rate X of the secured B channel (step S13)
, this transfer rate is Y, and if Y=0, it is determined that there is no data or image (steps S14, S15
). Therefore, in this case, the image switching unit 17 is
(step S16 in FIG. 9), and displays images from the system control unit 14. For example, the current transmission capability is displayed and the operator is informed that the image is not being transmitted. Alternatively, information on media other than moving images (drawings, still images, text data, etc.) is sent and received as data, and the system control unit 14 converts this data into an image and displays it. In the case of Y>0, if there is data as a capability, the data interface section (data section) 12 is activated (steps S17, S18), and the transfer rate B of the selected data is subtracted from the transfer rate Y. This transfer rate is set as Z (step S19), and if Z=0, it is determined that there is no image (step S20). Therefore, in this case, the image switching section 17 is connected between C and B (step S16), and the image from the system control section 14 is displayed. In the case of Z>0, if there is no image as a capability, the image switching section 17 is connected between C and B, and the image from the system control section 14 is displayed (steps S21, S16). If there is an image, the image section is started, and the synchronization detection section 18 detects synchronization/asynchronousness of the image signal using the transfer rate Z as the image transfer rate (step S22), and when the synchronization of the image signal is established. The system control unit 14 recognizes this, connects the image switching unit 17 between C and A (step S23), and transmits the other party's image from the image decoding unit 10b to the image output unit 9 and outputs it to the display unit 8. . Here, if Y<0 or Z<0, an error in capability selection is reported to the other terminal, and capability negotiation begins again (steps S27, S28). For example, there are methods such as returning to mode 0 and redoing the ability exchange. If the synchronization is out of synchronization, the image switching section 17 is connected between C and B, and the image created by the system control section 14 is displayed on the display section (step S24). For example, a message such as "Please wait for a while" may be displayed to notify the operator that the device is waiting to receive the other party's image. When there is a display switching instruction from the operation unit 13 by means such as key input (step S25), the system control unit 14 controls the image switching unit 17.
to the opposite side of the current switch position (step S
26). For example, if the connection between CA and A is established, it is switched to between C and B, and when the connection between C and B is connected, it is switched to between C and A.

Next, the case of changing the capability during communication will be explained. If the own terminal changes its capabilities (step S2
9) In response to an instruction to change the transmission ability from the operation unit 13,
The system control unit 14, via the demultiplexing unit 15,
The in-channel BAS of the channel notifies the partner terminal of the capability change (step S30). At the same time, the own terminal also changes its audio section, data section, image section (image input/output section 9 and image encoding/decoding section 10), and image switching section 17 in accordance with the change in capability (steps S11 to S34). In this case, when there is no voice, the transfer rate A=0, and when there is no data, the transfer rate B=0, and the process proceeds to step S21 (steps S31 to S34). When there is a change in capability from the other party's terminal, a command is received by the BAS, and the audio section, data section, image section, and image switching section 17 are changed accordingly.

Next, when communication ends (step S35)
) will be explained. When terminating from the own terminal (step S36), in response to an instruction from the operation unit 13, the system control unit 14 sends the destination terminal to mode 0 via the in-channel BAS of the B channel via the demultiplexing unit 15. (Step S37). At the same time, the own terminal is also set to mode 0 (step S37), the system control section 14 connects the image switching section 17 between C and B, and displays the image from the system control section 14 on the display section (step S39). For example, it may notify the operator that the line will be disconnected. After that, it sends a disconnection message from its own terminal (step S
40), receiving a release from the other party's terminal and terminating the communication. At this time, communication time, charges, etc. are displayed, and when preparations for the next communication are completed, the initial screen returns. When ending from the other party's terminal (step S36), the system controller 14 receives a notification of the capability change to mode 0 from the BAS (step S38), and connects the image switching unit 17 between C and B (step S38). S39), the image from the system control unit is displayed on the display unit. For example, it may notify the operator that the line will be disconnected. After that, the line is disconnected (step S4
0) When preparation for the next communication is completed, the system control unit 14 displays an initial screen on the display unit.

As explained above, according to this embodiment, in a multimedia terminal equipped with an audio input/output means, an image input/output means, and a data input/output means, the received image signal received from the partner terminal and the own terminal are controlled. an image switching section that selects one of the control image signals from the control section and outputs it to the image output section; means for causing the received image signal to be selected and displayed on the display section; and means for causing the image switching section to select the control image signal and display it on the display section when it is recognized that there is no image data in the communication data. By doing so, it is possible to recognize whether or not there is image data in the data received from the other party's terminal, and depending on the result, the display automatically switches between the received image and the image from the system control unit. This eliminates the need for the operator to manually switch between the above screens while looking at the display screen, or the troublesome effort of switching while checking with the other party.
This has the effect of improving the convenience of the interface.

[0029] Furthermore, when there is no received image, an image from the system control unit is displayed on the screen to visually inform the operator that the other party's terminal has changed its transmission data capability and that image data is no longer available. This has the effect of making it easier and more accurate to understand the situation.

[0030]

As described above, according to the present invention, if there is a received image, the received image is displayed, and if there is no received image, the image from the system control unit is displayed, making it easier to understand the situation. It is possible to prevent meaningless images and distorted images from being displayed.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a multimedia terminal according to an embodiment of the present invention.

FIG. 2 is a sequence diagram showing a D channel procedure.

FIG. 3 is a sequence diagram showing an in-channel control procedure.

FIG. 4 is a frame configuration diagram.

FIG. 5 is a diagram of the bit configuration in FAS.

FIG. 6 is a diagram of the bit configuration within the BAS.

FIG. 7 is a flowchart illustrating the operation of an embodiment of a multimedia terminal according to the present invention.

FIG. 8 is a flowchart illustrating the operation of an embodiment of a multimedia terminal according to the present invention.

FIG. 9 is a flowchart illustrating the operation of an embodiment of a multimedia terminal according to the present invention.

FIG. 10 is a flowchart illustrating the operation of an embodiment of a multimedia terminal according to the present invention. 1 Handset 2 Microphone 3 Speaker 4 Audio interface section 5 Audio encoding/decoding section 5a Audio encoding section 5b Audio decoding section 6 Camera 7 Document camera 8 Display section 9 Video interface section 9a Image input section 9b Image output section 10 Image code Image decoding unit 10a Image encoding unit 10b Image decoding unit 11 Data terminal 12 Data interface unit 13 Operation unit 14 System control unit 15 Demultiplexing unit 16 Line interface unit 17 Image switching unit 18 Synchronization detection unit

Claims (1)

[Claims] Claim 1: A multimedia terminal equipped with audio input/output means and image input/output means, which selects either a received image signal received from a partner terminal or a control image signal from a control unit that controls the own terminal. an image switching unit for outputting to an image output unit; the control unit having means for causing the image switching unit to select the received image signal and display it on the display unit when recognizing that there is image data in the communication data; . A multimedia terminal comprising means for causing the image switching unit to select the control image signal and display it on a display unit when it is recognized that there is no image data in the communication data.