JPH09282133A - Video communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent other applications which are executed at the same time from being disturbed by performing control so that an image is displayed large at the start of a communication and automatically reduced in size several seconds later when the object of the image is the face of an opposite person. SOLUTION: To start a real-time communication, current time Tnow is obtained and stored as start time Tstart. Then the size of a display image is set to large size, which is reported to a decoding process part 407. Then a receiving process part 404 receives one packet to acquire the current time Tnow. When the set display image size is large, the display image size is set to small size and reported to the decoding process part 407 if the difference between the acquired current time Tnon and stored start time Tstart is larger than a previously given threshold value. In this manner, an image received from the opposite party is displayed large at the start of the communication, but automatically becomes small several seconds later.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication system, and more particularly, to a communication system that handles multimedia data with a partner via a communication network and performs communication in real time.

[0002]

2. Description of the Related Art In real-time video communication such as a videophone, the main purpose is to see the face of the other party even in a remote place, and the user's operation is required to change the size of the video. there were.

[0003]

Conventionally, the main purpose of real-time interactive video communication such as that of a TV phone has been to see the face of the other party. In recent years, the personal computer is preparing an environment where it can collaborate with a partner in a remote place while exchanging data through a communication network, and the purpose of video communication is to see the face of the partner. It has been expanded to video support for collaborative work. However, in the conventional technique, for example, when using a shared blackboard (where a communicator can write while watching the same screen on a computer), a large meeting material can be displayed on a display that is not so large. In order to display the image, the user performs an operation of changing the image size to reduce the size of the screen showing the face of the other party.

A first object of the present invention is to provide a video communication system which does not disturb an application such as a collaborative work where the screen is widely used when the target of the video is the face of the other party. It is in. Further, the second aspect of the present invention
The purpose of is when the processing load of a certain application is very high, or when the processing load of a communication terminal becomes high by executing multiple applications simultaneously,
An object is to provide a video communication system capable of reducing the load.

[0005]

In order to solve the above-mentioned problems, according to the first aspect of the present invention, when the target of the image is the face of the other party, the image is displayed in a large size for several seconds at the start of communication. After that, a video communication system having a function of automatically reducing the video size so as not to interfere with one or more other applications simultaneously executed is provided.

In a second aspect, by observing the CPU load of the communication terminal, a function of suppressing the loss of real-time property due to the processing load of the communication terminal itself and the influence on one or more other applications being executed at the same time is provided. Provide a video communication system that has.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, a first embodiment of the present invention will be described with reference to FIGS.

FIG. 1 shows the configuration of a communication system according to an embodiment of the present invention. Reference numeral 101 is a packet communication network, and 102 is a communication terminal connected to the packet communication network 101. Communication terminal 1
Between the two, real time communication can be performed via the packet communication network 101. FIG. 2 shows the configuration of the communication terminal 102. Reference numeral 201 is an input device for inputting media information such as voice, video and data, 202 is an output device for outputting media information such as voice, video and data, 203 is a memory, 2
Reference numeral 04 is a communication network interface control unit for performing interface processing of the communication network, and 205 is C for performing communication control of each media data such as data communication, voice communication, and video communication.
PU (Central Processing Unit), 206 is a storage device, and 207 is a communication terminal internal bus.

FIG. 3 shows the configuration of the multimedia communication control unit in the communication terminal 102. This multimedia communication control unit is a communication program 3 stored in the memory 203.
01 to 305, and a CPU 205 that executes these communication programs. 301 manages communication such as simultaneous communication and switching of each media data,
A communication management unit for inputting / outputting data to / from the communication network interface control unit 204, a data communication control unit 302,
Is a voice communication control unit that controls voice communication, 304 is a video communication control unit that controls video communication, and 305 is another media communication control that controls communication of other media, such as a mail system and a FAX system. It is a department.

FIG. 4 shows a detailed configuration of the video communication control unit 304 shown in FIG. Reference numeral 401 denotes an encoding processing unit which digitizes and compresses video information input using the input device 201.
Reference numeral 02 designates a coding rate to instruct the coding processing unit 401, analyzes the information contained in the header of the received packet, and prompts the related processing unit to perform processing.
Reference numeral 03 is a transmission processing unit that packetizes the video data encoded by the encoding processing unit 401 and performs a transmission process to a partner terminal that performs real-time communication, and 404 is a communication management unit 301.
A reception processing unit that decomposes video packets received through
405 is a screen control unit that controls the screen for displaying the received video data on the output device 202, 406 is a current time acquisition unit that acquires the current time, and 407 is the screen control unit 4.
05 is a decoding processing unit that decodes the video information passed from 05 and outputs it to the output device 202.

FIG. 5 shows a detailed configuration of the screen control unit 405. Reference numeral 501 is a start time storage unit that stores the time acquired from the current time acquisition unit at the start of real-time communication, 502 is a comparison unit that compares the time stored in 501 with the current time acquired by 406, and 503 is a comparison of 502. A screen size determination unit that determines the display size of the received video data from the result, and 504 is an image data storage buffer that receives the video data from the reception processing unit 404 and temporarily stores the data until one frame of data is ready.

FIG. 6 shows an example of the overall flow of the reception processing of real-time video communication. When real-time video communication is started in step 601, the current time Tonow is acquired in step 602 and stored as the start time Tstart in step 603. In step 604, the front image size is set to a large size, and in step 605, the value set in 604 is notified to the decoding processing unit 407. In step 606, the reception processing unit 4
One packet is received at 04, and the current time T is received at step 607.
get now If the display image size set in step 608 is large, then in step 609 step 60
If the difference between the current time Tnow acquired in step 7 and the start time Tstart stored in step 603 is larger than a threshold value (image size change time) given in advance, step 6
The display image size is set to small in 10 and the decoding processing unit 407 is notified in step 611. If the display image size set in step 608 is small and step 60
If Tstart-Tnow is less than the image resize time in 9, the process proceeds to step 612. When the video data for one frame is prepared in step 612, the video data prepared for one frame is passed to the decoding processing unit 407 in step 614, and one frame image is output to the output device 202 in step 615. If one frame of data is not available in step 612, packets are received until one frame of data is available in step 613. If the real-time video communication is to be ended in step 616, this processing is ended. If not, the process returns to step 606 to repeat the series of processes.

According to the present embodiment, when real-time video communication is performed, a video communication system in which the video received from the other party is displayed in a large size at the start of communication but automatically decreases after a few seconds becomes possible.

Next, a second embodiment of the present invention will be described with reference to FIGS. In addition to the above-described first embodiment, the present embodiment is an example of a video communication system in which the user can turn off the automatic control of the display video size by setting the display video size to a fixed value in advance. . The overall configuration, the configuration of various communication control units, the incoming call processing, and the like are the same as those in FIGS. 1 to 4 and 6 of the first embodiment described above.
Only different parts will be described below.

FIG. 7 shows the configuration of the screen control unit in this embodiment (corresponding to FIG. 5 of the first embodiment). The same parts as those in FIG. 5 are designated by the same reference numerals, and the description thereof will be omitted. 7
Reference numeral 01 denotes a display video size automatic control mode setting unit (default is automatic control mode) which receives an event input from the user and sets whether or not to automatically control the display size.

FIG. 8 shows an example of the overall flow of processing on the receiving side when performing real-time video communication (corresponding to FIG. 6 of the first embodiment). When the user sets the display image size to the fixed mode and turns off the automatic control in step 801,
The process proceeds to step 613 without entering the automatic control processing routine.

According to this embodiment, when the user sets the display size of the video received from the communication partner at the start of the real-time video communication to be fixed, it is possible to realize a video communication system in which automatic control is disabled. Become.

Next, a third embodiment of the present invention will be described with reference to FIGS. In addition to the first and second embodiments described above, the present embodiment can display not only the image transferred from the communication partner but also the own image, and display the own image after a certain time after the start of communication. It is an example of a video communication system that automatically erases. The overall configuration, the configuration of various communication control units, the incoming call processing, and the like are the same as those in FIGS. 1 to 8 of the first and second embodiments. Only different parts will be described below.

FIG. 9 shows the structure of the screen control unit in this embodiment (corresponding to FIGS. 5 and 7). Reference numeral 901 denotes a self-portrait display mode setting unit that receives an event from the user and sets whether or not to display a self-portrait. Reference numeral 902 denotes an image input from a device (camera or the like) of the input device 201 that inputs a video to the output device 202. It is a self-portrait display processing unit that performs a process of displaying and outputting.

FIG. 10 shows an example of the overall flow of processing when performing real-time video communication. Step 1001
If real-time video communication is started in step 1,
In 002, it is determined whether the user has set the self-portrait display mode. When the self-portrait display mode is set, the current time Tnow is acquired in step 1003, and the current time T acquired in step 1003 in step 1004.
Store now as the start time Tstart. Step 100
In step 5, the video data of the user input from the input device 201 is passed to the self-image display processing unit 902. In step 1006, the current time Tonow is acquired, and in step 1007, step 10 is executed.
03 If the difference between the acquired current time Tnow and the start time Tstart stored in step 1004 is larger than a threshold value (self-portrait display time) given in advance, step 100
Use 8 to erase your video display. T at step 1007
If now-Tstart <threshold value, in step 1009, the video data passed to the self-portrait display processing unit in step 1005 is displayed. When the user turns off the self-portrait display mode in step 1002, the processes of steps 6102 to 615 are repeated without performing the self-portrait display processing routine. If the real-time video communication is to be ended in step 1010, this processing is ended. If the real-time video communication is not ended, the process returns to step 1002,
Repeat a series of processes.

According to the present embodiment, if the user is set to display his / her own video during real-time video communication, his / her own video can also be displayed, and the appearance condition can be confirmed. A video communication system becomes possible.

Next, a fourth embodiment of the present invention will be described with reference to FIGS. 11 to 13. This embodiment is the same as the above-mentioned third.
In addition to the above embodiment, this is an example of a video communication system in which the automatic change of the display screen size and the automatic deletion are turned off when the image transferred from the communication partner is to be widely displayed on the screen such as a document. The overall configuration, the configuration of various communication control units, the processing of outgoing and incoming calls, and the like are the same as those in FIGS. 1 to 10 of the above-described first to third embodiments. Only different parts will be described below.

FIG. 11 shows an example of the format of a packet transmitted / received when performing real-time video communication.
1101 is a field indicating whether the packet is a control command or data, audio or video data,
1102 is a field 1101 that indicates whether the data is audio, video, or data when the packet is media data. 1103 is normal data, or video data and a document. A field for indicating whether the image is reflected, 1104
Is a field indicating the sequence number of the packet after the real-time video communication is started, 1105 is a field indicating the frame number of the video frame after the real-time video communication is started when the packet data is video data, 1106 is A field indicating the packet length of the packet, 1107 is a field indicating video coding information (for example, a compression method or a video size) when the packet data is video data, 1108 is a field indicating packet transmission time information, and 1109 is This is a field showing the body of Medidata.

FIG. 12 shows an example of the overall flow of processing on the transmitting side when performing real-time video communication. When the user sets the document mode in step 1201, the self-portrait size is set to large in step 1202 and step 12
In 03, the self-portrait data captured from the input device 201 is passed to the display processing unit 902 and the encoding processing unit 401, and in step 1204, a header is set to 1103 indicating that the video data is data representing a document, and a packet is added. Assemble and send the packet in step 1205. If the user does not set the document mode in step 1201,
The processing after step 1003 is performed.

FIG. 13 shows an example of the overall flow of processing on the receiving side when performing real-time video communication. If the field 1103 indicating the normal data or the document data in the header of the received packet is turned on in step 1301, the display image size is set to the large and fixed modes in step 1302, and steps 611 to 616 are performed.
Is processed. If the document mode is off in step 1302, steps 607 to 616 are performed.

According to the present embodiment, when the image data to be displayed during real-time image communication is data such as a document which is desired to be displayed large, the image communication can be displayed large without automatically controlling the image size. The system becomes possible.

Next, a fifth embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. In the present embodiment, when another application (task) is operating during real-time video communication, the screen size displayed by another task is detected, and there is a task that widely uses the screen. Is an example of a video communication system for reducing the display video size.

FIG. 14 shows the configuration of the task management unit in this embodiment. A task management unit 1400 observes a task that is currently being executed. 1401 is a timer, 1402 is a task acquisition unit that detects whether or not another task is activated, and acquires information for identifying a task when a new task is detected, 1403 is acquired by the task acquisition unit 1402. The screen size acquisition unit that acquires the size of the screen displayed by the selected task, 1404 is a registration unit that registers the task information acquired by the task acquisition unit 1402 and the screen size acquired by the screen size acquisition unit 1403, and 1405 is registration It is a task management table for recording registration data from a department.

FIG. 15 shows an example of the overall flow of processing on the receiving side when performing real-time video communication. When real-time video communication is started in step 1501, the video size is set to large in step 1502, and step 1
At 503, the timer is set. If a timer interrupt is generated in step 1504, the current time is acquired in step 1505, and the task currently being executed is acquired by the task acquisition unit 1402 in step 1506. Step 15
In step 07, the task acquired in step 1506 is searched for in the task management table 1405, and if not registered, the screen size acquisition unit 1403 acquires the screen size displayed by the task acquired in step 1508. , Task in step 1509 and step 1505
The current time obtained in step 3 is registered in the task management table 1405. If task information is already registered in the task management table 1405 in step 1507, step 151
In step 4, the current time obtained in step 1505 is updated in the task management table 1405. In step 1510, the display screen size of the task is compared with a threshold value given in advance,
If it is larger than the threshold, the display image size of this communication task is set to small in step 1511, and step 151
In 3, the video size is passed to the decoding processing unit 407. If the display screen size of the task is smaller than the threshold value in step 1510, the image size is set large in step 1512 and the process of step 1513 is performed. Step 151
In step 5, it is determined whether or not the video data for one frame has been prepared, and reception of one packet is repeated in step 1516 until the data is prepared. When the video data for one frame is complete, the video data is passed to the decoding processing unit 407 in step 1517, and the video is displayed on the output device 202 in step 1518. When terminating the real-time video communication in step 1519, it is determined in step 1520 that the task information having the oldest time information among the task information in the task management table 1405 has already been terminated, and the task information is set as the task information. It is deleted from the management table 1405. If the real-time video communication is not ended in step 1519, the process returns to step 1504 to repeat the series of processes.

According to the present embodiment, when an application that widely uses the screen, such as a word processor or a spreadsheet application, is activated during real-time video communication, the display video size is automatically reduced so that the other application is It enables a video communication system that can be kept out of the way of use.

Next, a sixth embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. In addition to the first embodiment described above, the present embodiment is an example of a video communication system that automatically reduces the display video size when the CPU load of the communication terminal is high.

FIG. 16 shows the reception processing unit 40 in this embodiment.
4 is shown. Reference numeral 1601 denotes a reception waiting loop that performs processing for waiting reception of packet data transferred from a partner of real-time video communication, 1602 a packet decomposition unit that decomposes the packet data received in 1601 into header and video data, and 1603 controls 1601. A counter that counts when it arrives, 1604 is a current time acquisition unit, 1605 is a loop cycle calculation unit that calculates an interval at which control is passed to the 1601 data reception waiting loop from the current time information acquired by the counters 1603 and 1604, 1
A comparison unit 606 compares the calculation result of the loop period calculation unit 1605 with the coding rate passed from the communication control unit 402.

FIG. 17 shows an example of the overall flow of processing on the receiving side when performing real-time video communication. When real-time video communication is started in step 1701, the video size is set large in step 1702. When control is passed to the data reception waiting loop 1601 in step 1703, and when a packet is received in step 1704, the received packet is decomposed into a header and video data in step 1705. In step 1706, it is determined whether or not counter ≠ 0. If counter = 0, step 1
At 707, the current time is obtained and stored as T0.
If the counter is incremented in step 1708 and counter = n (given to the system in advance) in step 1709, the current time is acquired in step 1710 and T
In step 1711, the reception waiting loop period is calculated from the difference between the time T0 stored in step 1707 and Tn acquired in step 1710, and the counter number n.
In step 1712, the reception waiting loop cycle calculated in step 1709 is compared with the necessary loop cycle (set as a threshold) calculated in advance from the coding rate and the packet size. If the reception waiting loop cycle is larger than the threshold, In step 1713, the video size is set to small, and in step 1714 the real-time video communication partner is notified that the video size is changed to small. In step 1715, the video size is passed to the decoding processing unit 407, and in step 1716
Then, it is determined whether or not the video data for one frame has been obtained. If so, the video data is passed to the decoding processing unit 407 in step 1717, and is output to the output device 202 in step 1718. If the video data for one frame is not available in step 1716, step 17
Returning to 03, a series of processing is repeated. Step 1719
If the real-time video communication is to be ended, the present process is ended.

According to this embodiment, one or more applications can be operated at the same time during real-time video communication.
Alternatively, a certain application performs processing that occupies the CPU, and the processing load on its own terminal increases, causing a phenomenon in which the CPU performance required for video communication cannot be ensured and packet data cannot be received completely. In this case, it is possible to provide a video communication system capable of informing the communication partner to reduce the video size and suppressing the loss of the real-time property of the video data and the influence on other applications as much as possible.

[0035]

According to the present invention, when real-time video communication is performed, the video transmitted from the other party is displayed large at the start of communication, but after a few seconds, the video is automatically changed to a small size so that the video collaborates. Can be out of the way.

According to the second embodiment of the present invention, by setting the display image size to be fixed in advance, it is possible to support communication while looking at the face of the other party.

According to the third embodiment of the present invention, when the real-time video communication is started, not only the video transferred from the other party but also the video of the user is displayed to adjust the appearance of the camera of the user. be able to.

According to the fourth embodiment of the present invention, in order to convey the shape of a document or an object to be imaged, if the image is desired to be displayed with high definition and large size, the automatic control of the image display size is not performed. You can

According to the fifth embodiment of the present invention, the display image size is automatically reduced when an application that uses a wide screen is activated during real-time image communication.
The user's work can be prevented from being disturbed.

According to the sixth embodiment of the present invention, the image size is automatically reduced when the CPU load of the terminal itself becomes high due to a plurality of applications being activated during real-time image communication. Then, the influence on other applications can be suppressed.

[Brief description of drawings]

FIG. 1 is a block diagram of a communication system for implementing the present invention.

FIG. 2 is a block diagram of a communication terminal for implementing the present invention.

FIG. 3 is a block diagram of a multimedia communication control unit.

FIG. 4 is a block diagram of a video communication control unit in FIG.

5 is a block diagram of the screen control unit of FIG.

FIG. 6 is a flowchart of processing on the receiving side during real-time video communication.

7 is a block diagram when the screen control unit in FIG. 4 receives a setting of a display video size from a user.

FIG. 8 is a flowchart of processing when a user sets a display video size mode during real-time video communication.

9 is a block diagram of the screen control unit shown in FIG. 4 in which the user's video is also displayed.

FIG. 10 is a flowchart of a process for displaying the user's video at the start of real-time video communication.

FIG. 11 is an explanatory diagram showing an example of a video packet format.

FIG. 12 is a flowchart of processing on the transmitting side when video data to be sent to the other party is a document during real-time video communication.

FIG. 13 is a flowchart of processing on the receiving side when the video data sent from the other party is a document during real-time video communication.

FIG. 14 is a block diagram of a task management unit.

FIG. 15 is a flowchart of processing on the reception side when another application that widely uses a screen is activated during real-time video communication.

FIG. 16 is a block diagram of a reception processing unit when observing the CPU load of the terminal itself.

FIG. 17: CPU of own terminal during real-time video communication
The flowchart of the process of the receiving side when load becomes high.

[Explanation of symbols]

101 ... Packet communication network, 301 ... Multimedia communication control unit, 304 ... Video communication control unit, 402 ... Communication control unit, 405 ... Screen control unit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshiaki Koyama 810 Shimoimazumi, Ebina City, Kanagawa Prefecture Hitachi Systems Co., Ltd. Office Systems Division

Claims (7)

[Claims] 1. A communication system comprising a communication network for transmitting and receiving digital data, and a communication terminal connected to the communication network, wherein the communication terminal is an input device and an encoding processing unit for encoding input information. A transmission processing unit that packetizes and transmits the encoded information, a reception processing unit that receives packet information from the communication network, and manages communication conditions to the transmission processing unit and the reception processing unit. A communication control unit for performing communication control, a decoding processing unit for decoding encoded information,
An output device that outputs the decoded information, and a screen control unit that manages and controls the screen when outputting the video data transferred from the communication partner or the video data input from the input device to the output device The communication terminal decomposes the received video packet into encoded data in the reception processing unit, a unit for obtaining the current time, and a unit for measuring an elapsed time from the start of the real-time video communication, A means for comparing the elapsed time obtained by the measuring means with a threshold value given in advance; and a means for reducing the display size of the video data passed from the reception processing unit as a result of the comparing means. Characteristic video communication system. 2. The video communication system according to claim 1, wherein the communication terminal includes means for setting a display video size to be fixed. 3. The communication terminal displays not only the image received from the other party but also the image input from the input device of the own terminal, and the unit displaying the own image at the start of the real-time video communication. The video communication system according to claim 1 or 2, further comprising: a unit that erases its own image as a result of the comparison by the comparison unit. 4. The mode in which the communication terminal inputs data to be input when the image input from the input device of the terminal is data for which a large screen is to be displayed such as the shape of a document or an object. 4. The video communication system according to claim 3, further comprising: means for setting the display size of the video data of the user when the mode is set, and means for not deleting the video display. 5. The video communication system according to claim 3, wherein said communication terminal comprises means for turning off automatic control of the display video size when the video from the real-time video communication partner is a document. 6. The communication terminal has a task management unit for observing an application running on the own terminal, means for acquiring a task running on the own terminal during real-time video communication, and the means for acquiring the task. Means for acquiring the size of the screen displayed by the task, means for registering the task and screen size acquired by the means in the task management table, and comparison of the screen size acquired by the means with a threshold value given in advance Means, a means for reducing the image display size when the task is determined to widely use the screen by the comparing means, and task data that has been completed among the tasks registered in the task management table. The video communication system according to claim 1, further comprising means for erasing. 7. The communication terminal comprises means for observing the processing load of the terminal itself during real-time video communication, and means for comparing the processing load observed by the means with the current coding rate of video communication. The reception processing unit comprises means for notifying a communication partner to reduce the video size when it is determined by the comparison means that the processing load in the video communication needs to be reduced. The described video communication system.