JP2000059772A - Video displaying system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video displaying system for automatically compressing a displaying video or varying a volume at outputting of voice, in matching with the situation of a user. SOLUTION: A distance between the display part 102 of a terminal on a receiving side 100 and a user is measured by a distance measuring sensor 101 to vary the displaying image quality of the part 102 or the volume of voice outputted from a load speaker 103 corresponding to the distance, thereby as a result the volume and screen display according to the situation of the user is realized and the transmission quantity of video data sent from a transmission side is reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information terminal such as a videophone or a multimedia personal computer, which automatically converts a video and a sound transmitted via a communication line into a display video according to a user's operation status. The present invention relates to a video display method for compressing or changing the volume at the time of audio output.

[0002]

2. Description of the Related Art Conventionally, in a portable terminal such as a personal computer having a display unit and an audio output unit, when displaying a transmitted image, even if a user looks away at the display screen with his / her face away from the display screen, It is displayed with a fixed number of pixels. Also, when outputting sound, the sound is output at the initially set volume.

[0003]

However, in the state of looking at the display screen, the time for seriously looking at the contents is not so long unless the displayed contents are important. For example, in a videophone or the like, it is rare to observe the other party's face in detail for a long time. After a certain amount of time, the user often turns his / her face off the screen and looks at the displayed content vaguely.

In such a state, more attention is paid not to overlook the voice uttered by the other party than to look at the display screen. However, when the face is separated from the screen, the distance between the face and the screen becomes longer, so that the voice becomes smaller and it becomes difficult to hear. Therefore, it has been practiced to manually operate the volume control on the device side to increase the volume and make it easier to listen.

[0005] Further, whether the user is looking at the display screen at random or seriously, the screen is displayed with high definition, the transfer rate of the video sent from the transmission side does not change, and excessive video transmission is performed. Have been done.

The present invention has been made in view of such a conventional problem, and automatically adjusts to a user's situation.
It is an object of the present invention to provide a video display method for compressing a display video or changing a volume at the time of audio output.

[0007]

According to the present invention, there is provided a video display system comprising: a compression means for compressing a video and an audio; a display means for expanding a video compressed by the compression means and displaying the video on a display screen; Means for expanding and outputting the sound compressed by the means, means for determining the distance between the user facing the display screen and the display screen, and compression and output of the video according to the distance determined by the means. Means for changing at least one of the volume of the sound to be reproduced.

With such a configuration, it is possible to control the compression ratio of the video and the volume of the output audio according to the situation of the user. If video and audio are transmitted via a communication line, and if the communication line provides communication services to multiple users over multiple channels, the transmission capacity per person decreases. , It is possible to provide services for more users.

Here, means for stopping the operation of at least one of the display means and the sound output means when the distance becomes longer than a predetermined length may be provided. Further, the video display system according to the present invention includes a means for compressing video and audio, a transmission means for transmitting the video and audio compressed by this means via a multi-channel communication line, and a transmission means for transmitting the video and audio. Display means for expanding the compressed video and displaying it on a display screen, audio output means for expanding and outputting the compressed audio transmitted by the transmission means, a user facing the display screen, and the display screen Means for determining the distance between the video signal and the means for changing at least one of the video compression rate and the volume of the output audio in accordance with the distance determined by the means. And control means for controlling transmission of video with the temporarily secured communication capacity when the increased communication capacity cannot be secured.

Further, according to the video display system of the present invention, there are provided means for compressing video and audio, transmission means for transmitting the video and audio compressed by this means via a multi-channel communication line, and this transmission means. Display means for expanding the compressed video transmitted by the transmission means and displaying the same on a display screen; audio output means for expanding and outputting the compressed audio transmitted by the transmission means; and a user facing the display screen. Means for determining the distance from the display screen, means for changing at least one of the video compression ratio and the volume of the output audio according to the distance determined by the means, and compression of the video on a plurality of channels in the communication line. Means for securing a communication capacity by assigning a priority according to the distance between a plurality of channels when the rate decreases and a predetermined increase in the communication capacity cannot be secured.

The means for determining the distance between the user facing the display screen and the display screen includes: means for capturing an image including the user's face; means for extracting a face area from the image captured by the means; Means for calculating the distance to the user from the area of the face region extracted by this means.

[0012]

Embodiments of the present invention will be described below in detail with reference to the drawings. In the following drawings, the same symbols indicate the same or corresponding parts. First, the configuration and operation of a receiving terminal and a transmitting terminal common to Embodiments 1 and 2 described below will be described.

FIG. 1 is an external view of a receiving terminal 100 according to the first and second embodiments. Receiving terminal 100
May be connected to a terminal on the transmitting side, such as a personal computer, a mobile terminal, or a videophone, and may be of any type as long as it relates to the purpose of displaying an image transmitted from the transmitting side and outputting audio.

As shown in FIG.
A distance measurement sensor 101, a display unit 102, and a speaker 103 are attached. When the user faces his / her face in front of the receiving side terminal 100 and looks at the display unit 102, the distance measuring sensor 101 outputs infrared rays or ultrasonic waves, and captures infrared rays or ultrasonic waves reflected by the user's face. Then, based on the time difference between output and reception, the display unit 1
02 and the distance between the user's face.

FIG. 2 is a block diagram showing a circuit configuration of the receiving terminal 100 and the transmitting terminal 112 in the first and second embodiments. First, the receiving side terminal 100 will be described. Reference numeral 105 denotes an audio decoding unit for expanding compressed audio, 104 denotes an audio output unit for outputting expanded and reproduced audio, and audio is output from the speaker 103. . Reference numeral 107 denotes a video decoding unit that expands the compressed video and reproduces the original video.
Is displayed with. Reference numeral 106 denotes a control unit A which is configured by a microcomputer or the like and controls the entire receiving terminal 100. Reference numeral 108 denotes a demultiplexing unit that separates a video signal and an audio signal from the multiplexed video signal and audio signal transmitted from the transmitting terminal 112. Reference numeral 110 denotes a receiving unit A that receives the multiplexed video signal and audio signal. 109 is the receiving terminal 100
Is a transmission unit A for transmitting control information to the transmission side terminal 112 from the transmission side.

The receiving terminal 100 and the transmitting terminal 112 are connected by a communication line 111, and a signal in which a video signal and an audio signal are multiplexed and control information are mutually transmitted. Next, the configuration of the transmitting terminal 112 will be described. 118
Is a microphone for inputting voice. Reference numeral 117 denotes an audio encoding unit that compresses input audio. Reference numeral 120 denotes an image input unit for inputting an image such as a television camera. Reference numeral 121 denotes a motion detection unit that observes motion of a video input from the video input unit 120 over a plurality of frames and outputs an evaluation value. Reference numeral 116 denotes a control unit B which is configured by a microcomputer or the like and controls the entire transmission side terminal 112. A multiplexing unit 115 multiplexes the compressed audio and video. 114
Is a transmission unit B for transmitting multiplexed audio and video. 1
Reference numeral 13 denotes a receiving unit B that receives control information transmitted from the receiving terminal 100.

FIG. 3 is a block diagram showing a detailed circuit configuration of the video encoder 119 shown in FIG. Note that the video encoding unit 119 according to the present embodiment can select an INTER mode in which a difference from a previous frame (prediction value) is encoded for each screen, or an INTRA mode in which encoding is performed in the screen without taking a difference. . For example, for video or still images with little motion, IN
The TER mode is used, and the INTRA mode is used for a scene change even for a video having a large motion or a video having a small motion.

Also, the quantization step size is changed according to the amount of coded data to be generated, and frame skipping (frame skipping) is performed if necessary. In FIG.
Reference numeral 4 denotes an encoding mode selection circuit for selecting an INTRA mode or an INTER mode according to an energy comparison result between a pixel value from the video input unit 120 and a prediction error of the pixel value and an external control signal. In the INTRA mode, the encoding mode selection circuit 124 outputs the pixel value from the video input unit 120 as it is, and in the INTER mode, the difference (prediction error) from the predicted value (previous frame) in macroblock units as the encoded block. ) Is output.

Reference numeral 122 denotes a quantization table storage circuit for storing a quantization table; 123, a Huffman table storage circuit for storing a Huffman table; 125, a discrete cosine transform of the output of an encoding mode selection circuit 124;
Discrete cosine transform circuit (DCT) that outputs T coefficient data
, 126 converts the DCT coefficient data output from the discrete cosine transform circuit (DCT circuit) 125 into a quantization step designated by the quantization table storage circuit 122.
A Huffman coding circuit 127 for performing Huffman coding on the output of the quantization circuit 126 with reference to the Huffman table storage circuit 123;
Is a transmission buffer for buffering the output of the Huffman encoding circuit 127, and the output of the transmission buffer 128 is connected to the multiplexing unit 115.

Reference numeral 130 denotes an inverse quantization circuit that inversely quantizes the output of the quantization circuit 126, and 131 denotes an inverse discrete cosine transform circuit (an inverse DCT circuit) that performs an inverse discrete cosine transform on the output of the inverse quantization circuit 130. An adder 132 adds the predicted value to the output of the inverse discrete cosine transform circuit (inverse DCT circuit) 131 in the INTER mode and outputs the result, and outputs the output of the inverse discrete cosine transform circuit (inverse DCT circuit) 131 as it is in the INTRA mode. It is a vessel. Reference numeral 133 denotes a frame memory for motion-compensated inter-frame prediction, which stores the output (local decoded value) of the adder 132. A motion vector detection circuit 136 detects a motion vector by comparing the video signal input from the video input unit 120 with the video signal of the previous frame stored in the frame memory 133 on a macroblock basis.

Reference numeral 135 denotes a motion compensating circuit for canceling the motion by moving the data of the previous frame from the frame memory 133 within the screen in units of macro blocks in accordance with the motion vector detected by the motion vector detecting circuit 136. Note that the detection result of the motion vector detection circuit 136 is
It is also applied to the motion detection unit 121 in FIG. Reference numeral 134 denotes a low-pass filter for filtering the output of the motion compensation circuit 135 on a macroblock basis. The output of the low-pass filter 134 becomes a predicted value of the inter-frame prediction, and is applied to the adder 132 and the subtractor 129.

The subtractor 129 calculates the prediction error between the pixel data from the video input unit 120 and the output (predicted value) of the low-pass filter 134, and calculates the coding mode selection circuit 1
24.

Here, the operation of FIG. 3 will be described. Video data is input from the video input unit 120. Video input unit 120
Are input to the encoding mode selection circuit 124,
It is applied to the subtractor 129 and the motion vector detection circuit 136.

The subtractor 129 calculates the difference (prediction error) between the pixel data from the video input unit 120 and the predicted value output from the low-pass filter 134, and applies the difference to the coding mode selection circuit 124. Encoding mode selection circuit 12
4 is a pixel value from the video input unit 120 and a subtractor 129
Is compared with the prediction error, and the encoding mode is selected according to the comparison result and the control signal from the control unit B116.

In the INTRA mode, the input pixel value from the video input unit 120 is directly output to the discrete cosine transform circuit (DCT circuit) 125. In the INTER mode, the pixel value from the video input unit 120 and the subtractor 129 are output. The prediction error is converted into a discrete cosine transform circuit (DCT circuit) 12
5 is output. Discrete cosine transform circuit (DCT circuit) 1
Reference numeral 25 performs discrete cosine (DCT) conversion on the data from the encoding mode selection circuit 124 in block units, and outputs DCT coefficient data to the quantization circuit 126.

The quantization circuit 126 quantizes the DCT coefficient data from the discrete cosine transform circuit (DCT circuit) 125 at a quantization step size specified by the quantization table storage circuit 122. The Huffman encoding circuit 127
A significant block is determined with reference to the Huffman table storage circuit 123, and the quantized DCT coefficients are Huffman-coded.

The transmission buffer 128 buffers the Huffman coded data from the Huffman coding circuit 127 and outputs it to the multiplexing unit 115, and transmits the buffer accumulation amount to the control unit B116. An error correction coding circuit may be connected between the transmission buffer 128 and the multiplexing unit 115.

The inverse quantization circuit 130 includes a quantization circuit 126
The inverse quantization of the output of the quantization circuit 126 is performed with the same quantization step size as that selected in step (1), and the representative value of the DCT coefficient is output. Inverse discrete cosine transform circuit (Inverse DCT circuit)
131 performs an inverse discrete cosine transform on the output of the inverse quantization circuit 130.

In the INTER mode, the adder 132
The prediction value (the output of the low-pass filter 134) is added to the output of the inverse discrete cosine transform circuit (inverse DCT circuit) 131, and the output of the inverse discrete cosine transform circuit (inverse DCT circuit) 131 is output as it is in the INTRA mode. The output of the adder 132 is stored in the frame memory 133. The frame memory 133 has a storage capacity for at least two frames, and stores an output pixel value of the adder 132 (that is, a locally decoded value).

The motion vector detection circuit 136 compares the pixel data of the current frame from the video input unit 120 with the pixel data of the previous frame stored in the frame memory 133 to detect the motion of the video. Specifically, the pixel data of the previous frame is read from the frame memory 133 as a motion vector search window near the macroblock being processed in the current frame, and a motion vector is detected by performing a block matching operation.

The motion compensation circuit 135 moves the pixel data of the previous frame from the frame memory 133 in the screen direction according to the motion vector detected by the motion vector detection circuit 136 so as to cancel the motion. The low-pass filter 134 applies filter processing to alleviate the discontinuity at the block boundary to the pixel data of the previous frame that has been motion-compensated by the motion compensation circuit 135, and sends the processed data to the subtractor 129 and the adder 132 with the predicted value. Supply as

The control section B 116 controls the entire video encoding section 119 according to the image quality control signal and the amount of data stored in the transmission buffer 128. Specifically, the transmission buffer 1 is set so that the transmission buffer 128 does not overflow.
Changes in input video, scenes,
The quantization step size of the quantization circuit 126, the mode selection in the coding mode selection circuit 124, the significant block determination in the Huffman coding circuit 127, and the dropping of frames (frame・ Skip) is controlled.

The motion detecting section 121 includes a video encoding section 119
The motion vector output from the motion vector detection circuit 136 is stored over a plurality of frames. The motion vector to be stored may be a detected value of all blocks constituting the frame or a part of blocks at a predetermined position in the frame. The motion detection unit 121 determines the magnitude of the motion of the object in the field of view of the video input unit 120 based on the evaluation value such as the added value (or the average value) of the absolute values of the motion vectors over a plurality of frames stored as described above. Determine the frequency. The determination result is compared with a preset threshold, and when the determination result is equal to or greater than the threshold,
The motion detection signal is output to the control unit B116.

FIG. 4 is a block diagram of the video decoding unit 107. In the figure, 139 is an inverse discrete cosine transform circuit (inverse DCT circuit), 140 is an inverse quantization circuit, 141 is a Huffman decoding circuit, and 137 is an inverse quantization circuit 14
A quantization table storage circuit 138 stores a quantization table used by 0, and a Huffman table storage circuit 138 stores a Huffman table used by the Huffman decoding circuit 141.

The operation of the video encoder 119 and the video decoder 107 will be described below. First, at the time of compression, the quantization table storage circuit 122 and the Huffman
The table storage circuit 123 has a quantization circuit 12
The quantization table and the Huffman table necessary for the quantization in H.6 and the encoding in the Huffman encoding circuit 127 are loaded from the control unit B116.

A discrete cosine transform circuit (DCT circuit) 12
5 is input with video data for each coding block obtained by dividing the video information into square pixel blocks of 8 × 8 pixels in the screen. Discrete cosine transform circuit (DCT circuit)
Reference numeral 125 denotes a discrete cosine transform of the input video data. A quantization circuit 126 converts transform coefficient data output from the discrete cosine transform circuit (DCT circuit) 125 in accordance with the quantization table stored in the quantization table storage circuit 122. Quantize.

The Huffman coding circuit 127 refers to the Huffman table stored in the Huffman table storage circuit 123 and performs Huffman coding on the quantized transform coefficient data from the quantization circuit 126. Huffman encoding circuit 127
Is output to the multiplexing unit 115 as compressed video data.

At the time of decompression, the quantization table and the Huffman table used at the time of compression are stored in the quantization table storage circuit 137 and the Huffman table storage circuit 138 according to FIG.
The table is loaded from the control unit A106.

The compressed data applied from the demultiplexer 108 is input to the Huffman decoding circuit 141. The Huffman decoding circuit 141 is used for the Huffman table storage circuit 13.
With reference to the Huffman table stored in 8, the compressed video data is Huffman-decoded. The inverse quantization circuit 140 inversely quantizes the data output from the Huffman decoding circuit 141 with reference to the quantization table stored in the quantization table storage circuit 137, and then performs an inverse discrete cosine transform circuit (an inverse DCT circuit) At 139, inverse discrete cosine transform is performed. The data subjected to the inverse discrete cosine transform is output to the display unit 102 as the restored compressed video data. Hereinafter, a plurality of embodiments of the present invention will be described.

(Embodiment 1) First, Embodiment 1 will be described. The transmitting terminal 112 generates video and audio as a source. As for the video, a digital video is created by the video input unit 120. For example, when applied to a video phone system, the video input unit 120 corresponds to a TV camera, and captures an upper body image of the user on the transmission side. At the same time as the video is captured by the video input unit 120, the user's voice is heard by the microphone 118.

The digital video is input to the video encoder 119. Since the control unit B116 is in the initial state of the system, the quantization table and the Huffman table with the lowest compression ratio in this system are stored in the quantization table storage circuit 12 respectively.
2, and set in the Huffman table storage circuit 123.

The control unit B116 uses the video encoding unit 119 to compress the input video. At this time, the video encoding unit 119 compresses the video using the tables set in the quantization table storage circuit 122 and the Huffman table storage circuit 123.

On the other hand, the sound output from the microphone 118 is compressed by the sound coding unit 117. Then, the video data compressed by the video encoding unit 119 and the audio data compressed by the audio encoding unit 117 are combined by the multiplexing unit 115.
, And the two data are synchronized and transmitted by the transmitting unit B114.
Sent to

Since the bit rate of the video data to be transmitted frequently changes as will be described later, the transmission unit B114 can easily secure the required communication capacity at the time of transmission easily. A scheme with a small amount of delay is required.

The multiplexed video data and audio data are transmitted to the receiving section A11 of the receiving terminal 100 via the communication line 111.
0 is applied. Then, the multiplexed video data and audio data are separated by the demultiplexer 108 into audio data and video data. The audio data is expanded into the original audio data by the audio decoding unit 105, powered up by the audio output unit 104, and output from the speaker 103.

On the other hand, the video data is applied to the video decoding unit 107. In the video decoding unit 107, the control unit A106
After the quantization table and the Huffman table used at the time of compression are set in the quantization table storage circuit 137 and the Huffman table storage circuit 138, decompression processing is performed. The decompressed video data is displayed on the display unit 102.

Here, in this embodiment, the user's face measured by the distance measurement sensor 101 and the display unit 10
It is characterized in that the resolution of the video displayed on the display unit 102 is changed and the output of the sound output from the speaker 103 is changed at the same time in accordance with the distance to the display unit 102. Therefore, the distance between the user's face and the display unit 102 is divided into three stages as shown in FIG.
The purpose of the present invention is to appropriately set the resolution and volume of a video image in accordance with the situation where the video camera 2 faces.

The horizontal axes in FIGS. 5A to 5E correspond to the display unit 10.
2 shows the distance between the user and the user's face. Point A is the position of the display unit 102. The distance from the display unit 102 is a short distance range (within a distance α from the display unit 102), a medium distance range (more than a distance α from the display unit 102 and within β), and a long distance range (a distance β from the display unit 102) Above).

In the short range, the user is near the screen,
He observes the contents of the screen in detail, and also displays the screen at high resolution. In the middle range, the user determines that the content displayed on the screen is not so important, and removes his / her face from the screen to take a rest. However, the user is listening to the output voice so as not to miss it. The far range is a state where the user has moved away from the screen or completely separated, and it is only necessary to be able to judge whether the screen scene has changed or the sound has changed something when the user is looking at the screen from a distance. The screen may have a high compression rate and the volume may be reduced.

The distance by which the display unit 102 and the user's face are divided into the short range, the medium range, and the long range varies depending on the actual operation state. Display 1
02, the maximum display capacity of the display unit 102, the age of the user, and the like. Therefore, it is necessary to determine it by trial according to the actual system.

The vertical axis in FIGS. 5A and 5C indicates the display unit 1.
The setting state of the compression ratio of the video displayed on the display unit 102 corresponding to the distance between the user 02 and the user's face is shown. The compression ratio is divided into three levels: level 1, level 2, and level 3.

The vertical axis in FIG. 5B and FIG.
11 shows a change in the number of frames of a video image displayed on the display unit 102 corresponding to the distance between 02 and the user's face. The number of frames is also divided into three levels: level 1, level 2, and level 3.

The vertical axis of FIG. 5E shows the setting of the volume output from the speaker 103 with respect to the distance between the display unit 102 and the user's face. The volume is also divided into three levels: level 1, level 2 and level 3.

FIGS. 5A and 5B show the display unit 102.
3 shows the setting state of the compression ratio and the number of frames of the displayed video when the motion of the displayed video is large. In other words, the compression ratio is increased from level 1 to level 2 in the medium range in comparison with the short range, but the number of frames is set to level 3 which is the same as in the short range so as to follow the movement of the screen as much as possible. Similarly, the compression ratio is set to level 3 in the long distance region, but the number of frames is set to level 2 by lowering the level by one rank from the middle distance region.

The quantization table storage circuit 1 for encoding
22, the quantization table set in the quantization table storage circuit 137 for decoding is A1 in the short range, A2 in the middle range, and A3 in the long range. The Huffman table set in the Huffman table storage circuit 123 for encoding and the Huffman table storage circuit 138 for decoding is B1 in the short range, B2 in the medium range, and B3 in the long range.
And

FIGS. 5C and 5D show the display unit 102.
4 shows the setting state of the compression ratio and the number of frames of the displayed video when the motion of the displayed video is small. That is, the compression ratio is set to the same level 1 in the middle distance region and the short distance region in order to emphasize the image quality. However, since the movement of the screen is small, the number of frames is set to level 2 by lowering the number of frames in the middle range by one rank than in the short range. In the long distance range, the compression ratio is raised by one rank from the middle distance range to Level 2,
The number of frames is set at level 3 which is one rank lower than the middle distance range. In addition, the quantization table storage circuit 12
2. The quantization table set in the quantization table storage circuit 137 for decoding is A1 in the short range and A in the middle range.
1, A2 in the long distance range. Huffman encoding
The Huffman table set in the table storage circuit 123 and the Huffman table storage circuit 138 for decoding is B1 in the short range, B1 in the middle range, and B2 in the long range.

Here, the number of frames is set to a specified number of frames by performing frame dropping by controlling the amount of data stored in the transmission buffer 128 as described above. For example, FIG.
As shown in (D), in the quantization table A1 and the Huffman table B1 in the middle range, the number of frames is reduced from level 3 in the short range to level 2 by dropping frames.

Then, when the motion of the image is large (FIGS. 5A and 5B) and when the motion of the image is small (FIGS. 5C and 5D) in the middle distance range and the long distance range. Therefore, each level of the compression ratio and each level of the number of frames are matched so that the bit rate transmitted via the communication line 111 is the same.

FIG. 5E shows a setting state of the volume output from the speaker 103. In the short range, the user is located on the side of the screen, and the level is 2 at a medium volume. In the middle range, the user does not gaze at the screen and listens carefully to the voice. In the long distance range, the volume is set to the lowest level 1 because the user does not always watch the screen and does not listen to the voice.

FIG. 6 is a flowchart showing a process of setting the compression rate, the number of frames, and the volume of the display image according to the distance between the display unit 102 and the face of the user in FIG. 100 shows an operating state.

At the receiving terminal 100, a display unit 102,
The distance between the user and the opposing user's face is measured by the distance measurement sensor 101, and the distance is any of the short, medium, and long distance areas. Then, when the user moves between the areas, it notifies the transmitting terminal 112 of this fact.

In FIG. 6, a flag FL1 indicates where the user's face is located in the short range, the middle range, and the long range. When FL1 = 1, short range, FL
When 1 = 2, it indicates a middle range, and when FL1 = 3, it indicates a long range.

The flag FL2 indicates the state of the user's face. That is, FL2 =
In the case of 0, the user's face does not move and stays in the short range, middle range, and long range. When FL2 = 1, it indicates a state in which the user's face approaches the display unit 102.
In other words, the movement of the face from the middle range to the short range or the movement of the face from the far range to the middle range is shown. FL2
In the case of = 2, it indicates that the user's face has moved away from the display unit 102. That is, the movement of the face from the short range to the middle range or the movement of the face from the middle range to the long range is shown.

First, FL1 and FL2 are initialized (S0
1, S02). Display unit 102 with distance measurement sensor 101
The distance between the user and the user's face is measured (S03). It is checked whether the measured distance is a short range, a middle range, or a Liao range (S04, S06). The position of the face is currently in the short range, and the position of the previous face is checked (S05). If the position of the previous face is in the short distance range, the face is determined not to move and FL2 = 0 (S08). When the face approaches the short distance range from the middle distance range, FL2 = 1 is set (S07). The current distance area value is set as FL1 = 1 (S19).

The position of the face is currently in the middle distance range, and the position of the previous face is checked (S09, S11). If the position of the previous face has not moved in the middle distance range, FL2 = 0 is set (S14). When the position of the previous face moves to the middle range in the short range, FL
2 = 2 (S13). When the position of the previous face has moved to the middle range in the long range, FL2 = 1 is set (S12).
The current distance area value is set as FL1 = 2 (S2
0).

If the position of the face is currently in the long distance area, the position of the previous face is checked (S10). If the position of the previous face has not moved in the long distance area, FL2 = 0 is set (S18). When the position of the previous face has moved to the long distance area in the middle distance area, FL2 = 2 is set (S16). The current distance area value is set as FL1 = 3 (S21).

Next, it is checked whether the user's face has moved (S22). If not moved, the beginning of the loop (S03)
Return to When moving, the values of FL1 and FL2 are
The data is sent to the control unit B116 via the communication unit 109, the communication line 111, and the receiving unit B113 (S23).

The distance between the display unit 102 and the user's face is measured by the distance measuring sensor 101 in FIG.
The sequence for setting 1 and FL2 is controlled by the control unit A106 so as to periodically loop with a timer or the like.

FIG. 7 shows a processing sequence in the case where the face of the user approaches the display unit 102 on the receiving side in the transmitting terminal 112. First, if the user's face does not move, the process proceeds to (S24). If the user's face has moved away, the process proceeds to (S25). When the user's face approaches, the compression rate of the video displayed on the display unit 102 decreases, and the video data transmitted from the transmitting terminal 112 to the receiving terminal 100 increases, so that the data transmission amount of the communication line increases. I do. Suddenly, the capacity of the communication line that has been secured may not be secured. Therefore, it is checked whether the communication capacity has been secured (S2).
6) If not secured, start of sequence (S24)
And loop until the capacity can be secured.
The resolution of the video to be displayed on the display 2 remains unchanged.

When the capacity is secured, it is checked in which distance area the user's face is currently located (S2).
7). When the user's face has moved from the middle range to the short range, the flag FLC is set to 1, the FLF is set to 3, and the FLV is set to 2 (S29). The flag FLC is shown in FIGS.
2 shows the setting state of the compression ratio level in FIG. That is, in the case of level 1, FLC is set to 1. Similarly, FLF indicates the setting state of the level of the number of frames in FIGS. 5B and 5D. FLV indicates the setting state of the volume level in FIG.

Then, from the control unit B 116,
A quantization table and a Huffman
Regarding the table, the quantization table A1 and the Huffman table storage circuit 123 are stored in the quantization table storage circuit 122.
The Huffman table B1 is set (S32).

The FLC, FLF, and FLV are transmitted from the control unit B 116 to the control unit A 106 via the multiplexing unit 115, the transmission unit B 114, the communication line 111, and the reception unit A 110.
(S35).

Then, the video encoding unit 1 uses the quantization table A1 and the Huffman table B1 set as described above.
The compression process is performed at 19 and at the same time, the microphone 118
Is compressed by the voice encoding unit 117 (S36).

The compressed video data and audio data are multiplexed by the multiplexing unit 115 and sent to the demultiplexing unit 108 via the transmitting unit B114, the communication line 111 and the receiving unit A110 (S37). The video data and the audio data are separated by the demultiplexer 108, and the video data is separated by the video decoder 1
07, the audio data is applied to the audio decoding unit 105.

In step S27, if the position of the user's face has moved from the long range to the middle range, the receiving terminal 10
The motion of the video to be sent to 0 is checked (S28).
The motion of the video is examined by the motion detection unit 121. If the motion is large, the FLF is set to 3 in order to increase the number of frames as much as possible and set the same as the number of frames in the short range, and in order to increase the compression ratio instead of the short range. , FLC is set to 2 and the sound volume of the voice is set to be larger than that in the short range to set the FLV to 3 (S3
0). The quantization table A2 and the Huffman table B2 are set in the quantization table storage circuit 122 and the Huffman table storage circuit 123 (S33).

When the motion is small, the FLC is set to 1 in order to make the compression ratio the same as that in the short range, the FLF is set to 2 by reducing the number of frames, and the sound volume is set to be larger than the FLV in the short range. Is set to 3 (S31). Quantization table storage circuit 122 and Huffman table storage circuit 123
, Quantization table A1, Huffman table B1
Is set (S34).

Next, in step S24 of FIG. 7, when FL2 = 0, that is, when the user's face does not move (when the processing shifts to),
It moves to the sequence of FIG. First, it is checked whether the distance between the user's face and the display unit 102 is in a short distance range (S39). In the case of the short range, the process returns to the beginning of the loop (FIG. 7) without performing the processing. If the distance is not in the short range, it is checked whether it is in the middle range (S40). In the case of the middle distance range, the motion detection unit 121 checks the motion of the video (S44). In the following processing, the same processing as the sequence from S28 in FIG. 7 is performed, and FLC, FLF, and FLV
Are set (S45, S46), and a quantization table and a Huffman table are set in the quantization table storage circuit 122 and the Huffman table storage circuit 123 (S45).
47, S48).

The FLC, FLF, and FLV are transmitted from the control unit B 116 to the control unit A 106 via the multiplexing unit 115, the transmission unit B 114, the communication line 111, and the reception unit A 110.
(S51).

Then, the video encoding unit 1 uses the quantization table A1 and the Huffman table B1 set as described above.
The compression process is performed at 19 and at the same time, the microphone 118
Is compressed by the voice coding unit 117 (S52).

The compressed video data and audio data are multiplexed by the multiplexing unit 115 and sent to the demultiplexing unit 108 via the transmitting unit B114, the communication line 111, and the receiving unit A110 (S53). The video data and the audio data are separated by the demultiplexer 108, and the video data is separated by the video decoder 1
07, the audio data is applied to the audio decoding unit 105.

In FIG. 8, when FL1 = 3, that is, in the long distance range, the motion of the video is checked by the motion detecting section 121 (S41). If the movement is large, FLC is 3, FL
F is set to 2 and FLV is set to 1 (S42). The quantization table A3 and the Huffman table B3 are set in the quantization table storage circuit 122 and the Huffman table storage circuit 123 (S49).

When the motion is small, FLC is set to 2, instead of FLC, FLF is set to 1 and FLV is set to 1.
Is set to 1 (S43). Quantization table storage circuit 122
And the Huffman table storage circuit 123 are set with the quantization table A2 and the Huffman table B2 (S
50).

If FL2 = 1 in step S25 in FIG. 7, that is, if the user's face has moved away (the process proceeds to step S25), the process proceeds to the sequence in FIG. Check the destination (S
55). In the case of the middle distance area, the same processing as S28 and subsequent steps in FIG. 7 is performed (S59 to S63, S66 to S68). If the destination is a long distance (S56), the same processing as S41 and subsequent steps in FIG. 8 is performed (S57, S58, S64, S65,
S66 to S68).

FIG. 10 shows that the flags FLC, FLF, and FLV sent from the transmitting terminal 112 are stored in the receiving terminal 1.
This is the processing sequence after input by the control unit A106 of FIG. First, FLC, FLF, FL
V is received (S71). The contents of FLC are checked (S72, S73).

When FLC is 1, the quantization table A1,
The Huffman table B1 is stored in the quantization table storage circuit 13
7 and the Huffman table storage circuit 138 (S74). When FLC is 2, the quantization table A2,
The Huffman table B2 is stored in the quantization table storage circuit 13
7 and the Huffman table storage circuit 138 (S75). If FLC is 3, the quantization table A3,
The Huffman table B3 is stored in the quantization table storage circuit 13.
7 and the Huffman table storage circuit 138 (S76).

Check the contents of the FLF (S77, S
78). If the FLF is 1, the number of frames is set to level 1 (S79). If the FLF is 2, the number of frames is set to level 2 (S80). If the FLF is 3, the number of frames is set to level 3 (S81).

Check the contents of the FLV (S82, S
83). If the FLV is 1, the volume is set to level 1 (S84). If the FLV is 2, the volume is set to level 2 (S85). If the FLV is 3, the volume is set to level 3 (S86).

The video data and the audio data transmitted from the transmitting terminal 112 are received (S87). And
The video decoder 107 expands the video data based on the quantization table, the Huffman table, and the number of frames set in the quantization table storage circuit 137 and the Huffman table storage circuit 138 as described above. The audio data is also expanded by the audio decoding unit 105 (S88). The decompressed video data is displayed on the display unit 102. On the other hand, the expanded audio data is also set to the above-mentioned volume by the audio output unit 104 and output from the speaker 103 (S89).

Here, if the communication capacity cannot be secured in S26 of FIG. 7, the quality of the video is kept as it is, and the quality of the video is increased when the communication capacity is secured. For example, as shown in FIGS. 11A and 11B, the compression ratio is level 2 (a) and the number of frames is level 3 (b) in the middle distance range, and the user brings the face closer to the display unit 102 to reduce the distance to the short range. Moved,
It is assumed that the communication capacity cannot be secured. In such a case, as shown in FIGS. 11C and 11D, the compression ratio is lowered from level 2 to level 1 (c), the number of frames is lowered from level 3, and the number of frames (communication capacity is secured) The image is displayed in advance as d). And FIG.
When the communication capacity is secured as in (E) and (F),
The number of frames may be set to level 3 (f), which should be originally set.

Embodiment 2 Next, Embodiment 2 of the present invention will be described. In this embodiment, the compression ratio, the number of frames, and the volume are continuously changed in the short range and the middle range.

In the first embodiment, the distance between the user's face and the display unit 102 is divided into three stages of a short range, a medium range, and a long range with respect to the compression ratio, the number of frames, and the volume. However, for compression ratio, number of frames, and volume,
When fine control is necessary, FIG.
As shown in (B) and (C), the compression ratio, the number of frames, and the sound volume are continuously changed in the short range and the middle range, and are set to the same values in the long range as described above. You may.

(Embodiment 3) Next, Embodiment 3 of the present invention will be described. Receiving terminal 10 in this embodiment
13 is shown in FIG. 13, and the circuit configuration of the receiving terminal 100 and the transmitting terminal 112 is shown in FIG.

In the first and second embodiments, an infrared sensor or the like is used as the distance measuring sensor 101 to output infrared light and the like, capture infrared light reflected by the user's face, and obtain a time difference between the output time and the reception time. Has measured the distance between the display unit 102 and the user's face.

However, in a multimedia personal computer or a video telephone system, the receiving terminal 100
Is also equipped with a video input unit. On the receiving side, the user's face or an arbitrary scenery on the receiving side is photographed and transmitted as an image to the transmitting side, and the transmitting side observes the image photographed on the receiving side.

In this embodiment, instead of the distance measuring sensor 101, an image input unit A142 comprising a CCD camera or the like is attached as shown in FIG. 10
Measure the distance between 2 and the user's face.

In FIG. 14, an image input from an image input section A 142 such as a CCD camera extracts a face area of a person by a face area extraction section 143. For example, the method of JP-A-9-281605 can be applied to the face region extraction. In this method, a hue value and a saturation value are obtained for each pixel of the video data input from the video input unit A142, and a two-dimensional histogram of the hue value and the saturation value of the entire video is created. Then, the two-dimensional histogram is clustered for each mountain. By combining the divided mountains for each group, a region that is a candidate for a human face is extracted. Further, the area of the region extracted as the face candidate region is determined, the minute region is removed and renumbered, and the face candidate region is accurately extracted.

The present method is characterized in that all face regions can be extracted even if the faces of a plurality of persons appear in the video. Therefore, for example, when two persons are looking at the display unit 102 as shown in FIG. 15, it is preferable to extract the face area 144 located at the forefront and obtain the area of the area.

Although the face area 144 is a curve, the minor axis a and the major axis b which are approximately in contact with the curve are obtained, and the area is set to a × b. When the area is large, the user is approaching the display unit 102, and when the area is small, the user is far from the display unit 102.

Then, the horizontal axis in FIG. 5 is replaced with the area of the face area of the user, and the first and second embodiments are replaced.
The same effect can be expected. Then, when the face area cannot be extracted, the user may be located in the long distance area in the first embodiment.

(Modification) In the first embodiment, since the user does not face the display unit 102 in a long-distance region, there is little possibility that the user does not look at the screen and listens to the sound correctly. Therefore, the video was compressed to the maximum and the volume was reduced to the minimum.
However, at the same time as changing the volume of the voice, the voice can also be compressed, and it is only necessary to know how the speech is being made.

Further, when the user is far from the screen by a predetermined distance or more, the display of the video and the output of the sound may be stopped. In particular, when the audio output and the display of the video are stopped, the power consumption is saved, and the transmission side terminal 112 transmits the reception side terminal 100 to the reception side terminal 100.
Since there is no need to transmit the video to the communication channel, there is a margin in the capacity of the communication line, and more communication can be allocated for another channel.

Then, only either video or audio can be left. For example, if the display of the video is stopped and only the audio is output, the user may be able to understand the content to some extent if the user can hear the audio even when working far away from the screen. Alternatively, the video may be displayed and the audio output may be stopped. If the user is at a distance where the contents of the screen can be glimpsed from time to time, the voice output is noisy, but it is more convenient to display the screen. It is preferable that the user can appropriately select screen display or audio output.

In the first embodiment, the distance area is divided into three steps. However, the distance area may be roughly divided into two steps such as a short distance area and a long distance area. That is, when the user is watching the image, the image is displayed in high definition. When the user is away from the display unit 102 and cannot see the image, the image is displayed in low definition.
The audio output may be reduced. Then, in the long distance range, the display and the sound output may be stopped as described above.

In the above embodiment, the user's face and the display 10
2, the degree of compression of the displayed video and the volume of the output sound were simultaneously changed in accordance with the distance from the display device. However, the sound volume may be set to be large so that the sound can be heard even if the sound is slightly away, and only the image quality may be changed according to the distance. Conversely, if the communication capacity can always be sufficiently ensured, the image quality may be fixed and the sound volume may be changed according to the distance.

In the above embodiment, when the user's face approaches from the long range to the middle range or from the middle range to the short range, the communication capacity of the communication line increases and the capacity cannot be secured. They waited until they could be secured, and as soon as they could be secured, the displayed images were being refined. However, when communication is performed over multiple channels and the communication capacity becomes tight, the priority of channel reservation may be set.

For example, it is assumed that a first user who needs to increase the capacity from the middle range to the short range and a second user who needs to increase the capacity from the long range to the middle range occur at the same time. In the case of the first user, it is presumed that an important display is displayed on the screen and that the user needs to look seriously at the screen as the user approaches the short-range from the middle-range. On the other hand, it is presumed that the second user goes to the screen from the state where he has not been on the screen side and grasps the contents of the screen for the time being. Therefore, it is preferable to secure the communication capacity by giving the first user priority over the second user.

If the first user cannot secure the communication capacity by any means, the other users who are currently communicating in the middle range are set to correspond to the long range, and the surplus communication capacity is reduced to the first. May be redirected for users.

Furthermore, the distance measuring sensor 1 of the above embodiment
The purpose of use 01 is to measure the distance between the user's face and the display unit 102 when the user turns his / her face in front of the display unit 102. Here, if the directivity of infrared rays and the like emitted from the distance measurement sensor 101 is sharp,
Even if the face is slightly deviated from the screen, infrared rays do not shine and the face cannot be detected. Therefore, when the infrared rays are emitted with a broad directivity, they surely hit the user. However, infrared light may hit the user's face instead of the face.
Even if the distance is measured at a location other than the face, there is a possibility that an error from the distance measured accurately at the face may be within an allowable range. Therefore, the distance measurement sensor 101 may measure the distance by making the directivity relatively broad at the time of emission of infrared rays and capturing the infrared rays that have hit somewhere of the user.

[0109]

As described above, according to the video display method of the present invention, when the user views the video displayed on the display unit at the receiving terminal, the user can change the distance between the display unit and the user's face. Therefore, by changing at least one of the compression ratio of the display video and the volume of the output audio, it is possible to output the audio at an appropriate volume according to the distance without displaying an image with excessive image quality. As a result, the operability of the receiving terminal is improved, and the transmission amount of the video data transmitted from the transmitting side is reduced.

[Brief description of the drawings]

FIG. 1 is a perspective view showing the appearance of a receiving terminal according to Embodiments 1 and 2 of the present invention.

FIG. 2 is a block diagram showing a configuration of a system including a receiving terminal and a transmitting terminal according to the first and second embodiments of the present invention.

FIG. 3 is a block diagram showing a detailed configuration of a video encoding unit of the transmitting terminal shown in FIG. 2;

FIG. 4 is a block diagram showing a detailed configuration of a video decoding unit of the receiving terminal shown in FIG. 2;

FIG. 5 is a diagram illustrating a setting state of a compression ratio of a display image, a number of frames, and a volume of an output sound according to a distance between a display unit and a user according to the first embodiment of the present invention.

FIG. 6 is a flowchart showing an operation sequence of the transmitting terminal according to the first embodiment of the present invention.

FIG. 7 is a flowchart showing an operation sequence when a user's face approaches a display unit in the receiving terminal according to the first embodiment of the present invention.

FIG. 8 is a flowchart showing an operation sequence when the user's face is stationary at the receiving terminal according to the first embodiment of the present invention.

FIG. 9 is a flowchart showing an operation sequence when the user's face moves away from the display unit at the receiving terminal according to the first embodiment of the present invention.

FIG. 10 is a flowchart illustrating a processing sequence of a receiving terminal according to the first embodiment of the present invention.

FIG. 11 illustrates a setting state of a compression ratio of a display image, the number of frames, and a volume of an output sound when a communication line capacity cannot be secured even when a user's face approaches a display unit according to the first embodiment of the present invention. Figure.

FIG. 12 shows a compression ratio, a number of frames, and the like of a display image according to a distance between a display unit and a user according to the second embodiment of the present invention.
FIG. 6 is a diagram illustrating a setting state of a volume of an output sound.

FIG. 13 is a perspective view showing the appearance of a receiving terminal according to a third embodiment of the present invention.

FIG. 14 is a block diagram illustrating a configuration of a system including a receiving terminal and a transmitting terminal according to a third embodiment of the present invention.

FIG. 15 is an explanatory diagram showing an example of a face area extracted by a face area extraction unit according to the third embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 100 receiving terminal 101 distance measuring sensor 102 display unit 103 speaker 104 audio output unit 105 audio decoding unit 106 control unit A 107 video decoding unit 108 demultiplexing unit 109 transmission unit A 110 ... Receiving unit A 111 communication line 112 transmitting terminal 113 receiving unit B 114 transmitting unit B 115 multiplexing unit 116 control unit B 117 audio encoding unit 118 microphone 119 video encoding unit 120 Image input unit 121: Motion detection unit 122: Quantization table storage circuit 123: Huffman table storage circuit 124: Coding mode selection circuit 125: Discrete cosine transform circuit (DCT circuit) 126: Quantization circuit 127: Huffman coding circuit 128 transmission buffer 129 subtracter 130 inverse quantization circuit 131 inverse discrete cosine transform Conversion circuit (inverse DCT circuit) 132 ... Adder 133 ... Frame memory 134 ... Low-pass filter 135 ... Motion compensation circuit 136 ... Motion vector detection circuit 137 ... Quantization table storage circuit 138 ... Huffman table storage circuit 139 ... Inverse discrete Cosine transform circuit (inverse DCT circuit) 140 ... Inverse quantization circuit 141 ... Huffman decoding circuit 142 ... Video input unit A 143 ... Face region extraction unit

 ──────────────────────────────────────────────────続 き Continued on front page F term (reference) 5C026 DA05 DA14 5C059 KK15 KK33 KK47 MA01 MA23 MC11 MC14 MC31 MC38 ME02 NN01 NN21 NN28 NN41 PP04 PP14 RA01 RF04 RF05 RF19 SS07 TA07 TA17 TA18 TA21 TA41 TA45 TA47 TA50 TA57 TA58 TA62 TA65 TB07 TC08 TC11 TC18 TC20 TC36 TC38 TC45 TD03 TD06 UA02 UA05 UA11 UA34 5C064 AA01 AA02 AC01 AC02 AC06 AC11 AC12 AC16 AC22 AD02 AD14

Claims (5)

[Claims] 1. A compression means for compressing video and audio, a display means for decompressing a video compressed by the compression means and displaying it on a display screen, and expanding and outputting the audio compressed by the compression means. Audio output means, and means for determining the distance between the user facing the display screen and the display screen,
Means for changing at least one of the compression ratio of the video and the volume of the output audio in accordance with the distance obtained by the means. 2. When the distance is longer than a predetermined length,
2. The apparatus according to claim 1, further comprising a unit configured to stop an operation of at least one of the display unit and the audio output unit.
Video display system described in 1. 3. A means for compressing video and audio, a transmission means for transmitting the video and audio compressed by the means via a multi-channel communication line, and a means for compressing the compressed video transmitted by the transmission means. Display means for decompressing and displaying on a display screen, sound output means for decompressing and outputting the compressed sound transmitted by the transmission means, and determining a distance between a user facing the display screen and the display screen Means,
Means for changing at least one of the compression rate of the video and the volume of the output audio in accordance with the distance obtained by the means; And a control means for controlling transmission of a video with the temporarily secured communication capacity when the communication cannot be secured. 4. A means for compressing video and audio, a transmission means for transmitting the video and audio compressed by the means via a multi-channel communication line, and a means for compressing the compressed video transmitted by the transmission means. Display means for decompressing and displaying on a display screen, sound output means for decompressing and outputting the compressed sound transmitted by the transmission means, and determining a distance between a user facing the display screen and the display screen Means,
According to the distance obtained by this means, means for changing at least one of the compression rate of the video and the volume of the output audio, the compression rate of the video is reduced in a plurality of channels in the communication line, Means for securing a communication capacity by assigning a priority between the plurality of channels according to the distance when a predetermined increase in the communication capacity cannot be secured. 5. A means for determining a distance between a user facing the display screen and the display screen, comprising: means for capturing an image including a face of the user; and extracting a face area from the image captured by the means. 5. The apparatus according to claim 1, further comprising: a unit configured to calculate a distance to a user from an area of the face region extracted by the unit. 6. Video display system.