JPH09116879A - Audio visual communication terminal equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an audio visual communication equipment whose operability is satisfactory and device cost is inexpensive by adopting a function for executing an equipment operation only by means of executing an action such as to wave toward the video input means of a video camera and the like. SOLUTION: A moving picture signal outputted from a video camera device 14 being a moving picture video input means is inputted to moving picture CODEC 15 and a display control part 16, Moving picture CODEC 15 divides an inputted video signal into prescribed blocks, obtains differential, information with a previous frame in the block unit, encodes/compresses the moving picture video in the block unit based on differential information and detects that differential information of the respective blocks change. A system control part 1 calculates a position coordinate corresponding to the block which is detected to be changed and discriminates that a prescribed equipment operation is executed based on the position coordinate.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audiovisual communication terminal device having at least a communication function for exchanging audio information and moving image information.

[0002]

2. Description of the Related Art Conventionally, as an audiovisual communication terminal having at least a communication function for exchanging voice information and moving picture information, for example, a video conference communication terminal device or a video telephone terminal device has been put into practical use.

In these audiovisual terminal devices, various operations on the device are performed by a dedicated operation display device, or operation symbols corresponding to various operations are displayed on a video monitor device displaying a conference screen. (A so-called icon) is displayed, and this operation symbol is operated by using a remote controller or a pointing device (for example, a mouse).

[0004]

However, the conventional device has the following inconveniences.

That is, when performing various operations using the operation display device, the operator must look away from the screen of the video monitor device and look at the operation display device, and the line of sight moves to improve operability. It caused the inconvenience of being bad.

When operating symbols corresponding to various operations are displayed on the video monitor device and the operating symbols are operated using a remote controller or a pointing device,
Since it is necessary to separately provide a remote controller and a pointing device, the structure of the device becomes complicated and the cost of the device increases.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an audiovisual communication terminal device having good operability and low device cost.

[0008]

According to the present invention, in an audiovisual communication terminal device having at least a communication function for exchanging audio information and moving picture information, a picture inputting means for inputting a moving picture, and the above picture inputting means. The video signal output from is divided into predetermined blocks, the difference information with the previous frame is obtained for each block unit, and the moving image information is coded and compressed for each block based on the difference information, and at the same time for each block. The moving picture coding unit that detects that the difference information has changed, and the moving picture coding unit calculates the position coordinates corresponding to the block in which the difference information is detected to be changed, and calculate the position coordinates as the position coordinates. On the basis of this, a control means for determining that a predetermined device operation has been performed is provided.

Further, in an audiovisual communication terminal device having at least a communication function for exchanging voice information and moving image information, a predetermined image input means for inputting a moving image and a video signal output from the image input means are predetermined. The block is divided into blocks, the motion vector based on the difference from the previous frame is obtained for each block, the video information is coded and compressed in block units based on the motion vector, and the video content of each block is based on the motion vector. Is detected by the moving picture coding means, and the moving picture coding means calculates the position coordinates corresponding to the block in which the changing of the video content is detected, and based on the position coordinates A control means for determining that a predetermined device operation has been performed is provided.

Further, in an audiovisual communication terminal device having at least a communication function for exchanging audio information and moving image information, a predetermined image input means for inputting a moving image and a video signal output from the image input means are predetermined. The block is divided into blocks, the motion vector based on the difference from the previous frame is calculated for each block, and the moving image information is coded and compressed for each block based on the motion vector, and the motion vector corresponding to the absolute value of the motion vector Moving picture coding means for detecting that the video content of each block has changed based on the evaluation value, and position coordinates corresponding to the blocks for which the moving picture coding means has detected that the video content has changed And a control means for determining that a predetermined device operation has been performed based on the position coordinates. The moving image encoding means, while increasing the weighting for the vertical component than the horizontal component of the motion vector, is obtained to calculate the evaluation value of the motion vector.

Further, in an audiovisual communication terminal device having at least a communication function for exchanging audio information and moving picture information, a video input means for inputting a moving picture and a video signal outputted from the video input means are provided.
Divide into a predetermined block, determine the difference information with the previous frame in block units, code the moving image information in block units based on the difference information, and based on the difference information obtained during a predetermined time A moving image encoding unit that detects that the image of each block has changed, and the moving image encoding unit calculates position coordinates corresponding to the block in which the difference information is detected to have changed, Based on the position coordinates, a control means for determining that a predetermined device operation has been performed is provided.

Also, in an audiovisual communication terminal device having at least a communication function for exchanging audio information and moving image information, a predetermined image input means for inputting a moving image and a video signal output from the image input means are specified. The block is divided into blocks, the motion vector based on the difference from the previous frame is obtained in the block unit, and the moving image information is coded and compressed in the block unit based on the motion vector, and the motion vector obtained during a predetermined time. The moving picture coding means for detecting that the video content of each block has changed based on the above, and the position coordinates corresponding to the block where the video content is detected to have been changed by the moving picture coding means. It is provided with a control unit that calculates and determines based on the position coordinates that a predetermined device operation has been performed. .

Further, in an audiovisual communication terminal device having at least a communication function for exchanging audio information and moving image information, a predetermined image input means for inputting a moving image and a video signal output from the image input means are specified. The block is divided into blocks, the motion vector based on the difference from the previous frame is obtained in the block unit, and the moving image information is coded and compressed in the block unit based on the motion vector, and the motion vector obtained during a predetermined time. The moving picture coding means for detecting the change of the video content in each block based on the motion vector evaluation value corresponding to the absolute value of Calculate the position coordinates corresponding to the specified block, and perform a predetermined device operation based on the position coordinates The moving picture coding means is provided with a determination means, and the motion vector coding means calculates the evaluation value of the motion vector in a state in which the weighting of the vertical component of the motion vector is larger than that of the horizontal component. is there.

Further, the control means superimposes and displays the operation symbol on the display screen, and discriminates the operation content of the device based on the display coordinates of the operation symbol. Further, as the predetermined device operation, a shooting screen control operation of the video input means can be applied. Further, screen instruction symbol display can be applied as the predetermined device operation.

[0015]

BRIEF DESCRIPTION OF THE DRAWINGS FIG.
Embodiments of the present invention will be described in detail.

FIG. 1 shows a video conference communication terminal device according to an embodiment of the present invention. This video conference communication terminal device uses a basic interface of ISDN as a transmission path, and has a communication function of voice information, a communication function of moving image information, a communication function of still image information, a facsimile communication function, a telewriting function, and It has a multiplexed communication function of these communication functions, and is connected to the basic interface of ISDN, and has two information channels (B channel).
Data communication using is possible.

In FIG. 1, the system control unit 1 controls the respective units of the video conference communication terminal device, processes the upper layer of the video conference communication, and executes various application programs included in the video conference communication terminal device. It executes various control processing such as processing and help display processing corresponding to the help request from the user.
The ROM (Read Only Memory) 2 stores a part of the control processing program executed by the system control unit 1 and various data necessary for executing the control processing program. The access memory 3 constitutes a work area or the like of the system control unit 1.

The clock circuit 4 is for outputting current date and time information, the scanner 5 is for reading and inputting an original image at a predetermined resolution, and the plotter 6 is for reading an image at a predetermined resolution. It is for recording and outputting, and the encoding / decoding unit 7 encodes and compresses the image signal read by the scanner 5 by the encoding method of the group 4 facsimile function, and at the same time, based on the encoded image information. The image processing unit 8 executes various image processing such as image data scaling processing and resolution conversion processing, and the magnetic disk device 9 is system software. It is for storing a plurality of application programs and a large number of user data.

The operation display unit 10 is used as a telewriting operation display means in the telewriting communication function, and is used by the user to operate the video conference communication terminal device.

The voice input / output device 11 is for inputting a voice for a call with a microphone and outputting it with a speaker. An analog voice signal input from the voice input / output device 11 with a microphone is output to the voice CODEC 12. ,
The analog audio signal output from the audio input / output device 14 as a speaker is output from the audio CODEC 11 and added to the audio input / output device 11.

The audio CODEC 12 is for performing signal conversion processing of analog signal / digital data for transmitting an analog audio signal using the B channel of ISDN. The voice control unit 13 is for controlling the operation of the voice input / output device 11.

The video camera device 14 is for taking an image on the user side of the video conference communication terminal device, and the moving image signal output from the video camera device 14 is input to the image signal input terminal of the moving image CODEC 15. In addition to being added, it is added to the moving image signal input end of the display control unit 16. The video camera control unit 17 controls shooting on / off, zoom-in / zoom-out of the video camera device 14,
It is also for controlling operations such as bread.

The moving picture CODEC 15 converts the analog NTSC format moving picture signal input from the video camera device 14 into corresponding digital moving picture data, converts the moving picture data into predetermined CIF format moving picture data, and further, TTC. Standard JT-H. In addition to encoding and compressing to form moving image information, the moving image information in the encoded and compressed state is converted into original CIF format moving image data, and the moving image data is converted into NTSC format moving image data. Then, the moving image data is converted from digital to analog and converted into a moving image signal in an analog NTSC format.

The document camera device 18 is for photographing a still image, and the analog NTSC signal (still image signal) output from the document camera device 18 is added to the still image CODEC 19.

The still image CODEC 19 performs analog / digital conversion on a still image signal applied from the document camera device 18, and then encodes and compresses the still image signal by a predetermined encoding method (for example, JPEG method), and in a compressed state. The still image information is decoded into the original still image signal, and the still image signal input from the document camera device 18 or the still image signal obtained by decoding is output to the still image signal input terminal of the display control unit 16. ing. The document camera control unit 20 is for controlling operations such as photographing on / off and zoom-in / zoom-out of the document camera device 18.

The display controller 16 includes a video monitor device 21.
The display contents of the screen displayed on the screen are controlled by a command from the system control unit 1.
The display screen information of the transmitted moving image signal transmitted from the terminal itself, the display screen information of the received moving image signal input from the moving image CODEC 15, and the display screen information of the still image signal added from the still image CODEC 19 are formed, and The display screen of the video monitor device 21 is configured in a mode in which the display screen information is appropriately combined. For example, when the display screen of the transmitted moving image signal has the content shown in FIG. 2A and the display screen of the received moving image signal has the content shown in FIG. A screen as shown in FIG. 7C is displayed on the video monitor device 21.

The ISDN interface circuit 22 is an IS
In addition to being connected to the DN, the ISDN layer 1 signal processing function and the integration / separation function of the D channel signal and the two B channel signals are provided, and the D channel signal is connected to the D channel transmission control unit 23. Exchange and again
The two B channel signals are in communication with the multiplexer / demultiplexer 24.

The D channel transmission control section 23 has a D channel layer 2 signal processing function and a call control processing function for connecting / disconnecting one or two B channels.

The multiplexer / demultiplexer 24 multiplexes the data of a plurality of media such as voices, moving images, and general-purpose data (other data such as still image data) exchanged using the B channel to obtain the TTC standard JT. -H. In addition to forming frame data specified in H.221 and sending it to the line side, it separates data of a plurality of media multiplexed with the received frame data. The multiplexer / demultiplexer 24 converts audio data into audio CODE.
It exchanges with C12, and the video data is video CODE.
It exchanges data with the C15, and exchanges general-purpose data with the system control unit 1.

These system control unit 1, ROM 2, R
AM3, clock circuit 4, scanner 5, plotter 6, encoding / decoding unit 7, image processing unit 8, magnetic disk device 9, operation display unit 10, audio CODEC 11, audio control unit 12, moving image CODEC 15, display control unit 16, The video camera control unit 17, the document camera control unit 20, the D channel transmission control unit 23, and the multiplexing / demultiplexing device 24 have an internal bus 25.
Data exchange between these elements is mainly performed via the internal bus 25.

FIG. 3 shows an example of a schematic structure of the moving image CODEC 15.

In the figure, the transmitted moving picture signal VI is for converting the analog NTSC format moving picture data into a predetermined CIF format moving picture data by the format converter 27, and the output thereof is added to the moving picture encoder 28. Has been.

The moving picture encoder 28 converts the inputted moving picture data in the CIF format into the TTC standard JT-H. The moving picture information is formed by encoding and compressing by the coding method of H.261, and the output signal thereof is output to the next-stage device as transmission moving picture information VD1.

The moving picture decoder 29 converts the received moving picture information VD2 received from the partner terminal into original CIF format moving picture data, and the output signal thereof is added to the format converter 30.

The format converter 30 receives the input CI
The F format moving image data is converted into NTSC format moving image data, and the output signal thereof is output to the next stage device as a received moving image signal VO.

Further, the moving picture encoder 28 and the moving picture decoder 2
9 exchanges various control information with the system control unit 1, respectively. For example, the video encoder 28
When the system controller 1 requests the output of difference information,
Difference information indicating a difference in information from the previous frame is output to the system control unit 1 for each macroblock that is a unit of encoding moving image data. Further, when the motion picture encoder 28 is requested by the system control unit 1 to output a motion vector, the motion picture encoder 28 outputs the motion vector information to the system control unit 1 for each macroblock.

Here, the difference information and the motion vector information are the TTC standard JT-H. In the encoding method of H.261, the information to be encoded is formed inside the moving image encoder 28, and when requested by the system control unit 1, the difference information and the motion formed inside thereof are generated. The vector information is output to the system control unit 1 together with the coordinate information (macroblock coordinates) representing the macroblock.

In addition, the TTC standard JH-H. CIF (Common Inter) that is the target of encoding in H.261.
The median format (Common Intermediate Format) format is a basic format of moving image information. In this CIF format, a screen of 29.97 frames is displayed per second, and one frame (image frame) is displayed for the luminance component (Y). ) Is composed of 288 lines, one line is composed of 360 pixels, and color difference components (Cb, C
Regarding r), one frame is composed of 144 lines and one line is composed of 180 pixels.

The frame is divided into 12 GOBs (Grou, 2 × 6 (horizontal direction × vertical direction; the same applies hereinafter)).
p Of Block), and each GOB is divided into 33 macroblocks of 11 × 3.

Therefore, one screen (frame) is
As shown in FIG. 4, the macroblock is divided into 22 × 18 macroblocks, and the moving picture coding process is performed in macroblock units. Further, in this moving image coding process, one macroblock is divided into four 2 × 2 blocks for the luminance component and one × 1 block for the color difference component. Therefore, for the luminance component and the color difference component, one block is 8 pixels ×
It consists of 8 lines.

In the present embodiment, as shown in FIG.
Four operation symbols (icons) are displayed in a superimposed manner (superimpose display) on the display screen of the transmitted image on the display screen of the video monitor device 21, and when the user moves his or her hand at the display position of these operation symbols, the movement is performed. It is determined that the operation of the operation symbol displayed corresponding to the display coordinates is selected, and the process of the selected selection operation is executed.

In this way, by making a gesture such that the user operates the operation symbol displayed on the video monitor device 21, the process corresponding to the operation symbol can be activated, so that the user operates the operation symbol. For this reason, the necessary operation can be performed without performing the operation of removing the line of sight from the video monitor device 21 and moving the line of sight to the operation display unit 10 and operating the operation display unit 10. As a result, this video conference communication The operability of the terminal device is significantly improved.

FIG. 6 shows an example of processing executed by the system controller 1 in this case.

First, the system control unit 1 superimposes a predetermined operation symbol on the display screen of the transmission image of the operation symbol (process 101) and sets a detection area corresponding to the display area of the operation symbol (step 101). Processing 10
2). Here, the type, the number, and the display position of the operation symbols may be determined in advance by the setting by the user, or may be determined in advance in the system.

When the operation symbol is superimposed and displayed on the display screen of the transmitted image in this manner, a predetermined video change detection process (process 103) is performed until a change in the image is detected in the display area of any of the operation symbols. ; Later described) is repeatedly executed (NO loop of decision 104).

When a change in the image is detected in the display area of any of the operation symbols and the result of determination 104 is YES, the operation symbol displayed at the change coordinates detected at that time is determined ( After performing the operation process corresponding to the determined operation symbol (process 105) (process 106), the process returns to process 101 and the operation detecting operation of the next operation symbol is performed.

FIG. 7 shows an example of the image change detection processing.

First, for one macroblock, the difference value is read from the moving picture encoder 28 of the moving picture CODEC 15 (process 201), and it is checked whether or not it is included in the set detection area (decision 202). Judgment 20
When the result of 2 is NO, the process returns to step 201 and the difference value is read for the next macroblock.

This is the case where the read macroblock is included in the set detection area, and the result of judgment 202 is YE.
When it becomes S, it is checked whether or not the value of the difference value read at that time is the predetermined value KA or more (determination 20
3) If the result of determination 203 is YES, the macroblock address of the macroblock is stored (process 204).

Next, it is checked whether or not the processing of the inspection area has been completed for one frame (decision 205), and when the result of the judgment 205 is NO, the processing returns to step 201, and the same processing is repeated for the remaining macroblocks. .

When the result of judgment 205 is YES, it is checked whether any macroblock address is stored at that time (judgment 206). When the result of judgment 206 is YES, the memory is stored. The change detection is notified together with the macro block address being processed (process 207), and this process ends. When the result of the determination 206 is NO, no change detection is notified (process 208), and this process ends.

In this way, in this case, it is checked whether or not the difference value with respect to the previous frame is greater than or equal to the predetermined value KA for each macroblock, and the macroblocks having the above difference are selected by the user. It is determined that the operation has been performed.

Further, not only the difference value but also the motion vector can be used for this judgment. FIG. 8 shows an example of the image change detection processing in that case.

First, for one macroblock, the value of the motion vector is read from the moving picture encoder 28 of the moving picture CODEC 15 (process 301), and it is checked whether or not it is included in the set detection area (decision 30).
2) If the result of determination 302 is NO, process 3
Returning to 01, the value of the motion vector for the next macroblock is read.

In the case where the macroblock into which the value of the motion vector is read is included in the set detection area and the result of judgment 302 is YES, the absolute value of the value of the motion vector read at that time is calculated. (Process 30
3) Check whether or not the calculated absolute value is equal to or greater than a predetermined value (decision 304). If the result of decision 304 is YES, the macroblock address of the macroblock is stored (process 305). .

Next, it is checked whether or not the processing of the inspection area has been completed for one frame (decision 306). When the result of the judgment 306 is NO, the processing returns to step 301 and the same processing is repeated for the remaining macroblocks. .

When the result of judgment 306 is YES, it is checked whether any macroblock address is stored at that time (decision 307). When the result of judgment 307 is YES, the memory is stored. The change detection is notified together with the macro block address being processed (process 308), and this process ends. When the result of the judgment 307 is NO, no change detection is notified (process 309), and this process ends.

By the way, the screen shot by the video camera device 14 includes members other than the user who will operate the operation symbol (hereinafter referred to as the operating user), and the members other than the operating user are displayed on the screen. Move
When the movement range includes the display area of the operation symbol, there is a possibility that it may be determined that one of the operation symbols has been selected and operated by the movement of the member other than the operating user.

In order to eliminate such an inconvenience, the fact that the movements of the members other than the operating user are mainly in the horizontal direction is used, and the change in the image is detected by paying attention to the change in the image in the vertical direction. . An example of the image change detection processing in that case is shown in FIG.

First, with respect to one macroblock, the value of the motion vector is read from the moving picture encoder 28 of the moving picture CODEC 15 (process 401), and it is checked whether or not it is included in the set detection area (decision 40).
2) If the result of determination 402 is NO, process 4
Returning to 01, the value of the motion vector for the next macroblock is read.

In the case where the macroblock into which the value of the motion vector is read is included in the set detection area, and the result of the judgment 402 is YES, the following equation is calculated based on the value of the motion vector read at that time. (I) is calculated to calculate the motion vector evaluation value MVS (process 40).
3).

MVS = ((Vx × 0.2) ** 2+ (Vy × 1.8) ** 2)) ** (1/2) (I)

Here, Vx represents the x-direction (horizontal direction) component of the motion vector, and Vy represents the y-direction (vertical direction) component of the motion vector. In addition, the operator (x **
y) represents an operation in which x is raised to the power of y. In this case, the coefficient of Vx is 0.2 and the coefficient of Vy is 1.
Therefore, the weighting of the vertical component of the motion vector is made larger than that of the horizontal component of the motion vector, so that the influence of noise that causes erroneous detection can be eliminated.

Next, it is checked whether the calculated evaluation value MVS is equal to or more than a predetermined value (decision 404),
When the result of determination 404 is YES, the macroblock address of the macroblock is stored (process 4).
05).

Next, it is checked whether or not the processing of the inspection area has been completed for one frame (decision 406), and when the result of the judgment 406 is NO, the processing returns to step 401, and the same processing is repeated for the remaining macroblocks. .

When the result of judgment 406 is YES, it is checked whether any macroblock address is stored at that time (judgment 407). When the result of judgment 407 is YES, it is stored. The change detection is notified together with the macro block address being processed (process 408), and this process ends. When the result of determination 407 is NO, no change detection is notified (process 409) and this process ends.

By the way, in the embodiment shown in FIG. 10, the present invention is applied to the operation for panning the photographing screen of the video camera device 14 left and right.

In this case, the operation area is set in advance to an area having a constant width on the left and right of the display screen, and the transmission image display screen on the display screen of the video monitor device 21 is
Do not display operation symbols.

FIG. 11 shows a processing example of the system controller 1 in this case.

First, the system control unit 1 sets a detection area corresponding to the operation area (process 501). Next, a predetermined video change detection process (process 50) is performed until a change in the image is detected in the display area of any of the operation symbols.
2; already described) is repeatedly executed (NO loop of determination 503).

When an image change is detected in the display area of any of the operation symbols and the result of determination 503 is YES, the operation content displayed at the change coordinates detected at that time is determined ( After performing a camera operation process corresponding to the determined operation content (process 504) (process 505), the process returns to process 502, and the detection operation of the next camera operation is performed.

In this way, in this embodiment, the user does not need to operate the operation display unit 10 when panning the video camera device 14, so that the operability of the video camera device 14 is greatly improved. . In addition, such a pan operation is a relatively frequent operation, and therefore,
The operability of the video conference communication terminal device can be significantly improved.

In FIG. 10, when displaying the image on the side of the terminal itself, the image taken by the video camera device 14 is displayed by reversing the left and right to provide the user with an image that does not look uncomfortable. This shows a case where so-called mirroring processing is not executed.

Now, as shown in FIG. 12, for example, when a still image such as a catalog is displayed on the video monitor device 21, there is a request to point a part of the displayed content. In this example, a cursor (pointing symbol) is displayed such that the index finger is projected to indicate the target. In this case, the target for superimposing the cursor display is a still image.

In this way, when it is desired to point an arbitrary position of the still image, the image of the pointing cursor can be moved to a desired position simply by moving a hand or the like in the screen of the video camera device 14 in accordance with the pointing position. Since it can be displayed, the operability of such an operation is significantly improved. If a video monitor device for displaying a still image is additionally provided, the operability will be better.

FIG. 13 shows a processing example in this case.

First, it waits for a cursor display request to be commanded (NO loop of judgment 601). When the cursor display request is commanded and the result of judgment 601 becomes YES, a predetermined initial position (for example, , And the cursor is displayed in the upper right corner of the screen (process 602).

Next, a predetermined video change detection process (process 603; described later) is repeatedly executed until a change in the image is detected for any macroblock (decision 60).
4 NO loop).

When an image change is detected for any of the macroblocks and the result of determination 604 is YES, the display position of the cursor is moved to the display position corresponding to the change coordinates detected at that time. (Process 60
5).

Then, it is checked whether or not the end of the cursor display is instructed (decision 606). When the result of the decision 606 is NO, the process returns to the process 603 and the cursor moving process is performed. Further, when the end of the cursor display is instructed and the result of the determination 606 is YES, the cursor is erased (process 607) and this process ends.

FIG. 14 shows a video change detection process (process 60).
3) shows an example.

First, for each macroblock, the reference value table for storing the reference value used when detecting that the video is changing is cleared, and the timer is started (process 701).

Then, for one macroblock,
The difference value is read from the moving picture encoder 28 of the moving picture CODEC 15 (process 702), it is checked whether the value of the read difference value is equal to or more than a predetermined value (decision 703), and when the result of the judgment 703 is YES, Add the value of the difference value at that time to the value of the element of the reference value table corresponding to the macroblock address of the macroblock,
The reference value table is updated (process 704).

Then, it is checked whether or not the reading of the difference value has been completed for one frame (decision 705). When the result of the judgment 705 is NO, the process returns to step 702 and the same processing is repeated for the remaining macroblocks. .

When the result of the judgment 705 is YES, it is checked whether or not the value of the timer is equal to or larger than the predetermined value TA (decision 706). When the result of the judgment 706 is NO, the process returns to the process 702. The same process is repeated for the next frame.

When the value of the timer is equal to or larger than the predetermined value TA and the result of the judgment 706 is YES, the reference value table stores the information necessary for detecting the image change. Of the elements stored in the reference value table, the element with the maximum value is extracted (process 707), and it is checked whether or not the maximum value is equal to or more than a predetermined value (decision 708).

When the result of the judgment 708 is YES, the macroblock address of the macroblock corresponding to the element of the reference value table in which the maximum value is stored is set at the detection position, and the change detection is performed together with the macroblock address. The presence is notified (process 709) and this process ends. When the result of the determination 708 is NO, no change detection is notified (process 710) and this process ends.

FIG. 15 shows a video change detection process (process 60).
Another example of 3) is shown.

First, for each macroblock, the reference value table for storing the reference value used when detecting that the image is changing is cleared and the timer is started (process 801).

Then, for one macroblock,
A motion vector is read from the video encoder 28 of the video CODEC 15 (process 802), the absolute value of the read motion vector is calculated (process 803), and it is checked whether or not the calculated absolute value is equal to or more than a predetermined value ( Judgment 80
4) If the result of determination 804 is YES, the absolute value at that time is added to the value of the element of the reference value table corresponding to the macroblock address of the macroblock, and the reference value table is updated. (Process 805).

Then, it is checked whether or not the reading of the motion vector is completed for one frame (decision 80
6) If the result of determination 806 is NO, process 8
Returning to 02, the same processing is repeated for the remaining macroblocks.

When the result of the judgment 806 is YES, it is checked whether or not the value of the timer is equal to or more than the predetermined value TA (decision 807). When the result of the judgment 807 is NO, the process returns to the process 802. The same process is repeated for the next frame.

When the value of the timer is equal to or larger than the predetermined value TA and the result of the judgment 807 is YES, the reference value table stores the information necessary for detecting the video change. Of the elements stored in the reference value table, the element with the maximum value is extracted (process 808), and it is checked whether or not the maximum value is equal to or greater than a predetermined value (decision 809).

When the result of determination 809 is YES, the macroblock address of the macroblock corresponding to the element of the reference value table in which the maximum value is stored is set at the detection position, and the change detection is performed together with the macroblock address. The presence is notified (process 810) and this process ends. When the result of the determination 809 is NO, no change detection is notified (process 811) and this process ends.

FIG. 16 shows a video change detection process (process 60).
Still another example of 3) is shown.

First, for each macroblock, the reference value table for storing the reference value used when detecting that the video is changing is cleared and the timer is started (process 901).

Then, for one macroblock,
A motion vector is read from the moving picture encoder 28 of the moving picture CODEC 15 (process 902), and the above equation (I) is calculated based on the value of the motion vector read at that time to calculate the motion vector evaluation value MVS (process 903). ).

Then, it is checked whether or not the calculated absolute value is equal to or more than a predetermined value (judgment 904), and the judgment 90
When the result of 4 is YES, the value of the evaluation value MVS at that time is added to the value of the element of the reference value table corresponding to the macroblock address of the macroblock,
The reference value table is updated (process 905).

Then, it is checked whether or not the reading of the motion vector is completed for one frame (decision 90
6), when the result of determination 906 is NO, processing 9
Returning to 02, the same processing is repeated for the remaining macroblocks.

When the result of the judgment 906 is YES, it is checked whether or not the value of the timer is equal to or more than the predetermined value TA (decision 907). When the result of the judgment 907 is NO, the process returns to the process 902. The same process is repeated for the next frame.

When the value of the timer is equal to or larger than the predetermined value TA and the result of the judgment 907 is YES, the reference value table stores the information necessary for detecting the image change. Of the elements stored in the reference value table, the element with the maximum value is extracted (process 908), and it is checked whether or not the maximum value is equal to or greater than a predetermined value (decision 909).

When the result of judgment 909 is YES, the macroblock address of the macroblock corresponding to the element of the reference value table in which the maximum value is stored is set at the detection position, and the change detection is performed together with the macroblock address. The presence is notified (process 910), and this process ends. When the result of the determination 909 is NO, no change detection is notified (process 911), and this process ends.

In the above-mentioned embodiment, the case where the present invention is applied to the video conference communication terminal device shown in FIG. 1 has been described, but the present invention is similarly applied to other audiovisual communication terminal devices. Can be applied.

[0104]

As described above, according to the present invention,
The device can be operated simply by waving its hand toward the video input means, which greatly reduces the time and effort required for operating the device and greatly improves the operability of the audiovisual communication terminal device. Get the effect that you can.

Further, since the device operation can be performed only by waving the hand toward the video camera device so as to overlap with the operation symbol displayed on the video monitor screen, the device operation can be performed. This has the effect of significantly reducing the labor and improving the operability of the audiovisual communication terminal device.

Since the weighting of the vertical component of the motion vector is made larger than that of the horizontal component, the evaluation value used for detecting the change of the image is calculated. The effect of noise that causes erroneous detection can be eliminated, and the detection accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a video conference communication terminal device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing an example of a screen of a video monitor device.

FIG. 3 is a block diagram showing an outline of a moving image CODEC.

FIG. 4 is a schematic diagram for explaining a macroblock.

FIG. 5 is a schematic diagram showing an example of a case where the display of operation symbols on the video monitor device is superimposed.

FIG. 6 is a flowchart showing an example of processing executed by a system control unit.

FIG. 7 is a flowchart showing an example of video change detection processing.

FIG. 8 is a flowchart showing another example of the image change detection process.

FIG. 9 is a flowchart showing still another example of the image change detection process.

FIG. 10 is a schematic diagram showing a screen display example according to another embodiment of the present invention.

FIG. 11 is a flowchart showing another example of processing executed by the system control unit.

FIG. 12 is a schematic diagram showing a screen display example according to still another embodiment of the present invention.

FIG. 13 is a flowchart showing an example of processing executed by the system control unit in the case of the embodiment of FIG.

FIG. 14 is a flowchart showing another example of video change detection processing.

FIG. 15 is a flowchart showing another example of video change detection processing.

FIG. 16 is a flowchart showing still another example of the image change detection process.

[Explanation of symbols]

 1 System control unit 15 Video CODEC 28 Video encoder

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location H04N 7/24 9466-5K H04L 11/02 C // G10K 15/04 302 H04N 7/13 Z

Claims (9)

[Claims] 1. An audiovisual communication terminal device having a communication function for exchanging at least audio information and video information, wherein video input means for inputting a video image, and a video signal output from the video input means are provided. Divide into predetermined blocks, obtain the difference information from the previous frame for each block, encode and compress the moving image information for each block based on the difference information, and change the difference information for each block And a moving picture coding means for detecting the moving picture coding means for calculating position coordinates corresponding to the block in which the difference information is detected to have changed,
An audiovisual communication terminal device comprising control means for determining that a predetermined device operation has been performed based on the position coordinates. 2. An audiovisual communication terminal device having at least a communication function for exchanging audio information and video information, wherein a video input means for inputting a video image and a video signal output from the video input means are predetermined. The block is divided into blocks, the motion vector based on the difference from the previous frame is obtained for each block, and the video information is coded and compressed for each block based on the motion vector, and the video content of each block is based on the motion vector. Is calculated by the moving picture coding means, and the moving picture coding means calculates the position coordinates corresponding to the block in which the changing of the video content is detected,
An audiovisual communication terminal device comprising control means for determining that a predetermined device operation has been performed based on the position coordinates. 3. An audiovisual communication terminal device having at least a communication function for exchanging audio information and moving image information, wherein a predetermined image input means for inputting a moving image and a video signal output from the image input means are provided. The block is divided into blocks, the motion vector based on the difference from the previous frame is calculated for each block, and the moving image information is coded and compressed for each block based on the motion vector, and the motion vector corresponding to the absolute value of the motion vector A moving picture coding means for detecting that the video content of each block has changed based on the evaluation value, and position coordinates corresponding to the block for which the moving picture coding means has detected that the video content has changed And calculate
Based on the position coordinates, a control means is provided to determine that a predetermined device operation has been performed, and the moving picture coding means weights the vertical component of the motion vector more than the horizontal component. An audiovisual communication terminal device, wherein an evaluation value of the motion vector is calculated in a state. 4. An audiovisual communication terminal device having a communication function for exchanging at least audio information and video information, wherein video input means for inputting a video image, and a video signal output from the video input means are provided. Divide into a predetermined block, determine the difference information with the previous frame in block units, code the moving image information in block units based on the difference information, and based on the difference information obtained during a predetermined time A moving picture coding unit that detects that the video of each block has changed, and the above moving picture coding unit calculates position coordinates corresponding to the block in which the difference information is detected to change,
An audiovisual communication terminal device comprising control means for determining that a predetermined device operation has been performed based on the position coordinates. 5. An audiovisual communication terminal device having a communication function for exchanging at least audio information and moving picture information, wherein a video input means for inputting a moving picture and a video signal output from the video input means are predetermined. The block is divided into blocks, the motion vector based on the difference from the previous frame is obtained in the block unit, and the moving image information is coded and compressed in the block unit based on the motion vector, and the motion vector obtained during a predetermined time. The moving picture coding means for detecting that the video content of each block is changed based on the above, and the position coordinates corresponding to the block where the video content is detected to be changed by the moving picture coding means. Calculate,
An audiovisual communication terminal device comprising control means for determining that a predetermined device operation has been performed based on the position coordinates. 6. An audiovisual communication terminal device having at least a communication function for exchanging audio information and moving picture information, wherein a video input means for inputting a moving picture and a video signal output from the video input means are predetermined. The block is divided into blocks, the motion vector based on the difference from the previous frame is obtained in the block unit, and the moving image information is coded and compressed in the block unit based on the motion vector, and the motion vector obtained during a predetermined time. The moving picture coding means for detecting the change in the video content in each block based on the motion vector evaluation value corresponding to the absolute value of, and the moving picture coding means for detecting the change in the video content Position coordinates corresponding to the selected block,
The moving picture coding means includes a control means for judging that a predetermined device operation is performed based on the position coordinates, and the moving picture coding means weights the vertical component of the motion vector more than the horizontal component. An audiovisual communication terminal device, wherein an evaluation value of the motion vector is calculated in a state. 7. The apparatus according to claim 1, wherein the control means superimposes and displays an operation symbol on a display screen, and determines the device operation content based on the display coordinates of the operation symbol. The audiovisual communication terminal device according to claim 3, claim 4, claim 5, or claim 6. 8. The predetermined device operation is a shooting screen control operation of the video input means, claim 1, claim 2, claim 3, claim 4, claim 5, or. The audiovisual communication terminal device according to claim 6. 9. The method according to claim 1, wherein the predetermined device operation is screen indication symbol display. Audiovisual communication terminal equipment.