JPH09182043A - Visual telephone having movement tracking function - Google Patents

Abstract

PROBLEM TO BE SOLVED: To always keep a moving object in a sharp area and to suppress the useless information by deciding a new sharp area based on the motion vector of the block that is previously decided in the original sharp area. SOLUTION: An image encoding part 14 encodes an image frame stored in a frame storage part 12 and detects the motion vector of the block forming the image frame. A sharp area decision part 16 sets a boundary between a sharp area and other areas (deteriorated areas) based on the motion vector of a prescribed block among those motion vectors of a detected object and then outputs the position information of the boundary. An encoding control part 15 colors as designated to the background part (deteriorated area) of the image frame via a background input part 13 and based on the position information of the preceding boundary. At the same time, a correction code of high correction capability is given to the sharp area with a correction code of low correction capability given to the deteriorated area via an error correction control part 17 respectively. Otherwise, the control is carried out not to encode the image frame and the quantity of information to be transferred can be reduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a videophone, and more particularly to a videophone with a motion tracking function that moves a clear area in a screen according to the movement of a subject.

[0002]

[Prior art]

(First Conventional Example) A conventional videophone in which an image frame is divided into a clear area and a degraded area and transmitted to the videophone of the other party will be described below. By preliminarily distinguishing the inside of the screen into the subject portion and the background portion, and making the bandwidth of the background portion narrower than the bandwidth of the subject portion,
The amount of information generated in the background portion can be reduced. A method and apparatus for processing a moving image signal that smoothes the motion of a subject part by increasing the amount of information of the subject part by reducing the amount of information generated in the background part (Japanese Patent Laid-Open No. 2-20).
6293) has been devised. FIG. 19 is a diagram showing the structure of the moving image signal processing apparatus. In FIG. 19, the moving image signal processing device includes a control circuit, a time filter, and an encoder. The control circuit applies a background pattern, which is predetermined and selected from the background patterns stored in the ROM, to the temporal filter in synchronization with the input moving image signal. At this time, the control circuit controls the time filter such that the background part is strongly band-limited and the subject part is relatively lightly band-limited. As a result, it is possible to suppress useless information generated by movement of the background or the like other than the point of interest, and instead smooth the movement of the subject that is the point of interest.

(Second Conventional Example) A conventional videophone having a function of inputting an operation command by a non-contact operation will be described below. Point-of-view monitoring input device that enables input to the information processing device only by looking at the screen
No. 4526), or when the operator touches a virtual reality image in a space where the virtual reality image is formed, by outputting a signal indicating that this virtual reality image is selected, Virtual reality space data input device capable of non-contact input (Japanese Patent Laid-Open No. 6-274270)
Has been devised. FIG. 20 is a diagram showing the configuration of the viewpoint monitoring input device. 20, the viewpoint monitoring input device includes a video device 1, a display device 2, and an image processing device 3.
And a control device 4. The viewpoint monitoring input device in FIG. 20 detects the position of the operator's viewpoint by displaying the operator's eyes looking at the screen of the display device 2 with the video device 1 and analyzing the projected image with the image processing device 3. Then, it specifies which position on the display screen the operator has selected. Then, the control device 4 converts the display information displayed at that position into information indicating the input intention of the operator, and transmits the information to the device that performs information processing. The position of the operator's current viewpoint and the information immediately before the input are displayed on the screen of the display device 2.

FIG. 21 is a diagram showing the configuration of the virtual reality space data input device. In FIG. 21, the virtual reality space data input device includes a display unit 5, an image input device 6, a 3D control device 7, and a 3D eyeset 8. Display unit 5 of FIG.
Displays color image data on the display screen 5A. The image input unit of the image input device 6 includes an artificial retina element and an optical lens, captures an image of the front area 5B in front of the display screen 5A from above, recognizes the operator's finger, and detects the finger of the operator. Identify the location. 3D eye set 8 has 3
It is connected to the D control device 7 and controls the visual image of the operator. The 3D control device 7 causes the operator to visually recognize the virtual reality image by controlling the image displayed by the display unit 5 and the transmitted light amount of the 3D eyeset 6, respectively. Then, the selection item pointed by the operator's finger in the virtual reality space is output as a selection signal.

(Third conventional example) In many cases, a conventional videophone having a picture-in-picture function displays an image received from the other party on the main screen, and an image transmitted by itself is displayed on the main screen. It is displayed on a sub-screen at a predetermined position within.

[0006]

[Problems to be Solved by the Invention]

(Problems in First Conventional Example) The following problems exist in the video telephone in the first conventional example. The screen is divided into a clear area and a degraded area in advance, and their positions are fixed, so if the subject moves out of the clear area due to movement of the subject or camera shake, that is, the subject moves to the degraded area. If this happens, the intensity of the subject is band-limited and the image quality deteriorates.

Therefore, an object of the present invention is to provide a videophone capable of always capturing a subject within the clear region by moving the clear region according to the movement of the subject.

(Problems in Second Conventional Example) The video telephone in the second conventional example has the following two problems. The first problem is as follows. In the videophone having the function of inputting the operation command by the non-contact operation, the image device 1 needs to compose and focus on the eyes of the operator who is looking at the screen of the display device 2. Normally, it is difficult to use a videophone camera that shows the entire upper half of the operator's upper body together for the purpose of viewpoint monitoring. Therefore, the videophone requires the dedicated video device 1. In addition, even when the dedicated video device 1 is provided,
The operator needs to stay still at a predetermined position in front of the video device 1. In particular, when the present videophone is a portable videophone, it is difficult to always keep the operator's eyes and the video device 1 in a predetermined positional relationship. Further, the virtual reality space data input device needs to include an image input device using an expensive artificial retina element. Second
The challenges of are as follows. The viewpoint input device requires the image processing device 3 in order to specify the position of the operator's viewpoint from the projected image. In the virtual reality space data input device, the artificial retina element recognizes the operator's finger, but the artificial retina element performs special image processing for recognition.

Therefore, another object of the present invention is to provide a dedicated video input device and perform no special image processing,
An object of the present invention is to provide a non-contact operation input device capable of inputting a command in a non-contact operation by utilizing the functions of an image input device and an image compression device included in a general videophone.

(Problems in Third Conventional Example) The video telephone in the third conventional example has the following problems. Since the display position of the small screen is fixed, in a videophone having a picture-in-picture function, when the clear area is moved, the clear area may be hidden by the small screen.

Therefore, another object of the present invention is to provide a videophone having a picture-in-picture function so that the clear area is not hidden by the small screen even when the clear area is moved. .

[0012]

Means for Solving the Problems and Effects of the Invention The structure of the present invention for achieving the above object will be described below.
In order to clarify the correspondence relationship with the embodiments described later, each constituent element adopted in the present invention is provided with the corresponding reference numeral. However, it should be pointed out in advance that the reference numbers are given only for easy understanding and for reference, and do not limit the scope of the claims of the present invention.

According to the first aspect of the present invention, a moving object can always be captured in a clear area by determining a new clear area based on a motion vector of a predetermined block in the clear area. An image input means (11) for inputting a moving image, which is a videophone with a motion tracking function.
A frame storage means (12) for storing an image frame input by the image input means (11), and an image frame stored in the frame storage means (12) are encoded,
Further, based on the image coding means (14) for detecting the motion vector of the block forming the frame and the motion vector of the predetermined block among the motion vectors detected by the image coding means (14). , A predetermined area in the image frame stored in the frame storage means (12) is determined as a clear area for displaying the subject,
A clear area determination means (16) for outputting position information of a clear area / degraded area of the frame, and a clear area determination means (1
6) encoding control means (15) for controlling the image encoding means (14) so as to encode the deteriorated area of the frame with the image quality deteriorated more than that of the clear area; And a communication unit (18) for transmitting the image frame encoded by the image encoding unit (14).

As described above, according to the first aspect of the invention, the image input means inputs the image frame including the subject and the background portion. The frame storage means stores the image frame input by the image input means. The image encoding means encodes the frame stored in the frame storage means and detects a motion vector of a block forming the frame. The sharp area determination means determines a new clear area based on the motion vector of a predetermined block in the clear area among the motion vectors detected by the image coding means, and position information of the clear area / degraded area. Is output. The encoding control means controls the image encoding means based on the position information output by the clear area determination means. The image encoding means encodes the deteriorated area with the image quality deteriorated as compared with the clear area under the control of the encoding control means. The communication means transmits the image frame encoded by the image encoding means.

As described above, the amount of information generated from the deteriorated area can be reduced by distinguishing the image frame into the clear area / the deteriorated area. Further, by determining a new clear region based on the motion vector of a predetermined block in the clear region, even when the subject moves, the subject can always be kept in the clear region.

A second invention is characterized in that, in the first invention, the coding control means (15) controls the image coding means (14) so as not to code the deteriorated area of the frame. .

As described above, in the second invention according to the first invention, the coding control means controls the image coding means so as not to code the deteriorated area. As a result, since the deteriorated area is not encoded, the amount of generated information can be further reduced. Also, since the background portion is not transmitted, the privacy of the caller can be protected.

In a third aspect based on the first aspect, a background color input means (13) for coloring the image frame stored in the frame storage means (12) is further provided, and an encoding control means (15). Is the background color input means (1
3) is characterized by performing control so that the deteriorated area of the frame is colored based on the position information output by the clear area determination means (16).

As described above, in addition to the first invention, the third invention further comprises background color input means. The encoding control means further controls the background color input means to color the degraded area. Thus, by filling the background portion with the same color, the amount of generated information can be reduced. Further, since the background portion is filled with the designated color, the appearance of the screen can be improved and the privacy of the caller can be protected.

A fourth invention is the same as the first to third inventions,
Error correction control means (17) for adding an error correction code to the image data coded by the image coding means (14)
The coding control means (15) further includes an error correction control means (17), and a sharp area determination means (16).
Based on the position information output by, the control is performed such that an error correction code with high error correction ability is added to a clear area and an error correction code with low error correction ability is added to a deteriorated area. There is.

As described above, the fourth invention further comprises error correction control means in addition to the first to third inventions.
In the encoding control means, the error correction control means further applies an error correction code having a high error correction ability to the image data in the clear area and an error correction code having a low error correction ability to the image data in the deteriorated area. Control to add. In this way, by adding the error correction code to the encoded image data, it is possible to prevent the image quality from being deteriorated only in the clear area even when an error occurs in the transmission path. In addition, the amount of information to be transmitted can be reduced by adding a correction code having a lower error correction capability than the clear region, that is, by adding a short correction code to the degraded region.

A fifth aspect of the invention is the same as the first to fourth aspects of the invention.
The position in the clear region of the block from which the clear region determining means (16) acquires the motion vector is determined in relation to the position of the clear region in the image frame.

As described above, in the fifth invention, the first to fourth
In the invention, the position within the clear region of the block for which the clear region determining means acquires the motion vector via the image coding means is such that the object is within the clear region according to the position of the image frame in the clear region. It is set in such a position that it will always be held. Accordingly, even when the subject moves, the clear area can follow the movement of the subject while always capturing the subject inside itself.

A sixth aspect of the invention is the same as the first to fourth aspects of the invention.
The position in the clear region of the block for which the clear region determining means (16) acquires a motion vector is located at the left end of the clear region and at the left end of the image frame when the clear region is located at the right end of the image frame. If it is located, it is defined at the right end of the clear area, at the lower end of the clear area if it is located at the upper edge of the image frame, and at the upper end of the clear area if it is located at the lower edge of the image frame. I am trying.

As described above, in the sixth invention, the first to fourth
In the invention, the position within the clear region of the block for which the clear region determining means obtains the motion vector is such that the subject moves from near the center of the image frame to the right end of the image frame,
Further, it is defined at the left end of the sharp area in order to correspond to the case of protruding outside the right side of the image frame and returning to the inside from the right outside of the frame again. That is, the clear region is at the right end of the image frame according to the movement of the subject. At this time, if the block for detecting the motion vector is in the center of the clear region, the motion vector at the left end of the subject returning from the right side into the frame is detected and the movement is started. Therefore, the clear region follows the movement of the subject while capturing only the left end of the subject.

Therefore, in the above case, if the block for detecting the motion vector is set at the left end of the clear area, the object can be always captured in the center of the clear area. Similarly, even if the subject extends to the left outer side, the upper outer side, and the lower outer side of the image frame, setting the block for detecting the motion vector at the position as described above will always capture the subject in the center of the clear area. it can.

A seventh aspect of the present invention is a videophone with a motion tracking function on the receiving side, which displays only a clear area of a received image frame without degrading the image quality and displays the deteriorated area in color. A communication means (18) for receiving the coded image frame, decoding the coded image frame received by the communication means (18), and detecting a motion vector of a block forming the frame. Image decoding means (701) and image decoding means (70)
Frame storing means (12) for storing the image frame obtained by the decoding of 1), and image decoding means (701)
A clear area determination means (704) for determining a clear area of the image frame stored in the frame storage means (12) on the basis of the motion vector detected by, and outputting position information of the clear area / degraded area of the frame. And a background color input means (13) for coloring the image frame stored in the frame storage means (12) with a designated color, and the position information output by the clear area determination means (704). A decoding control means (703) for controlling the background color input means (13) and an image display means (702) for displaying the image frames stored in the frame storage means (12) so as to color the deteriorated area of I have it.

As described above, in the seventh aspect, the communication means receives the image frame transmitted by the transmitting side terminal. The image decoding means decodes the image frame received by the communication means and detects a motion vector of a block forming the image frame. The frame storage means stores the image frame decoded by the image decoding means. The clear area determination means outputs the position information of the clear area / degraded area based on the motion vector detected by the image decoding means. The decoding control means controls the background color input means based on the position information output by the clear area determination means. The background color input means colors the deteriorated area of the frame stored in the frame storage means under the control of the decoding control means. The display means displays the image frame stored in the image frame storage means.

As described above, since the background portion is filled with the designated color, the appearance of the screen can be improved and
The privacy of the caller can be protected. Also, by performing coloring on the receiving side, the coding processing on the transmitting side can be reduced. Furthermore, when connected to a videophone with a motion tracking function that encodes and transmits only the clear area, the receiving side videophone colors the degraded area, and therefore it is not necessary to transmit information regarding the color of the degraded area. Therefore, the amount of information to be transmitted is smaller than that in which the transmitting videophone performs coloring.

An eighth aspect of the present invention is a videophone having a picture-in-picture function for superimposing a small screen on a received image frame and displaying the image frame, and communicates with a communication means (18) for receiving the encoded image frame. An image decoding means (701) for decoding the encoded image frame received by the means (18) and detecting a motion vector of a block forming the frame, and an image decoding means (70).
Based on the motion vector detected by 1), the image decoding means (701) determines the clear area in the image frame decoded, and the clear area determining means (704) outputs the position information of the clear area of the frame. ) And the picture-in-picture position determining means (1303) that determines the position of the child screen based on the position information output by the clear area determination means (704) and outputs the position information of the child screen. Image input means (11) for inputting the image displayed on the screen and the image transmitted to the other party's videophone terminal, the position information output by the clear area determination means (704), and the picture-in-picture position determination means ( The image frame obtained by decoding by the image decoding means (701) based on the position information output by 1303) and displayed on the child screen input by the image input means (11). Picture-in-picture creating means (1301) and a picture-in-picture creating means (1301) for creating a picture-in-picture image including a child screen at the position determined by the picture-in-picture position determining means (1303). Image display means (702) for displaying the picture-in-picture image created by the above (1).

As described above, in the eighth aspect of the invention, the communication means receives the encoded image frame transmitted by the transmitting videophone. The image decoding means decodes the coded image frame received by the communication means and detects a motion vector of a block forming the frame. The clear area determination means determines a clear area based on the motion vector detected by the image decoding means, and outputs position information of the clear area. The image input means inputs an image displayed on the child screen and an image transmitted to the other party's videophone terminal. The picture-in-picture position determining means determines the position of the child screen within the image frame based on the position information output by the clear area determining means. The picture-in-picture creating means, based on the position information of the child screen output by the picture-in-picture position determining means, the image frame (parent screen) decoded by the image decoding means, and the child input by the image input means. The image displayed on the screen is combined with each other to output a picture-in-picture image including the child screen at the position determined by the picture-in-picture position determining means. The image display means displays the picture-in-picture image created by the picture-in-picture creating means. In this way, by moving the display position of the small screen in accordance with the position of the clear region, it is possible to prevent the clear region from being hidden by the small screen.

A ninth aspect of the present invention is a videophone having a picture-in-picture function for superimposing a small screen on a received image frame and displaying the image frame, and communicates with a communication means (18) for receiving the encoded image frame. The image decoding means (701) for decoding the encoded image frame received by the means (18) and the blocks constituting the image frame decoded by the image decoding means (701) are different from each other. It is determined whether or not the information is included, and when the difference information is included, the block is determined as a valid macroblock,
Based on the effective macroblock discriminating means (1302) which outputs the positional information of the block and the effective macroblock positional information which is output by the effective macroblock discriminating means (1302), the position of the small screen is determined, In-picture position determining means (1
303), an image displayed on the child screen, and an image transmitted to the other party's videophone terminal, image input means (11), and effective macroblock discrimination means (1302).
The image frame obtained by decoding by the image decoding means (701) based on the position information output by the image in-picture position determining means (1303) and the position information output by the picture-in-picture position determining means (1303), and the image input means (11). A picture-in-picture creating unit (1301) that creates a picture-in-picture image including the child screen at the position determined by the picture-in-picture position determining unit (1303) by combining the image displayed on the child screen input by And an image display means (702) for displaying the picture-in-picture image created by the picture-in-picture creating means (1301).

As described above, in the ninth invention, the communication means receives the image frame transmitted and encoded by the transmitting videophone. The image decoding means decodes the encoded image frame received by the communication means. The effective macroblock determination means determines whether or not each block forming the image frame decoded by the image decoding means has difference information, and determines that the block having difference information is an effective macroblock. Then, the position information of the block is output. The image input means inputs an image displayed on the small screen and an image transmitted to the other party's videophone. The picture-in-picture position determining means determines the position of the child screen in the image frame based on the position information of the valid macroblock output by the valid macroblock determining means. The picture-in-picture creating means displays the image frame (parent screen) decoded by the image decoding means on the basis of the position information output by the picture-in-picture position determining means, and the child screen input by the image inputting means. The picture-in-picture image including the child screen is output at the position determined by the picture-in-picture position determining means. The image display means displays the picture-in-picture image created by the picture-in-picture creating means. Since many effective macroblocks are detected at the position of the subject, by thus moving the display position of the sub-screen according to the position of the effective macroblock, the subject is prevented from being hidden by the sub-screen. be able to.

According to a tenth aspect of the present invention, the pixel located at a predetermined position in the image frame is included in a clear region which moves following the motion of the operator, so that the operator can make an input corresponding to the pixel. A non-contact operation input device capable of non-contact selection of items, the image input unit (11) for inputting a moving image, and a frame for storing an image frame input by the image input unit (11). Storage means (1
2) and an image coding means (14) for coding the image frame stored in the frame storage means (12) and detecting a motion vector of a block forming the frame.
And a predetermined area in the image frame stored in the frame storage means (12) is determined as a clear area for displaying the subject based on the motion vector output from the image encoding means (14), A clear area determining unit (16) that outputs position information of a clear area of the frame, and position information of one or more pixels in the image frame are stored, and the position information of the pixel and the clear area determining unit (16) are used. Based on the output position information, it is determined whether or not the clear region includes the pixel, and if it includes, the region position determining unit (905) that outputs the position information of the pixel and the region position determining unit ( Operation control means (906) for selecting an operation command in accordance with the position information output by (905).

As described above, in the tenth aspect, the image input means inputs a moving image. The frame storage means
The image frame input by the image input means is stored. The image encoding means encodes the image frame stored in the frame storage means and detects a motion vector of a block constituting the image frame. The clear area determination means determines a predetermined area of the frame stored in the frame storage means as a clear area based on the motion vector detected by the image coding means, and outputs the position information. The area position determination means compares the position information of the clear area output by the clear area determination means with the position information of the specific pixel in the image frame stored in itself, and the clear area includes the specific pixel. If it is included, the position information of the pixel is output. The operation control means selects an operation command according to the position information output by the area position determination means.
In this way, the clear area is moved according to the movement of the operator so that the clear area includes a specific pixel. At this time, by transmitting the position information of the pixel, a command corresponding to the position information of the pixel can be input in a non-contact operation.

An eleventh invention is the tenth invention, wherein
A character / graphic representing an input item at a position including the pixel based on the position information output by the clear area determination unit (16) and the position information of the pixel stored in the region position determination unit (905). For information, create an operation screen that displays the boundary of the clear area / degraded area at the position of the clear area,
Operation screen output means (902) for outputting the operation screen
And an image display unit (702) for displaying the operation screen output by the operation screen output unit (902).

As described above, in the eleventh invention, the tenth invention is provided.
The invention further includes an operation surface output means and an image display means. The operation screen output means represents an input command at the position of a specific pixel based on the position information of the clear area output by the clear area determination means and the position information of the specific pixel stored in the area position determination means. Display characters and figures,
Further, an operation screen is also displayed in which selection areas for selecting those characters and figures are displayed in an overlapping manner. The image display means displays the operation screen created by the operation screen output means. Thereby, the input operation of the operator can be visualized, and thus the non-contact input can be performed more easily.

[0038]

BEST MODE FOR CARRYING OUT THE INVENTION

(First Embodiment) A first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the configuration of a videophone with a motion tracking function according to the first embodiment of the present invention. In FIG. 1, a videophone with a motion tracking function includes an image input unit 11 and a frame storage unit 1.
2, a background color input unit 13, an image coding unit 14, a coding control unit 15, a clear area determination unit 16, an error correction control unit 17, and a communication unit 18.

The image input section 11 in FIG. 1 inputs an image frame. The frame storage unit 12 stores the image frame input from the image input unit 11. Background color input section 1
In No. 3, the background portion (deteriorated area) of the image frame stored in the frame storage unit 12 is colored with the designated color. That is, the information representing the designated color is stored in the block corresponding to the deteriorated area among the blocks forming the image frame stored in the frame storage unit 12. The image encoding unit 14 encodes the image frame stored in the frame storage unit 12 and detects a motion vector of a block forming the frame. Here, ITU Recommendation H.264. The encoding method defined by H.261 is assumed. The clear area determination unit 16 includes the image encoding unit 1
Based on the motion vector of a predetermined block among the motion vectors output by 4, the boundary between the clear region and the other region (deteriorated region) is set, and the position information is output. The error correction control unit 17 adds an error correction code having a high error correction ability to the clear area and an error correction code having a low error correction ability to the deteriorated area. The encoding control unit 15 determines whether the background color input unit 13, the image encoding unit 14, and the error correction control unit 17 are suitable for the clear region / deteriorated region based on the position information output by the clear region determination unit 16. Control to perform processing. The communication unit 18 transmits the image data output by the error correction control unit 17 at the designated transmission rate.

The coding method of the image coding unit 14 of FIG. 1 will be described below with reference to FIG. FIG. 2 shows FIG.
It is a figure which shows the structure of the image coding part 14 of FIG. In FIG. 2, a digitized moving image signal composed of a plurality of temporally continuous frames is input from a moving image input terminal 141. The conversion unit 142 uses the image signal input terminal 1
The moving image frame input from 41 is divided into blocks of 8 pixels * 8 pixels and output in block units. As a result, one frame includes 22 in the X direction and 18 in the Y direction (the image size is QCIF: Quater Common Int
ermediate Format). In the following processing, in the case of a color image, a macro block including six blocks (four luminance blocks having luminance information and two color difference blocks having color difference information) is used as a processing unit. Hereinafter, this macro block is simply called a block.

The frame memory 14d has a storage area for storing images for two frames. Output from the adder 14b to the storage area for one frame of these,
The currently encoded frame is stored in block units, and the reproduction frame (reference frame) immediately preceding the above frame is stored in the storage area for another frame. The motion vector search unit 14e searches the reference frame stored in the frame memory 14d for a block most similar to the block output by the conversion unit 142 (that is, a block having the smallest difference error), and the searched block. The vector (motion vector) that points to is output. By the way, ITU Recommendation H.264. In H.261, the motion vector search range is defined to be within ± 15 pixels around the target block.

The motion compensation unit 14f determines which block in the reference frame stored in the frame memory 14d corresponds to the currently encoded block based on the motion vector output from the motion vector search unit 14e. The image data (predicted value) of the predicted and predicted block is output to the difference unit 144 and the addition unit 14b. The difference unit 144 obtains the difference between the image data of the encoding target block output by the transform unit 142 and the prediction value provided from the motion compensation unit 14f, and outputs the obtained difference to the orthogonal transform unit 146. The difference is called a prediction error because it is the difference between the image data of the block to be encoded and the prediction value. The orthogonal transformation unit 146 orthogonally transforms the prediction error output by the difference unit 144 and outputs the prediction error to the quantization unit 147. The quantizer 147 quantizes the orthogonally transformed prediction error,
It outputs to the inverse quantization unit 148 and the variable length coding unit 14a. The dequantization unit 148 dequantizes the quantized prediction error and outputs the dequantized prediction error to the inverse orthogonal transform unit 149. The inverse orthogonal transformation unit 149 inversely orthogonally transforms the prediction error that has been subjected to the orthogonal transformation, and outputs it to the addition unit 14b.

The adding unit 14b reproduces the coding target block by adding the prediction error output from the inverse orthogonal transforming unit 149 and the prediction value output from the motion compensating unit 14f, and the reproduced coding target block. The frame memory 14
Output to d. The variable length coding unit 14a includes a quantization unit 14
The orthogonally-transformed and quantized prediction error output from 7 is variable-length coded and output to the multiplexing unit 14i. The differential motion vector detection unit 14g includes the motion vector search unit 1
4e outputs the motion vector of the encoding target block,
Find the difference with the motion vector of the block to the left of it,
It outputs to the motion vector variable length coding unit 14h. The motion vector variable length coding unit 14h performs variable length coding on the differential motion vector output by the differential motion vector detection unit 14g, and outputs the variable motion vector to the multiplexing unit 14i.

The coding control unit 143 switches the connection between the switch 145 and the switch 14c according to the magnitude of the prediction error output by the difference unit 144. That is, the encoding control unit 143 sets the switch 145 to 1-3 if the output prediction error is larger than a preset value.
Connection, and switch 14c is switched to connection 1-3. At this time, the image encoding unit 14 encodes the encoding target block in the intraframe mode. If the prediction error is smaller than the preset value, switch 145 to 2-3 connection, switch 14
c is switched to connection 1-2. At this time, the image encoding unit 14 encodes the encoding target block in the interframe mode.

The coding control unit 143 sets the quantized value in consideration of various factors at the time of coding the target block, and outputs the set quantized value to the quantizing unit 147. The side information such as the coding mode identification code indicating whether the block is coded in the inter-frame mode or the intra-frame mode and the quantized value are output to the multiplexing unit 14i. The multiplexing unit 14i includes a prediction error output from the variable length encoding unit 14a, a variable length encoded difference motion vector output from the motion vector variable length encoding unit 14h, and a side output from the encoding control unit 143. Information is multiplexed and output to the code output terminal 14j.

A method of determining the clear area by the clear area determining unit 16 of FIG. 1 will be described below with reference to FIGS. FIG. 3 shows the clear area determination unit 1 of FIG.
It is a figure which shows the structure of 6. In FIG. 3, the clear area determination unit 16 includes a clear area control unit 161 and a motion vector storage unit 162. FIG. 4 is a diagram showing the storage contents of the motion vector storage unit 162 of FIG. As shown in FIG. 4, the motion vector storage unit 162 stores the block address and motion vector of each block.

In FIG. 3, the clear area control section 161
When the motion vector of the block used for moving the clear area is input from the image coding unit 14, it is determined whether or not to move the clear area based on the motion vector of the block,
If it is necessary to move the clear area, a new boundary between the clear area and the deteriorated area is newly set, and the position information thereof is encoded by the encoding control unit 1.
Notify 5 The image encoding unit 14 includes an encoding control unit 15
When each block is coded under the control of 1, the coding mode identification code, the motion vector, and the block address of each block are output to the clear area determination unit 16. The clear area determination unit 16 uses the clear area control unit 161 to
Of the coding mode identification code, the motion vector and the block address of each block output from the image coding unit 14, the block address and the motion vector are stored in the motion vector storage unit 162.

Next, the manner in which the clear area follows the movement of the object is divided into two cases, one is when the block used for moving the clear area is encoded in the intra-frame mode and the other is encoded in the inter-frame mode. A description will be given with reference to 5 and 6. FIG. 5 shows a state in which the clear region moves within the frame when the block used for moving the clear region is encoded in the intra-frame mode, and FIG. 6 shows the block used for moving the clear region in the inter-frame mode. It is a figure which respectively shows a mode that a clear region moves in a frame in the case of being converted. Here, for simplicity, the direction of the motion vector is limited to the right. Further, in FIGS. 5 and 6, a shaded area represents a block used for moving the clear area.

As shown in FIG. 5, when the block used for moving the clear region is coded in the intra-frame mode, no motion vector is generated in the block.
The clear area cannot be moved. Therefore, the moving direction is determined based on the motion vector of the block before the block used for moving the clear region. That is, the motion of the subject is detected by the average of the motion vectors of the blocks stored in the motion vector storage unit 162 of FIG. 3 and used for moving the clear region, and the clear region is moved.

Further, as shown in FIG. 6, when the block used for moving the clear area is coded in the inter-frame mode, the time when the total motion vector of the block exceeds a predetermined threshold value. Use to move the clear area. ITU
Recommendation H. In H.261, since the area of one block (macro block) is defined as 16 * 16 pixels, the threshold value is set to ± 15. (1), (2) and (3) of FIG. 6 are temporally consecutive image frames,
The motion vectors of the blocks used for moving the clear region are +3, +10, and +5, respectively. Therefore,
In (2), the total motion vector of the flock is +13.
Therefore, the subject has moved by +13 pixels. Since the threshold value is set to ± 15 pixels, the clear area is not moved at the time of (2). In (3), since the total motion vector is +18, the clear region is moved to the right by one block at this point. When you move the clear area,
The motion vector sum is reset to zero.

A method of making a clear region follow the movement of a subject based on a motion vector will be described below with reference to FIG.
A description will be given with reference to 8, 9 and 10. FIG. 7 is a diagram showing how a clear area that captures a subject follows the movement of the subject. In FIG. 7, a black-painted area within the clear area indicates an area in which a motion vector used for moving the clear area is detected. The same applies to FIGS. 8, 9 and 10. Hereinafter, this area is simply referred to as a detection area. FIG. 8 is a diagram showing a state in which the clear region moves in accordance with the movement of the subject when the subject moved as shown in FIG. 7 further moves, once protrudes to the right side of the screen, and then returns to the inside of the screen again. is there. Since the clear area moves according to the motion vector of the detection area, when the object is captured within the clear area as shown in FIG. 7, the clear area follows the movement of the object while capturing the object.

However, as shown in FIG. 8, when the subject once protrudes to the right side of the screen, there are the following problems. In FIG. 8, the clear region is moving to the right end along with the movement of the subject, and the subject further extends to the right side of the image frame. When the subject comes back into the image frame from the right side of the image frame, the motion vector at the left end of the subject is detected, so
The clear area follows the subject without sufficiently capturing the subject. That is, the clear area moves so as to escape from the subject.

Therefore, a method for solving the above problems will be described with reference to FIGS. FIG. 9 is a diagram showing the relationship between the position of the clear area in the image frame and the position of the detection area in the clear area. As shown in FIG. 9, when the clear region is located at the right end of the image frame,
Set the detection area to the left edge of the clear area. Similarly, if the sharp area is at the left edge of the image frame, it is at the right edge of the sharp area, and if the sharp area is at the top edge of the image frame,
When the clear region is at the lower end of the clear region and at the lower end of the image frame, the detection region is set at the upper end of the clear region, and when the clear region is at any other position, the detection region is set at the center of the clear region.

FIG. 10 shows a case where the sharp area is set at the right end of the image frame and the object appears in the clear area from the right outer side of the frame, as in FIG. It is a figure which shows the movement of a clear area. In FIG. 10, the left edge of the subject approaches the detection region at the time of (3) when the clear region sufficiently captures the subject, and at this time, the movement of the clear region is started. Therefore, the clear region can follow the movement of the subject while sufficiently capturing the subject. In (4), since the clear area is separated from the right end of the image frame, the detection area is reset to the center of the clear area.

As described above, in the present embodiment, since the image frame is divided into the clear area / deteriorated area and the deteriorated area is not coded, the amount of information generated can be reduced. Further, since the clear region is made to follow the movement of the subject, the subject does not become unclear even if the subject moves and deviates from the center of the screen. Furthermore, by changing the block used for moving the clear region according to the position of the clear region in the image frame, the clear region can be made to follow the movement of the subject while the subject is always held in the clear region. it can.

In this embodiment, one block is used for moving the clear area, but a plurality of blocks may be used and the clear area may be moved based on the average of the motion vectors of these blocks.

Further, in this embodiment, when the clear region is moved, it is moved in block units, but it may be moved in pixel units.

Further, in this embodiment, the deteriorated area is colored with a predetermined color and transmitted, but the deteriorated area may not be transmitted, and the image quality of the deteriorated area may be deteriorated and transmitted.

Further, in the present embodiment, when the block used for moving the clear area is coded in the intra-frame mode, the clear area is moved based on the average of the motion vectors of the blocks prior to the block. However, instead of a simple average, it is also possible to move based on an average calculated by multiplying each motion vector by an appropriate weight.

Further, in the present embodiment, when the block used for moving the clear area is encoded in the intra-frame mode, the clear area is moved based on the average of the motion vectors of the blocks prior to the block. However, the clear region may be moved based on the average of the motion vectors of blocks around the block. Also in this case,
The clear region may be moved based on an average obtained by multiplying each motion vector by an appropriate weight, instead of a simple average of motion vectors of blocks around the block.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to the drawings.
FIG. 11 is a diagram showing a configuration of a transmission side terminal and a reception side terminal of the videophone with the motion tracking function according to the present embodiment. In FIG. 11, an encoder 71 is a transmitting side videophone with a motion tracking function, and a decoder 72 is a receiving side videophone with a motion tracking function. The encoder 71 is shown in FIG.
This is a configuration in which the background color input unit 13 and the error correction control unit 17 are removed from the configuration of the videophone with the motion tracking function.
The decoder 72 includes a communication unit 18, an image decoding unit 701, an image display unit 702, a frame storage unit 12, a background color input unit 1.
3, a decoding control unit 703 and a clear area determination unit 704. It should be noted that, of these constituent elements, detailed description of constituent elements similar to those shown in FIG. 1 will be omitted.

In the decoder 72, the image decoding unit 70
1 decodes the coded image frame received by the communication unit 18 and detects the motion vector of the block forming the frame. Clear area determination unit 704
Determines the clear region of the image frame stored in the frame storage unit 12 based on the motion vector detected by the image decoding unit 701, and outputs the position information. The decoding control unit 703 controls the background color input unit 13 based on the position information output by the clear area determination unit 704. The image display unit 72 displays the image frame stored in the frame storage unit 12.

The image decoding unit 701 of FIG. 11 will be described below.
The method of decoding the coded image frame will be described with reference to FIG. The image decoding unit 70
The image frame that 1 decodes is the image frame 1 shown in FIG.
4 is an encoded image frame. FIG.
7 is a diagram showing a configuration of an image decoding unit 701 of FIG. In FIG. 12, a separation unit 7012 separates the multiplexed image data received from the encoder 71 into a prediction error, a motion vector, and side information, and the prediction error is sent to the variable length decoding unit 7014. The motion vector is sent to the motion vector variable length decoding unit 7015, and the side information is sent to the decoding control unit 701.
To 3 respectively.

The variable length decoding unit 7014 has a separating unit 701.
2 performs variable length decoding on the coded prediction error output by 2 and outputs the result to the inverse quantization unit 148. Inverse quantizer 148
Dequantizes the quantized prediction error output from the variable length decoding unit and outputs the dequantized prediction error to the inverse orthogonal transform unit 149. The inverse orthogonal transformation unit 149 inversely orthogonally transforms the orthogonally transformed prediction error output from the inverse quantization unit 148, and outputs the result to the addition unit 14b. The motion vector variable length decoding unit 7015 performs variable length decoding on the variable length encoded motion vector output from the separation unit 7012, and outputs the variable length decoded motion vector to the motion compensation unit 14f.

The reception frame memory 7016 has a storage area for two frames, and a storage area for one frame has
At present, a frame to be decoded is stored, and a reproduction frame (reference frame) immediately before the frame is stored in the storage area for another frame. The motion compensation unit 14f predicts and predicts which block in the reference frame corresponds to the current decoding target block, based on the motion vector output from the motion vector variable length decoding unit 7015. The image data (predicted value) of the block is output.

The adding unit 14b adds the prediction error output from the inverse orthogonal transforming unit 149 and the prediction value output from the motion compensating unit 14f to reproduce the decoding target block and stores it in the reception frame memory 7016. Output. The decoding control unit 7013, based on the side information, switches 70
Switch connection of 1b. That is, if the received image data is encoded in the intra-frame mode, the switch 701b is switched to the connection of 1-3, and if it is encoded in the inter-frame mode, it is switched to the connection of 1-2. FIG.
The image decoding unit 701 of the
In the case of -3, the received image data is decoded in the intra-frame mode, and in the case of 1-2, it is decoded in the inter-frame mode.

A method of transmitting and receiving an image frame between the encoder 71 and the decoder 72 will be described below with reference to FIG.
1 will be described. In FIG. 11, the encoder 71
The clear area determination unit 16 determines the clear area and outputs information indicating the boundary between the determined clear area and the other area (deteriorated area) in the same manner as described in the first embodiment. . The encoding control unit 15 uses the clear area determination unit 1
The image coding unit 14 is controlled so that the degraded area is not coded based on the information output by the device 6. The image encoding unit 14
Under the control of the encoding control unit 15, only the clear region is encoded without encoding the deteriorated region. The communication unit 18 transmits the image frame encoded by the encoding unit 14 to the decoder 72.

The communication section 18 of the decoder 72 receives the encoded image frame transmitted by the communication section 18 of the encoder 71. The image decoding unit 701 decodes the coded image frame received by the communication unit 18 and detects the motion vector of the block forming the frame. The frame storage unit 12 stores the image frame decoded by the image decoding unit 701. Clear area determination unit 704
Determines a clear region based on the motion vector of each block output from the image decoding unit 701 and outputs information indicating the boundary between the determined clear region and the degraded region. The decoding control unit 703 inputs the background color so as to color the deteriorated area of the image frame stored in the frame storage unit 12 based on the information given by the clear area determination unit 704 and indicating the boundary between the clear area and the deteriorated area. The part 13 is controlled. The background color input unit 13 colors the deteriorated area of the image frame stored in the frame storage unit 12 with a designated color. That is, information representing the designated color is stored in the block corresponding to the deteriorated area. The image display unit 702 displays the image frame stored in the frame storage unit 12.

As described above, in this embodiment, since the encoder 71 does not code the deteriorated area, the total amount of information generated can be greatly reduced. Further, by increasing the amount of information in the clear region by the amount of information thus reduced, the image quality in the clear region can be improved.

Further, since the encoder 72 does not color the deteriorated area but the decoder 72 colors the deteriorated area, the processing of the encoder 71 can be reduced and the privacy of the caller on the encoder 71 side can be protected.

In this embodiment, the encoder 71 does not encode the deteriorated area. However, the encoder 71 may encode the deteriorated area by degrading the image quality and transmit the deteriorated area to the decoder 72 together with the clear area.

(Third Embodiment) A third embodiment of the present invention will be described below with reference to the drawings.
FIG. 13 is a diagram showing the configuration of the non-contact operation input device according to the present embodiment. In FIG. 13, the non-contact operation input device according to the present embodiment includes an image input unit 11 and a frame storage unit 1.
2, image coding unit 14, coding control unit 15, clear region determination unit 16, communication unit 18, image decoding unit 701, operation screen output unit 902, image display unit 702, region position determination unit 90
5, a system control unit 906 and a switch 903.

The communication unit 18 in FIG. 13 transmits / receives encoded image frames. The operation screen output unit 902 displays a plurality of input items including symbols, characters, or charts that represent input commands, and a selection area for selecting one of those items. The selected area here is a boundary line between the clear area and the degraded area, which is output by the clear area determining unit 16. The area position determination unit 905 registers the pixels in the image frame output by the operation screen output unit 902 as many as the input items, and the input items are displayed at positions including these pixels. The area position determination unit 905
It is determined whether or not any of the registered pixels is included in the selection area, and if it is included, selection information indicating which pixel is included is output.

The switch 903 selects either the image output by the image decoding unit 701 or the image output by the operation screen output unit 902 as the image frame stored in the frame storage unit 12. The system control unit 906 controls the entire videophone device based on the selection information output by the area position determination unit 905. For example, if the selection information is auto dial 1, the call is made to the other party registered in auto dial 1. Detailed description of the same components as those described in the first and second embodiments will be omitted.

An example of the operation of the non-contact operation input device of FIG. 13 will be described below with reference to FIGS. 13 and 14. FIG. 14 is a diagram showing an example of a display screen of the image display unit 702 of FIG. In FIG. 14, the display screen has a character 10 that indicates that the automatic dial 1 is called when selected.
02, a character 1003 that indicates that a call is made to the auto dial 2 when selected, a character 1004 that indicates that a call is made to the auto dial 3 when selected, and a message that a manual dial is selected when selected A character 1005 to be selected and a selection area 1001 for selecting these characters are displayed.

For connection of the switch 903 in FIG. 13, the operation screen output unit 902 side is selected. Operation screen output section 9
02 outputs the characters 1001 to 1005 and the selected area. As described in the first embodiment, the clear area determination unit 16 moves the clear area according to the movement of the subject, that is, the operator's hand, and the position information thereof is displayed on the operation screen output unit 902 and the area position determination unit. Output to 905. Here, the clear area determined by the clear area determination unit 16 is the selected area 1001. Area position determination unit 905
Is the pixel at the position where 1002 is displayed, the pixel at the position where 1003 is displayed, the pixel at the position where 1004 is displayed, and the position where 1005 is displayed as the above pixels. Pixels are registered in advance.

The area position determining unit 905 determines whether or not any of the registered pixels is included in the selected area 1001, and if so, a selection indicating which pixel is included. The information is output to the system control unit 906. For example, place your hand in front of the image input means 11,
When the hand is moved upward, the selection area 1001 moves up on the screen and eventually includes the characters of the auto dial 1. At this time, the area position determination unit 905 notifies the system control unit 906 that the pixel at the position 1001 has been selected. Upon receiving the notification, the system control unit 906 controls the videophone device so that the telephone number registered in the auto dial 1 is called.

As described above, according to this embodiment, non-contact operation input can be easily realized by using the image input device and the image compression device of a general TV telephone.

Although the number of operation input items is 4 in this embodiment, the number of input items is increased by increasing the number of pixels registered in the area position determination unit 905 or by hierarchizing the operation screen. Can be more.

(Fourth Embodiment) A fourth embodiment of the present invention will be described below with reference to the drawings.
FIG. 15 is a diagram showing the configuration of a videophone with a motion tracking function according to this embodiment. 15, the videophone with the motion tracking function of the present embodiment includes a communication unit 18, an image decoding unit 701, an image display unit 702, a PIP (picture in picture) creation unit 1301, and an effective macroblock determination unit 1.
302, PIP position determination unit 1303, and image input unit 1
1 is provided.

The effective macroblock discriminating unit 1302 determines the block having the difference information among the blocks forming the image frame to be decoded by the image decoding unit 701 based on the reference frame as the effective macroblock and the difference information. The block that did not exist is determined as an invalid macroblock, and the position information of the valid macroblock is output.

The image input unit 11 inputs an image (for example, a picture of himself) displayed on the child screen. The PIP position determination unit 1303 determines the display position of the child screen based on the position information of the valid macro block output by the valid macro block determination unit 1302. The PIP creating unit 1301 includes the child screen input by the image input unit 11 and the image decoding unit 70.
1 is combined with the image frame (for example, the video image of the other party) output by 1 to create a PIP image in which the child screen is displayed at the position determined by the PIP position determination unit 1303. The image display unit 702 displays the PIP image created by the PIP creating unit 1301.

A method of moving the display position of the child screen in accordance with the position of the effective macroblock in the videophone with the motion tracking function shown in FIG. 15 will be described below with reference to FIGS. 16 and 17. FIG. 16 is a diagram showing a position where a small screen is displayed in the display screen of the image display unit 702 of FIG. As shown in FIG. 16, it is assumed that there are two display position candidates for the small screen (A) and (B), and the current position of the small screen is (A). FIG. 17 is a diagram showing a state in which a subject is displayed in a display screen including a child screen of the image display unit 702 of FIG. 15 and a diagram showing the position of an effective macroblock at that time.

Since the effective macroblocks concentrate on the subject portion, the movement of the subject can be known by detecting the effective macroblocks. Therefore, if the subject portion overlaps the child screen as shown in FIG. 17, the PIP position determination unit 1303 determines the display position of the child screen based on the position information of the effective macroblock output by the effective macroblock determination unit 1302. , And moves from the position (A) in FIG. 16 to the position (B).

As described above, according to the present embodiment, the position of the effective macroblock is detected, and when the effective macroblock overlaps the small screen, the position of the small screen is moved.
Since the effective macro blocks concentrate on the subject portion, the subject can be prevented from being hidden by the small screen.

(Fifth Embodiment) A fifth embodiment of the present invention will be described below with reference to the drawings.
FIG. 18 is a diagram showing a configuration of a videophone with a motion tracking function according to this embodiment. The videophone with motion tracking function of FIG. 18 is the same as the videophone with motion tracking function of FIG.
A clear region determination unit 704 is provided instead of the effective macroblock determination unit 1302. Clear area determination unit 7 in FIG.
04 determines the clear area in the same manner as described in the second embodiment. PIP position determination unit 1303
In the same manner as described in the fourth embodiment, moves the sub screen when the clear region overlaps the sub screen based on the position information of the clear region output by the clear region determination unit 704.

As described above, according to the present embodiment, even when the clear area is moved in the videophone having the picture-in-picture function, the position of the small screen is moved based on the position information of the clear area. It is possible to prevent the clear area from being hidden by the inset screen.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a videophone with a motion tracking function according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of an image coding unit 14 of the videophone with the motion tracking function of FIG.

FIG. 3 is a diagram showing a configuration of a clear area determination unit 16 of the videophone with the motion tracking function shown in FIG.

4 is a diagram showing storage contents of a motion vector storage unit 162 of FIG.

5 is a diagram illustrating a clear region determination unit 1 of FIG. 1 when a block used for moving a clear region is encoded in an intra-frame mode.
6 is a diagram for explaining a method of determining a clear area. FIG.

6 is a diagram illustrating a clear region determining unit 1 of FIG. 1 when a block used for moving a clear region is encoded in an interframe mode.
6 is a diagram for explaining a method of determining a clear area. FIG.

FIG. 7 is a diagram showing a state in which a clear area capturing a subject moves in accordance with the movement of the subject.

FIG. 8 is a diagram showing how a clear region moves according to the movement of the subject when the subject once protruding outside the screen returns to the inside of the screen.

FIG. 9 is a diagram showing the relationship between the position of a clear area in an image frame and the position of a detection area in the clear area.

FIG. 10 shows a case where a detection area is set as shown in FIG.
It is a figure which shows the mode of movement of a clear area.

FIG. 11 is a diagram showing a configuration of a videophone with a motion tracking function according to a second embodiment of the present invention.

12 is a diagram showing a configuration of an image decoding unit 701 of FIG.

FIG. 13 is a diagram showing a configuration of a non-contact operation input device according to a third embodiment of the present invention.

14 is a diagram showing an example of a display screen of the image display unit 702 of FIG.

FIG. 15 is a diagram showing a configuration of a videophone with a motion tracking function according to a fourth embodiment of the present invention.

16 is a diagram showing the position of a child screen on the display screen of the videophone with the motion tracking function of FIG.

17 is a diagram showing a state in which a subject is displayed in a display screen on which a child screen is displayed and a detection position of an effective macro block at that time in the videophone with the motion tracking function of FIG.

FIG. 18 is a diagram showing a configuration of a videophone with a motion tracking function according to a fifth embodiment of the present invention.

FIG. 19 is a diagram showing an example of a configuration of a conventional moving image signal processing device.

FIG. 20 is a diagram showing an example of a configuration of a conventional viewpoint monitoring input device.

FIG. 21 is a diagram showing an example of a configuration of a conventional virtual reality space data input device.

[Explanation of symbols]

 Reference Signs List 11 image input unit 12 frame storage unit 13 background color input unit 14 image coding unit 15 coding control unit 16 sharp region determination unit 17 error correction control unit 18 communication unit 701 image decoding unit 702 image display unit 704 sharp region determination unit 902 Operation screen output unit 905 Area position determination unit 906 System control unit 1301 PIP creation unit 1302 Effective macroblock determination unit 1303 PIP position determination unit

Claims (11)

[Claims] 1. A motion tracking function capable of always catching a moving object in a clear area by determining a new clear area based on a motion vector of a predetermined block in the clear area. A videophone, an image input unit for inputting a moving image, a frame storage unit for storing an image frame input by the image input unit, an image frame stored in the frame storage unit, and An image coding unit that detects a motion vector of a block that constitutes the frame, and a motion vector detected by the image coding unit, based on a motion vector of a predetermined block, in the frame storage unit. A predetermined area in the stored image frame is determined as a clear area for displaying the subject, and the freshness of the frame is determined. And sharpness area determining means for outputting position information of the region / degradation region, on the basis of the position information where the clear area determining means has outputted,
Encoding control means for controlling the image encoding means so as to encode the deteriorated area of the frame with image quality deteriorated as compared with the clear area, and communication for transmitting the image frame encoded by the image encoding means. A videophone with a motion tracking function, including means. 2. The videophone with a motion tracking function according to claim 1, wherein the encoding control unit controls the image encoding unit so as not to encode the deteriorated area of the frame. 3. An image frame stored in said frame storing means is further provided with a background color inputting means for coloring, said encoding control means, further, said background color inputting means, said clear area determining means. The videophone with a tracking function according to claim 1, wherein control is performed so that the deteriorated area of the frame is colored based on the output position information. 4. An error correction control means for adding an error correction code to the image data coded by the image coding means, wherein the coding control means further comprises the error correction control means. Based on the position information output by the sharp area determination means, control is performed so as to add an error correction code having a high error correction ability to the clear area and an error correction code having a low error correction ability to the deteriorated area. The videophone with a motion tracking function according to claim 1, wherein the videophone has a motion tracking function. 5. The position in the clear region of the block from which the clear region determining means obtains the motion vector is determined in relation to the position of the clear region in the image frame. A videophone with a motion tracking function according to any one of to 4. 6. The position of the block for obtaining the motion vector by the clear region determining means in the clear region is such that, when the clear region is located at the right end of the image frame, the position of the block is set at the left end of the clear region. If it is located at the left edge of the clear area, it is defined at the right edge of the clear area, if it is located at the top edge of the image frame, it is at the bottom edge of the clear area, and if it is located at the bottom edge of the image frame, it is defined at the top edge of the clear area. The videophone with a motion tracking function according to any one of claims 1 to 4, characterized in that: 7. A clear area of a received image frame is displayed without deterioration of image quality, and the deteriorated area is
A videophone with a motion tracking function on the receiving side for displaying in color, the communication means receiving an encoded image frame, decoding the encoded image frame received by the communication means, and An image decoding means for detecting a motion vector of a block forming a frame, a frame storing means for storing an image frame obtained by the decoding of the image decoding means, and a motion detected by the image decoding means. Based on the vector, a clear region determination unit that determines the clear region of the image frame stored in the frame storage unit and outputs position information of the clear region / degraded region of the frame, and stores the sharp region in the frame storage unit. A background color input means for coloring the image frame with a designated color, and based on the position information output by the clear area determination means,
A motion tracking function on the receiving side, which comprises a decoding control means for controlling the background color input means so as to color the deteriorated area of the frame, and an image display means for displaying the image frame stored in the frame storage means. Videophone. 8. A videophone having a picture-in-picture function for displaying a received image frame by superimposing a child screen, the communication means receiving the encoded image frame, and the code received by the communication means. An image decoding means for decoding the encoded image frame and detecting a motion vector of a block forming the frame; and based on the motion vector detected by the image decoding means, the image decoding means Determining a clear area in the decoded image frame, based on the position information output by the clear area determination means for outputting the position information of the clear area of the frame, the clear area determination means,
Picture-in-picture position determining means for deciding the position of the small screen and outputting the position information of the small screen, an image for inputting the image displayed on the small screen and the image transmitted to the other party's videophone terminal An image frame obtained by decoding by the image decoding means on the basis of input means, position information output by the clear area determination means, and position information output by the picture-in-picture position determination means; Picture-in-picture creating means for creating a picture-in-picture image including the child screen at the position determined by the picture-in-picture position determining means by combining with the image displayed on the child screen input by the image input means; Image display means for displaying the picture-in-picture image created by the picture-in-picture creating means,
Videophone with motion tracking function. 9. A videophone having a picture-in-picture function of displaying a child screen in an overlapping manner on a received image frame, the communication means receiving the encoded image frame, and the code received by the communication means. The image decoding means for decoding the converted image frame and the blocks forming the image frame decoded by the image decoding means determine whether or not each has difference information, and have the difference information In this case, the sub-screen is determined based on the position information of the effective macro block which is determined by the effective macro block determining unit that determines the block as the effective macro block and outputs the position information of the block. A picture-in-picture position determining means for determining the position of the child screen and outputting position information of the child screen, and an image displayed on the child screen. Based on image input means for inputting an image to be transmitted to the other party's videophone terminal, position information output by the valid macroblock determination means, and position information output by the picture-in-picture position determination means, The image frame obtained by decoding by the image decoding means and the image displayed on the child screen, which is input by the image inputting means, are combined, and the child is placed at the position determined by the picture-in-picture position determining means. A picture-in-picture creating unit that creates a picture-in-picture image including a screen; and an image display unit that displays the picture-in-picture image created by the picture-in-picture creating unit.
Videophone with motion tracking function. 10. The operator does not touch the input item corresponding to the pixel by including a pixel located at a predetermined position in the image frame in a clear region that moves following the motion of the operator. A non-contact operation input device that can be selectively selected, including image input means for inputting a moving image, frame storage means for storing an image frame input by the image input means, and the frame storage means. An image encoding unit that encodes the stored image frame and detects a motion vector of a block forming the frame, and an image encoding unit that is stored in the frame storage unit based on the motion vector output from the image encoding unit. A clear area determination that determines a predetermined area in the image frame as a clear area for displaying the subject and outputs position information of the clear area of the frame. Means for storing the position information of one or more pixels in the image frame, and based on the position information of the pixel and the position information output by the sharp region determining means, whether the sharp region includes the pixel. In the case of including and determining whether or not it includes an area position determining unit that outputs position information of the pixel, and an operation control unit that selects an operation command according to the position information output by the area position determining unit. Touch operation input device. 11. A character / character representing an input item at a position including the pixel based on the position information output by the clear area determination unit and the position information of the pixel stored by the area position determination unit. The graphic information creates an operation screen in which a boundary between the clear area and the deteriorated area is displayed at the position of the clear area and outputs the operation screen, and the operation screen output means outputs the operation screen. The non-contact operation input device according to claim 10, further comprising image display means for displaying an operation screen.