JPH01141479A - Image communication equipment utilizing glance detection - Google Patents

Abstract

PURPOSE:To economically receive with high quality, an image obtained by means of a camera installed at an opponent side (transmitting side) by feeding back the glance information of a receiving person to a transmitting part and extracting transmitting image information from an original image based on it. CONSTITUTION:The image is reproduced from a receiving signal, it is displayed on an image displaying means 8, the movement of the glance of a receiving person 9 monitoring the image displaying means 9 is detected, and the glance information is transmitted to an image transmitting part 15. By the image transmitting part 15, the information quantity of the original image is weighted based on the glance position information, the transmitting image information is extracted, the extracted image information is transmitted to a receiving side, and the weighting is executed in the point of a transmitting information quantity between the image information of the central part of the glance important for the recognition of the receiving person and the image information of a gazing point periphery to have little concern to the recognition of a detail. Thus, the image information can be widely compressed, the image information can be made economical, and when the transmitting information quantity is fixed, an image communication which seems to have the high quality can be realized.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to an image communication device that uses line of sight detection, and particularly improves image quality by feeding back line of sight information of a receiver to a transmitting side. The present invention also relates to an image communication device using line of sight detection that can significantly compress information.

[Prior Art] FIG. 12 is a diagram schematically showing a conventional communication method used for image communication such as video conferences, video telephone calls, and access to a moving image database. In FIG. 12, the original image extraction unit 1 on the image transmission side captures a visual object 2 of a person with the camera 3. The original image of the visual object 2 taken by the camera 3 is given to the image encoding means 4, and the original image is encoded. The encoded original image is transmitted to the receiving side by the transmitting means 5. The encoded original image is received by the receiving means 6 on the image receiving side and restored to the original image by the decoding means 7.

The restored original image is displayed on the display means 8, and the recipient 9 views the image.

Here, in the following explanation, the image before being input to the image encoding means 4 will be referred to as an original image, and the image information whose information has been scanned (compressed, etc.) for transmission will be referred to as transmitted image information. In this method, the transmitted image information is encoded uniformly and directly in the entire screen area for each screen, and then transmitted to the receiving side. For this reason, 107~'10
The encoding rate is 8 bps, and a wide transmission band is required as a transmission path. For this reason, there is a problem in that the transmission cost becomes high when transmitting the formatted image information shown in FIG. 12. ′ Various studies have been conducted to reduce the bit rate to address the above-mentioned problems. The ones that have been put into practical use are: 1. Using statistical properties such as the tendency of screen shading and the correlation between pixels. There are ways to minimize transmitted information by making predictions spatially or temporally. For example, there are methods of extracting only the moving parts, or extracting and transmitting the contours. Furthermore, in recent years, vector quantization has attracted attention as a high-efficiency encoding method for moving images, and has been put into practical use to some extent. In this method, the original image to be encoded is divided into small blocks, each block is represented by a vector X-(X+l x2... = (Y+
) l=l, NIV+ -(Y+'+.

y12. ...y,,), and select the appropriate one from X.
, to y6. This is a method of encoding by replacing the subscript i with the subscript i. Here, Y is called a codebook.

FIG. 13 is a diagram for explaining a conventional image information compression method using vector quantization. The original image 10 is captured by the camera 3, and includes a visual object 11. This original image 10 is divided into blocks, and each block is represented by X. X, is 4
It is compared with N blocks in the pixel by 4 pixel codebook 12, the closest block i is selected, X( is replaced with this index i, and this is encoded.

Therefore, if this operation is performed efficiently, the amount of information will be smaller than if X is transmitted as is.

In this vector quantization, the problem is how to create the codebook 12 and how to search the codebook for the vector closest to the input vector of the original image when using the codebook 12. Various methods have been proposed for creating the codebook 12 in order to prevent the quality of reproduced images from being biased depending on the type of original image.

Also, regarding the search, performing successive comparison with all code vectors in the codebook would require too much calculation, so the vectors in the codebook are classified into several categories. Depending on the nature of the input vector, by selecting this category and searching, we can use this search method to reduce the number of searches. Multi-stage vector quantization that sequentially divides blocks into smaller blocks, and adaptive block size vector quantization that selects blocks of two sizes (large and small) by applying this to small screen display devices (Izuoka, Hase, Suzuki: Adaptive block size vector Small-screen video communication using quantization (see Institute of Electronics, Information and Communication Engineers technical report IE86-98) is known.

Furthermore, recently, consideration has been given to applying intelligent processing to encoding as a future image communication system aiming at significant compression and transmission of natural image quality. The transmitter side recognizes the characteristics of the original image, encodes and transmits this feature recognition information, and then the receiver side uses the stored image feature database to determine the object image and its movement in real time at high speed. It is something that is generated and displayed.

[Problems to be Solved by the Invention] The conventional information compression methods described above all have in common that the entire screen is processed according to a uniform algorithm.

Furthermore, regarding vector quantization, it can be said that the method of dividing the screen is uniform at least in the initial stage of quantization. However, in various image services such as video calls and video conferences, it is known from daily experience that if the image is meaningful to the user, the user's gaze will be focused on one part of the image. In other words, people do not perceive the entire screen at once, but try to understand the image by scanning the parts of interest. This tendency becomes stronger as the degree of interest increases and as the screen becomes larger.

Despite these human visual characteristics, the conventional method described above encodes the entire screen according to a uniform algorithm, so even parts that are not originally needed are encoded with a large amount of information. The disadvantage is that there is a lot of waste. Furthermore, if the amount of information that can be transmitted in real time is limited, transmitting the entire image using a uniform algorithm will result in coarse encoding, making it impossible to obtain a sufficient amount of information near the gaze point, and There were drawbacks such as poor quality.

In addition, regarding the intelligent image communication system, there is a drawback that processing time is increased when features are extracted and encoded in a manner similar to - for all visual objects on each screen.

Another disadvantage is that the processing device becomes large in order to execute in real time.

Therefore, the main purpose of this invention is to focus on human visual characteristics, and instead of uniformly compressing information on the entire screen using an algorithm, the main purpose of this invention is to compress the information of the receiver based on the gaze information fed back from the receiver. By weighting the amount of transmitted information between the image information in the center of the line of sight, which is important for recognition, and the image information around the gaze point, which is less relevant to the recognition of details, it is possible to significantly compress image information. It is an object of the present invention to provide an image communication device using line of sight detection that can make image information economical and realize image communication with high quality.

[Means for Solving the Problems] The present invention is an image communication device that includes an image receiving section and an image transmitting section and uses detection of the recipient's line of sight, and the image receiving section reproduces an image from a received signal. An image restoration means, an image display means for the receiver to monitor the reproduced image, a line of sight detection means for detecting the movement of the receiver's line of sight moving on the display screen of the image display means, and a line of sight detection means. and a line-of-sight information transmitting means for transmitting the line-of-sight information obtained by the above to the image transmitting section.

On the other hand, the image transmitting unit includes a transmission image information extraction unit that extracts transmission image information by weighting the information amount of the original image based on the gaze position information transmitted from the gaze information transmission unit; and a transmitting means for transmitting to the receiving section.

Here, as an example of a means for extracting transmitted image information, there is an example that includes a part that encodes the original image and a part that weights the encoding by using at least one parameter the distance from the receiver's gaze point. .

In this case, an example of encoding is vector quantization, and there is an example in which the distance from the point of interest is used as at least one weighting parameter when selecting a block size for the encoding. Also, examples of other parameters are:
There are receiver visual acuity characteristics that can be corrected by detecting gazed dust. Another example of the transmitted image information extraction means is an example that includes a part that recognizes image features from the original image and a part that weights feature recognition scanning based on line-of-sight information. In this case, there is an example in which the image restoration means on the receiving side restores the image by understanding the characteristics of the target image from a pre-stored image feature database regarding the target image and the feature recognition information transmitted from the transmitter.

[Function] ゛The image communication device using line of sight detection according to the present invention has the following features:
An image is reproduced from the received signal and displayed on the image display means, the movement of the line of sight of the recipient who is monitoring the image display means is detected, and the line of sight information is transmitted to the image transmitting section. By weighting the amount of information in the original image based on the gaze position information and extracting the transmitted image information, and transmitting the extracted image information to the receiving side, an image of the central part of the gaze that is important for the recipient's recognition is created. By weighting the amount of transmitted information between information and image information around the gaze point that is less relevant to the recognition of details, it is possible to significantly compress image information, making image information more economical. When the amount of information to be transmitted is constant, it is possible to realize image communication that feels high quality.

[Embodiment of the Invention] Figures 2 and 3 are diagrams for explaining the invention in detail. In particular, Figure 2 shows the visual acuity characteristics, and Figure 3 shows the standard characteristics of the display device. FIG. 2 is a diagram showing observation conditions.

First, the principle of this invention will be explained with reference to FIGS. 2 and 3. When a person obtains information about the outside world, 8
It is said that more than half of this is due to visual perception. On the other hand, as shown in Figure 2, visual acuity is not uniform on the retina; the area with high visual acuity is a small part called the fovea (approximately 1° in visual terms), and the area with visual acuity of 0.3 or higher is 0 of the entire retina.
This accounts for about 5%, and the majority of the rest have visual acuity of 0.1 or less.

Here, the range of several degrees of visual acuity where visual acuity is relatively high will be referred to as the central area, and the periphery where visual acuity is reduced will be referred to as peripheral vision.

Further, the standard viewing angle of a display device is as shown in FIG. 3, taking current televisions and high-definition televisions as examples, and many other display devices have viewing angles of 10° or more. Furthermore, since the sense of presence is proportional to the size of the screen, it is likely that visual perception will become even larger in the future. From the above, it can be seen that humans cannot perceive the entire range of the display screen at once with high visual acuity. Therefore, people obtain information by extracting an interesting object on the screen, fixing their line of sight on it, and moving their line of sight when the object of interest changes.

Therefore, this eye movement is measured in real time, this information is fed back to the transmitting side, and the transmitting side transmits the information on the part that the person is paying attention to with as much fidelity as necessary.
It is conceivable to compress the transmitted information by transmitting other parts of the information at a lower bit rate. Based on this principle, embodiments of the present invention will be described below.

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

First, the configuration of an embodiment of the present invention will be described with reference to FIG. The original image extraction unit 1 extracts an image of a visual object 2 such as a person using a camera 3 in the same manner as in the conventional example shown in FIG. 12 described above. A camera work control means 21 is provided in connection with the camera 3, and the camera work control means 21 controls the facing direction, magnification, focus, etc. of the camera 3. The image information output from the camera 3 is given to the image encoding means 14 and the original image is encoded. The encoded original image information is transmitted to the receiving side by the transmitting means 15.

A receiving means 16 included in the receiving section receives the transmitted image information, and a decoding means 17 restores the code of the received image information to a display signal. The display means 8 displays the restored display signal. The recipient 9 monitors the image displayed by the display means 8. A line-of-sight detection camera 18 is provided to detect the movement of the line-of-sight of the recipient 9, and the gaze point detection/transmission means 19 detects the gaze point as gaze information from the image obtained by the gaze detection camera 18. Send to the transmitter.

Note that the transmitter is provided with a gaze point weighting processing means 20. The point-of-gaze weighting processing means 20 controls the image decoding algorithm by the image encoding means 14, using the distance from the point of interest as at least one parameter, based on the point-of-gaze information fed back from the receiving section. It is.

FIG. 4 is a diagram for explaining the concept of transmitting an image by weighting the gaze point. In FIG. 4(a), the visual object 2 is included in the original image 22 captured by the camera 3 shown in FIG.

This original image 22 is subjected to gaze point weighting processing and transmitted image 2
It becomes 3. In FIG. 4(b), 24 is the receiver's gaze point, and 25 is the receiver's gaze point, and the signal is transmitted with the necessary degree of fidelity within the range corresponding to central vision. Furthermore, in the range corresponding to peripheral vision, image information is thinned out into two stages and transmitted.

FIG. 5 is a diagram for explaining an encoding method when the present invention is applied to vector quantization. Next, image encoding in one embodiment of the present invention will be described in detail with reference to FIGS. 1, 4, and 5. The original image 30 is input to the image encoding means 14 shown in FIG. 1, and is divided into 7×7 blocks xk as shown in FIG. Here, the left edge of the face in the block where the gaze point 31 is located is designated as XI6, and the right edge of the face in the block located at a symmetrical position between XI6 and the center of the face is designated as X20. Here, it is assumed that X, 6 and X20 have the same amount of information in FIG. It is also assumed that each block consists of 4×4 pixels.

Each block is sequentially compared with N blocks in the 4-pixel×4-pixel codebook 32, and the amount of distortion is calculated for each block. Here, assuming that the block with the smallest amount of distortion is i, this i was selected in the conventional example. In contrast, in one embodiment of the invention, this distortion ii'd (is further compared with a threshold value A(r,h), such that d, <A(r,
If the condition h) is satisfied, xk is replaced with this index i. Here, the threshold value A (r, h
) is a variable whose parameters are the distance Mr from the gaze point 31 and the visual characteristics such as the degree of attention. For simplicity, if r is the central view, then A (r+ )1) −
Let a + (constant), and if it is a peripheral vision area, then A
Let it be (r, h)-a2. In the block of X20,
Assuming that the condition dl<a2 is satisfied, i is selected as shown in FIG.

On the other hand, the xis block does not satisfy dl <a. That is, as shown in FIG. 5, d1≧A D, h
). In this case, a block of 4 pixels x 4 pixels
ll+ is further divided into four smaller blocks of 2 pixels x 2 pixels, which are compared with the 2 pixels x 2 pixels codebook 33, and the nearest block j, (1, m) is selected and output. .When encoding, each block X.

If the block is 4 pixels x 4 pixels, code 0 is set, and then the index in the 4 x 4 code book 32 is coded.

Further, if it is a block of four 2 pixels x 2 pixels, the code "1" is set, and then fl+m is coded into the index J in the 2 x 2 code book 33.

In this way, in the original image, X1li and X20 are the same information block, but by weighting encoding according to this embodiment, X20 can be made smaller than X1li. The same applies to other blocks. In this way, information can be reduced in the peripheral vision area where visual acuity is low.

Here, the amount of information compression will be explained. The cumulative speed of eyeball follow-up movement during moving image observation is 5°/s or less, accounting for 70% of the time. Within this speed, it can be assumed that a person can gaze, so it is assumed that the range of good visibility (central vision) is about 5 degrees, and the rest of the range is assumed to be peripheral vision. Next, when the recognition target is a person and the recognition limit pixel number is determined through a psychological experiment, it is 5.
It is known that if the pixels of an original image of 12 x 480 pixels are reduced to approximately 1720 or less, large facial expressions of a person cannot be distinguished, and fine facial expressions of 115 to 1/10 become indiscernible (Yorizawa, Miyata). Relationship between the number of pixels of two-person images and identifiable information, Television Society Technical Report ID87
-5)). Therefore, assuming that the amount of information can be reduced to 115 for the part of the image that is 5 degrees or more away from the viewpoint,
The information compression rate is about 1/3 for current television, and about 1/4 for high-definition.

Above, A (r, h) is a, due to central vision and peripheral vision.
We have described the case where the visual acuity changes in two stages (1 and 82), but human visual acuity is approximately proportional to the reciprocal of r and changes continuously. Furthermore, if the central point is an interesting object and the attention is concentrated there, the change in visual acuity relative to r becomes even more steep. Visual acuity also changes depending on when you are nervous or relaxed. A factor related to such a person's consciousness is represented by h. At this time, A(r
.

The value of h) may be changed based on the above. Note that the degree of attention can be estimated from measurements of the pupil area of the eyes, the frequency of gaze saccades, and the like. That is, as the degree of attention increases, pupils tend to dilate and saccades tend to decrease. These can be captured by camera 3.

By the way, the user's line of sight is considered to reflect their intentions. Therefore, as a more advanced control, the gazing visual object 100 shown in FIG. 1 is extracted from the gaze point information fed back from the receiving side, and other parts, such as the background, are regarded as less important information. , it is not possible to control the amount of information to be reduced. In this case, the user can clearly recognize the object of attention on the screen.

Above, we have described an example of feeding back the gaze point position as line-of-sight information, but in addition to this, there are other ways to feed back information on whether the user is looking at the screen, and interrupt image transmission if the user is not looking at the screen. It may also be possible to perform control of the following.

Next, referring to FIG. 1, the feedback of the gaze point information to the camera work control means 21 will be explained. When a visual target is estimated from the line-of-sight information, the camera work control means 21 rotates the camera 3 so that the visual target is centered on the display screen, and further controls the original image by enlarging the image. You can. Note that the visual target may be estimated on the receiving side and fed back to the transmitting side as line-of-sight information.

FIG. 6 is a diagram for explaining an encoding method in another embodiment of the invention. The embodiment shown in FIG.
This is a case where A(r,h) changes in three stages and the block size is set in four stages. A(r, h) is a shown in Figure 6.
Let l r 82 r aa. Taking block x4 in the vicinity of the gaze point 45 as an example, since the original image is 8 pixels by 8 pixels, it is compared with the 8×8 codebook 46. d,
<A (r.

h), i is coded together with the code "00" representing the block size of 8 pixels x 8 pixels. d1≧
If A(r,h), it is divided into four 4×4 blocks (2), and each is compared with the 4×4 codebook 47.

Here, if dff, < A (r, h), m is encoded with the code “011 representing block Move to ■.In the same way, 16 2X
It is divided into two blocks (code "10"), each of which is compared with the 2x2 code book 48. Here, do
If <A (r, h), then n is encoded and dn≧A
If (r, h), encode the not applicable Z and convert the pixel ■
(code “11”).

Figure 7 shows an example in which the original image is not evenly divided from the initial stage, but the central vision part is divided into small blocks, the peripheral vision part is divided into large blocks, and then the comparison with the codebook is performed. FIG. In the embodiment shown in FIG. 7, the block size is in two stages, similar to the embodiment shown in FIG. 5 described above.

The whole is divided into 16 large blocks, among which the blocks covering the area of central vision 27 are as follows:
At an early stage, it is further divided into smaller blocks. That is, at the beginning of pattern matching, the block size is weighted using the distance from the point of interest as a parameter. The encoding process is similar to the embodiment shown in FIG.

Furthermore, the present invention can be applied not only to video transmission, but also to transmission of high-definition still images at a low bit rate.

FIG. 8 is a diagram showing an example in which the present invention is applied to still image communication. In FIG. 8, a high-definition image 55 to be transmitted
contains the first fixation point■. At the beginning of transmission of the target image, quantization is performed in the same way as in the case of video transmission described above, and the information density to be transmitted is high when averaged around the point of interest, that is, there are many areas with small block sizes, and the area around the point of interest is On average, the area has a low information density, that is, there are many areas with large block sizes.

Here, as the gaze point moves like ■, ■,
Areas that previously had a small amount of information are replaced with images that have a large amount of information.

At this time, areas where the amount of information does not change do not need to be transmitted. By controlling in this manner, the user can receive the target image while perceiving it as having high definition from the initial stage of reception.

Although vector quantization has been explained above, this invention also applies to other compression methods, such as discrete cosine transformation, intelligent image communication method using feature extraction (Murakami et al.

Ichihara: A method for constructing an intelligent image communication system? ! ! (Refer to Technical Report IE87-14 of the Institute of Information and Communication Engineers).

FIG. 9 is a diagram showing an embodiment applied to the intelligent image communication system of the present invention. Conceptually, this embodiment recognizes the features of the target image by knowing the person's gaze point with respect to the displayed image, and recognizes the features of the target image finely in the part that is being watched, and coarsely in other parts, or by using computer animation technology. The aim is to significantly reduce the amount of information to be sent by using substitutions.

Note that FIG. 9 is constructed in the same manner as the embodiment shown in FIG. 1 described above except for the following points. That is, an image recognition encoding means 61 is provided in place of the image encoding means 14 on the transmitting side shown in FIG. 1, an image database 62 is provided, and an image generating means 63 is provided in place of the decoding means 17 on the receiving side. and an image database 64 are provided. The image recognition and encoding means 61 recognizes and encodes the characteristics of the original image. The image databases 62 and 64 on the sending and receiving sides each have the characteristics of each image necessary for recognition. The image generating means 63 generates an image database 64 based on the recognition information received by the receiving means 16.
The original image is restored by generating the image while referring to the image. Here, the gaze point weighting is performed by the processing means 2.
0 controls operations for recognition, such as the number of image divisions (blocks) and the number of instruction elements (commands) for recognition, when recognizing and encoding features of the original image.

Next, the operation of the embodiment shown in FIG. 9 will be explained.

Image database 62 provided on the sending and receiving sides
．． 64 can be roughly divided into two types: (1) static image databases and (2) dynamic image databases. The static image database (2) is prepared at the time of system configuration, and stores standard shapes when the target image is a human image.

Specifically, small plane approximation is performed by three-dimensional measurement using images taken of a person from the front and side directions. In other words, if you manually determine the direction and basic expressions of a person's face, a typical face ii! It is a database in which the J image is expressed. therefore,
During communication, the target person's movements and facial expressions are recognized, encoded and transmitted, and the receiving side uses this database to obtain a typical facial image. This database 62, 64 is called a plane batch method, and is a method of approximately representing an object using a small plane (or curved surface).

The dynamic image database ■ is a database for naturally expressing the vicinity of each small plane of a face image during actual communication. In other words, the image obtained from the static image database ■ is a model, and naturally the original image (image at the time of communication (!
Since this is different from a), the purpose is to correct the position and normal direction of the small plane so that it becomes closer to the original image. This database uses a method called an image representation method, in which objects are represented in units of surface pixels. Specifically, information such as the position of the surface from the base point (for example, the center) of the target object and the direction in which the normal line of the surface faces is stored.

Object descriptions are grouped as a command system, such as the head, eyes, nose, ears, cheeks, skin, etc., and are hierarchically described along with their interrelationships. Therefore, in communication, the transmitting side extracts the surface position and normal direction of each part of the area located on the small plane as feature recognition information, encodes it with a command, and transmits it. On the receiving side,
An image is reconstructed by combining the information recognized using the plane patch method and the information recognized using the image representation method.

As is clear from the above explanation, the representation of objects can be made arbitrarily fine in both the plane patch method and the pixel representation method, that is, the layering can be complicated,
This increases the degree of natural reproduction of the original image. However, as in the vector quantization embodiment, it is not necessary to uniformly segment the entire screen to a level greater than visual acuity characteristics, so in this embodiment, the line of sight information fed back from the receiving side is used for command selection during transmission. We will reflect this and perform necessary and sufficient small screen division and surface display.

Note that in the present invention, it is desirable that the line of sight be measured without contact. Therefore, an example of a non-contact line of sight detection method that does not require wearing a device such as glasses on the face will be briefly explained using FIGS. 10 and 11. In FIG. 11, the eyeball can be represented by a model in which two balls 71 and 72 are overlapped with their centers shifted as shown in FIG. The curvatures of the spheres 71 and 72 correspond to the curvatures of the vitreous body and the cornea, respectively. When the head is fixed and the eyes are rotated, in this model, the sphere 72 rotates at the center of the sphere 71 like a thick solid circle. Therefore, in this example, the direction of the line of sight is detected by measuring the position of a fixed feature point in the face, estimating the center coordinates of the sphere 72, measuring the center position of the pupil, and combining these position vectors. do.

FIG. 11 shows an example of a method for extracting immobile feature points.

The positions of the outer corners of both eyes shown in FIG. 11 do not move much with respect to the movement of the eyeballs. Therefore, it is possible to measure the positions of the left and right outer corners of the eyes and approximate the intermediate position as a fixed feature point. A three-point survey method using three cameras can be applied to measure the positions of immobile feature points and the center of the pupil. In FIG. 10, the direction of the line of sight is represented by the sum of vectors A, B, and C.

Here, the coordinates of the center of the sphere 71 can be determined by presenting a fixed target on the display screen and instructing the user to do so at the beginning of using this system. That is, it can be determined by calibration.

Although an example in which the line of sight detection camera 18 is used to detect the gaze point has been shown, other sensors, such as an EOG (Elec
A method (tro-oculogram) or a method of detecting a corneal reflection image using a position detection sensor may be used.

Furthermore, it goes without saying that the present invention may be used in combination with the information compression technology described in the prior art, which extracts and transmits only the motion part of the object.

[Effects of the Invention] As described above, according to the present invention, the receiver's line-of-sight information is fed back to the transmitter, and based on this, transmitted image information is extracted from the original image, so that, for example, videophone calls can be made. This makes it possible to economically receive images captured by a camera installed on the other party's side (sending side) with high quality in applications such as robot control and remote robot control. It can also be applied to image communication using binocular parallax.

[Brief explanation of the drawing]

FIG. 1 is a schematic block diagram showing an example of an image communication device that feeds back line-of-sight information according to an embodiment of the present invention. FIG. 2 is a diagram showing visual acuity characteristics with respect to vision. Third
The figure is a diagram for explaining standard viewing conditions of a display device. FIG. 4 is a diagram illustrating the concept of an image transmitted with weighted points of interest in order to explain the operation of an embodiment of the present invention. FIG. 5 is a diagram for explaining an encoding method when one embodiment of the present invention is applied to vector quantization. FIGS. 6 and 7 are diagrams for explaining an encoding method in another embodiment of the invention. FIG. 8 is a diagram for explaining an embodiment in which the present invention is used for still image communication. FIG. 9 is a schematic block diagram of an embodiment in which the present invention is applied to an intelligent image communication system. Figure 10 and 1
FIG. 1 is a diagram showing an example of a non-contact line of sight detection method. FIG. 12 is a schematic block diagram of a conventional cicada image encoding communication device. FIG. 13 is a diagram for explaining a conventional image information compression method using vector quantization. In the figure, 1 is an original image extraction unit, 3 is a camera, 8 is a display means, 14 is an image encoding means, 15 is a transmission means, 16
17 is a receiving means, 17 is a decoding means, 18 is a line of sight detection camera,
Reference numeral 19 denotes a gaze point detection/transmission means, 20 a gaze point weighting processing means, 21 a camera work control means, 61 an image recognition encoding means, 62 and 64 an image database, and 63 an image generation means. Patent applicant A.T.R. Communication Co., Ltd. Figure 2 Figure 3 Figure 10 Figure 11

Claims (6)

[Claims] (1) An image communication device that includes an image receiving section and an image transmitting section and uses detection of the recipient's line of sight, wherein the image receiving section includes an image restoring means that reproduces an image from a received signal, and an image display means for monitoring a reproduced image; a line-of-sight detection means for detecting movement of the line of sight of a receiver moving on a display screen of the image display means; and line-of-sight information obtained by the line-of-sight detection means. and a line-of-sight information transmitting means for transmitting the line-of-sight information to the image transmitting unit, the image transmitting unit transmitting image information by weighting the information amount of the original image based on the line-of-sight position information transmitted from the transmitting unit. 1. An image communication device using line of sight detection, comprising: a transmission image information extraction means for extracting a transmission image information; and a transmission means for transmitting the image information extracted by the transmission image information extraction means to the reception section. (2) The transmission image information extraction means includes an encoding means for encoding the source image, and a weighting means for weighting the encoding using a distance from the receiver's gaze point as at least one parameter. An image communication device using line-of-sight detection according to claim 1, characterized in that the device includes: (3) The encoding means encodes the original image by vector quantization, and when selecting a block size for encoding, the distance from the point of interest is used as at least one weighting parameter. , an image communication device using line of sight detection according to claim 2. (4) The image communication device using line of sight detection according to claim 2, wherein the encoding means uses visual acuity characteristics of the receiver as another weighting parameter. (5) The image communication device using line of sight detection according to claim 4, wherein the encoding means detects the degree of gaze and corrects visual acuity characteristics. (6) The transmitted image information extraction means includes means for recognizing image characteristics from the original image, and based on the line of sight information,
and means for weighting the feature recognition operation, and the image restoring means extracts the features of the target image from an image feature database related to the target image stored in advance and the feature recognition information transmitted from the image transmitter. An image communication device using line of sight detection according to claim 1, further comprising means for understanding and restoring an image.