JPH04293376A - Video display system - Google Patents

Abstract

PURPOSE:To reduce the information quantity of a video signal sent to a display device by limiting a band of a noted area component of the video signal in response to a length of a noted time. CONSTITUTION:The video display system in which the information quantity of a video signal sent from a video signal generator to a display device is reduced is indicated. A noticed area of an observer on a screen is detected based on an eyeball motion and a head motion. A noticed time discrimination section 83 of a picture processing unit 8 detects it that a noticed time length of an observer exceeds a prescribed time length. A control signal generating section 82 gives a control signal required to decrease the band of the noticed area component of the video signal to a switching section in response to the detection. A switching section 81 generates a synthesis video signal in which the band of the noted area component is limited in response to the given control signal and the signal is sent to the display device.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video display system, and more particularly to a video display system that can reduce the amount of information transmitted between a video signal generator and a display device.

0002

[Prior Art and Problems to be Solved by the Invention] Conventionally, when band compression of video signals or image signals is performed, redundancy contained in image data, that is, correlation, is Uniform processing is performed on the image data of the entire screen using the height. Therefore, when an image based on a band-compressed and transmitted image signal is displayed on the screen, the viewer can view the image displayed on the screen at a uniform resolution. That is, the image displayed on the screen has a uniform resolution everywhere, and the viewer can see the image having the same resolution everywhere on the screen.

However, since the resolution of the image is uniform, when the image is viewed in detail in the observer's gaze area,
You may feel that the resolution is insufficient due to the coarse pixels. In order to prevent this, it is necessary to widen the transmission band and improve resolution, but it is generally not desirable to widen the transmission band.

On the other hand, in the technical fields of broadcasting and image communication, the conventional format of showing the same video or image to many viewers under the same conditions has changed from the conventional format of showing the same video or image to many viewers, to allowing each viewer to view desired image parts individually and according to their preference. There is a shift to a form of observation. Therefore, it is not possible to meet this demand with uniform band compression of video signals or image signals.
The development of technology that compresses bandwidth individually according to the needs of observers has become an important issue.

The present invention was made in order to solve the above-mentioned problems, and it is an object of the present invention to provide a video display system that can reduce the amount of information transmitted between a video signal generator and a display device. purpose.

[0006]

[Means for Solving the Problems] A video display system according to the present invention includes: a video signal generating means; a video display means for displaying a video on a screen based on a video signal generated from the video signal generating means; gaze state detection means for detecting a gaze area and gaze time length on a screen of an observer observing a video image, and determining whether the detected gaze time length exceeds a predetermined time length. and video information reducing means for reducing the amount of video information in the gaze area of the video signal generated by the video signal generating means in response to the gaze time length determining means.

[0007]

[Operation] In the video display system of the present invention, the gaze time length determination means determines that the gaze time length of the observer detected by the gaze state detection means exceeds a predetermined time length. . When the gaze time length exceeds a predetermined time length, the video information reduction means responds to the gaze time length determination means to reduce the video information of the gaze area of the video signal generated by the video signal generation means. Decrease the amount. The video display means displays on the screen a video based on the video signal whose information amount has been reduced by the video information reduction means. Therefore, the amount of video information of the video signal transmitted to the video display means is reduced.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram of a video display system showing an embodiment of the present invention. Referring to FIG. 1, this video display system includes a video signal generator 1 that generates a video signal of a video to be displayed on the screen of a display device 10, and a low pass filter (LPF) 2 that limits the band of the video signal. , an image processing device 8 that performs image processing on the video signal according to changes in the observer's gaze area on the screen, an eye movement detection device 5 that detects the movement of the observer's eyeballs, and a magnetic sensor. a head movement detection device 6 that detects the head movement of the observer; a gaze point processing device 7 that calculates gaze data of the observer by processing the detected head movement data and eyeball movement data; and a gaze point processing device 7 that calculates gaze data of the observer; and a gaze area separation device 3 that obtains gaze area data of an observer based on. The image processing device 8 includes a switching unit 81 that receives the video signal Sa generated from the video signal generation device 1 and the video signal Sb provided through the low-pass filter 2, and generates a control signal based on the viewer's gaze area data. It includes a control signal generation section 82 and a gaze time determining section 83 that determines whether the observer's gaze time length exceeds a predetermined time length and generates a band control signal BW.

In operation, the eyeball movement and head movement of the observer are detected by the eye movement detection device 5 and the head movement detection device 6, respectively. Based on the detected movement data, the gaze point processing device 7 performs arithmetic processing to obtain line-of-sight data indicating the movement of the observer's line of sight. The obtained line-of-sight data is given to the gaze area separation device 3, where the coordinates of the gaze area on the screen that the observer is gazing at and the coordinates of its surrounding area are obtained. The coordinates of the gaze area and its surrounding area are given to a control signal generator 82 within the image processing device 8 . The above processing will be explained in detail later.

The video signal Sa generated from the video signal generator 1 is directly transmitted to the switching section 81 in the image processing device 8.
while also being given to the low-pass filter 2. The band-limited video signal Sb is outputted via the low-pass filter 2 and given to the switching section 81 . The control signal generation section 82 generates a control signal necessary to control the switching section 81 based on the gaze region data provided from the gaze region separation device 3 . The processing in the switching unit 81 and the gaze time determining unit 83 will be described below.

FIG. 10 is a graph showing the relationship between human gaze time and correct answer rate. Referring to FIG. 10, the relationship between gaze time and information processing in the brain will be described below. For details, please refer to the document titled "Gaze time control presentation device and visual search experiment using it" by Hongo, Yamada et al.
This is described in the Communication Society Image Engineering Study Group Materials (January 1991). The experimental results shown in Figure 10 are as follows: Subjects A, B, and C were given the task of counting the number of circles in a picture studded with patterns such as circles, squares, and triangles, and were given various fixation times. It shows how the correct answer rate for counting changes when Therefore, the horizontal axis shows the gaze time (msec), and the vertical axis shows the correct answer rate (%).
shows. From this figure, with a gaze time of 270 ms, the same correct answer rate as in the normal case where the gaze time is not limited can be obtained. That is, when such a task is given, the gaze time required to perform a correct count is 270 ms, and it is understood that there is no need to spend more time than that to present images to the subject. From various other visual experiments, the time Tv required for visual information processing in the human brain is 30
It is known that the time is around 0ms (±100ms). Therefore, in the video display system shown in FIG. 1, using the above experimental results, after the time Tv has elapsed since the start of gazing on the screen, the video signal of the gazing part is band-limited, so that observation is possible. The present invention aims to reduce the amount of information transmitted in a video signal without reducing the visual information processing ability of a person's images.

FIG. 2 is a timing chart for explaining the processing in the image processing device 8 shown in FIG. Figure 2 (
As shown in A), the gaze areas on the screen are C1, C2, and C.
Assume that they move in the order of 3. That is, Figure 2
As shown in (B) and (C), the observer at time t1
It is assumed that a user watches a region C1 on the screen during a period from t2 to t2, a region C2 from a time t3 to t6, and a region C3 from a time t7 to t8. The vertical axis of FIG. 2(B) represents changes in the observer's state;
In other words, it indicates whether the observer is in a gazing state or a non-gazing state. The vertical axis in FIG. 2(C) is the display device 1.
It shows the change in the band BW of the gaze area component of the video signal given to 0.

The gaze time determination section 83 shown in FIG. 1 determines whether the observer's line of sight remains within a certain area for the above-mentioned time length Tv or more based on the gaze area data generated from the gaze area separation device 3. Determine whether or not the That is, the period during which the observer is gazing at the gaze area C1 is between time t1 and t2, and the time length of this period is shorter than Tv. Therefore, the gaze time determining section 83 outputs a signal BWW that controls the video signal to have a wide band during this period, and provides this to the control signal generating section 82. Next, the observer gazes at area C2 during the period from time t3 to t6. When the gaze time exceeds t4, the gaze time length becomes Tv
Therefore, the gaze time determination unit 83 outputs a signal BWN for controlling the video signal band to be narrowed, and supplies this to the control signal generation unit 82. In response to the signal BWN, the control signal generating section 82 supplies a control signal to the switching section 81 so that the video signal component of the stop area C2 becomes a narrow band. After a predetermined time length Tm has elapsed from time t4, the band of the video signal in the gaze area C2 is widened again. That is, after the elapse of this time Tm, the gaze time determination unit 8
3 outputs the signal BWW, the control signal generating section 82 widens the band of the video signal in the viewing area C2.

[0014] The observer gazes at the region C3 during the period from time t7 to t8, but since this time length does not exceed Tv, the band of the video signal in the gaze region C3 remains wide.

Next, the operation of the control signal generating section 82 shown in FIG. 1 will be explained. The switching unit 81 receives a video signal S that is not band-limited, that is, has a wide band.
a and a video signal Sb whose band is limited by the low-pass filter 2, that is, has a narrow band. The control signal generation unit 82 generates a control signal necessary for controlling the band in the viewing area of the video signal to the switching unit 81.
give to That is, the control signal generation unit 82 generates a control signal necessary for controlling the band of the video signal in the gaze area in response to the band control signal BW given from the gaze time determination unit 83, and transmits the control signal to the switching unit. Give to 81. The switching unit 81 synthesizes a video signal based on the received video signals Sa and Sb in response to a given control signal. As a result, the switching section 81 generates a composite video signal Sc in which the band of the video signal in the viewing area is controlled, and this composite video signal is transmitted to the display device 10. Since the synthesized video signal Sc to be transmitted has a reduced band in the video signal component of the viewing area according to the viewer's viewing state, the amount of transmitted information is reduced.

In the embodiment shown in FIG. 1, a case has been described in which the low-pass filter 2 is used to limit the band of the video signal; however, the present invention is not limited to this. It has been pointed out that it is also effective to reduce the transmission band by making it monochrome in some cases.

Visual V that determines the radius of the gaze area on the screen
For example, 3° to 5° is appropriate for θ, including Parafovea, which has the second highest resolution after the fovea and performs parallel processing. As for the time length Tm, for example, 200 ms (±100 ms), which is the time between the next gaze points after the time Tv has elapsed, is appropriate. This means that after the time Tv has elapsed, the human eye generally
This is based on the fact that a decrease in visual function called Saccadic Suppression has been reported at this time because the patient starts a flying-like movement called Saccadic Suppression. In addition,
For time lengths Tv and Tm and visual Vθ,
It changes depending on the object to be observed, and can be arbitrarily set through visual experiments.

In the following explanation, processing for detecting the movement of the observer's line of sight and extracting the gaze area based on the movement in the video display system shown in FIG. 1 will be described.

FIG. 5 shows the head motion detection device 6 shown in FIG.
FIG. Referring to FIG. 5, this head motion detection device 6 includes an excitation coil 61 that generates a reference magnetic field, a drive circuit 65 for AC driving the excitation coil 61, and a sensor attached to the head of the observer. a detection circuit 63 that detects the voltage induced in the sensor coil 62 according to the movement of the observer's head; and a detection circuit 63 that detects the movement of the observer's head through arithmetic processing based on the voltage detected by the detection circuit 63. and a CPU 64 that calculates the . The CPU 64 also controls a drive circuit 65. The excitation coil 61 has three axes (X
, Y, Z) direction. Each coil is provided with a signal generated from the drive circuit 65.
Driving power supply voltages having different frequencies are supplied. Therefore, a magnetic field based on the three axes is generated by the excitation coil 61, and the observer is present in the magnetic field. The sensor coil 62 also includes three detection coils wound in the (X, Y, Z) directions. When the observer's head moves, an electromotive force is induced in each coil in the sensor coil 62 in accordance with the movement. By detecting the electromotive force generated in each coil by the detection circuit 63, a signal corresponding to the movement of the observer's head can be obtained. The CPU 64 is
By performing arithmetic processing on the basis of this signal, head motion data H indicating the movement of the observer's head is obtained and output.

FIG. 3 is a schematic diagram showing the direction of each motion parameter of the observer's head motion. The aforementioned sensor coil 62 is attached to the forehead of the observer. The parameters Hx, Hy, Hz, Hθ, Hφ, and Hψ shown in FIG. 3 each indicate the movement of the head movement in the direction shown in this figure. That is, Hx indicates horizontal parallel motion of the head, Hy indicates forward-backward motion, and Hz indicates vertical parallel motion. Furthermore, Hθ indicates a movement in the direction of tilting the neck to the left and right, Hφ indicates a rotational movement of the neck in the vertical direction, and Hψ indicates a rotational movement of the head in the left and right direction.

FIG. 4 is a schematic diagram showing how an observer looks at the display device 10. Referring to FIG. 4, the observer:
The user is observing the screen of the display device 10 while wearing glasses such as goggles. The above-mentioned excitation coil 61 is placed at a predetermined position. A sensor coil 62 (not shown) is attached to glasses 66. As the observer's head moves, the relative positional relationship between the excitation coil 61 and the sensor coil 62 changes, and head movement is detected.

FIG. 6 shows the eye movement detection device 5 shown in FIG.
FIG. 2 is a schematic diagram showing the detection principle in FIG. With this device,
An example of this method uses a scleral reflection method that utilizes the difference in reflectance between the white and iris of the corneal surface. As other methods, a corneal reflection method and a search coil method can also be applied. Referring to FIG. 6, sensors 71 and 72 are placed in front of left and right eyes E1 and E2, respectively. Each sensor 7
A light emitting element 73 is arranged at the center of 1 and 72. As the light emitting element 73, a light emitting diode that emits infrared radiation with a relatively wide directivity of about ±21° is used. A light receiving element 74 is provided on both sides of the light emitting element 73. As the light receiving element 74, a photodiode with sharp directivity of approximately ±10° is used. The light projected from the light emitting element 73 toward the eyeball is reflected at the white and iris portions of the eye. The whites of the eyes and the iris have different reflectances, so if you amplify this difference in reflectance and take the difference, you will get the horizontal (left and right) eye movements as an output, and if you add the sum, you will get the vertical (up and down) eye movements. The movement of is obtained as output.

The positions of the sensors 71 and 72 with respect to the eyeball are different between the horizontal direction and the vertical direction, with the horizontal sensor 71 detecting the reflected light at the center of the eyeball above and below, and the vertical sensor 72 detecting the reflected light downward. and are arranged so as to detect the reflected light. A horizontal sensor 71 is placed on one eye E1, and a vertical sensor 72 is placed on the other eye E2. If these are used simultaneously, two-dimensional eye movement can be detected. Using this detection principle, the moving speed, direction, distance, and
For example, a method for determining gaze time, etc.
It is disclosed in Japanese Patent No. 26140. Therefore, data E indicating the observer's eyeball movement is obtained from the eyeball movement detection device 5 shown in FIG. 1 and is provided to the gaze point processing device 7.

FIG. 7 is a schematic diagram of glasses 66 worn by an observer. The glasses 66 are provided with a horizontal sensor 71 for detecting eye movement and a sensor coil 62 for detecting head movement.

FIG. 8 is a schematic diagram for explaining the movement of the line of sight due to head movement. Referring to FIG. 8, generally when a visual object moves, the eyeball moves to follow the visual object. Instead of moving the eyeballs, it is also possible to track the visual object by moving the head. Usually, both movements are used together. Head movements include parallel movements caused by the orientation of the feet and spine, and rotational movements realized by the neck, spine, hips, legs, etc. This head movement is converted into an eyeball rotation angle by a method described later, and a head movement corrected eyeball rotation angle α shown in FIG. 8 is obtained. In the gaze point processing device 7 shown in FIG. 1, line of sight data is expressed by a combination of the head movement corrected eyeball rotation angle α and the eyeball rotation angle θ. Here, the sum of the rotation angles α and θ is collectively defined as the line of sight.

The processing in the gaze point processing device 7 shown in FIG. 1 will be explained below. The gaze point processing device 7 receives head movement data H and eye movement data E. It is assumed that (Hx, Hy, Hz) is given as the horizontal motion component of data H, and (Hφ, Hθ, Hψ) is given as the rotational motion component. The line of sight changes due to the horizontal component of head movement (Hx, Hy, Hz), which can be expressed as the eye rotation angle (Ex, Ey).
The following formula is used to convert to . Here, D is the distance between the subject and the observation target.

[0027]

[Math 1]

When the head is tilted by Hθ towards the left or right shoulder, the coordinates of the eye movement system rotate. Therefore, H
It is necessary to convert the eye movement coordinate system (Xe, Ye) tilted by θ to a coordinate system (Xe', Ye') orthogonal to the original observation object. Therefore, the following formula is used.

[0029]

[Formula 2] Xe′=Xe・cosHθ+Ye・sinHθ
...(3) Ye'=-X
e・sinHθ+Ye・cosHθ
(4) The movement of the line of sight (Xh, Yh) realized by head movement is expressed by the following equation from equations (1) and (2).

[0030]

[Math. 3] Xh=Ex+Hψ

...(5) Yh=Ey+Hφ

(6) Therefore, the movement of the line of sight (Xv, Yv) in consideration of the movement of the head is expressed by the following equation from equations (3) to (6).

[0031]

[Formula 4] Xv=Xe′+Xh

...(7) Yv=Ye′+Yh

...(8) By using equations (7) and (8), it is possible to reproduce the normal movement of the line of sight that is performed by combining head movement and eyeball movement. The above processing is performed in the gaze point processing device 7 shown in FIG. 1, and gaze data indicating the movement of the gaze is given to the gaze area separation device 3.

The gaze area separation device 3 obtains data indicating the observer's gaze area on the screen based on the given line of sight data. Specifically, as shown in FIG. 2A, the radius Dth of the gaze area C3 is determined by the product of a predetermined threshold speed Vth and a sampling interval Ts. That is, assuming the current viewpoint position as P1, the radius D
The area determined by th is defined as the gaze area C3 on the screen.

Since the coordinates of the data obtained by line-of-sight detection are different from the coordinates on the screen of the display device 10, the following coordinate transformation is performed. FIG. 9 is a coordinate diagram showing the correspondence between coordinates in line-of-sight detection and coordinates on the screen. Referring to FIG. 9A, it is assumed that the screen of the display device 10 is placed at a distance D from the viewer, and the screen size is Dv in height and Dh in width. Therefore, the coordinates on the screen are represented by (1, 1)...(Px, Py). Therefore, the viewing angle of the screen as seen from the observer is expressed by the following equation, as shown in FIG. 9(B).

[0034]

[Formula 5] a=180/π・tan-1Dh/(2・D)
...(9)b=18
0/π・tan−1Dv/(2・D)
...(10) Here, if the line-of-sight movement angle is Xv, Yv, then this is the coordinate Xp, Yp on the screen.
The formula for converting into is expressed by the following formula.

[0035]

[Formula 6] Xp=f(Xv)=p+(r-p)・Xv/(a-Xv
) ...(11) Yq=g(Yv)=q+(
s-q)・Yv/(b-Yv)...(12
) Therefore, by using the above equation, the coordinate data of the gaze area and its surrounding area obtained based on the line of sight detection can be converted into coordinate data on the screen. As a result, from the control signal generation section 82 shown in FIG.
It is possible to generate a control signal necessary for performing image processing while distinguishing between a gaze area and its surrounding area.

[0036]

[Effects of the Invention] As described above, according to the present invention, a video information reducing means is provided for reducing the amount of video information in the viewing area of the video signal to be transmitted in accordance with the length of the viewer's viewing time. Therefore, a video display system capable of reducing the amount of information in a video signal transmitted to a display device has been obtained.

[Brief explanation of drawings]

FIG. 1 is a block diagram of a video display system showing an embodiment of the present invention.

FIG. 2 is a timing diagram for explaining processing in the image processing apparatus shown in FIG. 1.

FIG. 3 is a schematic diagram showing the direction of each motion parameter of the observer's head motion.

FIG. 4 is a schematic diagram showing how an observer looks at the display device.

FIG. 5 is a block diagram of the head motion detection device shown in FIG. 1.

6 is a schematic diagram showing a detection principle in the eye movement detection device shown in FIG. 1. FIG.

FIG. 7 is a schematic diagram of glasses worn by an observer.

FIG. 8 is a schematic diagram for explaining the movement of the line of sight during head movement.

FIG. 9 is a coordinate diagram showing the correspondence between coordinates in line-of-sight detection and coordinates on the screen.

FIG. 10 is a graph showing the relationship between human gaze time and correct answer rate.

[Explanation of symbols]

1 Video signal generator 2 Low pass filter 7 Gaze point processing device 8 Image processing device 10 Display device 81 Switching section 82 Control signal generation section 83 Gaze time determination unit

Claims (2)

[Claims] 1. A video signal generating means, a video display means for displaying a video on a screen based on the video signal generated from the video signal generating means, and an observation for observing the video displayed by the video display means. gaze state detection means for detecting a gaze area and gaze time length on a screen of a person; and determining that the gaze time length detected by the gaze state detection means exceeds a predetermined time length. a gaze time length determining means; and a video information reducing means for reducing the amount of video information in the gaze area of the video signal generated from the video signal generating means in response to the gaze time length determining means; The video display system is a video display system in which the video display means displays video on the screen based on the video signal whose information amount has been reduced by the video information reduction means. 2. The predetermined time length is determined based on a human gaze time length-correct answer rate characteristic, and the video information reduction means responds to the gaze time length determination means to: 2. The video display system according to claim 1, further comprising video band reducing means for reducing a band of a gaze area component of the video signal generated by said video signal generating means.