JP2013186254A - Image presentation apparatus and image presentation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image presentation apparatus capable of improving image quality by suppressing optical shot noise by signal processing while holding performance of an imaging device at conventional one.SOLUTION: An image presentation apparatus 10 includes: sight line calculation means 11 for calculating a sight line position of a viewer A on a screen of a display apparatus 50 from an image of an imaging apparatus 30 of the viewer A; area determination means 12 for determining whether or not respective pixels on the screen of the display apparatus 50 are included in a preset area range around the sight line position of the viewer A; and addition processing means 13 for reducing time resolution by performing additional average processing of pixels of an image displayed on pixel positions determined as positions included in the area range by the area determination means 12 and pixels on the same positions in a preceding frame and reducing space resolution by performing additional average processing of pixels determined as pixels not included in the area range by the area determination means 12 and pixels of peripheral positions in the same frame.

Description

  The present invention relates to a video presentation device and a video presentation program that present video to a viewer via a display device, and in particular, an ultra-high definition and large screen that can provide a wide viewing angle to the viewer. The present invention relates to a video presentation device and a video presentation program using a display device such as a super high vision projector display device or a liquid crystal type or plasma display type direct view type display device.

  For example, as shown in Patent Document 1, a conventional video presentation device presents a video having a uniform spatial resolution and temporal resolution of a video to be displayed on the entire screen, that is, a spatial frequency and a frame frequency of the video to be displayed.

JP 2011-244217 A

  However, when the S / N ratio of the camera that shoots the video is poor, the noise component of the camera causes the video quality to deteriorate. Although the signal-to-noise ratio of a camera has been improved by advances in imaging device design / manufacturing technology and electronic circuit technology, the number of pixels of the imaging device has increased in recent years, and this is the case for shooting with a fixed light receiving area. As a result, the photodiode capacity per pixel is reduced, and an image having a smaller accumulated charge capacity of the image sensor is used. Further, for example, in full spec / super high-vision expected as a next-generation ultra-high-definition television system, 59.94, which has been used in conventional high-definition, etc., to shoot images with high image quality and little motion blur. Shooting is performed at a frame frequency of 120 frames / second exceeding field / second (29.97 frames / second). Therefore, in the future, the amount of light charge accumulated in the image sensor per frame will be further reduced, and the influence of light shot noise is expected to be greater than before.

  The light shot noise described above is random noise resulting from fluctuations in the amount of light incident on one pixel within a certain time, and is generated when light is photoelectrically converted by the image sensor. is there. In addition, since the light shot noise is determined by the amount of charge of light accumulated in one image sensor, as described above, the effect becomes larger as the amount of charge of light accumulated in the image sensor decreases. This hinders viewing. Since the influence of such light shot noise is fundamental, a technology for avoiding this has not been developed so far.

  The present invention has been made in view of the above points, and is an image presentation device capable of suppressing light shot noise by signal processing and improving image quality while maintaining the performance of an image pickup device photographed by a camera as usual. It is another object of the present invention to provide a video presentation program.

  In order to solve the above problem, the video presentation apparatus according to claim 1 recognizes the visual field of the viewer who views the video, generates video having different resolutions in the central region and the peripheral region of the visual field, and displays the video. The video presentation device that presents to the viewer via the display device includes a line-of-sight calculation unit, a region determination unit, and an addition processing unit.

  The video presentation apparatus having such a configuration calculates the viewer's line-of-sight position on the screen of the display device from the video captured by the imaging apparatus that captures the viewer's eyeballs by the line-of-sight calculation means. In addition, the video presentation device determines whether each pixel on the screen of the display device is included in a predetermined region range centered on the viewer's line-of-sight position by the region determination unit. Then, the video presentation device performs addition averaging processing on the pixel of the video displayed at the pixel position determined to be included in the region range by the region determination unit by the addition processing unit and the pixel at the same position in the previous frame. The spatial resolution is reduced by performing the averaging process on the pixels determined not to be included in the region range by the region determination means and the pixels at the peripheral positions in the same frame.

  Thus, the video presentation device calculates the viewer's line-of-sight position on the screen of the display device by the line-of-sight calculation means, and each pixel on the screen of the display device is placed in the central region of the viewer's field of view by the region determination means. It is determined whether or not it is included. The video presentation device can reduce the temporal resolution of the video by performing frame addition processing on the pixels in the central region of the viewer's visual field by the addition processing means, and can also reduce the peripheral region of the viewer's visual field. By performing the pixel addition process for the pixels within, it is possible to reduce the spatial resolution of the video. Therefore, the video presentation device can present videos having different temporal resolution or spatial resolution for each pixel position.

  Note that the shape of the region serving as a reference for determination by the region determining means can be changed flexibly according to the viewing characteristics of the viewer, and for example, regions of various shapes such as a perfect circle, an ellipse, and a polygon The area can be determined based on the above.

  According to a second aspect of the present invention, the video presentation device according to the first aspect includes a viewing distance calculation unit.

  The video presentation apparatus having such a configuration calculates the viewing distance, which is the distance between the viewer and the display device, from the video captured by the imaging device by the viewing distance calculation means. Then, the video presentation device determines whether each pixel on the screen of the display device is included in the region range set according to the viewing distance centered on the viewer's line-of-sight position by the region determination unit. judge.

  Thus, the video presentation apparatus can calculate the viewing distance of the viewer by the viewing distance calculation means, and perform the area determination by using the area range having a different size for each viewing distance of the viewer by the area determination means. it can. Therefore, the video presenting apparatus can change the size of the central area of the viewer's visual field, which is a reference for pixel addition and frame addition, according to the distance of the viewer to the display device. For example, when the viewer views from a position close to the display device, the video presentation device can perform region determination using a small central region and perform an averaging process based on the result, and the viewer can display When viewing at a position far from the apparatus, it is possible to perform area determination using a large central area and perform an averaging process based on the result.

  In order to solve the above problem, the video presentation device according to claim 3 recognizes a visual field of a viewer who views the video, generates a video having different resolutions in a central region and a peripheral region of the visual field, A video presentation device that presents a video to a viewer via a display device, provided in a line-of-sight calculation means, a region determination means, and an imaging device that outputs the video to the display device. A photoelectric conversion unit that converts the charge, a charge-voltage conversion unit that converts the charge transferred from the photoelectric conversion unit into a voltage and stores it, and a transfer gate transistor that transfers the charge stored in the photoelectric conversion unit to the charge-voltage conversion unit An imaging element having a selection transistor that selects whether or not to transfer the charge to the charge-voltage conversion unit by the transfer gate transistor, and a reset transistor that resets the charge accumulated in the photoelectric conversion unit If, and configured to include a.

  The video presentation apparatus having such a configuration calculates the viewer's line-of-sight position on the screen of the display device from the video captured by the imaging apparatus that captures the viewer's eyeballs by the line-of-sight calculation means. In addition, the video presentation device determines whether each pixel on the screen of the display device is included in a predetermined region range centered on the viewer's line-of-sight position by the region determination unit. Then, the video presentation device is configured to reset the reset transistor when charges corresponding to the pixels of the image displayed at the pixel position determined to be included in the region range by the region determination unit are accumulated in the photoelectric conversion unit by the image sensor. By performing no reset according to the above, pixels that are determined to be included in the region range are subjected to addition averaging processing with pixels at the same position in the previous frame to reduce time resolution, and are not included in the region range by the region determination unit The charge corresponding to the pixel of the image displayed at the pixel position determined to be and the surrounding pixels are simultaneously transferred to the charge-voltage converter by the transfer gate transistor according to the selection by the selection transistor, and included in the region range. For pixels that are determined to be averaged, the averaging process is performed with the pixels at the peripheral positions in the same frame. The decreasing spatial resolution by performing.

  Thus, the video presentation device calculates the viewer's line-of-sight position on the screen of the display device by the line-of-sight calculation means, and each pixel on the screen of the display device is placed in the central region of the viewer's field of view by the region determination means. It is determined whether or not it is included. The video presentation device can reduce the temporal resolution of the video by performing frame addition processing on the pixels in the central region of the viewer's visual field in the imaging device, and can also reduce the peripheral region of the viewer's visual field. By performing the pixel addition process for the pixels within, it is possible to reduce the spatial resolution of the video. Therefore, the video presentation device can present videos having different temporal resolution or spatial resolution for each pixel position.

  In order to solve the above problem, the video presentation device according to claim 4 recognizes a visual field of a viewer who views the video, generates a video having different resolutions in a central region and a peripheral region of the visual field, A video presentation device that presents video to a viewer via a display device, and includes a line-of-sight calculation unit, a region determination unit, and an addition processing unit.

  The video presentation apparatus having such a configuration calculates the viewer's line-of-sight position on the screen of the display device from the video captured by the imaging apparatus that captures the viewer's eyeballs by the line-of-sight calculation means. In addition, the video presentation device receives information about the speed at which the video changes from the game device that outputs the video to the display device to the region determination unit, and each region on the screen of the display device is displayed by the region determination unit. It is determined whether or not the pixel is included in an area range that is set in advance according to information about the speed centered on the viewer's line-of-sight position. Then, the video presentation device performs addition averaging processing on the pixel of the video displayed at the pixel position determined to be included in the region range by the region determination unit by the addition processing unit and the pixel at the same position in the previous frame. The spatial resolution is reduced by performing the averaging process on the pixels of the video displayed at the pixel position determined not to be included in the area range by the area determination means and the pixels at the peripheral positions in the same frame. Reduce.

  Thus, the video presentation device calculates the viewer's line-of-sight position on the screen of the display device by the line-of-sight calculation means, and each region of the pixel on the display device screen is viewed according to the video speed by the region determination means. It is determined whether or not it is included in the central region of the person's visual field. The video presentation device can reduce the temporal resolution of the video by performing frame addition processing on the pixels in the central region of the viewer's visual field by the addition processing means, and can also reduce the peripheral region of the viewer's visual field. By performing the pixel addition process for the pixels within, it is possible to reduce the spatial resolution of the video. Therefore, the video presentation device can present video having different temporal resolution or spatial resolution for each pixel position in accordance with the speed of the video displayed by the display device.

  In order to solve the above problem, the video presentation program according to claim 5 recognizes a visual field of a viewer who views the video, generates a video having different resolutions in a central region and a peripheral region of the visual field, In order to present the video to the viewer via the display device, the computer is configured to function as a line-of-sight calculation unit, a region determination unit, and an addition processing unit.

  The video presentation program having such a configuration calculates the viewer's line-of-sight position on the screen of the display device from the video captured by the imaging device that captures the viewer's eyeballs by the line-of-sight calculation means. Also, the video presentation program determines whether each pixel on the screen of the display device is included in a preset region range centered on the viewer's line-of-sight position by the region determination unit. Then, the video presentation program performs an addition averaging process on the pixels of the video displayed at the pixel positions determined to be included in the region range by the region determination unit by the addition processing unit and the pixels at the same position in the previous frame. The spatial resolution is reduced by performing the averaging process on the pixels determined not to be included in the region range by the region determination means and the pixels at the peripheral positions in the same frame.

  Thus, the video presentation program calculates the viewer's line-of-sight position on the screen of the display device by the line-of-sight calculation means, and each pixel on the screen of the display device is placed in the central region of the viewer's field of view by the region determination means. It is determined whether or not it is included. The video presentation program can reduce the temporal resolution of the video by performing frame addition processing on the pixels in the central region of the viewer's visual field by the addition processing means, and can also reduce the peripheral region of the viewer's visual field. By performing the pixel addition process for the pixels within, it is possible to reduce the spatial resolution of the video. Therefore, the video presentation program can present videos with different temporal resolution or spatial resolution for each pixel position.

  In order to solve the above problem, a video presentation program according to claim 6 recognizes a visual field of a viewer who views a video, generates a video having different resolutions in a central region and a peripheral region of the visual field, In order to present the video to the viewer via the display device, the computer is configured to function as a line-of-sight calculation unit, a region determination unit, and an addition processing unit.

  The video presentation program having such a configuration calculates the viewer's line-of-sight position on the screen of the display device from the video captured by the imaging device that captures the viewer's eyeballs by the line-of-sight calculation means. In addition, the video presentation program receives information about the speed at which the video changes from the game device that outputs the video to the display device to the region determination unit, and each pixel on the screen of the display device is viewed by the region determination unit. It is determined whether or not the image is included in an area range set in advance according to information related to the speed, centered on the person's line-of-sight position. Then, the video presentation program performs an addition averaging process on the pixels of the video displayed at the pixel positions determined to be included in the region range by the region determination unit by the addition processing unit and the pixels at the same position in the previous frame. The spatial resolution is reduced by performing the averaging process on the pixels of the video displayed at the pixel position determined not to be included in the area range by the area determination means and the pixels at the peripheral positions in the same frame. Reduce.

  Thus, the video presentation program calculates the viewer's line-of-sight position on the screen of the display device by the line-of-sight calculation means, and each region of the pixels on the display device screen is viewed according to the video speed by the region determination means. It is determined whether or not it is included in the central region of the person's visual field. The video presentation program can reduce the temporal resolution of the video by performing frame addition processing on the pixels in the central region of the viewer's visual field by the addition processing means, and can also reduce the peripheral region of the viewer's visual field. By performing the pixel addition process for the pixels within, it is possible to reduce the spatial resolution of the video. Therefore, the video presentation program can present videos having different temporal resolution or spatial resolution for each pixel position in accordance with the speed of the video displayed by the display device.

  According to the first and fifth aspects of the present invention, by presenting images having different temporal resolution or spatial resolution between the central region and the peripheral region of the viewer's line of sight, a place (pixel position) and time (frame) are presented. Therefore, it is possible to average the variation in light due to the light shot noise in which the light signal amount changes randomly, so that the influence of the light shot noise can be reduced without changing the performance of the image sensor.

  According to the second aspect of the present invention, since the central region and the peripheral region of the viewer's field of view are determined in consideration of the distance between the viewer and the display device, the time resolution or space between the central region and the peripheral region is determined. Even when the resolution is changed, it is possible to present a natural video that the viewer does not feel uncomfortable.

  According to the third aspect of the present invention, it is possible to average variations in light due to light shot noise by presenting images having different temporal resolution or spatial resolution between the central region and the peripheral region of the viewer's line of sight. Therefore, the influence of light shot noise can be reduced without changing the performance of the image sensor. According to the invention of claim 3, since the pixel addition process and the frame addition process can be performed by the image sensor in the imaging apparatus, the number of configurations of the entire apparatus can be reduced and the cost can be reduced.

  According to the inventions according to claim 4 and claim 6, by presenting images having different temporal resolution or spatial resolution between the central region and the peripheral region of the viewer's line of sight, light shot noise is generated in the image from the game device. Even if it is included, since the variation in light due to the light shot noise can be averaged, the influence of the light shot noise can be reduced without changing the performance of the image sensor. According to the fourth and sixth aspects of the present invention, when the video (game video) from the game device changes with speed, the temporal resolution or spatial resolution of the video can be changed according to the speed. Therefore, it is possible to present a realistic video to the viewer.

It is the schematic which shows the whole structure of the video presentation system containing the video presentation apparatus which concerns on this invention. It is a block diagram which shows the internal structure of the image | video presentation apparatus which concerns on 1st Embodiment of this invention. It is the schematic for demonstrating the process in the area | region determination means with which the video presentation apparatus which concerns on 1st Embodiment of this invention is provided. In the video presentation device concerning the present invention, it is a schematic diagram showing an example of a central field of a viewer's field of view, and a peripheral field. FIG. 6 is a schematic diagram illustrating another example of the center region and the peripheral region of the viewer's visual field in the video presentation device according to the present invention. It is the schematic for demonstrating the process in the addition process means with which the video presentation apparatus which concerns on 1st Embodiment of this invention is provided. It is a flowchart which shows operation | movement of the video presentation apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the internal structure of the image | video presentation apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the internal structure of the video presentation apparatus which concerns on 3rd Embodiment of this invention. It is the outline which shows the process in the imaging device with which the video presentation apparatus which concerns on 3rd Embodiment of this invention is provided, Comprising: It is a figure which shows the example of a frame addition process. It is the outline which shows the process in the imaging device with which the video presentation apparatus which concerns on 3rd Embodiment of this invention is provided, Comprising: It is a figure which shows the example of a pixel addition process. It is a block diagram which shows the internal structure of the video presentation apparatus which concerns on 4th Embodiment of this invention. It is the schematic which shows the example of an image | video presentation by the image | video presentation apparatus which concerns on 4th Embodiment of this invention, Comprising: (a) is a figure which shows the case where the car in an image | video has stopped, (b) is an image | video. The figure shows a case in which the car in the vehicle is traveling at a speed of 50 km / h and the center region of the viewer's field of view is a horizontally long ellipse, and (c) shows the car in the image traveling at a speed of 200 km / h. It is a figure which shows the case where it is a case, Comprising: The center area | region of a viewer's visual field is made into a horizontally long ellipse. It is the schematic which shows another presentation example of the image | video by the image | video presentation apparatus which concerns on 4th Embodiment of this invention, Comprising: (a) is the case where the car in the image | video is drive | working at 50 km / h, Comprising: FIG. 5B is a diagram showing a case where the center region of the viewer's field of view is a horizontally long rectangle; FIG. 7B is a case where the vehicle in the video is traveling at a speed of 200 km / h, and the center region of the viewer's field of view is a horizontally long rectangle; (C) is a diagram showing a case where the car in the video is traveling at a speed of 200 km / h, and the center region of the viewer's field of view is a vertically long rectangle.

  Hereinafter, a video presentation device according to an embodiment of the present invention will be described with reference to the drawings. In addition, the size, positional relationship, and the like of members shown in each drawing may be exaggerated for convenience of explanation. Furthermore, in the following description, the same name and reference sign indicate the same or the same members in principle, and the detailed description will be omitted as appropriate.

[First Embodiment]
The configuration of the video presentation device 10 according to the first embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 1, the video presentation device 10 presents video to the viewer A via the display device 50. Specifically, as shown in FIG. 1, the video presentation device 10 recognizes the visual field of the viewer A who views the video by the display device 50, and generates video with different resolutions in the central region and the peripheral region of the visual field. Then, the video is presented to the viewer A via the display device 50.

  Here, the video presentation apparatus 10 constitutes a part of the video presentation system 1 as shown in FIG. As shown in FIG. 1, the video presentation system 1 includes an imaging device 20, an imaging device 30, and a display device 50 in addition to the video presentation device 10. Hereinafter, configurations other than the video presentation device 10 of the video presentation system 1 will be described first.

  The imaging device (eyeball imaging device) 20 photographs the eyeball of the viewer A. The imaging device 20 is, for example, a camera provided in the home as shown in FIG. 1, and outputs the captured image of the eyeball to the line-of-sight calculation means 11 in the video presentation device 10 as shown in FIG. 2.

  The imaging device (video imaging device) 30 captures a video displayed by the display device 50. The imaging device 30 is, for example, a television camera for outdoor relaying as shown in FIG. 1 or a television camera in a studio of a television station, and as shown in FIG. Output to. Instead of the imaging device 30, as shown in FIG. 2, a configuration may be adopted in which video is input to the addition processing means 13 from a video playback device 40 installed in a television station or the like.

  The display device 50 displays the video imaged by the imaging device 30 to the viewer A. The display device 50 is, for example, a television set provided in a home, and more specifically, a super high-vision projector display device or a liquid crystal type or plasma display type direct-view type display device. As shown in FIG. 2, when the video reproduction device 40 is used instead of the imaging device 30, the display device 50 displays the video of the video reproduction device 40 to the viewer A.

  Hereinafter, the configuration of the video presentation device 10 will be described. Here, as shown in FIG. 2, the video presentation device 10 includes a line-of-sight calculation unit 11, a region determination unit 12, and an addition processing unit 13.

  The line-of-sight calculation means 11 calculates the line-of-sight position of the viewer A. As shown in FIG. 2, an image of the eyeball of the viewer A is input to the line-of-sight calculation unit 11 from the imaging device 20. The line-of-sight calculation unit 11 calculates the line-of-sight position of the viewer A on the screen of the display device 50 from the image of the eyeball input from the imaging device 20, and as shown in FIG. 12 is output. Note that the method for calculating the line-of-sight position of the viewer A is not particularly limited, and for example, the method disclosed in “JP-A-2005-323905” can be used.

  The area determination unit 12 determines whether each pixel of the screen of the display device 50 is included in the central area of the viewer A's visual field. As shown in FIG. 2, the region determination unit 12 is input with the line-of-sight position of the viewer A from the line-of-sight calculation unit 11 and pixel information of the display device 50 from the outside. Here, the pixel information of the display device 50 indicates information related to the number of pixels on the screen of the display device 50. Then, the region determination unit 12 determines whether or not each pixel on the screen of the display device 50 is included in a preset region range centered on the viewing position of the viewer A. In addition, the area | region range has shown the center area | region (refer FIG. 4 and FIG. 5) of the viewer's A visual field so that it may mention later.

  Here, several methods of region determination by the region determination unit 12 are conceivable. Hereinafter, a method of performing threshold processing on a distance for each pixel on the screen of the display device 50 using a preset threshold will be described. In this method, as shown in FIG. 3, the region determination unit 12 firstly has a line-of-sight position E (x, y, t) of the viewer A and each pixel position P (x, y) on the screen of the display device 50. , T) and a distance D (x, y). This distance D (x, y) can be expressed as, for example, the following expression (1).

  Here, x in Equation (1) indicates the horizontal coordinate on the screen of the display device 50, y indicates the vertical coordinate on the screen of the display device 50, and t indicates time. .

Next, the area determination unit 12 determines whether or not the distance D (x, y) calculated by the equation (1) is less than a preset threshold value D ₀ on the screen of the display device 50. The determination is made for each pixel position P (x, y, t), and these determination results are output to the addition processing means 13 as shown in FIG. The threshold D ₀ is calculated according to the viewing characteristics of the viewer A and is input to the area determination unit 12 in advance. By such processing, the region determination unit 12 determines whether each pixel on the screen of the display device 50 is included in the central region of the viewer A's field of view, as shown in FIG. Can be determined (whether it is not included in the central region of the viewer A's field of view). That is, the threshold value D ₀ used here is for selecting the central region and the peripheral region of the visual field in FIG. 4, and the distance D (x, y) from the line-of-sight position E (x, y, t). the pixel but is less than the threshold D _0, is in the central region of the visual field of the viewer a, the pixels sight position E (x, y, t) the distance between the D (x, y) is greater than the threshold value D _0, If it is within the peripheral region of the viewer A's field of view, it will be selected.

In addition, the central area of the visual field means, as shown in FIG. 4, an area range of a predetermined size centered on the viewer A's line-of-sight position. It means a portion other than the central region of the visual field on the screen of the display device 50. The size of the central region of the visual field is determined by the threshold value D ₀ described above, the central region as the threshold value D ₀ is large increases, the central region as the threshold value D ₀ is small decreases. Further, since the area determination unit 12 performs threshold processing using a fixed threshold D ₀ input from the outside, the threshold D ₀ is set to the radius of the central area of the viewer A's visual field as shown in FIG. Accordingly, the central area of the visual field of the viewer A derived by the threshold processing described above is a perfect circle as shown in FIG.

  In FIG. 4, since the region determination unit 12 performs threshold processing on the distance for each pixel on the screen of the display device 50 using one preset threshold value, the center region of the visual field of the viewer A is a perfect circle. However, the shape of the central region can be changed flexibly according to the viewing characteristics of the viewer A, for example, an ellipse shown in FIG. 5A or a shape shown in FIG. Various shapes such as a rectangular shape can be applied.

  For example, as illustrated in FIG. 5A, the region determination unit 12 performs region determination using an ellipse formula when the central region of the field of view is an ellipse. In this case, the major axis radius a of the ellipse and the minor axis radius b of the ellipse are calculated according to the viewing characteristics of the viewer A and input to the region determination unit 12 in advance. Further, as shown in FIG. 2, the region determining unit 12 receives the line-of-sight position E (x, y, t) of the viewer A from the line-of-sight calculating unit 11 and the pixel information of the display device 50 from the outside. Have been entered.

  Then, the area determination unit 12 uses the following formula (2) to make each pixel on the screen of the display device 50 an ellipse centered on the viewer A's line-of-sight position E (x, y, t). It is determined whether or not the region is included in the central region of the viewer A's visual field. That is, the region determination unit 12 sets the line-of-sight position E (x, y, t) of the viewer A to the origin of the ellipse, and then the pixel position P (x, y, t) that satisfies the following expression (2). Is determined to be within the central area of the visual field, and the pixel at the pixel position P (x, y, t) that does not satisfy the following expression (2) is determined to be outside the central area of the visual field. 2, the determination result for each pixel position P (x, y, t) is output to the addition processing means 13.

  Similarly, as shown in FIG. 5B, for example, the region determination unit 12 determines the region using a rectangular formula (not shown) when the central region of the viewer A's field of view is rectangular. The determination result is output to the addition processing means 13. Even when the central region of the viewer A's field of view has a shape other than an elliptical shape or a rectangular shape, the region determination can be performed by using an expression (not shown) of the corresponding shape.

  The addition processing means 13 performs addition averaging processing on the video. As shown in FIG. 2, a video is input to the addition processing unit 13 from the imaging device 30 or the video reproduction device 40, and a determination result is input from the region determination unit 12. Then, the addition processing means 13 performs a frame addition process on the pixel of the video displayed at the pixel position P (x, y, t) determined by the area determination means 12 when included in the preset area range. Then, pixel addition processing is performed on the pixel of the image displayed at the pixel position P (x, y, t) determined by the region determination unit 12 as not being included in the preset region range.

As shown in FIG. 6, the frame addition process means that the pixel values at the same position of two frames in the video are added and averaged, and the pixel addition process is the same frame in the video. It means that the pixel values at different positions are subjected to the averaging process. Hereinafter, the processing performed by the addition processing means 13 will be described separately for frame addition processing and pixel addition processing. In the following, the above-described area determination unit 12, as shown in FIG. 4, using a threshold D ₀ which is set in advance, based on the result of the area determination made in a true circular region, averaging process An example of performing is described.

(Frame addition processing)
Addition processing unit 13 is less than the threshold value D ₀ by the region determining means 12 (i.e. the field of view of the central region) and the determined pixel position P (x, y, t) for the pixels of the image displayed on the frame addition The time resolution is reduced by performing processing, that is, addition averaging processing (see FIG. 6) with pixels at the same position in the previous frame. More specifically, the addition processing means 13 performs low-pass filter processing (addition averaging processing with the previous frame) in the time direction as shown in the following expression (3), and displays the display data P _{frame_avg} (x, y, t ) Is calculated.

Here, P _{frame_avg} in Expression (3) indicates a pixel value after frame addition processing at an arbitrary pixel position P (x, y, t). Further, the lowercase letters p (x, y, t) and p (x, y, t−1) in the expression (3) are arbitrary pixel positions P (x, y, t) and P (x, y, t). The pixel value in -1) is shown. In Expression (3), x indicates the horizontal pixel number of the video, y indicates the vertical pixel number of the video, and t indicates the frame number.

  In this way, by performing the frame addition process by the addition processing unit 13, even if the time resolution (frame rate, frame frequency) of the display device 50 is constant, the same or similar display data is displayed at the same pixel position P (x , Y, t) is displayed a plurality of times, which is equivalent to reducing the temporal resolution of the display device 50 for the viewer A.

(Pixel addition processing)
Addition processing unit 13, exceeds the threshold value D ₀ by the region determining means 12 (i.e. the field of view of the peripheral region) and the determined pixel position P (x, y, t) for the pixels of the image displayed on the pixel addition process That is, the spatial resolution is reduced by performing the averaging process (see FIG. 6) with the pixels at the peripheral positions in the same frame. More specifically, as shown in the following formula (4), the addition processing means 13 performs a low-pass filter process (peripheral 4-pixel addition averaging process) in the spatial direction to display the display data P _{2d_avg} (x, y, t). Is calculated.

Here, P _{2d_avg} in Expression (4) indicates a pixel value after pixel addition processing at an arbitrary pixel position P (x, y, t). In addition, the lowercase letters p (x + 1, y, t), p (x-1, y, t), p (x, y + 1, t), and p (x, y-1, t) in Equation (4) are The pixel values at arbitrary pixel positions P (x + 1, y, t), P (x-1, y, t), P (x, y + 1, t), P (x, y-1, t) are shown. . In Expression (4), x represents the horizontal pixel number of the video, y represents the vertical pixel number of the video, and t represents the frame number.

  In this way, by performing pixel addition processing by the addition processing means 13, even if the spatial resolution (spatial frequency) of the display device 50 is constant, the same or similar display data is displayed at a plurality of pixel positions P (x, y). , T), this is equivalent to reducing the spatial resolution of the display device 50 for the viewer A.

Here, the processing in the addition processing means 13 as described above can be represented by an algorithm as shown in the following formula (5), for example. Note that P _disp in the following equation (5) indicates a pixel value after addition processing at an arbitrary pixel position P (x, y, t).

  The video presentation apparatus 10 having the above-described configuration can present video having different temporal resolution or spatial resolution for each frame constituting the video based on the viewer A's visual field. For example, a full-spec Super Hi-Vision signal as a video format used for shooting and display, that is, the resolution of the entire screen of the display device 50 is equivalent to 33 million pixels of horizontal 7680 pixels × vertical 4320 pixels, and the frame frequency (time resolution) is 120 Hz. Assume a case in which a video / second is presented. In this case, the video presentation apparatus 10 follows the line of sight of the viewer A, and in the center region of the visual field, the spatial resolution of the video to be displayed is set to a high resolution, for example, a spatial resolution corresponding to full spec, and the temporal resolution of the video, That is, an image with a frame frequency lower than the full specification, for example, 60 frames / second is generated and output to the display device 50. On the other hand, the video presentation device 10 sets the spatial resolution of the video to be displayed to a low resolution, for example, a spatial resolution corresponding to half of the full specification, and the temporal resolution of the video, that is, the frame frequency, in the peripheral region of the viewer A's visual field. Is generated at a high rate, for example, 120 frames / second, and is output to the display device 50.

  In this way, the video presentation device 10 calculates the line-of-sight position of the viewer A on the screen of the display device 50 by the line-of-sight calculation unit 11, and each pixel on the screen of the display device 50 is viewed by the region determination unit 12. It is determined whether or not it is included in the center region of the visual field of the person A. Then, the video presentation device 10 can reduce the temporal resolution of the video by performing the frame addition processing on the pixels in the central region of the viewer A's visual field by the addition processing means 13, and By performing pixel addition processing on pixels in the peripheral region of the field of view, the spatial resolution of the video can be reduced. Therefore, the video presentation device 10 can present videos having different temporal resolution or spatial resolution for each pixel position P (x, y, t).

  The reason why the central area of the viewer A's visual field is set to the high resolution and the low frame rate and the peripheral area of the visual field is set to the low resolution and the high frame rate is based on human visual characteristics. That is, human visual characteristics are characterized by being able to recognize even fine objects in the central area of the visual field, but not sensitive to movement, and sensitive to movement in the peripheral area of the visual field, but not recognizing fine objects. Therefore, by taking such human visual characteristics into consideration, the temporal resolution and spatial resolution of the video can be adjusted without making the viewer A feel uncomfortable.

  Therefore, according to the video presentation device 10, by presenting video having different temporal resolution or spatial resolution between the central region and the peripheral region of the viewer A's line of sight, light can be transmitted depending on the location (pixel position) and time (frame). Since it is possible to average the variation in light due to light shot noise whose signal amount changes randomly, the influence of light shot noise can be reduced without changing the performance of the image sensor. Further, according to the video presentation device 10, the viewer A's visual field and line of sight are recognized, and the video matching the visual characteristics of the viewer A is generated and presented in real time, so that the viewer A does not feel uncomfortable. It is possible to view with high video quality.

Hereinafter, the operation of the video presentation device 10 (in the case where a perfect circular area is a central area of the visual field (see FIG. 4)) will be described with reference to FIG. First, the video presentation apparatus 10 calculates the line-of-sight position E (x, y, t) of the viewer A by the line-of-sight calculation means 11 (step S1). Next, the video presentation device 10 calculates a distance D (x, y) between the line-of-sight position E (x, y, t) and the pixel position P (x, y, t) by the region determination unit 12 (step) S2). The video presentation device 10, by the region determining means 12, the distance D (x, y) is judged whether it is smaller than the threshold value _{D 0} which is set in advance (step S3).

Next, when the area determination unit 12 determines that the distance D (x, y) is less than the preset threshold D ₀ (Yes in step S3), the video presentation device 10 uses the addition processing unit 13 Then, frame addition processing is performed for the pixel position P (x, y, t) when the distance D (x, y) is calculated (step S4), and the result is output to the display device 50 (step S6). On the other hand, the video presentation device 10, by the region determining means 12, the distance D (x, y) if it is determined to exceed the threshold value _{D 0} which is set in advance (No in step S3), and the addition processing unit 13, the Pixel addition processing is performed for the pixel position P (x, y, t) when the distance D (x, y) is calculated (step S5) and output to the display device 50 (step S6).

  The video presentation device 10 performs the above-described procedure for each pixel on the screen of the display device 50, thereby performing frame addition processing on the pixels in the central region of the viewer A's field of view and reducing the time resolution. In addition, it is possible to reduce the spatial resolution by performing pixel addition processing on the pixels in the peripheral region of the visual field of the viewer A.

[Second Embodiment]
Hereinafter, an image presentation device 10A according to a second embodiment of the present invention will be described with reference to FIG. As shown in FIG. 8, the video presentation device 10 </ b> A includes a region determination unit 12 </ b> A instead of the region determination unit 12, and further includes a viewing distance calculation unit 14, according to the first embodiment. It has the same configuration as Therefore, in the following description, it demonstrates centering on difference with the video presentation apparatus 10, and detailed description is abbreviate | omitted about the structure which overlaps with the said video presentation apparatus 10, and operation | movement of 10 A of video presentation apparatuses.

The video presenting apparatus 10A is characterized in that the size of the area range (center area) used in the area determination unit 12A varies depending on the distance between the viewer A and the display device 50. That is, as shown in FIG. 8, the video presentation device 10 </ b> A determines a region using a different threshold D ₀ or a different expression according to the distance between the viewer A and the display device 50 calculated by the viewing distance calculation unit 14. I do.

  The viewing distance calculation unit 14 calculates a viewing distance that is the distance between the viewer A and the display device 50 from the video captured by the imaging device 20. Here, the method of calculating the viewing distance is not particularly limited, and for example, the method disclosed in “JP-A-2003-97914” can be used. Then, the viewing distance calculation means 14 outputs the calculated viewing distance to the area determination means 12A as shown in FIG.

In the video presentation device 10A provided with such a viewing distance calculation unit 14, each pixel on the screen of the display device 50 is centered on the line-of-sight position E (x, y, z) of the viewer A by the region determination unit 12A. It is determined whether it is included in the area range set according to the viewing distance. That is, the area determination unit 12A has the previously inputted from an external threshold value D _0, or the formula is corrected in accordance with the viewing distance, performs area determination based on a threshold D ₀ or expression after the correction.

For example, the area determination unit 12A, as shown in FIG. 4, using a threshold D ₀ which is set in advance, when performing area determination in a true circular area, increase or decrease the threshold value D ₀ depending on the viewing distance By doing so, the area determination is performed by enlarging or reducing the perfect circular area. In addition, as shown in FIG. 5A, the region determination unit 12A determines the viewing distance when performing region determination on an elliptical region using a preset major axis radius a and minor axis radius b. Accordingly, by increasing / decreasing the major axis radius a and the minor axis radius b, the area determination is performed by enlarging or reducing the elliptical area.

Here, the visual field of the viewer A generally decreases as the screen approaches the display device 50 and increases as the screen moves away from the display device 50 screen. Therefore, area determining means 12A, when the viewing distance is short, the threshold value D ₀ or major axis radius reduces a true circle and an ellipse-shaped area range by reducing a and a minor axis radius b, the area after the reduction Perform region judgment by range. On the other hand, when the viewing distance is long, the region determination unit 12A increases the threshold D ₀ , the major axis radius a, and the minor axis radius b to expand the perfect circle and ellipse region ranges, and the enlarged region range Determine the area with.

  The video presentation apparatus 10A having the above-described configuration calculates the viewing distance of the viewer A by the viewing distance calculation unit 14, and the region range having a different size for each viewing distance of the viewer A by the region determination unit 12A. The region can be determined using Accordingly, the video presentation device 10A can change the size of the central area of the viewer A's field of view as a reference for pixel addition and frame addition according to the distance of the viewer A to the display device 50. . For example, when the viewer A views the viewer A at a position close to the display device 50, the video presentation device 10 </ b> A can perform region determination using a small central region and perform an averaging process based on the determination result. When the user A views at a position far from the display device 50, the area determination is performed using the large central area, and the addition averaging process can be performed based on the result.

  Therefore, according to the video presentation device 10A, the central region and the peripheral region of the viewer A's visual field are determined in consideration of the distance between the viewer A and the display device 50. Even when the resolution or the spatial resolution is changed, it is possible to present a natural video that the viewer A does not feel uncomfortable.

[Third Embodiment]
Hereinafter, an image presentation device 10B according to a third embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 9, the video presentation device 10 </ b> B has a configuration similar to that of the video presentation device 10 according to the first embodiment except that the imaging device 15 is provided instead of the addition processing unit 13. Therefore, in the following description, it demonstrates centering on difference with the video presentation apparatus 10, and detailed description is abbreviate | omitted about the structure which overlaps with the said video presentation apparatus 10, and operation | movement of the video presentation apparatus 10B.

  The video presenting apparatus 10B is characterized in that the addition averaging process performed by the addition processing means 13 (first and second embodiments) is performed by the imaging element 15 in the imaging apparatus 30A. That is, the video presentation device 10 </ b> B feeds back the distance determination result based on the line-of-sight information from the imaging device 20 to the imaging device 15, and performs pixel addition processing and frame addition processing in the imaging device 15.

  The imaging element 15 converts light incident from a lens (not shown) of the imaging device 30A into an electrical signal. As illustrated in FIGS. 10 and 11, the imaging element 15 includes a plurality of photoelectric conversion units 151, a plurality of transfer gate transistors 152, a charge-voltage conversion unit 153, a selection transistor 154, and a reset transistor 155. ing. In FIG. 11, the reset transistor 155 is not shown.

  The photoelectric conversion unit 151 is a photodiode that converts light from a subject to be photographed into electric charges. Further, the transfer gate transistor 152 transfers the charge accumulated in the photoelectric conversion unit 151 to the charge voltage conversion unit 153. The charge-voltage converter 153 is a floating diffusion that converts the charge transferred from the photoelectric converter 151 into a voltage and stores it. The selection transistor 154 is a gate switch that selects whether or not the transfer gate transistor 152 transfers charges to the charge-voltage conversion unit 153. The reset transistor 155 is a reset switch that resets the charge accumulated in the photoelectric conversion unit 151.

The image sensor 15 having such a configuration performs a frame addition process and a pixel addition process according to the determination result by the area determination unit 12. Hereinafter, processing performed by the image sensor 15 will be described separately for frame addition processing and pixel addition processing. In the following, the above-described area determination unit 12, as shown in FIG. 4, using a threshold D ₀ which is set in advance, based on the result of the area determination made in a true circular region, averaging process An example of performing is described.

(Frame addition processing)
In the frame addition process, as shown in FIG. 10, the accumulated charge of each pixel (photoelectric conversion unit 151) inside the image sensor 15 is reset to the reset potential by the reset transistor 155 after the read operation in the normal operation. When the reset transistor 155 is operated and the reading operation and the reset operation are not performed, the accumulated charge of the photoelectric conversion unit 151 continues the accumulation operation for the next frame. When this is read after accumulating an appropriate number of frames, the read pixel signal becomes the pixel signal accumulated in the frame, so that frame addition processing can be performed inside the image sensor 15. The switch control of these reset transistors 155 is operated by a control signal included in the determination result of the region determination unit 12.

As described above, the image sensor 15 is a pixel of the image displayed at the pixel position P (x, y, t) determined to be within the predetermined region range (less than the threshold D ₀ ) by the region determination unit 12. when the corresponding electric charge is accumulated in the photoelectric conversion unit 151, by not reset by the reset transistor 155, the pixel determined to be less than the threshold value D _0, pixel and averaging the same position in the previous frame Perform processing to reduce temporal resolution.

(Pixel addition processing)
In the pixel addition process, as shown in FIG. 11, reading from each pixel (photoelectric conversion unit 151) inside the image sensor 15 is performed via the transfer gate transistor 152 (gate 1 to 4), and the common charge voltage conversion unit 153 ( Floating diffusion) and read out. By operating the selection transistor 154 (Gate switches 1 to 4) that controls on / off of the transfer gate transistors 152 (gate 1 to 4), it is possible to perform pixel mixing, that is, pixel addition in an arbitrary combination. For example, when Gate switch 1 and Gate switch 2 of the selection transistor 154 are turned on at the same time, the accumulated charges of the two pixels in the horizontal direction are simultaneously transferred to the charge-voltage conversion unit 153 and subjected to pixel addition processing. Similarly, when Gate switch 1 and Gate switch 3 of the selection transistor 154 are simultaneously turned on, direct pixel addition is performed, and when Gate switches 1 to 4 of the selection transistor 154 are simultaneously turned on, 4-pixel addition is performed. Note that the switch control of these selection transistors 154 is operated by a control signal included in the determination result of the region determination unit 12.

Thus, the image pickup device 15 is outside the region range by the region determining means 12 (threshold value D ₀ or higher) at which the determined pixel position P (x, y, t) of the image displayed on the pixel and its surrounding the electric charges corresponding to the pixel in accordance with the selection by the selection transistor 154, by simultaneously transferred to the charge-voltage conversion unit 153 by the transfer gate transistor 152, the pixel determined to the the threshold value D ₀ or more, the peripheral position in the same frame The spatial resolution is lowered by performing an averaging process with the other pixels.

  In the video presentation device 10B having the above-described configuration, the line-of-sight calculation unit 11 calculates the line-of-sight position E (x, y, t) of the viewer A on the screen of the display device 50, and the region determination unit 12 It is determined whether each pixel on the screen of the display device 50 is included in the central region of the viewer A's visual field. Then, the video presentation device 10B can reduce the temporal resolution of the video by performing the frame addition process on the pixels in the central region of the viewer A's field of view in the imaging device 15, and the viewer A's By performing pixel addition processing on pixels in the peripheral region of the field of view, the spatial resolution of the video can be reduced. Therefore, the video presentation device 10B can present videos with different temporal resolution or spatial resolution for each pixel position P (x, y, t).

  Therefore, according to the video presentation device 10B, it is possible to average the variation in light due to light shot noise by presenting videos having different temporal resolution or spatial resolution in the central region and the peripheral region of the viewer A's line of sight. Therefore, the influence of light shot noise can be reduced without changing the performance of the image sensor. Further, according to the video presentation device 10B, since the pixel addition process and the frame addition process can be performed by the imaging element 15 in the imaging device 30A, the number of configurations of the entire device can be reduced and the cost can be reduced.

[Fourth Embodiment]
Hereinafter, an image presentation device 10 </ b> C according to a fourth embodiment of the present invention will be described with reference to FIGS. 12 and 13. As shown in FIG. 12, the video presentation device 10 </ b> C has the same configuration as the video presentation device 10 according to the first embodiment, except that it includes a region determination unit 12 </ b> B instead of the region determination unit 12. Therefore, in the following description, it demonstrates centering on difference with the video presentation apparatus 10, and detailed description is abbreviate | omitted about the structure which overlaps with the said video presentation apparatus 10, and operation | movement of 10 C of video presentation apparatuses.

  The video presentation device 10 </ b> C is an application of the video presentation device 10 described above to a video from the game device 60. The video presentation device 10 </ b> C is characterized by performing a frame addition process and a pixel addition process on a video with a speed input from the game device 60. That is, as shown in FIG. 12, the video presentation device 10 </ b> C includes an area determination unit 12 </ b> B, and performs a frame addition process and a pixel addition process according to the determination result by the area determination unit 12 </ b> B.

  The game device 60 provides a video of the game to the viewer A via the video presentation device 10 </ b> C and the display device 50. The video of the game is a video reflecting the operation of the viewer A by input means (not shown) such as a controller connected to the game apparatus 60. For example, when the viewer A is playing a driving simulation game with the game device 60, an image corresponding to the field of view from the car in the game is displayed on the screen of the display device 50 as shown in FIG. Is done. Then, when the viewer A drives the car in the game via the controller, as shown in FIG. 13, an image flows (changes) according to the speed of the car.

  The area determination unit 12B has an area range in which each pixel on the screen of the display device 50 is set in advance according to information about the speed, centered on the line-of-sight position E (x, y, t) of the viewer A. It is determined whether or not it is included. The area range here is calculated based on the visual field of the viewer A according to the speed at which the video changes, and is input in advance to the area determination unit 12B. In addition, as shown in FIGS. 13B and 13C, this region range is set so as to increase as the image changing speed becomes slower in consideration of the visual characteristics of the viewer A, and the video changes. It is set so as to become smaller as the speed of performing is faster.

  For example, consider a case where the viewer A is playing a driving simulation game with the game device 60. In this case, the region determination unit 12B does not set the region range when the vehicle is stopped as shown in FIG. On the other hand, when the vehicle is traveling at a speed of 50 km / h as shown in FIG. 13B, the area determination unit 12B sets an area range having a size as shown in FIG. Further, as shown in FIG. 13 (c), the region determination means 12B is more effective when the vehicle is traveling at a speed of 200 km / h than when the vehicle is traveling at a speed of 50 km / h (see FIG. 13 (b)). Set a narrow area range. Then, the area determination unit 12B performs area determination using the area ranges of the respective sizes.

  The video presentation device 10C provided with such a region determination unit 12B is configured to display a pixel of a video displayed at the pixel position P (x, y, t) determined to be included in the region range by the region determination unit 12B. The temporal resolution is lowered by performing the averaging process with the pixel at the same position in the previous frame. In addition, the video presentation device 10 </ b> C uses the pixels at the peripheral positions in the same frame for the pixels of the video displayed at the pixel position P (x, y, t) determined not to be included in the region range by the region determination unit 12 </ b> B. The spatial resolution is lowered by performing the averaging process. That is, the video presentation device 10 </ b> C uses the addition processing means 13 to perform an averaging process according to the video speed in the game, thereby reducing the temporal resolution or spatial resolution of the video.

  In the video presentation device 10C having the above-described configuration, the line-of-sight calculation unit 11 calculates the line-of-sight position E (x, y, t) of the viewer A on the screen of the display device 50, and the region determination unit 12B It is determined whether each pixel on the screen of the display device 50 is included in the central region of the viewer A's field of view according to the video speed. Then, the video presentation device 10 </ b> C can reduce the temporal resolution of the video by performing the frame addition processing on the pixels in the center region of the viewer A's visual field by the addition processing unit 13, and By performing pixel addition processing on pixels in the peripheral region of the field of view, the spatial resolution of the video can be reduced. Therefore, the video presentation device 10 can present videos having different temporal resolution or spatial resolution for each pixel position P (x, y, t) according to the speed of the video displayed by the display device 50.

  Therefore, according to the video presentation device 10 </ b> C, the video from the game device 60 includes light shot noise by presenting video with different temporal resolution or spatial resolution between the central region and the peripheral region of the viewer A's line of sight. Even in this case, since the variation in light due to the light shot noise can be averaged, the influence of the light shot noise can be reduced without changing the performance of the image sensor. Further, according to the video presentation device 10C, when the video (game video) from the game device 60 changes depending on the speed, the video with a sense of presence can be obtained by changing the temporal resolution or spatial resolution of the video according to the speed. Can be presented to the viewer A.

[Video presentation program]
Here, the video presentation devices 10 and 10C described above can be realized by operating a general computer by a program that functions as each of the above-described units and units. This program can be distributed via a communication line, or can be written on a recording medium such as a CD-ROM for distribution.

  The video presentation apparatus and the video presentation program according to the present invention have been specifically described above by the embodiments for carrying out the invention. However, the gist of the present invention is not limited to these descriptions, and the scope of the claims Should be interpreted broadly based on the description. Needless to say, various changes and modifications based on these descriptions are also included in the spirit of the present invention.

  For example, each of the video presentation devices 10, 10A, 10B, and 10C selects the central region and the peripheral region of the viewer A's field of view, and performs an averaging process on each region in the video for each pixel position. The video having different temporal resolution or spatial resolution was presented, but the video having different temporal resolution or spatial resolution is presented for each pixel position without clearly selecting the central area and the peripheral area of the viewer A's visual field. You can also.

  In this case, the video presentation apparatus according to the modified example first calculates the line-of-sight position E (x, y, t) of the viewer A by the line-of-sight calculation unit, and then uses the above-described formula (1) by the region determination unit. Thus, the distance D (x, y) between the line-of-sight position E (x, y, t) of the viewer A and each pixel position P (x, y, t) on the screen of the display device 50 is calculated. .

Then, the video presentation device according to the modified example calculates the pixel value P _disp at each pixel position P (x, y, t) on the screen of the display device 50 using the following formula (6).

Here, c ₁ and c ₂ are functions according to the distance D (x, y). C ₁ in Equation (6) is set to be a larger value as the distance D (x, y) is smaller, and to a smaller value as the distance D (x, y) is larger. Further, c ₂ in equation (6), the distance D (x, y) becomes a smaller value the smaller a distance D (x, y) is set such that a larger value the larger. Then, P _{frame_avg} in the equation (6) can be calculated by the above equation (3), and P _{2d_avg} can be calculated by the above equation (4).

  As described above, the video presentation device according to the modification simultaneously performs the frame addition process and the pixel addition process for each pixel in the video, and at the same time, the distance D from the viewer A's line-of-sight position E (x, y, t). The closer the (x, y) distance is, the higher the influence of the frame addition processing is. The farther the distance D (x, y) from the viewer A's line-of-sight position E (x, y, t) is, the farther the pixel is. The influence of the addition process is increased. Therefore, the video presentation apparatus according to the modified example can present a more natural video to the viewer A.

  Further, in the video presentation device 10C described above, as shown in FIGS. 13B and 13C, the oval area (the central area of the visual field) is determined by the area determination unit 12B. The shape is not limited to this. For example, when the speed of the car in the game is 50 km / h, the video presentation device 10D can determine a rectangular area as shown in FIG. In addition, when the speed of the car in the game is increased from 50 km / h to 200 km / h, the video presentation device 10D can reduce the area so as to be horizontally long as shown in FIG. As shown in FIG. 14C, the region can be reduced to be vertically long.

DESCRIPTION OF SYMBOLS 1 Image | video presentation system 10, 10A, 10B, 10C Image | video presentation apparatus 11 Line-of-sight calculation means 12, 12A, 12B Area | region determination means 13 Addition processing means 14 Viewing distance calculation means 15 Imaging element 151 Photoelectric conversion part (photodiode)
152 Transfer Gate Transistor 153 Charge Voltage Converter 154 Selection Transistor 155 Reset Transistor 20 Imaging Device (Eyeball Imaging Device)
30, 30A imaging device (video imaging device)
40 video playback device 50 display device 60 game device A viewer D ₀ threshold

Claims (6)

A video presentation device that recognizes a visual field of a viewer who views a video, generates a video having different resolutions in a central region and a peripheral region of the visual field, and presents the video to a viewer via a display device. ,
Line-of-sight calculation means for calculating the line-of-sight position of the viewer on the screen of the display device from the video imaged by the imaging device that images the eyeball of the viewer;
Area determination means for determining whether each pixel on the screen of the display device is included in a preset area range centered on the line-of-sight position of the viewer;
For the pixels of the video displayed at the pixel positions determined to be included in the area range by the area determination means, the temporal resolution is reduced by performing an averaging process with the pixels at the same position in the previous frame, and the area determination Addition processing means for reducing the spatial resolution by performing addition averaging processing with pixels at peripheral positions in the same frame for pixels determined not to be included in the region range by the means;
A video presentation apparatus comprising: A viewing distance calculating means for calculating a viewing distance, which is a distance between the viewer and the display device, from a video imaged by the imaging device;
The area determination means determines whether each pixel on the screen of the display device is included in an area range set according to the viewing distance centered on the viewer's line-of-sight position. The video presentation apparatus according to claim 1. A video presentation device that recognizes a visual field of a viewer who views a video, generates a video having different resolutions in a central region and a peripheral region of the visual field, and presents the video to a viewer via a display device. ,
Line-of-sight calculation means for calculating the line-of-sight position of the viewer on the screen of the display device from the video imaged by the imaging device that images the eyeball of the viewer;
Area determination means for determining whether each pixel on the screen of the display device is included in a preset area range centered on the line-of-sight position of the viewer;
Provided in an imaging device that outputs an image to the display device, a photoelectric conversion unit that converts light from an object to be photographed into electric charge, and charge-voltage conversion that converts electric charge transferred from the photoelectric conversion unit into voltage and accumulates it A transfer gate transistor that transfers the charge accumulated in the photoelectric conversion unit to the charge voltage conversion unit, and a selection transistor that selects whether the transfer gate transistor transfers the charge to the charge voltage conversion unit An image sensor having a reset transistor that resets the charge accumulated in the photoelectric conversion unit,
The image sensor is
When the charge corresponding to the pixel of the image displayed at the pixel position determined to be included in the region range by the region determination unit is accumulated in the photoelectric conversion unit, the reset by the reset transistor is not performed. The pixel determined to be included in the region range is subjected to the averaging process with the pixel at the same position in the previous frame to reduce the time resolution,
Charges respectively corresponding to the pixels of the image displayed at the pixel position determined not to be included in the region range by the region determination means and the surrounding pixels are selected by the transfer gate transistor according to the selection by the selection transistor. The image is characterized in that the spatial resolution is lowered by performing an averaging process on the pixels determined to be included in the region range by performing simultaneous averaging with the pixels at the peripheral positions in the same frame by simultaneously transferring to the charge-voltage converter. Presentation device. A video presentation device that recognizes a visual field of a viewer who views a video, generates a video having different resolutions in a central region and a peripheral region of the visual field, and presents the video to a viewer via a display device. ,
Line-of-sight calculation means for calculating the line-of-sight position of the viewer on the screen of the display device from the video imaged by the imaging device that images the eyeball of the viewer;
Information relating to the speed at which the video changes is input from the game device that outputs the video to the display device, and each pixel on the screen of the display device is related to the speed in advance with the viewer's line-of-sight position as the center. Area determination means for determining whether or not the area is set according to the information;
For the pixels of the video displayed at the pixel positions determined to be included in the area range by the area determination means, the temporal resolution is reduced by performing an averaging process with the pixels at the same position in the previous frame, and the area determination Addition processing means for lowering the spatial resolution by performing an averaging process with pixels at peripheral positions in the same frame for the pixels of the video displayed at the pixel positions determined not to be included in the region range by the means;
A video presentation apparatus comprising: In order to recognize the visual field of the viewer who views the video, generate a video having different resolutions in the central region and the peripheral region of the visual field, and present the video to the viewer via the display device,
Line-of-sight calculating means for calculating the position of the viewer's line of sight on the screen of the display device from a video imaged by an imaging device that images the eyeball of the viewer;
Area determination means for determining whether each pixel on the screen of the display device is included in a preset area range centered on the line-of-sight position of the viewer;
For the pixels of the video displayed at the pixel positions determined to be included in the area range by the area determination means, the temporal resolution is reduced by performing an averaging process with the pixels at the same position in the previous frame, and the area determination Addition processing means for reducing the spatial resolution by performing an addition averaging process with pixels at peripheral positions in the same frame for the pixels determined not to be included in the region range by the means;
Video presentation program to function as In order to recognize the visual field of the viewer who views the video, generate a video having different resolutions in the central region and the peripheral region of the visual field, and present the video to the viewer via the display device,
Line-of-sight calculating means for calculating the position of the viewer's line of sight on the screen of the display device from a video imaged by an imaging device that images the eyeball of the viewer;
Information relating to the speed at which the video changes is input from the game device that outputs the video to the display device, and each pixel on the screen of the display device is related to the speed in advance with the viewer's line-of-sight position as the center. A region determination means for determining whether or not the region is included in the region range set according to the information;
For the pixels of the video displayed at the pixel positions determined to be included in the area range by the area determination means, the temporal resolution is reduced by performing an averaging process with the pixels at the same position in the previous frame, and the area determination Addition processing means for lowering the spatial resolution by performing an averaging process on pixels of a video displayed at a pixel position determined not to be included in the region range by the means with pixels at peripheral positions in the same frame;
Video presentation program to function as