JP5086711B2 - Video display device - Google Patents

Description

  The present invention relates to a video display device for managing video, and more particularly to a video display device that efficiently selects and displays moving image thumbnail candidates when displaying a moving image as a thumbnail.

  In recent years, with the development of network technology and the increase in capacity of data storage, it has become possible to hold a large amount of video content and view the content based on it. In addition to the large amount of video held by content holders such as broadcasting stations and production companies, the amount of video content will increase further in the future, such as a large amount of stored content due to the spread of hard disk recorders, etc., and a huge amount of video content on the Internet. Therefore, efficient video presentation and management technology is more demanded.

  Currently, DVD (Digital Versatile Disc), BD (Blue) are used in video playback and recording devices such as a DVD player, a hard disk recorder, or a PC having a video recording / playback function acquired via broadcasting or a network. -Thumbnail display as a means to efficiently present and manage a large number of videos recorded on various recording media such as ray disk (HD), high definition DVD (HD DVD), SD memory card, HDD (Hard Disk Drive) It is used.

  A thumbnail refers to a small image representing the video (usually a reduced image of the original image) and is displayed instead of the video. The thumbnail display refers to a list display format using thumbnails (representative reduced images) of the plurality of videos. Normally, a thumbnail is a still image, but a so-called moving image thumbnail display that displays a moving image instead of a still image is becoming a general function as the processing capability of the device is improved.

  Patent Document 1 is characterized by comprising display control means for performing display control so that the playback frame rates of the selected moving image and the unselected moving image are different. Specifically, the playback frame rate of the selected video is set higher than the playback frame rate of the video that was not selected, or the selected video is played back as a video, and the video that was not selected is played as a still image. An example of displaying as is shown. In addition, an example in which the playback frame rate is determined according to the number of moving images displayed on the screen is also shown.

  Further, Patent Document 2 is characterized in that the video or audio reproduction corresponding to the watched part is controlled by detecting the part that the user has watched for a predetermined time or more with a gaze detection device. It is said. Specifically, an example is shown in which the part that the user has watched for a predetermined time or more is reproduced as a moving image, and the other part is displayed as a still image.

  As a method for detecting the line-of-sight direction of Patent Document 2, the contours of both eyes are detected from image data obtained by photographing the user, and the center position of the eyeball is determined from the geometrical positional relationship between both ends of the eyes and the positions of the nostrils. A method for estimating the radius and detecting the line-of-sight direction is shown.

  Further, as other methods for detecting the line-of-sight direction, methods shown in Non-Patent Documents 1 and 2 are disclosed, which are known techniques. Non-Patent Document 1 describes, as a line-of-sight measurement method using image analysis, extracting a face region from a CCD camera image, performing model fitting with a face discovery template, obtaining a head position / posture, A method of estimating the position and detecting the gaze direction in real time is shown.

Further, in Non-Patent Document 2, as a line-of-sight measurement method using near-infrared rays, an infrared camera is used to obtain a true pupil position and Purkinje image from a three-dimensional model related to the human eyeball shape, an observed pupil image and Purkinje image. The method of estimating the position and calculating the gaze direction is shown.
JP 2004-032535 A Japanese Patent Laid-Open No. 2000-138872 Yoshio Matsumoto, Junichi Tsuji, Kentaro Takemura, Tsukasa Ogasawara, Development of Real-time Face / Gaze Measurement System and Application to Intelligent Interfaces, Information Processing Society of Japan Journal of Computer Vision and Image Media, vol. 47, no. SIG15 (CVIM16), pp. 10-21, 2006/10 Takehiko Ohno, Gaze-based interface, information processing, Vol. 44, no. 7, pp. 726-732, 2003 Mitsuo Ikeda (Author), What is the eye seeing? Visual information processing, Heibonsha, 1988 Ikeda Mitsuo (Author), Visual psychophysics, Morikita Publishing, 1975

  However, as described above, the moving image thumbnail display is used when a desired moving image is selected from the moving images displayed in a list, and is a format in which a thumbnail is reproduced by moving a conventional cursor one by one. Then, it is necessary to move the cursor each time to check the contents of the moving image, and the moving image cannot be selected quickly and efficiently.

  Further, if all thumbnails are always displayed as moving images, the above problem can be avoided, but displaying all thumbnails as moving images places a high processing load on the device. Although the method of Patent Document 1 shows that such a load is reduced by reducing the frame rate of each moving image, the frame rate of each moving image thumbnail decreases as the number of moving image thumbnails to be displayed increases. Therefore, the advantage of displaying a movie is lost. As described above, there remains a problem that the larger the number of displays, the more the same as the still image display.

  On the other hand, in the method of Patent Document 2, as described above, in order to display the moving image thumbnail, it is not necessary to move the cursor by performing a special operation each time, and the advantage of displaying the moving image is lost. All you have to do is watch the video you want to play.

  However, even if the line-of-sight position is obtained by the line-of-sight detection device, it is necessary to perform processing for stopping the reproduction of the video (position before the movement of the visual line) and starting the reproduction of the video (position after the movement of the visual line) every time the line-of-sight position changes. For this reason, there is a problem that video playback / stop processing frequently occurs and the load on the system increases.

  Furthermore, it is not clear from the obtained line-of-sight position whether only the video to be watched is concentrated or whether the entire plurality of thumbnail displays are viewed vaguely. As a result, when the line of sight is placed at the center position of the display and the whole is viewed vaguely, only one image at the center position is reproduced, and the other portion is in a state of a still image. As described above, in the technique disclosed in Patent Document 2, it is necessary to always concentrate (gaze) on the target video at the line-of-sight position. In order to view a plurality of thumbnail videos, each video has a predetermined time. You have to look closely. Therefore, with this method, the entire thumbnail cannot be viewed vaguely, which imposes a burden on the user.

  In other words, even if the gaze detection device identifies the gaze target, the gaze target video is played at a high frame rate, and the video other than the gaze target is played at a low frame rate, is the user's gaze target the entire thumbnail display? Since it is unclear whether it is a specific moving image, it is not easy to see what the user has watched, and the processing load of the moving image thumbnail display system is not reduced.

  Furthermore, as a result of the present inventors actually observing through experiments and examining the above problems, it has been found that there are the following new problems. In other words, even if you are looking at the focused video (gazing), you will be distracted by the video other than the focused video (mainly affected by changes in the brightness of the video scene, etc.) and concentrated. I can't see the video I'm watching. This is because human visual characteristics are more sensitive to luminance perception than peripheral color perception (color is unknown in peripheral vision) (see Non-Patent Document 3, pp. 209).

  In addition, such a situation has an elliptical characteristic (see Non-Patent Document 4, pp. 196, FIG. 10.3) in which the human effective visual field is horizontally long, but it is more effective than the right and left of the effective visual field. It was found to be strongly influenced by the images placed above and below the field of view.

  As described above, it is difficult to reliably select a desired moving image thumbnail image by the method based only on the detection of the line-of-sight position.

  Therefore, the present invention has been made in view of the above-described problems, and allows a user to easily check individual videos and efficiently select from among a plurality of thumbnails without imposing excessive processing load on the device. It is an object of the present invention to provide a video display device capable of selecting a candidate thumbnail and displaying a moving image.

  In order to solve the above-described problems, the present invention has the following features and has the following features.

That is, the video display device according to the present invention is a video display device that displays a list of a plurality of thumbnails respectively corresponding to a plurality of videos on a screen, and a photographing unit that captures an image of a face centered on a user's eyes, A gaze target detection unit that analyzes a captured image and detects a gaze position of the user; and a gaze target range calculation that calculates a gaze target range of the thumbnail centered on a gaze target position obtained from the detected gaze position And the calculated size of the gaze target range and a predetermined threshold, and when it is determined that the calculated size of the gaze target range is smaller than the predetermined threshold, the thumbnail in the gaze target range The video of the peripheral vision part outside the target area for viewing is reproduced at a low quality, and the display quality adjustment part and the display quality adjustment part are used for the display quality adjustment. The video of the thumbnail that is, the effective field of view shape of the human user in accordance with the effective field of view of the viewing distance are watching, a weighting function generated by superimposing the degree of visual acuity in the effective field of view And a display characteristic changing unit that performs a process of suppressing luminance change between video frames of thumbnails within the effective visual field range by performing gamma correction weighted so that the maximum output value is 1 or less. It is characterized by that.
In this way, it is possible to make it easier to see the watched thumbnail relatively without reproducibly reproducing the entire thumbnail, and to reduce the processing load of moving image thumbnail reproduction.

Further, in the video display device according to the present invention, the gaze target detection unit further analyzes the captured image, and calculates a body motion rate that is a movement amount of the user's body per unit time and a user per unit time. A blink rate that is the number of blinks, and the gaze target range calculation unit calculates the gaze target range according to at least one of the parameters of the body movement rate and the blink rate. The smaller the rate is, the larger the gaze target range is calculated, and the gaze target range is specified from the user's gaze position.

In the video display device according to the present invention, the display quality adjustment unit compares the calculated size of the gaze target range with the predetermined threshold, and the calculated size of the gaze target range is the predetermined size. Display adjustment processing is performed when it is determined that the threshold value is smaller than the threshold value, and when the calculated size of the gaze target range is determined to be equal to or greater than the predetermined threshold value, a plurality of thumbnail displays are vaguely viewed. Therefore , the display adjustment is not performed .

Further, the image display device according to the present invention, if the size of the gaze target range the calculated is determined to be smaller than the predetermined threshold value, the display quality adjustment section, the thumbnails in the gaze target range On the other hand, while performing the high quality image processing including at least one processing of enlargement processing, high image quality processing, high frame rate reproduction processing, and high color reproduction processing, for the thumbnail outside the target range, The low quality image processing including at least one of reduction processing, low image quality processing, low frame rate reproduction processing, and low color reproduction processing is performed to reduce the processing load of moving image thumbnail reproduction.

In the video display device according to the present invention, when the high quality image processing is performed on the thumbnail within the gaze target range, the display quality adjustment unit applies to the thumbnail outside the gaze target range. the omitted to run a low-quality image processing, wherein the relative gaze outside the scope of the thumbnail when performing low-quality image processing, the high-quality image processing on a thumbnail in the gaze target range Te By omitting the execution of the above, the processing load of moving image thumbnail reproduction is reduced.

Further, in the video display device according to the present invention, even if the display characteristic changing unit determines that the display quality adjusting unit is vaguely viewing the entire plurality of thumbnail displays, A change is suppressed, and a desired thumbnail is selected and displayed from all the displayed thumbnails.

  According to the video display device according to the present invention configured as described above, by calculating the gaze target range of the thumbnail centered on the gaze target position obtained from the user's gaze position, by limiting the gaze range, Without replaying the entire thumbnail in high quality, only the video within the appropriate gaze target range is played back with high quality, and the video of the peripheral vision area outside the gaze target range is played back with low quality. It is possible to make it easier to see the thumbnails that are not being watched, and to reduce the processing load of moving image thumbnail playback.

  In addition, by suppressing changes in luminance according to the effective visual field shape of humans, when watching a thumbnail image within the gaze target range, the thumbnail image outside the gaze target range is concentrated without being distracted. This makes it possible to view the video of the gaze target.

  In addition, according to the video display device of the present invention, even when a plurality of thumbnail displays are viewed vaguely, it is easy to see the entire thumbnail display by suppressing the luminance change according to the human effective visual field shape. It becomes possible.

  Hereinafter, an embodiment of a video display device according to the present invention will be described with reference to FIGS.

  FIGS. 1 to 8 are diagrams for explaining an embodiment of a video display device according to the present invention. In each figure, the same reference numerals denote the same parts.

First, a schematic configuration of the video display device 100 according to the present invention will be briefly described with reference to FIG.
FIG. 1 is a functional block diagram showing a configuration example of a video display apparatus according to the present invention.

  As shown in FIG. 1, a video display device 100 includes a hard disk HD device 101 that records a large number of videos, a recording / playback device 102 that drives an optical recording medium such as MO, CD, DVD, and the like. An I / OI / F 103 that is an interface for reading and writing data, a video display processing unit 107 that generates a moving image thumbnail image with improved visibility, which is a characteristic part of the video display device 100, and a video display processing unit 107 A gaze target information detection unit 108 that detects gaze target information used when calculating / deciding a gaze target range, which will be described later, and a display 110 connected to a display I / F 109 that displays the generated moving image thumbnail video and the like. If the video includes audio, the speaker 112 connected to the AVI / F 111 that outputs audio A CPU105 for performing overall control of the entire apparatus configured to include a memory 104 that stores a program for executing this control, etc., the.

  Further, a communication I / F 106 that captures video from a communication port and an AVI / F 111 that captures audio from Video, Mic, or the like may be provided.

  Note that the moving image thumbnail generation processing by the video display processing unit 107 may be entirely executed by hardware. Further, the display moving image thumbnail generated by the video display processing unit 107 may be temporarily stored in the memory 104 and output to the display 110 via the display IF 109, or the generated moving image thumbnail may be displayed. May be provided in the video display processing unit 107.

  Here, the still image data and the moving image data used for the thumbnail display may be obtained directly from the original video data represented by the thumbnail, or may be provided separately from the original video data.

  Further, there is no particular limitation on when and how these still image data and moving image data for thumbnail display are prepared. For example, every time thumbnails are displayed, moving image data for thumbnail display may be generated from the original video data, or video data for thumbnail display may be generated and stored separately when the original video data is stored. A configuration may be used. Of course, the original video data can be used as the video data for displaying the video thumbnail.

  Next, a basic configuration of the video display processing unit 107, which is a characteristic part of the video display device 100 according to the present invention configured as described above, and an operation concept thereof will be briefly described.

  In the so-called video thumbnail display that displays thumbnails that are displayed as a list, the thumbnails of the images that reflect the physiological visual characteristics of human beings are displayed on multiple thumbnails that are listed on a single screen, and the video to be watched It is easy to see only the video in the specified area in the direction of the line of sight, and even if it is determined that the entire thumbnail display is vaguely viewed, the thumbnail display In order to have the feature of making the whole view easy, the video display processing unit 107 of the video display device 100 includes the following three main components.

  FIG. 2 is a functional block diagram showing a configuration example of the video display processing unit 107 of the video display device 100 according to the present invention.

  As shown in FIG. 2, the gaze target range that identifies the gaze target range by combining at least one piece of information of the line-of-sight position detected by the gaze target information detection unit 108, the user's body motion rate, and the user's blink rate. A calculation unit 200, a display quality adjustment unit 210 that displays a high-quality video thumbnail within the specified gaze target range, and a low-quality video thumbnail that is outside the gaze target range. The display characteristic changing unit 220 changes the moving image thumbnail display to a display characteristic considering human visual characteristics.

  First, the basic operation concept of the gaze target range calculation unit 200 will be described.

  The gaze target range calculation unit 200 specifies a gaze target range for determining whether the gaze target is a thumbnail image in the gaze position direction or the entire thumbnail display centered on the gaze position. That is, the gaze target range calculation unit 200 calculates the degree of interest in the video in the direction of the gaze position from the body motion rate and blink rate detected from the gaze target information detection unit 108, and based on the calculated interest level and gaze position Identify the gaze target range.

  For example, the gaze target detection unit 108 generally detects a gaze position, a body motion rate, and a blink rate by capturing a face portion centered on the user's eyes with a camera and analyzing the obtained image. Is.

  Here, a specific calculation method for specifying the gaze target range from the degree of interest and the line-of-sight position will be described later, but the relationship between the body motion rate, the blink rate, and the degree of interest and the relationship with the gaze target range. Briefly described below.

  In research reports (Hideoki Tada, Tomio Yamada, Keisuke Fukuda (eds.), Psychology of blinking, Kitaoji Shobo, 1991, pp. 40, pp. 96, pp. 97, hereinafter referred to as “Research Report 1”) Research reports on changes in blinking when a human gazes at a stimulus target have been shown, and blinking increases in emotional states such as anger and fear and muscle tone. On the other hand, when attention is paid to a stimulus of interest, or during information processing (during thinking), blinking is reduced, and blinking temporarily increases as the stimulus ends or processing ends (pp) .49).

  Moreover, the result of having experimented about the relationship between blinking at the time of program viewing and a break is also shown (pp.197), and from the experiment of a video stimulus series, it is a blink between an interesting program and an uninteresting program ( It has been confirmed that there is a difference in the blink rate, and that this is an inverse correlation.

  Note that the blink rate is the number of blinks performed per unit time, and according to the above-mentioned research report 1, it has been reported that a human is usually about 20 times per minute. This value can be reflected in the relationship between blink rate and interest.

  In addition, it is described that body movement tends to increase under low interest conditions, and the body movement rate (the ratio of body movement) is a quantification of the amount of body movement per unit time. If it is large or intense, it becomes a large value, and if it is stationary, it becomes a small value.

  As explained above, when people are generally in a low-interest environment, there is a tendency for body movement to increase, and when they are interested or thinking about stimuli of interest, Therefore, when watching a program that the user is interested in, it is easy to expect that there will be less body movement and less blinking. It is said that it will be more appropriate as an index of (see Research Report 1, pp. 206). From this, the interest level is calculated using both blink and body movement.

  The size of the gaze target range calculated from the degree of interest calculated in this way is small if the degree of interest is high, and is small if the degree of interest is low. It gets bigger. This is because in the case of a high degree of interest, the video at the gaze position is concentrated and viewed, and the gaze target range is naturally narrowed.

  On the other hand, in the case of a low degree of interest, since a wide range centered on the gaze position is viewed vaguely, the gaze target range is widened.

  As described above, the gaze target range calculation unit 200 determines whether only the video of the gaze target is concentrated with the obtained gaze position as the center or whether the plurality of thumbnail displays are viewed vaguely. Means for determining the gaze target range for determination, and when only the video of the gaze target is viewed in a concentrated manner, it is easy to see only the sub-nails of the narrow area in the gaze position direction, Even when the entire thumbnail display is viewed vaguely, it is possible to provide information for determining the area of the gaze target range so that the entire thumbnail display can be easily viewed. With this information, it is possible to avoid an excessive processing load on the device.

  Next, the basic operation concept of the display quality adjustment unit 210 will be described.

  In order to determine whether the display quality adjustment unit 210 is viewing only a sub-nail image in a narrow region in the gaze position direction or whether the entire plurality of thumbnail displays are viewed vaguely, the gaze target range calculation unit When the size of the gaze target range generated is compared with the predetermined threshold value and the size of the gaze target range is smaller than the predetermined threshold value, the display quality of the entire sub nail is adjusted.

  As a display quality adjustment process, thumbnails within the gaze target range are subjected to high quality adjustment processing such as enlargement adjustment, high image quality adjustment, high frame rate adjustment, and high color reproduction (full color) adjustment. Low quality adjustment processing such as reduction adjustment, low image quality adjustment, low frame rate adjustment, and low color reproduction (monochrome) adjustment is performed.

  Here, the contents of each quality adjustment process will be briefly described.

  The enlargement adjustment processing refers to performing enlargement processing of the target video, and the reduction adjustment processing refers to performing video reduction processing.

  Also, the high image quality adjustment is a decoding process that is not simplified, and means reproduction with high image quality, and the low image quality adjustment is a reproduction of only an I picture at the time of decoding, a motion compensation filter at the time of decoding, or This means that the smoothing filter is omitted.

  Furthermore, the high frame rate adjustment refers to updating the video at the frame rate of the video itself that is not subjected to frame rate conversion, and the low frame rate adjustment is a stop (still image), only an I picture, or an I picture. This means that the frame rate is reduced so that only the P picture is reproduced.

  The high color reproduction adjustment means that the color of the video itself is reproduced as it is, and the low color reproduction means that the reproduction is performed in gray scale (monochrome). Note that human vision has a characteristic that it is difficult to perceive color in a small object or a peripheral visual field (Non-Patent Document 4, pp. 209). Even when displayed in monochrome, there is generally little discomfort (there are individual differences).

  Note that the enlargement process is intended to be relatively enlarged compared to the image within the gaze target range, so when the video outside the gaze target range is reduced, the enlargement process itself is not performed. It also includes displaying the original size as it is. Similarly, when a video outside the gaze target range is reduced, the enlargement process itself is not performed, and the original size is displayed as it is.

  In addition, since high-quality processing is aimed at obtaining a relatively high-quality image for a low-quality image, if low-quality processing is performed on a video outside the target range, This includes outputting the image quality as it is without performing any image quality processing itself. Similarly, when a low-quality image process is performed on a video outside the target range, the high-quality image processing itself is not performed, and output is included as it is.

  Note that a series of enlargement processing, high image quality processing, high color reproduction processing, and high frame rate conversion processing are processing for obtaining a high-quality video, and do not depend on this order. Similarly, a series of reduction processing, low image quality processing, low color reproduction processing, and low frame rate processing are performed in this order in order to obtain low-quality video for the purpose of reducing the processing load of the video display apparatus and method of the present invention. Is not dependent. Further, it is not necessary to perform all the processes, and for example, a combination of specific processes such as performing only the enlargement process may be used.

  Moreover, since high quality and low quality are compared relatively, the case where only high quality processing is performed, the case where only low quality processing is performed, and the case where both are performed simultaneously are included.

  As described above, when only the video within the gaze target range is concentrated, only the video in the gaze position direction is played back with high quality, and the video of the peripheral vision part outside the gaze target range is low. By reproducing with quality, it is possible to reduce the processing load of moving image thumbnail reproduction.

  Next, the basic operation concept of the display characteristic changing unit 220 will be described.

  As described above, when a video within the gaze target range is played at a high frame rate and a video outside the gaze target range is played at a low frame rate, or a video thumbnail within the gaze target range and outside the gaze target range is the same frame When playing at a rate, even if you are watching a video within the target area, you will be distracted by the video outside the target area (mainly affected by changes in brightness due to scene changes, etc.) As described above, the inventor's experiment revealed that the subject cannot concentrate on seeing the image of the gaze target. This is because human visual characteristics are more than that of color perception in peripheral vision. This is because it is sensitive to luminance perception (Non-patent Document 4, pp. 209).

  Therefore, in order to solve this problem, the display characteristic changing unit 220 includes display changing means that uses human visual characteristics to display an image within the gaze target range. In other words, the luminance change suppressing method is used in accordance with the effective visual field shape by utilizing the characteristic that the human effective visual field has an elliptical shape that is long horizontally and the visual acuity rapidly decreases as the distance from the central eye increases.

  As one of the luminance change suppression methods, there is a method of changing the gamma characteristic itself (gamma correction). An image is displayed on the display in accordance with the input signal, but depending on the characteristics of the display, there is a problem that the displayed brightness and saturation differ. Therefore, gamma correction is generally used to correct these errors.

  The gamma characteristic refers to the relationship between the input image data signal strength and the output signal strength. For example, in the case of a display, the input display luminance signal and the actual luminance of the fluorescent band on the display. This is the gamma characteristic.

  Gamma correction is a process for correcting the saturation and brightness of an image displayed on the display. Normally, any curve (gamma) from the darkest part to the brightest part is used. (Curve) is a process for correcting whether to change. As described above, the gamma correction adjusts the relative relationship between the color information output from the display and the actual output information, and usually performs color correction to obtain a display close to natural. The gamma value represents the ratio of the change in the voltage conversion value to the change in the brightness of the image.

  FIG. 7 shows a curve (gamma curve) in which input and output correspond one-to-one, and no correction is performed. On the other hand, FIG. 8 shows a gamma curve in which the characteristic is a straight line, the output value for the input 1 is 0.8, and the output characteristic is corrected to 0.8 times the output characteristic for all inputs.

  At this time, as shown in FIG. 9, only the upper part of the gamma curve is cut (the input is from 0 to 0.8, and the output is also a curve corresponding to 0 to 0.8. In the characteristic that the output value is 1 even if a large input is given), the luminance value is suppressed by gamma correction in the high luminance value part (bright part), but the gradation characteristic of the output with respect to the input is uniform No, a color change occurs. However, as shown in FIG. 8, when the overall luminance is lowered while the output characteristics with respect to the input are kept linear, the luminance value becomes linear with respect to the input and the overall luminance is reduced. Low gradation is corrected and the gradation characteristics are preserved.

  The gamma correction performed by the display characteristic changing unit 220 can reduce the luminance value itself and suppress the dynamic luminance change. Furthermore, by setting the luminance value to a low value by this gamma correction, the effect of being able to express richly the gradation and hue of the dark part of the image is also produced.

  As another method, it can be realized by blurring a plurality of corresponding frames before and after the dynamic luminance change (such as motion blur). Furthermore, a method of suppressing the dynamic contrast by providing an upper limit value (limiter) of the luminance change and suppressing the luminance at that time by gamma correction when the luminance change of a certain level or more occurs. However, these methods are not preferable methods because they give significant changes to the original video and may infringe the intention of the work or the work itself.

  Note that the grayscale conversion does not change the luminance value itself, and thus cannot suppress the dynamic luminance change.

  Next, a weighting function based on the effective visual field shape to which the gamma correction value for suppressing the dynamic luminance change is applied will be described.

  FIG. 10 is obtained by superimposing the visual acuity (effective visual acuity range) in the effective visual field on the human effective visual field range. 300a shown in FIG. 10 is a three-dimensional display portion that displays the degree of effective visual acuity with an upper limit of 1.0 and a lower limit of 0.0, and 300b is a portion that displays the effective visual acuity in contour lines. In other words, FIG. 10 is a diagram in which a weighting function obtained by weighting the effective visual field with the effective visual acuity is three-dimensionally displayed.

  10 may be considered as an effective visual acuity range (non-patent document 4, pp. 194) when the visual acuity capable of seeing a fine object is 0.3 or more. The visual acuity becomes 0.3 only when the degree of visual acuity is shifted by 5 degrees from the center of the visual field, and the visual acuity is reduced to 0.1 when the visual acuity is shifted by 10 degrees.

  In other words, the visual acuity is very good at the center of the visual field, but the visual acuity suddenly drops when it is slightly off the center.

  Therefore, FIG. 10 is a diagram showing the distribution of the weight of the output maximum value of the gamma correction expressing this characteristic, and the output maximum value of the gamma characteristic is an elliptical cone shape that is long on the left and right with the user's line-of-sight position as the center. Is given as a weight function.

  That is, this weighting function calculates the maximum output value by the gamma characteristic (the output value with respect to the input value 1.0, for example, 1.0 in FIG. 7 and 0.8 in FIG. 8) relative visual acuity (non- Patent Document 4, FIG. 10.1, and FIG. 10.2) are applied.

  Note that the X coordinate (degree) indicates the viewing angle in the left-right direction when the display is viewed directly, and the Y coordinate (degree) indicates the viewing angle in the up-down direction, and the value is the maximum output value (FIG. 7). , 8, and 9 for the input 1).

  In addition, since the effective visual field of human beings is indicated by an angle, strictly speaking, the value of gamma correction processing for an actual video changes depending on the viewing distance. Details of the gamma correction process will be described later.

  As described above, by performing gamma correction based on the weight function and performing it on each frame image of each moving image thumbnail, it is possible to generate a moving image thumbnail in which the dynamic luminance change is suppressed.

  Next, the flow of operation processing of the video display processing unit 107 shown in FIG. 1 will be described in detail with reference to FIGS. 3 and 4 to 6.

  FIG. 3 is a flowchart showing a schematic operation of the video display processing unit shown in FIG.

  As shown in FIG. 3, the video display processing unit 107 is divided into the following three stages of processing steps.

  First, in step S21, the gaze target range calculation unit 200 calculates the interest level of the video at the gaze position from the gaze position detected by the gaze target information detection unit 108, the body motion rate / blink rate of the user, and gazes. Identify the scope. After that, in step S22, the display quality adjustment unit 210 compares the specified gaze range with a predetermined size and determines that the user's interest in the video at the gaze position is high, that is, the gaze target range is narrow. In addition, the video sub-nails within the gaze target range are displayed with high quality, and the video sub-nails outside the gaze target range are displayed with low quality. Then, in step S23, the display characteristic changing unit 220 has a high degree of interest of the user in the video at the gaze position, that is, when the gaze target range is narrow, so that the video thumbnail of the gaze target can be easily viewed. Change the video thumbnail display to display characteristics that consider human visual characteristics.

  Next, the operation of each step (S21, 22, 23) will be described in more detail with reference to FIGS.

FIG. 4 is a flowchart illustrating in detail the processing operation of the gaze target range extraction process (step S21) described in FIG.
FIG. 5 is a flowchart illustrating in detail the processing operation of the display quality adjustment process (step S22) described in FIG.
FIG. 6 is a flowchart illustrating in detail the processing operation of the display characteristic changing process (step S23) described in FIG.

First, in step S21, the gaze target range calculation unit 200 acquires the detected body motion rate of the user by the gaze target information detection unit 108 (step S302), and similarly acquires the detected blink rate of the user. (Step S303). The degree of interest is calculated using the two acquired parameters (step S304). For example, when the body motion rate is b and the blink rate is d, the degree of interest f can be calculated by the following equation (1).
f = wb × (rb / b) + wd × (rd / d), (wb + wd = 1) (1)
Here, rb is a coefficient for converting the body motion rate to be used for calculating the degree of interest (for example, when a value in the range of the body motion rate is 0,..., 100 is obtained, the dynamic range is 0,. Rb = 0.01 in order to keep it within the range, rd is a coefficient for converting the blink rate so that the blink rate is used in the interest degree calculation. In addition, wb and wd are weighting factors that determine which of the body motion rate and blink rate is to be calculated when calculating the degree of interest, and when both factors are added, 1.0 (wb + wd = 1). .

  In order to avoid b = 0 and d = 0 in the above formula (1), it is necessary to avoid handling values less than a minute value such as 0.1 in advance. Moreover, since b is a body movement rate and d is a blink rate, it does not become a negative value.

  Thus, when the body motion rate is low or when the blink rate is low, it is determined that the degree of interest f is large. When the body motion rate is high or when the blink rate is high, it is determined that the degree of interest f is small. The In calculating the degree of interest f, b and d are not multiplied by the coefficients as they are, but the reciprocal is taken to calculate the degree of interest f.

Next, a gaze target range g (for example, a circular range with a radius r centered on the gaze position e) is calculated using the calculated interest degree f and the gaze position e (step S305). Therefore, the gaze target range g can be calculated as follows.
g = e ± w _f / f (2)
Here, the gaze target range g shown in the above formula (2) is conceptually expressed by the gaze position e and the range amount (w _f / f), and the range amount (w _f / The range of f) indicates the gaze target range. Further, as described above, the range amount (w _f / f) is inversely related to the interest level f, and is proportional to the reciprocal of the interest level f.

  Further, in the above example, an example in which the shape of the gaze target range is a circle is shown, but a rectangle may be used, for example.

  When the process of step S305 is completed, the process proceeds to step S22 (symbol A).

  Next, based on the gaze target range determined in step S21, in the display quality adjustment step in step S22, the display quality adjustment unit 210 adjusts all the video thumbnail display quality displayed on the display. .

First, the display quality adjustment unit 210 compares and determines the range amount (w _f / f) of the gaze target range and a predetermined threshold (step S306), and the range amount (w _f / f) of the gaze target range is the threshold value. Is smaller (step S306; Yes), only one of all the moving image thumbnails displayed on the display is selected (step S307). Then, it is determined whether or not the selected moving image thumbnail is within the gaze target range (step S308).

When the range amount (w _f / f) of the gaze target range is larger than the threshold value (step S306; No), this process ends (step S22).

  When it is determined that it is within the gaze target range (step S308; Yes), enlargement processing (step S309), high image quality processing (step S310), high color reproduction processing (step S311), and high frame rate conversion processing (step S312) are performed. Then, it is determined whether all moving image thumbnails have been processed (step S317). If all moving image thumbnails have not been completed (step S317; No), the process returns to step S307. When all the moving image thumbnails have been processed (step S317; Yes), the process proceeds to step 23 (symbol B).

  In step S308, if the selected moving image thumbnail is not within the gaze target range (step S308; No), the reduction process (step S313), the image quality reduction process (step S314), and the low color reproduction process (step S314). S315), a low frame rate conversion process (step S316) is performed.

  Next, in step S23, the display characteristic changing unit 220 has a characteristic that an effective human visual field has an elliptical shape that is horizontally long in order to suppress a change in luminance between video frames (hereinafter, dynamic luminance change). This is used to suppress the luminance change according to the human effective visual field shape and perform the display characteristic changing process.

  Usually, since the human effective visual field is given by an angle, the actual effective visual field range is strictly determined by the viewing distance, and is an ellipse that is long to the left and right with the user's line-of-sight position as the center. Therefore, first, in step S318, the viewing distance detected by the gaze target information detection unit 108 is acquired. As a method of detecting the viewing distance by the gaze target information detection unit 108, for example, the distance between the user and the display may be directly detected by a distance sensor. When there are a plurality of cameras, a triangulation technique is used. The distance between the user and the display may be calculated from the corresponding points of each camera. These are disclosed as a known technique (for example, Non-Patent Document 1).

In step S319, the effective visual field range in the left-right direction is calculated from the viewing distance obtained in step S318 by the following equation. When the viewing range is d and the viewing angle is 2 × θ, the effective viewing range e is given by the following equation (3).
e = 2 × d × tan (θ) (3)
This will be described with reference to FIG.

  In FIG. 11, when the viewing distance d is d1 and the viewing angle is 2 × θ, the effective viewing range e is e1.

  In addition, when the viewing distance d is d2 and the viewing angle is 2 × θ, the effective viewing range e is e2.

  The displays 801 and 802 are the same size. FIG. 12 shows the relationship between the viewing distance (distance) and the effective visual field range (view area). It can be seen that if the viewing distance d is short (d1), the effective visual field range is narrow, and if the viewing distance is long (d2), the effective visual field range is wide.

  Here, the upper limit of the effective visual field range is expressed as a ratio (1.0), and those having different display sizes such as a 20-inch display and a 45-inch display have an effective visual field range proportional to the size.

  For example, as shown in FIG. 11, if the viewing distance is short (when the display 801 at the viewing distance d1 is watched), the effective visual field range e1 is only a narrow range (black rectangular portion 803) from the center of the viewing position. The other corners of the screen are out of the effective field of view. On the other hand, if the viewing distance is long (when the display 802 at the viewing distance d2 is watched), the effective visual field range e2 is wider than the size of the display 802 (black rectangular portion 804) from the center of the viewing position. .

  In step S320, based on the weight function given in FIG. 10, the output maximum value in gamma correction (output value with respect to input value 1.0, for example, 1.0 in FIG. 7, 0.8 in FIG. 8) is calculated. . For example, the weight 1 is the maximum output value for gamma correction at the center of the gaze position, and the weight corresponding to the value of the distribution is the maximum output value for gamma correction around the gaze position.

When gamma correction is actually performed on a video thumbnail image, gamma correction is performed by linear correction with a low maximum output value. For example, in the case of an RGB image, it can be performed by the following formula (usually the minimum output value) Set to 0.0 and maximum output value 1.0 to perform nonlinear correction).
L1 = L0 × g (4)
L0 is the brightness of each RGB value of each pixel before gamma correction (when 0 ≦ L0 ≦ 255, 8 bits), and L1 is the brightness of each RGB value of each pixel after gamma correction (0 ≦ L1 ≦ 255). , 8 bits). g is an output value coefficient (0 ≦ g ≦ 1) of lightness of each RGB value. By performing this process on each frame image included in each moving image thumbnail, it is possible to generate a moving image thumbnail in which a change in dynamic luminance is suppressed.

  FIGS. 13 and 14 show three-dimensional distributions of gamma correction results on the display depending on the viewing distance. FIG. 13 shows the distribution of gamma correction results using the weighting function shown in FIG. 10 in the effective visual field range of the viewing distance d1 shown in FIGS. 11 and 12, and FIG. 14 shows the viewing shown in FIGS. The distribution of the gamma correction result using the weighting function of FIG. 10 is represented in the effective visual field range of the distance d2.

  The X and Y coordinates in FIGS. 13 and 14 represent the actual screen size (for example, inches), and when the viewing distance d is shown in FIGS. 11 and 12 (the viewing distance is short). 13 shows that gamma correction is performed in a narrow range as shown in FIG. 13. When the viewing distance is d2 (the viewing distance is far) as shown in FIGS. 11 and 12, the wide range is shown in FIG. Indicates that gamma correction is given.

  In addition, it is assumed that the line-of-sight position, the user's body movement rate, and the user's blink rate are detected from the camera image, but it is not necessary to detect them all from the camera image. May be detected.

  In addition, since the effective visual field range changes depending on the viewing distance, in this embodiment, the viewing distance is detected. However, in consideration of the environment in which a display such as a living room is installed in the home, the actual viewing distance is The actual viewing range due to the difference in viewing distance falls within the distance of twice to three times the height, and is almost in the range of error. In fact, there is no problem as a fixed viewing range that does not consider the viewing distance. In this case, step S318 shown in FIG. 6 can be omitted, and the effective visual field range of step S319 is a fixed value.

  In this way, when the viewing distance is regarded as fixed in consideration of the actual home viewing environment, a part of the processing of the display characteristic change processing step S23 of FIG. 2 is omitted, so the system load is reduced. can do.

  In addition, as described above, by specifying the gaze target range, character information for displaying information on the video displayed on the display, symbol information such as stereo audio symbols, reproduction status symbol / stop status symbol, etc. It is also possible to control display of video information other than thumbnail images such as control symbol information. That is, when a plurality of thumbnails are viewed vaguely, character information and control symbol information are not displayed, and the entire plurality of thumbnail images can be easily viewed. Furthermore, if you are looking at a specific thumbnail, the text information etc. is displayed at first, but if you keep watching, control information will be displayed, and operations such as stopping automatically will be performed. It is also possible to perform thumbnail display in consideration of the user's viewing state (whether vaguely looking or looking at a certain thumbnail).

  It should be understood that the embodiments disclosed herein are illustrative and non-restrictive in every respect. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims. Therefore, the video display apparatus according to the present embodiment is not limited to the above-described illustrated examples, and it is needless to say that various changes can be made without departing from the gist of the present invention. Furthermore, each process described in the present embodiment or each step in the process may be a program for causing a computer to execute the program, and the program may be provided by a computer-readable recording medium to be recorded.

It is a functional block diagram which shows the example of 1 structure of the video display apparatus concerning this invention. It is a functional block diagram which shows the structural example of the video display process part of the video display apparatus which concerns on this invention. It is a flowchart figure which shows schematic operation | movement of the video display process part shown in FIG. It is a flowchart figure which shows detailed operation | movement of a gaze target range part. It is a flowchart figure which shows the detailed operation | movement of a display quality adjustment part. It is a flowchart figure which shows the detailed operation | movement of a display characteristic change part. It is a figure which shows a gamma characteristic (no correction | amendment). It is a figure which shows a gamma characteristic (linear correction). It is a figure which shows a gamma characteristic (nonlinear correction). It is the figure which displayed the effective visual acuity range on the effective visual field range in three dimensions. It is a figure showing the relationship between viewing distance and a viewing angle. It is a figure showing the relationship between viewing distance and an effective visual field range. It is the figure which displayed in three dimensions the display characteristic change result when viewing distance is near. It is the figure which displayed three-dimensionally the display characteristic change result when viewing distance is far.

Explanation of symbols

100 Video display device 101 HDD
102 Recording / playback device (MO, CD, DVD)
103 I / O I / F
104 memory 105 CPU
106 Communication I / F
107 Video display processing unit 108 Gaze target information detection unit 109 Display I / F
110 Display 111 AV I / F
112 speaker 200 gaze target range calculation unit 210 display quality adjustment unit 220 display characteristic change unit 300a effective visual acuity three-dimensional display part 300b effective visual acuity contour display part 800 visual field range 801 viewing distance d1 display 802 viewing distance d2 Installed displays 803, 804 Black rectangular part

Claims (6)

A video display device that displays a list of a plurality of thumbnails respectively corresponding to a plurality of videos,
An imaging unit that captures an image of the face centered on the user's eyes;
A gaze target detection unit that analyzes a captured image and detects a user's gaze position;
A gaze target range calculation unit that calculates a gaze target range of the thumbnail around a gaze target position obtained from the detected gaze position;
When the calculated size of the gaze target range is compared with a predetermined threshold, and it is determined that the calculated size of the gaze target range is smaller than the predetermined threshold, only the thumbnail images within the gaze target range A display quality adjustment unit for reproducing the thumbnail image of the peripheral vision part outside the gaze target range at a low quality,
The degree of visual acuity in the effective visual field range to a human effective visual field shape corresponding to the effective visual field range of the viewing distance viewed by the user for the thumbnail image whose display quality has been adjusted by the display quality adjustment unit A process for suppressing a change in luminance between video frames of thumbnails within the effective visual field range by performing gamma correction using a weighting function generated by superimposing the weights so that the maximum output value is 1 or less. A display characteristic changing unit for performing
A video display device comprising: The gaze target detection unit further analyzes the captured image to detect a body motion rate that is the amount of movement of the user's body per unit time and a blink rate that is the number of blinks of the user per unit time. ,
The gaze target range calculation unit calculates the gaze target range by calculating at least one of the parameters of the body motion rate and the blink rate when the gaze target range is calculated. The video display device according to claim 1, wherein the video display device is largely calculated and the gaze target range is specified from the gaze position of the user. The display quality adjustment unit
Comparing the calculated size of the gaze target range with the predetermined threshold,
When it is determined that the calculated size of the gaze target range is smaller than the predetermined threshold, display adjustment processing is performed,
When it is determined that the calculated size of the gaze target range is equal to or greater than the predetermined threshold value, it is determined that the entire plurality of thumbnail displays are viewed vaguely , and display adjustment is not performed. Item 3. A video display device according to item 1 or 2. If the size of the gaze target range the calculated is determined to be smaller than the predetermined threshold value, the display quality adjustment section to the thumbnail in the gaze target range, enlargement processing, image quality improvement processing, high While performing the high-quality image processing including at least one processing of frame rate reproduction processing and high color reproduction processing, reduction processing, low image quality processing, and low frame rate reproduction are performed on the thumbnails outside the target range. 4. The process according to claim 1, wherein the low-quality image processing including at least one of processing and low-color reproduction processing is performed to reduce a processing load of moving image thumbnail reproduction. The video display device described in 1. When the high-quality image processing is performed on the thumbnail within the gaze target range, the display quality adjustment unit omits the execution of the low-quality image processing with respect to the thumbnail outside the gaze target range. When the low quality image processing is performed on the thumbnail outside the gaze target range, the execution of the high quality image processing on the thumbnail within the gaze target range is omitted, thereby reproducing the video thumbnail. The video display device according to claim 4, wherein the processing load of the video is reduced. The display characteristic changing unit
Even when the display quality adjustment unit determines that the entire plurality of thumbnail displays are viewed vaguely, the luminance change is suppressed, and a desired thumbnail is selected from all the displayed thumbnails. 4. The video display device according to claim 3 , wherein the video display device is selected and displayed.

Images