JP7153508B2 - Visual guidance device and its program - Google Patents

Description

TECHNICAL FIELD The present invention relates to a gaze guidance device that guides a viewer's gaze and a program therefor.

Conventionally, methods have been proposed for guiding the line of sight to a place where a person's attention should be directed. For example, Patent Literature 1 describes a line-of-sight guidance method for visual documents such as product catalogs. Further, Japanese Patent Laid-Open Nos. 2002-200002 and 2004-200031 describe a method of guiding the driver's gaze of the vehicle.

By the way, in recent years, research has progressed on ultra-high-definition video systems with a resolution of 8K (horizontal 7680 pixels×vertical 4320 pixels) or higher. The ultra-high-definition video system enables wide-field video presentation with the aim of enhancing the viewer's sense of presence. This wide-field video presentation can provide a video viewing experience with a high sense of realism as if the user were there by presenting the video at a viewing angle of approximately 100 degrees.

Viewing a video with a wide field of view means that the viewer views the video closer to the screen than in conventional TV viewing. The range in which human vision can perceive a visual image with high resolution (the range captured by the fovea of the retina) is only within a viewing angle of several tens of degrees. Therefore, in wide-field video presentation, it is difficult for the viewer to grasp the entire screen in the center of the field of view.

Therefore, in wide-field video presentation, when the visual objects are widely dispersed in the visual field or when the visual objects are moving, it is difficult for the viewer to find the visual object, and there is a possibility that the viewer may lose sight of the visual object. . In this way, the wide-field video presentation enhances the sense of reality, but makes it difficult to correctly convey the intention of the video creator. Furthermore, wide-field video presentation requires the task of searching for a visual target, which imposes a burden on the viewer. As a result, in wide-field video presentations, viewers tend not to view video in a wide field of view, ie, viewers tend to move away from the screen to view the video.

JP 2005-182746 A JP 2014-99105 A JP-A-2006-327310

Ultra-high-definition video systems are designed based on the idea that video quality can be maintained even when the viewer approaches the screen. In order to realize efficient information transmission while maintaining the special viewing experience brought about by the high quality of ultra-high-definition video systems, it is necessary to reduce the viewer's burden when presenting wide-field video. In other words, in wide-field video presentation, there is a demand for a method of guiding only the viewer's line of sight that is close to the screen.

However, the techniques described in Patent Literatures 1 to 3 described above have a mechanism for selectively exerting a gaze guidance effect according to the viewing distance, such as guiding the gaze only for viewers close to the screen in wide-field video presentation. not

SUMMARY OF THE INVENTION An object of the present invention is to provide a line-of-sight guidance device capable of guiding a viewer's line of sight according to a viewing distance, and a program therefor.

In view of the above-described problems, the visual guidance device according to the present invention is based on a contrast sensitivity curve that expresses the visibility of a pattern according to the viewing distance based on the relationship between the contrast of the pattern and the spatial frequency, on a wide-field image. A line-of-sight guidance device that guides a viewer's line of sight to a line-of-sight guidance position is provided with a line-of-sight guidance image generator and an image synthesizing unit.

According to such a configuration, the visual guidance image generation unit generates a visual guidance image in which a pattern is drawn in which the contrast sensitivity curve and the spatial frequency decrease as the distance from the visual guidance position increases on the wide-field image.
Then, the image synthesizing unit synthesizes the visual guidance image generated by the visual guidance image generating unit with the wide-field image.

Since the pattern of the visual guidance image is drawn based on the contrast sensitivity curve, whether or not the pattern is visible differs depending on the viewing distance. In other words, since the visual guidance device allows a viewer close to the screen to visually recognize the pattern for visual guidance, only the viewer close to the screen can guide the visual line. On the other hand, since a viewer who is far from the screen cannot visually recognize the pattern for visual guidance in the visual guidance device, the viewer who is far from the screen does not lose a sense of reality.

Note that the visual guidance device according to the present invention can also be realized by a program that causes a general computer to operate cooperatively as each of the means described above.

According to the present invention, it is possible to guide the viewer's line of sight according to the viewing distance.

(a) and (b) are explanatory diagrams for explaining a contrast sensitivity curve in the present invention. FIG. 10 is an explanatory diagram illustrating an outline of processing of the visual guidance device in the embodiment; 1 is a block diagram showing the configuration of a visual guidance device according to an embodiment; FIG. FIG. 4 is an explanatory diagram for explaining parameters to be set in the visual guidance device in the embodiment; FIG. 10 is an explanatory diagram for explaining a visual guidance image in the embodiment; 4 is a flowchart showing the operation of the visual guidance device of FIG. 3; FIG. 11 is an explanatory diagram for explaining a visual guidance image in a modified example; An example of each image in an Example is represented, (a) is a wide-field-of-view image, (b) is a visual guidance image, (c) is a synthetic image.

(Principle of the present invention: contrast sensitivity curve)
After describing the principle of the present invention with reference to FIG. 1, embodiments will be described in detail.
In the present invention, a viewer's line of sight guidance is encouraged by synthesizing and displaying a pattern for line of sight guidance (for example, concentrated lines and concentric circles) on a wide-field image. This pattern is drawn with a low contrast near the limits of human vision and a low spatial frequency (high spatial frequency is also possible).

As shown in FIG. 1(a), it is known that there is a relationship between the contrast in human vision and the spatial resolution on the retina. In FIG. 1, the vertical axis represents contrast and the horizontal axis represents spatial frequency. Human vision works like a bandpass filter. That is, if the spatial frequency of the pattern projected onto the retina is too low or too high, the pattern cannot be seen. Furthermore, this tendency increases when the contrast value of the pattern (difference between white and black luminance values) decreases. As shown in FIG. 1B, the boundary between the area visible to humans and the area invisible to humans is represented by a contrast sensitivity curve CV.
In FIG. 1B, the area below the contrast sensitivity curve CV is visible to humans, and the area above the contrast sensitivity curve CV is invisible to humans. FIG. 1(b) is a contrast sensitivity curve CV plotted in FIG. 1(a).

As described above, since spatial frequencies are defined on the retina, changes in viewing distance will cause the contrast sensitivity curve CV to move up and down. At this time, the curve of the contrast sensitivity curve CV also slightly changes. Therefore, the pattern drawn with the contrast and spatial frequency near the contrast sensitivity curve CV is divided between visible and invisible depending on the viewing distance. In other words, it can be said that the contrast sensitivity curve CV determines whether or not a pattern is visible according to the viewing distance from the relationship between the contrast and the spatial frequency. Using the characteristics of the contrast sensitivity curve CV, it is possible to draw a pattern that does not exert the visual guidance effect on viewers far from the screen, but exerts the visual guidance effect only on viewers close to the screen.
The viewing distance is the ratio of the screen height of the wide-field image display device 2 to the distance from the wide-field image display device 2 to the viewer.

This pattern adjusts the spatial frequency and contrast according to the viewing distance so that it is barely visible when viewed from close to the screen. As a characteristic of humans, even if we cannot consciously perceive visual images, we can expect the guidance of the line of sight by effects (for example, subliminal effects) that affect human behavior.

(embodiment)
[Configuration of Gaze Guidance Image Generating Device]
BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In addition, in the embodiments, the same means are denoted by the same reference numerals, and the description thereof is omitted.
The configuration of a gaze guidance image generation device (sight guidance device) 1 according to the present embodiment will be described with reference to FIGS. 2 and 3. FIG.

The gaze guidance image generation device 1 guides the viewer's gaze to a gaze guidance position (sight guidance position) on the wide-field image. Specifically, as shown in FIG. 2, the gaze guidance image generation device 1 generates a gaze guidance image (sight guidance image) E in which a gaze guidance pattern M is drawn based on the contrast sensitivity curve CV. Then, the gaze guidance image generation device 1 synthesizes the generated gaze guidance image E and the wide-field image F, and outputs the synthesized image G to the wide-field image display device 2 .

The wide-field image F is an image that can be viewed at a wide viewing angle of 100 degrees or more at a viewing distance (for example, 0.75H) assumed for the wide-field image display device 2 . For example, the wide-field image F is an image with a resolution of 8K or higher. For example, if the wide-field image F has a resolution of 8K or higher, the viewing angle per pixel is 1 minute (1/60 degrees) or less, and the occurrence of unintended spatial frequency components due to the pixel structure can be suppressed. can.
Further, the gaze guidance image E is an image in which a pattern M is drawn to guide the viewer's line of sight to the gaze guidance position on the wide-field image F (the position of the aircraft T in FIG. 2). In this embodiment, the gaze guidance image E is a concentration line image in which concentration lines radiating from the periphery of the aircraft T are drawn as the pattern M.
Also, the wide-field image F and the gaze-guided image E are assumed to have the same size and to correspond to each other in pixel position.

The wide-field image display device 2 is a general display device that displays the synthetic image G input from the gaze guidance image generation device 1 . For example, the wide-field image display device 2 may be a flat panel display such as a liquid crystal display, a plasma display, or an organic EL display.

As shown in FIG. 3, the gaze guidance image generation device 1 includes a video reception/decode unit 10, a gaze guidance parameter acquisition unit 20, a screen parameter acquisition unit 30, a gaze guidance image generation unit (sight guidance image generation unit ) 40 and an image synthesizing unit 50 .

The video reception/decoding unit 10 receives video to be displayed on the wide-field image display device 2 and decodes the received video. For example, the video receiving/decoding unit 10 receives video by broadcasting or communication, and decodes the video by a method corresponding to encoding. Then, the video receiving/decoding unit 10 outputs each frame image (wide-field image) F of the decoded video to the image synthesizing unit 50 .

The gaze-guidance parameter acquisition unit 20 acquires gaze-guidance parameters necessary for generating a gaze-guidance image E. FIG. For example, the gaze-guidance parameter acquisition unit 20 acquires gaze-guidance parameters set in advance by the video creator through broadcasting or communication. Then, the gaze guidance parameter acquisition unit 20 outputs the acquired gaze guidance parameters to the gaze guidance image generation unit 40 .

The screen parameter acquisition unit 30 acquires screen parameters related to the screen of the wide-field image display device 2 . For example, the screen parameter acquisition unit 30 acquires screen parameters preset by the user of the gaze guidance image generating device 1 . The screen parameter acquisition unit 30 then outputs the acquired screen parameters to the gaze guidance image generation unit 40 .

The gaze-guidance image generating unit 40 generates a gaze-guidance image E in which a pattern M in which the contrast sensitivity curve CV and the spatial frequency decrease as the distance from the gaze-guidance position on the wide-field image F is drawn. . Then, the gaze guidance image generation unit 40 outputs the generated gaze guidance image E to the image synthesizing unit 50 .
Details of the gaze guidance image generation unit 40 will be described later.

The image synthesizing unit 50 synthesizes the gaze-guided image E from the gaze-guided image generator 40 and the wide-field image F from the video reception/decoding unit 10, and outputs the synthesized image G to the wide-field image display device 2. It is something to do.

<Generation and Synthesis of Gaze Guidance Image>
4 and 5, generation and synthesis of the gaze guidance image E will be described (see FIGS. 1 to 3 as necessary).
In FIGS. 4 and 5, the black pattern M is illustrated on the white background for easy viewing of the drawings, but in reality, the white pattern M is drawn on the black background (see FIG. 8). In addition, in FIGS. 4 and 5, the intensity of the contrast of the pattern M is illustrated by the density of the dots.

As shown in FIG. 4, the gaze guidance parameters include gaze guidance position (P _x , P _y ), frequency f of pattern M, contrast c of pattern M, size s ₁ of gaze guidance area A ₁ , pattern area _The size s2 of _A2 is included.

The frequency (spatial frequency) f of the pattern M corresponds to the number of concentrated lines when the pattern M is a concentrated line.
The contrast c of the pattern M is the luminance ratio between the background of the gaze guidance image E and the pattern M. As shown in FIG.
The gaze _- guidance area A1 is an area to which the viewer's gaze is guided in the gaze-guidance image E, and the pattern M is not drawn inside this area. This gaze guidance area A ₁ is a circular area with a radius s ₁ centered at the gaze guidance position (P _x , P _y ).
The pattern area _A2 is an area where the pattern M is drawn in the gaze guidance image E, and is a circular area with a radius s2 centered at the gaze guidance position ₍ P _x , P _y ).

This gaze guidance parameter is set in advance with an arbitrary value. Also, the frequency f and contrast c of the pattern M should be set in consideration of the contrast sensitivity curve CV at the viewing distance (for example, 0.75H) assumed for the wide-field image display device 2 . That is, the frequency f and the contrast c of the pattern M are determined based on the contrast sensitivity curve CV so that viewers below the assumed viewing distance can see the pattern M, while viewers beyond that viewing distance cannot see the pattern M. can be set.

_In addition, in the gaze guidance image E, the pattern M is drawn between the gaze guidance area A1 and the pattern area _A2 . Also, the pattern M may be drawn only in the range where the pattern area _A2 is included in the gaze guidance image E. FIG. In other words, there is no need to draw the pattern M in the range where the pattern area _A2 deviates from the gaze guidance image E (FIG. 4).

The screen parameters include the screen size and resolution of the wide-field image display device 2 . This screen size represents the height and width of the screen of the wide-field image display device 2 . Also, the resolution represents the number of vertical and horizontal pixels of the wide-field image display device 2 .
First, the gaze guidance image generation unit 40 refers to the screen parameters and obtains the pixel pitch p from the physical screen size and resolution. Specifically, the gaze guidance image generator 40 divides the screen size by the resolution to calculate the pixel pitch p.

Next, the gaze guidance image generation unit 40 adjusts the gaze guidance parameters according to the pixel pitch p. Specifically, the gaze-guidance image generation unit 40 calculates the frequency f' of the pattern M, the contrast c' of the pattern M, the size _s1 ' of the gaze _- guidance area A1, the pattern area Calculate the magnitude s ₂ ' of A ₂ . Note that x, y, z, and w are coefficients preset for adjustment (0≤x, y, z, w≤1).

Here, one arbitrary position is set on the gaze guidance image E, and the brightness value _vg of this arbitrary position (p _x , p _y ) is determined. In this case, the gaze-guided image generation unit 40 may generate the gaze-guided image E centered on the gaze-guided position (P _x , P _y ) using Equation (2) (0≦v _g ≦1). . In this equation (2), the luminance value v _g ( _px , _py ) corresponding to the frequency _f ' is calculated at an arbitrary position (px, _py ).

In the vicinity of the gaze guidance position (P _x , P _y ), a perceptible pattern (folding distortion) is generated by folding, so the pattern M is not drawn in this vicinity (within the gaze guidance area A ₁ ). . Further, as the distance from the gaze guidance position (P _x , P _y ) increases, the pattern M becomes more visible as the spatial frequency of the pattern M decreases, so the contrast also needs to be decreased. Therefore, the gaze guidance image generator 40 calculates the brightness adjustment coefficient v _n ( _px , _py ) at the arbitrary position ( _px , _py ) using Equation (3).

Note that in equation (3), the value of the _first term on the right side decreases toward the outside of the pattern area _A2 , and the value of the second term on the right side decreases toward the outside of the gaze guidance area A1.

Then, the gaze guidance image generation unit 40 calculates the luminance value v(p _x ,p _y ) at the arbitrary position (p _x ,p _y ) using the equations (4-1) and (4-2). . In this way, by multiplying the luminance value _vg ( _px , _py ) at an arbitrary position ( _px , _py ) by the luminance adjustment coefficient _vn ( _px , _py ), a gaze is obtained as shown in FIG. A induced image E can be generated.

Note that max is a function that obtains the maximum value. In addition, the formula (4-1) indicates that the contrast and spatial frequency of the pattern M between the gaze guidance area A ₁ and the pattern area A ₂ decrease as the distance from the gaze guidance position (P _x , P _y ) increases. represents Also, the coefficient γ in equation (4-2) is for normalization. That is, Equation (4-2) expresses that the maximum value of the luminance value v(p _x , p _y ) is set to 1 in the gaze guidance image E.

Next, the image synthesizing unit 50 calculates RGB values r', g', b' at an arbitrary position (p _x , p _y ) of the synthetic image G using Equation (5). In this formula (5), the RGB values at an arbitrary position (p _x , p _y ) of the wide-field image F are r, g, b (0≦r, g, b≦1).
Note that the image synthesizing unit 50 preferably adjusts the RGB values r′, g′, and b′ to 1 when each of them exceeds 1.

Note that although one arbitrary position (p _x , p _y ) has been described, the above calculation is actually performed for each pixel of the gaze guidance image E. FIG. That is, the gaze guidance image generation unit 40 and the image synthesis unit 50 set each pixel of the gaze guidance image E as an arbitrary position (p _x , p _y ), and perform the calculations of formulas (1) to (5) for gaze guidance. This is done for each pixel of image E.

[Operation of gaze guidance image generation device]
The operation of the gaze guidance image generating device 1 will be described with reference to FIG.
As shown in FIG. 6, the screen parameter acquisition unit 30 acquires screen parameters (screen size and resolution) of the wide-field image display device 2 (step S1).

The video receiving/decoding unit 10 receives video to be displayed on the wide-field image display device 2 and decodes the received video (step S2).
The gaze-guidance parameter acquisition unit 20 uses gaze-guidance parameters as gaze-guidance position (P _x , P _y ), frequency f of pattern M, contrast c of pattern M, size s ₁ of gaze-guidance area A ₁ , pattern _The size s2 of the area _A2 is obtained (step S3).

The gaze-guidance image generation unit 40 generates a gaze-guidance image E that depicts a pattern M in which the contrast sensitivity curve CV and the spatial frequency decrease as the distance from the gaze-guidance position increases. Specifically, the gaze-guided image generation unit 40 generates the gaze-guided image E using the formulas (1) to (4) described above (step S4).

The image synthesizing unit 50 synthesizes the gaze guide image E generated in step S4 and the video (wide-field image F) received in step S1 (step S5). Specifically, the image synthesizing unit 50 generates a synthetic image G of the gaze guidance image E and the wide-field image F using Equation (5) described above (step S5).
The image synthesizer 50 outputs the synthesized image G generated in step S5 to the wide-field image display device 2 (step S6).

The gaze guidance image generation device 1 (image synthesizing unit 50) determines whether or not to end the process. For example, when the gaze guidance image generation device 1 has synthesized the final wide-field image F (Yes in step S7), the process ends.
If the gaze guidance image generation device 1 does not end the process (No in step S7), it returns to the process of step S2 and processes the wide-field image of the next frame.

[Action/effect]
As described above, the gaze guidance image generating device 1 can guide the viewer's line of sight according to the viewing distance. In other words, the gaze-guidance image generation device 1 guides the line of sight only for viewers close to the screen, so that the presence of viewers far from the screen is not lost, and degradation in video quality can be suppressed. Furthermore, the gaze-guided image generation device 1 can reduce the hindrance to video comprehension caused by the viewer missing a visual target in wide-field video presentation. As a result, the gaze guidance image generation device 1 further improves viewing experience in wide-field video presentation such as 8K, and contributes to promotion of wide-field video presentation.

Since the pattern M is often drawn with very low contrast and very high spatial frequency, the encoding may cause the pattern M to disappear. Therefore, the gaze guidance image generation device 1 on the video receiving side draws the pattern M separately and synthesizes it with the video, thereby preventing the pattern M from disappearing.

(Modification)
Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and includes design changes and the like within the scope of the present invention.
In the above-described embodiments, the gaze guidance image generation device and the wide-field image display device are described as separate devices, but both devices may be integrated.

In the above-described embodiment, the drawing of concentration lines is described as a pattern for gaze guidance, but the pattern is not limited to this. For example, the gaze-guided image generation device may generate a gaze _- guided image E2 in which concentric circles are drawn as the pattern _M2 , as shown in FIG. In this gaze guidance image E2, the contrast and spatial frequency of the concentric circles become _lower as the distance from the gaze guidance position increases.
In the gaze-guided image E2 of _FIG . 7, the black pattern _M2 is shown on the white background to make the drawing easier to see, but in reality, the white pattern _M2 is drawn on the black background.

In the above-described embodiment, the gaze-guided image generation device has been described as independent hardware, but the present invention is not limited to this. For example, the present invention can also be implemented by a program that causes hardware resources such as a CPU, memory, and hard disk provided in a computer to operate cooperatively as the gaze-guided image generation device described above. These programs may be distributed via a communication line, or may be distributed after being written in a recording medium such as a CD-ROM or flash memory.

As an embodiment of the present invention, a gaze guidance image E is illustrated.
As shown in FIG. 8A, letters A, B, and C are displayed on a wide-field image F with a monochromatic background, and the viewer's line of sight is guided to letter B. As shown in FIG. In this case, the gaze-guided image generating device generates a gaze-guided image E as shown in FIG. 8(b), and synthesizes the wide-field image F and the gaze-guided image E as shown in FIG. 8(c). In the synthetic image G of FIG. 8C, the concentrated lines are drawn centering on the character B, and the farther away from the character B, the less conspicuous the concentrated lines are.

Note that the gaze guidance image E in FIG. 8B has a frequency f=100, a contrast c=0.1, a size s ₁ =3, s ₂ =1500, a pixel pitch p=1, and x=y=z= The pattern M is drawn with the parameter w=1.

1 gaze guidance image generation device (sight guidance device)
2 wide-field image display device 10 video reception/decoding unit 20 gaze guidance parameter acquisition unit 30 screen parameter acquisition unit 40 gaze guidance image generation unit (sight guidance image generation unit)
50 Image synthesizing unit A ₁ gaze guidance area A ₂ pattern areas CV Contrast sensitivity curves E, E ₂ gaze guidance image (sight guidance image)
F wide-field image G composite image M, M ₂ patterns T aircraft

Claims (5)

A line-of-sight guidance device that guides the line of sight of a viewer to a line-of-sight guidance position on a wide-field image based on a contrast sensitivity curve that expresses whether or not a pattern can be viewed according to the viewing distance by the relationship between the contrast of the pattern and the spatial frequency. There is
a visual guidance image generation unit that generates a visual guidance image in which a pattern in which contrast and spatial frequency in the contrast sensitivity curve decrease as the distance from the visual guidance position on the wide-field image is drawn;
an image synthesizing unit that synthesizes the visual guidance image generated by the visual guidance image generating unit with the wide-field image;
A line-of-sight guidance device comprising: 2. The visual guidance device according to claim 1, wherein the visual guidance image generating unit generates the visual guidance image in which concentrated lines or concentric circles are drawn as the pattern. 3. The visual guidance image according to claim 1, wherein the visual guidance image generation unit generates the visual guidance image without drawing the pattern in a visual guidance area centered on the visual guidance position. Device. The visual guidance image generation unit is
pixel pitch p of the display device that displays the wide _- field image, frequency f of the pattern, contrast c of the pattern, size s1 of the visual guidance area, size _s2 of the pattern area for drawing the pattern, and Using Equation ( ₁ ) including preset coefficients x, y, z, and w, the frequency f' of the pattern, the contrast c' of the pattern, the size s1' of the visual guidance area, the pattern Calculate the size of the region s ₂ ',

Using the formula (2) including the visual guidance position (P _x , P _y ) and the arbitrary position ( _{px, p y ) of the visual guidance image, the arbitrary position (p x , p y} ) Calculate the brightness value v _g (p _x , p _y ),

Using equation (3), calculate the brightness adjustment coefficient v _n ( _px , p _y ) at the arbitrary position (p _x , p _y ),

Calculate the luminance value v (p _x , p _y ) at the arbitrary position (p _x , p _y ) using equation (4) including the function max for obtaining the maximum value,

The visual guidance device according to any one of claims 1 to 3, wherein the visual guidance image of the calculated brightness value v( _px , _py ) is generated. A program for causing a computer to function as the visual guidance device according to any one of claims 1 to 4.

Images