JP2023161493A - Display system, display method, and display program - Google Patents

Abstract

To provide a display system or the like, which allows a viewer to easily obtain a space expansion feeling and an immersion feeling against an image reflected onto a screen.SOLUTION: A display system comprises: a content data storage part that stores image data of a content; display means of enabling a display of the content to a screen; a camera that images a face of a viewer who views the screen; a setting value storage part that stores a position relation with the screen and the camera and an angle of an optical shaft of a camera to the screen; a detection part that detects the face of the viewer from an image imaged by the camera; a relative relation calculation part that calculates a relative position relation of the face of the viewer against the screen on the basis of a position of the face of the viewer in the image, the position relation between the screen and the camera, and the relative angle; and a display control part that makes an image region of at least one part of the content display on the screen, and controls the image region in the content displayed on the screen on the basis of the relative position relation.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display system, a display method, and a display program that display content configured using moving images or still images.

In recent years, content consisting of panoramic images taken with a spherical camera or fisheye lens, or panoramic images or 3D images created using three-dimensional computer graphics (3DCG) has become popular not only for entertainment such as games, but also for tourism information and real estate. It is used in a wide range of fields, including previews and vocational training.

For example, Patent Document 1 discloses a photographing system that generates a spherical image using a photographing device and outputs it to a smartphone.

JP2017-84422A

In general, images displayed on screens such as televisions, displays, and digital signage do not change no matter what angle the viewer views them from. In other words, unlike a real subject, viewers can only see the same image whether they are looking at the screen from the front or looking up from below. Therefore, the viewer only recognizes the image displayed on the screen as an image, and the viewer experiences a sense of spatial expansion, as if the space within the image is connected to real space, and a sense of immersion, as if entering the space within the image. It is difficult to have.

When a panoramic image is displayed on a display connected to a smartphone, tablet device, or personal computer, viewers can change the image area within the panoramic image displayed on the screen by performing swipe or drag operations. (For example, see Patent Document 1). There is also content in which the image area displayed on the screen changes by changing the orientation of the smartphone or the like. However, in any case, in order to change the image area within the panoramic image displayed on the screen, operations on the device such as swiping or changing the orientation of the device are required, so it is still possible for the viewer to experience a sense of spatial expansion and immersion. It is difficult to get a feel for it.

In recent years, goggle-type display devices called head-mounted displays or VR goggles have become popular. In such display devices, the image area displayed as a panoramic image or 3D image changes according to the head movement of the viewer wearing goggles, so the viewer can relatively easily become immersed in the image. can be obtained. However, in this case, it is necessary to prepare VR goggles, so it is difficult to say that it can be used easily. Furthermore, wearing heavy VR goggles itself may hinder the viewer's sense of immersion.

The present invention has been made in view of the above, and it is an object of the present invention to provide a display system, a display method, and a display program that allow viewers to easily obtain a sense of spatial expansion and immersion in images displayed on a screen. purpose.

In order to solve the above problems, a display system that is one aspect of the present invention includes a display unit that can display content configured using moving images or still images on a screen, and an image of the face of a viewer viewing the screen. a camera installed at a possible position, a setting value storage unit that stores the positional relationship between the screen and the camera, and the angle of the optical axis of the camera with respect to the screen; a detection unit that detects the viewer's face; and a detection unit that detects the viewer's face relative to the screen based on the position of the viewer's face in the image, the positional relationship between the screen and the camera, and the relative angle. a relative relationship calculation unit that calculates a relative positional relationship between the faces of the content, and displays at least a partial image area of the content on the screen, and displays the content on the screen based on the relative positional relationship and a display control section that controls an image area within the content.

In the above display system, when the viewer's face moves parallel to the surface of the screen, the display control unit controls the image area within the content displayed on the screen to It may be shifted in the opposite direction.

In the above display system, the content is a panoramic image in which coordinates corresponding to a celestial sphere are set, and the display control unit is configured to display an image included in an angle of view of a virtual camera set at the center of the celestial sphere. Displaying the area on the screen, and rotating a direction vector indicating the center direction of the angle of view of the virtual camera in the same direction as a rotation direction of a direction vector from the center point of the screen toward the viewer's face. It's okay.

In the display system, the setting value storage further stores the size of a display area on the screen where the content is displayed, and the display control unit stores the size of the display area and the relative size of the display area. An image area within the content displayed in the display area may be controlled based on the positional relationship.

In the display system, the display control unit narrows an image area within the content displayed on the screen when the viewer's face moves away from the screen, and also adjusts the size of the image area to the display screen. When the viewer's face approaches the screen, the image area within the content displayed on the screen is widened, and the size of the image area is adjusted to fit the display area. You can also reduce it.

In the above display system, the display control unit may perform control so that the content displayed on the screen is recognized by the viewer as a view through a window.
In the above display system, the display means may include a projector configured to project the content onto the screen.
In the above display system, the display means may be a display device having a liquid crystal panel or an organic EL panel.

A display method that is another aspect of the present invention is a display method that displays content composed of moving images or still images on a screen, and includes an imaging step of capturing an image of the face of a viewer viewing the screen with a camera. a detection step of detecting the viewer's face from an image captured by the camera; the position of the viewer's face in the image; the positional relationship between the screen and the camera; and the camera relative to the screen. a relative relationship calculation step of calculating the relative positional relationship of the face of the viewer with respect to the screen based on the angle of the optical axis of the screen; and displaying at least a partial image area of the content on the screen; The method further includes a display control step of controlling an image area within the content displayed on the screen based on the relative positional relationship.

A display program that is another aspect of the present invention is a display program that displays content composed of moving images or still images on a screen, and includes an imaging step of causing a camera to image the face of a viewer viewing the screen. a detection step of detecting the viewer's face from an image captured by the camera; the position of the viewer's face in the image; the positional relationship between the screen and the camera; and the camera relative to the screen. a relative relationship calculation step of calculating the relative positional relationship of the face of the viewer with respect to the screen based on the angle of the optical axis of the screen; and displaying at least a partial image area of the content on the screen; and a display control step of controlling an image area within the content displayed on the screen based on the relative positional relationship.

According to the present invention, since the image area within the content displayed on the screen is controlled based on the relative positional relationship of the viewer's face with respect to the screen, the viewer can feel a sense of spatial expansion with respect to the image displayed on the screen. It becomes possible to easily obtain a sense of immersion.

1 is a schematic diagram showing a schematic configuration of a display system according to a first embodiment of the present invention. 2 is a block diagram showing a schematic functional configuration of the information processing device shown in FIG. 1. FIG. FIG. 2 is a schematic diagram for explaining the display principle of a panoramic image. FIG. 2 is a schematic diagram showing how the scenery looks through the window. FIG. 2 is a schematic diagram showing how the scenery looks through the window (when the viewer moves in parallel). FIG. 2 is a schematic diagram showing how the scenery looks through the window (when the viewer moves away from the window). FIG. 2 is a schematic diagram showing how the scenery looks through the window (when the viewer approaches the window). 2 is a flowchart showing display control operations in the information processing apparatus shown in FIG. 1. FIG. FIG. 3 is a schematic diagram for explaining the relative positional relationship of a viewer to a screen. FIG. 3 is a schematic diagram illustrating an initial setting state in displaying a panoramic image. FIG. 2 is a schematic diagram illustrating an image obtained by capturing an image of a viewer with a camera. 2 is a schematic diagram for explaining display control (when a viewer moves in parallel) in the display system shown in FIG. 1. FIG. FIG. 2 is a schematic diagram for explaining display control in the display system shown in FIG. 1 (when the viewer moves away from the screen). 2 is a schematic diagram for explaining display control (when a viewer approaches a screen) in the display system shown in FIG. 1. FIG. FIG. 2 is a schematic diagram showing a schematic configuration of a display system according to a second embodiment of the present invention. FIG. 2 is a schematic diagram showing a schematic configuration of a display system according to a third embodiment of the present invention. 17 is a block diagram showing a schematic functional configuration of the display system shown in FIG. 16. FIG.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An image display system, a display method, and a display program according to embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these embodiments. In addition, in the description of each drawing, the same parts are denoted by the same reference numerals.

(First embodiment)
FIG. 1 is a schematic diagram showing a display system according to a first embodiment of the present invention. As shown in FIG. 1, a display system 1 according to the present embodiment includes a display unit 10 capable of displaying content (images) composed of moving images or still images on a screen, and a screen on which images are displayed. It includes a camera 20 installed at a position where it can image the face of the viewer 100, and an information processing device 30 that outputs image data to the display means 10 to display content.

In this embodiment, the display means 10 includes a projector 11 that projects an image onto the screen 12 based on image data output from the information processing device 30. Although a planar screen 12 is shown in FIG. 1, the shape of the screen is not limited to a planar shape. It's okay to do so. Further, the projector 11 may project an image onto a building wall or the like instead of the screen 12.

The size of the display area Sc, which is an area where an image is displayed (projected) on the screen 12, is determined by the distance between the projector 11 and the screen 12 and the projection angle from the projector 11. The projection angle can be adjusted by operating the projector 11 or controlling the projector 11 by the information processing device 30.

The camera 20 is a digital camera that includes a solid-state imaging device such as a CCD or CMOS and is capable of capturing moving images. The camera 20 generates an image signal by capturing an image of a subject, converts the image signal into a digital signal, and outputs the converted image signal to the information processing device 30 . The camera 20 may be connected to the information processing device 30 by wire using a USB (Universal Serial Bus) or the like, or may be connected via wireless communication such as Wi-Fi (Wireless Fidelity) or Bluetooth (registered trademark). good.

The specs of the camera 20 include, for example, a frame rate high enough to detect normal movements of the viewer 100 such as walking and moving the face, and a resolution high enough to detect the face 101 of the viewer 100 through image processing. There is no particular limitation if it is. For example, a general-purpose web camera may be used as the camera 20. Furthermore, as will be described later, a camera built into a smartphone, tablet terminal, or the like may be used.

Although the camera 20 is installed above the screen 12 in FIG. 1, the position of the camera 20 is not particularly limited as long as it can capture the face 101 of the viewer 100 viewing the screen 12. For example, the camera 20 may be installed on the side or below the screen 12, or the camera 20 may be installed at a position away from the screen 12.

The information processing device 30 can be configured by, for example, a general-purpose computer such as a personal computer (PC), a notebook PC, or a tablet terminal. The information processing device 30 causes the display means 10 to display content based on image data (hereinafter also referred to as content data) for displaying content configured using moving images or still images, and also displays content from the camera 20. The display of the content on the display means 10 is controlled based on the image signal thus obtained. The information processing device 30 may be connected to the display means 10 (projector 11 in FIG. 1) by wire or wirelessly.

FIG. 2 is a block diagram showing a schematic functional configuration of the information processing device 30. As shown in FIG. As shown in FIG. 2, the information processing device 30 includes an external interface 31, a storage section 32, and a processor 33. Further, the information processing device 30 may include a display section 34 and an operation input section 35.

The external interface 31 is an interface that connects the information processing device 30 to an external device and transmits and receives data to and from the external device. Further, when the information processing device 30 is wirelessly connected to the projector 11 or the camera 20, the information processing device 30 connects the information processing device 30 to a communication network such as a LAN or a telecommunications line, and connects the information processing device 30 to the communication network. The device may also include a communication interface for communicating with other devices. The communication interface can be configured using, for example, a soft modem, cable modem, wireless modem, ADSL modem, or the like.

The storage unit 32 can be configured using, for example, a semiconductor memory such as ROM or RAM, or a computer-readable storage medium such as a hard disk. The storage section 32 includes a program storage section 321 , a content data storage section 322 , and a setting value storage section 323 . The program storage unit 321 stores operating system programs, driver programs, application programs that execute various functions, and various parameters used during execution of these programs. Specifically, the program storage unit 321 stores a display program for controlling the display of content on the display means 10.

The content data storage unit 322 stores image data of content configured using moving images or still images. In this embodiment, a panoramic image displayed based on image data generated in Equirectangular format (equirectangular projection) will be described as an example of content. Furthermore, the content data storage unit 322 may store audio data that is output together with the content.

The setting value storage unit 323 stores various setting values and parameters used during execution of the display program stored in the program storage unit 321. The setting values stored in the setting value storage section 323 include the size of the display area Sc on the screen 12, the positional relationship between the screen 12 and the camera 20, the angle of the optical axis of the camera 20 with respect to the screen 12, and the like. Note that when the screen is curved, the angle of the optical axis of the camera 20 with respect to the normal line at the center point of the screen may be stored as a set value.

The processor 33 is configured using, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and centrally controls each part of the information processing device 30 by reading various programs stored in the program storage unit 321. At the same time, various calculation processes for displaying the content on the screen 12 are executed. Functional units realized by the processor 33 reading the display program stored in the program storage unit 321 include an image generation unit 331, a detection unit 332, a relative relationship calculation unit 333, and a display control unit 334.

The image generation unit 331 generates an image based on the image signal input from the camera 20.
The detection unit 332 detects the face of the viewer from the image by performing face detection processing on the image generated by the image generation unit 331, and outputs the position (coordinates) of the face in the image.

The relative relationship calculation unit 333 calculates the viewer's relative position relative to the screen 12 based on the position of the viewer's face in the image, the positional relationship between the screen 12 and the camera 20, and the angle of the optical axis L of the camera relative to the screen 12. The relative positional relationship of 100 faces 101 is calculated.

The display control unit 334 causes at least a part of the content (for example, a panoramic image) to be displayed on the screen 12, and also controls the display on the screen 12 based on the relative positional relationship of the face 101 of the viewer 100 with respect to the screen 12. control the image area within the content that is displayed.

The display unit 34 is, for example, a liquid crystal display or an organic EL display. The operation input unit 35 is an input device such as a keyboard, a pointing device such as a mouse, or a touch panel provided on the display unit 34, and accepts input operations for the information processing device 30.

FIG. 3 is a schematic diagram for explaining the display principle of a panoramic image. In the image data of a panoramic image generated in the Equirectangular format, as shown in FIG. The coordinates have been set. If a virtual camera Cv is placed at the center of such a celestial sphere Sp and this camera Cv captures an image of the inner surface of the celestial sphere Sp, the image area that falls within the field of view Fv of the camera Cv is ( As shown in b), it is displayed in the display area Sc of the screen 12. The display control unit 334 controls the display of content in the display area Sc by changing the size and direction of the viewing angle Fv of the camera Cv based on the relative positional relationship of the face of the viewer 100 with respect to the screen 12. .

Next, with reference to FIGS. 4 to 7, a description will be given of how the viewer views the outside scenery through the window. FIGS. 4 to 7 are schematic diagrams showing how the scenery looks through the window. Among these, FIG. 5 shows a case where the viewer 100 moves parallel to the surface of the window W compared to the state shown in FIG. FIG. 6 shows a case where the viewer 100 moves away from the window W. FIG. 7 shows a case where the viewer 100 approaches the window W. 4 to 7 (a) show the positional relationship between the viewer 100 and the window W when the viewer 100 is viewed from the top of the head, and FIGS. 4 to 7 (b) show the positional relationship between the viewer 100 and the window W. The view outside the window W seen from 100 is shown.

As shown in (a) of FIG. 4, when the viewer 100 is inside the window W, the viewer 100 is able to understand the distance D0 between the viewer 100 and the window W, as shown in (b) of FIG. , the outside scenery Sn can be seen within the field of view V within the window frame determined by the size of the window (width X, height Y).

On the other hand, as shown in FIG. 5, when the viewer 100 moves in parallel while maintaining the distance D0 from the window W, the outside scenery Sn that enters the field of view V within the window frame will be different from the direction in which the viewer 100 is moving. Shift in the opposite direction. For example, as shown in (a) of FIG. 5, when the viewer 100 moves to the right with respect to the window W, the scenery Sn on the left as seen from the viewer 100 comes into the field of view V, while the view on the right The view Sn is blocked by the window frame and cannot be seen.

Further, as shown in FIGS. 6 and 7, when the distance between the viewer 100 and the window W changes, the outside scenery Sn that falls within the field of view V within the window frame changes according to the distance between the two. For example, as shown in (a) of FIG. 6, when the viewer 100 moves away from the window W (distance D0<D1), as shown in (b) of FIG. As the size of the viewed window W becomes smaller, the range of the outside scenery Sn that falls within the field of view V within the window frame becomes narrower. On the other hand, as shown in (a) of FIG. 7, when the viewer 100 approaches the window W (distance D0>D2), the position of the viewer 100 changes due to perspective, as shown in (b) of FIG. As the window W seen from above becomes larger, the range of scenery Sn that falls within the field of view V within the window frame becomes wider.

In the display method according to the present embodiment, the display area Sc on the screen 12 is likened to a window W, and the content displayed in the display area Sc is presented to the viewer 100 as a view through the window as shown in FIGS. 4 to 7. It controls the display so that it is recognized.

FIG. 8 is a flowchart showing the operation of display control by the information processing device 30. FIG. 9 is a schematic diagram for explaining the relative positional relationship of the viewer 100 with respect to the screen 12. FIG. 10 is a schematic diagram illustrating an initial setting state in displaying a panoramic image. Of these, FIG. 10(a) shows the positional relationship between the viewer 100 and the screen 12 as seen from the top of the head, and FIG. 10(b) shows the horizontal direction of the celestial sphere Sp to which the panoramic image is pasted. FIG. 10C shows an image area within the content displayed in the display area Sc of the screen 12. FIG. FIG. 11 is a schematic diagram illustrating an image obtained when the camera 20 images the viewer 100.

First, the information processing device 30 performs initial settings (step S10). Specifically, the information processing device 30 uses the center of the display area Sc of the screen 12 as the origin, the horizontal direction of the screen 12 as the x axis, the vertical direction of the screen 12 as the y axis, and the normal direction of the screen 12 as the z axis. Set the screen coordinate system O. Then, the information processing device 30 determines the coordinates (xc, yc, zc) of the camera 20 in the screen coordinate system O, the relative angle (θ, φ) of the optical axis L of the camera 20 with respect to the normal to the screen 12, Set the size (width w0, height h0) of the display area Sc. Here, the angle θ indicates an angle in a horizontal plane (xz plane) (that is, an angle around the y axis), and the angle φ indicates an angle in a vertical plane (yz plane) (that is, an angle around the x axis).

The information processing device 30 also performs settings in the coordinate system of a panoramic image displayed as content. Specifically, as shown in FIG. 10(a), the information processing device 30 is located at a position (0, 0, d0) is set as the initial value of the coordinates of the face 101 of the viewer 100 in the screen coordinate system O. Then, based on the width w0 of the display area Sc (see FIG. 9), the horizontal angle a0 at which the viewer 100 faces the display area Sc from this position (0, 0, d0) is determined in the horizontal direction of the celestial sphere Sp. This is set as the size of the angle of view Fv (see (b) in FIG. 10). The viewing angle in the vertical direction of the celestial sphere Sp is similarly set based on the height h0 of the display area Sc.

Further, as shown in FIG. 10(b), the information processing device 30 converts the direction vector C indicating the optical axis direction of the virtual camera Cv on the celestial sphere Sp, that is, the center direction of the angle of view Fv, to the celestial sphere. Align with reference point c0 on Sp.

Further, the information processing device 30 associates a direction vector V from the center of the screen coordinate system O toward the face 101 of the viewer 100 with a direction vector C of the camera Cv.

Thereby, the image area included in the angle of view Fv centered on the reference point c0 on the celestial sphere Sp is initially set as the range displayed in the display area Sc of the screen 12 (see (c) in FIG. 10). Note that image display on the screen 12 may start after the camera 20 starts capturing an image of the viewer 100.

Subsequently, the information processing device 30 detects the viewer's face from the image captured by the camera 20, and obtains the coordinates of the face in the image (step S20). Specifically, as shown in FIG. 11, the image generation unit 331 of the image processing device 30 generates the image M1 based on the image signal captured from the camera 20. The detection unit 332 detects the face M2 by performing face detection processing on the image M1, and outputs the coordinates (xf, yf, zf) of the face M2 in the image M1. The coordinates of the face M2 in the image M1 may be, for example, the center coordinates of the frame M3 indicating the area of the face M2, or may be the coordinates indicating the position of a specific subject, such as the intermediate coordinates of both eyes. Various known methods can be used for the face detection process, and a publicly available library may be read. Note that the face detection process in this embodiment is performed to detect the position of the face M2 in the image M1, so the image M1 illustrated in FIG. 11 does not particularly need to be displayed.

Subsequently, the information processing device 30 calculates the position (xf, yf, zf) of the face M2 in the image M1, the coordinates (xc, yc, zc) of the camera 20 with respect to the screen 12, and the light of the camera with respect to the normal line of the screen 12. The relative coordinates (xf', yf', zy') of the face 101 of the viewer 100 with respect to the screen 12 are calculated based on the relative angle (θ, φ) of the axis L (step S30, see FIG. 9). . Specifically, the relative relationship calculation unit 333 converts the coordinates of the face 101 in the camera coordinate system (that is, the coordinates of the face M2 in the image M1) to the coordinates of the face 101 in the screen coordinate system O. As the coordinate transformation, a known method such as affine transformation can be used.

Subsequently, the information processing device 30 determines the direction of the direction vector C of the virtual camera Cv on the celestial sphere Sp or the size of the angle of view Fv based on the change in the relative coordinates of the face 101 of the viewer 100 with respect to the screen 12. Adjust (step S40).

Further, the information processing device 30 outputs image data corresponding to the image area included in the adjusted viewing angle Fv to the projector 11, and displays the content on the screen 12 based on the image data (step S50).

12 to 14 are schematic diagrams for explaining display control in the display system 1. Among these, FIG. 12 shows a case where the viewer 100 moves parallel to the surface of the screen 12 with respect to the initial setting state shown in FIG. FIG. 13 shows a case where the viewer 100 moves away from the screen 12. FIG. 14 shows a case where the viewer 100 approaches the screen 12. 12 to 14 (a) show the positional relationship between the viewer 100 and the screen 12 as seen from the top of the head, and FIGS. 12 to 14 (b) show the screen 12 on which the panoramic image is pasted. 12 to 14 (c) show an image area within the content displayed in the display area Sc of the screen 12. FIG.

For example, as shown in (a) of FIG. 12, when the face 101 of the viewer 100 moves in parallel to the right while maintaining the distance from the screen 12, the display control unit 334 ), the angle of view Fv is shifted in the direction opposite to the moving direction of the face 101 (to the left). Thereby, as shown in FIG. 12C, the image area displayed in the display area Sc changes in accordance with the movement of the face 101. Thereby, the viewer 100 can recognize the content displayed in the display area Sc in the same way as when the viewer 100 moves in parallel while looking at the outside scenery through the window (see FIG. 5).

As for vector processing on the celestial sphere Sp, when the direction of the direction vector V from the center of the display area Sc toward the face 101 changes due to the movement of the face 101, the display control unit 334 converts the change in the direction vector V into the camera Cv. This is reflected in the direction vector C indicating the optical axis direction of. Specifically, when the direction vector V rotates counterclockwise around the y-axis due to movement of the face 101 in the horizontal plane (xz plane), the display control unit 334 rotates the direction vector V in the horizontal plane of the celestial sphere Sp. Rotate the direction vector C in the same direction as the direction. Thereby, the angle of view Fv shifts in the direction opposite to the moving direction of the face 101. The same applies when the face 101 moves in the vertical direction.

The amount of rotation of the direction vector C of the camera Cv may be the same as the amount of rotation of the direction vector V of the face 101, or may be the amount obtained by multiplying the amount of rotation of the direction vector V by a predetermined coefficient k. In the latter case, regarding the movement of the face 101 in the horizontal direction, the display control unit 334 obtains a projection vector by projecting the direction vector V after the movement onto the xz plane, and calculates the angle of the projection vector with respect to the normal vector N of the screen 12. Calculate |Δa|. Then, the direction vector C of the camera Cv is rotated from the direction of the reference point 0 by an angle |k·Δa| obtained by multiplying this angle |Δa| by a coefficient k. Furthermore, regarding the movement of the face 101 in the vertical direction, the display control unit 334 obtains a projection vector by projecting the direction vector V onto the yz plane and performs similar processing.

Here, if the display area Sc is much larger than the viewer 100, the change in the position of the face 101 will be relatively small, and therefore the change in the image area in the display area Sc will be slight. In such a case, it becomes difficult for the viewer 100 to recognize changes in the image area in the display area Sc with respect to his or her own movements. On the other hand, when the display area Sc is smaller than the viewer 100, the change in the position of the face 101 becomes relatively large, so even a slight movement of the viewer 100 causes a large change in the image area in the display area Sc. Resulting in. Therefore, as described above, by scaling the amount of rotation of the direction vector V using the coefficient k, it is possible to change the image area in the display area Sc in a way that feels natural to the viewer 100.

As shown in (a) of FIG. 13, when the viewer 100 moves away from the screen 12, the display control unit 334 acquires the distance d1 between the viewer 100 and the screen 12 after the movement. Then, based on the width w0 and distance d1 of the display area Sc, an angle a1 at which the viewer 100 faces the display area Sc is calculated, and as shown in FIG. Set as Fv. Here, as the distance d1 between the viewer 100 and the screen 12 increases, the viewing angle Fv becomes smaller compared to before the movement (see FIG. 10). Then, the image area included in the narrowed angle of view Fv is enlarged and displayed in accordance with the size of the display area Sc. The same applies to the vertical direction.

When the viewer 100 moves away from the screen 12, the size of the display area Sc seen from the viewer's 100 position decreases due to perspective. That is, since the enlarged image area is displayed in the display area Sc that appears small, the viewer 100 perceives that the size of the object within the display area Sc has not changed from before the movement. Thereby, the viewer 100 can recognize the content displayed in the display area Sc in the same way as when the viewer 100 moves away from the window while looking at the outside scenery through the window (see FIG. 6).

Further, as shown in (a) of FIG. 14, when the viewer 100 approaches the screen 12, the display control unit 334 acquires the distance d2 between the viewer 100 and the screen 12 after the movement. Then, based on the width w0 and distance d2 of the display area Sc, an angle a2 at which the viewer 100 faces the display area Sc is calculated, and as shown in FIG. Set as Fv. Here, when the distance d2 between the viewer 100 and the screen 12 becomes smaller, the viewing angle Fv becomes larger compared to before the movement (see FIG. 10). Then, the image area included in the widened angle of view Fv is reduced and displayed in accordance with the size of the display area Sc. The same applies to the vertical direction.

When the viewer 100 approaches the screen 12, the size of the display area Sc seen from the viewer's 100 position increases due to perspective. That is, since the reduced image area is displayed in the display area Sc that appears larger, the viewer 100 perceives that the size of the object within the display area Sc has not changed from before the movement. Thereby, the viewer 100 can recognize the content displayed in the display area Sc in the same way as when the viewer 100 approaches the window while looking at the outside scenery through the window (see FIG. 7).

As explained above, according to the present embodiment, the image area within the content displayed on the screen 12 is controlled based on the relative positional relationship of the face 101 of the viewer 100 with respect to the screen 12, so that the viewer The user 100 can recognize the image displayed in the display area Sc of the screen 12 as if viewing the outside scenery through a window. Thereby, the viewer 100 can feel the image displayed in the display area Sc realistically, and can obtain a natural sense of spatial expansion.

Furthermore, according to the present embodiment, by acquiring the position of the face in the image captured by the camera 20 of the viewer 100, the content displayed in the display area Sc can be adjusted in a natural manner according to the movement of the face. The image area can be changed. In other words, the viewer 100 can view the content by simply moving his or her face in the direction in which he or she wants to view the content, without having to operate the screen 12, thereby providing a deeper sense of immersion in the content. becomes.

Further, according to the present embodiment, the size of the angle of view Fv is adjusted according to the distance between the viewer 100 and the screen 12, and the image area included in the angle of view Fv is expanded or expanded to match the display area Sc. Since the object is displayed in a reduced size, the viewer 100 can recognize the object in the content as having the same size as the actual object.

Furthermore, according to the present embodiment, the viewer can view content with a sense of spatial expansion and immersion without using an input device or VR goggles. Therefore, the display method according to this embodiment can be applied to signage and displays that are viewed by unspecified people.

(Modification 1)
In the first embodiment, a panoramic image is exemplified as the content displayed on the screen 12, but it is also possible to display content other than panoramic images. For example, a 3D space created with 3DCG may be displayed as content. In this case, not only the position of the viewer's face but also the direction of the face is detected from the image M1 captured by the camera 20, and the inclination of the object in the 3D space displayed in the display area Sc is determined according to the direction of the face. may be changed. For example, when the viewer looks down, the top surface of the object in the 3D space may be displayed in the display area Sc.

(Modification 2)
In the first embodiment, the image area within the content displayed in the display area Sc is controlled based on the position of the face 101 of the viewer 100. The image area may be controlled according to the following. As an example, a predetermined gesture such as "spreading the right hand sideways" is detected based on the posture of the viewer 100 in the image M1 captured by the camera 20, and the display area Sc is The image area within the panoramic image displayed may be shifted in a predetermined direction (for example, rightward).

Alternatively, a 3D space created by 3DCG may be displayed as the content, and objects in the content may be displayed so as to move forward toward the viewer in response to a gesture such as "raise your right hand." In this case, the viewer can be made to perceive that the viewer is moving relatively forward. By controlling the display according to such gestures of the viewer, it is also possible to configure a function that allows the viewer to move around in the 3D space.

(Second embodiment)
FIG. 15 is a schematic diagram showing a schematic configuration of a display system according to a second embodiment of the present invention. As shown in FIG. 15, the display system 2 according to this embodiment includes a display device 40 instead of the projector 11 and screen 12 shown in FIG. The configuration and operation of the display system 2 other than the display device 40 are the same as those shown in FIG. 1.

The display device 40 has a screen made of a liquid crystal panel or an organic EL panel, and displays content composed of moving images or still images on the screen based on image data output from the information processing device 30.

The information processing device 30 may set the entire screen of the display device 40 as the content display area Sc, or may set a part of the screen as the content display area Sc. By setting the size of the display area Sc according to the size of the object in the content, the viewer can recognize the object in the content with the same sense of size as the actual object.

(Third embodiment)
FIG. 16 is a schematic diagram showing a schematic configuration of a display system according to a third embodiment of the present invention. FIG. 17 is a block diagram showing a schematic functional configuration of the display system shown in FIG. 17. As shown in FIGS. 16 and 17, the display system 3 according to this embodiment includes a display device 40 and a tablet terminal 50 connected to the display device 40 by wire or wirelessly.

The display device 40 has a screen made of a liquid crystal panel or an organic EL panel, and displays content composed of moving images or still images on the screen based on image data output from the tablet terminal 50.

As shown in FIG. 17, the tablet terminal 50 includes an external interface 31, a storage section 32, a processor 33, an imaging section 51, a display section 52, an operation input section, and an audio input/output section 54. Among these, the configuration and operation of the external interface 31, the storage unit 32, and the processor 33 are the same as those of the external interface 31, the storage unit 32, and the processor 33 included in the information processing device 30 shown in FIG. Further, when the tablet terminal 50 is wirelessly connected to the display device 40, the tablet terminal 50 includes a communication interface for connecting the tablet terminal 50 to a communication network.

The imaging unit 51 is a camera built into the tablet terminal 50 and generates an image signal by imaging a subject, digitally converts the image signal, and inputs it to the processor 33 . As shown in FIG. 16, the tablet terminal 50 is installed at a position where the face of the viewer 100 is within the viewing angle of the imaging unit 51. In FIG. 16, the tablet terminal 50 is installed horizontally below the display device 40, but the position and orientation of the tablet terminal 50 are not limited to this. In short, it is sufficient that the image capturing unit 51 can image the face of the viewer 100, and for example, the tablet terminal 50 may be installed parallel to the display device 40.

The display section 52 is, for example, a liquid crystal display or an organic EL display. The operation input section 53 is a touch panel provided on the display section 52. The audio input/output section 54 includes a microphone and a speaker.

In the present embodiment, an image of the viewer 100 is captured by the imaging unit 51 built into the tablet terminal 50, and content display control on the display device 40 is performed based on the position of the viewer's face in the image generated thereby. conduct. Therefore, according to this embodiment, the configuration of the display system 3 can be simplified.

As a modification of this embodiment, a notebook PC or a smartphone equipped with a built-in camera may be used instead of the tablet terminal 50. Furthermore, as another modification of the present embodiment, the tablet terminal 50 may be connected to the projector 11 that projects an image onto the screen 12, and image data may be output from the tablet terminal 50 to the projector, as in FIG. 1. .

The present invention is not limited to the first to third embodiments and modifications, but can be achieved by appropriately combining a plurality of components disclosed in the first to third embodiments and modifications. , various inventions can be formed. For example, some components may be excluded from all the components shown in the first to third embodiments and modifications, or the first to third embodiments and modifications may be formed by excluding some components. The illustrated components may be combined as appropriate.

1, 2, 3...Display system, 10...Display means, 11...Projector, 12...Screen, 20...Camera, 30...Information processing device, 31...External interface, 32...Storage unit, 33...Processor, 34, 52... Display section, 35, 53... Operation input section, 40... Display device, 50... Tablet terminal, 51... Imaging section, 54... Audio input/output section, 100... Viewer, 101... Face, 321... Program storage section, 322... Content data storage section, 323... Setting value storage section, 331... Image generation section, 332... Detection section, 333... Relative relationship calculation section, 334... Display control section

Claims (10)

A display means capable of displaying content composed of moving images or still images on a screen;
a camera installed at a position capable of capturing an image of the face of a viewer viewing the screen;
a setting value storage unit that stores the positional relationship between the screen and the camera and the angle of the optical axis of the camera with respect to the screen;
a detection unit that detects the face of the viewer from an image captured by the camera;
A relative positional relationship of the viewer's face with respect to the screen is calculated based on the position of the viewer's face in the image, the positional relationship between the screen and the camera, and the relative angle. a relative relationship calculation section;
a display control unit that causes at least a part of the image area of the content to be displayed on the screen and controls the image area within the content that is displayed on the screen based on the relative positional relationship;
A display system equipped with. The display control unit is configured to move an image area within the content displayed on the screen in a direction opposite to the moving direction of the viewer's face when the viewer's face moves parallel to the surface of the screen. The display system according to claim 1, wherein the display system is shifted. The content is a panoramic image in which coordinates corresponding to a celestial sphere are set,
The display control unit displays on the screen an image area included in the angle of view of the virtual camera set at the center of the celestial sphere, and also displays a direction vector indicating the center direction of the angle of view of the virtual camera. 3. The display system according to claim 2, wherein is rotated in the same direction as a direction of rotation of a direction vector from the center point of the screen toward the viewer's face. The setting value storage unit further stores the size of a display area that is an area where the content is displayed on the screen,
The display system according to claim 1, wherein the display control unit controls an image area within the content displayed in the display area based on the size of the display area and the relative positional relationship. The display control section includes:
when the viewer's face moves away from the screen, narrowing an image area within the content displayed on the screen and expanding the size of the image area to match the display area;
widening an image area within the content displayed on the screen when the viewer's face approaches the screen, and reducing the size of the image area to match the display area;
The display system according to claim 4. 6. The display system according to claim 1, wherein the display control unit performs control so that the content displayed on the screen is recognized by the viewer as scenery through a window. The display system according to any one of claims 1 to 5, wherein the display means includes a projector configured to project the content onto the screen. 6. The display system according to claim 1, wherein the display means is a display device having a liquid crystal panel or an organic EL panel. A display method for displaying content composed of moving images or still images on a screen, the method comprising:
an imaging step of imaging the face of the viewer viewing the screen using a camera;
a detection step of detecting the face of the viewer from an image captured by the camera;
The relative position of the viewer's face with respect to the screen is determined based on the position of the viewer's face in the image, the positional relationship between the screen and the camera, and the angle of the optical axis of the camera with respect to the screen. a relative relationship calculation step for calculating positional relationships;
a display control step of displaying at least a partial image area of the content on the screen and controlling the image area of the content displayed on the screen based on the relative positional relationship;
Display method including. A display program that displays content composed of moving images or still images on a screen,
an imaging step of causing a camera to take an image of the face of the viewer viewing the screen;
a detection step of detecting the face of the viewer from an image captured by the camera;
The relative position of the viewer's face with respect to the screen is determined based on the position of the viewer's face in the image, the positional relationship between the screen and the camera, and the angle of the optical axis of the camera with respect to the screen. a relative relationship calculation step for calculating positional relationships;
a display control step of displaying at least a partial image area of the content on the screen and controlling the image area of the content displayed on the screen based on the relative positional relationship;
A display program that causes a computer to execute

Images