JP5598790B2 - Video display system - Google Patents

Description

  The present invention relates to a video display system that displays a partial video in a viewing area of a video display unit viewed by a user.

  Japanese Patent Application Laid-Open No. 9-284676 (Patent Document 1), Japanese Patent No. 3796776 (Patent Document 2), Japanese Patent Application Laid-Open No. 9-93700 (Patent Document 3), Japanese Patent No. 2550832 (Patent Document 4), etc. A technique is disclosed in which a direction in which a user is facing is detected by a sensor mounted on a headphone mounted on a user's head, and an image displayed on a head-mounted display mounted by the user is switched according to the detected direction. Yes. On the head-mounted display, a high immersive feeling is obtained, but the symptoms of screen sickness have been pointed out.

  Therefore, in order to prevent screen sickness, a technology has been developed to project the image on the wall or screen and to localize the sound image of the image viewed by the user. For example, Japanese Patent Laid-Open No. 2003-284196 (Patent Document 5) discloses a technique related to a sound image localization signal processing apparatus that can localize a sound image in the direction of a moving image even when the image is moved and reproduced. ing. This gazette has a function to cut out a rectangular image from a 360 ° image, and can be changed to a free viewpoint by key operation, and further, the image and sound image can be localized by information on the changed angle of the image. The technology that enables the above is specifically disclosed.

JP-A-9-284676 Japanese Patent No. 3796776 Japanese Patent Laid-Open No. 9-93700 Japanese Patent No. 2550832 JP2003-284196

  Although the technique disclosed in Patent Document 5 can prevent screen sickness, it is necessary to change the video to the viewpoint by key operation, and thus the problem is that the video cannot be displayed in the direction that the user is intuitively facing. is there.

  An object of the present invention is to provide an image display system that can display an image in a direction intuitively viewed by a user without using a head-mounted display and without requiring a key operation. .

  In addition to the above object, another object of the present invention is to provide an image display system that can display an image in a direction intuitively viewed by a user with a simple configuration and low power consumption. .

  In addition to the above object, another object of the present invention is to provide a video display system capable of displaying video in a direction intuitively viewed by the user simply by wearing the headphones. .

  In addition to the above object, another object of the present invention is to provide a video display system capable of displaying a video with no sense of incongruity regardless of the shape of the projection screen that displays the video.

  In addition to the above object, an object of the present invention is to provide a video display system capable of displaying a video in a direction intuitively viewed by a user and providing a sound image.

  In addition to the above object, an object of the present invention is to provide a video display system that makes it easy to observe the behavior of children.

  The video display system of the present invention includes a video display unit, a viewing area specifying unit, a video data storage unit, and a partial video display device as basic components. The video display unit is arranged at a position where the user can see his / her body and head. As a typical image display unit, a projection screen can be used. However, an electrical display device such as a liquid crystal display may be used as the video display unit. The watching area specifying unit specifies the watching area of the video display unit viewed by the user. As a technique for specifying the viewing area of the video display unit viewed by the user, a known technique may be used and is arbitrary. Here, the “viewing area” in the present specification means a predetermined area within the visual range that the user can see without moving his / her head. Although the maximum range of the viewing area coincides with the visual range, the viewing area may of course be narrower than the visual range. The video data storage unit stores video data necessary for displaying video on the entire video display unit. The video data is all data for displaying video on the entire video display unit, and may be either still image data or moving image data. The partial video display device acquires partial video data for the partial video to be displayed in the viewing area specified by the viewing area specifying unit from the video data storage unit based on the output of the viewing area specifying unit. The partial video display device displays the partial video in the viewing area that the user is viewing on the video display unit.

  According to the present invention, the partial video is displayed in the viewing area viewed by the user on the video display unit, and no video is displayed on the other video display units. Therefore, according to the present invention, it is possible to display an image in a direction that the user views intuitively without using a head mounted display and without requiring a key operation. In other words, according to the present invention, since the video is not displayed on the entire video display unit including the area that is not viewed by the user, the complicated video display device conventionally required to display the entire video is provided. do not need. Therefore, the power consumption required for video display can be greatly reduced as compared with the conventional case.

  When a projection screen is used as the video display unit, the viewing area specifying unit is mounted on the user's head and includes a head posture / azimuth detection unit having a gyro function that detects the posture / orientation of the head, It is possible to use one that includes a viewing area determination unit that determines a viewing area from the attitude direction output from the body orientation detection unit. Regarding the head posture / orientation detection unit having such a gyro function, the above-described Patent Documents 1 to 4 also disclose specific configurations. The viewing area determination unit determines the viewing area based on the posture direction detected by the head posture direction detection unit. Specifically, the viewing area can be determined in principle so that a virtual line extending in the attitude direction detected by the head attitude direction detection unit passes through the center of the viewing area or the vicinity of the center. As will be described later, when the projection screen has a boundary portion, it is also permitted to define the viewing area according to a predetermined criterion related to the posture orientation in which the position of the viewing area is exceptionally detected. .

  The projection screen may be a wall or ceiling of a building other than a dedicated screen. When a projection screen is used, a partial video display device equipped with a projector that is mounted on the user's head and projects the partial video onto the viewing area can be used. In this specification, the term “projector” refers to a CRT projector that enlarges and projects an image displayed on a CRT using an optical system, and a light from a light source lamp that uses a discharge light. The projector includes at least a projection portion of a known projector device such as a liquid crystal projector that transmits the light and enlarges and projects it on a screen using a lens.

  If such a configuration is adopted when a projection screen is used, a device for basic projection will be mounted on the user, so that if there is a projection screen anywhere, an image can be displayed. it can.

  Note that the partial video display device displays the partial video data according to the viewing area so that the video projected on the projection screen becomes a video with distortion corrected when viewed from the user according to the shape of the projection screen. It is preferable to further include a data correction unit for correction. Since the data correction unit corrects the image distortion by correcting the partial image data, it is possible to apply various correction techniques that have been developed in the projection technology of a known projector. Even in the projection of the partial video, by applying this known correction technique, it is possible to show the partial video without any sense of incongruity to the user.

  In addition, when using the multi-surface projection screen comprised combining two or more projection surfaces so that a boundary part may be formed as a projection screen, a data correction part uses the correction formula used on the basis of this boundary part. Change it. For example, a boundary portion is always formed between a non-inclined plane and an inclined surface. Further, the method of image distortion differs between when projecting on a non-inclined plane and when projecting on an inclined surface. Therefore, if the correction formula is changed at the boundary, appropriate correction can be executed.

  In addition, in order to show the user an uncomfortable image, the conversion from the image data to the image data obtained through the standard lens required according to the type of lens used when obtaining the image data is eliminated. A cut area setting unit for setting a cut area of the partial video data to be cut may be provided. Also in this case, the partial video display device corrects the partial video data so that the video projected on the projection screen becomes a video with distortion corrected when viewed from the user, according to the shape of the projection screen. It is preferable to provide a data correction unit. It should be noted that the cut area setting unit preferably changes the shape and size of the cut area of the partial video data cut from the video data with reference to the boundary. The basic data for determining the shape and size of the cut-out area may be obtained by a preliminary test according to the shape of the projection screen to be used and the type of lens used for obtaining the image data. By providing such a cut-out area setting unit, for example, a conversion step for converting video data acquired through a fisheye lens into video data obtained through a standard lens can be omitted.

  Video data captured using a camera equipped with a fisheye lens can be used. When projecting video on a projection screen based on video data captured using a camera equipped with a fisheye lens, a correction method for outputting a rectangular image from an image obtained from a fisheye lens, or for operating in real time Various techniques for speeding up conversion have already been proposed. For example, Japanese Patent Application Laid-Open Nos. 2007-13791 and 2009-75646 disclose techniques for suppressing image deterioration when an image such as a fisheye lens is cut out. In addition, WO2004 / 102479 discloses a technique for preventing pixel loss at the periphery of a fisheye image. Furthermore, Japanese Patent Laid-Open No. 2008-311890 discloses a technique for creating a wide-angle image with little distortion. Furthermore, Japanese Patent Laid-Open No. 2000-11166 and Japanese Patent Laid-Open No. 2004-157999 disclose a technique for speeding up fisheye correction conversion, and Japanese Patent Laid-Open No. 2009-176273 discloses a user specifying parameters in advance. Thus, a technique for reducing the computational burden of fisheye conversion is disclosed. Japanese Patent Application Laid-Open No. 11-243508 and Japanese Patent Application Laid-Open No. 6-36020 have a function of cutting an image desired to be viewed from an image obtained by a fisheye lens or a 360 ° image, and further switching and enlarging / reducing the image. JP-A-2006-287942, JP-A-2000-83242 and the like. Therefore, if these known techniques are used, partial video data can be easily obtained from image data obtained by a fisheye lens. These known techniques do not disclose or suggest a technique for intuitively selecting a desired direction as in the present invention.

  An audio data storage unit having audio data for letting the user hear a sound image synchronized with the image displayed on the entire image display unit, and a sound image generating unit for providing the user with a sound image synchronized with the image You may have. Note that a sound image localization technique for a video is described in Patent Document 5 and the like. If a sound image synchronized with the video is provided to the user, music video such as live music can be provided to the user with a sense of reality. In this case, the speaker unit of the sound image generation unit may be a headphone worn on the user's head, and the projector may be worn on the headphone. In this way, basically, if there is a projection screen and headphones, it is possible to easily display an image in a direction intuitively viewed by the user.

  The content of the video shot by the camera may be anything. If a surveillance camera attached to a child is used as a camera equipped with a fisheye lens, the image can be displayed only in the direction that the supervisor who monitors the child intuitively sees, so that the surveillance using the intuition is maximized. It becomes possible.

It is a block diagram which shows the structure of 1st Embodiment of the video display system of this invention. (A) thru | or (C) is a figure used in order to demonstrate the use condition of the video display system of 1st Embodiment. (A) And (B) is a figure for demonstrating horizontal rotation angle (theta) and elevation angle (phi). It is a figure used in order to explain the problem of projection by a projector. It is a figure for demonstrating the mode change based on a boundary part. It is a figure which shows the flow of a process to the projection including correction | amendment in the case of using the video data obtained using the fisheye lens. It is a figure which shows the illustration of the step which converts the video data obtained through the fish-eye lens into the video data obtained through the standard lens of arbitrary focus, and the video at that time. It is a figure which shows the flowchart of the algorithm of the process at the time of setting the boundary part of a projection screen. It is a figure which shows the algorithm of the process for obtaining the partial video data correct | amended from the partial video data in the data correction part. It is a conceptual diagram which shows processing the partial video data i as data of several division images, when obtaining the corrected partial video data from the partial video data. (A) thru | or (C) are the figures used in order to demonstrate step ST22 of FIG. (A) And (B) is a figure used in order to demonstrate step ST23 of FIG. It is a block diagram which shows the structure of 2nd Embodiment of the video display system of this invention. It is a figure used in order to explain the use condition of the picture display system of a 2nd embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. In the embodiment described below, for example, at home, live images such as videos and DVDs can be placed in a place other than a live house such as a home without using a head-mounted display or requiring key operations. Even so, it is intended to provide a video display system that allows you to enjoy a live presence. Conventionally, there is a technology called CubicVR that can convert a wide range of images such as a fisheye lens and display a part of the image. Cubic VR is a spherical panorama that can be seen not only at 360 ° per circle but also directly above and below. It is possible to look around the surroundings by taking a picture that covers the entire circumference as a material and connecting them together. However, CubicVR does not mean that people are actually moving, and the images on the display are rotated and moved by mouse operation, so if you watch a moving image for a long time, you may become screen sick. In the video display system of the present embodiment, the video (screen) displayed on the projection screen changes in conjunction with the natural movement of the user.

  FIG. 1 is a block diagram showing the configuration of the first embodiment of the video display system of the present invention. FIG. 2 is a diagram used for explaining a use state of the video display device. The video display system 1 in FIG. 1 includes, as basic components, a projection screen (video display unit) 3, a viewing area specifying unit 5, a boundary setting unit 7, a video data storage unit 9, and a partial video. A display device 11 and a sound image generation device 13 are provided. As shown in FIG. 2, in the present embodiment, the projection screen 3 as the video display unit forms a predetermined angle between the front projection surface 31 located in front of the user P and the front projection surface 31. The right end of the front projection surface 31 is formed so as to form a predetermined angle between the front projection surface 31 and the left projection surface 32 that is continuous with the left end portion of the front projection surface 31 and extends toward the user P side. And a right projection surface 33 extending to the user P side. The shape of the projection screen is not limited to the shape shown in FIG. 2, and various shapes can be used.

  The watching area specifying unit 5 includes a head posture / azimuth detecting unit 5A and a watching area determining unit 5B in order to specify the watching area WR of the projection screen 3 that the user is looking at. The head orientation detection unit 5A used in the present embodiment has a gyro function that is attached to the user A's head and detects the orientation of the head. Patent Documents 1 to 4 described above also disclose a specific configuration of the head posture abandonment detection unit having a gyro function. Then, the viewing area determination unit 5B determines the viewing area WR from the posture direction output by the head posture direction detection unit 5A. The viewing area WR is a predetermined area whose maximum range is a visual range that the user can see without moving his / her head. The watching area WR is an area that overlaps an area for projecting a partial video described later. The user P sees the periphery of the viewing area WR together with the viewing area WR. The head posture / azimuth detection unit 5A is attached to the user's head and, as shown in FIGS. 3A and 3B, the horizontal rotation angle θ (−π / 2 ≦ θ of the user's head). ≦ π / 2) and elevation angle φ (−π / 4 ≦ φ ≦ π / 4) are acquired. As a specific sensor that can be used for the head posture / azimuth detection unit 5A, 3DM (trademark) manufactured by MicroStrain can be used. This 3DM is a sensor having a structure in which a magnetometer and an accelerometer are arranged orthogonally, and has a wide range of posture direction, that is, azimuth information, that is, Yaw (horizontal rotation angle θ), Pitch (elevation angle φ), and Roll (lens rotation). It can be measured in the measurement range. The viewing area determination unit 5B basically determines the viewing area WR so that a virtual line extending in the posture direction detected by the head posture direction detection unit 5A passes through the center or the vicinity of the center of the viewing area WR. In this embodiment, as will be described later, when the projection screen has a boundary portion, it can be determined according to a predetermined standard related to the posture orientation in which the position of the viewing area is exceptionally detected. To do. In order to determine a region (projection region) for displaying a partial video, which will be described later, and a partial video to be displayed in the region (region of partial video data cut out from the video data), the viewing region WR is determined.

  The boundary setting unit 7 determines the positions of the boundary portions BL1 and BL2 between the front projection surface 31 and the left projection surface 32 of the projection screen 3. Specifically, first, the user P looks at the boundary portion BL1 and presses an input switch for boundary portion registration (not shown) at that time. Information on posture orientation when this input switch is pressed is acquired from the head posture orientation detection unit 5A, and the position of the boundary portion BL1 is set. Similarly, the user P looks at the boundary portion BL2 and presses an input switch for boundary portion registration (not shown) at that time. Information on posture orientation when this input switch is pressed is acquired from the head posture orientation detection unit 5A, and the position of the boundary portion BL2 is set. The position information of the boundary portions BL1 and BL2 is given to the partial video display device 11 together with the information on the viewing area WR determined by the viewing area determination section 5B. The position information of the boundary portions BL1 and BL2 is used in correction calculation by the data correction unit 11B described later.

  The video data storage unit 9 stores video data necessary for displaying video on the entire projection screen (video display unit) 3. The video data is all data for displaying video on the entire video display unit (projection screen) 3 and may be either still image data or moving image data. If a plurality of video display devices are used as in the prior art, the video can be projected on the entire projection screen 3, but in the present invention, the video is not projected on the entire projection screen. A partial image to be projected on the viewing area is displayed only in the viewing area WR to be viewed. A technique for acquiring partial video data for a partial video from video data will be described later. Note that the video data in the video data storage unit 9 is used in synchronization with the corresponding acoustic data stored in the sound image data storage unit 13A in the sound image generation device 13 described later.

  The partial video display device 11 includes a partial video display unit 11A and a data correction unit 11B, and the viewing area determination unit 5B identifies the viewing area determination unit 5B based on the output of the viewing area determination unit 5B. Partial video data for a partial video to be displayed in the viewing area WR is acquired (cut out) from the video data storage unit 9. As will be described in detail later, the partial video data is corrected by the data correction unit 11B based on the position of the projection region WR relative to the boundary portion BL. Then, the partial video display unit 11A displays the partial video in the viewing area WR viewed by the user on the projection screen 3 based on the corrected partial video data.

  In the present embodiment, a projector that is attached to the user's head and projects a partial video onto the viewing area is used as the partial video display device 11. In the present embodiment, a partial video display unit 11A and a data correction unit 11B are built in the projector. As a projector, a CRT projector that enlarges and projects an image displayed on a CRT using an optical system and a liquid crystal panel are built in, and light from a light source lamp using discharge light is transmitted through the lens. It is possible to use a projector device having a known structure such as a liquid crystal projector that is enlarged and projected onto a screen, to which a correction function is added. In the present embodiment, as shown in FIG. 2, the partial video display device 11 composed of a projector is mounted at the center of a headphone HP that is worn on the head of the user P. That is, the partial video display device 11 is mounted near the top of the user P's head. When the projector constituting the partial video display device 11 is mounted on the headphones, the partial video can be easily displayed in the direction intuitively viewed by the user if there is basically a projection screen and headphones. The data correction unit 11B may be provided on the video display device main body side where the video data storage unit 9 is disposed.

  The sound image generation device 13 includes an acoustic data storage unit 13A having acoustic data for letting the user hear a sound image synchronized with the image displayed on the entire projection screen 3 (image display unit), and a sound image synchronized with the image. A sound image generating unit 13B provided to the user and a speaker 13C are further provided. Since the sound image localization technology for video is described in Patent Document 5 and the like, it will be omitted. If a sound image synchronized with the video is provided to the user, music video such as live music can be provided to the user with a sense of reality. In the present embodiment, the speaker 13C of the sound image generation device 13 is mounted on a headphone HP that is worn on the user's head.

  As shown in FIGS. 2A to 2C, according to the present embodiment, the partial video is displayed on the viewing area WR viewed by the user P on the projection screen 3, and the other video display units are displayed. No video is displayed on the top. Therefore, it is possible to display an image in a direction intuitively viewed by the user without using a head-mounted display and without requiring a key operation. In other words, according to the present embodiment, since the video is not displayed on the entire projection screen 3 including the area that is not viewed by the user P, the complicated video that has been conventionally required to display the entire video. No display device is required. Therefore, the power consumption required for video display can be greatly reduced as compared with the conventional case.

  The data correction unit 11B described above responds to the viewing area WR so that the image projected on the projection screen 3 becomes an image with distortion corrected when viewed from the user P according to the shape of the projection screen 3. Then, the partial video data cut out from the video data storage unit 9 is corrected. For example, as shown in FIG. 4, when projecting the end of the stage on the front projection surface 31 of the projection screen 3, the projector is rotated in the left-right direction to display a partial image of the stage end of the projection screen 3. Projecting onto the front projection surface 31. In such a situation, the image of the best position BP reflected in the square is also tilted. That is, as shown in FIG. 4, the video portion in which the position of the front projection surface 31 is in front of the best position BP is shorter than the normal size, and the position of the front projection surface 31 is in the back of the best position BP. The video part becomes longer than the normal size. An image projected in this way appears to be trapezoidal, so it is called trapezoidal distortion. This correction for making the trapezoidal image appear square is called trapezoidal distortion correction (keystone correction). The data correction unit 11B is configured to display the partial video data according to the tilt so that the partial video is not deformed into a trapezoid even when the projector (11) is tilted with respect to the projection plane (tilted in the horizontal direction or the vertical direction). Perform correction processing. As an example of the correction process, Apple Inc. It is conceivable to perform correction for deforming a partial video image in a shape opposite to the projected shape by using an image rotation function of “Quicktime (trademark)” of a multimedia program developed by the company.

  In particular, when using a projection screen arranged at an angle with respect to the front projection surface 31 such as the left projection surface 32 and the right projection surface 33 as in the projection screen 3 of the present embodiment, the boundary The angle of the screen changes at the part BL. Therefore, the data correction unit 11B also performs correction in this case. Such a screen situation arises when not only one wall of a room but also the left and right walls of the one wall are used as a screen. The reason why such a screen is used is that it is visually unnatural to display all images over 180 ° on one flat screen. Therefore, the inventor considered using a projection screen capable of projecting an image using not only the front but also the left and right walls on the premise of using indoors. However, when displaying a partial image on the left and right walls (the left projection surface 32 and the right projection surface 33), the image is deformed when the projector is directed toward the boundary portion BL. For this reason, when the posture orientation of the user is directed to the corner (boundary portion) of the projection screen, it is preferable to switch the image to either the front or the left and right walls (left and right projection surfaces).

  In order to execute this image switching, a partial video display unit 11A of the partial video display device 11 that has a projection angle change function capable of changing the projection angle in accordance with a command is used. Then, when the boundary portion is included in the original viewing area WR, the data correction unit 11B instructs to change the projection angle so as to project the partial video onto the area adjacent to the boundary part BL in the projection screen 3. Is configured to output. In this way, since no video is shown in the area including the boundary portion BL, it is possible to provide a video display without a sense of incongruity. Which of the two adjacent screen portions (the front projection surface 31 and the left or right projection surface 32 or 33) displays the partial image is based on, for example, the area of the partial image divided by the boundary portion. Select the projection plane. If the areas are the same, one projection plane is selected according to a predetermined rule.

For example, as shown in FIG. 5, calibration is performed by measuring the distance to the boundary portion BL in advance so that the video can be switched at a certain angle based on the boundary portion BL. The distance to the boundary BL and the distance to the front wall (front projection surface 31) are measured, and angles θ1 and θ2 are obtained. Then, θ ₀ is calculated from the angles θ1, θ2. When the horizontal rotation angle of the user is within the angle range θ ₀ , the projection is performed on the front projection surface 31 (front mode), and when the horizontal rotation angle exceeds the angle range θ ₀ , the left projection surface 32 according to the exceeded direction. Alternatively, the projection mode is changed so that the projection is performed on one of the right projection surfaces 33 (left mode or right mode). The data correction unit 11B changes the correction formula used in the front mode, the left mode, and the right mode with reference to the boundary part BL. In the case of projecting on the front projection surface 31 that is not inclined with respect to the user P and in the case of projecting on the surface that is inclined with respect to the user P, such as the left projection surface 32 and the right projection surface 33, image distortion is caused. Is different. Therefore, as described above, if the projection plane of the image is switched at the boundary portion BL and the correction formula is changed, appropriate correction can be executed.

Video data taken using a camera equipped with a fisheye lens can be used as the video data. FIG. 6 is a diagram showing a flow of processing up to projection including correction when video data I ₀ obtained using a fisheye lens is used. FIG. 7 shows an example of a step of converting video data I ₀ obtained through a fisheye lens into video data I ₁ obtained through a standard lens with an arbitrary focus and an image at that time. As shown in FIG. 7, when an image is projected based on image data obtained through a fisheye lens, a curved image as displayed on a spherical surface is obtained (step ST1). This is converted into video data I ₁ obtained through a standard lens of arbitrary focus in step ST2. When projected based on the converted video data, a standard video is obtained as shown in FIG. In step ST3, the azimuth angles (horizontal rotation angles) θ = −90 °, 90 °, and the elevation angle φ obtained by acquiring the three end portions of the image of the converted video data I ₁ with the 3DM sensor including the acceleration sensor. = Set so that the positions are −90 ° and 90 °. The converted video data I ₁ set in this way is stored in the video data storage unit 9.

FIG. 8 shows a flowchart of a processing algorithm for setting the boundary BL of the projection screen 3. In step ST11, it is determined whether or not a reference direction setting switch (SW) (not shown) has been pressed. When the reference direction setting switch is pressed, the current direction is set to the front. That is, the user takes a posture of staring at the center of the front projection surface of the projection screen 3 and presses the reference direction setting switch SW. By this operation, the angle that the user is facing at that time is set as a reference angle (θ = 0 °, φ = 0 °). The position of the reference angle becomes the center of the image. Next, in step ST13, it is determined whether or not the left corner switch has been pressed. The direction in which the user is facing when the left corner switch is pressed is set as the left corner (boundary portion BL1 in FIG. 5) (step ST14). Next, in step ST15, it is determined whether or not the right corner switch has been pressed. When the right corner switch is pressed, the direction in which the user is facing is set as the right corner (boundary portion BL2 in FIG. 5) (step ST16). By setting the left corner and the right corner, the angle θ _L and the angle θ _R shown in FIG. 5 are determined, and as a result, the angle θ ₀ is determined. By this determination, when the azimuth angle (horizontal rotation angle) of the posture azimuth determined by the head posture azimuth detecting unit 5A exceeds θ _L or θ _R from the reference angle, the image is displayed on the front projection surface 31. Can perform signal processing to switch to the left projection surface or the right projection surface. In the present embodiment, the switching process is performed when the azimuth angle (horizontal rotation angle) = θ _L or θ _R. Therefore, correction processing is performed so that the partial video is displayed on the projection surface before switching.

  Returning to FIG. 6, when projecting a partial image on the projection screen 3, based on the posture orientation determined by the head posture orientation detection unit 5 </ b> A, the viewing area determination unit 5 </ b> B allows the user to use a standard lens. Judging by the video converted into an image, it is determined which part is being viewed, and the viewing area WR is determined. In order to project the partial video onto the determined viewing area WR, the partial video display unit 11A cuts the partial video data i from the video data according to the viewing area WR. Then, the data correction unit 11B performs trapezoidal distortion correction on the cut partial video data i in consideration of the horizontal rotation angle θ and the elevation angle φ (tilt) of the partial video display device (projector) 11. The corrected partial video data i ′ is output. Then, the partial video display unit 11A performs projection based on the corrected partial video data i ′, so that a partial video with no sense of incongruity is displayed in the viewing area WR of the projection screen 3. According to this processing technique, signal processing is relatively simple because research on conversion from an eye to an image with an arbitrary focus has already been performed. However, since the conversion is performed twice, there is a problem that the image is deteriorated, but there is no problem in practical use.

  FIG. 9 shows a processing algorithm for obtaining the partial video data i ′ corrected from the partial video data i in the data correction unit 11B. In FIG. 10, when the corrected partial video data i ′ is obtained from the partial video data i, the partial video data i is processed as data of a plurality of segment images (24 segment data in the example of FIG. 10). It is a conceptual diagram which shows that. In FIG. 10, numerals 1 and 24 entered in the square sections indicate the section numbers (n). In step ST21 of FIG. 9, n = 1 is set. Therefore, in step ST22, the correction calculation of the projection image (partial video data) is executed with respect to the horizontal rotation angle θ for the first section. This correction calculation will be described later. Next, in step ST23, correction calculation of the projection image (partial video data) is executed with respect to the elevation angle φ. In step ST24, an enlargement / reduction correction process is performed in consideration of both the horizontal rotation angle θ and the elevation angle φ. In step ST25, it is determined whether n has reached 24. If n = 24, n + 1 is made in step ST26. Steps ST22 to ST24 are repeated until the correction calculation is completed for the first to 24th sections. When the correction is completed for all the sections, the correction of the partial video data by the data correction unit 11B is completed. The corrected partial video data i ′ is sent to the partial video display unit 11A, and the partial video display unit 11A projects an image on the projection screen 3 based on the corrected partial video data i ′.

The correction calculation in step ST22 of FIG. 9 will be described with reference to FIGS. In the following description, an “image” is a partial video image corresponding to each section of FIG. In the correction in step ST22, first, the enlargement magnification of the left side of the projected image is calculated. Therefore, as shown in FIG. 11A, the distance between the projector and the projection screen when the projector as the partial video display device 11 is projected onto the front projection surface of the projection screen (not shown) is A, and the screen The height of the upper image is H, the width is W, the projected horizontal rotation angle is θ, the projected elevation angle is φ, and the distance to the edge of the image is X. As shown in FIG. 11B, when the projector is tilted from the front by an angle θ ₁ , the distances between the left and right edges P and S of the image projected on the screen are A ₁ and A _4. the width is W _1. D is a virtual center line passing through the center of the image, and the angle θ ₁ is an angle between the virtual line indicating the distance A and the virtual center line D. In this case, the angle between the virtual center line D and the virtual line indicating the distance A ₁ is θ / 2. When the distance A ₁ is expressed by an equation under such conditions, it is as follows.

A ₁ = A / cos (θ ₁ −θ / 2)
At the point P, it is considered that the image has been enlarged by A ₁ / X times than when projected in front. A ₁ / X is the magnification on the left side.

Next, the magnification of the right side of the projected image is calculated. In this calculation, the distance A ₄ is expressed by the following equation as described above.

A ₄ = A / cos (θ ₁ + θ / 2)
At the point S, the image is considered to have been enlarged by A ₄ / X times than when projected in front. Therefore, A ₄ / X is the magnification on the right side.

Then, the output image is reduced based on the calculated magnification. The “original image” shown in FIG. 11C is an image obtained when projected from the front onto the screen as shown in FIG. The “projected image” is an image projected without correction in the case of FIG. Therefore, the corrected image to be output from the partial video display device 11 is obtained by multiplying PQ in FIG. 11C by A ₁ / X and reducing RS by A ₄ / X. Such correction is performed for all sections of the partial video.

Next, based on FIG. 9, the correction calculation in step ST23 is demonstrated using FIG. 11 (A), FIG. 12 (A), and (B). In the following description, an “image” is a partial video image corresponding to each section of FIG. In the correction in step ST23, first, the magnification of the lower side of the projected image is calculated. Therefore, as shown in FIG. 12A, the distances between the upper and lower edges R and S of the image projected on the screen when the projector is tilted from the front by the elevation angle φ ₁ are A ₂ and A _3. Is set to H ₁ . D is a virtual center line passing through the center of the image, and the angle φ ₁ is an angle between the virtual line indicating the distance A and the virtual center line D. In this case, the angle between the virtual center line D and the virtual line indicating the distance A ₂ is φ / 2. When the distance A ₂ is expressed by an equation under such conditions, it is as follows.

A ₂ = A / cos (φ ₁ −φ / 2)
At the point R, the image is considered to have been enlarged by A ₂ / X times than when projected in front. A ₂ / X is the magnification of the lower side.

Next, the magnification of the upper side of the projected image is calculated. In this calculation, the distance A ₃ is expressed by the following equation as described above.

A ₃ = A / cos (φ ₁ + φ / 2)
At the point S, it is considered that the image has been enlarged by A ₃ / X times than when projected in front. Therefore, A ₃ / X is the magnification of the upper side.

Then, the output image is reduced based on the calculated magnification. The “original image” shown in FIG. 12B is an image obtained when projected from the front onto the screen as shown in FIG. The “projected image” is an image projected without correction in the case of FIG. Accordingly, the corrected image to be output from the partial video display device 11 is obtained by multiplying QR of FIG. 12B by A ₂ / X and reducing PS by A ₃ / X. Such correction is performed for all sections of the partial video.

  The above description is a case where the projection screen has no boundary. When the projection screen is a multi-surface projection screen having a boundary, the correction formula used in the data correction unit 11B is changed according to the shape of the projection surface to be projected with the boundary as a reference. The correction formula corresponding to each projection plane in the multi-plane projection screen can be basically determined based on the formula described above.

Further, in order to show the user an uncomfortable image, a cut area setting unit 15 may be provided as in the second embodiment of the present invention shown in FIG. In the embodiment of FIG. 13, the basic structure is the same as that of the embodiment shown in FIG. 1 except that a cut area setting unit 15 is added. Therefore, description of common parts is omitted. Cutoff area setting unit 15, based on the output of the head posture direction detection section 5A (profile orientation), image data obtained through standard lens for the relevant type of lens used in obtaining the image data I ₀ A cut-out area of partial video data i cut out from the video data is set so that conversion to video data is unnecessary. FIG. 14 is a diagram showing a flow of processing up to projection including correction in the case of using video data I ₀ obtained using a fisheye lens using the embodiment of FIG. In the present embodiment, the clipping region setting unit 15 can eliminate the need for conversion to video data obtained through a standard lens that is required according to the fish lens used when acquiring the video data I ₀ . A cut area for cutting out the partial video data i from the fisheye video data I ₀ is set (the position and shape of the cut out partial video data i are determined). The partial video display unit 11A cuts the partial video data i in the cut region set (determined) by the cut region setting unit 15 from the fisheye video data I ₀ stored in the video data storage unit 9, and the data correction unit 11B. Enter. The data correction unit 11B corrects the partial video data i for projection so that the video obtained by projecting the partial video data i on the projection screen 3 becomes a video with distortion corrected when viewed from the user. 'Is generated and output to the partial video display unit 11A. In this embodiment, by providing the cutoff area setting unit 15, the conversion step of converting the image data I ₀ obtained through the fisheye lens to the image data obtained through standard lens becomes unnecessary. As a result, according to the present embodiment, it is possible to reduce the number of times of data conversion as compared with the first embodiment, and thus it is possible to prevent deterioration in image quality.

  The basic data for determining the shape and size of the cut-out area may be obtained by a preliminary test according to the shape of the projection screen to be used and the type of lens used when acquiring the image data.

  Even in the present embodiment, when a multi-surface projection screen configured such that two or more projection surfaces are combined to form a boundary portion is used as the projection screen, the viewing area shape determination unit 5C includes the boundary portion. The basic shape and size of the viewing area are changed based on the above.

  According to each of the above embodiments, the headphone HP is used in which the user actually moves his / her neck and displays an image in a desired direction. If a projector and a sensor are mounted on the headphone HP, an interface that can be used by an intuitive operation of moving the neck can be provided. If the system is mounted on the headphone HP, the system of the present invention can be used anywhere such as the wall of the room, and the convenience that it can be moved to every corner such as the feet can be obtained. For this reason, the entire stage is not visible, but it is possible to know information on the movements of the players around by the change in the sound heard from the headphones (the sound approaches or moves away). In addition, a zoom function may be installed for a user who wants to see a detailed part such as the movement of the performer's hand, and the performance of the user can be supported.

  In the above embodiment, the projection screen 3 is a three-surface projection screen, but it is needless to say that other shapes of projection screens may be used. In the above embodiment, a fish-eye lens is used to obtain video data. However, the present invention is naturally applicable to a case where video data is acquired using another lens. The content of the video shot by the camera may be anything. If a surveillance camera attached to a child is used as a camera equipped with a fisheye lens, the image can be displayed only in the direction that the supervisor who monitors the child intuitively sees, so that the surveillance using the intuition is maximized. It becomes possible.

  According to the present invention, the partial video is displayed in the viewing area viewed by the user on the video display unit, and no video is displayed on the other video display units. Therefore, it is possible to display an image in a direction intuitively viewed by the user without using a head-mounted display and without requiring a key operation. Further, according to the present invention, the power consumption required for video display can be greatly reduced as compared with the prior art.

1 Video display system 3 Projection screen (video display unit)
DESCRIPTION OF SYMBOLS 5 Watching area specific | specification part 7 Boundary part setting part 9 Video data storage part 11 Partial video display apparatus 13 Sound image generation apparatus 15 Cut area setting part

Claims (6)

A video display unit arranged at a position where a user can move his / her body and head, and
A viewing area identifying unit that identifies a viewing area of the video display unit that the user is viewing;
A video data storage unit including video data necessary for displaying video on the entire video display unit;
Based on the output of the viewing area specifying unit, the video display unit acquires partial video data about the partial video to be displayed in the viewing area specified by the viewing area specifying unit, and the video display unit A partial video display device for displaying the partial video in the viewing area above ,
The video display unit is a projection screen,
The viewing area specifying unit is mounted on the user's head and has a gyro function that detects the posture of the head, and the head posture direction detection unit outputs the head posture direction detection unit. A viewing area determination unit that determines the viewing area from a posture orientation,
The partial video display device includes a projector that is mounted on the user's head and projects the partial video to the viewing area.
A cutout for setting a cutout area of the partial video data cut out from the video data so that conversion to video data obtained through a standard lens required according to the type of lens used when obtaining the video data can be made unnecessary An area setting unit,
The partial video display device corrects the partial video data according to the shape of the projection screen so that the video projected on the projection screen becomes a video with distortion corrected when viewed from the user. A data correction unit;
The video data was taken using a camera equipped with a fisheye lens,
The projection screen is a multi-surface projection screen configured such that two or more projection surfaces are combined to form a boundary portion.
The video display system, wherein the cut area setting unit changes a shape and a size of the area of the partial video data cut from the video data with reference to the boundary . The video display system according to claim 1 , wherein the data correction unit changes a correction formula to be used on the basis of the boundary portion. The partial video display device has a projection angle change function capable of changing the projection angle according to a command,
The data correction unit issues a command to change the projection angle so that a partial image is projected onto an area adjacent to the boundary part in the projection screen when the boundary part is included in the viewing area. The video display system according to claim 1 , wherein the video display system is configured to output. An acoustic data storage unit comprising acoustic data for letting the user hear a sound image synchronized with the video displayed on the entire video display unit;
The video display system according to claim 1, further comprising: a sound image generation unit that provides the user with the sound image synchronized with the video. An acoustic data storage unit comprising acoustic data for letting the user hear a sound image synchronized with the video displayed on the entire video display unit;
A sound image generator for providing the user with the sound image synchronized with the video;
The video display system according to claim 1 , wherein a speaker unit of the sound image generation unit is a headphone attached to the head of the user, and the projector is attached to the headphone. The video display system according to claim 1 , wherein the camera is a surveillance camera attached to a child.