JP2005092363A - Image generation device and image generation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image generation device that always generates an undistorted image when a viewer watches the image displayed on the display screen of a television set or the like. <P>SOLUTION: The image generation device 1 is provided with; a viewer position detection means 11 which detects the position of the viewer; a virtual space setting means 12 which sets the viewer's position and the screen area of the display screen, as a virtual viewer's position and a virtual screen area, on a virtual space; a virtual plane setting means 13 which makes the virtual viewer's position a virtual viewpoint, makes an area surrounded by the intersection points of straight lines, which go through the virtual viewpoint and the apexes of the virtual screen area, and virtual plane as a conversion virtual screen area, and sets on the virtual plane a rectangular area including the conversion virtual screen area as a generation image area; an image synthesizing means 15 which synthesizes a displaying image on the conversion virtual screen area; and an image conversion means 16 which converts an image in the conversion virtual screen area in association with the generation image area. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for generating an image to be displayed on a display screen, and more particularly to an image generation apparatus and an image generation program for generating an image to be displayed on a display screen according to the position of a viewer with respect to the display screen.

  Conventionally, video displayed on a display screen such as a television receiver or a computer display is displayed on the premise that the viewer sees the screen facing the screen. Therefore, when the viewer is not facing the display screen, the video on the display screen appears distorted when viewed from the viewer.

  Further, when a projection from a position that does not face the screen, such as tilt projection, is performed from a video projection device such as a projector, trapezoidal distortion may occur in the projected image. Therefore, the projected image is obtained by adding distortion that cancels this trapezoidal distortion to the image projected according to the projection position and the position of the projection surface so that the image does not generate distortion on the screen when projecting. A video projection apparatus having a function of displaying without causing distortion is disclosed (see Patent Document 1).

Furthermore, a virtual reality system has been proposed in which an image to be projected is updated in real time according to the position of the viewer with respect to the screen, and an image that should always be seen from the position of the viewer is projected (see Non-Patent Document 1). This system has a three-dimensional structure that should be visible from the viewpoint and line of sight by detecting the viewer's position by the viewer wearing stereoscopic glasses connected by cables from the ceiling into a room surrounded by a screen on all sides. The image is projected onto the screen.
JP 2003-78842 A (paragraphs 0009 to 0030, FIG. 3, etc.) Shunji Imon, “CAVE Study Group Report”, [online], Department of Information Engineering, Saitama Institute of Technology, [July 18, 2003 search], Internet <URL: http://www.viz-journal.kgt.co.jp /topics/cave.html>

  However, the video displayed on the display screen of a normal television receiver, computer display, etc., and the video projection device of Patent Document 1 are based on the premise that the viewer sees the video displayed from the front, If the viewer views the video from an oblique direction, the viewer must see a distorted video. For this reason, there arises a problem that it is difficult for the viewer to recognize the content of the video, or it cannot be correctly recognized.

  Further, in the system of Non-Patent Document 1, a stereoscopic video is projected onto a plurality of screens in accordance with the viewer's viewpoint. To realize this system, large-scale equipment dedicated to the system is required. In addition, in order to generate an image to be projected, three-dimensional information of a subject to be projected as an image is required, and an image of each screen is generated based on the information. There is a problem that it is not possible to view the video and the like with this system.

  The present invention has been made to solve the above-mentioned problems of the prior art, and is viewed from the viewer's position when displaying on a display screen using a general display device such as a television receiver or a computer display. To provide an image generation apparatus and an image generation program that can always generate an image without distortion from an existing image such as an image distributed via a broadcast wave or a network, an image captured in real time, or the like. Objective.

  In order to solve the above problem, the image generating apparatus according to claim 1, when displaying the display image on the display screen, the viewer displays the display image at a position deviated from the directly-facing position with respect to the display screen. An image generation device that generates a reverse distortion-added image in which reverse distortion is added to the display screen in order to remove distortion of the display image that occurs when viewed, the viewer position detecting means, and a virtual space setting Means, virtual plane setting means, image composition means, and image conversion means.

  According to this configuration, the image generation apparatus detects the position of the viewer by the viewer position detection unit, and the viewer position detected by the viewer position detection unit by the virtual space setting unit and the screen of the display screen The area is set as a virtual viewer position and a virtual screen area in the virtual space. Then, the virtual plane setting means sets a virtual plane having the virtual viewer position set by the virtual space setting means as a virtual viewpoint and having a half line passing through the center of the virtual screen area as a normal line from this virtual viewpoint. Furthermore, an area surrounded by the intersection of the virtual viewpoint and a straight line passing through each vertex of the virtual screen area and the virtual plane is set as a converted virtual screen area on the virtual plane, and includes the converted virtual screen area. A rectangular area having the aspect ratio of the virtual screen area is set as a generated image area on the virtual plane.

  Then, the display image is synthesized by the image synthesis unit so that at least a part of the display image is included in the converted virtual screen area set by the virtual plane setting unit, and the display image synthesized by the image synthesis unit by the image conversion unit The image in the converted virtual screen area including at least a part of the image is converted in association with the generated image area to generate an inverse distortion added image. By this conversion, reverse distortion is added to the display image. The reverse distortion added to the display image is a distortion that corrects the distortion of the display image that occurs when the viewer views the display image at a position that is deviated from the normal position with respect to the display screen. The reverse distortion-added image to which reverse distortion is added becomes an image without distortion when viewed from the viewer when displayed on the display screen. As a result, even when the viewer sees the display screen at a position deviated from the front-facing position, a reverse distortion-added image that is not distorted by the viewer when displayed on the display screen is generated from the display image. can do.

  Here, the display screen is a presentation unit that presents a reverse distortion-added image. For example, when displaying a reverse distortion-added image on a display, the display screen corresponds to the display screen, and the reverse-distortion-added image is displayed on a projector. In the case of projecting onto a screen by a video projection device such as the above, the display screen corresponds to a region where a video is projected by this video projection device.

  As a method for detecting the viewer by the viewer position detecting means, for example, the viewer is imaged by two or more cameras, and the viewer's face is detected from the captured image by subject detection technology such as face detection technology. The position of the viewer may be specified by detection, or the viewer may be detected by attaching a sensor for position detection to the viewer. Furthermore, the position of the viewer represents the position of the viewer's viewpoint (eyes). For example, when the viewer's face is detected from a captured image captured by a camera and the position of the viewer's eyes is specified. The position of the eye may be specified by adding an offset to the eye position to the detected face position, but the method of detecting the viewer of the present invention is not limited to these methods, Any device that can detect the position of the viewer's viewpoint (eyes) may be used.

  Furthermore, the image synthesis means may synthesize the display image so that at least part of the display image is included in the converted virtual screen area. For example, the entire display image is included in the converted virtual screen area. The images may be combined so that they fit, or the display image may be combined with the entire generated image area. Here, when the entire display image is synthesized so as to be within the converted virtual screen area, the image in the converted virtual screen area is converted in association with the generated image area, and then the reverse distortion added image including the entire input display image is included. Is generated. Further, when the display image is combined with the entire generated image area, the input display image is combined on the entire surface after conversion, and a reverse distortion added image having no blank portion is generated.

  An image generation apparatus according to claim 2 is the image generation apparatus according to claim 1, wherein the three-dimensional image data storage means stores three-dimensional image data for generating the display image, and the three-dimensional image data. A display image generation unit that generates the display image corresponding to the position of the viewer based on image data, and the image synthesis unit synthesizes the display image generated by the display image generation unit; did.

  According to such a configuration, the image generation apparatus stores the three-dimensional image data for generating the display image by the three-dimensional image data storage unit. Then, a display image corresponding to the viewer's position is generated by the display image generating means based on the three-dimensional image data, and this display image is synthesized by the image synthesizing means. The content of the additional image can be changed.

  The image generation program according to claim 3 is generated when a viewer views the display image at a position deviated from a directly-facing position with respect to the display screen when the display image is displayed on the display screen. In order to generate a reverse distortion-added image in which reverse distortion is added to the display screen in order to remove distortion of the display image, a viewer position detecting means for detecting the position of the viewer, Virtual space setting means for setting the position of the viewer and the screen area of the display screen detected by the viewer position detection means as a virtual viewer position and a virtual screen area on the virtual space, the virtual space setting means The set virtual viewer position is a virtual viewpoint, and the virtual viewpoint passes through the virtual viewpoint and each vertex of the virtual screen area on a virtual plane whose normal is a half line passing through the center of the virtual screen area. Virtual plane setting means for setting a region surrounded by an intersection of a line and the virtual plane as a conversion virtual screen region, and setting a rectangular region including the conversion virtual screen region and having an aspect ratio of the virtual screen region as a generation image region An image composition means for composing the display image so that at least a part of the display image is included in the converted virtual screen area set by the virtual plane setting means, and the display image synthesized by the image composition means. The image in the converted virtual screen area including at least a part thereof is converted in association with the generated image area, and functions as an image converting means for generating the inverse distortion added image.

  According to such a configuration, the image generation program detects the position of the viewer by the viewer position detection unit, and the viewer position and the screen of the display screen detected by the viewer position detection unit by the virtual space setting unit. The area is set as a virtual viewer position and a virtual screen area in the virtual space. Then, the virtual plane setting means sets a virtual plane having the virtual viewer position set by the virtual space setting means as a virtual viewpoint and having a half line passing through the center of the virtual screen area as a normal line from this virtual viewpoint. Furthermore, an area surrounded by the intersection of the virtual viewpoint and a straight line passing through each vertex of the virtual screen area and the virtual plane is set as a converted virtual screen area on the virtual plane, and includes a converted virtual screen area. A rectangular area having the aspect ratio of the screen area is set as a generated image area on the virtual plane.

  Then, the display image is synthesized by the image synthesis unit so that at least part of the display image is included in the converted virtual screen area set by the virtual plane setting unit, and the image synthesized by the image synthesis unit is displayed. An image in the converted virtual screen area including at least a part of the image is converted in association with the generated image area to generate a reverse distortion added image. As a result, even when the viewer sees the display screen at a position deviated from the normal position, a reverse distortion-added image that is not distorted when viewed from the viewer when displayed on the display screen is displayed from the display image. Can be generated.

  The image generation apparatus and the image generation program according to the present invention have the following excellent effects.

  According to the first or third aspect of the present invention, the position of the viewer is detected, and distortion generated when viewed from the position of the viewer when displayed on the display screen is removed based on the position. Therefore, an image to which reverse distortion is added can be generated from the display image. As a result, when the generated reverse distortion added image is displayed on the display screen, the viewer correctly recognizes the content even when the viewer views the image at a position deviated from the normal position with respect to the display screen. become able to. In addition, a reverse distortion-added image can be generated from an existing image, and a dedicated display device is not required. For example, video distributed via broadcast waves or the like is displayed on the screen of a television receiver. The present invention can be applied to various images (videos) displayed on the display screen, such as when the video is taken in real time like a videophone or when the video is displayed on the display screen.

  According to the second aspect of the present invention, the image generating device generates and synthesizes the display image based on the position of the viewer, and therefore changes the content of the reverse distortion added image according to the position of the viewer. Can do. This makes it possible to generate an image that can provide information according to the position of the viewer. For example, when the position is changed so that the image of the portion that the viewer wants to view is displayed, It is possible to display an image of a desired portion on the display screen.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, two embodiments will be described.

[Configuration of Image Generating Device (First Embodiment)]
FIG. 1 is a block diagram showing a configuration of an image generation apparatus according to the first embodiment of the present invention. The image generation apparatus 1 receives a video such as a TV program, and reverse-distortion-added images such that each frame of the input video becomes an image without distortion when viewed from the viewer's position when displayed on the display screen. Is generated. Here, the image generation apparatus 1 includes a viewer position detection unit 11, a virtual space setting unit 12, a virtual plane setting unit 13, an input unit 14, an image composition unit 15, an image conversion unit 16, and a display unit. 17. Here, the image generation apparatus 1 is connected to the outside with two position detection cameras 2 and 2 for detecting the position of the viewer and a display apparatus 4 for displaying images and the like.

  The viewer position detection means 11 receives captured images from the position detection cameras 2 and 2 and detects the position of the viewer's eyes (viewpoint). The viewer position detection unit 11 includes a viewer detection unit 11a and a viewer position calculation unit 11b.

  The viewer detection unit 11 a detects the viewer's face from the captured images of the position detection cameras 2 and 2. Here, the viewer is imaged by the two position detection cameras 2 and 2, and the viewer's face is detected from each captured image. The face detection in the viewer detection unit 11a can be realized using a known face detection technique. Further, the method for detecting the position of the viewer from the captured images captured by the position detection cameras 2 and 2 is not limited to face detection, and any detection method for detecting a person from the captured images may be used. Furthermore, here, the position of the viewer is detected from the captured images captured by the position detection cameras 2 and 2. For example, a signal for detecting the position is attached to the viewer and a signal generated from the sensor is detected. Based on this, the viewer detection unit 11a may detect the viewer.

  The viewer position calculation unit 11b calculates the position of the viewer's eyes (viewpoint) from the detection result detected by the viewer detection unit 11a. The viewer position calculation unit 11b receives from the viewer detection unit 11a the detection result of detecting the viewer's face in the captured images captured by the position detection cameras 2 and 2, and the viewer's eyes (from the detection result) The position of the (viewpoint) is calculated. Here, the configuration includes two position detection cameras 2, but the image generation apparatus 1 only needs to include two or more position detection cameras in order to calculate the three-dimensional position of the viewer's eyes. . The calculation result calculated here is output to the virtual space setting unit 12.

  The virtual space setting means 12 is based on the viewer eye position calculation result input from the viewer position calculator 11b and the position and size of the display screen of the display device 4 registered in advance. The position of the eyes and the area of the display screen of the display device 4 are set on the virtual space. The virtual space setting unit 12 includes a virtual screen area setting unit 12a and a virtual viewer position setting unit 12b.

  The virtual screen area setting unit 12a sets the coordinate axis of the virtual space based on the position and size of the display screen of the display device 4 registered in advance, and places the display screen area of the display device 4 on this virtual space. This is set as a virtual screen area. Then, the virtual screen area setting unit 12 a outputs virtual screen area information representing the virtual screen area in the virtual space to the virtual plane setting unit 13.

  The virtual viewer position setting unit 12b is based on the viewer's eye position calculation result input from the viewer position calculation unit 11b and the positions of the position detection cameras 2 and 2 registered in advance. The eye position is set as a virtual viewer position on the virtual space set by the virtual screen area setting unit 12a. Then, the virtual viewer position setting unit 12 b outputs virtual viewer position information representing the virtual viewer position in the virtual space to the virtual plane setting unit 13.

  Here, an example in which a virtual viewer position and a virtual screen area are set in the virtual space will be described with reference to FIG. 2 (refer to FIG. 1 as appropriate). 2A is an explanatory diagram showing a state in which the display device 4, the position detection cameras 2 and 2, and the viewer V are arranged in the real space, and FIG. 2B is a virtual viewer position in the virtual space. It is explanatory drawing which showed the example which sets a virtual screen area | region.

  As shown in FIG. 2A, the display device 4, the position detection cameras 2 and 2, and the viewer V are arranged in the real space. Then, the position of the viewer calculated based on the captured images captured by the position detection cameras 2 and 2 is input to the virtual space setting unit 12 from the viewer position calculation unit 11b. Then, the virtual screen area setting unit 12a sets the coordinate axes of the virtual space shown in FIG. 2B based on the position of the display screen of the display device 4 registered in advance. That is, the intersection of the perpendicular to the ground plane from the center point Po of the display screen of the display device 4 and the ground plane is defined as the origin O of the spatial coordinates, the X axis passing through the origin O and parallel to the longitudinal direction of the display screen, The Y axis is set in the vertical direction from the origin O, and the Z axis is set in the direction orthogonal to the X axis and the Y axis.

  Then, based on the position and size of the display screen of the display device 4 registered in advance, a virtual screen area Dv is set in this virtual space. Here, it is registered in advance that the height of the display screen is h from the ground surface of the center Po of the display screen, and the width and height of the display screen are 2w and 2t, respectively, as the size of the display screen. Suppose. Therefore, the virtual screen area setting unit 12a has the coordinates Po of the center Po of the virtual screen area Dv Po = (0, h, 0) and the coordinates of the vertices Pa, Pb, Pc, Pb of the virtual screen area Dv Pa = (−. A virtual screen region Dv is set such that w, h + t, 0), Pb = (− w, ht, 0), Pc = (w, ht, 0), Pd = (w, h + t, 0). The coordinates of the center Po of the virtual screen area Dv and the coordinates of the vertices Pa, Pb, Pc and Pb are output to the virtual plane setting means 13 as virtual screen area information.

  In addition, the virtual viewer position setting unit 12b performs viewing based on the calculation result of the viewer's eye position input from the viewer position calculation unit 11b and the positions of the position detection cameras 2 and 2 registered in advance. The position of the viewer's eyes is set as the virtual viewer position on the virtual space shown in FIG. Here, the coordinates of the viewer position Pp are determined based on the position detection cameras 2 and 2, the position of the display screen, and the calculation result of the viewer's eye position input from the viewer position calculation unit 11 b. It is assumed that Pp = (xp, yp, zp) is calculated. Therefore, the virtual viewer position setting unit 12b outputs the coordinates of the viewer position Pp to the virtual plane setting unit 13 as virtual viewer position information.

  Returning to FIG. 1, the description will be continued. The virtual plane setting unit 13 sets the virtual viewer position as a virtual viewpoint based on the virtual viewer position information and the virtual screen area information input from the virtual space setting unit 12, and the center of the virtual screen area from the virtual viewpoint. A virtual plane whose normal is a half line passing through is set, and a converted virtual screen area and a generated image area are set on the virtual plane. The virtual plane setting unit 13 includes a conversion virtual screen area setting unit 13a and a generated image area setting unit 13b.

  The converted virtual screen area setting unit 13a passes from the virtual viewpoint (virtual viewer position) to the center of the virtual screen area based on the virtual viewer position information and the virtual screen area information input from the virtual space setting means 12. A virtual plane whose normal is a half line is set, and a conversion virtual screen area is set on the virtual plane. This converted virtual screen area is an area surrounded by the intersection of a straight line passing through each vertex of the virtual screen area and the virtual viewpoint and the virtual plane.

  The generated image area setting unit 13b includes a converted virtual screen area set by the converted virtual screen area setting unit 13a, and sets a rectangular area having the same aspect ratio as the virtual screen area as a generated image area on the virtual plane. Is. Then, the generated image area setting unit 13b sets coordinate axes on the virtual plane, and converts the generated image area and the converted virtual screen area set by the converted virtual screen area setting unit 13a into coordinates on the virtual plane. Then, the converted virtual screen area information indicating the converted virtual screen area on the virtual plane and the generated image area information indicating the generated image area are output to the image composition means 15.

  Here, an example in which the converted virtual screen area and the generated image area are set on the virtual plane will be described with reference to FIG. 3 (refer to FIG. 1 as appropriate). FIG. 3A is an explanatory diagram showing an example of setting a converted virtual screen area on a virtual plane, and FIG. 3B is an explanatory diagram showing an example of setting a generated image area on a virtual plane.

  First, virtual viewer position information and virtual screen area information are input from the virtual space setting means 12 to the converted virtual screen area setting unit 13a. Then, as shown in FIG. 3A, the conversion virtual screen area setting unit 13a sets the viewer position indicated by the virtual viewer position information as the virtual viewpoint Pp, and the center Po of the virtual screen area Dv from the virtual viewpoint Pp. A virtual plane with the half line passing through as a normal is set. For example, the virtual viewer position information includes Pp = (xp, yp, zp) as the coordinates of the virtual viewpoint Pp, and the virtual screen area information includes Po = (0, h, 0) as the coordinates of the center Po of the virtual screen area Dv. In this case, the conversion virtual screen area setting unit 13a sets a half line passing through the center Po of the virtual screen area from the virtual viewpoint Pp, and sets a virtual plane having the half line as a normal line. That is, a vector v = (− xp, h−yp, −zp) from the virtual viewpoint Pp to the center Po of the virtual screen area Dv is calculated as a vector having the same inclination as the half line, and is orthogonal to the vector v. A virtual plane S is set. In FIG. 3A, the virtual plane S is set between the virtual viewpoint Pp and the center Po of the virtual screen area Dv. However, this virtual plane defines the center Po of the virtual screen area Dv when viewed from the virtual viewpoint Pp. It may be set to a position that is separated, or may be set to a position that includes the center Po of the virtual screen area Dv.

  Furthermore, the conversion virtual screen area setting unit 13a sets a conversion virtual screen area on the virtual plane. That is, as shown in FIG. 3A, the converted virtual screen area setting unit 13a has straight lines La, Lb, Lc, Ld passing through the vertices Pa, Pb, Pc, Pd and the virtual viewpoint Pp of the virtual screen area Dv. A region surrounded by the intersections Pa ′, Pb ′, Pc ′, and Pd ′ between the virtual plane S and the virtual plane S is set as a converted virtual screen region Dv ′.

  Then, the generated image region setting unit 13b sets a generated image region including the converted virtual screen region set by the converted virtual screen region setting unit 13a on the virtual plane S. That is, as shown in FIG. 3B, the generated image region setting unit 13b includes the converted virtual screen region Dv ′ set by the converted virtual screen region setting unit 13a on the virtual plane S, and is the same as the virtual screen region Dv. A rectangular area having an aspect ratio is set as the generated image area Iv.

  Further, the generated image area setting unit 13b sets coordinate axes on the virtual plane, and converts the generated image area and the converted virtual screen area into coordinates on the virtual plane. For example, as shown in FIG. 3B, the generated image area setting unit 13b has coordinates spa ″, spb ″, and vertices Pa ″, Pb ″, Pc ″, and Pd ″ of the generated image area Iv on the virtual plane S. The virtual plane S is set such that spc ″, spd ″ are spa ″ = (0, 1), spb ″ = (0, 0), spc ″ = (1, 0), spd ″ = (1, 1). Coordinate axes sx and sy are set, and the vertices Pa ′, Pb ′, Pc ′, and Pd ′ of the converted virtual screen area Dv ′ are converted into coordinates spa ′, spb ′, spc ′, and spd ′ on the virtual plane. Then, the generated image area setting unit 13b uses the coordinates on the virtual plane S of the vertices spa ′, spb ′, spc ′, and spd ′ of the converted virtual screen area Dv ′ as converted virtual screen area information, and the vertices of the generated image area Iv The coordinates on the virtual plane S of spa ″, spb ″, spc ″, spd ″ are output to the image composition means 15 as generated image area information.

  Returning to FIG. 1, the description will be continued. The input means 14 is for inputting a display image. The display image input by the input unit 14 is converted by, for example, an image conversion unit 16 to be described later, and distortion generated when viewed at a position deviated from the directly-facing position with respect to the display screen is removed. is there. The input display image is output to the image composition means 15.

  The image composition unit 15 sets an image composition region, which is a region for compositing a display image, on the virtual plane set by the virtual plane setting unit 13 and composes the display image input from the input unit 14 in this image composition region. To do. The image composition unit 15 includes an image composition area setting unit 15a and an image composition execution unit 15b.

  The image composition area setting unit 15 a sets an image composition area, which is an area for synthesizing the display image input from the input unit 14, on the virtual plane set by the virtual plane setting unit 13.

  The image composition execution unit 15b synthesizes the display image input from the input unit 14 within the image composition region set by the image composition region setting unit 15a. At this time, the image composition execution unit 15b synthesizes the display image so that at least a part of the display image is included in the converted virtual screen area. Then, the image on the generated image area obtained by synthesizing the display image by the image synthesis execution unit 15 b is output to the image conversion means 16.

  Here, in order for the image composition execution unit 15b to compose the display image so that at least a part of the display image is included in the conversion virtual screen region, the image composition region setting unit 15a includes the image composition region and the conversion virtual screen. It is necessary to set the image composition area so that at least a part of the area overlaps. Then, an image included in the reverse distortion-added image generated by converting the image in the overlapping range between the image composition area set by the image composition area setting unit 15a and the converted virtual screen area by the image conversion means 16 described later. It becomes. Therefore, for example, when the image composition area setting unit 15a sets the image composition area so that the entire image composition area is included in the converted virtual screen area, the inverse distortion generated by the conversion by the image conversion means 16 is generated. The additional image can include the entire combined display image. Further, when the image composition area setting unit 15a sets the image composition area so that a part of the image composition area is included in the converted virtual screen area, the inverse distortion addition generated by the image conversion means 16 is generated. The image can include only an image in a range where the image synthesis area and the conversion virtual screen area overlap among the synthesized display images.

  Here, with reference to FIG. 4 (refer to FIG. 1 as appropriate), an example in which the image composition unit 15 sets an image composition area on a virtual plane and composes a display image will be described. FIG. 4 is an explanatory diagram showing an example in which an image synthesis area is set on a virtual plane and a display image is synthesized.

  The image composition area setting unit 15a sets an image composition area on the virtual plane set by the virtual plane setting unit 13 based on the converted virtual screen area information and the generated image area information input from the generated image area setting unit 13b. Set. For example, as illustrated in FIG. 4, the generated image area setting unit 13 b causes the converted virtual screen area Dv ′ having vertices spa ′, spb ′, spc ′, and spd ′ on the virtual plane S and the vertices spa ″, spb ″. , Spc ″, spd ″ and the generated image area Iv are set. Then, the image composition area setting unit 15a sets an image composition area Di surrounded by the vertices spe, spf, spg, and sph in the converted virtual screen area Dv ′.

  Further, the image composition execution unit 15b synthesizes the display image input from the input unit 14 in the image composition area. In FIG. 4, assuming that a display image in which the characters “correct aspect” are drawn is input from the input unit 14, the display image is combined with the image combining region Di. Then, the image composition execution unit 15 b outputs the image in the generated image area Iv to the image conversion unit 16.

  Returning to FIG. 1, the description will be continued. The image conversion means 16 converts the image in the converted virtual screen area from the image on the generated image area input from the image composition execution unit 15b in association with the generated image area, and generates a reverse distortion added image. is there. Here, the image conversion means 16 matches the image in the converted virtual screen area included in the image on the generated image area input from the image composition execution unit 15b with the converted virtual screen area to the generated image area. Deform (convert). Then, this converted image is output to the display means 17 as a reverse distortion added image.

  Here, referring to FIG. 4 and FIG. 5 (refer to FIG. 1 as appropriate), the image in the converted virtual screen area is associated with the generated image area from the image on the generated image area input from the image composition execution unit 15b. An example of generating an inverse distortion-added image will be described. FIG. 5A is an explanatory view showing a reverse distortion added image obtained by converting the image in the converted virtual screen area of FIG. 4 in association with the generated image area, and FIG. 5B is a reverse distortion added image of FIG. It is explanatory drawing which showed the reverse distortion addition image when displaying on a display screen and seeing from the position of a viewer.

  When an image in the generated image area Iv as shown in FIG. 4 is input from the image composition execution unit 15b, the image conversion means 16 associates the image in the converted virtual screen area Dv ′ with the generated image area. Conversion is performed to generate an inverse distortion added image. That is, the image conversion means 16 uses the vertices spa ′, spb ′, spc ′, spd ′ of the converted virtual screen area Dv ′ of FIG. 4 as the vertices spa ″, spb ″, spc ″, spd ″ of the generated image area Iv, respectively. Is performed so as to deform the image in the converted virtual screen area Dv ′. The conversion of the image in the conversion virtual screen area Dv ′ can be realized by an image conversion technique such as perspective conversion.

  By this conversion, the entire image in the converted virtual screen area Dv ′ is deformed, and the image in the converted virtual screen area Dv ′ is deformed corresponding to the position of the viewer as shown in FIG. This image, that is, an image to which reverse distortion is added, is displayed on the display screen and viewed from the viewer's position as shown in FIG.

  Returning to FIG. 1, the description will be continued. The display means 17 analyzes the reverse distortion added image input from the image conversion means 16, converts it into a displayable output format, and outputs it to the display device 4.

  The configuration of the image generation apparatus 1 according to the present invention has been described above, but the present invention is not limited to this. For example, here, the case where the reverse distortion added image generated by the image generation device 1 is displayed by the display device 4 connected to the outside has been described. However, instead of the display device 4, a video projection device is connected to the outside. Thus, the reverse distortion-added image generated by the image generation device 1 may be projected from the video projection device and displayed on a screen or the like. In this case, the image generating apparatus 1 registers in advance the size and position of the area on the screen where the image is projected, and the virtual space setting unit 12 virtually determines the position of the viewer and the area where the image is projected. This can be realized by setting the virtual viewer position and the virtual screen area on the space, and outputting the reverse distortion added image converted by the image conversion means 16 to the video projection device.

  The image generation apparatus 1 can be realized by causing a general computer to execute a program and operating an arithmetic device or a storage device in the computer. This program (image generation program) can be distributed via a communication line, or can be recorded and distributed on a recording medium such as a CD-ROM.

[Operation of Image Generating Device (First Embodiment)]
Next, the operation of the image generation apparatus 1 according to the present invention will be described with reference to FIG. 6 (refer to FIG. 1 as appropriate). FIG. 6 shows an operation in which the image generation apparatus generates an inverse distortion-added image from which image distortion is removed, which occurs when the viewer views the display image at a position deviated from the directly-facing position with respect to the display screen. It is the shown flowchart.

  The image generation device 1 registers the position and size of the display screen of the display device 4 and the positions of the position detection cameras 2 and 2 (step S11). Here, the position and size of the display screen are, for example, the center position of the display screen (height from the ground surface), the width and height of the display screen, and the height from the ground surface at the lower left edge of the display screen. It may be expressed by the length of the vertical and horizontal sides of the screen, or a reference coordinate axis may be provided and expressed by the coordinates. Further, the position of the position detection cameras 2 and 2 may be any position that can specify the position from the display screen.

  Further, the image generating apparatus 1 inputs a captured image obtained by capturing the viewer from the position detection cameras 2 and 2 (step S12).

(Viewer position detection step)
Then, the viewer detection unit 11a of the viewer position detection unit 11 detects the viewer's face from each captured image input from the position detection cameras 2 and 2 (step S13). Further, the viewer position calculation unit 11b of the viewer position detection unit 11 calculates the position of the viewer's eyes (viewpoint) from the detection result detected by the viewer detection unit 11a (step S14).

(Virtual space setting step)
Then, the virtual screen area setting unit 12a of the virtual space setting unit 12 sets the coordinate axis of the virtual space (step S15). Further, the virtual screen area setting unit 12a sets a virtual screen area on the virtual space based on the position and size of the display screen registered in step S11 (step S16). Further, the virtual viewer position setting unit 12b of the virtual space setting unit 12 calculates the viewer's eye position calculated by the viewer position calculation unit 11b in step S14 and the position detection camera 2 registered in advance. , 2 is set as the virtual viewer position in the virtual space based on the positions of the viewers (step S17).

(Virtual plane setting step)
Then, the conversion virtual screen area setting unit 13a of the virtual plane setting means 13 and the position of the center of the virtual screen area set in step S16 (coordinates in the virtual space) and the virtual viewer position set in step S17 ( On the basis of the coordinates in the virtual space), a virtual plane having a normal line from the virtual viewpoint (virtual viewer position) passing through the center of the virtual screen area is set (step S18). Furthermore, the conversion virtual screen area setting unit 13a calculates the coordinates of the intersections between the straight line passing through each vertex of the virtual image area set in step S16 and the virtual viewpoint and the virtual plane, and uses these intersections on the virtual plane. The enclosed area is set as the conversion virtual screen area (step S19). Further, the generated image area setting unit 13b of the virtual plane setting unit 13 includes a converted virtual screen area set by the converted virtual screen area setting unit 13a in step S19, and a rectangular area having the same aspect ratio as the virtual screen area. The generated image area is set on the virtual plane (step S20). Further, the generated image area setting unit 13b sets coordinate axes on the virtual plane, and converts the generated image area and the converted virtual screen area into coordinates on the virtual plane (step S21).

  And the input means 14 inputs the display image for displaying on a display screen (step S22).

(Image composition step)
The image composition area setting unit 15a of the image composition means 15 sets an image composition area on the virtual plane set by the virtual plane setting means 13 in step S18 (step S23). Then, the image composition execution unit 15b of the image composition unit 15 composes the display image input in step S22 with the image composition area on the virtual plane (step S24).

(Image conversion step)
Further, the image conversion means 16 converts the image in the converted virtual screen area in association with the generated image area from the image on the generated image area obtained by synthesizing the display image in step S24, and generates a reverse distortion added image. (Step S25).

  Then, the display unit 17 displays the reverse distortion added image generated in step S25 on the display screen of the display device 4 (step S26).

  If the operation for generating the display screen is not completed (No in step S27), that is, if a new display image is input from the input unit 14, the process returns to step S12 and is imaged from the position detection cameras 2 and 2. The operation after the image input operation is performed. On the other hand, when the operation for generating the display screen is ended (Yes in step S27), that is, when a new display image is not input from the input unit 14, the operation is ended.

  With the above operation, the image generating apparatus 1 detects the position of the viewer from the captured image input from the position detection camera, and displays the position on the display screen based on the position of the viewer and the position and size of the display screen. On the other hand, it is possible to generate a reverse distortion-added image in which reverse distortion for removing distortion that occurs when the viewer looks at a position shifted from the directly-facing position is added to the input display image. Since the viewer's position is detected for each input display image and an image with reverse distortion added is generated, the viewer's position on the display screen is always detected, and no distortion is always seen from the viewer. Video can be generated.

  Here, after the generated image area setting unit 13b of the virtual plane setting unit 13 sets the generated image area on the virtual plane (step S20), the coordinate axis is set on the virtual plane to generate the converted virtual screen area. The image area is converted into plane coordinates (step S21), and then the image composition area setting unit 15a of the image composition unit 15 sets the image composition area on the virtual plane (step S23), and further executes image composition. Although the case where the unit 15b synthesizes the display image with this image synthesis region (step S24) has been described, the generated image region set by the generated image region setting unit 13b is determined by the image conversion means 16 within the converted image region. For example, after the image conversion apparatus 1 sets the conversion virtual screen area (step S19), the image may be set before the image conversion (step S25). A coordinate axis is set on the virtual plane to convert the converted virtual screen area into a plane coordinate (step S21), and an image composition area is set on the virtual plane (step S23), and a display image is displayed in the image composition area. May be combined (step S24), and then the generated image area may be set on the virtual plane (step S20).

[Usage example 1 of image generating apparatus]
Next, referring to FIG. 7 (refer to FIG. 1 as appropriate), an example of use of the image generation apparatus according to the present invention will be described. Here, when a composite video is produced in a television studio, characters or figures are shown to the performer by the technique of intermittent projection light that is visible to the performer but not reflected on the television camera (see Japanese Patent Application No. 2001-142262). An example in which the image generation apparatus 1 is applied when presenting the information will be described. FIG. 7A is an explanatory view showing an example in which a display image for presenting information to performers is projected from the projection device, and FIG. 7B is a display image for presenting information to performers. It is explanatory drawing which shows the state which removes the distortion of the image seen from the performer, and projects from a projection apparatus with an image generation apparatus (not shown).

  Conventionally, as shown in FIG. 7A, when a display image for presenting information to the performer V is projected as it is by the projection device R, the performer V is near the projected video I. When viewed from this performer V, video I looks distorted.

  Therefore, position detection cameras 2 and 2 (not shown) are installed on the ceiling or the like in the studio, and the inside of the studio including the performer V is imaged, and this captured image is input to the image generation apparatus 1. . In addition, a display image including information such as characters and figures for the performer V is input to the image generation device 1 and a projection device for projecting the display image is connected to the generated reverse distortion added image. Is projected from the projection device onto a screen or the like. In addition, the position and size of the portion where the image is projected by the projection device R are registered in the image generation device 1 in advance. Then, the image generation device 1 detects the position of the performer V from the captured image captured by the position detection camera 2, and according to the position of the performer V and the position and size of the portion on which the image is projected. Thus, a reverse distortion-added image in which the distortion of the image viewed from the performer V is removed is generated. Then, as shown in FIG. 7B, the projection device R projects an image I ′ that is not distorted when viewed from the performer V. As described above, when the image generating device 1 is applied when presenting information such as characters and figures to the performer, there is a performer (viewer) near the projected image (image). However, it is possible to display an image from which distortion is removed when viewed from the performer, and to effectively present information to the performer.

[Usage example 2 of image generating apparatus]
Next, with reference to FIG. 8 (refer to FIG. 1 as appropriate), another usage example of the image generation apparatus 1 according to the present invention will be described. Here, the image generation apparatus 1 is applied to a television receiver, and a reverse distortion added image including a television program and other information is generated according to the position of the viewer from the television receiver to generate the television receiver. An example of displaying on the screen will be described. FIG. 8A is an explanatory view showing a state in which the viewer sees a reverse distortion-added image generated by an image generation device (not shown) from a position facing the display screen, and FIG. An explanatory view showing a state in which a reverse distortion added image generated by an image generation device (not shown) is viewed from the upper position with respect to the display screen. FIG. FIG. 5 is an explanatory diagram showing a state in which a reverse distortion added image generated by an image generation device (not shown) is viewed.

  As shown in FIG. 8A, when the viewer V views the display screen F from a position facing the display screen F of the television receiver, the image generation device 1 is input from the position detection cameras 2 and 2. The position of the viewer V is detected from the captured image and the video of the television program is displayed on the entire display screen F.

  When the viewer V views the display screen F at a position greatly deviated from the directly-facing position of the display screen F, the image generation device 1 detects the position of the viewer V from the captured images input from the position detection cameras 2 and 2. Is detected. Then, according to the position of the viewer V with respect to the detected display screen F, the image generating device 1 uses the image composition area setting unit 15a of the image composition means 15 to place two or more in the converted virtual screen area on the virtual plane. Set the image composition area. Here, an image including other information in addition to the video of the TV program is input from the outside to the image generation apparatus 1, and the image composition execution unit 15 b displays different images in the set image composition areas. Composite the images. Accordingly, a reverse distortion added image including a television program and other information can be displayed on the television receiver according to the position of the viewer from the television receiver.

For example, the example of FIG. 8B shows a case where the viewer V views the display screen F from a position above the display screen F of the television receiver. The setting unit 15a sets up and down two image composition areas in the converted virtual screen area according to the detected position of the viewer V. Then, the image generation device 1 uses the image composition execution unit 15b to display the image of the TV program in the upper image composition area and the time inputted from the outside, for example, in the lower image composition area. Combine the images to be displayed. Furthermore, the image generation apparatus 1 converts the image by the image conversion unit 16 to generate a reverse distortion added image, and causes the display unit 17 to display the reverse distortion addition image on the display screen F. As a result, although the viewer V is viewing from the upper position, the reverse distortion-added image displayed on the television receiver becomes an image without distortion as viewed from the viewer V, and further, the video of the TV program is displayed. A time image I ₁ is displayed below.

8C shows a case where the viewer V views the display screen F from the left position with respect to the display screen F of the television receiver. At this time, the image generating apparatus 1 is detected. Two image composition areas are set on the left and right in the converted virtual screen area according to the position of the viewer V. Then, the image generating apparatus 1 displays an image of a television program in the image composition area located on the left, and displays, for example, an adjustment menu of the television receiver input from the outside in the image composition area located on the right. Composite the images. Further, the image generation apparatus 1 converts the image to generate a reverse distortion added image, and displays the reverse distortion added image on the display screen F. As a result, the displayed reverse distortion added image becomes an image having no distortion when viewed from the viewer V, and further, the image I ₂ of the adjustment menu of the television receiver is displayed on the right of the video of the television program. It has become.

  Thereby, the image generating apparatus 1 inputs a plurality of display images at the same time, synthesizes one or more display images based on the position of the viewer with respect to the display screen, and distorts when viewed from the viewer when displayed on the display screen. A reverse distortion-added image that does not occur is generated. At this time, an image obtained by combining a plurality of display images can be displayed according to the position of the viewer. Therefore, by changing the position with respect to the display screen in order to present an image including information desired by the viewer, the image The generation apparatus 1 can generate an image corresponding to the position, that is, an inverse distortion-added image including an image desired by the viewer, and display the image on the display screen.

[Image generation apparatus (second embodiment)]
Next, with reference to FIG. 9, an image generating apparatus 1B according to the second embodiment of the present invention will be described. FIG. 9 is a block diagram showing a configuration of an image generation apparatus according to the second embodiment of the present invention. As shown in FIG. 9, the image generating apparatus 1B generates an image (computer graphics) based on the position of the viewer, and generates a reverse distortion added image from the image.

  The image generation device 1B is configured by adding a virtual plane setting unit 13B, an image synthesis unit 15B, a display image generation unit 18, and a drawing database 19 to the image generation device 1. The configuration other than the virtual plane setting unit 13B, the image synthesis unit 15B, the display image generation unit 18 and the drawing database 19 in the image generation apparatus 1 is the same as that shown in FIG. Description is omitted.

  The virtual plane setting unit 13B sets a virtual plane having a virtual viewer position as a virtual viewpoint and a normal line extending from the virtual viewpoint and passing through the center of the virtual screen area, and a converted virtual screen area, a generated image area, Is set. Then, the generated image area setting unit 13Bb outputs the converted virtual screen area information representing the converted virtual screen area on the virtual plane and the generated image area information representing the generated image area to the image composition unit 15. The generated image area setting unit 13Bb generates generated image area information representing a generated image area in the virtual space, and virtual viewer position information and virtual screen area information input from the virtual space setting unit 12. Output to means 18. The virtual plane setting unit 13B has the same function as the virtual plane setting unit 13 (FIG. 1) except that the number of output destinations is increased.

  The display image generating unit 18 generates a display image based on the virtual viewer position information, virtual screen region information, and generated image region space information input from the virtual plane setting unit 13B. The display image generation unit 18 includes a display image generation range setting unit 18a and a display image generation execution unit 18b.

  The display image generation range setting unit 18a sets a display image generation range based on virtual viewer position information, virtual screen area information, and generated image area space information input from the virtual plane setting unit 13B. Here, the display image generation range represents a range in which a display image (computer graphics) is generated by the display image generation execution unit 18b described later. The virtual viewer position is a virtual viewpoint, and the virtual viewpoint is converted into a virtual screen. A range surrounded by a straight line connecting the vertices of the region.

  Here, referring to FIG. 10 (refer to FIG. 9 as appropriate), the display image generation range setting unit 18a performs display image generation range based on the virtual viewer position information, virtual screen area information, and generated image area space information. An example of setting will be described. FIG. 10 is an explanatory diagram illustrating an example in which a display image generation range is set based on virtual viewer position information, virtual screen area information, and generated image area space information.

  First, the display image generation range setting unit 18a uses the display image generation execution unit 18b described later for the virtual viewer position and the generated image area in the virtual space set by the virtual space setting unit 12 and the virtual plane setting unit 13B. The display image (computer graphics) is converted into a reference coordinate (hereinafter referred to as CG coordinate) when it is generated. Here, the relationship between the virtual screen region and the coordinate axis of the CG coordinate or the relationship between the coordinate axis of the virtual space and the coordinate axis of the CG coordinate is determined in advance, so that the virtual viewer position and the coordinates of the generated image region are set to the CG coordinate. Can be converted to For example, as shown in FIG. 10, the display image generation range setting unit 18a sets the coordinate axes gX, gY, gZ of the CG coordinates, and the virtual viewer position Pp and the virtual plane setting unit 13B set by the virtual space setting unit 12. The generated image area Iv set by is converted into CG coordinates. That is, the coordinates of the virtual viewer position Pp and each vertex Pa ″, Pb ″, Pc ″, Pd ″ of the generated image area Iv are converted to CG coordinates, and the virtual viewer position gPp on the CG coordinates and each of the generated image areas are converted. Vertices gPa ″, gPb ″, gPc ″, and gPd ″ are calculated.

  The display image generation range setting unit 18a sets the virtual viewer position as a virtual viewpoint, and sets a range surrounded by a straight line connecting each vertex of the converted virtual screen area from the virtual viewpoint as a display image generation range. That is, as shown in FIG. 10, the display image generation range setting unit 18a sets the virtual viewer position gPp as the virtual viewpoint gPp. Further, the display image generation range setting unit 18a is surrounded by straight lines gLa, gLb, gLc, and gLd connecting the vertices gPa ″, gPb ″, gPc ″, and gPd ″ of the generated image region Iv from the virtual viewpoint gPp. Let U be the display image generation range.

  Returning to FIG. 9, the description will be continued. Based on the drawing database 19, the display image generation execution unit 18b generates an image viewed from a virtual viewpoint within the display image generation range set by the display image generation range setting unit 18a. Here, the generation of an image (computer graphics) by the display image generation execution unit 18b can be realized by a known computer graphics generation technique. Then, the display image generated by the display image generation execution unit 18 b is output to the image composition unit 15.

  The drawing database 19 (three-dimensional image data storage means) stores various data necessary for generating images (computer graphics) in the display image generation execution unit 18b, and is generally used for semiconductor memories, hard disks, and the like. Is a typical recording medium. The drawing database 19 is referred to by the display image generation execution unit 18b and is used when generating an image within the display image generation range viewed from the virtual viewpoint.

  The image synthesizing unit 15B sets an image synthesizing region, which is a region for synthesizing the display image, on the virtual plane, and synthesizes the display image input from the display image generating unit 18 in this image synthesizing region. The image composition unit 15B includes an image composition area setting unit 15Ba and an image composition execution unit 15Bb.

  The image composition area setting unit 15Ba is for setting an image composition area, which is an area in which the display image input from the display image generating means 18 is composed. Here, the image composition area is an area for synthesizing the input display image. Since the input display image is an image corresponding to the generated image area generated by the display image generating means 18, the image composition is performed. The area setting unit 15Ba sets the generated image area as an image synthesis area.

  The image composition execution unit 15Bb is for synthesizing the display image input from the display image generation means 18 within the image composition region set by the image composition region setting unit 15Ba. Then, the image on the generated image area obtained by synthesizing the display image by the image synthesis execution unit 15Bb is output to the image conversion means 16. The image composition execution unit 15Bb has the same function as the image composition execution unit 15b (FIG. 1) except that the input destination of the display image is changed from the input unit 14 (FIG. 1) to the display image generation unit 18. is there.

  Here, referring to FIG. 11 and FIG. 12 (refer to FIG. 9 as appropriate), the image composition means 15B composes a display image by setting an image composition area, and the image conversion means 16 converts the image. explain. FIG. 11A is an explanatory view showing an example of a display image input from the display image generating means 18, and FIG. 11B is an explanatory view showing an example of setting an image composition area on a virtual plane. () Is an explanatory diagram showing an example of synthesizing a display image in an image synthesis area. FIG. 12A is an explanatory diagram showing an image in the converted virtual screen area in the image synthesized in FIG. 11, and FIG. 12B is an image converted in the converted virtual screen area in FIG. It is explanatory drawing which showed the example which produces | generates a reverse distortion addition image.

  As shown in FIG. 11 (a), the display image G is input from the display image generating means 18, and as shown in FIG. 11 (b), the generated image area Iv and the converted virtual screen area Dv are set by the virtual plane setting means 13B. 'Is set, the image composition area setting unit 15Ba of the image composition means 15B sets the generated image area Iv as the image composition area Di. Then, the image composition execution unit 15Bb synthesizes the display image G in the image composition region Di (Iv) set by the image composition region setting unit 15Ba, as shown in FIG. Further, the image composition execution unit 15Bb outputs the image in the generated image area Iv to the image conversion unit 16.

  Further, the image conversion means 16 corresponds to the generated image area Iv from the image (FIG. 11C) input from the image composition execution unit 15Bb, and the image of the converted screen area Dv ′ shown in FIG. And transform to generate a reverse distortion added image. In other words, the image converting means 16 has the vertices spa ′, spb ′, spc ′, spd ′ of the converted virtual screen area Dv ′ coincide with the vertices spa ″, spb ″, spc ″, spd ″ of the generated image area Iv, respectively. As described above, a process of deforming the image in the converted virtual screen area Dv ′ is performed. By this processing, the entire image in the converted virtual screen area Dv ′ is deformed. Then, as shown in FIG. 12B, the image in the converted virtual screen area Dv ′ becomes an image deformed corresponding to the position of the viewer, and this inverse distortion added image is displayed on the display screen of the display device 4. When viewed from the viewer's position, the image shown in FIG. 12A looks like an image in the region Dv ′.

  Thereby, the image generation device 1B generates a display image corresponding to the position of the viewer, and generates a reverse distortion added image that does not cause distortion when viewed on the display screen from the display image. . Then, the image generation device 1B sets a virtual screen region and a virtual viewer position (virtual viewpoint) on the CG coordinates, generates a display image (computer graphics) viewed from this virtual viewpoint, and uses this display image. Since the display screen is generated, if the viewer changes the position relative to the display screen, the reverse distortion added image displayed on the display screen is displayed at the viewer's position as if the viewer is looking at the scenery beyond the window. It can change with movement. For example, when the viewer looks down on the display screen from above, the image generation device 1B detects the position of the viewer and generates an inverse distortion-added image corresponding to the position. Then, the display screen displays an image corresponding to a portion below the image displayed when the viewer looks from a position facing the display screen. Therefore, for example, when the position of the viewer is changed so that the portion of the image that the viewer wants to view is displayed, the image generation device 1B displays the image of the portion that the viewer wants to view on the display screen of the display device 4. Can be displayed.

  The configuration of the image generation apparatus 1B has been described above. However, the image generation apparatus 1B can also realize each unit as each function program in a computer, and combine the function programs to operate as an image generation program. Is also possible.

  Here, the display image generation range setting unit 18a of the display image generation means 18 uses the range surrounded by a straight line connecting the vertices of the generation image area from the virtual viewpoint as the display image generation range, and the image of the image composition means 15B. The synthesis area setting unit 15Ba sets the generated image area as the image synthesis area. For example, the display image generation range setting unit 18a is surrounded by a straight line connecting each vertex of the converted virtual screen area from the virtual viewpoint. The area may be set as a display image generation range, and the image composition area setting unit 15Ba may set the converted virtual screen area as an image composition area, or the display image generation range setting unit 18a may select an arbitrary area ( (Not shown), a display image generation range is set based on an arbitrary area on this virtual plane and a virtual viewpoint, and the image composition area setting unit 15Ba Any region of the upper may be an image combining region.

  Further, even if the coordinates of the virtual space set by the virtual space setting means 12 and the coordinates (CG coordinates) used as a reference when generating the image (computer graphics) by the display image generation execution unit 18b are set to be the same. In this way, the display image generation range setting unit 18a does not need to convert the virtual viewer position, the virtual screen area, and the generated image area in the virtual space into CG coordinates.

  Further, here, the display image generation means 18 is configured to generate computer graphics within the display image generation range based on the drawing database. However, the image generation apparatus 1B is an input means for inputting an image (FIG. (Not shown), an image may be input via the input means, and an image (display image) corresponding to the display image generation range may be generated (extracted) from the image. At this time, the image input from the input unit is input to the display image generation unit 18. Then, a correspondence relationship between the input image and the virtual screen area is determined in advance, and based on this, the display image generation range setting unit 18a sets a three-dimensional coordinate corresponding to the CG coordinate. Further, the display image generation range setting unit 18a sets an input image, a virtual screen region, a virtual viewpoint, and a generated image region on the three-dimensional coordinates, and sets a display image generation range. The display image generation execution unit 18b extracts an image corresponding to the display image generation range from the image input from the input unit, so that the display image generation unit 18 displays a display image corresponding to the position of the viewer. Can be generated from the input image.

[Usage example 3 of image generating apparatus]
Here, with reference to FIG. 13 (refer to FIG. 9 as appropriate), a usage example of the image generation apparatus 1B according to the present invention will be described. Here, an example in which the image generation device 1B is applied to a videophone will be described. FIG. 13A is an explanatory diagram showing a state in which the room of the other party is captured with a wide-angle camera, and FIG. 13B is generated by the image generation device 1B (not shown) from the image captured in FIG. Explanatory drawing which shows the state which an operator sees a reverse distortion addition image from various angles, (c) is an image generation apparatus 1B (not shown) by a wide angle camera corresponding to the position with respect to a display screen of an operator. It is explanatory drawing which shows the image extracted from the imaged image.

As shown in FIG. 13A, a wide-angle camera C is installed in the room of the other party to image the room. Then, this image is input to the image generation apparatus 1B via a communication line or the like. Here, when the operator is at the position facing the display screen F of the videophone, that is, at the position of the operator (viewer) V _A in FIG. The position of the operator V _A is detected, and a display image corresponding to the position of the operator V _A is extracted (generated) from the image of the wide-angle camera C. Here, if an image as illustrated in FIG. 13C is input to the image generation apparatus 1B from the wide-angle camera C, the image generation apparatus 1B includes the image captured by the wide-angle camera C and the wide-angle camera C. The virtual viewpoint and the generated image area are set on the three-dimensional coordinate by associating the position of the wide-angle camera on the three-dimensional coordinate with the virtual screen area. Thus, an image corresponding to the position of the operator with respect to the display screen can be extracted (generated). Here, the image generating apparatus 1B, as an image corresponding to the operator V _A being in a position directly facing the display screen F videophone extracts image G _A of the center portion of the image in FIG. 13 (c). Then, the image generating apparatus 1B, the image G _A when displayed on the display screen F, is converted so as not to cause distortion when viewed from the operator V _A, is displayed on the display screen F videophone.

Further, when the operator views the display screen F of the videophone from the left side, that is, from the position of the operator V _B in FIG. 13B, the image generating device 1B displays the operator V _B. And a display image corresponding to this position is extracted (generated) from the image of the wide-angle camera C. Image generating apparatus 1B, the input as a display image corresponding to the operator V _B that are in the position of the left side with respect to the display screen F, the image G _B of the right portion of the image in FIG. 13 (c), from the wide-angle camera C Extract from extracted images. Then, the image generating apparatus 1B, this time displaying the image G _B on the display screen F, is converted so as not distorted when viewed from the operator V _B, is displayed on the display screen F videophone. Similarly, when the operator views the display screen F of the videophone from the right side of the facing position, that is, from the position of the operator V _C in FIG. When the image G _C in the left part of the image c) is extracted from the image input from the wide-angle camera _C and displayed on the display screen F, there is no distortion when viewed from the operator V _C. To be displayed on the display screen F.

  As a result, in the conventional videophone, the operator (viewer) changed the position relative to the display screen so that only the image in the range displayed on the videophone display screen could be seen so far. It is possible to display a video that has not been performed. Since the display image to be displayed on the display screen is extracted according to the position of the operator, if the operator changes the position with respect to the display screen of the videophone, it is as if the operator is looking at the scenery beyond the window. The reverse distortion added image displayed on the display screen of the videophone can be changed as the operator moves.

1 is a block diagram illustrating a configuration of an image generation apparatus according to a first embodiment of the present invention. (A) is explanatory drawing which showed the state where the display apparatus, the position detection camera, and the viewer are arrange | positioned in real space, (b) is a virtual viewer position and virtual screen area | region on virtual space. It is explanatory drawing which showed the example which sets this. (A) is explanatory drawing which showed the example which sets a conversion virtual screen area | region on a virtual plane, (b) is explanatory drawing which showed the example which sets a production | generation image area | region on a virtual plane. It is explanatory drawing which showed the example which sets an image composition area | region on a virtual plane, and synthesize | combines a display image. (A) is an explanatory view showing a reverse distortion added image obtained by converting the image in the converted virtual screen area of FIG. 4 in association with the generated image area, and (b) displays the reverse distortion added image of (a). It is explanatory drawing which showed the reverse distortion addition image when it displays on a screen and it sees from the position of the viewer. It is the flowchart which showed the operation | movement which produces | generates the reverse distortion addition image which remove | eliminated the distortion of an image which generate | occur | produces when an image generation apparatus sees an image in the position which shifted | deviated from the facing position with respect to the display screen. (A) is explanatory drawing which showed the example which projects the display image for presenting the information to a performer from a projection apparatus, (b) is a display image for presenting the information to a performer, an image It is explanatory drawing which shows the state which removes the distortion of the image seen from the performer by the production | generation apparatus, and projects from a projection apparatus. (A) is explanatory drawing which shows the state which a viewer looks at a reverse distortion addition image from the position which directly faces a display screen, (b) is a viewer seeing a reverse distortion addition image from the upper position with respect to a display screen. An explanatory view showing the state, (c) is an explanatory view showing a state in which the viewer sees the reverse distortion added image from the left position with respect to the display screen. It is the block diagram which showed the structure of the image generation apparatus which is 2nd embodiment in this invention. It is explanatory drawing which showed the example which sets a display image production | generation range based on virtual viewer position information, virtual screen area | region information, and production | generation image area | region space information. (A) is explanatory drawing which showed the example of the display image input from the display image production | generation means, (b) is explanatory drawing which showed the example which sets an image composition area | region on a virtual plane, (c), It is explanatory drawing which showed the example which synthesize | combines a display image in an image composition area | region. (A) is explanatory drawing which shows the image in the conversion virtual screen area | region in the image synthesize | combined in FIG. 11, (b) is a reverse distortion addition image by converting the image in the conversion virtual screen area | region of (a). It is explanatory drawing which showed the example which produces | generates. (A) is explanatory drawing which shows the state which images a room with a wide-angle camera, (b) is an operator with various back distortion addition images produced | generated by the image production | generation apparatus from the image imaged in (a). Explanatory drawing which shows the state seen from an angle, (c) is explanatory drawing which shows the image which an image generation apparatus extracts from the image imaged with the wide angle camera corresponding to the position with respect to a display screen of an operator.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,1B Image generation apparatus 11 Viewer position detection means 12 Virtual space setting means 13, 13B Virtual plane setting means 15, 15B Image composition means 16 Image conversion means 18 Display image generation means 19 Drawing database (three-dimensional image data storage means)
Pp Virtual viewer position (virtual viewpoint)
Dv virtual screen area Dv 'conversion virtual screen area Iv generation image area

Claims (3)

In order to remove the distortion of the display image that occurs when the viewer views the display image at a position deviating from the directly-facing position with respect to the display screen when displaying the display image on the display screen. An image generation device for generating a reverse distortion added image obtained by adding reverse distortion to a display image,
Viewer position detecting means for detecting the position of the viewer;
Virtual space setting means for setting the position of the viewer and the screen area of the display screen detected by the viewer position detection means as a virtual viewer position and a virtual screen area on the virtual space;
The virtual viewer position set by the virtual space setting means is a virtual viewpoint, and the virtual viewpoint and the virtual screen area are on a virtual plane having a half line passing through the center of the virtual screen area from the virtual viewpoint as a normal line. An area surrounded by an intersection of a straight line passing through each vertex of the virtual plane and the virtual plane is defined as a converted virtual screen area, and a rectangular area that includes the converted virtual screen area and has an aspect ratio of the virtual screen area is defined as a generated image area. A virtual plane setting means for setting;
Image combining means for combining the display image so that at least a part of the display image is included in the converted virtual screen area set by the virtual plane setting means;
Image conversion means for converting the image in the converted virtual screen area including at least a part of the display image synthesized by the image synthesis means in association with the generated image area, and generating the reverse distortion added image; An image generation apparatus comprising: 3D image data storage means for storing 3D image data for generating the display image;
Display image generating means for generating the display image corresponding to the position of the viewer based on the three-dimensional image data;
The image generating apparatus according to claim 1, wherein the image synthesizing unit synthesizes the display image generated by the display image generating unit. In order to remove the distortion of the display image that occurs when the viewer views the display image at a position deviating from the directly-facing position with respect to the display screen when displaying the display image on the display screen. In order to generate a reverse distortion-added image with reverse distortion added to the display screen,
Viewer position detecting means for detecting the position of the viewer;
Virtual space setting means for setting the position of the viewer and the screen area of the display screen detected by the viewer position detection means as a virtual viewer position and a virtual screen area on the virtual space;
The virtual viewer position set by the virtual space setting means is a virtual viewpoint, and the virtual viewpoint and the virtual screen area are on a virtual plane having a half line passing through the center of the virtual screen area from the virtual viewpoint as a normal line. An area surrounded by an intersection of a straight line passing through each vertex of the virtual plane and the virtual plane is defined as a converted virtual screen area, and a rectangular area that includes the converted virtual screen area and has an aspect ratio of the virtual screen area is defined as a generated image area. Virtual plane setting means for setting,
Image combining means for combining the display image so that at least a part of the display image is included in the converted virtual screen area set by the virtual plane setting means;
Function as image conversion means for converting the image in the converted virtual screen area including at least a part of the display image synthesized by the image synthesis means in association with the generated image area and generating the inverse distortion added image An image generation program characterized in that

Images