JP5183277B2 - Stereoscopic image display device - Google Patents

Description

  The present invention relates to a stereoscopic image display device that realizes depth expression of stereoscopic content by displaying a two-dimensional image on each display layer.

  As a conventional stereoscopic image display device that realizes depth expression, for example, there is a content reproduction device disclosed in Patent Document 1. This apparatus sets depth information indicating a relative depth relationship from a predetermined plane as a reference plane for two-dimensional graphic data, and generates graphic data corresponding to each display layer based on the depth information. It was displayed. Furthermore, a left-eye image and a right-eye image are created from this content and displayed on the 3D liquid crystal. Then, the left-eye image enters only the viewer's left eye, and the right-eye image enters only the right eye, so that the content is stereoscopically viewed.

JP 2004-145832 A

  Since the conventional stereoscopic image display apparatus is configured as described above, it is intended to realize depth expression when the display is viewed from the front, and consideration is given to depth expression when viewed from the top, bottom, left, and right directions. Was not. Large screen displays are characterized by a wide range of viewing point movement for viewers, and when installed on the walls of buildings, halls, etc., the view from the top and bottom is very important in addition to the left and right direction. . Therefore, particularly on a large screen display, there is a problem that it is difficult to see an image displayed depending on the viewing angle.

  The present invention has been made to solve the above-described problems, and by displaying according to the viewpoint of a moving person, a stereoscopic image that can display an image with a sense of depth from the vertical and horizontal directions of the display. An object is to obtain a display device.

A stereoscopic image display apparatus according to the present invention includes a hierarchical coordinate management unit that defines the position and size of a display hierarchy based on coordinate data based on three-dimensional spatial coordinates on a screen on which three-dimensional spatial coordinates are set, and a viewer's viewpoint 3D for adjusting the display position of the two-dimensional image displayed on the display layer based on the viewpoint coordinates formed by converting the detected viewpoint position into the position of the viewpoint on the display layer. An effect calculation unit, a distortion correction processing unit that transforms a two-dimensional image displayed on the display layer according to projective transformation in accordance with the viewpoint coordinates, and a superimposition that superimposes the two-dimensional image displayed on each display layer in an order based on the coordinate data a display processing unit, among the two-dimensional image to be displayed on each display hierarchy, a cylindrical image generating unit that generates a cylindrical image by connecting the start and end frames of an arbitrary two-dimensional image, the movement of the viewpoint coordinate Is obtained by so and a scroll display processing unit scrolls the cylindrical image Te Align.

  According to the present invention, the position and size of the display hierarchy are defined by the coordinate data based on the three-dimensional space coordinates on the screen on which the three-dimensional space coordinates are set, and the viewer's viewpoint position is determined from the viewpoint on the display hierarchy. The display position of the two-dimensional image displayed on the display layer is adjusted based on the viewpoint coordinates converted into the position of the two-dimensional image, and the two-dimensional image displayed on the display layer is modified by projective transformation according to the viewpoint coordinates, Since the 2D images to be displayed are superimposed on each display layer in the order based on the data, display with a sense of depth is easy to see from the top, bottom, left and right of the display by displaying according to the viewpoint of the moving person. A stereoscopic image display device that can be obtained can be obtained.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the overall configuration of a stereoscopic image display apparatus according to Embodiment 1 of the present invention. The stereoscopic image display apparatus shown in FIG. 1 includes a sensor 1 for acquiring viewer's viewpoint information, a viewer viewpoint recognition unit 2 that detects the viewer's viewpoint using an output value of the sensor 1, and a viewer's viewpoint movement. A 3D effect calculation unit 3 that calculates a 3D effect corresponding to the image, a hierarchical coordinate management unit 4 that manages coordinate data of a display layer that displays an image, an image recording unit 5 that stores an image, and an image that is displayed even if the entire screen is scrolled A cylindrical image generation unit 6 that generates an uninterrupted cylindrical image, an automatic mask generation unit 7 that generates a mask for the image, an imaging unit 8 that captures an image, and a distortion correction process that shapes the image according to the viewpoint direction of the viewer 9 and 3D effect calculation unit 3 that performs the 3D effect calculated by the image on the image, the scroll display processing unit 11 that scrolls the image, and the superimposed display process that superimposes the images of each display layer. 12 includes an image boundary processing unit 13 for changing the transparency of the mask boundary line of the mask with the image, the screen display unit 14 for displaying the superimposed image. The stereoscopic image display device described in the present embodiment is assumed to be applied mainly to a large screen display.

  FIG. 2 is an explanatory diagram showing an example of a screen displayed by the stereoscopic image display apparatus according to Embodiment 1 of the present invention. The screen display unit 14 illustrated in FIG. 1 displays a superimposed image 18 in which images displayed on three display layers are superimposed. This superimposed image 18 is farthest from the viewer's position and is the far-field image 15 displayed on the display layer deeper from the screen surface, the middle-view image 16 displayed on the previous display layer, and the viewer's position. A foreground image 17 to be displayed is superimposed on the display layer near the screen surface. In the present embodiment, the image of the blue sky through which the clouds flow is displayed in the distant view image 15, the image of the tree blown in the wind prepared as the image with the mask in the intermediate view image 16, and the image with the mask in the close view image 17. We prepared images of the dancers that were prepared. The image having the mask data is superimposed on the background image through the designated area by the mask data when superimposed by the superimposed display processing unit 12.

  A sensor 1 shown in FIG. 1 includes a camera, a pressure sensitive sensor, and the like. The sensor 1 may be configured with only the camera or only the pressure sensor, or may be configured using both the camera and the pressure sensor. When a camera is used as the sensor 1, a position where the camera faces the screen display unit 14 is photographed, and the acquired image is output as a detection value to the viewer viewpoint recognition unit 2. When a pressure sensor is used as the sensor 1, the pressure sensor is disposed on the front floor surface of the screen display unit 14, measures the pressure applied by the viewer, and outputs the detected value to the viewer viewpoint recognition unit 2. .

  When the detection value of the sensor 1 is input, the viewer viewpoint recognition unit 2 detects the position of the viewer and the height of the viewpoint using the detection value. When a camera is used as the sensor 1, the viewer viewpoint recognition unit 2 has a face image detection program and detects the coordinates of the face from the camera image. And the three-dimensional relative coordinate which shows the plane position and the height position of a viewpoint with which a viewer is with respect to the screen display part 14 is acquired from the imaging range information of a camera image.

  When a pressure sensor is used as the sensor 1, the viewer viewpoint recognition unit 2 calculates a two-dimensional relative coordinate indicating the planar position of the viewer based on the detection value of the pressure sensor, and the pressure value is further calculated. The height of the viewer is estimated based on the above, and three-dimensional relative coordinates indicating the height position of the viewpoint are obtained.

  When 3D relative coordinate data is input from the viewer viewpoint recognition unit 2, the 3D effect calculation unit 3 is displayed on the display hierarchy from the 3D relative coordinates and the 3D spatial coordinates of the screen display unit 14 prepared in advance. Relative coordinates (viewpoint coordinates) where the viewpoint is present are calculated for each display layer. Then, for example, as shown in FIG. 3, images displayed on each display layer so that the superimposed image displayed on the screen display unit 14 has a 3D effect in response to the viewer's viewpoint movement represented by relative coordinates. The optimum parameters regarding the size of the image, the coordinates for displaying the image, the image effect, etc. are calculated. Hereinafter, an example of the 3D effect calculated by the 3D effect calculation unit 3 will be described with reference to FIG.

  FIG. 3 is an explanatory diagram for explaining the 3D effect calculation processing by the stereoscopic image display apparatus according to Embodiment 1 of the present invention. In this example, the display position of the image displayed on the distant display layer by the 3D effect calculation unit 3 is fixed. The images displayed on the display layers for the middle scene and the near scene were moved in the direction opposite to the viewpoint and in the same direction with the screen center as a reference. 3 are images with masks as described above, the outer frame of each image is not displayed on the screen, but the outer frame is shown here for convenience of explanation. .

  FIG. 3A is a superimposed image displayed by the screen display unit 14 when the viewer is in the front direction 41 a of the screen display unit 14. When the viewer views from the front direction 41a, the 3D effect calculation unit 3 sets the display coordinates of each image so that the distant view image 25a, the middle view image 26a, and the foreground image 27a are displayed in the center of the screen. calculate.

  Next, a superimposed image displayed by the image display unit 14 when the viewer is in the left direction 41b with respect to the screen display unit 14 is shown in FIG. When the viewer views from the left direction 41b, the middle scene image 26b is displayed shifted to the right with respect to the fixed distant view image 25b, and the foreground image 27b is leftward with respect to the middle scene image 26b. The 3D effect calculation unit 3 calculates the display coordinates of each image so that the images are displayed in a shifted manner.

  Further, FIG. 3C shows a superimposed image displayed by the screen display unit 14 when the viewer is in the right direction 41c with respect to the screen display unit 14. At this time, the middle scene image 26c is displayed while being shifted to the left with respect to the fixed far scene image 25c, and the foreground image 27c is displayed with being shifted to the right with respect to the middle scene image 26c. The 3D effect calculation unit 3 calculates the display coordinates of each image. Relative coordinates calculated by the 3D effect calculation unit 3 and the parameter data of each display layer are output to the distortion correction processing unit 9.

  Furthermore, it is assumed that a large screen display is viewed while a person walks in front of it. The 3D effect calculation unit 3 changes the display coordinates of the image in each display layer according to the speed at which the person walks, so that an easy-to-view display can be performed while walking. In that case, the 3D effect calculation unit 3 sequentially calculates the relative coordinates where the viewpoint on each display layer exists based on the three-dimensional relative coordinate data sequentially output from the viewer viewpoint recognition unit 2, and matches the relative coordinates. Display coordinates for changing each image as shown in FIGS. 3A, 3B, and 3C are sequentially calculated. Note that the 3D effect calculation unit 3 calculates the parameters so that the display coordinates of the images displayed on each display layer are moved up and down depending on the difference in the height position of the viewpoint.

  In addition to the display coordinates of the image as described above, the 3D effect calculation unit 3 calculates parameters such as the brightness, contrast, and blue balance of the image for each display layer with respect to, for example, image effects. For example, in order to increase the perspective of the superimposed image, the 3D effect calculation unit 3 calculates a parameter for performing color tone correction that increases the blue balance as the use of the foreground image becomes a distant view with reference to the blue balance of the foreground image. . In this way, the perspective of the superimposed image can be increased by performing color tone correction that enhances the blue color for an image displayed on a display layer of a distant view.

  Further, in order to increase the perspective of the superimposed image, for example, the 3D effect calculation unit 3 calculates a parameter for performing correction to increase the blurring degree from the near view to the far view with reference to the blurring degree of the middle scene image. . As described above, the perspective effect of the superimposed image can be increased by strongly adding the blurring effect to the image to be displayed on the distant view display layer.

  The hierarchical coordinate management unit 4 holds coordinate data of each display hierarchy for displaying each image data for distant view, foreground and foreground. As shown in FIG. 2, the three-dimensional image display device is preset with the three-dimensional spatial coordinates of the screen, the X axis parallel to the vertical direction of the screen, the Y axis parallel to the horizontal direction of the screen, and the front of the screen. It consists of a Z axis parallel to the direction toward the back. In the coordinate data of each display layer, the size of the display layer is defined by the XY axis, and the depth is defined by the Z axis. In the first embodiment, an example in which three types of display hierarchies are provided, that is, a distant view, a middle view, and a foreground, is shown. A number display hierarchy may be defined by each coordinate data. The hierarchical coordinate management unit 4 outputs the coordinate data to the distortion correction processing 9, the scroll display processing unit 11, and the superimposed display processing unit 12 as necessary.

  The image recording unit 5 records each image data for distant view, foreground and foreground by a moving image or a still image input from the image pickup unit 8. The image recording unit 5 may have a real-time capture function. By providing the real-time capture function, the image data can be recorded in the image recording unit 5 and displayed on the stereoscopic image display device almost simultaneously with the photographing by the imaging unit 8.

  The cylindrical image generation unit 6 generates a cylindrical image for scrolling the image without interruption. When image data is sent from the image recording unit 5 to the cylindrical image generation unit 6 as necessary, the cylindrical image generation unit 6 generates a cylindrical image.

  For example, the image recording unit 5 sends the distant view image data to the tubular image generating unit 6 in order to make the distant view image into a cylindrical image. FIG. 4 is an explanatory diagram for explaining a cylindrical image generation process of the stereoscopic image display apparatus according to Embodiment 1 of the present invention. FIG. 4A shows the first frame 21 and the end frame 22 of the image data input to the cylindrical image generation unit 6. In this distant view image, clouds flow from right to left. The joint image 23 shown in FIG. 4B is prepared as an image with a mask in order to hide the joint between the first frame 21 and the end frame 22 of the distant view image, and is input from the outside, for example. The cylindrical image generating unit 6 superimposes the joint image 23 on the joint of the distant view image to generate a cylindrical image 24 shown in FIG. As described above, the cylindrical image generation unit 6 performs image processing for merging the start and end of the image frame so that the image becomes endless when the scroll display processing unit 11 scrolls the entire image to the left and right. . Further, image processing for superimposing a joint image at the joint is performed. However, when there is no joint image 23, the cylindrical image generation unit 6 performs only merge processing. The generated cylindrical image 24 is output again to the image recording unit 5 and is held by the image recording unit 5.

  The mask automatic generation unit 7 automatically generates mask data for designating the screen display portion of the image data for the foreground and foreground. The image recording unit 5 sends image data to the mask automatic generation unit 7 as necessary, and causes the automatic generation unit 7 to automatically generate mask data of the image data.

  For example, in order to use the image data captured by the imaging unit 8 as a foreground image or a foreground image, mask data designating a portion to be displayed on a screen and a portion to be displayed on the screen when these are superimposed on a distant view image are automatically masked. The generation unit 7 creates it. The generated mask data is output to the image recording unit 5 together with the image data, and is held by the image recording unit 5. As a method of automatically generating mask data, for example, image contour analysis is performed by wavelet transform of image data, and mask data is generated with a detected contour as a boundary. In order to determine an image region to be masked from the detected contours, the user may specify a contour for mask creation or set parameters such as a luminance threshold value or a designated region in advance. Since the stereoscopic image display device includes the automatic mask generation unit 7, it is possible to generate an image for a foreground or a foreground with mask data using image data taken with a background of a natural image other than a chroma key background. It becomes.

  The imaging unit 8 captures images for distant view, foreground and foreground to be displayed on the stereoscopic image display device, and outputs the captured image data to the image recording unit 5. The image data may be a moving image or a still image.

  The distortion correction processing unit 9 uses the relative coordinates where the viewer's viewpoint exists calculated by the 3D effect calculation unit 3 and the coordinate data managed by the hierarchical coordinate management unit 4 to perform each display by projective transformation, trapezoidal correction, and the like. The image displayed in the hierarchy is transformed according to the relative coordinates where the viewpoint exists. Hereinafter, an example of the correction process performed by the distortion correction processing unit 9 will be described with reference to FIG.

  FIG. 5 is an explanatory diagram for explaining distortion correction processing by the stereoscopic image display apparatus according to Embodiment 1 of the present invention. Similarly to FIG. 3, the middle and foreground images shown in FIG. FIG. 5A shows a superimposed image obtained by superimposing a distant view image 28a, a foreground image 29a, and a foreground image 30a displayed on the image display unit 14 when a viewer is present in front of the screen display unit 14. Indicates. If the viewer is viewing from the front direction 42a at the same height as the screen, the image is not deformed. However, if the viewer's viewpoint is above or below the screen, it is adjusted to the height of the viewpoint. Then, the distortion correction processing unit 9 changes the lengths of the upper, lower, left, and right sides of the middle scene image 29a and the foreground image 30a to correct the keystone. For example, when the viewpoint is in the upward direction from the screen, the distortion correction processing unit 9 sets each of the middle scene image 29a and the foreground image 30a so that the upper side where the viewpoint exists is short and the lower side opposite to the viewpoint is long. Change the side length to correct the keystone.

  FIG. 5B shows an example of image processing performed by the distortion correction processing unit 9 when the viewer is viewing the screen from the left direction 42b. The distortion correction processing unit 9 shortens the left side where the viewpoints of the foreground image 29b and the foreground image 30b exist and shortens the right side opposite to the viewpoint in order to obtain a natural image without distortion corresponding to the viewpoint direction 42b. The keystone is corrected by changing the length of each side to make it longer.

  FIG. 5C shows an example of image processing performed by the distortion correction processing unit 9 when the viewer is viewing the screen from the right direction 42c. In order to obtain a natural image without distortion corresponding to the viewpoint direction 42c, the distortion correction processing unit 9 shortens the right side where the viewpoints of the foreground image 29c and the foreground image 30c exist and shortens the left side opposite to the viewpoint. The keystone is corrected by changing the length of each side to make it longer.

  The effect of the distortion correction processing by the distortion correction processing unit 9 is particularly noticeable when a superimposed image is displayed on a large screen display. FIG. 6 is an explanatory diagram showing the distortion correction processing effect by the stereoscopic image display apparatus according to Embodiment 1 of the present invention. The examples shown in FIGS. 6A to 6E all assume a case where an image is displayed on a large screen display. FIG. 6A shows an example in which the superimposed image displayed on the screen is viewed from the front direction. When the distortion correction processing by the distortion correction processing unit 9 is not performed, when the screen is viewed from the left direction or the right direction, the superimposed image is as shown in FIG. 6B or 6C, and what is displayed on the screen. It will be deformed so that it cannot be recognized.

  On the other hand, when the distortion correction processing by the distortion correction processing unit 9 is performed, when the screen is viewed from the left direction or the right direction, the superimposed image is as shown in FIG. 6 (d) or FIG. 6 (e). Although the range in which images for distant, middle, and foreground are displayed on the screen is reduced, it is possible to realize a display close to the screen display facing the viewer's viewpoint direction.

  In the above example, the case where the superimposed image is viewed from the left-right direction of the screen has been described. However, the distortion correction processing unit 9 similarly performs the distortion correction processing when viewed from the vertical direction. A viewer who views a large-screen display installed on the wall surface of the atrium hall, for example, rides on an elevator facing the screen and moves in the vertical direction. By displaying a superimposed image that has been subjected to distortion correction processing for such a viewer, an image that is close to the screen display when the viewer is facing the screen is always provided even though the viewer is moving vertically can do.

  The image rendering unit 10 for each level adds a filter effect in real time to the image data for the distant view, the foreground, and the foreground based on the parameters such as the image display coordinates calculated by the 3D effect calculation unit 3 and the image rendering. 3D effect is realized.

  The scroll display processing unit 11 performs a process of moving the image display area to either the left or the right. FIG. 7 is an explanatory diagram for explaining scroll display processing by the stereoscopic image display apparatus according to Embodiment 1 of the present invention. As shown in FIG. 7, the scroll display processing unit 11 moves the distant view image 19 in either the left or right direction 43 with respect to the image range 20 displayed on the screen display unit 14. The moving direction is matched with the viewer's viewpoint movement detected by the viewer viewpoint recognition unit 2. That is, when the viewer is moving from the right to the left with respect to the screen, the image range 20 is moved from the right to the left of the distant view image 19 to scroll the image.

  When scroll display processing is performed by the scroll display processing unit 11, an image 19 having a sufficient size with respect to the image range 20 displayed on the screen display unit 14 is used. The image range 20 in this case is one frame of actual display. In addition to the processing of the 3D effect calculation unit 3 that aligns the image with the viewer's viewpoint, the scroll display process can make a live screen display rich in change even with a small amount of image data, without getting the viewer bored.

  Here, the scroll display process performed mainly on the image data displayed on the display hierarchy for distant views is described as an example, but the process may be performed on the image data displayed on other display hierarchy. The scroll display processing unit 11 also performs scroll display processing so that the cylindrical image generated by the cylindrical image generation unit 6 is moved in the left-right direction with respect to the screen. When a cylindrical image is used, the image range 20 is scrolled in an endless manner as indicated by an arrow 44 because there is no end of the image, unlike simply moving the image range 20 from the beginning to the end of the distant view image 19. .

  The image boundary processing unit 13 performs a blurring process on the masked image by changing the transparency of the boundary portion in the mask data, that is, the edge portion, and prevents the image superimposed on each display layer from floating from the surroundings. , Blend into the background.

  As shown in FIG. 2, the superimposed display processing unit 12 superimposes the images for distant view, foreground and foreground that have been processed by the above units in the order based on the coordinate data managed by the hierarchical coordinate management unit 4. A simple superimposed image. At that time, the superimposition display processing unit 12 superimposes an area designated by the mask data on the image having the mask data.

  The screen display unit 14 is configured with a display or the like, and displays the superimposed image input from the superimposed display processing unit 12.

  Next, the operation of the stereoscopic image display device will be described. Image data photographed by the imaging unit 8 is input to the image recording unit 5, and the image recording unit 5 records these image data. The hierarchical coordinate management unit 4 manages the coordinate data of the display hierarchy for displaying each image data.

  The image recording unit 5 outputs the image data for the middle scene and the foreground to the mask automatic generation unit 7. The automatic mask generation unit 7 generates, for each image data, mask data designating an area to be displayed on the screen and a transparent area, and outputs the mask data to the image recording unit 5 together with the image data. Further, the image recording unit 5 outputs the distant view image data to the cylindrical image generation unit 6. The cylindrical image generation unit 6 generates a cylindrical image 24 shown in FIG. 4C and outputs it to the image recording unit 5.

  On the other hand, the detection value detected by the sensor 1 is output to the viewer viewpoint recognition unit 2. The viewer viewpoint recognition unit 2 calculates the three-dimensional relative coordinates indicating the position and viewpoint direction of the viewer existing in the vicinity of the stereoscopic image display device using the detection value. The obtained three-dimensional relative coordinate data is output to the 3D effect calculation unit 3.

  When the 3D relative coordinate data is input, the 3D effect calculation unit 3 calculates the relative coordinates where the viewer's viewpoint exists from the 3D relative coordinates and the 3D space coordinates of the screen display unit 14 prepared in advance. To do. Then, display coordinates and a display range as shown in FIG. 3 are calculated for each of the images for distant view, foreground, and foreground according to the direction in which the viewer views the screen. The 3D effect calculation unit 3 also calculates parameters related to image effects such as brightness, contrast, blue balance, and blurring effect of each image as 3D effects other than the display coordinates. Each parameter is output to the image rendering unit 10 for each layer through the distortion correction processing unit 9.

  Subsequently, using the relative coordinate information calculated by the 3D effect calculation unit 3 by the distortion correction processing unit 9, as shown in FIG. 5, for distant view, foreground and foreground according to the viewpoint direction of the viewer Each image is subjected to distortion correction processing by projective transformation or the like. Each image data subjected to the distortion correction processing is output to the image rendering unit 10 for each layer.

  Next, each stage effect unit 10 inputs from the distortion correction processing unit 9 based on parameters such as the size of each display layer calculated by the 3D effect calculation unit 3, the coordinates for displaying the image, the display range of the image, and the image effect. A 3D effect is realized by applying a filter effect in real time to each image data for a far view, a middle view, and a near view.

  The scroll display processing unit 11 scrolls the cylindrical image that is the image data for distant view according to the viewpoint direction. Further, the image boundary processing unit 13 performs a blurring process on the boundary portion of the mask data with respect to the image data for the middle scene and the foreground that are the images with the mask. The superimposed display processing unit 12 superimposes each image data subjected to these processes and displays it on the screen display unit 14. The stereoscopic image display apparatus performs the above-described series of operations within one frame processing time, and performs superimposed image processing in accordance with the viewer's viewpoint movement.

  In the above description, the stereoscopic image display device is configured to include all of the cylindrical image generation unit 6, the mask automatic generation unit 7, the imaging unit 8, the scroll display processing unit 11, and the image boundary processing unit 13. However, the present invention is not limited to this, and each part may be appropriately combined.

  In addition, when a large screen display is used as the stereoscopic image display device and the sensor 1 configured with a camera for detecting the viewer's viewpoint monitors a wide range, a plurality of viewers may be detected. In that case, using the image data acquired by the sensor 1, the viewer viewpoint recognition unit 2 recognizes the position of each of the plurality of viewers and their attributes (gender, age, etc.), and satisfies predetermined conditions. A matching viewer's viewpoint may be selected. For example, the viewer viewpoint recognizing unit 2 calculates the three-dimensional relative coordinates on the basis of the attribute recognition data, regarding the viewer at the center of the group with the largest number of people as the representative viewpoint. Alternatively, when the image displayed on the stereoscopic image display device is CM content or the like, for example, if it is a CM for women, the viewpoint of a woman is selected from a plurality of detected viewers, and three-dimensional relative coordinates are calculated. Then, the viewpoint of the viewer having the attribute according to the display content is selected to calculate the three-dimensional relative coordinates.

  As described above, according to the first embodiment, the hierarchical coordinate management unit 4 that sets the relative depth relation in the three-dimensional space to the plurality of display hierarchies that respectively display images by the coordinate data, and the output of the sensor 1 Viewer viewpoint recognition unit 2 that detects the position of the viewer's viewpoint from the value, and an image to be displayed for each display hierarchy based on the relative coordinates obtained by converting the detected viewpoint position into the position of the viewpoint on the display hierarchy A 3D effect calculation unit 3 that adjusts the display position, a distortion correction processing unit 9 that deforms an image to be displayed for each display layer according to projective transformation according to relative coordinates, and a depth relationship based on the coordinate data in each display layer An overlay display processing unit 12 that superimposes an image to be displayed is provided. Therefore, it is possible to obtain a stereoscopic image display device that can display an image with a sense of depth in an easy-to-view manner from the top, bottom, left, and right directions of the display by performing display according to the viewpoint of the moving person.

  Further, according to the first embodiment, the cylindrical image generation unit 6 that generates a cylindrical image that connects the start frame and the end frame of the image, and the scroll display processing unit that scrolls the cylindrical image as the viewpoint moves. 11 and so on. Therefore, it is possible to perform a live screen display rich in changes even with a small amount of image data without getting viewers bored.

  In addition, according to the first embodiment, the mask automatic generation unit 7 that detects the contour line of the object in the image and generates mask data that specifies the area that is not displayed on the screen and the area that is displayed is provided. When the unit 12 superimposes the image, the area not displayed on the screen specified by the mask data is transmitted. Therefore, layered images can be superimposed and displayed using automatically generated mask data.

  In addition, according to the first embodiment, the image boundary processing unit 13 that changes the transparency of the peripheral portion of the area displayed on the screen specified by the mask data is provided. For this reason, it is possible to prevent the image superimposed on each display layer from floating from the surroundings, and to blend in the background.

  Further, according to the first embodiment, the imaging unit 8 that captures an image is provided. For this reason, the video can be displayed on the screen almost simultaneously with the photographing, and a superimposed image using an actual image such as a natural scenery as a distant view image can be displayed.

  Further, according to the first embodiment, the 3D effect calculation unit 3 performs color tone correction that enhances the blue color of the image in the order of the display hierarchy for the foreground, the background, and the background. In addition, correction for increasing the degree of blurring of the image is performed in the order of the display hierarchy for the foreground, the middle, and the far view. Therefore, a perspective screen display can be performed.

It is a block diagram which shows the whole structure of the three-dimensional image display apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows an example of the screen which the three-dimensional image display apparatus concerning Embodiment 1 of this invention displays. It is explanatory drawing explaining the 3D effect calculation process by the three-dimensional image display apparatus concerning Embodiment 1 of this invention. It is explanatory drawing explaining the cylindrical image generation process of the three-dimensional image display apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing explaining the distortion correction process by the three-dimensional image display apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the distortion correction process effect by the three-dimensional image display apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing explaining the scroll display process by the three-dimensional image display apparatus which concerns on Embodiment 1 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Sensor, 2 Viewer viewpoint recognition, 3 3D effect calculation part, 4 Hierarchical coordinate management part, 5 Image recording part, 6 Cylindrical image generation part, 7 Mask automatic generation part, 8 Imaging part, 9 Distortion correction process part, 10 Image rendering unit for each layer, 11 Scroll display processing unit, 12 Superimposed display processing unit, 13 Image boundary processing unit, 14 Screen display unit, 15, 19 Distant view image, 16 Middle view image, 17 Near view image, 18 Superimposed image 20 image display range, 21 first frame, 22 end frame, 23 joint image, 24 cylindrical image, 25a, 25b, 25c 3D processed far view image, 26a, 26b, 26c 3D processed mid view image, 27a, 27b, 27c 3D-processed foreground image, 28a, 28b, 28c Distortion corrected distant view image, 29a, 29b, 29c Distortion corrected middle scene Image, 30a, 30b, 30c Distorted image for near view, 41a, 42a Viewpoint of viewer in front, 41b, 42b Viewpoint of viewer in left direction, 41c, 42c Viewpoint of viewer in right direction, 43 Scroll movement of image, 44 Scroll movement of cylindrical image.

Claims (7)

A hierarchical coordinate management unit that sets a relative depth relationship in a three-dimensional space to a plurality of display hierarchies that respectively display a two-dimensional image;
A viewer viewpoint recognition unit for detecting the position of the viewer's viewpoint;
A 3D effect calculation unit that adjusts the display position of a two-dimensional image to be displayed for each display layer based on the viewpoint coordinates obtained by converting the detected position of the viewpoint into the position of the viewpoint on the display layer;
A distortion correction processing unit that transforms the two-dimensional image to be displayed by projective transformation in accordance with the viewpoint coordinates for each display hierarchy;
A superimposed display processing unit that superimposes a two-dimensional image to be displayed on each display layer in the order of the depth relationship set by the hierarchical coordinate management unit ;
A cylindrical image generation unit that generates a cylindrical image connecting a start frame and an end frame of any two-dimensional image among the two-dimensional images displayed for each display layer;
A stereoscopic image display device comprising: a scroll display processing unit that scrolls the cylindrical image in accordance with the movement of the viewpoint coordinates . An automatic mask generation unit that detects a contour line of an object in a two-dimensional image and creates mask data that specifies a region that is not displayed on the screen and a region that is displayed using the contour line;
The stereoscopic image display apparatus according to claim 1 , wherein the superimposed display processing unit transmits a two-dimensional image region that is not displayed on the screen specified by the mask data. The stereoscopic image display apparatus according to claim 2, further comprising an image boundary processing unit that changes transparency of a peripheral portion of an area to be displayed on a screen specified by mask data in a two-dimensional image. The stereoscopic image display device according to any one of claims 1 to 3 , further comprising an imaging unit that captures an image used for a two-dimensional image. 3D effect calculation unit, of claim 1, wherein the performing color correction to enhance the blue two-dimensional image toward the display hierarchy positioned on the far side from the display hierarchy positioned on the screen front side of claim 4 The stereoscopic image display device according to any one of the above. 3D effect calculation unit, of claim 1, wherein the corrected to enhance the blurring degree of the two-dimensional image toward the display hierarchy positioned on the far side from the display hierarchy positioned on the screen front side of claim 5 The stereoscopic image display device according to any one of the above. Hierarchical coordinate management unit, the three display hierarchy, among the foreground stratifying toward the screen front side to the back side, from claim 1, characterized in that setting the depth relationship between middle ground and distant of claim 6 The stereoscopic image display device according to any one of the above.

Images