JP2015142148A - Image processing device, display device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of displaying an easy-to-view stereoscopic image on a display unit.SOLUTION: The image processing device comprises: an input unit which receives an inputted image; and a generator which, on the basis of a portion of the inputted image, generates first information for displaying an image on a first display unit and, on the basis of at least the other portion of the inputted image, generates second information for displaying an image on a second display unit disposed at a depth position different from that of the first display unit.

Description

  The present invention relates to an image processing device, a display device, and a program.

  A two-dimensional image is displayed on each of a plurality of display surfaces having different depth positions, and the object to be displayed is line of sight with respect to the plurality of display surfaces at different depth positions as viewed from the observer. There is a known three-dimensional display method that generates a three-dimensional image by generating a two-dimensional image projected from the direction and assigning a luminance ratio according to each depth position for each display surface of the generated two-dimensional image. (See Patent Document 1).

Japanese Patent Laid-Open No. 2001-112025

In the conventional method, there are cases where a stereoscopic image that is easy to visually recognize cannot be displayed.
An object of the present invention is to display a three-dimensional image that is easy to visually recognize on a display unit.

One embodiment of the present invention generates, based on an input unit to which an input image is input, first information for displaying an image on a first display unit based on a part of the input image, and at least the input A generating unit that generates second information for displaying an image on a second display unit arranged at a depth position different from that of the first display unit, based on another part excluding the part from the image And an image processing apparatus.
According to another aspect of the present invention, there is provided an image processing apparatus according to an aspect, the first display unit that displays an image based on first information generated by the image processing apparatus, and the image processing apparatus. And a second display unit that displays an image based on the generated second information.
In another aspect of the present invention, first information for causing a computer that controls the image processing apparatus to input an input image and causing the first display unit to display the image based on a part of the input image. And a second display unit configured to display an image on a second display unit arranged at a depth position different from the first display unit based on at least another part obtained by removing the part from the input image. It is a program that generates the information.

  According to one embodiment of the present invention, a stereoscopic image that is easy to visually recognize can be displayed on the display unit.

It is a figure which shows an example of a structure of the display apparatus 1 containing the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows typically an example of the pixel structure of the 1st display part which concerns on 1st Embodiment, and a 2nd display part. It is a figure which shows a mode that the 1st display part and the 2nd display part were seen from the user side in FIG. It is explanatory drawing for demonstrating the process by the distribution part which concerns on 1st Embodiment. It is a figure which shows typically a mode that depth information is reflected on an image. It is a figure which shows the relationship of the depth position between the display surface of the 1st display part 10, and the display surface of a 2nd display part. It is a figure which shows an example of the depth position of the stereo image VI. It is an example of the flowchart which shows the flow of the process performed by the display information generation part of 1st Embodiment. It is explanatory drawing for demonstrating the process which changes the grade of thinning. It is a figure which shows the other example of the pixel structure which a display apparatus has. It is a figure which shows typically the pixel structure of the display apparatus which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating the process by the distribution part which concerns on 2nd Embodiment. It is an example of the flowchart which shows the flow of the process performed by the display information generation part of 1st Embodiment. It is explanatory drawing for demonstrating the change of the arrangement | positioning according to the attitude | position of a display apparatus.

  Hereinafter, embodiments of an image processing device, a display device, and a program according to the present invention will be described with reference to the drawings. Hereinafter, in the description of each drawing, an XYZ rectangular coordinate system is set, and the positional relationship of each part will be described with reference to this XYZ rectangular coordinate system. The direction in which the display device displays an image is the positive direction of the Z axis, and the directions perpendicular to each other on a plane perpendicular to the Z axis direction are the X axis direction and the Y axis direction, respectively. Here, the X-axis direction is the horizontal direction of the display device, and the Y-axis direction is the vertically upward direction of the display device 1.

<First Embodiment>
[Constitution]
FIG. 1 is a diagram illustrating an example of a configuration of a display device 1 including an image processing device 30 according to the first embodiment of the present invention. The display device 1 is used for applications such as a television receiver, a display device for a computer or a mobile phone, and a digital signage. The display device 1 is arranged so as to be on the near side when viewed from the user U who is the viewer, and so as to be on the far side from the first display unit 10 when viewed from the user U. And a second display unit 20 to be arranged.

  FIG. 2 is a diagram schematically illustrating an example of the pixel structure of the first display unit 10 and the second display unit 20 according to the first embodiment. FIG. 3 is a diagram illustrating a state in which the first display unit 10 and the second display unit 20 are viewed from the user U side in FIG. The first display unit 10 and the second display unit 20 each include a plurality of pixels PX (10) and PX (20) arranged in the X direction. The pixels PX (10) and PX (20) are pixel units each including, for example, three element pixels of R (red), G (green), and B (blue). The pixels PX (10) and PX (20) may be provided with a minimum unit number of R, G, and B three-element pixels, or may each include a plurality of R, G, and B three-element pixels. It may be a thing. Further, the pixels PX (10) and PX (20) may be composed of only one pixel.

  Portions other than the pixel PX (10) in the first display unit 10 are formed of a transparent member (for example, glass or resin). Such a part may be an empty space. The pixels PX (10) in each column arranged in the X direction in the first display unit 10 and the pixels PX (20) in each column arranged in the X direction in the second display unit 20 are, for example, in the Y direction in the figure. Is offset. For this reason, when viewed from the user U side, the display image by the pixel columns of the second display unit 20 can be visually recognized from the gaps between the pixel columns of the first display unit 10. When the pitch interval in the Y direction of PX (10) is K, the offset interval between PX (10) and PX (20) is, for example, K / 2. In FIGS. 2 and 3, the display surfaces of the first display unit 10 and the second display unit 20 are shown to have the same size, but actually, the line of sight from the user U is expanded. Therefore, the size of the display surface of the second display unit 20 may be slightly larger than the size of the display surface of the first display unit 10.

  Returning to FIG. 1, the functional configuration of the image processing apparatus 30 will be described. The image processing apparatus 30 includes an input buffer 32, a pixel structure storage unit 34, a display information generation unit 40, a first output buffer 50, and a second output buffer 52. The input buffer 32, the first output buffer 50, and the second output buffer 52 are, for example, a RAM (Random Access Memory) and a register, and the pixel structure storage unit 34 is a ROM (Read Only Memory), a flash memory, An HDD (Hard Disk Drive), a RAM, a register, and the like.

  An input image IM is input to the input buffer 32. In the input image IM, for example, information on the depth position (hereinafter, depth information) is given to each pixel. The depth information is information for designating how far the user U feels that the input image IM is present on the back side or the back side when the input image IM is displayed as a stereoscopic image. Depth information is represented by a continuous value or a discrete value from 0 to 100, for example, when the foremost side is defined as 0 and the farthest side is defined as 100 or the like. The input buffer 32 stores the inputted input image IM and depth information.

  The pixel structure storage unit 34 stores the pixel structures of the first display unit 10 and the second display unit 20 illustrated in FIGS. 2 and 3. The pixel structure storage unit 34 is, for example, the number of the pixels PX (10) in the first display unit 10 in the X direction and the Y direction, and the number of the pixels PX (20) in the second display unit 20 in the X direction and Y direction. , The pitch interval, the offset interval, and information for distinguishing whether to alternately display in the X direction or in the Y direction are stored. The example in FIGS. 2 to 4 corresponds to “display alternately in the Y direction”. The image processing apparatus 30 according to the present embodiment refers to the information stored in the pixel structure storage unit 34 so that the total number of pixels of the input image IM and the number of pixels PX (10) and PX (20) can be calculated. Even in a different case, image data can be generated by performing appropriate thinning processing.

  The display information generation unit 40 includes, for example, a thinning processing unit 42, a distribution unit 44, and a depth reflection unit 46. These functional units are, for example, software functional units that function when a CPU (Central Processing Unit) executes a program stored in a program memory (not shown). The present invention is not limited to this, and a hardware function unit such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit) may be used.

  The thinning processing unit 42 performs a thinning process on the input image IM. For example, the input image IM is 640 × 480 pixels, and the number of pixels PX (10) in the X direction and the number of pixels PX (20) in the X direction are both 320 in the pixel structure storage unit 34. It is assumed that the number of PX (10) in the Y direction and the number of pixels PX (20) in the Y direction are both 120, and that “display alternately in the Y direction” is stored. Is). In this case, the thinning processing unit 42 performs a thinning process for converting the input image IM of 640 × 480 pixels into 320 × 240 pixels. That is, the thinning processing unit 42 matches the number of pixels after thinning to, for example, the sum of the number of pixels PX (10) and the number of pixels PX (20). As a result, the superimposed image shown in FIG. 3 is an image in which pixels are thinned out from the input image IM. The thinning-out process is performed, for example, by obtaining an average of pixel values of four adjacent pixels when thinning down to 1/2 size in the vertical and horizontal directions as described above. Further, the thinning processing unit 42 performs a calculation for making the depth information given to each pixel of the input image IM correspond to the pixel after thinning, similarly to the pixel value. When the thinning processing unit 42 thins the image to 1/2 size in the vertical and horizontal directions as described above, the depth information corresponds to the pixels after thinning, for example, by obtaining the average of the depth information of adjacent four pixels.

  The distribution unit 44 distributes the pixel values of the image after the thinning process to the pixels PX (10) of the first display unit 10 and the pixels PX (20) of the second display unit 20. FIG. 4 is an explanatory diagram for explaining processing by the distribution unit 44 according to the first embodiment. As shown in FIG. 4, the distribution unit 44 calculates the pixel value of each pixel (1-1 to 1-n, 2-1 to 2n,...) Of the image IM * after the thinning process for each row. 10), distributing to the pixels PX (20). Through such processing, the distribution unit 44 generates information (first information) for displaying an image on the first display unit 10 based on “part” of the input image IM * after thinning, and at least Based on the portion obtained by excluding the “part” from the thinned input image IM *, the second display unit 20 generates information (second information) for displaying the image.

  The depth reflecting unit 46 reflects the depth information on the pixel values distributed to the pixels PX (10) and PX (20). The depth reflecting unit 46 becomes brighter as the value indicated by the depth information is smaller (that is, the pixel value distributed to the pixel PX (10) is set to be displayed closer to the front). If the pixel value is distributed to the pixel PX (20), the pixel value is corrected to be darker. On the contrary, the depth reflecting unit 46 is a pixel value distributed to the pixel PX (10) as the value indicated by the depth information is larger (that is, as the depth information is set to be displayed on the deeper side). Then, the pixel value is corrected so as to be darker, and if the pixel value is distributed to the pixel PX (20), the pixel value is corrected so as to be brighter. Any correction method for the pixel value based on the depth information may be used as long as the above tendency is realized (the pixel PX (10) is brightened on the near side and the pixel PX (20) is brightened on the far side). May be.

  When the processing by the depth reflecting unit 46 is performed, the pixel value (that is, the first information) assigned to the pixel PX (10) of the first display unit 10 is stored in the first output buffer 50, and the second The pixel value (that is, the second information) assigned to the pixel PX (20) of the display unit 20 is stored in the second output buffer 52. The first display unit 10 displays an image based on the pixel value stored in the first output buffer 50, and the second display unit 20 displays the image based on the pixel value stored in the second output buffer 52. Is displayed. Note that the processing in the order of the thinning processing unit 42 → the distributing unit 44 → the depth reflecting unit 46 is merely an example, and the order of these processes may be arbitrarily changed.

  Thereby, the image processing device 30 and the display device 1 can display a stereoscopic image that is easy to visually recognize on the first display unit 10 and the second display unit 20. As shown in FIG. 3, the images displayed by the first display unit 10 and the second display unit 20 are in the Y direction when viewed from a predetermined direction (for example, the direction facing the first display unit 10). , The image by the pixel PX (10) and the image by the pixel PX (20) appear to alternate. For this reason, for an image element that is desired to be displayed at the depth position on the near side as viewed from the user U, the pixel PX (10) is displayed brighter and the pixel PX (20) is displayed darker, thereby realizing an easy-to-view stereoscopic image. be able to. FIG. 5 is a diagram schematically showing how the depth information is reflected in the image. As shown in FIG. 5, the depth reflection unit 46, for example, in the face area F of the person whose depth information is set on the near side, makes the pixel PX (10) brighter than the distributed pixels. The pixel PX (20) is corrected to be darker. Further, the depth reflecting unit 46 darkens the pixel PX (10) with respect to the distributed pixel and the pixel PX (20) with respect to the distributed pixel for the mountain region M in which the depth information is set on the back side. To brighten.

  Here, the principle by which the display device 1 can realize a stereoscopic image will be described. FIG. 6 is a diagram illustrating the relationship of the depth position between the display surface of the first display unit 10 and the display surface of the second display unit. As illustrated, the pixel PX (10) of the first display unit 10 emits light L1 from the depth position Z1 in the (+ Z) direction, that is, in the direction of the viewpoint position VP. On the other hand, the pixel PX (20) of the second display unit 20 emits light L2 from the depth position Z2 in the (+ Z) direction, that is, in the direction of the viewpoint position VP. Further, the light-transmitting part of the first display unit 10 transmits the light L1 from the pixel PX (10) of the second display unit 10 in the (+ Z) direction, that is, in the direction of the viewpoint position VP.

  When the user U views an image of the light L1 from the viewpoint position Zvp, a subjective contour may be formed in the user U's brain. For example, when the user U views the image by the light L1 from the viewpoint position Zvp, the outline of the display target included in the image distributed to the first display unit 10 in the input image IM is formed in the user U's brain. May be. When the subjective contour is formed in the brain of the user U in this way, the user U can recognize the contour of the image displayed by the pixel PX (10). That is, even when the interval between the pixels PX (10) is relatively wide, the user U can recognize the contour of the image displayed by the pixel PX (10).

  Similarly, when the user U views the image of the light L2 from the viewpoint position Zvp, a subjective contour may be formed in the user U's brain. For example, when the user U views the image by the light L2 from the viewpoint position Zvp, the outline of the display target included in the image distributed to the second display unit 20 in the input image IM is formed in the brain of the user U. May be. When the subjective contour is formed in the brain of the user U in this way, the user U can recognize the contour of the image displayed by the pixel PX (20). That is, even when the interval between the pixels PX (20) is relatively wide, the user U can recognize the contour of the image displayed by the pixel PX (20). As a result, when the (−Z) direction is viewed from the viewpoint position VP, an image (synthesized image) in which the image by the light L1 and the image by the light L2 are combined is generated.

  FIG. 7 is a diagram illustrating an example of the depth position of the stereoscopic image VI. There is a difference as illustrated between the depth position Z1 of the display surface of the first display unit 10 and the depth position Z2 of the display surface of the second display unit 20. For this reason, when the user U sees the first display unit 10 and the second display unit 20 in an overlapping manner from the viewpoint position Zvp, parallax (both parallaxes between the subjective contour by the light L1 and the subjective contour by the light L2) Eye parallax) occurs. The user U recognizes the combined image of the light L1 and the light L2 as a stereoscopic image VI based on the binocular parallax. The stereoscopic image VI is generated at a depth position (for example, a depth position Zvi) between the depth position Z1 of the first display unit 10 and the depth position Z2 of the second display unit 20.

The depth position where the stereoscopic image VI is generated can be controlled by the light emission intensity of the pixel PX (10) and the light emission intensity of the pixel PX (20) as shown in the following equation (1). For example, when it is desired to show a certain display object to the near side, the light emission intensity of the pixel PX (10) that displays the display object is relatively increased, and the light emission of the pixel PX (10) that displays the display object. What is necessary is just to make intensity | strength relatively weak.
Zvi = {light emission intensity of the pixel PX (10) / (light emission intensity of the pixel PX (10) + light emission intensity of the pixel PX (20)) × | Z1-Z2 |} + Z2 (1)

  In the display device 1 according to the present embodiment, since the image displayed on the second display unit 20 is displayed from the gap between the images displayed on the first display unit 10, even if a slight line-of-sight shift occurs, the second As long as the image displayed on the display unit 20 is visible, a stereoscopic image is projected on the eyes of the user U. Therefore, the image processing device 30 and the display device 1 can increase the tolerance for a shift in the line-of-sight direction. Note that this embodiment does not exclude that the pixel PX (10) of the first display unit 10 has light transmittance.

  Further, as shown in FIG. 3, the images displayed by the first display unit 10 and the second display unit 20 are, for example, the pixels PX (10) and PX ( 20) is displayed with a certain gap in the X and Y directions. For this reason, the image processing apparatus 30 and the display apparatus 1 can further improve the tolerance with respect to the shift | offset | difference of a gaze direction. Further, the processing load and power consumption can be reduced by the thinning process.

  In addition, the configuration is not limited to this, and the pixels PX (10) and PX (20) may be densely arranged in the X direction and the Y direction without gaps in the overlapped state. In this case, the significance of the thinning process is to correct the input image IM to a size corresponding to the resolution of the display device 1.

[Processing flow]
FIG. 8 is an example of a flowchart illustrating a flow of processing executed by the display information generation unit 40 of the first embodiment. The process of the flowchart in FIG. 8 is executed for each image of one frame, for example.

  First, the thinning processing unit 42 performs a thinning process on the input image IM (step S100). Next, the distribution unit 44 sets initial values of the parameters x and y to 1, and sets the maximum values MAX (x) and MAX (y) of x and y (step S102). The maximum values MAX (x) and MAX (y) are values respectively corresponding to the number of pixels in the horizontal direction and the number of pixels in the vertical direction in the superimposed image shown in FIG. The maximum values MAX (x) and MAX (y) may be stored in the pixel structure storage unit 34 as prescribed values.

  Next, the distribution unit 44 extracts the pixel value of the pixel at the coordinate (x, y) in the input image IM * after thinning (step S104), and determines whether or not the parameter y is an odd number (step S106). . When the parameter y is an odd number, the distribution unit 44 assigns the pixel value extracted in step S104 to the pixel PX (10) at the coordinates (x, (y + 1) / 2) (step S108). On the other hand, when the parameter y is an even number, the distribution unit 44 assigns the pixel value extracted in step S104 to the pixel PX (20) at the coordinate (x, y / 2) (step S110). Note that the “coordinates” of PX (10) or PX (20) in steps S108 and S110 are not the coordinates in the superimposed image shown in FIG. 3, but the coordinates in the array of only PX (10), or PX (20 ) Only means the coordinates in the array.

  Next, the distribution unit 44 increases the parameter x by 1 (step S112), and determines whether the parameter x exceeds the maximum value MAX (x) (step S114). If the parameter x does not exceed the maximum value MAX (x), the process returns to step S104. On the other hand, when the parameter x exceeds the maximum value MAX (x), the distribution unit 44 returns the parameter x to the initial value 1 and increases the parameter y by 1 (step S116). Next, the distribution unit 44 determines whether or not the parameter y exceeds the maximum value MAX (y) (step S118). If the parameter y does not exceed the maximum value MAX (y), the process returns to step S104.

  When the parameter y exceeds the maximum value MAX (y), since the pixel values of all the pixels in the input image IM * after thinning have been distributed to PX (10) or PX (20), the depth reflecting unit 46 depth information Based on the above, the pixel value of each pixel is corrected (step S120), and the processing of this flowchart ends.

[Change thinning amount]
Here, the thinning processing unit 42 may change the degree of thinning according to the content of the image in the input image IM. FIG. 9 is an explanatory diagram for explaining processing for changing the degree of thinning. In FIG. 9, the area F-1 in the area F of the face includes a three-dimensional structure such as a nose and a mouth, so that the variation of the depth position is large, and the change in color and brightness is large. On the other hand, the region F-2 is a region where the variation of the depth position is moderate and the change in color and brightness is small. For this reason, even if the resolution of the region F-2 is lowered, it is considered that the influence on the visibility is small.

  Therefore, the thinning processing unit 42 changes the degree of thinning larger than the region F-1 for the region F-2 having the above tendency. Thereby, the tolerance with respect to the shift | offset | difference of a gaze direction about the area | region F-2 can further be improved, and processing load and power consumption can further be reduced.

  For example, when information indicating whether or not large thinning is allowed is added to the input image IM for each region or for each pixel, the thinning processing unit 42 determines the thinning amount with reference to the information. You can do it. In addition, the thinning processing unit 42 calculates the amount of variation in the depth position for each arbitrarily divided region, reduces the degree of thinning for a region with a large amount of variation, and increases the degree of thinning for a region with a small amount of variation. May be.

  According to the image processing device, the display device, and the program (hereinafter referred to as an image processing device or the like) of the first embodiment described above, it is possible to display (display) a stereoscopic image that is easy to visually recognize. In addition, according to the image processing apparatus or the like, it is possible to increase the tolerance for a shift in the line of sight direction. In addition, according to the image processing apparatus or the like, it is not necessary to arrange pixels in a portion through which the image by the second display unit 20 is transmitted, and a transparent member or the like can be used for the portion. For this reason, the image by the 2nd display part 20 can be permeate | transmitted more clearly, and the stereoscopic image which is easy to visually recognize can be displayed.

  Further, according to the image processing apparatus or the like of the present embodiment, the pixel structure storage unit 34 that stores the pixel structures of the first display unit 10 and the second display unit 20 is provided, and information stored therein is used. In order to perform the thinning process, for example, even if the number of pixels of the input image IM is different from the total number of the pixels PX (10) and PX (20), an appropriate thinning process is performed to obtain the image data. Can be generated.

  In addition, according to the image processing apparatus or the like of the present embodiment, a thinning process is performed to generate a display image, so that it is possible to further increase the tolerance for a shift in the viewing direction. Further, the processing load and power consumption can be reduced by the thinning process.

  Further, according to the image processing apparatus or the like of the present embodiment, by changing the degree of thinning according to the content of the image in the input image IM, it is possible to further increase the tolerance for the shift in the line-of-sight direction for the greatly thinned region. And the processing load and power consumption can be further reduced.

[Other examples of pixel arrangement]
The display device 1 according to the first embodiment is not limited to the pixel structure illustrated in FIGS. 2 and 3 and may have another pixel structure. FIGS. 10A to 10C are diagrams illustrating other examples of the pixel structure included in the display device 1. As shown in FIG. 10A, the first display unit 10 and the second display unit 20 may each include a plurality of pixels PX (10) and PX (20) arranged in the Y direction. As shown in FIG. 10 (B), a plurality of pixels PX (10) and PX (20) arranged in an oblique direction may be provided, or as shown in FIG. A plurality of pixels PX (10) and PX (20) that form a pattern may be provided. In these cases, the processing in the flowchart shown in FIG. 8 is replaced with processing in a mode suitable for each pixel structure. In addition, the first display unit 10 and the second display unit 20 do not necessarily have to display at the same time, and may display alternately by time division.

Second Embodiment
Hereinafter, an image processing device, a display device, and a program according to a second embodiment of the present invention will be described. FIG. 1 is used for the entire configuration, and the description of matters common to the first embodiment is omitted.

  In the display device 2 according to the second embodiment, the first display unit 10 can switch between pixels that are to be displayed and pixels that are not to be displayed, and the second display unit 20 is displayed by the first display unit 10. The image is displayed to the user U through the portion that is not performed. The image processing device 30 according to the second embodiment generates display information for the display device 2 having such an aspect.

  FIG. 11 is a diagram schematically illustrating a pixel structure of the display device 2 according to the second embodiment. In such a configuration, the first display unit 10 displays an image with, for example, the odd-numbered pixels PX (10) and makes the even-numbered pixels PX (10) light-transmitting. The image is displayed with the pixels PX (20) in even rows. As a result, a superimposed image similar to that shown in FIG. 3 can be displayed to the user U.

  Since the processing of the thinning processing unit 42 and the depth reflection unit 46 is the same as that of the first embodiment, detailed description thereof is omitted. Similarly to the first embodiment, the distribution unit 44 according to the second embodiment uses the pixel value of the image after the thinning process as the pixel PX (10) of the first display unit 10 and the pixel of the second display unit 20. Distribute to PX (20). FIG. 12 is an explanatory diagram for explaining processing by the distribution unit 44 according to the second embodiment. As shown in FIG. 12, the distribution unit 44 calculates the pixel value of each pixel of the image IM after the thinning process (represented simply by numbers 1 to 36 in the pixel in FIG. 12) for each row. Distribution to PX (10) and pixel PX (20). For example, the distribution unit 44 distributes the pixel values to the pixels PX (10) in the odd rows and distributes the pixels to the pixels PX (20) in the even rows. In this case, the distribution unit 44 gives an instruction to enter the light transmissive state to the pixels PX (10) in the even-numbered rows, and instructs to turn off the pixels PX (20) in the odd-numbered rows (for example, the pixel values are all zero). give. With this processing, the distribution unit 44 generates information (first information) for displaying an image on the first display unit 10 based on “part” of the input image IM * after thinning, and also performs thinning. Based on the portion excluding the “part” from the subsequent input image IM *, the second display unit 20 generates information (second information) for displaying the image.

  Thereafter, when processing by the depth reflection unit 46 is performed, the pixel value (that is, the first information) assigned to the pixel PX (10) of the first display unit 10 is stored in the first output buffer 50, A pixel value (that is, second information) assigned to the pixel PX (20) of the second display unit 20 is stored in the second output buffer 52. The first display unit 10 displays an image based on the pixel value stored in the first output buffer 50, and the second display unit 20 displays the image based on the pixel value stored in the second output buffer 52. Is displayed. Note that the processing in the order of the thinning processing unit 42 → the distributing unit 44 → the depth reflecting unit 46 is merely an example, and the order of these processes may be arbitrarily changed.

  Accordingly, the image processing device 30 and the display device 2 can display a stereoscopic image that is easy to visually recognize on the first display unit 10 and the second display unit 20. As shown in FIG. 3, the images displayed by the first display unit 10 and the second display unit 20 are in the Y direction when viewed from a predetermined direction (for example, the direction facing the first display unit 10). In FIG. 5, the image by the pixel PX (10) and the image by the pixel PX (20) appear alternately. For this reason, for an image element that is desired to be displayed at the depth position on the near side as viewed from the user U, the pixel PX (10) is displayed brighter and the pixel PX (20) is displayed darker, thereby realizing an easy-to-view stereoscopic image. be able to.

  FIG. 13 is an example of a flowchart showing a flow of processing executed by the display information generation unit 40 of the first embodiment. The process of the flowchart in FIG. 13 is executed for each image of one frame, for example.

  First, the thinning processing unit 42 performs a thinning process (step S200). Next, the distribution unit 44 sets initial values of the parameters x and y to 1, and sets the maximum values MAX (x) and MAX (y) of x and y (step S202). The maximum values MAX (x) and MAX (y) are values respectively corresponding to the number of pixels in the horizontal direction and the number of pixels in the vertical direction in the superimposed image shown in FIG. The maximum values MAX (x) and MAX (y) are set according to the pixel structures of the first display unit 10 and the second display unit 20 stored in the pixel structure storage unit 34. The maximum values MAX (x) and MAX (y) may be stored in the pixel structure storage unit 34 or the like as specified values.

  Next, the distribution unit 44 extracts the pixel value of the pixel at the coordinates (x, y) in the input image IM (step S204), and determines whether or not the parameter y is an odd number (step S206). When the parameter y is an odd number, the distribution unit 44 assigns the pixel value extracted in step S204 to the pixel PX (10) at the coordinate (x, y), and sets the pixel PX (20) at the coordinate (x, y) as a non-value. The display state is set (step S208). On the other hand, when the parameter y is an even number, the distribution unit 44 assigns the pixel value extracted in step S204 to the pixel PX (20) at the coordinate (x, y), and the pixel PX (10) at the coordinate (x, y). Is set to the light transmission state (step S210).

  Next, the distribution unit 44 increases the parameter x by 1 (step S212), and determines whether the parameter x exceeds the maximum value MAX (x) (step S214). If the parameter x does not exceed the maximum value MAX (x), the process returns to step S204. On the other hand, when the parameter x exceeds the maximum value MAX (x), the distribution unit 44 returns the parameter x to the initial value 1 and increases the parameter y by 1 (step S216). Next, the distribution unit 44 determines whether or not the parameter y exceeds the maximum value MAX (y) (step S218). If the parameter y does not exceed the maximum value MAX (y), the process returns to step S204.

  When the parameter y exceeds the maximum value MAX (y), since the pixel values of all the pixels in the input image IM * after thinning have been distributed to PX (10) or PX (20), the depth reflecting unit 46 depth information Based on the above, the pixel value of each pixel is corrected (step S220), and the process of this flowchart ends.

  The image processing apparatus 30 of the second embodiment may perform the same processing as that of the first embodiment for changing the thinning amount. In addition, regarding the change in the arrangement of pixels to be displayed, the second embodiment can adopt various modes illustrated in FIGS. 10A to 10C, and can also switch these modes. The flowchart shown in FIG. 13 is replaced with a process in a mode suitable for each arrangement according to which pixel among the pixels PX (10) is in the display state and which pixel is in the light transmission state.

  When the display device 2 is a device held by a person such as a mobile phone, the display device 2 may change the arrangement of pixels to be displayed according to the orientation of the display device 2 grasped by an acceleration sensor or the like. . 14A and 14B are explanatory diagrams for explaining a change in arrangement according to the attitude of the display device 2. As shown in FIG. 14A, when the display device 2 is in a vertically held state (a state in which the longitudinal direction is inclined with respect to a horizontal plane), pixels to be displayed on each display unit are arranged in the X direction. As shown in (B), when the display device 2 is in a side-holding state (a short side direction is inclined with respect to a horizontal plane), the display device 2 is arranged so that the pixels displayed on each display unit are arranged in the Y direction. You can switch. 14A and 14B may be reversed. Further, such pixel arrangement switching may be performed by user selection.

  According to the image processing device, the display device, and the program (hereinafter referred to as an image processing device, etc.) of the second embodiment described above, a stereoscopic image that is easy to view is displayed (displayed) as in the first embodiment. Can do. In addition, according to the image processing apparatus or the like, it is possible to increase the tolerance for a shift in the line of sight direction. Further, according to the image processing apparatus or the like, even if the number of pixels of the input image IM is different from the total number of the pixels PX (10) and PX (20), an appropriate thinning process is performed. Image data can be generated.

  In addition, according to the image processing apparatus or the like of the second embodiment, it is possible to flexibly switch between the pixel PX (10) that is displayed on the first display unit 10 and the pixel PX (10) that is in the light transmission state. . Thereby, it is possible to change the arrangement according to the orientation of the display device 1.

  A part of the image processing apparatus in the above-described embodiment may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The “computer system” here is a computer system built in the image processing apparatus, and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In this case, a volatile memory inside a computer system that serves as a server or a client may be included that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. Further, a part or all of the image processing apparatus in the above-described embodiment may be realized as an integrated circuit such as an LSI. Each functional block of the image processing apparatus may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on the technology may be used.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, In the range which does not deviate from the summary of this invention, various deformation | transformation and substitution Can be added. Moreover, what combined the above-mentioned embodiment suitably is also contained in this invention.

For example, although the display devices 1 and 2 are provided with the first display unit 10 and the second display unit 20, the display devices 1 and 2 include more display units and superimpose three or more display images. Also good.
In addition, the second display unit 20 displays only a portion that is visible through the portion that the first display unit 10 does not display (that is, displays only an image that fills the gap of the pixel PX (10)). However, the second display unit 20 may display the entire input image IM or the thinned input image IM *.

  DESCRIPTION OF SYMBOLS 1, 2 ... Display apparatus, 10 ... 1st display part, 20 ... 2nd display part, 30 ... Image processing apparatus, 32 ... Input buffer, 34 ... Pixel structure memory | storage part, 40 ... Display information generation part

Claims (10)

An input unit for inputting an input image;
Based on a part of the input image, first information for displaying the image on the first display unit is generated, and based on at least the other part excluding the part from the input image, the first information is displayed. A generating unit for generating second information for displaying an image on a second display unit arranged at a depth position different from the display unit of
An image processing apparatus comprising: The image processing apparatus according to claim 1,
A storage unit that stores information on the pixel structure of the first display unit and information on the pixel structure of the second display unit;
The generation unit generates the first information and the second information based on information stored in the storage unit.
Image processing device. The image processing apparatus according to claim 1 or 2,
In the first display unit, at least a portion where no image is displayed transmits light,
When viewed from a predetermined direction, the pixels of the first display unit and the pixels of the first display unit are visible so that an image displayed on the second display unit can be visually recognized from a gap between images displayed on the first display unit. The pixels of the second display section are arranged;
Image processing device. The image processing apparatus according to claim 1 or 2,
The first display unit is a display unit that transmits light at least where an image is not displayed;
The generation unit displays the second display unit from a gap between images displayed on the first display unit when the first display unit and the second display unit are viewed from a predetermined direction. Generating the first information and the second information so that the image is visible;
Image processing device. An image processing apparatus according to any one of claims 1 to 4,
The generation unit generates the first information and the second information based on information about a depth position attached to the input image.
Image processing device. The image processing apparatus according to claim 5, wherein
The generation unit displays the pixels of the first display unit brighter than the pixels of the second display unit for an image with the near depth position, and the second display unit for an image with a far depth position. Generating the first information and the second information in a tendency to display the pixels of the first display section brighter than the pixels of the first display unit.
Image processing device. The image processing apparatus according to any one of claims 1 to 6,
The generation unit is configured to display the first information so that an image obtained by superimposing an image displayed by the first display unit and an image displayed by the second display unit becomes an image obtained by thinning the input image. And generating the second information,
Image processing device. The image processing apparatus according to claim 7,
The generation unit changes the degree of thinning according to the content of the image in the input image.
Image processing device. The image processing apparatus according to any one of claims 1 to 8,
The first display unit for displaying an image based on the first information generated by the image processing apparatus;
The second display unit for displaying an image based on the second information generated by the image processing apparatus;
A display device comprising: In the computer that controls the image processing apparatus,
Input the input image,
Based on a part of the input image, first information for displaying an image on the first display unit is generated,
Generating at least a second information for displaying an image on a second display unit arranged at a depth position different from the first display unit, based on at least another part of the input image excluding the part. Let
program.

Images