JP2013183391A - Image processing apparatus, image processing method, electronic apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent stereoscopic vision from being reduced when a left eye image and a right eye image, which can be viewed as a three-dimensional image, are converted into a multi-parallax image.SOLUTION: If a viewer often views an image from a front face when a parallax amount of a left eye image P1 and a right eye image P5 is a parallax amount D, the parallax amount of multi-parallax images P2' and P3 and P3 and P4' on the left side and the right side in a front face direction is set to be a parallax amount 2D/6, and the parallax amount of multi-parallax images P1 and P2' and P4' and P5 on the left side and the right side in an end direction is set to be a parallax amount D/6. The multi-parallax image is generated by interpolation so that the parallax amount in a direction where possibility of viewing is high is increased and the parallax amount in a direction where viewing possibility is low is reduced. This technology can be applied to an electronic apparatus including a display function which can display a three-dimensional image.

Description

  The present technology relates to an image processing device, an image processing method, an electronic device, and a program, and in particular, converts a three-dimensional image including a set of parallax images composed of a left-eye image and a right-eye image into a multi-parallax image. Even so, the present invention relates to an image processing apparatus, an image processing method, an electronic device, and a program that can suppress a reduction in stereoscopic effect when viewed from a predetermined direction.

  A technique for enabling viewing of a three-dimensional image with the naked eye is becoming popular. In general, a technique for displaying a three-dimensional image with the naked eye is such that left-eye images and right-eye images are alternately arranged in a horizontal direction for each predetermined pixel column, and a parallax barrier or lenticular lens is used. The image and the right eye image can be viewed with the left eye and the right eye, respectively.

  The image for the left eye and the image for the right eye can be viewed from a specific viewpoint at a predetermined angle with respect to the display image. It could not be viewed as a three-dimensional image.

  Therefore, a technique has been proposed in which a multi-parallax image is generated from an image for the left eye and an image for the right eye by interpolation, and a three-dimensional image can be viewed from multiple viewpoints. For example, it is assumed that an image in which the parallax amount between the left-eye image and the right-eye image is halved is generated from the left-eye image and the right-eye image by interpolation. In this case, the original left-eye image and the newly generated image can be viewed as a three-dimensional image when viewed with the left eye and the right eye, respectively. Similarly, when a newly generated image and an original right-eye image are viewed with the left eye and the right eye, respectively, it is possible to view the image as a three-dimensional image. That is, it is possible to newly create a three-dimensional image by multi-parallax images from two directions by using three images including an image newly generated by interpolation.

  However, when generating a multi-parallax image from the left-eye image and the right-eye image in this way, the mutual parallax amount between the left-eye image and the right-eye image is divided, and according to the divided parallax amount A new image is generated by interpolation, and a multi-parallax image is generated. For this reason, in the generated multi-parallax image, as the number of viewpoints increases, the amount of individual parallax decreases, so that the stereoscopic effect obtained by viewing as a three-dimensional image may be reduced.

  In view of this, a technique has been proposed in which a high-quality three-dimensional image can be viewed even from multiple viewpoints by performing processing on content including a left-eye image and a right-eye image displayed as a three-dimensional image. (See Patent Document 1).

JP 2008-249809 A

  However, the technique described in Patent Document 1 aims to improve image quality by generating multi-parallax images having the same amount of parallax for some of the multi-parallax images. There was a fear that it could not be improved.

  The present technology has been made in view of such a situation, and in particular, the amount of parallax is unequal compared to the left-eye image and the right-eye image that can be viewed as a three-dimensional image from a predetermined direction with respect to the display screen. By dividing the image into two and generating a multi-parallax image by interpolation, it is possible to view a three-dimensional image from multiple viewpoints while suppressing a reduction in stereoscopic effect for the viewpoint from a specific direction. It is.

  The image processing device according to the first aspect of the present technology divides the parallax amount of a set of parallax images for viewing a three-dimensional image unequally, and sets one set for each non-uniformly divided parallax amount. A multi-parallax image conversion unit that converts the set of parallax images into a multi-parallax image by generating a multi-viewpoint image so that a parallax image is configured, and the multi-parallax image conversion unit And a display unit that synthesizes and displays the parallax image.

  The multi-parallax image conversion unit divides the parallax amount of a set of parallax images for viewing the three-dimensional image unevenly so that the parallax amount is large with respect to the front direction of the display unit. And generating a multi-viewpoint image so that a set of parallax images is formed for each non-uniformly divided amount of parallax, thereby converting the set of parallax images into a multi-parallax image. Can do.

  In the multi-parallax image conversion unit, the parallax amount of a set of parallax images for viewing the three-dimensional image is set so that the parallax amount increases in the direction in which the viewer viewing the display unit exists. By dividing non-uniformly and generating a multi-viewpoint image so that one set of parallax images is formed for each non-uniformly divided amount of parallax, the one set of parallax images is converted into a multi-parallax image. Can be converted.

  A viewer position detecting unit for detecting a position of the viewer who views the display unit with respect to the display unit can be further included, and the multi-parallax image converting unit includes the viewer position detecting unit. Based on the detected position information of the viewer, the amount of parallax of a set of parallax images for viewing the three-dimensional image is equal to the direction in which the viewer viewing the display unit is present. The set of parallax images is generated by generating a multi-viewpoint image so that the set of parallax images is formed for each of the non-uniformly divided parallax amounts. It can be made to convert into a multi parallax image.

  The viewer position detection unit picks up an image, detects the viewer's face image, and detects the position of the face image as the position of the viewer viewing the display unit with respect to the display unit. It can be detected or detected as a position relative to the display unit of a viewer who views the display unit based on an image of heat distribution detected by thermography.

  The image processing method according to the first aspect of the present technology divides the parallax amount of a set of parallax images for viewing a three-dimensional image unevenly, and sets one set for each non-uniformly divided parallax amount. By generating a multi-viewpoint image so that a parallax image is formed, a multi-parallax image conversion process that converts the set of parallax images into a multi-parallax image, and a multi-parallax image conversion process Display processing for combining and displaying parallax images.

  The program according to the first aspect of the present technology divides the parallax amount of a set of parallax images for viewing a three-dimensional image unevenly, and sets one parallax image for each non-uniformly divided parallax amount. By generating a multi-viewpoint image, a multi-parallax image conversion step that converts the set of parallax images into a multi-parallax image and a multi-parallax image conversion step that is converted by the processing of the multi-parallax image conversion step. Causing the computer to execute processing including a display step of combining and displaying the parallax images.

  The electronic device according to the second aspect of the present technology divides the amount of parallax of a set of parallax images for viewing a three-dimensional image unevenly, and sets one set of parallax for each non-uniformly divided amount of parallax. A multi-parallax image conversion unit that converts the set of parallax images into a multi-parallax image by generating a multi-viewpoint image so that an image is configured, and the multi-parallax image converted by the multi-parallax image conversion unit And a display unit for combining and displaying images.

  In the first or second aspect of the present technology, the parallax amount of one set of parallax images for viewing a three-dimensional image is divided unevenly, and one set is set for each non-uniformly divided parallax amount. By generating multi-viewpoint images so that parallax images are formed, the one set of parallax images is converted into multi-parallax images, and the converted multi-parallax images are combined and displayed.

  The image processing apparatus of the present technology may be an independent apparatus or a block that performs image processing.

  According to one aspect of the present technology, a set of parallax images is converted into a multi-parallax image so that viewing of a three-dimensional image from multiple viewpoints can be displayed. Can be suppressed.

It is a figure which shows the structural example of one Embodiment of the portable terminal to which this technique is applied. 4 is a flowchart for describing multi-parallax image display processing by the mobile terminal of FIG. 1. It is a figure explaining the division | segmentation method of the amount of parallax and the conventional amount of parallax. It is a figure explaining the division | segmentation method of the amount of parallax by the portable terminal of FIG. It is a figure explaining the principle by which the 3D image which can be viewed by the synthetic | combination method of a multi-parallax image and the synthesized image which can be viewed can be viewed. It is a figure which shows the structural example of embodiment which becomes a modification of a portable terminal. It is a figure showing an example of composition of an embodiment of a television receiver to which this art is applied. It is a flowchart explaining the multi parallax image display process by the television receiver of FIG. It is a figure explaining the multi parallax image which can be viewed by the multi parallax image display process by the television receiver of FIG. It is a figure explaining the multi parallax image which can be viewed by the multi parallax image display process by the television receiver of FIG. And FIG. 11 is a diagram illustrating a configuration example of a general-purpose personal computer.

Hereinafter, an embodiment of the present disclosure (hereinafter referred to as the present embodiment) will be described. The description will be given in the following order.
1. First embodiment (an example of a portable terminal that converts a three-dimensional image captured by an imaging unit into a multi-parallax image)
2. Modified example (an example of a portable terminal that converts a three-dimensional image composed of CG (Computer Graphics) into a multi-parallax image)
3. Second Embodiment (Example of a television receiver that specifies a viewer position and converts a three-dimensional image received from a broadcast wave into a multi-parallax image corresponding to the viewer position)

<1. First Embodiment>
[Configuration example of mobile terminal]
FIG. 1 shows a configuration example of an embodiment of a mobile terminal to which the present technology is applied. The mobile terminal 11 in FIG. 1 is, for example, a mobile phone or a PDA (Personal Digital Assistant), which captures and stores a three-dimensional image and displays the stored three-dimensional image in a state that can be viewed with the naked eye. To do.

  The mobile terminal 11 includes a control unit 21, an input unit 22, an imaging unit 23, a display control unit 24, and a display unit 25.

  The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) (not shown), and the CPU expands programs and data stored in the ROM into the RAM. Various processes are executed to control the entire operation of the mobile terminal 11. The control unit 21 captures a three-dimensional image by controlling the imaging unit 23 based on an operation signal supplied from the input unit 22 when the input unit 22 including a keyboard and a touch panel is operated by the user. In addition, the captured three-dimensional image data is supplied to the display control unit 24.

  The imaging unit 23 includes, for example, a compound eye CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like for capturing an image for the left eye and an image for the right eye. Based on an instruction from the control unit 21, the imaging unit 23 captures a left-eye image and a right-eye image that can be viewed as a three-dimensional image so as to generate parallax, and supplies the parallax to the control unit 21. To do. Here, the parallax indicates a pixel shift in the display area of the same subject in each of the left-eye image and the right-eye image. In addition, the number of pixels indicating the amount of shift of the display area of the subject due to the parallax between the left-eye image and the right-eye image is referred to as a parallax amount.

  The display control unit 24 acquires and stores a three-dimensional image captured by the imaging unit 23 via the control unit 21, that is, a two-parallax image including a left-eye image and a right-eye image. In addition, the display control unit 24 reads the stored three-dimensional image, converts it into a viewable image as a multi-parallax three-dimensional image, and displays a display unit composed of an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) 25.

  More specifically, the display control unit 24 includes a three-dimensional image storage unit 31, a multi parallax image conversion unit 32, and a multi parallax image synthesis unit 33. The three-dimensional image storage unit 31 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores a left-eye image and a right-eye image that form a three-dimensional image in a predetermined image compression format. Remember. The three-dimensional image storage unit 31 reads out the stored three-dimensional image and supplies it to the multi-parallax image conversion unit 32.

  The multi-parallax image conversion unit 32 sequentially reads out the three-dimensional images stored in the three-dimensional image storage unit 31, and converts the left-eye image and the right-eye image constituting the three-dimensional image into a three-dimensional image from multiple viewpoints. Is converted into a multi-parallax image that enables viewing, and is supplied to the multi-parallax image combining unit 33. At this time, the multi-parallax image conversion unit 32 divides the parallax amount of the left-eye image and the right-eye image so that only the region close to the front direction is enlarged, and the multi-parallax image corresponding to the divided parallax amount is obtained. Generate. By generating a multi-parallax image in this way, a user using the mobile terminal 11 can view a three-dimensional image without reducing the stereoscopic effect in the front direction of the display unit 25 of the mobile terminal 11 that is normally viewed. In addition, it is possible to realize viewing of a three-dimensional image from multiple viewpoints.

  The multi-parallax image synthesizing unit 33 synthesizes an image so that the multi-parallax image can be displayed in the horizontal direction for each predetermined pixel width based on the information of the multi-parallax image by the multi-parallax image converting unit 32, and the display unit 25 Supply and display.

  The display unit 25 is provided with a separation filter 25a made of a parallax barrier, a lenticular lens, or the like. The separation filter 25a can view a multi-parallax image obtained by interpolating the left-eye image and the right-eye image constituting the three-dimensional image as an image having each parallax amount at each viewpoint. To separate and permeate. With such display processing, viewing of a three-dimensional image from multiple viewpoints is realized, and the amount of parallax with respect to the front direction of the display unit 25 that is the viewing direction of the viewer who is the main user of the mobile terminal 11 Therefore, the user can view from multiple viewpoints without impairing the stereoscopic effect due to the three-dimensional image.

[Multi-parallax image display processing by mobile terminal]
Next, multi-parallax image display processing by the mobile terminal 11 of FIG. 1 will be described with reference to the flowchart of FIG.

  In step S <b> 1, based on an instruction from the control unit 21, the imaging unit 23 captures an image, captures a left-eye image and a right-eye image that form a three-dimensional image, and supplies the images to the control unit 21. The control unit 21 supplies the left eye image and the right eye image constituting the three-dimensional image supplied from the imaging unit 23 to the display control unit 24.

  In step S <b> 2, the three-dimensional image storage unit 31 of the display control unit 24 corresponds to the left-eye image and the right-eye image constituting the three-dimensional image supplied from the imaging unit 23 via the control unit 21. Add and remember.

  In step S3, the multi-parallax image conversion unit 32 reads out the left-eye image and the right-eye image, which are three-dimensional images stored in the three-dimensional image storage unit 31, and converts them into a multi-parallax image using them. To do. More specifically, the multi-parallax image conversion unit 32 increases the amount of parallax between the left-eye image and the right-eye image with respect to the image of the viewpoint in the front direction of the display unit 25, and the end of the display unit 25. The viewpoint image from the partial direction is divided so as to be small, and the left-eye image and the right-eye image are converted into multi-parallax images by interpolation, according to the divided amount of parallax. The multi parallax image conversion unit 32 supplies the converted multi parallax image to the multi parallax image synthesis unit 33.

  That is, as shown in the left part of FIG. 3, the left-eye image PL and the right-eye image PR are viewed with the left eye EL of the viewer H because the parallax amount D exists. Then, the right-eye image PR is viewed with the right eye ER, so that a stereoscopic effect as a three-dimensional image can be felt.

  Conventionally, when generating a multi-parallax image including five viewpoints from the left-eye image PL and the right-eye image PR having such a parallax amount D, as shown in the right part of FIG. 3, the left-eye image The left-eye image PL and the right-eye image are divided so as to be equal to the parallax amount D / 4 with respect to the parallax amount D of the PL and right-eye image PR, and to correspond to the parallax amount D / 4. Images P1 to P5 were generated as multi-parallax images of five viewpoints by interpolation from PR. Here, the images P1 and P5 are the left-eye image PL and the right-eye image PR, respectively.

  The multi-parallax images P1 to P5 generated in this way can be viewed as a three-dimensional image corresponding to the corresponding viewpoint. That is, as shown in FIG. 3, for example, when the images P1 and P2 are viewed by the viewer H's left eye EL and right eye ER, respectively, a three-dimensional image having a parallax amount D / 4 in the viewing direction. As viewed. When the images P2 and P3 are viewed by the left eye EL and the right eye ER of the viewer H, respectively, they are viewed as a three-dimensional image with a parallax amount D / 4 in the viewing direction. Further, when the images P3 and P4 are viewed by the viewer H's left eye EL and right eye ER, respectively, they are viewed as a three-dimensional image with a parallax amount D / 4 in the viewing direction. Further, when the images P4 and P5 are viewed by the left eye EL and the right eye ER of the viewer H, respectively, they are viewed as a three-dimensional image with a parallax amount D / 4 in the viewing direction.

  When the multi-parallax image is generated in this way, when viewing the display unit 25 of the mobile terminal 11, if the angle of the display unit 25 with respect to the viewing direction of the viewer changes, a three-dimensional image corresponding to the change is viewed. It becomes possible. However, if the three-dimensional image can be viewed even if the angle of the display unit 25 with respect to the viewing direction of the viewer changes, the parallax amount in the original left-eye image PL and right-eye image PR is the parallax amount D. On the other hand, in the three-dimensional image that can be viewed at each angle with respect to the display unit 25, the parallax amount is D / 4. In general, the larger the parallax amount is, the more the stereoscopic effect such as depth can be made conscious. Therefore, there is a possibility that the stereoscopic effect is greatly reduced by converting the left-eye image and the right-eye image into a multi-parallax image by the conventional method.

  On the other hand, in the mobile terminal 11 to which the technology disclosed in the present application is applied, the multi-parallax image conversion unit 32 in the display control unit 24 is the front of the display unit 25 for the user who is the viewer of the mobile terminal 11. For an image from the viewing direction close to the direction, the amount of parallax is increased, and for an image from the viewing direction close to the end, the amount of parallax is decreased, so that the amount of parallax is divided unevenly. Then, the multi-parallax image conversion unit 32 converts the parallax amount into the multi-parallax image by interpolation using the left-eye image PL and the right-eye image PR based on the unevenly divided parallax amounts.

  That is, when the parallax amount between the left-eye image PL and the right-eye image PR is the parallax amount D, the multi-parallax image converting unit 32 is first shown in FIG. As described above, for the range close to the front direction, the parallax amount is set to 2D / 6, for example, and for the range close to the end direction, the parallax amount is set to D / 6. Then, the multi-parallax image conversion unit 32 interpolates the left-eye image PL and the right-eye image PR according to the divided parallax amounts, thereby multi-parallax images P1, P2 ′, P3, P4 ′. , P5. The images P1, P3, and P5 are the same as the images P1, P3, and P5 in FIG.

  The multi-parallax images P1, P2 ', P3, P4', and P5 generated in this way can be viewed as a three-dimensional image corresponding to the corresponding viewing direction. That is, as shown in FIG. 4, for example, when the images P1 and P2 ′ are viewed by the left eye EL and the right eye ER of the viewer H, respectively, the images P1 and P2 ′ are parallax in the viewpoint direction. It is viewed as a three-dimensional image of amount D / 6. Further, when the images P2 ′ and P3 are viewed by the left eye EL and the right eye ER of the viewer H, respectively, the images P2 ′ and P3 are viewed as a three-dimensional image with a parallax amount 2D / 6 in the viewpoint direction. Is done. Further, when the images P3 and P4 ′ are viewed by the left eye EL and the right eye ER of the viewer H, respectively, the images P3 and P4 ′ are viewed as a three-dimensional image with a parallax amount 2D / 6 in the viewpoint direction. Is done. When the images P4 ′ and P5 are viewed by the left eye EL and the right eye ER of the viewer H, respectively, the images P4 ′ and P5 are viewed as a three-dimensional image with a parallax amount D / 6 in the viewpoint direction. Is done.

  When the multi-parallax image is generated in this way, when viewing the display unit 25 of the mobile terminal 11, if the angle of the display unit 25 with respect to the viewing direction of the viewer changes, a three-dimensional image corresponding to the change is viewed. It becomes possible. Further, in a range close to 90 degrees, which is the viewing direction when the mobile terminal 11 is normally used, which is the front direction with respect to the display unit 25, the parallax amount 2D / 6 even if the number of viewpoints increases. Therefore, as shown in FIG. 3, it is possible to suppress a reduction in stereoscopic effect and display a three-dimensional image more stereoscopically, as in the case where the amount of parallax is divided equally as in the prior art. . In addition, since the amount of parallax is smaller at the end than in the conventional case where the amount of parallax is divided equally, the degree of stereoscopic effect is reduced from the viewpoint near the end, but in actual use, The possibility of viewing from a viewpoint is low, and even if the viewing is performed with the display unit 25 being changed, the 3D image can be viewed to the extent that there is no sense of incongruity.

  In step S <b> 4, the multi-parallax image synthesizing unit 33 synthesizes the multi-parallax images so as to be displayed as a multi-parallax three-dimensional image on the display unit 25, generates a synthesized image, and supplies the synthesized image to the display unit 25. .

  In step S5, the display unit 25 displays a combined image obtained by combining the multi-parallax images.

  In step S6, the control unit 21 determines whether or not the input unit 22 has been operated to instruct the end of display. If the end has not been instructed, the process returns to step S1. That is, the processes in steps S1 to S6 are repeated until an end is instructed. Then, if it is determined in step S6 that the end has been instructed, the process ends.

  That is, normally, a two-viewpoint three-dimensional image consisting only of the left-eye image PL and the right-eye image PR is each of the left-eye image PL and the right-eye image PR, as shown in the right part of FIG. The pixel rows having a predetermined width are alternately arranged in the horizontal direction to generate a composite image. By passing through the separation filter 25a provided on the surface of the display unit 25 by such a composite image, the viewer's left eye views the viewing image VL including the left eye image PL, and the right eye has the right eye. By viewing the viewing image VR made up of the image PR, the viewer recognizes it as a three-dimensional image and views it as a stereoscopic image. The separation filter 25a in FIG. 5 is an image of a parallax barrier. However, the separation filter 25a only needs to be able to separate and transmit the left-eye image PL and the right-eye image PR in predetermined directions. Therefore, for example, a lenticular lens may be used.

  On the other hand, the multi-parallax image combining unit 33 converts, for example, the multi-parallax images P1, P2 ′, P3, P4 ′, and P5 obtained by converting the left-eye image and the right-eye image by interpolation to the left part of FIG. The composite image is generated by arranging as shown in FIG. That is, as shown in the left part of FIG. 5, the multi-parallax image composition unit 33 is arranged in the order of the left part of FIG. To synthesize it repeatedly in the horizontal direction.

  By generating such a composite image, when the viewer views the display unit 25 from a direction close to the left end portion in FIG. 5, the left eye views the viewing image V1 including the image P1, and the right eye Thus, by viewing the viewing image V2 including the image P2 ′, it is possible to view a three-dimensional image having the parallax amount D / 6. In addition, when the viewer views the display unit 25 from the left direction close to the front in FIG. 5, the viewing image V2 composed of the image P2 ′ is viewed with the left eye, and the viewing image V3 composed of the image P3 is viewed with the right eye. Can view a three-dimensional image having a parallax amount of 2D / 6. Furthermore, when the viewer views the display unit 25 from the right direction close to the front in FIG. 5, the viewing image V3 composed of the image P3 is viewed with the left eye, and the viewing image V4 composed of the image P4 ′ is viewed with the right eye. Can view a three-dimensional image having a parallax amount of 2D / 6. Then, when the viewer views the display unit 25 from the direction close to the right end in FIG. 5, the viewing image V4 composed of the image P4 ′ is viewed with the left eye, and the viewing image V5 composed of the image P5 with the right eye. Can view a three-dimensional image having a parallax amount D / 6.

  As a result, when using the mobile terminal 11, when viewing from the front direction with respect to the display unit 25 that is generally frequently used, than when a multi-parallax image is configured by dividing the parallax amount evenly. In addition, since the amount of parallax can be increased, it is possible to suppress a reduction in stereoscopic effect when viewing a three-dimensional image, and to increase the stereoscopic effect. Also, when viewing from a direction other than the front direction, for example, from the end direction of the display unit 25, although the amount of parallax is small, it can be viewed as a three-dimensional image. Even in such a case, it is possible to view a stereoscopic image.

  In the above description, an example has been described in which the parallax amount near the front is 2D / 6 and the parallax amount near the end is D / 6 with respect to the parallax amount D of the left-eye image and the right-eye image. However, as long as the amount of parallax in the vicinity of the front direction can be set larger than the amount of parallax in the other direction, it may be unequally divided into other amounts of parallax. Moreover, although the example which produces | generates the image for five eyes by the interpolation by dividing the image for left eyes and the image for right eyes unevenly, the image of the number of viewpoints other than five viewpoints is produced | generated. Alternatively, it may be generated not only by interpolation but also by extrapolation. Furthermore, in the above description, an example in which a 3D image is displayed while being captured has been described, but after the captured 3D image is temporarily stored, it is converted into a multi-parallax image and displayed using the stored 3D image. You may do it.

<2. Modification>
[When the supplied image is a CC image that can be viewed as a three-dimensional image in advance]
In the above description, an example in which a three-dimensional image captured by the imaging unit 23 that captures a three-dimensional image is converted into a multi-parallax image has been described. However, the left-eye image and the right eye can be viewed in advance as a three-dimensional image. If a business image is generated, it may be used.

  That is, when a left-eye image and a right-eye image that can be viewed as a three-dimensional image, such as a CG (Computer Graphics) image, are generated in advance, the three-dimensional image is acquired and the amount of parallax is calculated. The image may be divided unevenly and converted into a multi-parallax image by interpolation based on the unevenly divided amount of parallax.

  FIG. 6 shows an example of a left-eye image and a right-eye image that are generated in advance as images that can be viewed as a three-dimensional image such as a CG (Computer Graphics) image. The example of a structure of the portable terminal 11 made to convert into a multi parallax image is shown. In FIG. 6, configurations having the same functions as those in FIG. 1 are denoted by the same names and the same reference numerals, and description thereof will be omitted as appropriate. 6 is different from the portable terminal 11 in FIG. 1 in that the communication unit 41 is provided in place of the imaging unit 23 and the three-dimensional image storage unit 31 is deleted.

  That is, in the mobile terminal 11 of FIG. 6, the communication unit 41 including a wireless LAN (Local Area Network) card or the like causes a three-dimensional image including a CG image or the like from an external server (not illustrated) via a network (not illustrated). An image is acquired and supplied to the control unit 21.

  The control unit 21 directly supplies a left-eye image and a right-eye image, which can be viewed in advance as a three-dimensional image including the supplied CG image, to the multi-parallax image conversion unit 32. By this processing, the multi-parallax image conversion unit 32 converts the left-eye image and the right-eye image that can be viewed as a pre-generated three-dimensional image including the supplied CG image or the like by the above-described method. Convert to image.

  Note that the multi-parallax image display process in the mobile terminal 11 in FIG. 6 is performed by the communication unit 41 instead of the 3D image capturing process by the image capturing unit 23 in step S1 described with reference to the flowchart in FIG. Since the process of acquiring a three-dimensional image is only skipped, the process of storing the three-dimensional image by the three-dimensional image storage unit 31 in step S2 is skipped, and the description thereof is omitted.

<3. Second Embodiment>
[Example configuration of a television receiver]
In the above description, an example in which a three-dimensional image is displayed by the display unit 25 provided in the mobile terminal 11 has been described. However, the present invention is applicable to any electronic device including a display unit made of an LCD, an organic EL, or the like. For example, it may be applied to a television receiver. When the technique of the present disclosure is applied to a television receiver, it is necessary to consider that the viewer is not one person with respect to the display screen like the mobile terminal 11. For example, the position of the viewer is specified, The parallax amount is set to be large with respect to the direction in which the specified viewer exists, and the parallax amount is set to be small with respect to the other directions, and the left-eye image and the right-eye image constituting the three-dimensional image are multi-parallax You may make it convert into an image.

  FIG. 7 shows a three-dimensional image in which the position of the viewer is specified and divided so that the amount of parallax is increased in the direction in which the specified viewer exists, and the amount of parallax is reduced in the other directions. FIG. 2 is a diagram illustrating a configuration example of a television receiver that converts a left-eye image and a right-eye image that form a multi-parallax image and displays them.

  The television receiver 101 includes a control unit 121, an input unit 122, a tuner 123, a display control unit 124, and a display unit 125. The control unit 121, the input unit 122, the display control unit 124, and the display unit 125 correspond to the control unit 21, the input unit 22, the display control unit 24, and the display unit 25 of FIG. In addition, a function unique to the television receiver 101 is provided.

  That is, the control unit 121 includes a CPU, a RAM, and a ROM (not shown), and the CPU expands programs and data stored in the ROM to the RAM, executes various processes, and the television receiver. The entire operation 101 is controlled. The control unit 121 controls the tuner 123 based on an operation signal from the input unit 122 generated when the operation panel or remote controller is operated by the user, and is transmitted by a broadcast wave having a predetermined frequency 3. A program of a three-dimensional image is received. Further, the control unit 121 supplies the display control unit 124 with the data of the three-dimensional image of the program received by the tuner 123.

  The camera 102 captures a direction in which a user who is a viewer who views the image displayed on the display unit 125 of the television receiver 101 exists, and supplies the captured image to the television receiver 101.

  The display control unit 124 acquires and stores a three-dimensional image captured by the tuner 123 via the control unit 121, that is, a left-eye image and a right-eye image. Further, the display control unit 124 reads out the stored three-dimensional image, converts it into a viewable image as a multi-parallax three-dimensional image, and displays a display unit made up of an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) 125.

  More specifically, the display control unit 124 includes a three-dimensional image storage unit 131, a multi-parallax image conversion unit 132, and a multi-parallax image synthesis unit 133. The three-dimensional image storage unit 131 includes a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores a left-eye image and a right-eye image that form a three-dimensional image in a predetermined image compression format. Store or read, and supply to the multi-parallax image conversion unit 132.

  The position specifying unit 134 detects a face image from the image captured by the camera 102, regards the detected face image as a user who is a viewer, and determines the position where the user exists in the detected face image. Position information, that is, information specifying the viewer's direction with respect to the display unit 125 is generated and supplied to the multi-parallax image conversion unit 132. The camera 102 only needs to be able to specify the position of the user. For example, instead of the camera 102, a thermography that measures a temperature distribution in an image to be captured is used, and the position specifying unit 134 captures the image. The range in which the human body temperature is distributed may be specified as the position of the user from the temperature distribution in the obtained image.

  The multi-parallax image conversion unit 132 sequentially reads out the three-dimensional image stored in the three-dimensional image storage unit 131, and converts the left-eye image and the right-eye image that form the three-dimensional image into a multi-viewpoint three-dimensional image. Is converted into a multi-parallax image that enables viewing, and supplied to the multi-parallax image combining unit 133. At this time, the multi-parallax image conversion unit 132 views the parallax amounts of the left-eye image and the right-eye image based on the information specifying the direction of the viewer user with respect to the display unit 125 by the position specifying unit 134. The parallax amount is divided so that the parallax amount in the direction in which the user who is the user is present is increased, and a multi-parallax image corresponding to the divided parallax amount is generated. By doing in this way, it is possible to realize viewing of a three-dimensional image from multiple viewpoints while suppressing a reduction in stereoscopic effect in the direction in which the viewer who views the television receiver 101 exists. Become.

  The multi-parallax image synthesizing unit 133 synthesizes an image so that the multi-parallax image can be displayed in the horizontal direction for each predetermined pixel width based on the information of the multi-parallax image by the multi-parallax image conversion unit 132, and the display unit 125 Supply and display.

  The display unit 125 is provided with a separation filter 125a made of a parallax barrier, a lenticular lens, or the like. This separation filter 125a separates a multi-parallax image obtained by interpolating the left-eye image and the right-eye image constituting the three-dimensional image in each direction so as to maintain the respective parallax amounts. Make it transparent. With such display processing, viewing of a three-dimensional image from multiple viewpoints is realized, and the amount of parallax with respect to the direction of the viewer of the display unit 125 that is the gaze direction of the main user of the mobile terminal 11 is present. Is divided with large allocation. As a result, the user can view from multiple viewpoints while suppressing a reduction in stereoscopic effect due to the three-dimensional image.

[Multi-parallax image display processing by a television receiver]
Next, multi-parallax image display processing by the television receiver 101 of FIG. 7 will be described with reference to the flowchart of FIG.

  In step S31, based on an instruction from the control unit 121, the tuner 123 receives a broadcast wave program composed of a three-dimensional image, and converts the left-eye image and the right-eye image constituting the three-dimensional image into the control unit. 121 is supplied. The control unit 121 supplies the left eye image and the right eye image constituting the three-dimensional image supplied from the tuner 123 to the display control unit 124.

  In step S <b> 32, the three-dimensional image storage unit 131 of the display control unit 124 associates the left-eye image and the right-eye image constituting the three-dimensional image supplied from the tuner 123 via the control unit 121. And remember.

  In step S <b> 33, the camera 102 captures the direction in which the user who is the viewer who views the display unit 125 exists, and supplies the captured image to the television receiver 101.

  In step S <b> 34, the position specifying unit 134 detects a face image in the image supplied from the camera 102, specifies the direction in which the viewer user exists with respect to the display direction of the display unit 125, and specifies the direction. Information on the determined direction is supplied to the multi-parallax image conversion unit 132.

  In step S35, the multi-parallax image conversion unit 132 reads out the left-eye image and the right-eye image, which are three-dimensional images stored in the three-dimensional image storage unit 131, and converts them into multi-parallax images using them. To do. More specifically, the multi-parallax image conversion unit 132 increases the amount of parallax between the left-eye image and the right-eye image with respect to the image of the viewpoint in the direction in which the viewer is present with respect to the display unit 125. The viewpoint image from other directions is divided so as to be smaller, and the left-eye image and the right-eye image are converted into multi-parallax images by interpolation, according to the divided amount of parallax. The multi-parallax image conversion unit 132 supplies the converted multi-parallax image to the multi-parallax image synthesis unit 133.

  In step S <b> 36, the multi-parallax image synthesizing unit 133 generates a synthesized image by synthesizing the multi-parallax images so that the display unit 125 can display the multi-parallax three-dimensional image, and supplies the synthesized image to the display unit 125. .

  In step S37, the display unit 125 displays a composite image obtained by combining the multi-parallax images.

  In step S38, the control unit 121 determines whether or not the input unit 122 has been operated to instruct the end of display. If the end has not been instructed, the process returns to step S31. That is, the processes in steps S31 to S38 are repeated until the end is instructed. Then, if it is determined in step S38 that termination has been instructed, the process ends.

  That is, in step S35, the multi-parallax image conversion unit 132 basically performs the left-eye image and the right-eye using the same method as that used by the multi-parallax image conversion unit 32 described with reference to FIGS. The business image is converted into a multi-parallax image. Here, as in the case described with reference to FIGS. 3 to 5, it is assumed that the left-eye image and the right-eye image are converted into multi-parallax images of five viewpoints. Further, as shown in FIG. 9, it is assumed that multi-parallax images from five viewpoints of viewing images V11 to V15 can be viewed on the display unit 125.

  Further, the position specifying unit 134 allows the user H11 who is a viewer to view the viewing images V12 and V11 with the left eye and the right eye, respectively, with respect to the angle of the viewing direction of the viewing images V11 to V15. It is assumed that the user H12 is in a direction in which the viewing images V15 and V14 can be viewed with the left eye and the right eye, respectively. At this time, the multi-parallax image conversion unit 132 sets the parallax amount of the multi-parallax images that can be viewed in the viewing images V12 and V11 and the viewing images V15 and V14 to be large, and reduces the parallax amount so that the other parallax amounts are small. The image is divided and converted into a multi-parallax image by interpolation.

  Further, as shown in FIG. 10, the position specifying unit 134 allows the user H11 who is the viewer to view the viewing image V14 with the left eye and the right eye with respect to the angle of the viewing direction of the viewing images V11 to V15. , V13 can be viewed, and the user H12 can view the multi-parallax images of the viewing images V15 and V14 with the left eye and the right eye, respectively. At this time, the multi-parallax image conversion unit 132 sets the parallax amount of the multi-parallax image that allows viewing the viewing images V14 and V13 and the viewing images V15 and V14 to be large, and sets the parallax amount so as to reduce the other parallax amounts. The image is divided and converted into a multi-parallax image by interpolation.

  As described above, based on the image captured by the camera 102, the two-parallax image in the direction with respect to the display unit 125 where the viewer user exists is set to increase the parallax amount, and the other parallax images are set. For, the amount of parallax is divided so that the amount of parallax is small, and a multi-parallax image is generated according to the divided amount of parallax. Thereby, a user who is a viewer who views the television receiver 101 can always view a multi-parallax image from various directions while a certain amount of parallax is guaranteed.

  9 and 10, the angles formed by the straight lines connecting the television receiver 101 and the viewing images V11 to V15 represent the amount of parallax between the viewing images. In FIG. It is expressed that the angles formed by the images V12 and V11 and the viewing images V15 and V14 are larger than the other angles, thereby being larger than other parallax amounts. Similarly, in FIG. 10, it is expressed that the angles formed by the viewing images V14 and V13 and the viewing images V15 and V14 are larger than other angles by being configured to be larger than the other angles. Has been.

  In the above description, an example in which a set of parallax images including a left-eye image and a right-eye image is converted into a multi-parallax image by interpolation is used, but there is a viewer as described above. As long as the amount of parallax is divided so as to increase the amount of parallax in the direction, a multi-parallax image may be generated by extrapolation.

  Through the above processing, when a pair of two-viewpoint images composed of the left-eye image and the right-eye image are divided into parallax amounts and converted into multi-parallax images by interpolation, the viewer views the image in the viewing direction. By increasing the amount of parallax and dividing the amount of parallax non-uniformly so as to reduce the amount of parallax in other directions, it is possible to view a three-dimensional image more than the conventional state where the amount of parallax is divided evenly. It is possible to suppress a reduction in stereoscopic effect. In addition, by detecting the viewer's direction and converting it into a multi-parallax image while changing the amount of parallax according to the direction in real time, the viewer has a certain amount of parallax without being aware of the viewing position. It is possible to sense the stereoscopic effect in the three-dimensional image in a guaranteed state.

  By the way, the series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 11 shows a configuration example of a general-purpose personal computer. This personal computer incorporates a CPU (Central Processing Unit) 1001. An input / output interface 1005 is connected to the CPU 1001 via a bus 1004. A ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.

  An input / output interface 1005 includes an input unit 1006 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 1007 for outputting a processing operation screen and an image of a processing result to a display device, a program, and various data. Are connected to a storage unit 1008 including a hard disk drive and the like, and a local area network (LAN) adapter and the like, and a communication unit 1009 that executes communication processing via a network represented by the Internet. Also, a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), a magneto-optical disk (including an MD (Mini Disc)), or a semiconductor A drive 1010 for reading / writing data from / to a removable medium 1011 such as a memory is connected.

  The CPU 1001 is read from a program stored in the ROM 1002 or a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded from the storage unit 1008 to the RAM 1003. Various processes are executed according to the program. The RAM 1003 also appropriately stores data necessary for the CPU 1001 to execute various processes.

  In the computer configured as described above, the CPU 1001 loads the program stored in the storage unit 1008 to the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 1001) can be provided by being recorded on the removable medium 1011 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 1008 via the input / output interface 1005 by attaching the removable medium 1011 to the drive 1010. Further, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. In addition, the program can be installed in advance in the ROM 1002 or the storage unit 1008.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  In this specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .

  The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

  For example, the present technology can take a configuration of cloud computing in which one function is shared by a plurality of devices via a network and is jointly processed.

  In addition, each step described in the above flowchart can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

In addition, this technique can also take the following structures.
(1) The parallax amount of a set of parallax images for viewing a three-dimensional image is divided unevenly, so that one set of parallax images is configured for each non-uniformly divided parallax amount. A multi-parallax image conversion unit that converts the set of parallax images into a multi-parallax image by generating a viewpoint image;
An image processing apparatus comprising: a display unit that synthesizes and displays the multi-parallax image converted by the multi-parallax image conversion unit.
(2) The multi-parallax image conversion unit divides the parallax amount of a set of parallax images for viewing the three-dimensional image unevenly so that the parallax amount is large with respect to the front direction of the display unit. Then, the one set of parallax images is converted into a multi-parallax image by generating multi-viewpoint images so that one set of parallax images is formed for each non-uniformly divided amount of parallax. ).
(3) The parallax image conversion unit increases a parallax amount of a set of parallax images for viewing the three-dimensional image with respect to a direction in which a viewer viewing the display unit exists. In this way, the one set of parallax images is generated as multi-parallax by generating multi-viewpoint images so that one set of parallax images is formed for each non-uniformly divided amount of parallax. The image processing device according to (1) or (2), wherein the image processing device converts the image.
(4) a viewer position detector that detects a position of the viewer who views the display unit with respect to the display unit;
The multi-parallax image conversion unit is configured to display a parallax amount of a set of parallax images for viewing the three-dimensional image based on information on a viewer position detected by the viewer position detection unit. The multi-viewpoint is configured so that the parallax amount is increased in the direction in which the viewer who views the video is present, and a set of parallax images is configured for each non-uniformly divided amount of parallax. The image processing device according to any one of (1) to (3), wherein the one set of parallax images is converted into a multi-parallax image by generating the image of (1).
(5) The viewer position detection unit captures an image, detects the face image of the viewer, and detects the position of the face image with respect to the display unit of the viewer who views the display unit. It detects as a position, or it detects as a position with respect to the said display part of the viewer who views the said display part based on the image of the heat distribution detected by thermography. (1) thru | or (4) Image processing apparatus.
(6) The parallax amount of a set of parallax images for viewing a three-dimensional image is divided unevenly, so that one set of parallax images is configured for each non-uniformly divided parallax amount. A multi-parallax image conversion process for converting the set of parallax images into a multi-parallax image by generating a viewpoint image;
An image processing method comprising: a display process for combining and displaying the multi-parallax image converted by the multi-parallax image conversion process.
(7) The parallax amount of one set of parallax images for viewing a three-dimensional image is divided unevenly, so that one set of parallax images is configured for each non-uniformly divided parallax amount. A multi-parallax image conversion step of converting the set of parallax images into a multi-parallax image by generating a viewpoint image;
A program for causing a computer to execute a process including a display step of combining and displaying the multi-parallax image converted by the process of the multi-parallax image conversion step.
(8) The parallax amount of one set of parallax images for viewing a three-dimensional image is divided unevenly, and a set of parallax images is configured for each non-uniformly divided parallax amount. A multi-parallax image conversion unit that converts the set of parallax images into a multi-parallax image by generating a viewpoint image;
An electronic apparatus comprising: a display unit that synthesizes and displays the multi-parallax image converted by the multi-parallax image conversion unit.

  DESCRIPTION OF SYMBOLS 11 Portable terminal, 21 Control part, 22 Input part, 23 Imaging part, 24 Display control part, 25 Display part, 31 Three-dimensional image memory | storage part, 32 Other parallax image conversion part, 33 Multi parallax image synthetic | combination part, 41 Communication part, DESCRIPTION OF SYMBOLS 101 Television receiver, 102 Camera, 121 Control part, 122 Input part, 123 Tuner, 124 Display control part, 125 Display part, 131 Three-dimensional image memory | storage part, 132 Other parallax image conversion part, 133 Multi parallax image synthetic | combination part

Claims (8)

Multi-viewpoint images so that the parallax amount of a set of parallax images for viewing a three-dimensional image is divided unevenly, and one set of parallax images is configured for each non-uniformly divided amount of parallax. A multi-parallax image conversion unit that converts the set of parallax images into a multi-parallax image by generating
An image processing apparatus comprising: a display unit that synthesizes and displays the multi-parallax image converted by the multi-parallax image conversion unit. The multi-parallax image conversion unit divides the parallax amount of a set of parallax images for viewing the three-dimensional image unevenly so that the parallax amount is large with respect to the front direction of the display unit, The set of parallax images is converted into a multi-parallax image by generating a multi-viewpoint image so that a set of parallax images is configured for each non-uniformly divided amount of parallax. Image processing apparatus. The multi-parallax image conversion unit may not set a parallax amount of a set of parallax images for viewing the three-dimensional image so that the parallax amount is large with respect to a direction in which a viewer who views the display unit exists. The set of parallax images is converted into a multi-parallax image by generating multi-viewpoint images so that one set of parallax images is formed for each of the non-uniformly divided parallax amounts. The image processing apparatus according to claim 1. A viewer position detecting unit for detecting a position of the viewer who views the display unit with respect to the display unit;
The multi-parallax image conversion unit is configured to display a parallax amount of a set of parallax images for viewing the three-dimensional image based on information on a viewer position detected by the viewer position detection unit. The multi-viewpoint is configured so that the parallax amount is increased in the direction in which the viewer who views the video is present, and a set of parallax images is configured for each non-uniformly divided amount of parallax. The image processing apparatus according to claim 3, wherein the one set of parallax images is converted into a multi-parallax image by generating a plurality of images. The viewer position detection unit picks up an image, detects a face image of the viewer, and detects a position where the face image is detected as a position of the viewer who views the display unit with respect to the display unit. The image processing apparatus according to claim 4, wherein the image processing apparatus detects a position of a viewer who views the display unit with respect to the display unit based on an image of heat distribution detected by thermography. Multi-viewpoint images so that the parallax amount of a set of parallax images for viewing a three-dimensional image is divided unevenly, and one set of parallax images is configured for each non-uniformly divided amount of parallax. A multi-parallax image conversion process for converting the set of parallax images into a multi-parallax image by generating
An image processing method comprising: a display process for combining and displaying the multi-parallax image converted by the multi-parallax image conversion process. Multi-viewpoint images so that the parallax amount of a set of parallax images for viewing a three-dimensional image is divided unevenly, and one set of parallax images is configured for each non-uniformly divided amount of parallax. A multi-parallax image conversion step for converting the one set of parallax images into a multi-parallax image by generating
A program for causing a computer to execute a process including a display step of combining and displaying the multi-parallax image converted by the process of the multi-parallax image conversion step. Multi-viewpoint images so that the parallax amount of a set of parallax images for viewing a three-dimensional image is divided unevenly, and one set of parallax images is configured for each non-uniformly divided amount of parallax. A multi-parallax image conversion unit that converts the set of parallax images into a multi-parallax image by generating
An electronic apparatus comprising: a display unit that synthesizes and displays the multi-parallax image converted by the multi-parallax image conversion unit.

Images