JP2015119360A - Image processor, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image forming apparatus capable of, when synthesizing additional information such as a text with a three-dimensional image for display, displaying the additional information in a state that it is made to always continuously face a viewpoint even when the viewpoint moves.SOLUTION: This image forming apparatus includes: three-dimensional image input means 400 for inputting three-dimensional image information consisting of at least a pair of two-dimensional image information for the right eye and the left eye; additional information input means 401 for inputting additional information; synthesis means 402 for synthesizing the input three-dimensional image information with the additional information; and display means 403 for three-dimensionally displaying the three-dimensional image synthesized by the synthesis means 402. The synthesis means 402 synthesizes one of the two-dimensional image information for the right eye and the left eye with the input additional information, and the display means 403 three-dimensionally displays the three-dimensional image information consisting of one of the two-dimensional information with which the additional information has been synthesized and the other two-dimensional image with which the additional information has not been synthesized.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly, to an image processing apparatus, an image processing method, and a program for displaying by adding additional information to three-dimensional display information.

  2. Description of the Related Art In recent years, an image processing apparatus has been proposed that combines three-dimensional image information with information such as text to display a three-dimensional image and the information such as the text simultaneously in three dimensions. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for combining and displaying text information three-dimensionally in the viewpoint direction according to the viewpoint when combining and displaying text information on a medical image to be displayed in a multi-viewpoint three-dimensional manner. .

JP 2012-234447 A

  However, in order to synthesize and display additional information such as text in a three-dimensional space on a multi-viewpoint three-dimensional image, it is necessary to synthesize by avoiding the three-dimensional interference between the three-dimensional image and the additional information for each viewpoint. is there. Further, in the synthesized multi-viewpoint three-dimensional image, the additional information may be difficult to see depending on the viewpoint, and the additional information must be synthesized toward the viewpoint direction at a different position for each viewpoint, and the processing is complicated. Special tools may be required, which takes time.

  Therefore, the present invention has been made in view of the above problems, and when displaying additional information such as text on a three-dimensional image, it is always continuously facing the viewpoint even if the viewpoint moves. An object of the present invention is to provide an image forming apparatus capable of displaying additional information in a state.

  In order to solve the above problems, an image forming apparatus according to the present invention inputs at least three-dimensional image input means for inputting three-dimensional image information composed of a pair of two-dimensional image information for right eye and left eye, and additional information. Additional information input means, combining means for combining the three-dimensional image information input by the three-dimensional image input means and additional information input by the additional information input means, and display for displaying the three-dimensional image information combined by the combining means in three dimensions Means for synthesizing either the right-eye or left-eye two-dimensional image information with the input additional information, and the display means for synthesizing the two-dimensional image information with the additional information. And the other two-dimensional image in which the additional information is not synthesized is three-dimensionally displayed.

  According to the present invention, it is possible to provide an image forming apparatus that can display a three-dimensional image that does not interfere unnaturally by synthesizing additional information with three-dimensional image information without performing complicated processing. . Further, it is possible to provide an image forming apparatus capable of displaying additional information continuously facing the viewpoint even when the viewpoint moves, when displaying a multi-viewpoint three-dimensional image obtained by combining the additional information. .

It is a block diagram which shows an example of the image processing apparatus which concerns on one Embodiment of this invention. It is a figure explaining the three-dimensional display apparatus of a Barrax barrier system. It is a figure which shows the relationship between a viewpoint and a parallax image pair. It is a figure which shows an example of the functional block of an image processing apparatus. It is a flowchart of the process which displays the three-dimensional image information which synthesize | combined additional information. It is a figure which shows an example of a three-dimensional image and additional information. It is a figure which shows the detailed example of the functional block of an image processing apparatus. It is a flowchart of the process which displays the three-dimensional image information which synthesize | combined additional information. It is a figure which shows an example of the three-dimensional image which synthesize | combined additional information. It is a figure explaining the 3D display device of a lenticular system. It is a figure which shows the relationship between a viewpoint and a multiview image. It is a figure which shows an example of the multiview image in each viewpoint shown in FIG. It is a flowchart of the process which displays the three-dimensional image information which synthesize | combined additional information. It is a figure which shows an example of the three-dimensional image which synthesize | combined additional information. It is a figure which shows the detailed example of the functional block of an image processing apparatus.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  Prior to the description of the embodiments, terms used in this specification will be briefly described. First, the two-dimensional image indicates a one-screen two-dimensional image itself that is recorded and displayed by a display or a printer, for example, representing a person or a figure. The three-dimensional image indicates a one-screen three-dimensional image itself that is three-dimensionally displayed on a three-dimensional display and represents a subject such as a person or a figure.

  A three-dimensional image is composed of a plurality of two-dimensional images taken from different positions. For example, it is an image obtained by displaying a pair of two-dimensional images for left eye and right eye with a predetermined display element and viewing the image with each eye. When each two-dimensional image is generated, each angle indicating the relative position between the subject and the photographing means is called a viewing angle. A multi-viewpoint image is a plurality of two-dimensional images obtained continuously from a substantially identical subject at a predetermined viewing angle, and can be displayed as a three-dimensional image according to the viewing angle of each two-dimensional image, or a so-called panorama. It is an image called an image.

  An image object refers to a part of an image existing in a two-dimensional image or a three-dimensional image, or a specific object. The user viewpoint is an angle from an arbitrary reference when the user views the display or the subject, and the left and right two-dimensional images for the left eye and the right eye have a user viewpoint at approximately the center of the viewing angle of both images. If there is, you can see the image in three dimensions.

  The two-dimensional image information or the three-dimensional image information indicates a signal source such as a luminance signal or a density signal for displaying the two-dimensional image or the three-dimensional image. In particular, the three-dimensional image information includes at least a pair of two-dimensional image information for the left eye and right eye, and the image pair is referred to as a parallax image pair. The angle formed by the pair of parallax images with the subject, that is, the difference between the viewing angles of the two images is called the parallax angle, and is equal to the apex angle of a triangle connecting the human eyes and the center of the display surface. Note that the multi-viewpoint image information may be image information captured by a camera, or may be artificially produced by computer graphics or the like.

  Next, a three-dimensional display for displaying a multi-viewpoint three-dimensional image used in the present invention will be described. In general, a three-dimensional display device simultaneously displays a plurality of pieces of two-dimensional image information on the screen of the display device as three-dimensional image information. Specifically, the two-dimensional image displayed on the three-dimensional display is based on two pieces of two-dimensional image information viewed by the user with the left eye and the right eye at a plurality of viewpoints. The optical member is optically devised to show with each eye.

(First embodiment)
FIG. 1 is a block diagram illustrating an example of an image processing apparatus according to the present embodiment. As shown in FIG. 1, the image processing apparatus according to this embodiment is a general personal computer (hereinafter simply referred to as “PC”) that acquires image information and the like from a network line 110 and the like via a network control unit 90. 1 is provided. The image processing apparatus includes at least a three-dimensional display device 2 that stereoscopically displays a three-dimensional image based on three-dimensional image information.

  The PC 1 includes a CPU 10, a ROM 60, a RAM 70, a storage unit 80, an operation input unit 50, and a network control unit (network) 90. The CPU 10 is an abbreviation for Central Processing Unit, and is a control unit that controls the operation of the PC 1. The ROM 60 is an abbreviation for Read Only Memory, and stores various settings and programs. The RAM 70 is an abbreviation for Random Access Memory, and is a storage unit that functions as a temporary storage unit. The storage unit 80 stores and holds 3D image information, a program used in the present embodiment, and the like. The operation input unit 50 is an input unit such as a mouse or a touch pad, and enables various input operations from the user to the PC 1.

  The three-dimensional display device 2 includes a three-dimensional display 30, which is a three-dimensional display unit, a touch panel 40, and an imaging unit 20. The three-dimensional display 30 is a display that displays a three-dimensional image. The touch panel 40 is provided on the display surface of the three-dimensional display, and selects an image or a displayed input key by touching (touching) the displayed three-dimensional image. Allows various input operations to the PC. The imaging means 20 is usually provided in a general display, and is provided in the vicinity of the three-dimensional display 30 in order to recognize the user's position and user's operation.

  The three-dimensional display 30 in the present embodiment uses a Barrax barrier system, and the display principle will be described with reference to FIG. The three-dimensional display 30 of the Barrax barrier system includes a liquid crystal display 32 having right-eye display pixels and left-eye display pixels arranged alternately in stripes one by one in the direction in which the left and right eyes are arranged. The three-dimensional display 30 has an optical barrier 31 for blocking light from the display pixels for the other eye on the line of sight as viewed from the left and right eyes in front of the display screen.

  As shown in FIG. 2, from the right eye, the light path of the left-eye pixel is shielded by the ballarax barrier 31, and conversely, from the left eye, the right-eye pixel is shielded by the ballarax barrier 31. As a result, since both eyes can see images of different viewpoints taken from the viewpoints of both eyes, the images can be viewed as a stereoscopic image that is equal to an image normally viewed by a person. In FIG. 2, the left-eye pixel and the right-eye pixel of the three-dimensional display 30 are respectively a left-eye image and a right-eye image captured at a certain angle from the viewpoint of viewing the subject 200, as shown in FIG. Each pixel information of the parallax image pair is displayed.

  FIG. 4 is a diagram illustrating an example of an image processing function block of the image processing apparatus according to the present embodiment. An overview of image processing according to the present embodiment will be described with reference to FIG. The 3D image information input unit (3D image input unit) 400 receives 3D image information including at least a pair of 2D images from the network control unit 90 and the like, and stores and holds the 3D image information in the storage unit 80 as necessary.

  The additional information input unit (additional information input unit) 401 receives the additional information from the network control unit 90 in the same manner as the previous three-dimensional image information input unit 400. The additional information may be general text information or the like created directly with the keyboard 50 or the like, and is information to be displayed superimposed on the three-dimensional image. The additional information combining unit (synthesizing unit) 402 selects one two-dimensional image from at least one pair of parallax images input by the three-dimensional image information input unit 400 and inputs the additional information input by the additional information input unit 401. And the compositing process.

  The three-dimensional image display unit (display unit) 403 writes the parallax image pair synthesized by the additional information synthesis unit 402 in a video RAM (hereinafter referred to as VRAM) described later. The image information written in the VRAM is read at a constant cycle and displayed on the three-dimensional display 30. As described above, the three-dimensional image information and the additional information can be displayed three-dimensionally at the same time.

  In the present embodiment, since the additional information is synthesized with only one two-dimensional image of the parallax image pair and displayed in three dimensions, the additional information displayed on the three-dimensional display 30 is displayed in three dimensions. I do not. That is, since the additional information is displayed so as to overlap with one of the synthesized parallax images, in principle there is no three-dimensional unnatural interference with the three-dimensional image object input by the three-dimensional image information input unit 400. . Further, the additional information is always displayed facing the direction of the line of sight when the user views the synthesized image.

  FIG. 5 is a flowchart of processing for displaying the three-dimensional image information obtained by combining the additional information. First, an overall flow of processing for displaying three-dimensional image information obtained by synthesizing additional information will be described with reference to FIG. First, in step S501, the 3D image information input unit 400 reads at least a pair of parallax image information as 3D image information. In step S502, the additional information input unit 401 reads additional information such as text information. In step S503, the additional information combining unit 402 combines the additional information input in step S502 with the right (left) eye image information that is one of the parallax image pairs input in step S501.

  Next, in step S504, the 3D image display unit 403 displays the right (left) eye image information combined with the additional information in step S503 on the VRAM for displaying the right (left) eye image information. Write. In step S505, the three-dimensional image display unit 403 outputs the other left (right) eye image information that has not been combined with the additional information to the VRAM for displaying the left (right) eye image information as it is. To do. With the above processing, the right (left) eye image information and the left (right) eye image information combined with the additional information are displayed on the three-dimensional display 30 shown in FIG. Therefore, the user can view the additional information input in step S502 on the three-dimensional image input in step S501 at the same time in a three-dimensional manner.

  In step S506, it is determined whether or not the display of the three-dimensional image is completed. If the display of the three-dimensional image has not ended (NO), the process returns to step S501 and the process is continued until the display of the three-dimensional image is ended. When the display of the three-dimensional image is finished (YES), the process is finished. Accordingly, the additional information and the 3D image information are read at any time in steps S502 and S501, and the combining process is performed in step S503. Therefore, both the 3D image and the additional information to be combined may be displayed as a moving image. Is possible.

  Next, FIG. 5B is a flowchart for explaining in more detail step S503 for synthesizing the additional information of FIG. FIG. 6 is a diagram illustrating an example of three-dimensional image information and additional information in the process illustrated in FIG. In FIG. 6, the additional information 603 is white characters “not for sale”.

  First, in step S510, for example, the position where the additional information 603, that is, the character “not for sale” is combined in the right-eye image 600 shown in FIG. 6 is determined. The combination position of the additional information may be determined by a user instruction (not shown), or may be determined by selecting from a plurality of positions determined in advance on the screen regardless of the three-dimensional image.

  In step S511, it is determined whether each pixel position of the right-eye image 600 is a pixel position where the additional information 603 is synthesized, according to the additional information synthesis position determined in step S510. In step S511, when it is a pixel position which synthesize | combines the additional information 603 (YES), it progresses to step S512 and implements a synthetic | combination process. That is, the additional information 603 is synthesized by replacing (replacing) the pixel information of the additional information 603 for synthesizing the image information at the corresponding pixel position on the right-eye image information 600. Accordingly, the additional information is overwritten on the right-eye image information 600.

  On the other hand, if it is not the pixel position where the additional information 603 is synthesized in step S511 (NO), the process proceeds to step S513, and the process from step S511 to step S513 is repeated until the process for all the pixel positions of the right-eye image information 600 is completed. . The right-eye image 600 obtained by combining the additional information 603 is defined as a right-eye image 604.

  When the right-eye image 604 combined with the additional information 603 is viewed with the right eye and the left-eye image 601 is viewed with the left eye, the camera as a three-dimensional image object looks three-dimensional, and the additional information 603, that is, the characters “not for sale” are Seen in front of a part of the camera, hidden in letters. However, since the character “not for sale” as additional information exists only in the right-eye image, the camera and the three-dimensional position of the image object displayed three-dimensionally are not determined. In other words, regardless of the shape and orientation of the stereoscopically displayed three-dimensional image object, it can always be displayed directly in front of the right-eye image object without interference in the depth direction.

  Next, FIG. 7 is a diagram showing a detailed example of functional blocks of the image processing apparatus outlined in FIG. The right-eye image 600, the left-eye image 601 and the additional information 603 of the three-dimensional image information stored in the storage unit 80 are input to the additional information combining unit 402 by the three-dimensional image information input unit 400 and the additional information input unit 401. . Then, the additional information is synthesized with either the right eye image or the left eye image. The combined image information for one eye and the image information for the other eye for which no additional information is combined are written in the VRAMs 701 and 702 for the right eye or the left eye, respectively.

  Thus, the drivers 703 and 704 read out the image information written in the respective VRAMs at a constant cycle, and drive the right-eye image display element 705 and the left-eye image display element 706. The three-dimensional image display unit 403 has two sets of display means for the left and right eyes. The right-eye display means includes a right-eye VRAM 701 that handles a right-eye image, a driver 703, and a right-eye image display element 705. The left-eye display means includes a left-eye VRAM 702 that handles a left-eye image, a driver 704, and a left-eye image display element 706.

  As described above, according to the present embodiment, it is possible to provide an image forming apparatus that can display a three-dimensional image that does not interfere unnaturally by synthesizing additional information with three-dimensional image information without performing complicated processing. be able to.

(Second Embodiment)
Next, a second embodiment of the present invention will be described with reference to FIGS. In the first embodiment, the additional information is displayed by being combined with one of the parallax image pairs, that is, the right or left eye image. On the other hand, in the present embodiment, after the elapse of a predetermined time, the left and right eye images for synthesizing the additional information are interchanged and synthesized. Specifically, the additional information is displayed by being combined with the right-eye image, and after a certain period of time, the additional information is combined with the left-eye image, and the left-eye image is combined with the additional information. Will be replaced.

  The above process will be described with reference to FIG. In addition, the same number is attached | subjected about the same process as FIG. 5 demonstrated in 1st Embodiment, and the description is abbreviate | omitted. First, in step S503, the additional information is synthesized, and in steps S504 and S505, the right (left) eye image information and the left (right) eye image information obtained by synthesizing the additional information are output to corresponding VRAMs. Then, after displaying the above image for a predetermined time in step S800, the additional information combining unit 402 reverses the parallax image obtained by combining the additional information input in step S502 in step S503 in step S801. Synthesize with information.

  That is, in step S801, additional information is synthesized with the left (right) eye image. Next, in step S802, the left (right) eye image combined with the additional information is output to the left (right) eye VRAM. Next, in step S803, the right (left) eye image with no additional information synthesized is output to the right (left) eye VRAM. Then, the left (right) eye image information and the right (left) eye image information combined with the additional information are displayed on the three-dimensional display 30 shown in FIG.

  FIG. 9 shows an example of an image synthesized by the process shown in FIG. The same components as those in FIG. 6 described in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted. In step S800 of FIG. 8, after a predetermined time has elapsed, in step S801, the additional information 603 is combined with the left-eye image 601. The left-eye image 601 obtained by combining the additional information 603 is set as a right-eye image 901. At this time, the additional information combining position of the right-eye image 604 combined with the additional information 603 and the additional information combining position of the left-eye image 901 combined with the additional information 603 maintain the same positional relationship as that of the image object.

  That is, while the processing from step S504 to step S801 is being performed, the user is viewing the right-eye image 604 combined with the additional information 603 in FIG. 9 with the right eye and simultaneously viewing the left-eye image 601 with the left eye. ing. After a predetermined time, the right eye image 600 in FIG. 9 is viewed with the right eye, and at the same time, the left eye image 901 in which the additional information 603 is combined is viewed. Since the camera of the three-dimensional image object is always viewed as a parallax image with both eyes, it appears to the user as the same stereoscopic image. Actually, the character “not for sale” of the additional information is viewed as a character in the right-eye image 604 with the right eye, and after a predetermined time has elapsed, as a character in the image of the left-eye image 901 with the left eye for a certain period of time. You will see them alternately.

  Further, if the position where the additional information is combined with the two-dimensional image information for the left and right eyes is substantially the same relative position to the image object in the additional information, the additional information can be displayed stably. Further, the time in step S800 is adjusted to a short time such that the additional information at the moment viewed with each eye does not remain as an afterimage and is not longer than 100 milliseconds and the characters do not flicker.

  As described above, according to the present embodiment, the additional information is always combined with either the left or right eye image information by alternately synthesizing the additional information with the right eye image information or the left eye image information at predetermined time intervals. Has been displayed. For this reason, for example, even when a user with high visual acuity for both eyes or a user with one blind eye sees a 3D image, it does not interfere unnaturally without performing complicated processing. Additional information of the three-dimensional image can be seen.

(Third embodiment)
In the present embodiment, a process for displaying a multi-viewpoint three-dimensional image in which additional information is combined will be described. FIG. 10 is a diagram for explaining the three-dimensional display device 2 shown in FIG. 1 in more detail. FIG. 10A is a cross-sectional view of the display surface of the three-dimensional display device 2. Note that the three-dimensional display 30 in the present embodiment is a multi-viewpoint three-dimensional display.

  As shown in FIG. 10A, the multi-viewpoint three-dimensional display 30 includes a liquid crystal display 300 and a lenticular lens 301 installed on the entire surface thereof. The lenticular lens 301 is a lens that displays an image three-dimensionally using a shift in image position (human binocular parallax) seen by the right eye and the left eye. A touch panel 40 is disposed on the display surface side of the lenticular lens 301. In addition, as illustrated in FIG. 10A, in the present embodiment, the imaging device 20 is disposed at the center of the screen of the display device 30.

  FIG. 10B is a diagram showing a lenticular lens 301 that displays a multi-viewpoint image and each liquid crystal display pixel whose relative position is guaranteed. The multi-viewpoint three-dimensional display 30 shown in FIG. 10B has pixels in each lenticular lens so that the multi-viewpoint three-dimensional display 30 can see a multi-viewpoint three-dimensional image from seven directions. It has eight display pixels that are continuous from 302 to 303. The multi-viewpoint three-dimensional display 30 can perform color display, but in order to simplify the description, the multi-viewpoint three-dimensional display 30 in the present embodiment is a display that performs monochromatic display. Also in the subsequent drawings, the arrangement of display elements and pixels is expressed one-dimensionally. In the following, when simply referred to as “viewpoint”, it refers to the position of the user with respect to the multi-viewpoint three-dimensional display 30, and the user's eyes viewed from one “viewpoint” are respectively the image for the right eye and the image for the left eye. Images are delivered.

  FIGS. 10C and 10D are diagrams illustrating pixels that the user can see from different viewpoints 1 and 4. As shown in FIG. 10C, only the pixel 5 is visible to the right eye of the user at the viewpoint 4, and only the pixel 4 is visible to the left eye of the user at the viewpoint 4. Further, as shown in FIG. 10D, due to the optical characteristics of the lenticular lens 301, only the pixel 2 is visible to the right eye of the user at the viewpoint 1, and only the pixel 1 is visible to the left eye of the user at the viewpoint 1. The display of a three-dimensional image by such a lenticular lens is a known principle, and further description is omitted in this specification.

  FIG. 11 is a diagram illustrating the relationship between seven viewpoints and captured images that are captured to display a multi-viewpoint three-dimensional image in accordance with the viewpoints. As shown in FIG. 11, before displaying on a multi-viewpoint three-dimensional image display, two two-dimensional images corresponding to images seen by the left and right eyes of the user are taken from each viewpoint, and three-dimensional image information is obtained. Is made. It should be noted that, in order to create a two-dimensional image at each viewing angle in accordance with the optical characteristics of the multi-viewpoint three-dimensional display, the viewing angle and the viewpoint generally coincide substantially. When the same subject 200 is continuously displayed in three dimensions from the viewpoint 1 to the viewpoint 7, the image N delivered to the user's left eye at the viewpoint N (N is an integer of 1 to 7) is the viewpoint N−. 1 is the same as the image delivered to the user's right eye.

  When each of these two-dimensional image information is divided in the horizontal direction and displayed by liquid crystal display pixels that maintain a relative position with the lenticular lens 301, a stereoscopic image close to the case where the subject 200 is viewed from each viewpoint shown in FIG. It becomes possible. That is, the pixel 1 shown in FIGS. 10C and 10D is a part of the two-dimensional image information indicating the two-dimensional image (image 1) photographed on the left side of the viewpoint 1, and the pixel 2 is the right side of the viewpoint 1. This is a part of the two-dimensional image information indicating the two-dimensional image (image 2) photographed in FIG.

  The pixel 4 is a part of two-dimensional image information indicating a two-dimensional image (image 4) photographed on the left side of the viewpoint 4, and the pixel 5 is a two-dimensional image (image 5) photographed on the right side of the viewpoint 4. It is a part of the two-dimensional image information shown. As shown in FIG. 11, the left-eye pixel and the right-eye pixel of the multi-viewpoint three-dimensional display 30 are respectively a left-eye image and a right-eye image captured at a certain viewing angle θ from the viewpoint of viewing the subject 200. That is, each pixel information of the parallax image pair is displayed.

  Next, FIG. 12 shows an image example of image 1 to image 8 for explaining the processing of the image information generation unit 401 according to the present embodiment. Image 1 (1201) is an image for showing the subject camera with the left eye of viewpoint 1 in FIG. 11, and image 2 (1202) is an image for showing the subject with the right eye of viewpoint 1. Similarly, left and right eye images from viewpoint 2 to viewpoint 7 are shown as image 3 (1203) to image 8 (1208). If these eight pieces of image information are displayed on the multi-viewpoint three-dimensional display 30, the user can see the subject camera three-dimensionally according to the direction of each viewpoint from the viewpoint 1 to the viewpoint 7. it can.

  FIG. 13 is a flowchart of image processing for synthesizing additional information according to the present embodiment. In the first embodiment, the processing described in FIG. 5 is assigned the same number, and description thereof is omitted. In the present embodiment, processing different from that in the first embodiment will be described. First, multi-viewpoint three-dimensional image information and additional information are input in steps S501 and S502. Next, in step S1301, the additional information combining unit 402 converts the image 1 (image 2), the image 3 (image 4), the image 5 (image 6), and the image 7 (image 8) from among the eight types of images. Synthesize additional information. In step S1302, the three-dimensional image display unit 404 outputs the image information shown in FIG. 14 to the VRAM in charge of displaying eight types of images in order to display each multi-viewpoint three-dimensional image.

  Specifically, at the viewpoint 1 shown in FIG. 11, since the image 1 is viewed with the left eye and the image 2 is viewed with the right eye, the additional information may be combined with one of the three-dimensional images. Therefore, in order to synthesize additional information with a three-dimensional object for display at all viewpoints, it is only necessary to synthesize the image for either the right eye or the left eye at any viewpoint. That is, the additional information is always combined with the image seen by the left (right) eye at viewpoints 1, 3, 5, and 7, and is always added to the image seen by the right (left) eye at viewpoints 2, 4, and 6. Information is synthesized.

  FIG. 14 shows the result of combining additional information in step S1301 with image 1, image 3, image 5, and image 7 of the multi-viewpoint image information shown in FIG. 12 input in step S501. Specifically, an image 1401 obtained by combining the additional information 1402 of the character “* camera *” with the image 1, an image 1403 obtained by combining the additional information 1404 with the image 3, an image 1405 obtained by combining the additional information 1406 with the image 5, An image 1407 obtained by combining the additional information 1408 with the image 7 is shown.

  In the second embodiment, the additional information is synthesized at substantially the same position of the three-dimensional image object. However, in this embodiment, it is desirable that the position where the additional information is synthesized is synthesized at substantially the same position with respect to the screen to be displayed. . This is because in the case of a multi-viewpoint three-dimensional image, the same image object does not necessarily exist in eight types of images having different viewing angles.

  In other words, in the present embodiment, the multi-viewpoint three-dimensional image display 30 displays a screen (in this embodiment, the lower portion of the screen) regardless of the viewpoint even when the image object is stereoscopically displayed while changing its orientation according to the viewpoint. In addition, the combined additional information is always displayed facing each other. For this reason, the synthesized additional information can be viewed without a sense of incongruity. Further, in the present embodiment, all the same additional information is combined and displayed on the three-dimensional image at all viewpoints. However, the present invention is not limited to this, and for example, different additional information may be displayed depending on the viewpoint. .

  For example, the viewpoint 1 shown in FIG. 11 displays only the additional information 1402 with the left eye, the viewpoint 2 displays only the additional information 1404 with the right eye, the viewpoint 3 displays only the additional information 1404 with the left eye, and the viewpoint 4 displays with the right eye. Only the additional information 1406 is displayed. Similarly, in viewpoint 5, only the additional information 1406 is displayed with the left eye, in viewpoint 6, only the additional information 1408 is displayed in the left eye, and in viewpoint 7, only the additional information 1408 is displayed with the right eye. For this reason, viewpoint 1, viewpoint 2 and viewpoint 3, viewpoint 4 and viewpoint 5, viewpoint 6 and viewpoint 7 have continuity without any sense of incongruity even with additional information 1402 to 1408 that differ depending on the orientation of the image object displayed stereoscopically. The additional information can be combined with the three-dimensional image and displayed.

  FIG. 15 is a diagram illustrating a detailed example of functional blocks of the image processing apparatus according to the present embodiment. The image information in which the additional information 1402 to 1408 is combined by the additional information combining unit 402 is written into the image 1 VRAM 1501 regardless of whether or not the additional information combining process is performed, and the driver 1511 The image 1 display element 1521 is driven to display.

  This embodiment is different from the first and second embodiments, and the three-dimensional image display unit 403 has eight sets of display means including a VRAM, a driver, and a display element in charge of each multi-viewpoint image. Yes. That is, for image 1 to image 8, image 1 VRAM 1501 to image 8 VRAM 1508, driver 1511 to driver 1518, image 1 display element 1521 to image 8 display element 1528.

  As described above, according to the present embodiment, when displaying a multi-viewpoint three-dimensional image obtained by synthesizing additional information, even if the viewpoint moves, it is possible to always display the additional information while facing the viewpoint continuously. An apparatus can be provided.

(Other embodiments)
In the present invention, the example in which the three-dimensional display unit 402 displays on the liquid crystal displays 31 and 300 has been disclosed. However, the same effect can be obtained even in a so-called recorded image having a display image at the same optical position. Is obtained. That is, the three-dimensional display unit 402 has a lenticular lens on its front surface, and is configured by means for separating parallax images into stripes at predetermined intervals and recording them on the same recording material surface using various recording materials. Therefore, the present invention may be applied.

  In addition, the present invention supplies software (program) for realizing the functions of the above-described embodiments to a system or apparatus via a network or various storage media, and a computer (CPU, MPU, etc.) of the system or apparatus executes the program. It is also realized by reading and executing.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

400 3D image information input unit 401 Additional information input unit 402 Additional information composition unit 403 3D image display unit

Claims (8)

3D image input means for inputting 3D image information comprising at least a pair of 2D image information for the right eye and left eye;
Additional information input means for inputting additional information;
Combining means for combining the three-dimensional image information input by the three-dimensional image input means and the additional information input by the additional information input means;
Display means for three-dimensionally displaying the three-dimensional image information synthesized by the synthesis means;
Have
The synthesizing unit synthesizes the two-dimensional image information for the right eye or the left eye and the input additional information, and the display unit is the one-dimensional two-dimensional image information obtained by synthesizing the additional information. And three-dimensional image information composed of the other two-dimensional image not synthesized with the additional information. The combining unit combines the other two-dimensional image information with the additional information after a predetermined time has elapsed, and the display unit combines the other two-dimensional image information with the additional information combined with the additional information. The image processing apparatus according to claim 1, wherein three-dimensional image information including the one two-dimensional image information that is not synthesized is three-dimensionally displayed. The synthesizing unit synthesizes the additional information so that the additional information is displayed at the same position when the three-dimensional display is performed by the display unit, regardless of whether the additional information is synthesized for the right eye or the left eye. The image processing apparatus according to claim 1, wherein:   The three-dimensional image input means inputs a plurality of pieces of three-dimensional image information at a plurality of viewpoints with respect to the subject, the combining means combines the additional information with the plurality of three-dimensional image information, and the display means includes The image processing apparatus according to claim 1, wherein the three-dimensional image information of multiple viewpoints corresponding to the plurality of viewpoints is displayed. The composition means synthesizes additional information that differs depending on the viewpoint among the plurality of viewpoints into three-dimensional image information corresponding to the viewpoint among the plurality of three-dimensional image information. 5. The image processing apparatus according to 4. Each of the two-dimensional image information and the additional information includes pixel information,
The said synthetic | combination means replaces the pixel information of the said two-dimensional image of the position which synthesize | combines the said additional information with the pixel information of the pixel information of the said additional information, The any one of Claims 1-5 characterized by the above-mentioned. Image processing device. A three-dimensional image input step of inputting three-dimensional image information comprising at least a pair of two-dimensional image information for the right eye and left eye;
An additional information input step for inputting additional information;
Combining the 3D image information input in the 3D image input step and the additional information input in the additional information input step;
A display step of three-dimensionally displaying the three-dimensional image information synthesized in the synthesis step;
Have
In the synthesizing step, the two-dimensional image information for either the right eye or the left eye is synthesized with the input additional information, and the one-dimensional two-dimensional image information in which the additional information is synthesized in the display step. And a three-dimensional display of three-dimensional image information including the other two-dimensional image.   The program for functioning a computer as a means of any one of Claims 1-6.

Images