JP2012160058A - Image processor, stereoscopic image printing system, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor generating, using a degree of stereoscopic vision, a natural composite stereoscopic image allowing a viewer to easily stereoscopically view.SOLUTION: The image processor includes: stereoscopic image data input means for inputting stereoscopic image data; subject extracting means for extracting a subject from the stereoscopic image data; degree-of-stereoscopic-vision calculating means that calculates a degree of stereoscopic vision of the subject from a plurality of plane image data items with different viewpoints that are included in the stereoscopic image data; stereoscopic image data recording means that records the degree of the stereoscopic vision and the stereoscopic image data in association with each other; template selecting means that selects a predetermined number of stereoscopic image templates from a plurality of types of stereoscopic image templates recorded in the template recording means, based on the degree of stereoscopic vision recorded associated with the stereoscopic image data; composing means that composes the stereoscopic image data with a single stereoscopic image template selected by the user from the predetermined number of stereoscopic image templates that is selected by the template selecting means and outputs a composite stereoscopic image data constituting a composite stereoscopic image.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for generating a stereoscopic image by performing image processing on stereoscopic image data.

  Conventionally, stereoscopic viewing using parallax has been realized by combining and displaying a plurality of images with different lines of sight. For example, (1) A method of performing stereoscopic viewing using polarized glasses by applying orthogonal linearly polarized light to the left-eye and right-eye images, and (2) alternately displaying the left-eye and right-eye images. And a method of performing stereoscopic viewing using glasses equipped with a liquid crystal shutter synchronized with the display means, and (3) displaying red and blue light on the left and right eye images, respectively, And a method of performing stereoscopic viewing using glasses with red and blue color filters.

  As a so-called autostereoscopic method, for example, a plurality of images are cut into strips and arranged alternately, and each image is viewed with the left and right eyes using a parallax barrier (parallax barrier) or a lenticular lens. Thus, a method of performing stereoscopic viewing is also mentioned.

Here, Patent Documents 1 to 4 are listed as prior art documents related to the present invention.
In Patent Document 1, when a plurality of positions are taken by a plurality of imaging devices and a desired position and a desired shooting time are specified, a plurality of frame images having different shooting times are selected. A stereoscopic image forming apparatus is disclosed in which a stereoscopic image forming unit is combined and stereoscopically printed so that it can be observed sequentially according to the movement of the observation viewpoint position when observed through a predetermined optical system.

  In Patent Document 2, based on a three-dimensional display surface or a detection unit that detects the position, posture, or shape of a real object arranged in the vicinity of the three-dimensional display surface, and the shape, position, or posture of the real object, A three-dimensional image obtained by performing a drawing process different from a region other than the shielding region on the shielding region, and a shielding region calculation unit that calculates a shielding region that is a region that shields the light beam irradiated by the three-dimensional display surface of the real object A stereoscopic image generation apparatus including a drawing unit that draws the image is disclosed.

  In Patent Document 3, luminance image acquisition means for acquiring a luminance image representing the luminance value of each point captured by the imaging system when an imaging space is imaged, and the spatial position of each point in a predetermined coordinate system. 3D information acquisition means for acquiring the represented 3D information and an input for specifying an offset value for each axial direction of the coordinate system from a predetermined input device, and storing each offset value in a predetermined memory; Disclosed is a file generation device that includes an offset origin setting unit that sets a point obtained by displacing the origin of a coordinate system by an offset value as an offset origin, and generates a file of a predetermined format having an area for storing each information. .

  Patent Document 4 includes, as image control information, a tag (stereoscopic intensity tag) indicating the stereoscopic intensity of a stereoscopic image, which indicates that a stereoscopic image with a greater stereoscopic effect is displayed as the level value of the stereoscopic intensity is larger. There is disclosed a stereoscopic image processing apparatus capable of managing designation of a display device for displaying a stereoscopic image as attached information.

JP 2006-165595 A JP 2008-90617 A JP 2008-252493 A JP 2004-334833 A

  When obtaining a three-dimensional print (stereophoto) in which a three-dimensional image is printed, there is a case where it is desired to obtain a composite image obtained by synthesizing a subject of a three-dimensional image and a character of a three-dimensional image template, like a normal flat image print. However, it is necessary to consider the stereoscopic degree (depth amount) of the stereoscopic image. If the stereoscopic degree of the stereoscopic image (user image) taken by the user and the stereoscopic degree of the template are greatly different, it is difficult to stereoscopically view the depth. There was a problem that an imbalance occurred in the feeling.

  An object of the present invention is to change the stereoscopic degree of a stereoscopic image template for composition and the direction of a character in the template in accordance with the stereoscopic degree of a stereoscopic image taken by a user, thereby enabling a more natural and stereoscopic stereoscopic view. An object is to provide an image processing apparatus, an image processing method, and a program for generating a composite image.

In order to solve the above problems, the present invention provides a stereoscopic image data input unit for inputting stereoscopic image data constituting a stereoscopic image from the outside,
Subject extraction means for extracting a subject from the stereoscopic image data;
Stereoscopic degree calculating means for calculating the stereoscopic degree of the subject from a plurality of planar image data having different viewpoints included in the stereoscopic image data;
Stereoscopic image data recording means for recording the stereoscopic degree and the stereoscopic image data in association with each other;
Template recording means in which a plurality of types of stereoscopic image templates are recorded;
A template selection unit that selects a predetermined number of stereoscopic image templates based on the stereoscopic degree recorded in association with the stereoscopic image data from among a plurality of types of stereoscopic image templates recorded in the template recording unit;
Combining the stereoscopic image data with one stereoscopic image template selected by a user from a predetermined number of stereoscopic image templates selected by the template selection means, and outputting combined stereoscopic image data constituting a combined stereoscopic image And an image processing apparatus.

  Here, it is preferable that the template selection means selects a stereoscopic image template including a character having a stereoscopic degree in a range larger than N / 3 and smaller than 3N, where N is the stereoscopic degree of the subject.

Further, the subject extracting means further extracts a face of the subject,
Furthermore, it is preferable to have a face direction detecting means for detecting the face direction of the subject.

  Further, it is preferable that the template selection unit further selects the stereoscopic image template based on the orientation of the face of the subject in the stereoscopic image and the position of the subject.

  Further, the template selection means is arranged such that the face direction of the subject is on the left side or the right side, the subject is arranged on the left side or the right side of the stereoscopic image, and the character included in the stereoscopic image template is arranged on the right side or the left side. If the direction of the face of the subject faces the character, the direction of the face of the character opposite to the direction of the face of the subject in the up-down direction is not selected, and the direction of the face of the subject Is not directed toward the character, the direction of the character's face in the left / right direction is opposite to the direction of the face of the subject, and the direction of the subject's face in the up / down direction is not selected. It is preferable that the direction of the face of the character in the reverse direction is not selected and a stereoscopic image template including the character of the other face direction is selected. .

  Further, the template selection means may be configured such that the face direction of the subject is on the left side or the right side, the subject is arranged in the center, and the character included in the stereoscopic image template is arranged on the right side or the left side of the subject. Regardless of the orientation of the face of the subject, the stereoscopic image template including the character of the face orientation other than the orientation of the character in the up and down direction is deselected and the face orientation of the character is not selected. Is preferably selected.

  In addition, when the orientation of the subject's face is the center, the template selection unit is configured to display the stereoscopic image in a direction opposite to the orientation of the subject's face in the vertical direction, regardless of the arrangement of the subject and the orientation of the face. It is preferable that the face direction of the character included in the template is not selected and a stereoscopic image template including the character having the other face direction is selected.

  Further, it is preferable that the template selection means is configured to set a priority order for the stereoscopic degree and the face direction of the character included in the stereoscopic image template and select the stereoscopic image template according to the priority order.

  Further, the template selection means displays the stereoscopic degree and face direction of the character included in the stereoscopic image template on a preview screen, and is selected by the user from a plurality of stereoscopic image templates displayed on the preview screen. It is preferable that one stereoscopic image template to be selected is selected.

In addition, the subject extraction means extracts a plurality of parts of the subject,
It is preferable that the three-dimensionality calculating unit calculates the three-dimensionality of a plurality of parts of the subject.

  The stereoscopic image data recording unit preferably records the stereoscopic degree of the subject in an Exif tag of the stereoscopic image data.

  Further, it is preferable that the stereoscopic image data recording means records the stereoscopic degree of the subject in a file different from the stereoscopic image data.

  The stereoscopic degree calculating means calculates a parallax of the subject and a distance from the viewpoint to the subject from a plurality of planar image data having different viewpoints, and calculates the stereoscopic degree of the subject from the parallax and the distance. It is preferable that

The present invention also provides an image processing apparatus according to any one of the above,
And a printing apparatus that prints a combined stereoscopic image corresponding to the combined stereoscopic image data output from the image processing apparatus.

Further, the present invention provides a stereoscopic image data input step for receiving stereoscopic image data constituting a stereoscopic image input from the outside,
A subject extraction step of extracting a subject from the stereoscopic image data;
A stereoscopic degree calculating step of calculating the stereoscopic degree of the subject from a plurality of planar image data having different viewpoints included in the stereoscopic image data;
A stereoscopic image data recording step of recording the stereoscopic degree and the stereoscopic image data in association with each other;
A template selection step of selecting a predetermined number of stereoscopic image templates based on the stereoscopic degree recorded in association with the stereoscopic image data from a plurality of types of stereoscopic image templates recorded in advance;
Combining the stereoscopic image data with one stereoscopic image template selected by the user from the selected predetermined number of stereoscopic image templates, and outputting combined stereoscopic image data constituting a combined stereoscopic image; An image processing method is provided.

  The present invention also provides a program for causing a computer to execute each step of the image processing method described above.

  According to the present invention, it is possible to obtain, without failure, a composite image of a user image and a character image having a layout that is more natural and easy to stereoscopically view, and a three-dimensional print on which these are printed, without being aware of the three-dimensionality.

(A) is explanatory drawing which shows the axis | shaft of planar image data, (b) is explanatory drawing which shows an example of the axis | shaft of stereoscopic image data and stereoscopic degree which concern on this invention. 1 is a block diagram showing an embodiment of a stereoscopic image printing system according to the present invention. It is explanatory drawing which shows another example of the axis | shaft of the stereo image data based on this invention, and a stereoscopic degree. It is explanatory drawing which shows an example of the three-dimensionality of the direction of a face. It is explanatory drawing which shows an example of a three-dimensional print. It is explanatory drawing which shows another example of a three-dimensional print. It is explanatory drawing which shows another example of a three-dimensional print. It is explanatory drawing which shows another example of a three-dimensional print. It is explanatory drawing which shows another example of a three-dimensional print. It is a flowchart which shows an example of the flow of a process in one Embodiment of the image processing apparatus which concerns on this invention. It is explanatory drawing which shows an example of the file structure of stereo image data. It is explanatory drawing which shows an example of the header of the file of stereo image data. It is explanatory drawing which shows another example of the header of the file of stereo image data. It is explanatory drawing which shows another example of the header of the file of stereo image data. It is a flowchart which shows an example of the flow of the stereoscopic printing process in the stereoscopic image printing system which concerns on this invention. FIG. 16 is a flowchart showing a continuation of the flowchart of FIG. 15. FIG. FIG. 16 is a flowchart showing a continuation of the flowchart of FIG. 15. FIG. FIG. 16 is a flowchart showing a continuation of the flowchart of FIG. 15. FIG.

  An image processing apparatus and a stereoscopic image printing system that implement an image processing method according to the present invention will be described in detail below based on preferred embodiments shown in the accompanying drawings.

First, the stereoscopic degree of the subject in the stereoscopic image according to the present invention will be described.
FIGS. 1A and 1B are diagrams comparing planar image data and stereoscopic image data of a user image.
FIG. 1A is a diagram showing the axes of plane image data (two-dimensional data of X and Y axes) constituting a conventional plane image, and the horizontal axis is expressed as the X axis and the vertical axis is expressed as the Y axis. Yes. FIG. 1B is a diagram showing an example of the axis and stereoscopic degree of stereoscopic image data (three-dimensional data of X, Y, and Z axes) constituting a stereoscopic image according to the present invention. , Y axis, and the Z axis perpendicular to the plane composed of the X axis and Y axis of the plane image data, and the Z axis represents the projecting direction of the subject (depth amount or popping amount, that is, subject solidity). It is. That is, the degree of stereo represents the Z-axis data among the X, Y, and Z-axis three-dimensional data included in the stereo image data.

  As stereo image data, the image data of the stereo image containing the plane image for right eyes and the plane image for left eyes can be illustrated, for example. That is, the image data of the stereo image includes the image data of the right-eye plane image and the image data of the left-eye plane image in one image data. Note that the stereoscopic image data is not limited to the image data of a stereo image, and may be any data that includes image data of a plurality of planar images with different viewpoints in one stereoscopic image data.

  Here, in order to obtain the stereoscopic degree of a subject in a stereoscopic image, first, for example, by a method described in JP 2006-165955 A, JP 2001-346226 A, JP 2008-90617 A, or the like. The subject is drawn as volume data (three-dimensional data on the X, Y, and Z axes), and the corresponding point detection result by template matching between the right-eye and left-eye flat images (main image and sub-image) is used for the right eye. The parallax (parallax amount) between the two-dimensional image and the left-eye planar image and the distance from the viewpoint to the subject are calculated.

  Next, as described in, for example, Japanese Patent Application Laid-Open No. 2010-45584, etc., by calculating the value of the axis of stericity represented by the Z axis from the obtained parallax and distance, (Z) Can be obtained. That is, the stereoscopic degree is proportional to the parallax and the distance, and increases as the parallax increases, and increases as the distance increases. Accordingly, if the distance is obtained, the stereoscopic degree can be uniquely calculated according to the parallax. For example, in FIG. 1B, assuming that the value of the Z axis (the amount of protrusion in the Z axis direction) is Z1, the solidity is (Z1) (starting point coordinates (0, 0, 0), end point coordinates (0, 0, Z1))). In addition, the calculation method of a three-dimensionality is not limited at all, and various methods including an existing method can be used.

Next, the stereoscopic image printing system according to the present invention will be described.
FIG. 2 is a block diagram of an embodiment representing a configuration of a stereoscopic image printing system according to the present invention that implements the image processing method according to the present invention.
The stereoscopic image printing system 1 illustrated in FIG. 2 includes an image processing device 10 and a printing device 46. The image processing apparatus 10 includes a stereoscopic image data input unit 12, a subject extraction unit 14, a stereoscopic degree calculation unit 16, a stereoscopic image data recording unit 18, a template recording unit 40, a template selection unit 42, and a combining unit 44. The

  The stereoscopic image data input means 12 is an input means for inputting stereoscopic image data, and has an external interface for inputting stereoscopic image data from the outside of the image processing apparatus 10. There are no particular limitations on the method of creating the stereoscopic image data, and there are various types such as moving image / still image data shot by a user with a 3D-compatible digital still camera or digital video camera, and moving image / still image data downloaded from the network. 3D image data can be input. The external interface may be a USB (Universal Serial Bus), a memory card R / W (reader / writer), an optical disk, a wired / wireless LAN (Local Area Network), or the like. Stereoscopic image data is input from the outside via an external interface. The stereoscopic image data input means 12 receives stereoscopic image data input from the outside and supplies it to each part of the image processing apparatus 10.

  Stereoscopic image data is input to the subject extraction unit 14 from the stereoscopic image data input unit 12. The subject extraction means 14 analyzes the stereoscopic image data and extracts a subject (region) in the stereoscopic image. As a subject extraction method, for example, various methods including existing methods such as edge detection and color detection can be used. In addition, each region is extracted for each part of the subject, for example, the face, left and right hands, torso, and feet. The subject extracting unit 14 generates information on the extracted subject and information on each part of the subject, and outputs the information as subject information.

The subject extracting unit 14 includes a face direction detecting unit 20.
The face direction detection means 20 calculates the face direction (angle) in the vertical direction and the face direction (angle) in the left-right direction from the face area of the subject. The face orientation detection method includes, for example, a method described in Japanese Patent Application Laid-Open No. 2006-202276 for detecting a position of a point indicating a landmark from a face image using a machine learning technique, and image data having a certain size. Various methods including an existing method such as the method described in Japanese Patent Application Laid-Open No. 2004-334836 for cutting out the image and collating it with the image data of the feature portion can be used. Here, regarding the face recognition, a plurality of human faces may be recognized, or a specific human face may be recognized. Further, among a plurality of persons, a specific person may detect each part including a face, and the remaining persons may detect only a face. Note that face orientation information is also output as subject information.

  In this embodiment, the face direction detection means 20 is provided inside the subject extraction stage means 14, but is not limited thereto, and the face direction detection means 20 is provided outside the subject extraction means 14. May be.

  Stereoscopic image data is input from the stereoscopic image data input unit 12 and subject information is input from the subject extraction unit 14 to the stereoscopic degree calculation unit 16. As described above, the three-dimensionality calculation unit 16 performs template matching of the subject between the main image data and the sub-image data included in the stereoscopic image data to detect corresponding points, and based on the corresponding point detection result, The parallax and distance are calculated for each part of the subject. Further, the stereoscopic degree is calculated from the calculated parallax and distance, and the stereoscopic degree (including information on the face direction of the subject) is output.

In the example of FIG. 3, the solidity calculation means 16 sets the face solidity to (Z2) (start point coordinates (0, 0, 0), end point coordinates (0, 0, Z2), and so on), right hand. Is set to (Z3), the left hand is set to (Z4), the torso is set to (Z5), and one or more of them are output as the subject. Note that the output stericity is not limited to these, and the stericity of other parts such as feet may be output.
FIG. 4 shows an example of the face direction and the stereoscopic degree of the subject. For example, “upper left (1)” indicates that the face is directed to the upper left, and the stereoscopic degree is (1). The same applies to the face direction and the stereoscopic degree of other subjects.

  The stereoscopic image data recording unit 18 is input with stereoscopic image data from the stereoscopic image data input unit 12 and the stereoscopic degree and face orientation from the stereoscopic degree calculation unit 16. The stereoscopic image data recording means 18 records the stereoscopic degree and the face direction in association with the stereoscopic image data. The stereoscopic image data recording means 18 may be a semiconductor memory such as a DRAM (Dynamic Random Access Memory) or a flash memory, or a magnetic recording medium such as an HDD (Hard Disk Drive).

  Further, the value of the stereoscopic degree may be written in a part of an Exif (Exchangeable image file format) tag of the stereoscopic image data, or the stereoscopic image data file (for example, file name: Image0001.jpg) and another file (for example, , File name: Image0001.vct).

  In the template recording means 40, a plurality of types of stereoscopic image templates to be combined with the user's stereoscopic image data are recorded in advance. Examples of the stereoscopic image template for synthesis include those using characters in a theme park or the like, and obtaining a stereoscopic composite image in which the subject of the user's stereoscopic image (user image) and the character of the stereoscopic image template are combined. It is something that can be done.

  The template selection unit 42 receives the stereoscopic degree and face orientation recorded in association with the stereoscopic image data from the stereoscopic image data recording unit 18 and the stereoscopic image template from the template recording unit 40. The template selection unit 42 selects a predetermined number of appropriate stereoscopic image templates from a plurality of types of stereoscopic image templates stored in the template recording unit 40 based on the stereoscopic degree and the face orientation, and displays them on a display device (not shown). indicate. The template selection unit 42 outputs a stereoscopic image template selected by the user via an input unit (not shown) from a predetermined number of stereoscopic image templates displayed on the display device.

  In the present embodiment, the information on the face direction of the subject (subject information) included in the subject information input from the subject extracting unit 14 to the stereoscopic degree calculating unit 16 is collected together with the stereoscopic degree and the stereoscopic degree calculating unit 16. However, the present invention is not limited to this, and information on the face direction of the subject (subject information) output from the subject extraction unit 14 and the stereoscopic degree of the subject output from the stereoscopic degree calculation unit 16 May be separately input to the stereoscopic image data recording unit 18 and the template selection unit 42, respectively.

Here, a method of selecting a stereoscopic image template by the template selection unit 42 will be described.
FIG. 5 shows an example in which the subject 61 of the user's stereoscopic image is arranged on the right side and the character 62 of the stereoscopic image template is arranged on the left side on the stereoscopic image 60a. The stereoscopic degree 66 of the subject 61 is (1), and the stereoscopic degree 68 of the character 62 is (3). The face direction of the subject 61 and the character 62 is the lower left. In this state, the faces of both are the same, but the stereoscopic degree of the character 62 is extremely larger than the stereoscopic degree of the subject 61, and the positional relationship of the subject 61 and the character 62 in the Z-axis direction is inconsistent. Will occur.

  For this reason, the template selection unit 42 determines the stereoscopic degree corresponding to the stereoscopic degree 69 of the subject 63 based on the stereoscopic degree of the subject 61, for example, the stereoscopic degree 69 of the subject 63 as in the stereoscopic image 60b shown in FIG. An appropriate stereoscopic image template is selected so that characters 64 having the same degree of stereoscopic degree 70 (both stereoscopic degree is (3)) and the same face direction can be arranged.

  The example of the stereoscopic image 60c shown in FIG. 7 is an arrangement in which the face direction of the subject 72 is upper left, the face direction of the character 74 is upper right, and the subject 72 and the character 74 face each other. However, since the solid degree 78 of the character 74 is (3) and is extremely larger than the solid degree 76 (1) of the subject 72, the character 74 is more conspicuous than the subject 72, which is not preferable.

  Further, in the example of the stereoscopic image 60d shown in FIG. 8, the face direction of the subject 88 is the upper left, the face direction of the character 90 is the upper right, and the subject 88 and the character 90 face each other. However, since the stereoscopic degree 94 of the character 90 is (1) and the stereoscopic degree 92 of the subject 88 is (3), the balance between the subject 88 and the character 90 is lost, which is not preferable.

  Among the combinations of the stereoscopic degree of the subject of the user's stereoscopic image (user image), the orientation of the face, and the arrangement, and the stereoscopic degree and the orientation of the face of the character (described as “Character” in Table 1) of the synthesis stereoscopic image template, Table 1 shows an example of the matrix when the orientation of the subject's face is upward and the stereoscopic degree is (N). When the face direction of the subject is downward, the face direction of the character that is OK (appropriate) and NG (inappropriate) is a reverse combination.

  When the subject of the user image is arranged on the left side of the stereoscopic image, the character of the character image is arranged on the right side of the subject, and when the subject is arranged on the right side, the character is arranged on the left side of the subject. . Further, when the subject is arranged at the center, the character is arranged on the right side or the left side of the subject (which side is arranged is selected by the user).

  When the subject's face is on the left side (upper left, positive left, lower left) or right side (upper right, right positive, lower right) and the subject is placed on the left side or right side, the subject's face direction is toward the character , The character face direction opposite to the subject face direction in the vertical direction is set to NG (non-selected). Also, if the direction of the subject's face does not face the character, the character's face orientation in the left-right direction opposite to the subject's face direction is set to NG, and the subject's face in the vertical direction Let NG be the direction of the face of the character in the opposite direction.

  If the subject's face is on the left or right side and the subject is centered, the subject's face orientation depends on whether the character is placed on the left or right side of the subject. Changes whether the character faces the character or not. Therefore, when the subject is arranged at the center, regardless of the face direction of the subject, the character face direction opposite to the subject face direction in the vertical direction is set to NG.

  In addition, when the subject's face is centered (upper center, center positive, lower center), the subject's face orientation is no matter where the subject is placed on the left side, center, or right side. Is in a neutral orientation regardless of the character placement. Therefore, when the direction of the subject's face is the center, regardless of the subject's arrangement and the direction of the face, the character's face direction opposite to the subject's face direction in the vertical direction is set to NG.

  For example, when the orientation of the subject's face is arranged on the left and on the left, the orientation of the subject's face does not face the character arranged on the right. Accordingly, the direction of the face in the right direction of the character is NG, and the direction of the face in the lower direction of the character is NG. That is, among the character face orientations, the upper right, right positive, and lower right included in the right direction are NG, the lower right, lower center, and lower left included in the lower direction are NG, and the other upper center, upper left, The left positive and the central positive are OK (selected).

  When the orientation of the subject's face is located on the upper left side and on the right side, the orientation of the subject's face faces the character placed on the left side. Therefore, the direction of the face in the lower direction of the character is NG. That is, among the character face orientations, the lower right, lower center, and lower left included in the lower direction are NG, and the other right positive, upper right, upper center, upper left, left positive, and central positive are OK.

  When the orientation of the face of the subject is located at the upper left and in the center, regardless of the orientation of the face of the subject, the face orientation in the lower direction of the character is NG. That is, among the character face orientations, the lower right, lower center, and lower left included in the lower direction are NG, and the other right positive, upper right, upper center, upper left, left positive, and central positive are OK.

  Subsequently, when the orientation of the subject's face is located on the upper right side and on the left side, the orientation of the subject's face is on the upper left side and on the right side, the situation is reversed. Therefore, in this case, the direction of the face of the character to be NG may be reversed left and right, and the direction of the face in the lower direction of the character is NG. That is, the lower right, lower center, and lower left included in the lower direction are NG, and the other right positive, upper right, upper center, upper left, upper left, and central positive are OK.

  When the orientation of the subject's face is located on the upper right side and on the right side, similarly, the orientation of the subject's face is on the upper left and the left side is reversed. Therefore, similarly, the direction of the face of the character to be NG is reversed left and right, the face direction of the left side of the character is NG, and the face direction of the lower side is NG. That is, the upper left, upper left, lower left, and lower right, lower center, and lower left included in the left direction are NG, and the other right positive, upper right, upper center, and central positive are OK. .

  Similarly, when the orientation of the subject's face is located at the upper right and in the center, the orientation of the subject's face is reversed left and right with respect to the case where the orientation of the subject's face is located at the upper left and the center. Accordingly, similarly, the direction of the face of the character to be NG is reversed left and right, and the direction of the face in the lower direction of the character is NG. That is, the lower right, lower center, and lower left included in the lower direction are NG, and the other right positive, upper right, upper center, upper left, upper left, and central positive are OK.

  Subsequently, when the orientation of the face of the subject is at the center, regardless of the placement of the subject and the orientation of the face, the face orientation in the lower direction of the character is set to NG. That is, the lower right, lower center, and lower left included in the lower direction are NG, and the other right positive, upper right, upper center, upper left, upper left, and central positive are OK.

  In Table 1, the solidity of the character is set to (N or less) in the sense that it is a range that does not exceed the stereoscopic degree (N) of the subject. The degree and the three-dimensionality of the character may be different. Specifically, for example, it is desirable that the character has a solidity greater than (N / 3) and smaller than (3N) with respect to the subject's solidity (N).

  In addition, priority may be given to the stereoscopic degree and face direction of characters that can be combined, and the stereoscopic image template may be selected according to the priority. For example, as shown in FIG. 9 and Table 2, when the user stereoscopic degree is upper left (1), the stereoscopic degree and face direction of the character 104 with respect to the stereoscopic degree and face direction 106 of the subject 102 in the stereoscopic image 100 are displayed. 108a to 108h are prioritized such that 108b is No. 1 and 108d is No. 2, and in the case of an inappropriate combination of stereoscopic degree and face orientation 108f, 108g, and 108h, NG is selected so as not to be selected. To do.

  Further, instead of the priority order, these three-dimensionality and face orientation may be displayed on the preview screen so that the user can select them. Furthermore, in order to save the storage capacity, not all combinations are prepared, but a stereoscopic image template close to the most appropriate combination may be prepared.

  Subsequently, the stereoscopic image data associated with the stereoscopic degree and the face direction and the stereoscopic image template selected by the template selection unit are input to the synthesizing unit 44 from the stereoscopic image data recording unit 18. The synthesizing unit 44 synthesizes the stereoscopic image data and the stereoscopic image template selected by the template selection unit (that is, the subject of the stereoscopic image and the character of the stereoscopic image template), and forms the combined stereoscopic image data. Is generated and output.

  The combined stereoscopic image data is input from the combining unit 44 to the printing device 46. The printing device 46 is a printer that can print out a composite stereoscopic image corresponding to the composite stereoscopic image data. As a print of a stereoscopic image, for example, a plurality of images are cut into strips and alternately arranged, and printed using a parallax barrier (parallax barrier) or a lenticular lens so that autostereoscopic viewing is possible. It is done.

  Next, operations of the stereoscopic image printing system 1 and the image processing apparatus 10 according to the present invention that realize the image processing method according to the present invention will be described.

FIG. 10 is a flowchart showing an example of the processing flow of the image processing method according to the present invention.
First, stereoscopic image data as a material is input via the stereoscopic image data input unit 12 (step S10). The stereoscopic image data is input to the subject extraction means 14 and the stereoscopic image is analyzed (step S12), and the subject is extracted. In addition, each region is extracted for each part of the subject, for example, the face, the left and right hands, the torso, and the foot. For the face of the subject, the orientation of the face is detected, and the extracted subject, information on each part of the subject, and information on the orientation of the subject's face are generated and output as subject information (step S14).

  The stereoscopic image data and the subject information are input to the stereoscopic degree calculation means 16, and matching of the subject (face) is performed between the main image data and the sub-image data of the stereoscopic image data (step). S16). Based on the corresponding point detection result, the parallax and the distance are calculated for each subject and each face of the subject (step S18). Further, the degree of stereo is calculated and output from the calculated parallax and distance (step S20).

  When the stereoscopic degree is obtained, it is determined whether or not the calculation of the stereoscopic degree has been completed for all persons appearing in the stereoscopic image (step S22). If the calculation of the stereoscopic degree has not been completed for all persons, step Returning to S16, the three-dimensionality is obtained for the remaining persons ("N" in step S22).

  If the calculation of the stereoscopic degree for all persons has been completed (“Y” in step S22), the stereoscopic image data, the stereoscopic degree, and the face orientation are input to the stereoscopic image data recording means 18, and the stereoscopic degree and The orientation of the face (hereinafter, the stereoscopic degree includes the orientation of the face) and the stereoscopic image data are associated, that is, the stereoscopic degree is written to the Exif tag of the stereoscopic image data or output to another file. Recording is performed (step S24). For example, in the example of the file format shown in FIG. 11, the stereoscopic degree is written in the Exif tag in the header 32 of the stereoscopic image data 30.

  As an example of entry in the Exif tag in the header 32, FIG. 12 shows a case where only the stereoscopic degree of the subject is recorded, and FIG. 13 shows a case where the stereoscopic degree for each part of the subject is recorded. Furthermore, as an example of entry in the Exif tag when there are a plurality of subjects, as shown in FIG. 14, the stereoscopic degree of each part including the facial stereoscopic degree can be recorded for each subject.

  Thus, by recording the stereoscopic degree and face orientation of the subject in association with the stereoscopic image data, the accuracy of subject extraction can be improved, and a more natural and stereoscopic image can be obtained.

  Subsequently, FIGS. 15 to 18 show the stereoscopic image data and the stereoscopic image template using the stereoscopic image data and the stereoscopic degree (user stereoscopic image data and user stereoscopic degree) recorded in the stereoscopic image data recording unit 18. It is a flowchart which shows an example of the flow of a process which synthesize | combines and prints stereo image data.

  First, user stereoscopic image data, user stereoscopic degree, and subject information (from the Exif tag of stereoscopic image data or from a stereoscopic image data file (Image0001.vct)) are read (step S104). Next, based on the subject (user) information and the user stereoscopic degree, an appropriate stereoscopic image template is read from the template recording means 40 and displayed on a display device (not shown) (step S106), and a stereoscopic image for synthesis by the operator. A template (character) is selected (step S108). Note that the arrangement of the stereoscopic image template (character) selected here is not determined. Further, the selection of the stereoscopic image template (character) for synthesis may be performed automatically.

  When the stereoscopic image template (character) for synthesis is selected, the subject arrangement position of the stereoscopic image data is detected or selected (step S110). When the subject (user) is placed on the left (“left” in step S110), a matrix when the subject is placed on the left is selected (step S112), and when the subject is placed at the center. (“Center” in step S110), a matrix is selected when the subject is placed in the center (step S114), and when the subject is placed on the right (“right” in step S110), the subject is turned to the right. A matrix when arranged is selected (step S116).

  Subsequently, a matrix is selected based on the stereoscopic degree of the subject (user stereoscopic degree). When the left-right direction of the user stereoscopic degree (face orientation) is left (“left” in step S118), and when the vertical direction of the user stereoscopic degree (face orientation) is up (in step S120) “Up”), a matrix whose user stereoscopic degree is at the upper left is selected (step S126). Similarly, when the left-right direction of the user stereoscopic degree is left (“left” in step S118) and the vertical direction of the user stereoscopic degree is front (“front” in step S120), the user solid A matrix with the left front is selected (step S128). Further, when the direction of the user stereoscopic degree in the left-right direction is left (“left” in step S118), and the vertical direction of the user stereoscopic degree is down (“down” in step S120), the user stereoscopic degree The lower left matrix is selected (step S130).

  Similarly, in the case where the left-right direction of the stereoscopic degree is the center (“center” in step S118), in steps S122 and S132 to S136, any matrix whose user stereoscopic degree is the center upper, center front, or center lower is the same. Selected. Similarly, when the left-right direction of the user stereoscopic degree is right (“right” in step S118), in steps S124 and S138 to S142, any matrix in which the user stereoscopic degree is upper right, right front, or lower right is obtained. Selected.

  When a matrix is selected on the basis of the user stereoscopic degree, a plurality of three-dimensional image templates that can be combined (appropriate) are presented based on the positional relationship between the subject (user) and the character included in the matrix, A stereoscopic image template having a stereoscopic degree is selected by the operator (step S144). Note that a stereoscopic image template having the most appropriate character placement position and stereoscopic degree may be automatically selected.

  The stereoscopic image template having the character arrangement position and the stereoscopic degree, and the stereoscopic image data are input to the synthesizing unit 44. The stereoscopic image template having the character arrangement position and the stereoscopic degree and the stereoscopic image data are combined, and the combined stereoscopic image data is generated and output (step S146).

  The combined stereoscopic image data is input to the printing device 46, and a stereoscopic image is printed (step S148).

  In this way, the user can select an appropriate composite stereoscopic image template without being aware of the stereoscopic value, and can select an inappropriate stereoscopic image based on the arrangement of the subject and the stereoscopic degree (user stereoscopic degree). It becomes possible to exclude that a template is selected, and satisfaction can be improved.

  Further, by changing the character arrangement and the stereoscopic degree of the stereoscopic image template in accordance with the arrangement of the subject, that is, by selecting the stereoscopic image template that best matches the stereoscopic effect of the subject and the character, more entertainment properties can be obtained. It is possible to obtain a three-dimensional print on which a three-dimensional image with high and high satisfaction is printed.

In the present invention, each step of the above-described image processing method may be configured as an image processing program for causing a computer to execute, or the computer may be configured as each unit for performing each step of the image processing method. Or you may comprise as an image processing program made to function as each means which comprises the image processing apparatus mentioned above.
Further, the present invention may be configured as a computer-readable medium or a computer-readable memory for the above-described image processing program.

  The image processing apparatus, the stereoscopic image printing system, the image processing method, and the program according to the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiment, and the scope of the present invention is not deviated. Various improvements and changes may be made.

DESCRIPTION OF SYMBOLS 1 Stereoscopic image printing system 10 Image processing apparatus 12 Stereoscopic image data input means 14 Subject extraction means 16 Stereogenicity calculation means 18 Stereoscopic image data recording means 20 Face direction detection means 30 Stereoscopic image data 32 Header 40 Template recording means 42 Template selection means 44 Composing means 46 Printing device 60a to 60d, 100 Stereoscopic image 61, 63, 72, 88, 102 Subject 62, 64, 74, 90, 104 Character 66, 68, 69, 70, 76, 78, 92, 94, 106 , 108a to 108h Three-dimensionality (including face orientation)

Claims (16)

Stereoscopic image data input means for inputting stereoscopic image data constituting the stereoscopic image from the outside;
Subject extraction means for extracting a subject from the stereoscopic image data;
Stereoscopic degree calculating means for calculating the stereoscopic degree of the subject from a plurality of planar image data having different viewpoints included in the stereoscopic image data;
Stereoscopic image data recording means for recording the stereoscopic degree and the stereoscopic image data in association with each other;
Template recording means in which a plurality of types of stereoscopic image templates are recorded;
A template selection unit that selects a predetermined number of stereoscopic image templates based on the stereoscopic degree recorded in association with the stereoscopic image data from among a plurality of types of stereoscopic image templates recorded in the template recording unit;
Combining the stereoscopic image data with one stereoscopic image template selected by a user from a predetermined number of stereoscopic image templates selected by the template selection means, and outputting combined stereoscopic image data constituting a combined stereoscopic image And an image processing apparatus.   2. The image processing according to claim 1, wherein the template selection unit selects a stereoscopic image template including a character having a stereoscopic degree in a range larger than N / 3 and smaller than 3N, where N is the stereoscopic degree of the subject. apparatus. The subject extracting means further extracts a face of the subject,
The image processing apparatus according to claim 1, further comprising a face direction detecting unit configured to detect a face direction of the subject.   The image processing apparatus according to claim 3, wherein the template selection unit is further configured to select the stereoscopic image template based on a face orientation and a position of the subject in the stereoscopic image.   The template selection means, when the face direction of the subject is on the left or right side, the subject is arranged on the left or right side of the stereoscopic image, and the character included in the stereoscopic image template is arranged on the right or left side, When the direction of the face of the subject faces the character, the character's face direction opposite to the direction of the subject's face in the up-down direction is not selected, and the face direction of the subject is When not facing the character, the direction of the character's face opposite to the direction of the subject's face in the left-right direction is not selected, and the direction opposite to the direction of the face of the subject in the vertical direction 5. The image according to claim 4, wherein the face orientation of the character is not selected and a stereoscopic image template including the character of any other face orientation is selected. Processing apparatus.   The template selecting means is arranged such that when the face of the subject is on the left side or the right side, the subject is placed in the center, and the character included in the stereoscopic image template is placed on the right side or the left side of the subject, Regardless of the orientation of the subject's face, the character's face orientation in the up / down direction opposite to the subject's face orientation is not selected, and a stereoscopic image template including the character with any other face orientation is selected. The image processing apparatus according to claim 4.   When the face direction of the subject is the center, the template selection unit applies the stereoscopic image template in the up-down direction opposite to the face direction of the subject regardless of the arrangement of the subject and the face direction. The image processing apparatus according to claim 4, wherein the face direction of the included character is not selected, and a stereoscopic image template including the character having the other face direction is selected.   The image processing apparatus according to claim 4, wherein the template selection unit sets a priority order for the stereoscopic degree and the face direction of the character included in the stereoscopic image template, and selects the stereoscopic image template according to the priority order. .   The template selection means displays a character's stereoscopic degree and face orientation included in the stereoscopic image template on a preview screen, and is selected by the user from a plurality of stereoscopic image templates displayed on the preview screen. The image processing apparatus according to claim 1, wherein one stereoscopic image template is selected. The subject extracting means extracts a plurality of parts of the subject,
The image processing apparatus according to claim 1, wherein the three-dimensionality calculation unit calculates the three-dimensionality of a plurality of parts of the subject.   The image processing apparatus according to claim 1, wherein the stereoscopic image data recording unit is configured to record the stereoscopic degree of the subject in an Exif tag of the stereoscopic image data.   The image processing apparatus according to claim 1, wherein the stereoscopic image data recording unit records the stereoscopic degree of the subject in a file different from the stereoscopic image data.   The stereoscopic degree calculating means calculates the parallax of the subject and the distance from the viewpoint to the subject from a plurality of planar image data having different viewpoints, and calculates the stereoscopic degree of the subject from the parallax and the distance. The image processing apparatus according to claim 1. An image processing device according to any one of claims 1 to 13,
A stereoscopic image printing system, comprising: a printing apparatus that prints a combined stereoscopic image corresponding to the combined stereoscopic image data output from the image processing apparatus. A stereoscopic image data input step for receiving stereoscopic image data constituting the stereoscopic image input from the outside;
A subject extraction step of extracting a subject from the stereoscopic image data;
A stereoscopic degree calculating step of calculating the stereoscopic degree of the subject from a plurality of planar image data having different viewpoints included in the stereoscopic image data;
A stereoscopic image data recording step of recording the stereoscopic degree and the stereoscopic image data in association with each other;
A template selection step of selecting a predetermined number of stereoscopic image templates based on the stereoscopic degree recorded in association with the stereoscopic image data from a plurality of types of stereoscopic image templates recorded in advance;
Combining the stereoscopic image data with one stereoscopic image template selected by the user from the selected predetermined number of stereoscopic image templates, and outputting combined stereoscopic image data constituting a combined stereoscopic image; An image processing method comprising:   A program for causing a computer to execute each step of the image processing method according to claim 15.