JP2011119825A - Video processor and video processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To output message data at a proper image forming position without any discomfort even when the image forming position of an object or the like is changed due to convergence. <P>SOLUTION: The video processor 100 includes: a data acquisition part 156 for acquiring three-dimensional video data and convergence information; a message acquisition part 158 for acquiring message data; an insertion surface determination part 160 for determining an insertion surface 184 for superimposing the message data and deriving the parallax of the message data; a parallax change part 162 for changing the parallax of the message data so as not to change the positional relation of two imaging parts 110a, 110b and the insertion surface in an imaging space on the basis of the convergence indicated in the convergence information; a data superimposing part 164 for superimposing the message data on the three-dimensional video data according to the parallax; and a data output part 128 for outputting the superimposed three-dimensional video data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a video processing apparatus and a video processing method for generating stereoscopic video data for causing stereoscopic video to be perceived by binocular parallax.

  In recent years, an image processing apparatus that presents two images (stereoscopic image data) having horizontal parallax (binocular parallax) on a display and perceives the viewer as if the subject exists in a three-dimensional manner. I'm bathing. The stereoscopic video data used in such a video processing device is video captured by two imaging units with different viewpoints.

  In general, message data related to video (for example, channel and telop) and message data indicating the status of the video processing device (for example, time, operation mode, icon, input / output line guidance, error information, etc.) Shown on the display. However, message data in a video processing device that realizes a stereoscopic video cannot be correctly transmitted to an observer unless it is processed so as to be perceived stereoscopically. Therefore, the video processing apparatus also provides parallax to the message data and superimposes the message data on the stereoscopic video data, thereby realizing a stereoscopic effect of the message data.

  In addition, as an application example of the display method of message data using such parallax, parallax is provided to text arbitrarily input by the observer, superimposed on the stereoscopic video data, and the perspective of the message data according to the button press. Has been disclosed (for example, Patent Document 1). Furthermore, a technique for superimposing message data on the depth position of the attention point in advance based on the amount of change in the parallax angle of the attention point (the angle at which the visual axis intersects) and reproducing it later is disclosed (for example, patents). Reference 2).

JP 2004-104331 A JP 2009-135686 A

  In the technique described above, in particular, in the technique of Patent Document 2, although the position of the message data is changed according to the parallax angle, the point of interest is artificially specified, and the photographer manually operates while watching the captured video. In such a case, it was forced to perform complicated work such as specifying a point of interest. Further, since the message data is superimposed at the time of imaging, message data relating to the video cannot be deleted at the time of reproduction, and message data indicating the state of the video processing device, for example, generated at the time of reproduction cannot be superimposed at the time of imaging. Furthermore, since there is no way of grasping the position of the pre-superimposed message data at the time of reproduction, new message data cannot be superposed avoiding the pre-superimposed message data.

  On the other hand, the video processing apparatus can also adjust the angle at which the optical axes of the two imaging units converge, the so-called convergence angle. This is because the stereoscopic effect around the subject is more clearly expressed in relation to the foreground and background existing before and after the subject for which imaging is desired. However, since the techniques of Patent Documents 1 and 2 described above do not support the change in the convergence angle, even if the convergence angle changes while the subject is stationary, the message data can be stored in an appropriate depth position corresponding to the subject. Could not be followed.

  In view of such a problem, the present invention provides a video processing apparatus capable of causing message data to be perceived at an appropriate imaging position without a sense of incongruity even when the imaging position of a subject or the like changes with the convergence angle. And an image processing method.

  In order to solve the above-described problem, the video processing apparatus of the present invention has a convergence angle formed by the optical axis of two imaging units that capture stereoscopic video data and stereoscopic video data for perceiving stereoscopic video due to binocular parallax. A data acquisition unit for acquiring convergence angle information, a message acquisition unit for acquiring message data to be superimposed on stereoscopic video data, and an insertion for superimposing message data that intersects perpendicularly with a bisector of the convergence angle The positional relationship between the two imaging units and the insertion plane in the imaging space does not change based on the insertion plane determination unit that determines the plane and derives the parallax of the message data and the convergence angle indicated in the convergence angle information The parallax changing unit for changing the parallax of the message data, the data superimposing unit for superimposing the message data on the stereoscopic video data according to the parallax, and the data for outputting the superimposed stereoscopic video data Characterized by comprising an output section.

The parallax changing unit may change the parallax based on a ratio obtained by multiplying each of cosα _B / cosα _A and sinα _B / sinα _A by a predetermined coefficient when the convergence angle is changed from 2α _A to 2α _B. Good.

  An image processing apparatus includes a subject tracking unit that tracks a specified subject, and a distance deriving unit that derives a subject distance that is a distance between the two imaging units and the tracked subject in the bisector direction of the convergence angle. And a convergence angle control unit that controls the convergence angle based on the derived subject distance.

  In order to solve the above-described problems, the video processing method of the present invention provides a stereoscopic image data for perceiving stereoscopic video due to binocular parallax and a convergence angle formed by the optical axes of two imaging units that have captured stereoscopic video data. In order to superimpose message data that intersects perpendicularly to the bisector of the convergence angle if the message data is obtained for the first time, the message data to be superimposed on the stereoscopic video data is acquired. If it is not the first time, based on the convergence angle indicated in the convergence angle information, the positional relationship between the two imaging units and the insertion surface is determined. The parallax of the message data is changed so as not to change, the message data is superimposed on the stereoscopic video data according to the parallax, and the superimposed stereoscopic video data is output.

  As described above, according to the present invention, message data can be perceived at an appropriate image formation position with no sense of incongruity even when the image formation position of a subject or the like changes with the convergence angle. Therefore, the observer can clearly feel the stereoscopic effect of the stereoscopic image data without feeling tired eyes through the stereoscopic image data on which the message data is superimposed.

It is the functional block diagram which showed the schematic function of the video processing apparatus. It is the external view which showed an example of the video processing apparatus. It is explanatory drawing for demonstrating a convergence angle. It is explanatory drawing for demonstrating a convergence angle. It is explanatory drawing for demonstrating the detailed operation | movement of an insertion surface determination part. It is explanatory drawing for demonstrating the change of the depth feeling of message data by the change of a convergence angle. It is explanatory drawing for demonstrating the detailed operation | movement of a parallax change part. It is explanatory drawing for demonstrating the detailed operation | movement of a parallax change part. It is explanatory drawing for demonstrating perception of the message data by the change of a convergence angle. It is the flowchart which showed the flow of the specific process of the video processing method.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

  In the video processing apparatus, the optical axes of the two imaging units are congested in the imaging space in order to more clearly represent the stereoscopic effect around the subject in relation to the foreground and background existing before and after the subject represented by a person. The convergence angle is adjusted. When the convergence angle changes in this way, the subject image formation position in the reproduction space also changes. However, if the message data image formation position is fixed in the reproduction space, the subject's depth perception changes. The data was stationary, and the observer sometimes felt uncomfortable and the eyes were tired. Here, the imaging space indicates a space representing the actual positional relationship between the video processing apparatus and the subject when imaging is being performed, and the reproduction space is an observer when reproducing the captured stereoscopic video data And a space representing the positional relationship between the subject and the imaging position of the subject. The purpose of this embodiment is to cause message data to be perceived at an appropriate imaging position without any sense of incongruity even if the imaging position of a subject or the like in the reproduction space changes with a change in the convergence angle in the imaging space. Hereinafter, the configuration of the video processing apparatus will be described, and then the processing flow of the video processing method will be described in detail.

(Video processing apparatus 100)
FIG. 1 is a functional block diagram illustrating schematic functions of the video processing apparatus 100, and FIG. 2 is an external view illustrating an example of the video processing apparatus 100. As shown in FIG. 1, the video processing apparatus 100 includes an imaging unit 110, a distance deriving unit 112, a convergence angle control unit 114, a convergence angle deriving unit 116, a video processing unit 118, a data buffer 120, and an operation. Unit 122, stereoscopic display unit 124, video compression unit 126, data output unit 128, and central control unit 130. Here, a video camera is cited as the video processing apparatus 100, but various electronic devices capable of imaging such as a digital still camera can be employed.

  The imaging unit 110 includes two imaging units 110a and 110b. In the two imaging units 110a and 110b, the optical axes 102a and 102b shown in FIG. 2 are converging (intersection) at any position in the imaging direction. When the photographer grips the main body 104 of the video processing apparatus 100 horizontally, the optical axes 102a and 102b are arranged so as to be included in the same horizontal plane.

  In the imaging unit 110, the left-eye video data for the viewer's left eye and the right-eye video data for the viewer's right eye perceived by the respective imaging units 110a and 110b. The three-dimensional video data for perceiving the stereoscopic video by binocular parallax is generated by merging. The stereoscopic video data can be formed as either a moving image or a still image. The stereoscopic video data is formed by various methods such as a side-by-side method and a top-and-bottom method depending on the physical arrangement or temporal arrangement of the left-eye video data and the right-eye video data.

  The two imaging units 110a and 110b are formed so as to be rotatable about the vertical axis, and the convergence angle control unit 114 (to be described later) rotates the two imaging units 110a and 110b, and the convergence angles formed by the optical axes 102a and 102b. Is controlled. Such control of the convergence angle can be realized not only by hardware rotation of the imaging units 110a and 110b but also by adjusting the cut-out position and range of the captured stereoscopic video data.

3 and 4 are explanatory diagrams for explaining the convergence angle. For example, assuming that the convergence point (intersection of the optical axes 102a and 102b) is at the position B in FIG. 3 in the imaging space, the observer perceives the subject P as popping out in the reproduction space, and the subject Q To perceive. At this time, when an equal distance between the position B is a point of convergence between the object P and the object Q, 2 one imaging unit 110a, 110b of the optical axis 102a, even if the distance _{X PB} and _{X QB} and 102b equal. The distances X _PB and X _QB with respect to the optical axes 102a and 102b correspond to the parallax of the stereoscopic video data captured by the two imaging units 110a and 110b. Therefore, when the stereoscopic image data captured with the position B as the convergence point is reproduced, the distance between the imaging position where the subject P is projected and perceived and the stereoscopic display unit 124, and the imaging position where the subject Q is perceived deeply. And the three-dimensional display unit 124 become substantially equal.

On the other hand, if the convergence point is at the position A in FIG. 3 in the imaging space, the distance X _PA between the subject P and the optical axes 102a and 102b of the two imaging units 110a and 110b is shortened, while the subject Q and the two imagings. parts 110a, 110b of the optical axis 102a, the distance _{X QA} and 102b becomes longer. Here, when the stereoscopic video data captured with the position A as the convergence point is reproduced, the distance between the imaging position perceived by the subject P popping out and the stereoscopic display unit 124 is shortened, and the subject Q is perceived deeper. The distance between the imaging position and the stereoscopic display unit 124 becomes longer. That is, the feeling of popping out of the subject P in the reproduction space is impaired, and the feeling of depth of the subject Q is amplified.

  In other words, even if the positional relationship between the imaging unit 110 and the subjects P and Q in the imaging space does not change, that is, even if the subjects P and Q are stationary, the convergence point is changed in the depth direction to reproduce the image. It is possible to change the depth of the subjects P and Q in the space. In the present embodiment, the convergence angle is controlled in accordance with a policy of how to express the subjects P and Q in the reproduction space.

  For example, when it is desired that a person as a subject always jumps forward even if the person moves in the depth direction back and forth, as shown in FIG. 4 (a), the person is placed at a position C far from the subject R in the imaging space. Set the convergence point. However, the farther the convergence point is, the less the sense of depth between the subjects around the subject R (for example, the subject R and the subject S) in the reproduction space becomes, and a sufficient visual stereoscopic effect cannot be obtained. On the other hand, as shown in FIG. 4B, when a convergence point is set at a position D that is substantially equal to the subject R in the imaging space, the sense of depth between subjects around the subject R increases in the reproduction space, but the subject R Depending on whether or not it is before or after the convergence point, it will pop out or be deepened.

  Therefore, it is desirable to set a convergence point at an appropriate position that makes the subject R pop out to some extent while maintaining the sense of depth between the subjects around the subject R. At this time, since the subject R does not always remain stationary, when the subject R moves in the depth direction, it is preferable to change the convergence point (convergence angle) accordingly. Specifically, using the distance deriving unit 112 and the convergence angle control unit 114 shown below, a subject distance, which is a distance from the imaging unit 110 (focal position) to the subject R, is derived, and automatic according to the subject distance. The convergence angle is changed.

  The distance deriving unit 112 measures the time spent on reflection of light (TOF: Time Of Flight) based on the light cutting method, so that the two imaging units 110a and 110b in the bisector direction of the convergence angle, A subject distance from a subject R tracked by a subject tracking unit 152 described later is derived.

  In order to more clearly represent the stereoscopic effect around the subject R, the convergence angle control unit 114 is based on the imaging mode at the time of imaging such as a macro, a person, and a landscape, and the subject distance derived by the distance deriving unit 112, for example. The convergence angle is controlled by rotating the two imaging units 110a and 110b around the vertical axis so that the convergence point is located at a predetermined distance from the subject distance. With this configuration, it is possible to make the observer perceive a good stereoscopic image in which the stereoscopic effect around the subject R is clearly expressed.

  The convergence angle deriving unit 116 adds the rotation angles of the two imaging units 110a and 110b with respect to the main body 104 to derive the convergence angle formed by the optical axes 102a and 102b of the two imaging units 110a and 110b. When the convergence angle is adjusted by a lens or mirror in the imaging unit 110, the rotation angle is added to derive the convergence angle. The convergence angle derivation process is not limited to the above case, and various existing derivation processes can be applied.

  Here, since the convergence angle is uniquely and automatically derived, it is possible to efficiently generate appropriate stereoscopic video data without requiring a complicated manual operation of manually inputting the convergence angle.

  The video processing unit 118 performs video processing such as R (Red) G (Green) B (Blue) processing (γ correction and color correction), enhancement processing, noise reduction processing, and the like on the stereoscopic video data generated by the imaging unit 110. Perform signal processing. The stereoscopic video data that has undergone predetermined video signal processing is temporarily held in the data buffer 120.

  The data buffer 120 includes a RAM (Random Access Memory), a flash memory, an HDD (Hard Disk Drive), and the like, and temporarily holds stereoscopic video data. At this time, together with the stereoscopic video data, the convergence angle information indicating the convergence angle at the time of capturing the stereoscopic video data is also stored in association with the stereoscopic video data.

  The operation unit 122 includes operation keys, a cross key, a joystick, and a switch such as a touch panel arranged on a display surface of a stereoscopic display unit 124 described later, and receives an operation input from the photographer.

  The stereoscopic display unit (viewfinder) 124 is configured by a liquid crystal display, an organic EL (Electro Luminescence) display, or the like, and is formed so that, for example, the polarization characteristics are different (for each line) (line sequential method). The photographer perceives stereoscopic video data stereoscopically through the stereoscopic display unit 124 during imaging, and operates the operation unit 122 based on the perceived stereoscopic video, thereby capturing the subject R at a desired position and occupied area. It becomes possible.

  Here, for example, when a line sequential method is adopted as the stereoscopic display unit 124, the subject and the background included in the stereoscopic video data can be displayed in either a stereoscopic view (crossing method) or a parallel view (parallel method). Therefore, depending on the parallax, the subject or the background can be imaged at either the front side or the back side of the display surface of the stereoscopic display unit 124.

  The display method of the stereoscopic display unit 124 is not limited to the line sequential method. For example, the left-eye video data and the right-eye video data are alternately displayed frame by frame and visually viewed through the electronic shutter glasses. A sequential method, a lenticular method for controlling the light traveling directions of the left-eye image and the right-eye image via a lenticular lens, or the like can also be used.

  The video compression unit 126 converts the stereoscopic video data held in the data buffer 120 into M-JPEG (Motion-Joint Photographic Experts Group), MPEG (Moving Picture Experts Group) -2, H, in response to a control command from the central control unit 130. . The encoded data is encoded by a predetermined encoding method such as H.264 and recorded on an arbitrary recording medium 180. As the arbitrary recording medium 180, an optical disk medium such as a DVD (Digital Versatile Disk) and a BD (Blu-ray Disc), a medium such as a RAM, an EEPROM, a nonvolatile RAM, a flash memory, and an HDD can be applied.

  The data output unit 128 outputs the stereoscopic video data superimposed by the data superimposing unit 164 described later to the display 182 connected to the video processing device 100. The display 182 is configured by a liquid crystal display, an organic EL display, or the like, like the stereoscopic display unit 124, and is formed so that, for example, the polarization characteristics are different for each line.

  The central control unit 130 manages and controls the entire video processing apparatus 100 by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs and the like, a RAM as a work area, and the like. In the present embodiment, the central control unit 130 includes a subject specifying unit 150, a subject tracking unit 152, a storage control unit 154, a data acquisition unit 156, a message acquisition unit 158, an insertion plane determination unit 160, a parallax change unit 162, data It also functions as a superimposing unit 164 and a display control unit 166.

  The subject specifying unit 150 specifies a subject in the stereoscopic video data through an operation input of a touch panel arranged on the display surface of the stereoscopic display unit 124 as the operation unit 122, for example. The subject specifying unit 150 can also automatically specify based on conditions such as relative movement with respect to the background in the video, for example.

  When the subject specified by the subject specifying unit 150 is a person, the subject tracking unit 152 tracks the face image of the person using an existing image processing technique, for example. With this configuration, the distance deriving unit 112 can appropriately derive the subject distance even when the subject moves during imaging.

  The storage control unit 154 stores the convergence angle information indicating the convergence angle derived by the convergence angle deriving unit 116 in the data buffer 120 in association with the stereoscopic video data. Here, since the convergence angle for superimposing the message data is stored instead of the message data itself, it is possible to superimpose the message data or the like on the stereoscopic video data later using the convergence angle.

  The data acquisition unit 156 acquires stereoscopic video data and convergence angle information from the data buffer 120. In addition, when the video processing apparatus 100 functions as a playback apparatus, the data acquisition unit 156 acquires stereoscopic video data and convergence angle information directly from the outside.

  The message acquisition unit 158 acquires message data to be superimposed on the stereoscopic video data. Here, the message data is data that visually represents information of the video processing apparatus 100 itself such as OSD (On Screen Display), channel, telop, time, operation mode, icon, input / output line guidance, error information, and the like. It is.

  The insertion surface determination unit 160 determines a virtual insertion surface for superimposing the message data that intersects the bisector of the convergence angle in the imaging space, and derives the parallax of the message data. In the reproduction space, the insertion plane determination unit 160 sets the insertion plane on the two imaging units 110a and 110b side in the imaging space and reproduces the message data when the viewer wants the viewer to perceive the message data from the subject included in the stereoscopic video data. When it is desired to perceive message data farther than the subject in the space, an insertion surface is set on the background side in the imaging space.

FIG. 5 is an explanatory diagram for describing a detailed operation of the insertion surface determination unit 160. Here, in order to facilitate understanding, the parallax related to one eye will be described, and since the derivation method is substantially the same for the parallax related to the other eye, the description thereof will be omitted. Insertion face determination unit 160, when an attempt afterwards insert message data to insertion surface 184 in FIG. 5, the convergence angle 2.alpha _A at that time, deriving the disparity of the message data (corresponding to the distance X _A) from the following be able to.

Two imaging units 110a, 110b a distance from each object-side principal point to the center point T s, and the distance to the insertion face 184 from the center point T p, and 2.alpha _A convergence angle at the position A, the center point distance L _a from T position to a, a congestion point is represented by equation 1.
L _A = s / tan α _A = s × cos α _A / sin α _A (Formula 1)

_A distance XA between the optical axes 102a and 102b of the two imaging units 110a and 110b is expressed by Equation 2.
X _A = (L _A −p) × sin α _A (Formula 2)
Thus, once the distance p from the center point T and determines the insertion surface 184 of the message data to the insertion plane 184, insertion plane determination unit 160 is able to uniquely derive the distance X _A corresponding to parallax. The data superimposing unit 164 described later superimposes the message data on the stereoscopic video data with the parallax (distance X _A ) derived in this way. With this configuration, message data can be inserted afterwards, and the message data can be perceived as if it existed on the insertion surface 184 in the imaging space.

  Based on the convergence angle indicated by the convergence angle information, the parallax changing unit 162 changes the parallax of the message data so that the positional relationship between the imaging units 110a and 110b and the insertion surface in the imaging space does not change.

  FIG. 6 is an explanatory diagram for explaining a change in the depth feeling of the message data due to a change in the convergence angle. The observer perceives the subject R in a three-dimensional manner, for example, because the subject R is provided with parallax in the stereoscopic video data. At this time, when the convergence point changes from the position A to the position B, the distance between the subject R and the convergence point in the imaging space increases, and the imaging position of the subject R in the reproduction space changes to the observer side. Perceives that the subject R gradually pops out.

On the other hand, when the message data 186 is fixed with the parallax (distance X _A ) derived by the insertion plane determination unit 160, the image formation position of the message data 186 does not change in the reproduction space, so that the observer It is perceived that the message data 186 is relatively deeper as indicated by the white arrow as compared to the pop-up. Therefore, in the present embodiment, even if the imaging position of the subject or the like changes with the convergence angle, the message data is perceived at an appropriate imaging position with no sense of incongruity. Here, the parallax changing unit 162 changes the parallax of the message data in accordance with the change in the imaging position of the subject or the like accompanying the change in the convergence angle.

  7 and 8 are explanatory diagrams for explaining the detailed operation of the parallax changing unit 162. FIG. Here, in order to facilitate understanding, the parallax related to one eye will be described, and the description of the parallax related to the other eye will be omitted because the derivation method is substantially the same. At the stage where the insertion surface determination unit 160 determines the insertion surface 184, message data is arranged on the insertion surface 184 with the convergence point at the position A. At this time, when the convergence point moves from the position A to the position B as indicated by the white arrow in FIG. 7, the parallax changing unit 162 derives the parallax of the message data as follows.

As in FIG. 5, the distance from the object principal point of each of the two imaging units 110a and 110b to the center point T is s, the distance from the center point T to the insertion surface 184 is p, and the convergence angle at the position A is 2α _A , When the convergence angle at the position B and 2.alpha _B, the distance L _B between the distance L _a between the center point position to a, a congestion point to the position B is expressed by equations 1 and 3.
L _A = s / tan α _A = s × cos α _A / sin α _A (Formula 1)
L _B = s / tan α _B = s × cos α _B / sin α _B (Formula 3)

Further, the distance X _A between the optical axes 102a and 102b of the two imaging units 110a and 110b when the convergence point is the position A, and the distance X between the optical axes 102a and 102b when the convergence point is the position B _B is expressed by Equation 2 and Equation 4.
X _A = (L _A −p) × sin α _A (Formula 2)
X _B = (L _B −p) × sin α _B (Formula 4)

Here, by substituting the distance _{L A} and the distance _{L B} indicated by Equations 1 and 3 in Formula 2 and Formula 4, the distance is a ratio between _{X A} and the distance _{X B X} B / _{X A} in Equation 5 expressed.
_{X B / X A = (s} × cosα B -p × sinα B) / (s × cosα A -p × sinα A) ... ( Equation 5)
Further, as shown in FIG. 8A, when the congestion point changes across the message data, X _A / X _B is expressed by Expression 6.
X _B / X _A = (s × cos α _B −p × sin α _B ) / (p × sin α _A −s × cos α _A ) (Formula 6)
Further, as shown in FIG. 8B, when the convergence point changes in an area closer to the message data, X _B / X _A is expressed by Expression 7.
X _B / X _A = (p × sin α _B −s × cos α _B ) / (p × sin α _A −s × cos α _A ) (Expression 7)
However, Expression 7 is substantially equal to Expression 5, and Expression 6 is obtained by inverting the signs of Expression 5 and Expression 7.

Returning to FIG. 7, the distance p from the center point T to the insertion face 184, a distance _{L A} and the coefficient F (F is a positive number) with, for example, when p = F × _{L A,} Equation 5 and Equation 6 Can be transformed into Equation 8 and Equation 9.
_{X B / X A = 1 /} (1-F) × cosα B / cosα A -F / (1-F) × sinα B / sinα A ... ( Equation 8)
_{X B / X A = 1 /} (F-1) × cosα B / cosα A -F / (F-1) × sinα B / sinα A ... ( Equation 9)

As can be understood with reference to Equation 8 and Equation 9, the distance s from the object principal point to the center point T of each of the two imaging units 110a and 110b, the distance p from the center point T to the insertion surface 184, and the center distance L _a between the point T to the position _a, even without information of the distance L _B between the center point T to the position B, the convergence angle 2.alpha _a at position _a, if even grasp convergence angle 2.alpha _B at position B, a distance X _a X _B / X _A , which is the ratio of the distance to the distance X _B , can be determined uniquely.

Therefore, even when the convergence angle changes from 2α _A to 2α _B , the parallax changing unit 162 sets the formula 8 and the formula to the distance X _A that is the parallax at the convergence angle 2α _A derived by the insertion plane determination unit 160. by multiplying X _{B /} X _a by 9, it is possible to derive a new distance X _B, i.e. a new disparity in convergence angle 2.alpha _B. In this way, the parallax of the message data can be changed as needed so that the message data can follow the depth of the subject etc. with respect to the convergence angle, and the message data can be perceived at an appropriate imaging position in the reproduction space. Is possible.

  Also, the parallax changing unit 162 does not necessarily need to calculate using the formulas 8 and 9, and derives the parallax using a table in which the transition of the convergence angle and the ratio of the parallax at that time correspond one-to-one. You can also With this configuration, it is possible to significantly reduce the calculation load for calculating Equation 8 and Equation 9.

  The data superimposing unit 164 superimposes the message data on the stereoscopic video data according to the parallax derived by the insertion plane determining unit 160 or the parallax changed by the parallax changing unit 162.

  The display control unit 166 displays the stereoscopic video data on which the message data is superimposed by the data superimposing unit 164 on the stereoscopic display unit 124. At this time, the display control unit 166 processes the stereoscopic video data according to the display format of the stereoscopic display unit 124, for example, the line sequential method, the frame sequential method, or the like.

  FIG. 9 is an explanatory diagram for explaining perception of message data due to a change in the convergence angle. When the captured stereoscopic video data is reproduced after the convergence point is imaged as the position A, the position A of the convergence point in the imaging space corresponds to the position of the stereoscopic display unit 124 in the reproduction space, as shown in FIG. Thus, the observer perceives the subject R and the message data 186 as protruding from the stereoscopic display unit 124 through the left eye 188a and the right eye 188b. Then, when the stereoscopic video data with the convergence point changed to the position B is reproduced, as shown in FIG. 9B, the position B of the convergence point in the imaging space corresponds to the position of the stereoscopic display unit 124 in the reproduction space. As shown by the white arrow, the imaging position of the subject R shifts to the observer side, and the observer compares the subject R with the case of FIG. 9A through the left eye 188a and the right eye 188b. And perceive that it is more popping out.

  Further, in this embodiment, since the parallax changing unit 162 applies the same parallax to the message data 186 as in the subject R, the observer can also compare the message data 186 in the same way as the subject R in FIG. Perceived as popping out.

(Another embodiment of the video processing apparatus 100)
In the video processing apparatus 100 described above, the configuration in which the message data is superimposed in real time on the stereoscopic video data captured by the imaging unit 110 has been described. However, the object of the present embodiment can be achieved if the convergence angle, the corresponding convergence point, or the distance to the convergence point is associated with the stereoscopic video data. For example, after the stereoscopic video data and the convergence angle information are associated with each other by the video processing apparatus 100 described above and once recorded on an arbitrary recording medium 180, the parallax of the message data is subsequently used to determine the depth perception of the subject using the convergence angle information. Can be followed.

  Therefore, in the video processing apparatus 100 of the present embodiment, as long as the convergence angle is associated with the stereoscopic video data, new message data can be easily added at the time of reproduction different from that at the time of imaging. It is possible to appropriately change the parallax.

(Video processing method)
Next, a video processing method for superimposing message data on stereoscopic video data using the video processing apparatus 100 described above will be described.

  FIG. 10 is a flowchart showing a specific processing flow of the video processing method. First, the imaging units 110a and 110b of the video processing apparatus 100 generate stereoscopic video data for perceiving stereoscopic video based on binocular parallax (S200), and the subject tracking unit 152 specifies the subject specified in the stereoscopic video data. (S202), the distance deriving unit 112 derives a subject distance that is a distance between the two imaging units 110a and 110b and the tracked subject in the bisecting direction of the convergence angle (S204). The angle control unit 114 controls the convergence angle based on the derived subject distance (S206).

  Then, the convergence angle deriving unit 116 derives the convergence angles of the two imaging units 110a and 110b (S208), and the storage control unit 154 associates the convergence angle information indicating the derived convergence angle with the stereoscopic video data and performs data processing. Store in the buffer 120 (S210).

  Subsequently, the data acquisition unit 156 acquires stereoscopic video data and convergence angle information from the data buffer 120 (S212). Here, when message data that can be superimposed is generated on the stereoscopic display unit 124 (YES in S214), the message acquisition unit 158 acquires message data to be superimposed on the stereoscopic video data (S216).

  If the message data acquisition is the first time (YES in S218), the insertion surface determination unit 160 determines the message data insertion surface that intersects perpendicularly with the bisector of the convergence angle (S220). And the parallax of the message data is derived using Equation 2 (S222), and if not the first time (NO in S218), the parallax changing unit 162 uses Equation 8 and Equation 9 to express the convergence angle indicated in the convergence angle information. Based on the above, the parallax of the message data is changed so that the positional relationship between the two imaging units 110a and 110b and the insertion surface 184 in the imaging space does not change (S224).

  The data superimposing unit 164 superimposes the message data on the stereoscopic video data according to the derived parallax (S226), and the data output unit 128 outputs the stereoscopic video data on which the message data is superimposed to the stereoscopic display unit 124 (S228). However, if no message data is generated (NO in S214), the data output unit 128 outputs only the original stereoscopic video data on which the message data is not superimposed to the stereoscopic display unit 124 (S228).

  By using such a video processing method, it is possible to perceive message data at an appropriate image formation position without a sense of incongruity even when the image formation position of a subject or the like changes with the convergence angle.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each step of the video processing method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for a video processing apparatus and a video processing method that generate stereoscopic video data for causing stereoscopic video to be perceived by binocular parallax.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 110a ... Imaging part 110b ... Imaging part 112 ... Distance deriving part 114 ... Convergence angle control part 116 ... Convergence angle deriving part 118 ... Video processing part 120 ... Data buffer 152 ... Subject tracking part 154 ... Storage control part 156 ... data acquisition unit 158 ... message acquisition unit 160 ... insertion plane determination unit 162 ... parallax change unit 164 ... data superimposition unit

Claims (4)

A data acquisition unit for acquiring stereoscopic video data for perceiving stereoscopic video due to binocular parallax, and convergence angle information indicating a convergence angle formed by optical axes of two imaging units that have captured the stereoscopic video data;
A message acquisition unit for acquiring message data to be superimposed on the stereoscopic video data;
An insertion plane determination unit that determines an insertion plane for superimposing the message data, which intersects perpendicularly with a bisector of the convergence angle, and derives a parallax of the message data;
A parallax changing unit that changes the parallax of the message data so that the positional relationship between the two imaging units and the insertion surface in the imaging space does not change based on the convergence angle indicated in the convergence angle information;
A data superimposing unit that superimposes the message data on the stereoscopic video data according to the parallax;
A data output unit for outputting the superimposed stereoscopic video data;
A video processing apparatus comprising: The parallax changing unit changes the parallax based on a ratio obtained by multiplying each of cosα _B / cosα _A and sin α _B / sin α _A by a predetermined coefficient and adding the same when the convergence angle changes from 2α _A to 2α _B. The video processing apparatus according to claim 1. A subject tracking unit for tracking the identified subject;
A distance deriving unit for deriving a subject distance that is a distance between the two imaging units and the tracked subject in the bisector direction of the convergence angle;
A convergence angle control unit for controlling the convergence angle based on the derived subject distance;
The video processing apparatus according to claim 1, further comprising: Obtaining stereoscopic video data for perceiving stereoscopic video due to binocular parallax, and convergence angle information indicating a convergence angle formed by optical axes of two imaging units that have captured the stereoscopic video data;
Obtaining message data to be superimposed on the stereoscopic video data;
If acquisition of the message data is the first time, the insertion plane for superimposing the message data that intersects perpendicularly with the bisector of the convergence angle is determined, and the parallax of the message data is derived for the first time. Otherwise, based on the convergence angle indicated in the convergence angle information, change the parallax of the message data so that the positional relationship between the two imaging units and the insertion surface in the imaging space does not change,
Superimposing the message data on the stereoscopic video data according to the parallax;
A video processing method comprising outputting the superimposed stereoscopic video data.

Images