JP2013031057A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a user to visually recognize video image data switched from 2D video scene to 3D video scene without feeling strangeness while the user is visually recognizing mixed video image data of the 2D video scene and the 3D video scene.SOLUTION: An image processing apparatus comprises a video acquisition section 112 for acquiring mixed video image data of 2D video scene and 3D video scene to be perceived in a three-dimensional state; a switchover detection section 130 for detecting the timing point of a switchover from the 2D video scene to the 3D video scene with the video acquisition section 112; a parallax derivation section 134 for deriving parallaxes at a prescribed condition in the 3D video scene; a target set section 136 for setting a parallax as a target value at a prescribed time after the switchover timing point among parallaxes derived by the parallax derivation section; and a parallax adjustment section 138 for adjusting the 3D video scene to make an absolute value of parallaxes in the 3D video scene gradually increase from a predetermined value to the target value over a predetermined time after the changeover timing point.

Description

  The present invention relates to a video processing apparatus and a video processing method capable of reproducing video data for perceiving a stereoscopic video.

  In recent years, the technology of stereoscopic video (3D video) that presents two videos having binocular parallax (horizontal parallax) on a display and makes the user perceive that the subject image exists stereoscopically has attracted attention. I'm bathing. Two images used in such a technique are images taken at different optical axes (viewpoints), for example. Hereinafter, an image having no stereoscopic effect is referred to as a 2D image with respect to such a 3D image.

  When broadcasting such a video, for example, when switching between a channel broadcasting 2D video and a channel broadcasting 3D video, the user transitions from a video with no parallax to a video with parallax. The sudden parallax change may make you feel uncomfortable. Therefore, a configuration has been proposed in which the parallax is changed and output sequentially over time when switching channels (for example, Patent Document 1).

  In addition to broadcasting, there are display devices that can generate and display 3D video from 2D video acquired from a storage medium or the like, for example. For such a display device, the mutual switching timing between 2D video and 3D video is determined from a preset viewing time limit, etc., so that the parallax gradually increases when switching from 2D video to 3D video. A technique for generating and outputting a 3D video has been proposed (for example, Patent Document 2).

  On the other hand, regarding an imaging apparatus capable of generating video data in which 2D video scenes (2D video scenes (scenes)) and 3D video scenes (3D video scenes (scenes)) are mixed, are 2D video scenes or 3D video scenes? A technique for outputting control information indicating whether or not as a control signal has been proposed (for example, Patent Document 3).

JP-A-11-164328 Japanese Patent Laid-Open No. 2004-328566 JP 2008-42645 A

  Even when a user views video data including a 2D video scene and a 3D video scene generated by the technique described in Patent Document 3 described above, 3D from a 2D video scene is switched in the same manner as the above-described switching of the broadcast channel. When switching to a video scene, the user may feel uncomfortable with a sudden change in parallax.

  Therefore, in view of such problems, the present invention allows a user to visually recognize a video without a sense of incongruity when switching from a 2D video scene to a 3D video scene while the user is viewing video data in which a 2D video scene and a 3D video scene are mixed. It is an object of the present invention to provide a possible video processing apparatus and video processing method.

  In order to solve the above-described problems, a video processing apparatus according to the present invention includes a video acquisition unit that acquires video data including a 2D video scene and a 3D video scene that is perceived stereoscopically, and a 3D video from the 2D video scene. Of the parallax derived from the parallax derived by the parallax deriving unit, the parallax deriving unit for deriving the parallax of the predetermined condition in the 3D video scene, the parallax after the predetermined time from the switching time is the target value And a parallax adjustment unit that adjusts the 3D video scene so that the absolute value of the parallax of the 3D video scene gradually increases from the predetermined value to the target value over a predetermined time from the switching time point. It is characterized by.

  The video processing apparatus may further include a subject specifying unit that specifies a predetermined subject, and the parallax deriving unit may derive the parallax of the predetermined subject specified by the subject specifying unit.

  In order to solve the above problems, the video processing method of the present invention acquires 2D video scenes and video data in which 3D video scenes that are perceived stereoscopically are mixed, and switches from 2D video scenes to 3D video scenes. Detect a time point, derive a parallax of a predetermined condition in the 3D video scene, set a parallax after a predetermined time from the switching time point as a target value among the derived parallaxes, and set the parallax of the 3D video scene for a predetermined time from the switching time point The 3D video scene is adjusted so that the absolute value of the value gradually increases from a predetermined value to a target value.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to visually recognize a video without a sense of incongruity when the user is viewing video data in which a 2D video scene and a 3D video scene are mixed while switching from a 2D video scene to a 3D video scene.

It is the functional block diagram which showed the schematic function of the video processing apparatus. It is explanatory drawing for demonstrating a parallax adjustment process. It is a flowchart which shows the flow of a process of a video processing method.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Video processing apparatus 100)
FIG. 1 is a functional block diagram illustrating schematic functions of the video processing apparatus 100. In FIG. 1, solid arrows indicate the flow of data, and broken arrows indicate the flow of control signals. As shown in FIG. 1, the video processing apparatus 100 includes an operation unit 110, a video acquisition unit 112, a temporary storage unit 114, a decoding unit 116, a synthesis unit 118, a video output unit 120, and a central control unit 122. It is comprised including.

  The operation unit 110 includes operation keys, a cross key, a joystick, and the like, and accepts user operation inputs. The video acquisition unit 112 mixes a 3D perceived 2D video scene and a 3D video scene from a 3D / 2D imaging device, a DVD, a BD (Blu-ray Disc), a nonvolatile memory, a digital TV tuner, and the like. Acquired video data.

  In such video data, at least a switching time point from the 2D video scene to the 3D video scene (hereinafter referred to as a 3D switching time point) is specified, so whether the scene (scene) is a 2D video scene or a 3D video scene. Control information, which is information indicating

  Here, the 3D video scene is a scene of a stereoscopic video (3D video) that presents two videos having binocular parallax (horizontal parallax) and makes the user perceive as if the subject image exists stereoscopically. is there. Such two images are, for example, images taken at different optical axes (viewpoints). In the present embodiment, a video scene without a stereoscopic perception effect is referred to as a 2D video scene with respect to a 3D video scene having a stereoscopic perception effect.

  The temporary storage unit 114 includes an HDD, a flash memory, a RAM, and the like, and temporarily holds the video data acquired by the video acquisition unit 112. The decoding unit 116 reads the video data held in the temporary storage unit 114 and performs a decoding process. In addition, when the video data is a 2D video scene, the decoding unit 116 outputs the decoded video data directly to the video output unit 120 according to the control of the central control unit 122. When the video data is a 3D video scene, video data (left-eye video data and right-eye video data) that is two video data having decoded binocular parallax is output to the synthesis unit 118.

  The synthesizing unit 118 displays two pieces of video data output from the decoding unit 116, for example, side-by-side method, top-and-bottom method, line-by-line method, frame sequential method, and the like, according to control of the parallax adjustment unit 138 described later. It is synthesized into a format corresponding to 150 stereoscopic display methods.

  The video output unit 120 outputs the video data of the 2D video scene output from the decoding unit 116 or the video data including the 3D video scene adjusted by the parallax adjustment unit 138 to the display device 150.

  The central control unit 122 manages and controls the entire video processing apparatus 100 by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, and the like. In the present embodiment, the central control unit 122 also functions as the switching detection unit 130, the subject specifying unit 132, the parallax derivation unit 134, the target setting unit 136, and the parallax adjustment unit 138.

  The switching detection unit 130 acquires the decoded video data from the decoding unit 116, and detects the 3D switching point based on the control information of the acquired video data. In addition, when there is no control information in the video data, the switching detection unit 130 can determine whether the scene is a 2D video scene or a 3D video scene from the difference in the data structure, and can detect the 3D switching time point.

  The subject specifying unit 132 specifies a predetermined subject. For example, the subject specifying unit 132 may use a person detected by face detection or the like as a predetermined subject. Furthermore, when a plurality of persons are detected, for example, a person who is reflected largely or a foreground person specified by parallax or the like may be set as a predetermined subject. Since various existing techniques can be applied to the identification of the subject, for example, face detection, detailed description is omitted here.

  The parallax deriving unit 134 derives parallax under a predetermined condition in the 3D video scene. In the present embodiment, the parallax of the predetermined condition is the parallax of the subject specified by the subject specifying unit 132. That is, the parallax deriving unit 134 derives the parallax of the predetermined subject specified by the subject specifying unit 132.

  The target setting unit 136 sets a parallax of a predetermined subject in a 3D video scene after a predetermined time (for example, 3 seconds) from the 3D switching time point as a target value among the parallaxes derived by the parallax deriving unit 134.

  The parallax adjustment unit 138 adjusts the 3D video scene so that the absolute value of the parallax of the predetermined subject in the 3D video scene gradually increases from a predetermined value (for example, 0) to a target value over a predetermined time from the 3D switching time point, The combining unit 118 performs the combining.

  For the adjustment that gradually increases the absolute value of the parallax for the video data of the 3D video scene, the parallax adjustment unit 138 may linearly increase the absolute value of the parallax from 0 to the target value with respect to the time axis, for example, It may be increased in a curve or may be increased in stages.

  Subsequently, the parallax derivation processing by the parallax derivation unit 134 will be described in detail, and then the parallax adjustment processing by the parallax adjustment unit 138 will be described in detail.

(Parallax derivation process)
For example, the parallax deriving unit 134 performs pattern matching on the subject specified by the subject specifying unit 132 with respect to the two pieces of video data acquired from the decoding unit 116, and arranges each frame (in time series constituting one video). The parallax in the three-dimensional video is referred to as the left and right still image data.

  However, since the parallax occurs only in the horizontal direction in the 3D video scene, the direction of the parallax can be simply indicated by a positive / negative sign (+ −). Here, for example, the direction in which the image is formed on the user side (protruding side) from the display position of the video processing device 100 is normal, and the direction in which the image is formed on the opposite side (back side) from the display position of the video processing device 100 Is negative.

  By such pattern matching processing, the parallax deriving unit 134 derives the parallax of the subject at the 3D switching time of the video data of the 3D video scene and the parallax of the subject after a predetermined time from the 3D switching time.

  Here, the flow of the process of deriving the parallax of the subject after a predetermined time from the 3D switching time differs depending on the acquisition destination of the video data. Hereinafter, each of the two types of acquisition means will be described.

  For example, when the video acquisition unit 112 acquires video data stored in a storage medium such as a DVD, a BD, or a non-volatile memory, video data to be reproduced already exists. The control information included in the video data is searched via the acquisition unit 112. Then, the switching detection unit 130 detects all 3D switching points in the video data.

  The video acquisition unit 112, under the control of the switching detection unit 130, detects the video data of the 3D video scene at all 3D switching points detected by the switching detection unit 130, and the video of the 3D video scene after a predetermined time at each 3D switching point. Data is acquired and stored in the temporary storage unit 114.

  The subject specifying unit 132 specifies a predetermined subject at each 3D switching time point based on the video data held in the temporary storage unit 114. The parallax deriving unit 134 derives the parallax of the predetermined subject after each 3D switching time and a predetermined time after each 3D switching time. The target setting unit 136 sets the parallax of a predetermined subject after a predetermined time from each 3D switching time as a target value.

  At this time, it is desirable that the predetermined subject specified at the 3D switching time point is reflected as a subject also in the video data of the 3D video scene after a predetermined time from the 3D switching time point. Therefore, the subject specifying unit 132 preferentially specifies a subject that is reflected in any video data after a predetermined time from the 3D switching time and the 3D switching time as a predetermined subject.

  In the present embodiment, it is assumed that the parallax adjustment processing is not performed when a subject that is reflected in any video data at a 3D switching time point and a predetermined time after the 3D switching time point is not specified.

  As described above, when acquiring video data from the storage medium, the parallax deriving unit 134 derives parallax necessary for parallax adjustment of the video data in advance by search processing before reproducing the video data.

  For example, when the video acquisition unit 112 acquires video data from a digital TV tuner that has received a broadcast, the video data to be played back is acquired in real time. Temporarily hold video data. The decoding unit 116 sequentially decodes the video data delayed from the temporary storage unit 114 by a predetermined time.

  When the switching detection unit 130 detects the 3D switching point in the video data, the subject specifying unit 132 causes the decoding unit 116 to decode the video data after a predetermined time from the 3D switching point held in the temporary storage unit 114. The decoded video data is acquired. Then, the subject specifying unit 132 specifies a predetermined subject based on the 3D switching time and the video data of the 3D video scene of the video data after a predetermined time from the 3D switching time.

  Then, the parallax deriving unit 134 derives the parallax of the predetermined subject after the 3D switching time and a predetermined time after the 3D switching time. The target setting unit 136 sets the parallax of a predetermined subject after a predetermined time from the 3D switching time as a target value.

  Thus, when acquiring video data through broadcasting, the parallax deriving unit 134 can also derive the parallax of the subject after a predetermined time from the 3D switching time by reproducing the video data with a predetermined time delay.

(Parallax adjustment processing)
FIG. 2 is an explanatory diagram for explaining the parallax adjustment processing. FIG. 2A to FIG. 2D show a time series of video data extracted at 1.5 second intervals. FIG. 2A shows a 2D video scene, and FIGS. ) Shows video data of a 3D video scene, the video data before parallax adjustment is shown on the left side of the axis (indicated by arrows) indicating the passage of time, and the video data after parallax adjustment is shown on the right side.

  2B to 2D, the video data of the 3D video scene is shown by superposing the left-eye video data and the right-eye video data for easy understanding.

  In FIG. 2A, the video data of the 2D video scene shows only the video data 160a before the adjustment because the parallax is not adjusted. Then, the video data 160b shown in FIG. 2B is switched to a 3D video scene. At this time, the subject specifying unit 132 specifies the person 170 as a predetermined subject from the video data 160b before adjustment.

  The parallax deriving unit 134 derives the parallax of the person 170 in the video data 160b, and further derives the parallax of the person 170 in the video data after a predetermined time (for example, 3 seconds). The target setting unit 136 sets the latter parallax as a target value. Here, it is assumed that the parallax deriving unit 134 derives the parallax of the person 170 in the video data 160d illustrated in FIG. In order to facilitate understanding, it is assumed that the parallax of the person 170 does not change in the video data 160b, 160c, and 160d before adjustment.

  The parallax adjustment unit 138 adjusts the video data so that the parallax of the person 170 becomes 0 as illustrated in FIG. 2B (see the video data 162b). Here, the parallax adjustment unit 138 relatively shifts the entire left-eye video data and the entire right-eye video data in the horizontal direction. Specifically, the parallax adjustment unit 138 converts the left-eye video data to the right-eye video data. To the left and adjust the parallax by superimposing the person 170. The shift amount at this time is equal to the magnitude of the parallax of the predetermined subject at the time of 3D switching.

  At this time, for example, it is assumed that the number of horizontal pixels of the left-eye video data and the right-eye video data is larger by a predetermined number than the horizontal pixel number of the output video data. As a result of the parallax adjustment, even if there is a portion where the left-eye video data and the right-eye video data do not overlap, it is possible to ensure the number of horizontal pixels to be output only by the overlapping portion.

  Then, the parallax adjustment unit 138 sequentially adjusts the parallax of the video data, and after 1.5 seconds from the 3D switching time, adjusts the parallax of the video data 160c as illustrated in FIG. Video data 162c in which the absolute value of the parallax is half the target value.

  In this way, the parallax adjustment unit 138 sequentially adjusts the parallax of the video data, and after 3 seconds from the 3D switching time point, adjusts the parallax of the video data 160d as illustrated in FIG. It is assumed that the video data 162d has an absolute value of a target value derived in advance.

  As described above, the video processing apparatus 100 according to the present embodiment derives the parallax of a predetermined subject after a predetermined time in advance at the time of 3D switching, and gradually sets the parallax of the predetermined subject to the target value using the parallax as a target value. Adjust the 3D video data so that they are closer. Therefore, the user can visually recognize the video without feeling uncomfortable without being exposed to a sudden change in parallax.

  Also, the parallax differs for each subject in the video. In the present embodiment, the parallax adjustment unit 138 adjusts the 3D video scene so that the parallax of a predetermined subject that is likely to be watched by the user changes gradually. It is possible to more reliably avoid a situation in which the user who feels uncomfortable feels uncomfortable.

  Also, as shown in FIGS. 2B to 2D, in the video data 160b, 160c, and 160d before parallax adjustment, the tower 172 as the background is assumed to have 0 parallax, for example. In this case, in the adjusted video data 162b, the tower 172 has the same parallax as the parallax of the person 170 before the adjustment. Then, as the tower 172 shifts to the video data 162c and 162d, the parallax gradually becomes zero.

  Thus, regarding the tower 172, the change in parallax is large at the time of 3D switching. However, since it is assumed that the user is gazing at the person 170 as the predetermined subject, it can be said that the user is less likely to feel discomfort than when the person 170 has a similar change in parallax.

  Hereinafter, a specific processing flow of the video processing apparatus 100 described above will be described with reference to a flowchart of FIG. Here, a process for obtaining video data through broadcasting is shown.

(Video processing method)
FIG. 3 is a flowchart showing the flow of processing of the video processing method. The process shown in FIG. 3 is repeatedly executed at a predetermined frame period while the video processing apparatus 100 acquires video data.

  As shown in FIG. 3, the video acquisition unit 112 acquires video data in which a 2D video scene and a stereoscopically perceived 3D video scene are mixed through broadcasting (S200). Then, the temporary storage unit 114 temporarily holds the video data (S202).

  The decoding unit 116 sequentially reads out the video data held in the temporary storage unit 114 from the latest video data held for a predetermined time (S204) and decodes it (S206). The switch detection unit 130 determines whether the decoded video data is a 3D video scene based on the control information (S208). If it is not a 3D video scene (NO in S208), the process proceeds to video data output processing step S222.

  When it is a 3D video scene (YES in S208), the switching detection unit 130 determines whether or not the previous frame was a 2D video scene (S210). If it is not a 2D video scene but a 3D video scene one frame before (NO in S210), the process proceeds to a parallax adjustment end determination processing step S220.

  If the previous frame was a 2D video scene (YES in S210), the switching detection unit 130 regards that a 3D switching time point has been detected, and the subject specifying unit 132 specifies a predetermined subject for the 3D switching time point. (S212). At this time, the subject specifying unit 132 causes the decoding unit 116 to decode the video data after a predetermined time from the 3D switching time held in the temporary storage unit 114, and also refers to the video data after the predetermined time from the 3D switching time. The subject is identified.

  The parallax deriving unit 134 derives the parallax of the predetermined subject at the time of 3D switching, and derives the parallax of the predetermined subject after a predetermined time from the time of 3D switching (S214). The target setting unit 136 sets, as a target value, the parallax of a predetermined subject that is a predetermined time after the 3D switching time among the parallaxes derived by the parallax deriving unit 134 (S216).

  Then, the parallax adjustment unit 138 adjusts the 3D video scene so that the absolute value of the parallax of the 3D video scene gradually increases from 0, which is a predetermined value, to a target value over a predetermined time from the 3D switching time point. 118 is synthesized (S218). After the frame, the 3D video scene is adjusted for a predetermined time.

  Therefore, when the switching detection unit 130 does not detect the 3D switching time (NO in S210), the parallax adjustment unit 138 determines whether or not the parallax adjustment is finished (S220). If the parallax adjustment has been completed (YES in S220), the process proceeds to video data output processing step S222.

  If the parallax adjustment has not been completed (NO in S220), the parallax adjustment unit 138 adjusts the 3D video scene so that the absolute value of the parallax of the subject gradually increases to the target value set in the target setting processing step S216. (S218).

  Then, the video output unit 120 outputs the video data of the 2D video scene output from the decoding unit 116 or the video data including the 3D video scene adjusted by the parallax adjustment unit 138 to the display device 150 (S222).

  According to the above-described video processing method, the user can view the video without any sense of incongruity at the time of 3D switching while the user is viewing the video data in which the 2D video scene and the 3D video scene are mixed.

  In the embodiment described above, the subject specifying unit 132 specifies a predetermined subject based on the 3D switching time and the video data after a predetermined time from the 3D switching time when the switching detection unit 130 detects the 3D switching time. However, when the switching detection unit 130 detects the 3D switching time point, it uses the video data held in the temporary storage unit 114 and based on video data having a time width from the 3D switching time point to a predetermined time after the 3D switching time point. Thus, a predetermined subject may be specified. In this case, for example, of the faces detected for video data for a predetermined time, a person having a face with the highest average value or maximum value such as a point indicating the certainty of face detection may be set as the predetermined subject.

  In the above-described embodiment, when acquiring video data from the storage medium, the 3D switching time point is first searched and detected for the entire video data. However, when acquiring video data from a storage medium, the video processing apparatus 100 may derive a parallax after a predetermined time by providing a buffer for a predetermined time as in the case of acquiring video data through broadcasting. .

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each step in the video processing method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include parallel or subroutine processing.

  INDUSTRIAL APPLICABILITY The present invention can be used for a video processing apparatus and a video processing method that can reproduce video data for perceiving a stereoscopic video.

DESCRIPTION OF SYMBOLS 100 ... Video processing apparatus 112 ... Video acquisition part 120 ... Video output part 130 ... Switching detection part 132 ... Subject specific | specification part 134 ... Parallax derivation | leading-out part 136 ... Target setting part 138 ... Parallax adjustment part

Claims (3)

A video acquisition unit that acquires video data in which a 2D video scene and a stereoscopically perceived 3D video scene are mixed;
A switching detection unit for detecting a switching time point from the 2D video scene to the 3D video scene;
A parallax deriving unit for deriving parallax under a predetermined condition in the 3D video scene;
Among the parallaxes derived by the parallax deriving unit, a target setting unit that sets a parallax after a predetermined time from the switching time point as a target value;
A parallax adjustment unit that adjusts the 3D video scene so that the absolute value of the parallax of the 3D video scene gradually increases from a predetermined value to the target value over the predetermined time from the switching time;
A video processing apparatus comprising: A subject specifying unit for specifying a predetermined subject;
The video processing apparatus according to claim 1, wherein the parallax deriving unit derives parallax of the predetermined subject specified by the subject specifying unit. Obtain video data in which 2D video scenes and 3D video scenes that are perceived stereoscopically are mixed,
Detecting a switching time from the 2D video scene to the 3D video scene;
Deriving a predetermined parallax in the 3D video scene,
Of the derived parallax, set a parallax after a predetermined time from the switching time as a target value,
A video processing method comprising adjusting the 3D video scene so that an absolute value of parallax of the 3D video scene gradually increases from a predetermined value to the target value over a predetermined time from the switching time.

Images