JP2013085054A - Stereoscopic video processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic video processor capable of providing optimal parallax for plural viewers by adjusting the parallax for stereoscopic video according to the position and distance of the viewers with respect to the TV.SOLUTION: When a stereoscopic video is being viewed by plural persons, viewing attitude of each of the viewers is estimated based on the position and direction of the face of the viewer. The parallax is adjusted by weighing to each viewer. With this, optimal parallax for viewers who are interested in the video is provided while neglecting the influence by a viewer who is not interested in the video.

Description

  The present invention relates to a stereoscopic video processing apparatus. More specifically, the present invention relates to a stereoscopic video processing apparatus that adjusts the parallax of a stereoscopic video.

  In recent years, display devices that display stereoscopic images have been developed.

  2. Description of the Related Art Conventionally, a display device that displays stereoscopic images is known that adjusts the amount of parallax according to coordinates from coordinate detection mounted on glasses (see, for example, Patent Document 1).

JP 2011-141381 A

  According to the above conventional technique, an appropriate stereoscopic video is displayed by adding parallax according to the position and distance of the viewer with respect to the television.

  However, when a plurality of people watch a stereoscopic video at the same time, the stereoscopic video cannot be displayed appropriately.

  For example, it is assumed that two viewers A and B are viewing stereoscopic video. Here, if the parallax is adjusted in accordance with the position and distance of the viewer A, the adjusted parallax may be inappropriate for the viewer B. As a result, the viewer B may feel an excessive three-dimensional effect or may not obtain the three-dimensional effect.

  In order to solve the conventional problems, a stereoscopic video viewing apparatus according to the present invention includes a viewer image acquisition unit that acquires an image of a viewer who is viewing a stereoscopic video, and the viewer image acquisition unit that acquires the viewer image. An inclination detection unit that analyzes an image and detects a tilt of a face for each viewer; a position detection unit that analyzes the image acquired by the viewer image acquisition unit and detects a viewing position for each viewer; A parallax amount adjustment unit that performs weighting for parallax amount adjustment for each viewer based on the face inclination for each viewer detected by the tilt detection unit and the viewing position for each viewer detected by the position detection unit And a parallax synthesis unit that adds parallax to the left-eye video and the right-eye video constituting the stereoscopic video based on the weighting of the parallax amount adjustment unit.

  According to the stereoscopic video processing apparatus of the present invention, the positions and viewing states of a plurality of viewers on the television are determined, so that a parallax suitable for all viewers can be added to the stereoscopic video.

1 is a block diagram including a video processing apparatus according to a first embodiment and external inputs to the video processing apparatus. FIG. 3 is a block diagram of an inclination detection unit according to the first embodiment. FIG. 3 is a block diagram of a weight calculation unit according to the first embodiment. The figure which showed the aspect of the viewer who uses the video processing apparatus concerning Embodiment 1.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
(Embodiment 1)
The first embodiment will be described below with reference to FIGS.
(Device configuration)
The configuration of the video processing apparatus according to the first embodiment will be described below with reference to FIG.

  FIG. 1 is a block diagram including a video processing apparatus according to the first embodiment and external inputs to the video processing apparatus.

  Hereinafter, each component shown in FIG. 1 will be described.

  The video processing apparatus 100 adds parallax to video content based on viewer images acquired from the outside. The video processing apparatus 100 includes a video analysis unit 104, a parallax generation unit 105, and a parallax synthesis unit 106. Specifically, the video processing apparatus 100 is realized by processing a large-scale circuit element such as an LSI or a program.

  The left-eye video acquisition unit 101 acquires or generates a left-eye video signal in a stereoscopic video. The left-eye video acquisition unit 101 acquires a left-eye video signal from a stereoscopic display recording-compatible camera, package media in which stereoscopic display content is recorded, a broadcast signal, or the like. In order for the left-eye video acquisition unit 101 to acquire the left-eye video signal from the stereoscopic display recording compatible camera, for example, a signal from an image sensor that captures the left-eye video may be acquired. In order for the left-eye video acquisition unit 101 to acquire the left-eye video signal from the package media or the broadcast signal, a signal constituting a frame corresponding to the left-eye video may be acquired from the decoded video signal. In order to acquire the signal which comprises the said flame | frame concretely, selectively acquiring the signal which comprises an even-numbered frame or an odd-numbered frame from a video signal according to a fixed rule is mentioned as an example. Alternatively, when a specific frame of a video signal is composed of a combination of a left-eye video and a right-eye video, an example is to selectively acquire a video signal constituting a specific area of the frame. The case where a specific frame of a video signal is composed of a combination of a left-eye video and a right-eye video is specifically a case where the video signal is configured according to a format format such as side-by-side or top-and-bottom. Further, as an example in which the left-eye video acquisition unit 101 generates a left-eye video signal, there is a case where a video that does not depend on an external video source, such as a three-dimensional OSD (On Screen Display), is generated.

  The right-eye video acquisition unit 102 acquires or generates a right-eye video signal in a stereoscopic video. Detailed functions are the same as those of the left-eye video acquisition unit 101.

  The viewer image acquisition unit 103 acquires an image obtained by photographing the viewer. Specifically, the viewer image acquisition unit 103 acquires an image captured by an external device such as a camera. The camera may be built in the video processing apparatus 100 or may be arranged around the viewer, but the position facing the viewer from the vicinity of the display that displays the video processed by the video processing apparatus 100 It is desirable to arrange in. This is because the video analysis unit 104 described later needs to detect the position of the viewer with respect to the display from the image acquired by the viewer image acquisition unit 103.

  The video analysis unit 104 analyzes the image acquired by the viewer image acquisition unit 103.

  The parallax generation unit 105 acquires the parallax of the original video from the left-eye video and the right-eye video. Furthermore, the parallax generation unit 105 calculates the amount of parallax to be combined with the left-eye video and the right-eye video based on the acquired parallax and the analysis result of the video analysis unit.

  The parallax synthesis unit 106 synthesizes the parallax generated by the parallax generation unit 105 into a left-eye video and a right-eye video.

  Hereinafter, each component of the video analysis unit 104 will be described.

  The face area detection unit 401 detects the presence of the viewer's face from the image acquired by the viewer image acquisition unit 103. Specifically, the face area detection unit 401 detects the presence of a face by detecting a skin-colored circular area from the acquired image.

  The inclination detection unit 402 detects the inclination of each face detected by the face area detection unit 401. Specifically, the inclination detection unit 402 cuts out a black rectangular area in the face area and calculates the inclination of the rectangular area. The black rectangular region corresponds to the region of the stereoscopic display glasses worn by the viewer. If the viewer is tilted, the glasses worn by the viewer are also tilted. Therefore, the inclination of the black rectangular area can be regarded as the inclination of the viewer's neck.

  The tilt detection unit 402 will be described in more detail with reference to FIG. FIG. 2 is a block diagram of the inclination detecting unit according to the first embodiment.

  If the tilt detection unit 402 is expressed in detail by a functional block diagram, the tilt detection unit 402 includes a glasses region detection unit 421 and a tilt detection unit 422.

  The eyeglass region detection unit 421 further detects a region of stereoscopic display glasses from the face region detected by the face region detection unit 401. Specifically, in the detected face area, the black area occupies a certain area or more of the face area, and detects that the black area has a high degree of squareness. The black area can be detected according to the luminance of the pixel. The degree of squareness can be determined by calculating the X-axis range and the Y-axis range of the black region, defining the rectangular region, and then determining the proportion of black in the rectangular region.

  The tilt detection unit 422 detects how much the region of the 3D display glasses detected by the glasses region detection unit 421 is tilted with respect to the television. In order to detect the inclination, the detected eyeglass area is matched with a rectangular area prepared for several predetermined angles, and the inclination is detected from the corresponding pattern.

  Returning to FIG. 1 again, the remaining components will be described.

  The position detection unit 403 detects the position of the viewer from the face detected by the face area detection unit 401. Specifically, the position detection unit 403 detects in which position the face detected by the face area detection unit 401 exists in the image acquired by the viewer image acquisition unit 103. There is a method in which a spectacle region is acquired from the spectacle region detection unit 421 and performed. The viewer's position is determined by assuming that the viewer is at the center of the acquired glasses region. Here, the position is at least the distance of the face from the center of the image (whether the viewer is in front or at the end). Further, the size of the face (whether the face is near or far away) may be further detected.

  The viewer number detection unit 404 counts the viewer's face or position information detected by the face area detection unit 401 or the position detection unit 403, and specifies the number of viewers.

  The weight calculation unit 405 determines the viewing status of the viewer based on the output of the tilt detection unit 402 and the output of the position detection unit 403, and weights each viewer for parallax amount adjustment.

  The weight calculation unit 405 will be described in detail with reference to FIG. FIG. 3 is a block diagram of the weight calculation unit according to the first embodiment.

  The first weight calculating unit 451 calculates a weight for adjusting the parallax amount from the tilt amount detected by the tilt detecting unit 402. Specifically, the first weight calculation unit 451 increases the weight for the viewer as the inclination detected by the inclination detection unit 402 is closer to the television.

  The second weight calculation unit 452 is for calculating a weight for adjusting the parallax amount from the position of the viewer with respect to the television detected by the position detection unit 403. Specifically, the second weight detection unit increases the weight for the viewer located at the left and right center of the acquired image based on the viewer position information detected by the position detection unit 403.

The weight synthesis unit 453 determines the final weight when adjusting the parallax amount from the respective weights calculated by the first weight calculation unit 451 and the second weight calculation unit 452 and the number of viewers detected in 404. Is. Details will be described later.
(About weighting)
Hereinafter, video processing in the video processing apparatus configured as described above will be described with reference to FIG.

  FIG. 4 is a diagram illustrating a viewer using the video processing apparatus according to the first embodiment.

  FIG. 4A is a diagram illustrating a state in which each viewer is viewing the video processed by the video processing device with the viewer facing each other and substantially equally positioned with respect to the center of the screen.

  The first weight calculation unit 451 equally weights each viewer. This is because the inclination of each viewer's face is parallel.

  The second weight calculation unit 452 weights the viewer positioned at the center more heavily than the viewer positioned at the left and right.

  In such a case, the parallax synthesis unit 106 synthesizes the parallax using the same parallax as the current video.

  FIG. 4B is a diagram showing a state in which each viewer is facing directly, the right viewer is slightly deviated from the center, and the left viewer is deviated greatly from the center.

  The first weight calculation unit 451 equally weights each viewer. This is because the inclination of each viewer's face is parallel.

  The second weight calculation unit 452 weights the right viewer heavily. This is because the viewer on the left can determine that he is not interested in the screen.

  In such a case, the parallax synthesis unit 106 performs parallax synthesis in accordance with the viewer located at the center.

  FIG. 4C is a diagram illustrating a state where the viewer on the left side is viewing sideways.

  The first weight calculation unit 451 weights the right viewer heavily. This is because it can be determined that the viewer on the left has not concentrated on the screen because his / her face is tilted.

  The second weight calculation unit 452 equally weights each viewer. This is because the distance from the center is almost equal for the left and right viewers.

  In such a case, the parallax synthesis unit 106 performs parallax synthesis in accordance with the viewer positioned on the right side.

  FIG. 4D is a diagram illustrating a situation in which the left viewer is viewing sideways and the right viewer is viewing away from the center.

  The first weight calculation unit 451 weights the right viewer heavily. This is because it can be determined that the viewer on the left has not concentrated on the screen because his / her face is tilted.

  The second weight calculation unit 452 weights the left viewer heavily. This is because it can be determined that the viewer on the right side is not interested in the screen.

  In such a case, video processing is performed so that the viewer can appropriately set which one of the weights of the first weight calculation unit 451 and the weight of the second weight calculation unit 452 the parallax synthesis unit 106 prioritizes. It is conceivable that the apparatus 100 is configured. This is because a guideline can be given when the calculation result of the first weight calculation unit 451 and the calculation result of the second weight calculation unit 452 are in conflict.

  In the first weight calculation unit 451, the weight is linearly set according to the inclination of the glasses, but the weight may be set in steps of several to several tens of steps. Similarly, the second weight calculation unit 452 may set a weight linearly or set a stepped weight according to the distance from the center of the screen.

  The present invention is effective as a video processing apparatus in a home television or the like.

100 Video processing apparatus 101 Left-eye video acquisition unit 102 Right-eye video acquisition unit 103 Viewer image acquisition unit 104 Video analysis unit 105 Parallax generation unit 106 Parallax synthesis unit 401 Face area detection unit 402 Inclination detection unit 403 Position detection unit 404 Viewer Number detecting unit 405 Weight calculating unit 421 Glasses region detecting unit 422 Inclination detecting unit 451 First weight calculating unit 452 Second weight calculating unit 453 Weight combining unit

Claims (1)

A viewer image acquisition unit that acquires an image of a viewer who is viewing stereoscopic video;
An inclination detection unit that analyzes the image acquired by the viewer image acquisition unit and detects the inclination of the face for each viewer;
A position detection unit that analyzes the image acquired by the viewer image acquisition unit and detects a viewing position for each viewer;
A parallax amount adjustment unit that performs weighting for parallax amount adjustment for each viewer based on the face inclination for each viewer detected by the tilt detection unit and the viewing position for each viewer detected by the position detection unit; ,
A parallax synthesis unit that adds parallax to the left-eye video and the right-eye video that form the stereoscopic video based on the weighting of the parallax amount adjustment unit;
3D image processing apparatus.

Images