JP2017525198A - Stereoscopic depth adjustment and focus adjustment - Google Patents

Abstract

A method of adjusting depth in a stereoscopic video may include generating a left eye view frame of the stereoscopic video. The left eye view frame may include a plurality of left eye view frame elements. The method may also include generating a right eye view frame of the stereoscopic video. The right eye view frame may correspond to the left eye view frame and may include a plurality of right eye view frame elements. Further, each right eye view frame element may correspond to one of the left eye view frame elements. The method may further include determining an offset between each right eye view frame element and its corresponding left eye view frame element. Further, the method may include applying a uniform multiplication factor to each offset such that the perceived depth associated with the stereoscopic video is adjusted on a substantially uniform scale.

Description

  The present disclosure relates to stereoscopic data processing, including depth adjustment and focus adjustment.

  Three-dimensional (stereoscopic) imaging for generating three-dimensional video (eg, television, movies, etc.) has become increasingly popular in recent years. One reason is that there have been significant advances in the cameras and post-production used to create 3D video. Another reason that 3D video has become widespread is that the entertainment crowd appears to be willing to pay a premium for this special effect.

  However, in the three-dimensional imaging technique, it is considerably more expensive to take a video using the three-dimensional technique than to use a two-dimensional (monoscopic) technique. In addition, there are numerous 2D videos that have already been generated but have not been shot using 3D technology.

  As such, many people have attempted to convert 2D video to 3D video. However, these methods of converting 2D video to 3D video may not work, require a lot of resources, and / or do not produce acceptable results (eg, cardboard cut-out effect).

  The claimed subject matter is not limited to embodiments that overcome or operate any disadvantages only in an environment such as those described above. Rather, this background is only provided to illustrate one example technology area where some embodiments described herein can be practiced.

  According to an aspect of an embodiment, a method for adjusting depth in a stereoscopic video may include generating a left eye view frame of the stereoscopic video. The left eye view frame may include a plurality of left eye view frame elements. The method may also include generating a right eye view frame of the stereoscopic video. The right eye view frame may correspond to the left eye view frame and may include a plurality of right eye view frame elements. Further, each right eye view frame element may correspond to one of the left eye view frame elements. The method may further include determining an offset between each right eye view frame element and its corresponding left eye view frame element. Further, the method may include applying a uniform multiplication factor to each offset such that a perceived depth associated with the stereoscopic video is adjusted on a substantially uniform scale.

  According to another aspect of the embodiment, a method for adjusting a focus in a stereoscopic video may include generating a left-eye view frame of the stereoscopic video. The left eye view frame may include a plurality of left eye view frame elements. The method may also include generating a right eye view frame of the stereoscopic video. The right eye view frame may correspond to the left eye view frame and may include a plurality of right eye view frame elements. Further, each right eye view frame element may correspond to one of the left eye view frame elements. The method may further include determining an offset between each right eye view frame element and its corresponding left eye view frame element. Further, the method uniformly shifts each right eye view frame element by substantially the same amount relative to its corresponding left eye view frame element so that the perceived focus associated with the stereoscopic video is adjusted. To apply a uniform summing factor to each offset.

  The objects and advantages of the embodiments will be realized and attained by means of the elements, features and combinations particularly pointed out in the appended claims.

  It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed. It should be understood that there is no.

Exemplary embodiments will be described and explained with additional specificity and detail through the accompanying drawings.
FIG. 1 illustrates an example system for generating stereoscopic (3-D) video based on planar (2-D) video. FIG. 2 shows an example block diagram illustrating the generation of a modified frame associated with a planar video based on the movement of elements between different frames of the planar video. FIG. 3 shows an example block diagram illustrating the generation of modified frames associated with a planar video, where one or more modified frames may be generated based on the one or more modified frames of the planar video. . FIG. 4 shows an example block diagram illustrating the generation of a left-eye view frame and a right-eye view frame of a stereoscopic video based on a right pan effect associated with a stereoscopic video. FIG. 5 shows an example block diagram illustrating the generation of a left-eye view frame and a right-eye view frame of a stereoscopic video based on a left pan effect associated with a planar video. FIG. 6 shows an example block diagram illustrating the generation of a left-eye view frame and a right-eye view frame of a stereoscopic video based on a zoom-out camera effect associated with a stereoscopic video. FIG. 7 shows an example block diagram illustrating the generation of a left-eye view frame and a right-eye view frame of a stereoscopic video based on a zoom-in camera effect associated with a planar video. FIG. 8 shows an example frame of planar video, which includes the fastest moving element, the slow moving element, and the dominant element. FIG. 9 illustrates an example method for determining the foreground and / or background of a scene based on at least one of the fastest, slowest, and dominant elements associated with the scene. It is a flowchart. FIG. 10 illustrates an example block diagram for generating a left-eye view frame and a right-eye view frame of a stereoscopic video from the planar video based on foreground and / or background motion associated with the stereoscopic video. . FIG. 11 is a flowchart of an example method for converting a planar video to a stereoscopic video based on a camera effect. FIG. 12A shows an example scene that may be perceived by the left and right eyes. FIG. 12B illustrates an example grid showing example offsets that may be found between elements in the left and right eye view frames of the stereoscopic video associated with the scene of FIG. 12A. FIG. 12C illustrates an example grid that shows the offset of the elements of FIG. 12B for each of their left eye and right eye view frames after applying a uniform multiplication factor to the offset of FIG. 12B. FIG. 12D illustrates an example grid that shows the offsets of the elements of FIG. 12B for each of their left-eye and right-eye view frames after applying a uniform addition factor to the offset of FIG. 12B. FIG. 13 is a flowchart of an exemplary method for adjusting the depth of a stereoscopic video. FIG. 14 is a flowchart of an example method for adjusting the focus of a stereoscopic video, all arranged according to at least some embodiments described herein.

  Humans have a binocular vision system that uses two eyes that are approximately 2.5 inches apart (approximately 6.5 centimeters). Each eye sees the world from a slightly different perspective. The brain uses these viewpoint differences to calculate or measure distance. This binocular vision system is partially responsible for the ability to determine the distance of an object with relatively good accuracy. The relative distance of multiple objects in the field of view can also be determined with the help of binocular vision.

  Three-dimensional (stereoscopic) imaging presents two images to the viewer, one image presented to one eye (eg left eye) and the other image presented to the other eye (eg right eye) By using the depth perceived by binocular vision. The images presented to the two eyes may contain substantially the same elements, but the elements in the two images mimic the offset viewpoint that can be perceived by the observer's eyes in daily life. Can be offset from each other. Therefore, the observer can perceive depth in the element drawn in the image.

  Traditionally, 3D video has used two video sources (eg, cameras) mounted side by side about 3-8 inches apart to obtain settings that can be perceived by two different eyes. Has been created. This distance is often referred to as “interaxial” or “interocular distance”. Thus, two video sources create two videos, one for the left eye and one for the right eye. The two videos are combined as a stereoscopic (or 3D) video where the right eye video is presented to the viewer's right eye, the left eye video is presented to the viewer's left eye, and the viewer perceives the video in three dimensions. obtain. In contrast, according to some embodiments of the present disclosure, a stereoscopic video may be derived from video obtained using a single (planar) video source. These videos derived from a planar video source may be referred to as “2D” or “planar video”.

  The term “video” may refer to any motion type picture and includes, but is not limited to, movies, television shows, recorded events (eg, sporting events, concerts, etc.), home movies, internet videos, etc. obtain. A video is made up of a series of “frames” (referred to as “frames” or “video frames”) that display images of a scene, each of which may contain elements or objects (commonly referred to as “elements”). The element may be moving or may be stationary. For example, the frame may be a landscape image that may include elements such as mountains, hills, trees, animals, buildings, airplanes, trains, cars, and the like.

  According to some embodiments, the movement of an element between a first frame of a planar video and a second frame of the planar video (eg, the next frame or the previous frame of the first frame) is the first It can correspond to a frame and can be used to generate one or more modified first frames that can be used to generate a stereoscopic video. In some embodiments, the first and second frames may be consecutive frames, and in other embodiments, one or more intermediate frames may be included between the first and second frames. The first frame and the modified first frame have one or more elements offset from each other in the first frame and the modified first frame based on the movement of the elements between the first frame and the second frame; It can include substantially the same elements. In addition, the movement of the element can be analyzed with respect to the first and second frames for camera effects (eg, zoom effects, panning effects, rotating effects, and / or stationary effects). Can be used. In some embodiments, the modified first frame may be generated based on the analyzed camera effect.

  In some embodiments, the left eye view frame may be configured to be viewed by the viewer's left eye, but the determined camera effect and at least one of the first frame and the modified first frame. Can be generated based on one. Similarly, the right eye view frame may be configured to be viewed by the viewer's right eye, but based on the determined camera effect and at least one of the first frame and the modified first frame. Can be generated. This process may be repeated for multiple frames of the stereoscopic video to generate corresponding left and right eye view frames of the stereoscopic video. Thus, in such an embodiment, a stereoscopic video can be generated from the planar video based on the analyzed camera effect and the motion between the first frame and the associated second frame.

  Further, in some embodiments, the generation of the left eye view frame and the right eye view frame is to determine movement of at least one of the foreground and background of the planar video scene associated with the first frame and the second frame. Can be based. In these or other embodiments, the foreground and / or background is based on the fastest, slowest, and / or dominant factors included in the scene associated with the first and second frames. Can be requested.

  Further, the amount of depth in the stereoscopic video can be modified by adjusting the offset between the elements contained in the left eye view frame and the right eye view frame. Moreover, the perception of which elements are included in the foreground or background of a scene associated with a stereoscopic video can also be modified by adjusting the offset between the elements.

  The right eye view frame and left eye view frame described herein may also be referred to as “right eye image” and “left eye image”, respectively. Further, the right eye view frame and left eye view frame can be up / down, left / right, SENSIO® Hi-Fi 3D, Blu-ray® 3D, or any other applicable 3D format. Any suitable 3D format can be used to generate the stereoscopic video.

  FIG. 1 illustrates an example system 100 according to some embodiments of the present disclosure that generates a stereoscopic (3-D) video 103 based on a planar view (2-D) video. The system 100 may include a three-dimensional (stereoscopic) video generation module 104 (hereinafter referred to as “stereoscopic video module 104”). The stereoscopic video module 104 may include any suitable system, apparatus, or device that receives the stereoscopic video 101 and converts the stereoscopic video 101 to the stereoscopic video 103. For example, in some embodiments, the stereoscopic video module 104 may be software that includes computer-executable instructions that cause the processor to perform an operation to convert the stereoscopic video 101 to the stereoscopic video 103.

  Stereoscopic video module 104 may be configured to generate stereoscopic video 103 based on the movement of one or more elements between frames of planar video 101. In some embodiments, the stereoscopic video module 104 may generate a modified frame based on the motion. The generation of modified frames that may be performed by the stereoscopic video module 104 is described in further detail below with respect to FIGS. In some embodiments, the stereoscopic video module 104 may also be configured to analyze or determine a camera effect associated with the planar video 101 based on the movement of elements between frames. In these embodiments, the stereoscopic video module 104 may be configured to generate a modified frame and left and right eye view frames for the stereoscopic video 103 based on the determined camera effect and associated motion. . Further, in some embodiments, the stereoscopic video module 104 generates a modified frame and left and right eye view frames based on determining motion associated with the foreground and background in the scene of the stereoscopic video 101. Can be configured. The analysis of camera effects and the generation of modified frames and left and right eye view frames that can be performed by the stereoscopic video module 104 based on the analyzed camera effects are described in further detail with respect to FIGS. 4-7 and 10.

  In some embodiments, the stereoscopic video module 104 can also be configured to determine the foreground and / or background of the scene of the planar video 101. In some of these embodiments, the stereoscopic video module 104 is based on the fastest, slowest, and / or dominant factors in the scene of the stereoscopic video 101 based on the foreground and / or Or it may be configured to determine the background. The determination of the fastest, slowest and / or dominant elements is described in more detail with respect to FIG. Determining the foreground and / or background that can be performed by the stereoscopic video module 104 is described in further detail below with respect to FIG.

  Further, in some embodiments, the stereoscopic video module 104 adjusts the offset between the elements included in the left and right eye view frames to adjust the depth perceived by the viewer watching the stereoscopic video 103. It can be configured to adjust the amount. The modification of depth perception that may be performed by the stereoscopic video module 104 is described in further detail below with respect to FIGS. 12A-12C and 13. Further, in some embodiments, the stereoscopic video 103 is focused so that the stereoscopic video module 104 can modify the perception of what elements may be in the foreground or background of the scene perceived by the viewer. The stereoscopic video module 104 can also be configured to adjust. The focus modifications that may be performed by the stereoscopic video module 104 are described in further detail below with respect to FIGS. 12A, 12B, 12D, and 14.

  As described above, the stereoscopic video module 104 generates modified frames that can be used to generate left and right eye view frames of the stereoscopic video 103 based on the movement of one or more elements between the frames. Can be configured. FIG. 2 shows an example block diagram 200 illustrating the generation of a modified frame 202 'based on the movement of elements between different frames 202 of the planar video 201, according to some embodiments of the present disclosure. In some embodiments, and as illustrated by way of example with respect to FIG. 2, the modified frame 202 ′ may be generated by a stereoscopic video module, such as the stereoscopic video module 104 described above with respect to FIG. . As illustrated in FIG. 2, the planar video 201 may be substantially similar to the planar video 101 described with respect to FIG. 1 and may include images of one or more scenes associated with the planar video 201. A series of frames 202 may be included. According to some embodiments of the present disclosure, the stereoscopic video module may generate a modified frame 202 ′ based on the horizontal movement of one or more elements (not explicitly shown) between the frames 202. .

  For example, an element in frames 202a and 202b (frame 202b may be a frame that follows frame 202a) such that the element can move horizontally from left to right between frames 202a and 202b. And the frame 202b may have different positions. Accordingly, the stereoscopic video module may modify the modified frame 202a ′ based on the horizontal movement of the element between the frames 202a and 202b such that the element is offset to the right in the modified frame 202a ′ relative to the frame 202a. Can be generated. Similarly, the stereoscopic video module may generate a modified frame 202b ′ based on the frames 202b and 202c (the frame 202c may be a frame following the frame 202b), and a modified frame 202c ′ based on the frames 202c and 202d (the frame 202d can be a frame following frame 202c). Thus, to mimic the horizontal offset of the image as perceived by the viewer's right and left eyes, based on the horizontal movement of the elements between frames, the modified frame 202 ′ It can be offset horizontally relative to the corresponding element.

  In some embodiments, the movement of the elements between the frames 202 can also be in the vertical direction (referred to as “vertical movement”). However, since the viewer's eyes are generally not substantially offset in the vertical direction, the stereoscopic video module substantially reduces the vertical movement of the elements between the frames 202 when generating the modified frame 202 ′. Can be ignored. Thus, a modified frame 202 'can be generated based on horizontal movement such that those elements are offset horizontally in the modified frame 202' with respect to corresponding elements in the associated frame 202. However, the elements of the modified frame 202 'cannot be substantially offset in the vertical direction relative to the corresponding elements of the associated frame 202, even if there is vertical movement between the frames.

  Further, in some embodiments, the movement of an element from one frame to another frame causes a subsequent frame in which one or more elements in one frame can be used to generate a modified frame. Can result in non-existence. In such cases, the stereoscopic video module 104 may copy the missing element from the original frame and place it in the associated modified frame.

  For example, the element of the frame 202a can be a human hand, and the person's movement between the frame 202a and the frame 202b can be such that the person's hand is not visible on the frame 202b. Further, a modified frame 202a 'can be generated based on the person's horizontal movement from frame 202a to frame 202b. Thus, when generating the modified frame 202a ′ based on the person's horizontal movement, the person is in the modified frame 202a ′ based on the person's horizontal movement between the frames 202a and 202b. It can be offset horizontally. However, the person's hand can be copied from the frame 202a and inserted into the modified frame 202a '. As a result, even if the hand disappears from the frame 202b from which the horizontal offset of the element of the correction frame 202a 'is extracted, the hand can be present in the correction frame 202a'.

  Modifications, additions, or omissions may be made to FIG. 2 without departing from the scope of the present disclosure. For example, the planar video 201 may include more or fewer frames than are explicitly illustrated. Further, in some embodiments, one or more intermediate frames may be included between the illustrated frames. For example, the planar video 201 may include one or more intermediate frames that may be between frames 202a and 202b, between frames 202b and 202c, and / or between frames 202c and 202d. Further, in some embodiments, the scene in the planar video 201 may vary such that one frame 202 may be associated with one scene and subsequent frames may be associated with different scenes. In this disclosure, “scene” refers to a change in the camera angle or perspective of the same scene and / or a change in the scene depicted in the video. It may refer to a specific camera field of view or angle for a specific scene. In such embodiments, the modified frame may be generated based on a speculative frame that may be based on the movement of one or more elements identified in the previous frame.

  FIG. 3 illustrates an exemplary block diagram 300 illustrating the generation of a modified frame 302 ′ according to some embodiments of the present disclosure, where one or more modified frames 302 ′ are one or more of the planar view video 301. Can be generated based on the estimated frame 305. In some embodiments, and as provided as an example with respect to FIG. 3, the modified frame 302 'may be generated by a stereoscopic video module, such as the stereoscopic video module 104 described above with respect to FIG. As illustrated in FIG. 3, the planar video 301 includes a series of frames 302 (shown as frames 302a-302d) that may include images of one or more scenes associated with the planar video 301. obtain. In the described embodiment, frames 302 a-302 c may be associated with scene 306 a of planar video 301 and frame 302 d may be associated with scene 306 b that is a different scene of planar video 301.

  According to some embodiments of the present disclosure, the stereoscopic video module may be between frames 302a and 302b and between frames 302b and 302c, similar to the generation of modified frames 202a ′ and 202b ′ described above. Based on the movement of each one or more elements (not specifically shown), the modified frames 302a ′ and 302b ′ may be configured to be generated. For example, the modified frame 302a 'can be generated based on the movement of one or more elements between the frames 302a and 302b (which can follow the frame 302a). Further, a modified frame 302b 'can be generated based on the movement of one or more elements between frames 302b and 302c (which can follow frame 302b).

  However, in the described embodiment, frame 302c may be the last frame associated with scene 306a of planar video 301. Thus, a subsequent frame, such as frame 302d, may be associated with a different scene, or may not have substantial movement associated with it before transitioning to another frame, so that element movement associated with frame 302c. Cannot exist between frame 302c and a frame subsequent to frame 302c. In such a case, the stereoscopic video module may be configured to generate a speculative frame that can represent a subsequent frame of the scene if the scene continues since its last frame. For example, in the described embodiment, the stereoscopic video module is configured to generate a speculative frame 305 that can represent a subsequent frame of the frame 302c of the scene 306a if the scene 306a continues from frame 302c. obtain.

  In some embodiments, the stereoscopic video module may be configured to determine a scene change and may determine a final frame of the scene based on the detected scene change. The stereoscopic video module may also be configured to generate a speculative frame based on detecting a scene change. For example, the stereoscopic video module may be configured to detect a scene change from scene 306a to scene 306b, may determine that frame 302c is the last frame of scene 306a, and is from scene 306a to scene 306b. A guess frame 305 may be generated based on detecting a scene change.

  In some of these embodiments, the stereoscopic video module may be configured to determine a scene change based on an analysis of pixel spacing and / or color distribution between frames. If the distribution of pixels is substantially similar from frame to other frame, the stereoscopic video module may determine that the frame 302 is associated with the same scene 306. However, if the distribution of pixels differs substantially between frames, the stereoscopic video module may require that a scene change has occurred. For example, the stereoscopic video module seeks that the distribution of pixels between frames 302a, 302b, and 302c is substantially similar, and frames 302a, 302b, and 302c are part of scene 306a. You can ask for that. The stereoscopic video module also requires that the distribution of pixels between frames 302c and 302d be substantially different and a scene change from scene 306a to scene 306b occurs between frames 302c and 302d. You can ask that.

  In response to detecting a scene change from the first scene to the second scene, the stereoscopic video module may be configured to detect movement of one or more elements between frames of the first scene. Based on the movement of elements between frames of the first scene, the stereoscopic video module may be configured to generate a speculative frame for the first scene. Upon generating the speculative frame, the stereoscopic video module may be configured to generate a modified frame for the final frame of the first scene based on the movement of elements between the final frame of the first scene and the speculative frame. .

  For example, after detecting a scene change from scene 306a to scene 306b, the stereoscopic video module is configured to detect movement of one or more elements between frames 302b and 302c in response to detecting the scene change. obtain. Based on the detected motion between frames 302b and 302c, the stereoscopic video module can determine what motion between frame 302c and the subsequent frames of scene 306a if scene 306a continues from frame 302c. May be configured to infer whether can occur. In some embodiments, motion estimation between elements assumes that the motion of elements between frames 302b and 302c will continue beyond frame 302c unless scene 306a ends after frame 302c. Based on The stereoscopic video module can therefore match the frames 302b and 302c of the scene 306a so that the speculative frame 305 can reflect the motion that would have continued in the scene 306a if the scene 306a continued from frame 302c. It may be configured to generate a guess frame 305 for the scene 306a based on the movement of elements in between.

  Once the inferred frame 305 is generated, the stereoscopic video module generates modified frames 302a ′ and 302b ′ based on the motion between frames 302a and 302b and between frames 302b and 302c, respectively. Similarly, a modified frame 302c ′ may be generated based on the movement of one or more elements between the frame 302c and the guess frame 305. Thus, the modified frame 302c ′ is based on the motion that can occur between the frame 302 and a subsequent frame having substantially the same elements, even if no such subsequent frame can exist. Can be generated.

  Modifications, additions, or omissions may be made to FIG. 3 without departing from the scope of the present disclosure. For example, the planar video 301 may include more or fewer frames 302 and / or scenes 306 than are explicitly illustrated. Further, each scene 306 can include any number of frames 302. Further, in some embodiments, one or more intermediate frames may be included between the illustrated frames 302.

  Returning to FIG. 1, as described above, the stereoscopic video module 104 may be configured to generate the left eye and right eye view frames of the stereoscopic video 103 based on the camera effect and the modified frame of the stereoscopic video 101. A modified frame may be generated based on the movement of elements between frames, as described above with respect to FIGS. Accordingly, the stereoscopic video module 104 may also be configured to analyze the camera effect of the planar video 101 according to one or more embodiments of the present disclosure. In such an embodiment, the stereoscopic video module 104 may determine whether the camera effect is a pan effect or a zoom effect, and based on whether the camera effect is a pan effect or a zoom effect, A right eye view frame may be generated. The generation of a left eye view frame and a right eye view frame based on determining that the camera effect is a pan effect is described in further detail below with respect to FIGS. The generation of left and right eye view frames based on determining that the camera effect is a zoom effect is discussed in more detail below with respect to FIGS.

  Further, if the stereoscopic video module 104 determines that the camera effect is neither a panning effect nor a zooming effect, the stereoscopic video module 104 generates left-eye and right-eye view frames based on this determination. Can do. In some of these embodiments, the stereoscopic video module 104 may generate a left eye view frame and a right eye view frame based on foreground and background motion associated with the frame of the stereoscopic video 101. Foreground and background detection is described in more detail with respect to FIGS. The generation of a left eye view frame and a right eye view frame based on the requirement that the camera effect is neither a pan effect nor a zoom effect is discussed in further detail below with respect to FIG.

  FIG. 4 illustrates the generation of a left-eye view frame 410 and a right-eye view frame 412 of a stereoscopic video 403 based on a panning right effect associated with the frames 402a and 402b of the stereoscopic video 401 according to some embodiments of the present disclosure. FIG. 4 shows an exemplary block diagram 400 illustrating the. In the example described, the stereoscopic video 403 may be generated by a stereoscopic video module, such as the stereoscopic video module 104 described above with respect to FIG.

  Frames 402a and 402b may include one or more elements 408 that may be included in both frame 402a and frame 402b, respectively. For example, in the described embodiment, each frame 402a and 402b may include elements 408a and 408b. Frames 402a and 402b may also include other elements not explicitly shown in FIG.

  When the camera pans, the entire scene filmed by the camera moves as the camera moves so that the elements in the frame associated with the pan effect move in a substantially uniform direction and in a substantially uniform direction. . For example, frames 402a and 402b are frames 402a generated by the camera such that elements 408a and 408b in frames 402a and 402b move uniformly from right to left between frames 402a and 402b by substantially the same amount. And 402b can be related to a right-panning effect, such as can be panned from left to right.

  The right pan effect may be generated based on actually panning the camera from left to right or based on a simulation of panning the camera from left to right. In some embodiments, the pan effect can be related to the rotation of the camera about an axis or to any suitable simulation that produces the pan effect. In other embodiments, the pan effect may be applied to the overall camera movement in the horizontal direction with respect to the scene being shot (e.g., from left to right) or to any suitable simulation that produces the pan effect. May be related.

  The stereoscopic video module is based on analyzing the movement of elements such as elements 408a and 408b between frames 402a and 402b, and whether a panning effect to the right exists between frames 402a and 402b. Can be configured to ask for For example, in some embodiments, the stereoscopic video module may analyze the movement of various elements located in various areas of the frames 402a and 402b (eg, upper right, upper left, lower right, and lower left areas). In some embodiments, the stereoscopic video module may analyze the motion of the element by analyzing the motion associated with the pixels of frames 402a and 402b that may correspond to the element. In these or other embodiments, the stereoscopic video module may analyze the motion associated with each pixel in frames 402a and 402b to determine the motion of the elements between frames 402a and 402b. If the motion associated with elements (or pixels in some embodiments) located in various areas of frames 402a and 402b all move from right to left to substantially the same extent, the stereoscopic video module It can be sought that the camera effect is a panning effect to the right.

  The stereoscopic video module may generate a modified frame 402a 'associated with the frame 402a based on the determined right panning effect and associated motion. For example, as described above, a panning effect to the right can result in elements 408a and 408b moving substantially the same amount from frame 402a to frame 402b, from right to left. Thus, in some embodiments, the stereoscopic video module causes frame 402b to be displayed due to the horizontal position difference of elements 408a and 408b between frames 402a and 402b caused by a panning effect to the right. Can be used to generate a modified frame 402a ′. In some embodiments, where there may be little or no vertical offset of elements 408a and 408b between frames 402a and 402b, frame 402b may be used as a modified frame 402a '. In other embodiments, where there may be a vertical offset in elements 408a and 408b between frames 402a and 402b, the difference in the vertical position of elements 408a and 408b between frames 402a and 402b is The vertical offset of elements 408a and 408b between frame 402a and modified frame 402a ′ can be removed during the generation of modified frame 402a ′ so that it cannot be essential.

  The left-eye view frame 410 and the right-eye view frame 412 of the stereoscopic video 403 may be related to the view frame 402a of the planar video 401, but based on the panning effect to the right, the view frame 402a, and the modified view frame 402a ′. Can be generated. For example, due to a panning effect to the right, frame 402a may be designated as left eye view frame 410 and modified frame 402a 'may be designated as right eye view frame 412. Thus, the left eye view frame 410 and the right eye view frame 412 of the stereoscopic video 403 may be generated based at least in part on the right pan effect and the modified frame 402a '. The modified frame 402a 'may relate to the movement of the elements 408a and 408b between the frames 402a and 402b.

  Modifications, additions, or omissions other than those explicitly described may be made to the generation of the stereoscopic video 403. For example, left eye and right eye view frames associated with other frames 402 of planar video 401 (eg, left eye and right eye view frames associated with frame 402b) may be similarly generated. Further, in some embodiments, the left eye and right eye view frames are frames at the end of each scene, or frames 402 that may be near the end of the frame, whereas a modified frame associated with frame 402 has been described above with respect to FIG. It can be generated in such a way that it can be generated based on a guess frame determined in such a way.

  FIG. 5 illustrates a left-eye view frame 510 and right eye of a stereoscopic video 503 based on a left-panning effect associated with the frames 502a and 502b of the planar video 501 according to some embodiments of the present disclosure. An example block diagram 500 illustrating the generation of a view frame 512 is shown. In the illustrated example, the stereoscopic video 503 can be generated by a stereoscopic video module, such as the stereoscopic video module 104 described above with respect to FIG.

  Frames 502a and 502b can include one or more elements 508, each of which can be included in both frame 502a and frame 502b. For example, in the illustrated embodiment, each of the frames 502a and 502b can include elements 508a and 508b.

  As described above, frames 502a and 502b move frames 502a and 502b so that elements 508 in frames 502a and 502b can move substantially the same amount from left to right between frames 502a and 502b. This can be related to the left panning effect, where the producing camera can be panned from right to left. For example, in the illustrated embodiment, elements 508a and 508b move from left to right between frames 502a and 502b by substantially the same amount based on the left pan effect associated with frames 502a and 502b. obtain.

  Similar to the right pan effect, the left pan effect is generated based on actually panning the camera from right to left or based on a simulation of panning the camera from right to left. obtain. Also, in some embodiments, the pan effect can be related to the rotation of the camera about the axis or to any suitable simulation that produces the pan effect. In other embodiments, the pan effect can be applied to the overall camera movement in the horizontal direction with respect to the scene being shot (eg, from right to left) or to any suitable simulation that produces the pan effect. May be related.

  As discussed above with respect to FIG. 4, similar to determining if there is a panning right effect, the stereoscopic video module is between elements 502a and 502b as elements 508a and 508b. Configured to determine whether a left panning effect exists between frames 502a and 502b, based on analyzing motion associated with a particular element (or pixel in some embodiments). obtain. If the motion associated with elements (or pixels in some embodiments) located in different areas of frames 502a and 502b all move from left to right to substantially the same extent, the stereoscopic video module It can be sought that the camera effect is a left panning effect.

  The stereoscopic video module may generate a modified frame 502a 'associated with frame 502a based on the determined left pan effect and associated motion. For example, as described above, a panning effect to the left can result in elements 508a and 508b moving substantially the same amount from frame 502a to frame 502b, from left to right. Thus, in some embodiments, the stereoscopic video module may cause the frame 502b to move due to a difference in the horizontal position of the elements 508a and 508b between the frames 502a and 502b caused by the left pan effect. Based on this, a modified frame 502a ′ may be generated. In some embodiments, where there may be little or no vertical offset of elements 508a and 508b between frames 502a and 502b, frame 502b may be used as a modified frame 502a '. In other embodiments, where there may be a vertical offset in elements 508a and 508b between frames 502a and 502b, the difference in the vertical position of elements 508a and 508b between frames 502a and 502b is , Any vertical offset of elements 508a and 508b between frame 502a and modified frame 502a ′ can be removed during generation of modified frame 502a ′.

  The left-eye view frame 510 and the right-eye view frame 512 of the stereoscopic video 503 may be related to the view frame 502a of the stereoscopic video 501 but are based on the panning effect to the left, the view frame 502a, and the modified view frame 502a ′. Can be generated. For example, due to a panning effect to the left, the modified frame 502a 'can be designated as the left eye view frame 510 and the frame 502a can be designated as the right eye view frame 512. Thus, the left eye view frame 510 and the right eye view frame 512 of the stereoscopic video 503 may be generated based at least in part on the left pan effect and the modified frame 502a '. The modified frame 502a 'may relate to the movement of the elements 508a and 508b between the frames 502a and 502b.

  Modifications, additions, or omissions other than those explicitly described may be made to the generation of the stereoscopic video 503. For example, left eye and right eye view frames associated with other frames 502 of planar video 501 (eg, left eye and right eye view frames associated with frame 502b) may be similarly generated. Further, in some embodiments, the left-eye and right-eye view frames are frames that may be the last or close frames of their respective scenes, whereas a modified frame associated with frame 502 has been described above with respect to FIG. It can be generated in such a way that it can be generated based on a guess frame determined in such a way.

  FIG. 6 illustrates the generation of a left-eye view frame 610 and a right-eye view frame 612 of a stereoscopic video 603 based on the zoom-out camera effect associated with the frames 602a and 602b of the stereoscopic video 601 according to some embodiments of the present disclosure. An exemplary block diagram 600 is shown, illustrating FIG. In the illustrated embodiment, the stereoscopic video 603 may be generated by a stereoscopic video module, such as the stereoscopic video module 104 described above with respect to FIG.

  Frames 602a and 602b can include one or more elements 608, each of which can be included in both frame 602a and frame 602b. For example, in the illustrated embodiment, each of the frames 602a and 602b can include elements 608a, 608b, 608c, and 608d.

  When the camera zooms out, the elements in the frame generated by the camera can move from one frame to the other toward the center of the scene captured by the frame. For example, as shown in FIG. 6, frames 602a and 602b are the scenes in which elements 608a, 608b, 608c and 608d in frames 602a and 602b are captured by frames 602a and 602b from frame 602a to frame 602b. It can be related to the zoom-out camera effect as it moves toward the center of the camera. The zoom-out camera effect can be generated based on actual camera zoom-out or simulation of zoom-out.

  As discussed above with respect to FIGS. 4 and 5, as with determining whether there is a right panning effect or a left panning effect, the stereoscopic video module performs various areas of frames 602a and 602b. Elements 608a, 608b, 608c that may be located in the upper left, upper right, lower left, and lower right areas of the elements located in (eg, upper left, upper right, lower left, and lower right areas), for example, frames 602a and 602b, respectively. And 608d may be configured to determine whether a zoom-out camera effect exists between frames 602a and 602b based on analyzing the movement between elements 602a and 602b. In some embodiments, the stereoscopic video module may analyze the motion of the element by analyzing the motion associated with the pixels in frames 602a and 602b that may correspond to the element. In these or other embodiments, the stereoscopic video module may analyze the motion associated with each pixel in frames 602a and 602b to determine the motion of the elements between frames 602a and 602b. If the motion associated with elements (or pixels in some embodiments) located in various areas of frames 602a and 602b moves substantially towards the center of the scene captured by frames 602a and 602b The stereoscopic video module may require that the camera effect is a zoom-out camera effect.

  The stereoscopic video module may generate a modified frame 602a ′ and a modified frame 602a ″ associated with the frame 602a based on the determined zoom-out camera effect and associated motion. This can result in the elements (eg, elements 608a and 608c) included on each left side 614a and 614b of 602a and 602b moving substantially the same amount from frame 602a to frame 602b and from left to right. The camera effect can result in the elements (eg, elements 608b and 608d) included on the right side 616a and 616b of each of the frames 602a and 602b moving substantially the same amount from frame 602a to frame 602b and from right to left. .

  In some embodiments, the stereoscopic video module may generate the left side 614c of the modified frame 602a ′ based on the horizontal offset of the elements between the left side 614a and 614b of each of the frames 602a and 602b. . In addition, the stereoscopic video module may ignore any vertical offset of elements between the left side 614a and 614b during the generation of the left side 614c. For example, elements 608a and 608c may have substantially the same horizontal position on left side 614c of modified frame 602a 'as on left side 614b of frame 602b. However, elements 608a and 608c may have substantially the same vertical position on the left side 614c of the modified frame 602a 'as on the left side 614a of the frame 602a.

  Further, the stereoscopic video module may modify the modified frame 602a ′ based on both the horizontal and vertical positions of the elements included in the right side 616a of the frame 602a so that the right side 616c may be substantially similar to the right side 616a. Of the right side 616c. For example, elements 608b and 608d may have substantially the same horizontal and vertical positions on the right side 616c of the modified frame 602a 'as on the right side 616a of the frame 602a.

  The stereoscopic video module may generate the right side 616d of the modified frame 602a "based on the horizontal offset of the elements between the right side 616a and 616b of each of the frames 602a and 602b. Further, the stereoscopic video module. Can ignore any vertical offset of the elements between the right side 616a and 616b during the generation of the right side 616d. For example, the elements 608b and 608d can have the frame 602b May have substantially the same horizontal position as on the right side 616b. However, elements 608b and 608d may have substantially the same vertical position on the right side 616d of the modified frame 602a ″ as on the right side 616a of the frame 602a.

  In contrast, the stereoscopic video module may modify the modified frame based on both horizontal and vertical positions of elements included in the left side 614a of the frame 602a so that the left side 614d may be substantially similar to the left side 614a. The left side 614d of 602a "can be generated. For example, elements 608a and 608c have horizontal and vertical positions on the left side 614d of modified frame 602a" that are substantially the same as on left side 614a of frame 602a. obtain.

  As shown in FIG. 6, the configuration of modified frames 602a ′ and 602a ″ for the zoom out effect described above is such that elements 608a, 608b, 608c and 608d are relative to modified frame 602a ″ in modified frame 602a ′. Result in a horizontal offset to the right. Further, elements 608a, 608b, 608c and 608d may have little or no offset in the correction frame 602a 'perpendicular to the correction frame 602a ".

  The left-eye view frame 610 and the right-eye view frame 612 of the stereoscopic video 603 may be related to the view frame 602a of the stereoscopic video 601 but are generated based on the zoom-out camera effect and the modified view frames 602a ′ and 602a ″. For example, for zoom-out camera effects, the modified frame 602a ′ may be designated as the left eye view frame 610 and the modified frame 602a ″ may be designated as the right eye view frame 612. Thus, the left eye view frame 610 and the right eye view frame 612 of the stereoscopic video 603 may be generated based at least in part on the zoom-out camera effect and the modified frames 602a ′ and 602a ″. The modified frames 602a ′ and 602a ″ are It may be related to the movement of elements 608a, 608b, 608c and 608d between frames 602a and 602b.

  Modifications, additions, or omissions other than those explicitly described may be made for the generation of the stereoscopic video 603. For example, left eye and right eye view frames associated with other frames 602 of planar video 601 (eg, left eye and right eye view frames associated with frame 602b) may be generated as well. Further, in some embodiments, the left-eye and right-eye view frames are frames 602 that may be the last frame or the last frame of each scene, while the modified frame associated with frame 602 is related to frame 602 and FIG. It can be generated such that it can be generated based on an associated guess frame generated in the manner as described above.

  FIG. 7 is an example block illustrating the generation of a left eye view frame 710 and a right eye view frame 712 for a zoom-in camera effect associated with frames 702a and 702b of a planar video 701, according to some embodiments of the present disclosure. A diagram 700 is shown. Frames 702a and 702b can include one or more elements 708, each of which can be included in both frame 702a and frame 702b. For example, in the illustrated embodiment, each of the frames 702a and 702b can include elements 708a, 708b, 708c, and 708d.

  When the camera zooms in, the elements in the frame generated by the camera can move from one frame to another and away from the center of the scene captured by the frame. For example, as shown in FIG. 7, frames 702a and 702b are the scenes where elements 708a, 708b, 708c and 708d in frames 702a and 702b are captured by frames 702a and 702b from frame 702a to frame 702b. It can be related to the zoom in camera effect to move away from the center of the camera. The zoom-in camera effect can be generated based on actual camera zoom-in or a zoom-in simulation.

  As discussed above with respect to FIGS. 4-6, similar to determining whether there is a right panning effect, a left panning effect, or a zoom-out camera effect, the stereoscopic video module performs the element 708a. , 708b, 708c and 708d, for example, to analyze the movement between the frames 702a and 702b of elements located in various areas of the frames 702a and 702b (eg, upper right, upper left, lower right and lower left areas). Based on that, it may be configured to determine if a zoom-in camera effect exists between frames 702a and 702b. In some embodiments, the stereoscopic video module may analyze the motion of the element by analyzing the motion associated with the pixels in frames 702a and 702b that may correspond to the element. In these or other embodiments, the stereoscopic video module may analyze the motion associated with each pixel in frames 702a and 702b to determine the motion of the elements between frames 702a and 702b. If the motion associated with elements (or pixels in some embodiments) located in various areas of frames 702a and 702b moves substantially away from the center of the scene captured by frames 702a and 702b If so, the stereoscopic video module may require that the camera effect is a zoom-in camera effect.

  The stereoscopic video module may generate a modified frame 702a ′ and a modified frame 702a ″ associated with the frame 702a based on the determined zoom-in camera effect and associated motion. For example, the zoom-in camera effect may include the frames 702a and 702a. The elements contained in each left side 714a and 714b of 702b (eg, elements 708a and 708c) may result in substantially the same amount of movement from frame 702a to frame 702b, from right to left. , Elements included in the respective right side 716a and 716b of frames 702a and 702b (eg, elements 708b and 708d) may result in substantially the same movement from frame 702a to frame 702b and from left to right.

  In some embodiments, the stereoscopic video module may generate the left side 714c of the modified frame 702a ′ based on the horizontal offset of the elements between the left side 714a and 714b of each of the frames 702a and 702b. . In addition, the stereoscopic video module may ignore any vertical offset of elements between the left side 714a and 714b during the generation of the left side 714c. For example, elements 708a and 708c may have substantially the same horizontal position on left side 714c of modified frame 702a 'as on left side 714b of frame 702b. However, the elements 708a and 708c may have substantially the same vertical position on the left side 714c of the modified frame 702a 'as on the left side 714a of the frame 702a.

  Further, the stereoscopic video module may modify the modified frame 702a ′ based on both the horizontal and vertical positions of the elements included in the right side 716a of the frame 702a such that the right side 716c may be substantially similar to the right side 716a. Of the right side 716c. For example, the elements 708b and 708d may have substantially the same horizontal and vertical positions on the right side 716c of the correction frame 702a 'as on the right side 716a of the frame 702a.

  The stereoscopic video module may generate the right side 716d of the modified frame 702a "based on the horizontal offset of the elements between the right side 716a and 716b of each of the frames 702a and 702b. Further, the stereoscopic video module. Can ignore any vertical offset of the elements between the right side 716a and 716b during the generation of the right side 716d. For example, the elements 708b and 708d are the frame 702b at the right side 716d of the modified frame 702a ″. May have substantially the same horizontal position as on the right side 716b. However, the elements 708b and 708d may have substantially the same vertical position on the right side 716d of the modified frame 702a "as on the right side 716a of the frame 702a.

  Further, the stereoscopic video module may modify the modified frame 702a "based on both the horizontal and vertical positions of the elements included in the left side 714a of the frame 702a so that the left side 714d may be substantially similar to the left side 714a. For example, elements 708a and 708c may have substantially the same horizontal and vertical positions on left side 714d of modified frame 702a ″ as on left side 714a of frame 702a.

  As shown in FIG. 7, the configuration of modified frames 702a ′ and 702a ″ for the zoom-in effect described above is such that elements 708a, 708b, 708c and 708d are relative to modified frame 702a ″ in modified frame 702a ′. This can result in a horizontal offset to the left. Further, elements 708a, 708b, 708c and 708d may have little or no offset in the correction frame 702a 'perpendicular to the correction frame 702a ".

  The left-eye view frame 710 and the right-eye view frame 712 of the stereoscopic video 703 can be related to the view frame 702a of the stereoscopic video 701, but can be generated based on the zoom-in camera effect and the modified view frames 702a ′ and 702a ″. For example, for the zoom-in camera effect, the modified frame 702a ″ may be designated as the left eye view frame 710 and the modified frame 702a ′ may be designated as the right eye view frame 712. Thus, the left-eye view frame 710 and the right-eye view frame 712 of the stereoscopic video 703 can be generated based at least in part on the zoom-in camera effect and the modified frames 702a ′ and 702a ″. The modified frames 702a ′ and 702a ″ are frames. It may relate to the movement of elements 708a, 708b, 708c and 708d between 702a and 702b.

  Modifications, additions, or omissions other than those explicitly described may be made for the generation of the stereoscopic video 703. For example, left eye and right eye view frames associated with other frames 702 of planar video 701 (eg, left eye and right eye view frames associated with frame 702b) may be similarly generated. Further, in some embodiments, the left-eye and right-eye view frames are frames 702 that can be the last or close frames of the respective scene, while the modified frame associated with frame 702 is related to frame 702 and FIG. It can be generated such that it can be generated based on the corresponding guess frame generated in the manner as described above.

  Returning to FIG. 1, the stereoscopic video module 104 is thus configured to generate left and right eye view frames for the stereoscopic video 103 based on the pan or zoom effect as described above with respect to FIGS. 4-7. obtain. However, in some examples, the camera effect may be different from the pan or zoom effect. For example, the camera effect can be a rotating camera effect or a stationary camera effect. In such a case, the stereoscopic video module 104 may determine that the camera effect is not a pan or zoom effect, and may generate left and right eye view frames accordingly. In some of these embodiments, the stereoscopic video module 104 is based on determining that the camera effect is not a pan or zoom effect and foreground and / or associated with a frame of the stereoscopic video 101. A left eye view frame and a right eye view frame may be generated based on the background motion.

  Accordingly, in some embodiments, the stereoscopic video module 104 may include the foreground and background associated with the stereoscopic video 101 such that the stereoscopic video module may analyze motion associated with the foreground and / or background. It may be configured to determine at least one. In some of these embodiments, the stereoscopic video module 104 has the fastest motion associated with the frame of the stereoscopic video 101 to determine the foreground and / or background based on the frame of the stereoscopic video 101. It may be configured to seek elements, slow moving elements, and / or dominant elements.

  FIG. 8 shows exemplary frames 802a and 802b of a planar video 801 according to some embodiments of the present disclosure, where the frames 802a and 802b are the fastest moving element 820, the slow moving element 822, and the dominant. A typical element 824. The planar video 801 may be substantially similar to the planar video 101 of FIG. In the illustrated embodiment, frames 802a and 802b may be associated with the same scene of planar video 801, and frame 802b may be a successor frame of frame 802a. A stereoscopic video module, such as the stereoscopic video module 104 of FIG. 1, may be configured to identify the fastest moving element 820, the slow moving element 822, and / or the dominant element 824.

  The stereoscopic video module may analyze frames 802a and 802b to determine the fastest moving element 820, slow moving element 822, and / or dominant element 824. For example, the stereoscopic video module may analyze the displacement of elements in frame 802b relative to frame 802a to identify the fastest moving element 820 and slow moving element 822. A fast moving element can move from frame 802a to frame 802b a longer distance than a slow moving element. Thus, the stereoscopic video module can analyze the displacement of the element from frame 802a to frame 802b, and the element with the greatest amount of displacement can be determined to be the element with the fastest movement.

  In the illustrated embodiment, the fastest moving element 820 is from the frame 802a so that the stereoscopic video module can identify the fastest moving element 820 as the fastest moving element from frame 802a to frame 802b. There may be a maximum displacement into the frame 802b. Furthermore, as shown in FIG. 8, the slow moving element 822 can also move between the frames 802a and 802b, but to a lesser extent than the fastest moving element 820. Accordingly, the stereoscopic video module may identify the slow moving element 822 as the slower moving element than the fastest moving element 820.

  Further, the stereoscopic video module may be configured to determine that the dominant element 824 is the dominant element in frames 802a and 802b. In some embodiments, the stereoscopic video module may determine whether the elements in frames 802a and 802b are based on an analysis of the pixels associated with frames 802a and 802b so that the number of pixels associated with the particular element may be determined. Can be specified. Accordingly, in some embodiments, the stereoscopic video module may control the dominant element 824 based on the fact that the dominant element 824 is associated with substantially more pixels than the other elements included in the frames 802a and 802b. Specific elements 824 may be configured. Thus, by analyzing frames 802a and 802b, the stereoscopic video module may be able to identify the fastest moving element 820, the slow moving element 822, and the dominant element 824. As described in detail below, identifying these elements can help in determining the foreground and / or background associated with frames 802a and 802b.

  The illustration in FIG. 8 is merely to introduce the concept behind the fastest, slowest and dominant elements. Accordingly, modifications, additions, or omissions may be made to FIG. 8 without departing from the scope of the present disclosure. In some embodiments, the amount of displacement between the fastest moving element 820 and the slow moving element 822 frames 802a and 802b is the difference in speed between the fastest moving element 820 and the slow moving element 822. Can vary based on. For example, in some examples, the fastest-moving element 820 and the fastest-moving element 820 and the slow-moving element 822 have the fastest-moving element 820 and the moving so that the amount of displacement between the frames 802a and 802b is substantially the same. Slow element 822 can move at substantially the same speed. Further, in some embodiments, dominant element 824 can move between frames 802a and 802b, while in other embodiments, dominant element 824 cannot move. Further, in some examples, frames 802a and 802b may not include dominant elements. Also, the direction of movement of the fastest moving element 820, slow moving element 822, and / or dominant element 824 may be different.

  As described above, the foreground and / or background of a scene may be determined based on the fastest, slowest, and / or dominant factors associated with the scene. FIG. 9 illustrates the foreground of a scene based on at least one of the fastest, slowest, and dominant elements associated with the scene, according to some embodiments of the present disclosure. 10 is a flowchart of an example method 900 for determining a background. The method 900 may be implemented in some embodiments by a stereoscopic video module, such as the stereoscopic video module 104 of FIG. For example, the stereoscopic video module 104 may be configured to execute computer instructions to perform operations for determining the foreground and / or background, as represented by one or more blocks of the method 900. . Although illustrated as separate blocks, depending on the desired implementation, various blocks may be separated into further blocks, combined into a smaller number of blocks, or deleted.

  The method 900 may begin at block 902 where element and / or pixel motion between the first frame and the second frame may be determined. At block 904, it can be determined whether all elements and / or pixels are moving in substantially the same orientation. If all elements and / or pixels are moving in substantially the same direction, the method 900 may proceed to block 906. If all elements and / or pixels are not moving in substantially the same direction, the method 900 may proceed to block 914.

  At block 906, the fastest and slowest motion elements associated with the first and second frames may be determined. As described above with respect to the fastest moving element 820 and slow moving element 822 between frames 802a and 802b of FIG. 8, in some embodiments, the fastest moving element and the slow moving element are It can be determined based on the displacement of the element in the second frame relative to one frame.

  At block 908, it may be determined whether there is a large speed difference between the fastest moving element and the slow moving element. If there is a large speed difference, the method 900 may proceed to block 910. If there is no significant speed difference, the method 900 may proceed to block 912. In some embodiments, the speed difference may be determined by measuring the offset difference between the fastest and slowest moving elements from the first frame to the second frame. Further, in some embodiments, the speed difference (eg, the difference in offset) can be compared to a threshold, and if the speed difference is greater than the threshold, method 900 can proceed to block 910 and the speed difference is less than the threshold. If so, the method 900 may proceed to block 912. In some examples, the threshold may vary depending on the type of movie. For example, the threshold may be higher than the drama for action movies.

  At block 910, the speed difference between the fastest and slow moving elements is substantially large enough (eg, greater than a threshold) so that the fastest moving element can be considered in the foreground. The foreground can be associated with the fastest moving element. In block 912, because the speed difference between the fastest and slowest moving elements is not substantially large enough (eg, less than a threshold), the entire scene with the background associated with the first and second frames Can be associated with Accordingly, at block 912, the entire scene associated with the first frame and the second frame can be considered to be in the background.

  Returning to block 904, as described above, the method 900 may proceed to block 914 if not all elements and / or pixels have the same motion. At block 914, it may be determined whether a dominant element is present in the scene associated with the first and second frames. In some embodiments, determining whether there are dominant elements may be determined based on the number of pixels associated with each element, as described above with respect to dominant element 824 in FIG. . If a dominant element is present, method 900 may proceed to block 916. If there is no dominant element, method 900 may proceed to block 922.

  At block 916, it can be determined whether the dominant element determined at block 914 is substantially in the middle of the scene associated with the first and second frames. If the dominant element is substantially centered, the method 900 may proceed to block 918. If the dominant element is not substantially centered, the method 900 may proceed to block 920.

  At block 918, the foreground may be associated with the dominant element so that the dominant element may be considered the foreground because the dominant element is substantially in the middle of the scene. At block 920, the background can be associated with the dominant element so that the dominant element can be considered the background because the dominant element is not substantially in the middle of the scene.

  Returning to block 914, if it becomes apparent that no dominant element is present, the method 900 may proceed to block 922, as described above. At block 922, the fastest and slowest motion elements associated with the first and second frames may be determined, similar to what was done at block 906. At block 924, it can be determined whether there is a large speed difference between the fastest and slowest moving elements, as was done at block 908. If there is a large speed difference, the method 900 may proceed to block 926. If there is no significant speed difference, the method 900 may proceed to block 928.

  In block 926, the foreground is moved in such a way that the fastest element can be considered to be in the foreground because the speed difference between the fastest and slowest elements is substantially large enough. Can be associated with the fastest element. At block 928, the background or foreground may be associated with the entire scene associated with the first frame and the second frame because the speed difference between the fastest and slowest motion elements is not substantially large enough. . Accordingly, at block 928, the entire scene associated with the first frame and the second frame may be considered to be in the background or foreground.

  Accordingly, the method 900 can be used to determine the background and / or foreground of the scene. Those skilled in the art will appreciate that for the various processes and methods disclosed herein, the functions performed in the processes and methods can be performed in different orders. Further, the outlined steps and operations are provided as examples only, and some of the steps and operations are optional and do not detract from the essence of the disclosed embodiments. Or can be extended to further steps and operations.

  Returning to FIG. 1, in embodiments where the stereoscopic video module 104 may determine that the camera effect is not a pan or zoom effect, as described above, the stereoscopic video module 104 is associated with a frame of the stereoscopic video 101. A left eye view frame and a right eye view frame may be generated based on foreground and / or background motion.

  FIG. 10 illustrates a left-eye view frame 1010 and a right eye of a stereoscopic video 1003 from a stereoscopic video 1001 based on foreground and / or background motion associated with the stereoscopic video 1001, according to some embodiments described herein. An example block diagram 1000 for the generation of a view frame 1012 is shown. The left eye view frame 1010 and the right eye view frame 1012 may be generated by a stereoscopic video module, such as the stereoscopic video module 104 of FIG. The left-eye view frame 1010 and the right-eye view frame 1012 of the stereoscopic video 1003 may correspond to the frame 1002a of the stereoscopic video 1001, and may be generated based on the frame 1002a and its associated modified frame 1002a '. In the illustrated embodiment, the modified frame 1002a ′ may be generated based on movement between the frames 1002a and 1002b in the manner described above with respect to FIG. 2, but in other embodiments, the modified frame 1002a. 'May be generated using a guess frame in the manner described above with respect to FIG.

  In the illustrated embodiment, the stereoscopic video module is based on determining that the camera effect associated with frames 1002a and 1002b is not a pan or zoom effect, and foreground and associated with frames 1002a and 1002b. A left eye view frame 1010 and a right eye view frame 1012 may also be generated based on the relative movement of the background (and thus also based on the modified frame 1002a ′). In some embodiments, the stereoscopic video module may determine the background and foreground using the method 900 described above with respect to FIG. In other embodiments, the background and foreground can be determined by any other suitable method.

  For example, in some examples, the foreground may move faster than the scene between frames 1002a and 1002b, and the stereoscopic video module may require such. In such a case, if the foreground is moving to the right, the stereoscopic video module may designate the frame 1002a as the left-eye view frame 1010, regardless of in which direction the background may move, and the modified frame 1002a ′. It may be designated as the right eye view frame 1012. Further, in such a case, if the foreground is moving to the left, the stereoscopic video module may designate the modified frame 1002a ′ as the left eye view frame 1010, regardless of which direction the background may move. 1002a may be designated as the right eye view frame 1012.

  In other cases, the background may move faster than the foreground between frames 1002a and 1002b, and the stereoscopic video module may seek such. In such a case, if the background is moving to the left, the stereoscopic video module may designate the frame 1002a as the left-eye view frame 1010, regardless of in which direction the foreground may move, and the modified frame 1002a ′. It may be designated as the right eye view frame 1012. Further, in such a case, if the background is moving to the right, the stereoscopic video module may designate the modified frame 1002a ′ as the left eye view frame 1010 regardless of which direction the foreground may move. 1002a may be designated as the right eye view frame 1012.

  Accordingly, the stereoscopic video module is based on the relative motion of the background and foreground associated with frames 1002a and 1002b in accordance with the requirement that the camera effect associated with frames 1002a and 1002b is not a pan or zoom effect. It may be configured to generate a left eye view frame 1010 and a right eye view frame 1012 (and thus also based on the modified frame 1002a ′).

  Modifications, additions, or omissions may be made to FIG. 10 without departing from the scope of the present disclosure. For example, as described above, in the illustrated embodiment, frame 1002b may follow frame 1002a, and modified frame 1002a 'may be generated based on the motion between frames 1002a and 1002b. However, in some cases where a modified frame has been generated for the end of a particular scene or a frame near the end, the modified frame in such a case is a speculative frame as described above with respect to FIG. Can be generated based on Further, in some embodiments, the foreground and background are detected according to the method 900 of FIG. 9, and in other embodiments, the foreground and background can be detected in other acceptable ways.

  According to some embodiments described herein, a stereoscopic video module (e.g., the stereoscopic video module 104 of FIG. 1) is used to determine motion between frames of a stereoscopic video and the associated camera for the stereoscopic video The above description of FIGS. 1-10 shows that based on the effect, it can be configured to generate left and right eye view frames for stereoscopic video. FIG. 11 is a flowchart of an example method 1100 for converting a stereoscopic video to a stereoscopic video based on camera effects according to some embodiments of the present disclosure. The method 1100 may be performed by a stereoscopic video module, such as the stereoscopic video module 104 of FIG. 1, in some embodiments. For example, the stereoscopic video module 104 is configured to execute computer instructions to perform operations for converting a stereoscopic video to a stereoscopic video, as represented by one or more blocks of the method 1100. Can be done. Although illustrated as separate blocks, the various blocks may be divided into further blocks, combined into fewer blocks, or deleted, depending on the desired implementation.

  Method 1100 may begin, but at block 1102, motion between a first frame of a planar video and a second frame of the planar video may be determined. At block 1104, a modified first frame may be generated based on the movement. In some embodiments, the modified first frame may be generated based on the description given with respect to FIG. In other embodiments, the modified first frame may be generated based on a speculative frame as described with respect to FIG.

  At block 1106, the camera effect may be analyzed and determined based on the motion. For example, the camera effect may be determined as a pan effect, a zoom effect, or neither. At block 1108, a left-eye view frame of the stereoscopic video may be determined based on the camera effect analysis and at least one of the first frame and the modified first frame. At block 1110, a right-eye view frame of the stereoscopic video may be determined based on the camera effect analysis and at least one of the first frame and the modified first frame. In the case of a zoom effect, in some embodiments, a second modified first frame may also be generated based on the first frame and the second frame, with the first modified first frame and the second modified first frame being Based on this, left and right eye view frames can be generated. In some embodiments, left and right eye view frames may be generated according to the description given in FIGS. 4-10.

  Thus, the method 1100 can be used to convert a stereoscopic video to a stereoscopic video. Those skilled in the art will appreciate that for this and other processes and methods disclosed herein, the functions performed in the processes and methods can be performed in different orders. Further, the outlined steps and operations are provided as examples only, and some of the steps and operations may be optional and do not detract from the essence of the disclosed embodiments, and may be coupled to fewer steps and operations. Can be extended to further steps and operations.

  For example, in some embodiments, the method 1100 includes steps associated with determining a background and / or foreground that can now be used to generate a stereoscopic video, as described above with respect to FIGS. May be included. In other embodiments, the foreground and background may be determined in other ways or specified by input.

  Returning to FIG. 1, as described above, the stereoscopic video module 104 adjusts the offset between the elements contained in the left and right eye view frames to adjust the amount of depth associated with the stereoscopic video 103. It can also be configured to adjust. As described above, in some embodiments, the stereoscopic video module 104 may modify the stereoscopic perception perceived by the viewer as to which elements may be in the foreground or background of the scene. Video module 104 may also be configured to adjust the focus point of stereoscopic video 103. Depth adjustment and focus adjustment are described below with respect to FIGS. 12A-14.

  FIG. 12A illustrates an example scene 1200 that may be perceived by the left eye 1202 and the right eye 1204 according to some embodiments. The scene 1200 can include a near element 1206, an intermediate element 1208, and a far element 1210. In the illustrated example, left eye 1202 and right eye 1204 can be focused on intermediate element 1208 such that intermediate element 1208 can be in the focus of left eye 1202 and right eye 1204. Thus, the near element 1206 can be in the foreground perceived by the left eye 1202 and the right eye 1204. Further, because the left eye 1202 and right eye 1204 are focused on the intermediate element 1208, the distant element 1210 can be in the background perceived by the left eye 1202 and right eye 1204.

  FIG. 12B is an example that may be seen between elements 1206, 1208, and 1210 in the left and right eye view frames of a stereoscopic video associated with a scene 1200, according to some embodiments disclosed herein. A grid 1201 showing an offset is illustrated. As described above, the offset between elements 1206, 1208, and 1210 may mimic different viewpoints of left eye 1202 and right eye 1204 for scene 1200.

  In the illustrated embodiment, the left near element 1206a may represent the near element 1206 of FIG. 12A from the viewpoint of the left eye 1202 of FIG. 12A, and the right near element 1206b may be the near element of FIG. 12A from the viewpoint of the right eye 1204 of FIG. 1206 may be represented. Thus, the difference in the position of the left near element 1206a for the right near element 1206b in the grid 1201 may represent the offset of the near element 1206 in the left eye view frame for the right eye view frame of the associated stereoscopic video.

  In the illustrated embodiment, the left proximate element 1206a can be on the right. This is because the near element 1206 can be to the right of the line of sight 1212 associated with the left eye 1202, as shown in FIG. 12A. Further, the right near element 1206b may be on the left. This is because the near element 1206 can be to the left of the line of sight 1214 associated with the right eye 1204. This phenomenon is often referred to as “negative parallax” and occurs for an object in front of the focus of the second eye, and in the illustrated embodiment the focus can be set to the intermediate element 1208.

  Intermediate element 1208a / b of grid 1201 may represent intermediate element 1208 of FIG. 12A from both the left eye 1202 and right eye 1204 viewpoints of FIG. 12A. In the illustrated embodiment of FIG. 12A, the intermediate element 1208 can be at the focus of the left eye 1202 and the right eye 1204 such that there can be little or no offset between the viewpoints of the left eye 1202 and the right eye 1204. Thus, the intermediate element 1208 can be in substantially the same position in the right and left eye view frames of the associated stereoscopic video. The intermediate element 1208a / b in FIG. 12B thus illustrates the overlap of the left intermediate element 1208a perceived by the left eye 1202 and the right intermediate element 1208b perceived by the right eye 1204.

  Further, the far left element 1210a and the far right element 1210b may represent the far element 1210 of FIG. 12A from the viewpoint of the left eye 1202 and right eye 1204 of FIG. 12A, respectively. Thus, the difference in position of the far left element 1210a for the far right element 1210b in the grid 1201 may represent the offset of the far element 1210 in the left eye view frame for the right eye view frame of the associated stereoscopic video.

  In the illustrated embodiment, the far left element 1210a can be on the left. This is because the distant element 1210 can be to the left of the line of sight 1212 associated with the left eye 1202, as shown in FIG. 12A. Further, the far right element 1210b may be on the right. This is because the distant element 1210 can be to the right of the line of sight 1214 associated with the right eye 1204. This phenomenon is often referred to as “positive parallax” and occurs for objects behind the focus of the second eye, and in the illustrated embodiment the focus can be set to the intermediate element 1208.

  The amount of stereoscopic video depth can be adjusted by adjusting the offset between corresponding elements of the left-eye view frame and its associated right-eye view frame. For example, the amount of depth associated with the stereoscopic video associated with FIG. 12B can be determined by adjusting the offset between the left near element 1206a and the right near element 1206b to provide an offset between the left intermediate element 1208a and the right intermediate element 1208b. And / or by adjusting the offset between the far left element 1210a and the far right element 1210b. In some embodiments of the present disclosure, the depth may be adjusted by applying a uniform multiplication factor to the offset between elements.

  FIG. 12C shows the offsets of the elements of FIG. 12B with respect to their respective left eye and right eye view frames after applying a uniform multiplication factor to the offset of FIG. 12B, according to some embodiments disclosed herein. An exemplary grid 1203 is illustrated. A uniform multiplication factor may be applied to offsets associated with substantially all elements in the left and right eye view frames so that the offset between elements is adjusted by substantially the same scale. In the illustrated embodiment, the uniform multiplication factor may have a value of 2, so that the offset between elements 1206, 1208, and 1210 in FIG. 12C may each be doubled relative to the corresponding offset in FIG. 12B. To the offset between elements 1206, 1208 and 1210 of FIG.

  For example, in FIG. 12B, the offset of the near element between the center of the left near element 1206a and the center of the right near element 1206b may be about “2” grid units. Thus, applying the multiplication factor “2” to the offset of the near element results in an offset of the near element between the center of the left nearest element 1206a and the center of the right nearest element 1206b in FIG. 12C of about 12 as shown in FIG. 12C. The result can be a “4” grid unit.

  Further, the intermediate element offset between the left intermediate element 1208a and the right intermediate element 1208b may be about “0” grid units. Thus, when the multiplication factor “2” is applied to the intermediate element offset, the number multiplied by “0” is still “0”, so the intermediate element between the left intermediate element 1208a and the right intermediate element 1208b in FIG. 12C. It can result in the offset still being about “0” grid units.

  Further, in FIG. 12B, the offset of the far element between the center of the far left element 1210a and the center of the far right element 1210b may be about “3” grid units. Thus, applying the multiplication factor “3” to the offset of the far element, as shown in FIG. 12C, the offset of the far element between the center of the far left element 1210a and the center of the right far element 1210b in FIG. 12C. Can result in approximately “6” grid units.

  In the illustrated embodiment, in FIG. 12C relative to FIG. 12B, the right near element 1206b associated with the right eye view frame may be shifted approximately “2” grid units to the left, and the left near element 1206a associated with the left eye view frame is Cannot be shifted. Further, in the illustrated embodiment, in FIG. 12C as compared to FIG. 12B, the far right element 1210b associated with the right eye view frame may be shifted to the right by “3” grid units, the far left element associated with the left eye view frame. Element 1210a cannot be shifted. Thus, by adjusting the right eye view frame, the offset of nearby elements can increase from “2” grid units to “4” grid units in FIG. 12C relative to FIG. 12B without having to adjust the left eye view frame. However, the offset of the far element may increase from “3” grid units to “6” grid units.

  In an alternative embodiment, in FIG. 12C versus FIG. 12B, instead of shifting the right near element 1206b and right far element 1210b in the right eye view frame, the left near element 1206a and left far element 1210a in the left eye view frame are shifted. By doing so, the offsets of near and far elements can be adjusted. In other embodiments, in FIG. 12C relative to FIG. 12B, the offset of the near element may be adjusted by shifting both the left near element 1206a and the right near element 1206b. Further, in some of these or other embodiments, in FIG. 12C relative to FIG. 12B, the offset of the far element may be adjusted by shifting both the far left element 1210a and the far right element 1210b.

  In the above example, a specific uniform multiplication factor “2” was used, but any suitable multiplication factor may be used. For example, a suitable multiplication factor greater than “1” may be used to increase the amount of depth perceived by the viewer. Furthermore, an appropriate multiplication factor smaller than “1” may be used to reduce the amount of depth perceived by the viewer. Further, while the uniform multiplication factor has been described above as being applied to the offset associated with the corresponding element, in some embodiments, each element may include one or more pixels, and the uniform multiplication factor corresponds. It can be applied to the offset associated with the pixel. In some embodiments, a uniform multiplication factor may be applied to any offset associated with any element and / or pixel.

  Thus, according to some embodiments of the present disclosure, a stereoscopic video module (eg, stereoscopic video module 104) may correspond to corresponding elements and / or associated with a left-eye view frame and a corresponding right-eye view frame of the stereoscopic video. Or it may be configured to adjust the depth associated with the stereoscopic video by applying a uniform multiplication factor to the offset between pixels. In contrast, conventional depth adjustment procedures cannot apply uniform scaling of offsets to adjust depth.

  The focus of the stereoscopic video can also be adjusted by adjusting the offset between corresponding elements of the left eye view frame and the associated right eye view frame. For example, the stereo video focus associated with FIG. 12B may adjust the offset between the left far element 1210a and the right far element 1210b by adjusting the offset between the left near element 1206a and the right near element 1206b. And by adjusting the offset between the left intermediate element 1208a and the right intermediate element 1208b. In some embodiments, the focus can be adjusted by applying a uniform summing factor to the offset between elements. A uniform summing factor may be applied to offsets associated with substantially all elements in the left eye and right eye view frames. Further, in some embodiments, a uniform summing factor may be applied to the offset associated with the corresponding pixel in the left eye and right eye view frames.

  FIG. 12D shows the offsets of the elements of FIG. 12B with respect to their respective left eye and right eye view frames after applying a uniform summing factor to the offset of FIG. 12B according to some embodiments disclosed herein. An exemplary grid 1205 is illustrated. In the illustrated embodiment, a uniform summing factor may be applied such that the element associated with the right eye view frame is shifted to the left with respect to the element associated with the left eye view frame. Such a shift of the right eye view frame element to the left with respect to the left eye view frame element can result in moving the focus back so that more elements are perceived to be in the foreground.

  In other embodiments, a uniform summing factor may be applied such that the element associated with the right eye view frame is shifted to the right with respect to the element associated with the left eye view frame. Such shifting of the right eye view frame element to the right with respect to the left eye view frame element can result in moving the focus forward so that more elements are perceived to be in the background.

  In the illustrated embodiment, the uniform summing factor may have the value “−3” and the right eye view frame elements may each be shifted 3 grid units to the left relative to their corresponding left eye view frame elements. As can be applied to the offset between elements 1206, 1208 and 1210 of FIG. 12B. In the illustrated embodiment, a negative addition factor can result in a left shift of the right eye view frame element with respect to the left eye view frame element, and a positive addition factor can be the right eye view frame element with respect to the left eye view frame element. This can result in a shift of the frame element to the right. However, in other embodiments, a positive addition factor can result in a left shift of the right eye view frame element relative to the left eye view frame element, and a negative addition factor can result in the right eye view frame element relative to the left eye view frame element. This can result in a right shift of the view frame element.

  For example, the close element offset between the center of the left near element 1206a and the center of the right near element 1206b in FIG. 12B may be a “2” grid unit. Therefore, when the addition coefficient “−3” is applied to the offset of the close element illustrated in FIG. 12B, the offset of the close element between the center of the left nearest element 1206a and the center of the right nearest element 1206b is “ As a result, it can result in shifting the right near element 1206b to the left by "3" grid units relative to the left near element 1206a. This is illustrated in FIG. 12D.

  Further, the intermediate element offset between the left intermediate element 1208a and the right intermediate element 1208b may be about “0” grid units. Therefore, when the addition coefficient “−3” is applied to the intermediate element offset illustrated in FIG. 12B, the offset of the intermediate element between the center of the left intermediate element 1208 a and the center of the right intermediate element 1208 b is applied after the addition coefficient is applied. It may result in shifting the right middle element 1208b to the left by "3" grid units relative to the left middle element 1208a, as may be "3". This is illustrated in FIG. 12D.

  Further, in FIG. 12B, the offset of the far element between the center of the far left element 1210a and the center of the far right element 1210b may be a “3” grid unit. Therefore, when the addition coefficient “−3” is applied to the offset of the far element illustrated in FIG. 12B, the offset of the far element between the center of the left far element 1210 a and the center of the right far element 1210 b after the addition coefficient is applied. Can result in shifting the far right element 1210b to the left by "3" grid units relative to the far left element 1210a, such that can be a "0" grid unit. This is illustrated in FIG. 12D.

  In the illustrated embodiment, the right near element 1206b, the right middle element 1208b, and the far right element 1210b associated with the right eye view frame are each shown in FIG. 12D with respect to FIG. 12B to perform the desired amount of offset adjustment, respectively. The “3” grid unit may be shifted to the left, and the near left element 1206a, the left middle element 1208a, and the far left element 1210a associated with the left eye view frame cannot be shifted. In an alternative embodiment, the near offset, middle offset, far offset may be the left near element 1206a, right near element 1206b, left middle element 1208a, right middle element 1208b, left far element 1210a and / or It can be adjusted by shifting any one of the far right elements 1210b.

  In addition, the illustrated embodiment allows the focus to move backwards so that more elements can be perceived by the viewer to be in the foreground, but the elements associated with the right eye view frame FIG. 6 illustrates shifting to the left with respect to their corresponding elements in the left eye view frame. However, as shown above, in other embodiments, in order to move the focus forward, it may allow more elements to be perceived by the observer as being in the background, Elements associated with the right eye view frame may be shifted to the right with respect to their corresponding elements in the left eye view frame.

  Thus, according to some embodiments of the present disclosure, a stereoscopic video module (eg, stereoscopic video module 104) may include corresponding elements and / or associated with a left-eye view frame and a corresponding right-eye view frame of the stereoscopic video. Or it may be configured to adjust the focus associated with the stereoscopic video by applying a uniform summing factor to the offset between pixels.

  Those skilled in the art will appreciate that for this and other processes and methods disclosed herein, the functions performed in the processes and methods can be performed in different orders. Further, the outlined steps and operations are provided as examples only, and some of the steps and operations may be optional and do not detract from the essence of the disclosed embodiments, and may be coupled to fewer steps and operations. Can be extended to further steps and operations.

  FIG. 13 is a flowchart of an example method 1300 for adjusting the depth of a stereoscopic video according to some embodiments of the present disclosure. The method 1300 may be performed by a stereoscopic video module, such as the stereoscopic video module 104 of FIG. 1, in some embodiments. For example, the stereoscopic video module 104 may be configured to execute computer instructions for performing operations for adjusting the depth of the stereoscopic video, represented by one or more blocks of the method 1300. Although illustrated as separate blocks, various blocks may be divided into further blocks, combined into fewer blocks, or deleted depending on the desired implementation.

  Although the method 1300 may begin, at block 1302, a left-eye view frame of a stereoscopic video may be generated. The left eye view frame may include multiple left eye view frame elements. In some embodiments, the left eye view frame element can be substantially all of the elements included in the left eye view frame such that the left eye view frame element can include substantially the entire left eye view frame. Further, each left eye view frame element may include one or more pixels.

  At block 1304, a right-eye view frame of the stereoscopic video may be generated. The right eye view frame may correspond to the left eye view frame and may include multiple right eye view frame elements. Each right eye view frame element may correspond to one of the left eye view frame elements. In some embodiments, the right eye view frame element can be substantially all elements included in the right eye view frame such that the right eye view frame element can include substantially the entire right eye view frame element. Further, each right eye view frame element may include one or more pixels.

  At block 1306, an offset between each left eye view frame element and its corresponding right eye view frame element may be determined. In some embodiments, the offset can be determined in pixels. At block 1308, a uniform multiplication factor may be applied to each offset so that the depth associated with the stereoscopic video may be adjusted at a substantially uniform scale.

  Thus, the method 1300 can be used to adjust the depth of the stereoscopic video. Those skilled in the art will appreciate that for this and other processes and methods disclosed herein, the functions performed in the processes and methods can be performed in different orders. Further, the outlined steps and operations are provided as examples only, and some of the steps and operations may be optional and do not detract from the essence of the disclosed embodiments, and may be coupled to fewer steps and operations. Can be extended to further steps and operations.

  For example, in some embodiments, the method 1300 may include steps associated with generating a left eye view frame and a right eye view frame. In some of these embodiments, the left eye view frame and right eye view frame may be generated according to one or more manners described above with respect to FIGS. 1-11.

  FIG. 14 is a flowchart of an example method 1400 for adjusting the focus of a stereoscopic video according to some embodiments of the present disclosure. The method 1400 may be performed by a stereoscopic video module, such as the stereoscopic video module 104 of FIG. 1, in some embodiments. For example, the stereoscopic video module 104 may be configured to execute computer instructions for performing operations for adjusting the focus of the stereoscopic video, represented by one or more blocks of the method 1400. Although illustrated as separate blocks, various blocks may be divided into further blocks, combined into fewer blocks, or deleted depending on the desired implementation.

  Although the method 1400 may begin, at block 1402, a left-eye view frame of a stereoscopic video may be generated. The left eye view frame may include multiple left eye view frame elements. In some embodiments, the left eye view frame element can be substantially all of the elements included in the left eye view frame such that the left eye view frame element can include substantially the entire left eye view frame. Further, each left eye view frame element may include one or more pixels.

  At block 1404, a right-eye view frame of the stereoscopic video may be generated. The right eye view frame may correspond to the left eye view frame and may include multiple right eye view frame elements. Each right eye view frame element may correspond to one of the left eye view frame elements. In some embodiments, the right eye view frame element can be substantially all elements included in the right eye view frame such that the right eye view frame element can include substantially the entire right eye view frame element. Further, each right eye view frame element may include one or more pixels.

  At block 1406, an offset between each left eye view frame element and its corresponding right eye view frame element may be determined. In some embodiments, the offset can be determined in pixels. In block 1408, a uniform addition factor may be applied to each offset. A uniform summing factor may be applied such that each right eye view frame element is uniformly shifted by substantially the same amount relative to its corresponding left eye view frame element. Thus, shifting can adjust the focus associated with the stereoscopic video.

  Thus, the method 1400 can be used to adjust the focus of a stereoscopic video. Those skilled in the art will appreciate that for this and other processes and methods disclosed herein, the functions performed in the processes and methods can be performed in different orders. Further, the outlined steps and operations are provided as examples only, and some of the steps and operations may be optional and do not detract from the essence of the disclosed embodiments, and may be coupled to fewer steps and operations. Can be extended to further steps and operations.

  For example, in some embodiments, the method 1400 may include steps associated with generating a left eye view frame and a right eye view frame. In some of these embodiments, the left eye view frame and right eye view frame may be generated according to one or more manners described above with respect to FIGS. 1-11.

  The embodiments described herein may include the use of special purpose or general purpose computers, including various computer hardware or software modules, as discussed in more detail below.

  The embodiments described herein may be implemented using a computer-readable medium for holding or having computer-executable instructions or data structures stored thereon. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can be RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or computer-executable instructions or data. A tangible computer readable storage medium, including any other storage medium that can be used to hold or store the desired program code in the form of structures, or that can be accessed by a general purpose or special purpose computer Can be included. Combinations of the above may also be included within the scope of computer-readable media.

  Computer-executable instructions may be executed by a processing device and may include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Although the subject matter is described in language specific to structural features and / or methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Should be understood. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

  As used herein, the term “module” or “component” refers to a software object or routine that executes on a computing system. The various components, modules, engines, and services described herein may be implemented as objects or processes that execute on the computing system (eg, as separate threads). Although the systems and methods described herein may be implemented in software, implementations in hardware or a combination of software and hardware are possible and contemplated. In this description, a “computing entity” can be any computing system, as defined herein, or any module or combination of modules running on a computing system.

  All examples and terms used herein are intended for educational purposes to help the reader understand the invention and the concepts that the inventors have contributed to and promote technology, and are not limited. Should be construed as such specifically described examples and conditions. Although embodiments of the present invention have been described in detail, various exchanges, substitutions, and changes can be made herein without departing from the spirit and scope of the present invention.

Claims (26)

Generating a left-eye view frame of a stereoscopic video, wherein the left-eye view frame includes a plurality of left-eye view frame elements;
Generating a right-eye view frame of the stereoscopic video, wherein the right-eye view frame corresponds to the left-eye view frame and includes a plurality of right-eye view frame elements, each right-eye view frame element being the left-eye view frame element Generating, corresponding to one of
Determining an offset between each right eye view frame element and its corresponding left eye view frame element; and
Applying a uniform multiplication factor to each offset such that the perceived depth associated with the stereoscopic video is adjusted on a substantially uniform scale;
A method for adjusting the depth in a stereoscopic video. The plurality of left eye view frame elements includes substantially all of the left eye view frame; and
The plurality of right eye view frame elements includes substantially all of the right eye view frame;
The method of claim 1. The plurality of left eye view frame elements includes left eye view frame pixels; and
The plurality of right eye view frame elements include right eye view frame pixels;
The method of claim 2. The uniform multiplication factor is less than 1 to reduce the perceived depth associated with the stereoscopic video;
The method of claim 1. The uniform multiplication factor is greater than 1 to increase the perceived depth associated with the stereoscopic video;
The method of claim 1. Determining a motion between a first frame of a planar video and a second frame of the planar video;
Determining a camera effect based on the movement; and
Generating the left eye view frame and the right eye view frame based on the camera effect;
The method of claim 1 further comprising: The camera effect includes at least one of a pan effect, a zoom effect, a rotation effect, and a static effect,
The method of claim 6. Determining at least one of a foreground and a background based on the first frame and the second frame;
Determining a motion between the first frame and the second frame associated with at least one of the foreground and the background; and
Determining the left eye view frame and the right eye view frame based on the movement associated with at least one of the foreground and the background;
The method of claim 6 further comprising: Determining at least one of the fastest, slowest, and dominant elements associated with the first frame and the second frame; and
At least one of the foreground and the background based on at least one of the fastest, slowest and dominant elements associated with the first and second frames. Seeking
The method of claim 8 further comprising: The method of claim 6, wherein the second frame is a subsequent frame to the first frame. The method of claim 6, wherein the second frame is a previous frame with respect to the first frame. Detecting a scene change in a planar video from a scene related to the first frame and the second frame to another scene;
Estimating subsequent frames of the scene associated with the first frame and the second frame in response to detecting the scene change and based on the motion between the first frame and the second frame And
Generating at least one of the left eye view frame and the right eye view frame based on the inferred subsequent frame;
The method of claim 11 further comprising: A computer-readable storage medium comprising computer-executable instructions for causing a system to perform an operation for adjusting a depth in a stereoscopic video, the operation comprising:
Generating a left-eye view frame of a stereoscopic video, wherein the left-eye view frame includes a plurality of left-eye view frame elements;
Generating a right-eye view frame of the stereoscopic video, wherein the right-eye view frame corresponds to the left-eye view frame and includes a plurality of right-eye view frame elements, each right-eye view frame element being the left-eye view frame element Generating, corresponding to one of
Determining an offset between each right eye view frame element and its corresponding left eye view frame element; and
Applying a uniform multiplication factor to each offset such that the perceived depth associated with the stereoscopic video is adjusted on a substantially uniform scale;
Including a computer-readable storage medium. Generating a left-eye view frame of a stereoscopic video, wherein the left-eye view frame includes a plurality of left-eye view frame elements;
Generating a right-eye view frame of the stereoscopic video, wherein the right-eye view frame corresponds to the left-eye view frame and includes a plurality of right-eye view frame elements, each right-eye view frame element being the left-eye view frame element Generating, corresponding to one of
Determining an offset between each right eye view frame element and its corresponding left eye view frame element; and
In order to shift each right eye view frame element uniformly by substantially the same amount relative to its corresponding left eye view frame element so that the perceived focus associated with the stereoscopic video is adjusted, Applying an additive factor to each offset;
A method for adjusting a focus in a stereoscopic video. The plurality of left eye view frame elements includes substantially all of the left eye view frame; and
The plurality of right eye view frame elements includes substantially all of the right eye view frame;
The method of claim 14. The plurality of left eye view frame elements includes left eye view frame pixels; and
The plurality of right eye view frame elements include right eye view frame pixels;
The method of claim 15. 15. The method of claim 14, wherein the uniform summing factor is applied such that each right eye view frame element is uniformly shifted to the right by substantially the same amount relative to its corresponding left eye view frame element. 15. The method of claim 14, wherein the uniform summing factor is applied such that each right eye view frame element is uniformly shifted to the left by substantially the same amount relative to its corresponding left eye view frame element. Determining a motion between a first frame of a planar video and a second frame of the planar video;
Determining a camera effect based on the movement; and
Generating the left eye view frame and the right eye view frame based on the camera effect;
15. The method of claim 14, further comprising: The camera effect includes at least one of a pan effect, a zoom effect, a rotation effect, and a static effect,
The method of claim 19. Determining at least one of a foreground and a background based on the first frame and the second frame;
Determining a motion between the first frame and the second frame associated with at least one of the foreground and the background; and
Determining the left eye view frame and the right eye view frame based on the movement associated with at least one of the foreground and the background;
20. The method of claim 19, further comprising: Determining at least one of the fastest, slowest, and dominant elements associated with the first frame and the second frame; and
At least one of the foreground and the background based on at least one of the fastest, slowest and dominant elements associated with the first and second frames. Seeking
The method of claim 21, further comprising: The method of claim 19, wherein the second frame is a subsequent frame to the first frame. 20. The method of claim 19, wherein the second frame is a previous frame with respect to the first frame. Detecting a scene change in a planar video from a scene related to the first frame and the second frame to another scene;
Estimating subsequent frames of the scene associated with the first frame and the second frame in response to detecting the scene change and based on the motion between the first frame and the second frame And
Generating at least one of the left eye view frame and the right eye view frame based on the inferred subsequent frame;
The method of claim 24, further comprising: A computer-readable storage medium comprising computer-executable instructions for causing a system to perform an operation for adjusting a focus in a stereoscopic video, the operation comprising:
Generating a left-eye view frame of a stereoscopic video, wherein the left-eye view frame includes a plurality of left-eye view frame elements;
Generating a right-eye view frame of the stereoscopic video, wherein the right-eye view frame corresponds to the left-eye view frame and includes a plurality of right-eye view frame elements, each right-eye view frame element being the left-eye view frame element Generating, corresponding to one of
Determining an offset between each right eye view frame element and its corresponding left eye view frame element; and
In order to shift each right eye view frame element uniformly by substantially the same amount relative to its corresponding left eye view frame element so that the perceived focus associated with the stereoscopic video is adjusted, Applying an additive factor to each offset;
Including a computer-readable storage medium.