CN104243950B - For the method and apparatus that 2 dimension contents are converted to 3-dimensional content in real time - Google Patents
For the method and apparatus that 2 dimension contents are converted to 3-dimensional content in real time Download PDFInfo
- Publication number
- CN104243950B CN104243950B CN201410240725.8A CN201410240725A CN104243950B CN 104243950 B CN104243950 B CN 104243950B CN 201410240725 A CN201410240725 A CN 201410240725A CN 104243950 B CN104243950 B CN 104243950B
- Authority
- CN
- China
- Prior art keywords
- pixel
- picture frame
- motion vector
- subset
- speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The application relates to a kind of method and apparatus for 2 dimension contents are converted to 3-dimensional content in real time.The various aspects of method for video processing and equipment can include the calculating device being communicatively coupled with external device (ED).This calculating device can operate the average vertical speed for determining the pixel subset in picture frame and Mean Horizontal Velocity, and the Mean Horizontal Velocity of motion vector, the average vertical speed of pixel subset and the pixel subset calculated based on the pixel in pixel subset determines the depth value of each pixel in pixel subset.
Description
To Cross-Reference to Related Applications
The application is referred to the U.S. Patent application of the Serial No. 13/432,065 that on March 28th, 2012 submits to, and it is complete
Portion's content is incorporated herein by reference.
The application is referred to the U.S. Patent application of the Serial No. 13/036,656 that on February 28th, 2011 submits to, and it is complete
Portion's content is incorporated herein by reference.
Technical field
The various embodiments of present disclosure relate to Content Transformation.More specifically, the various embodiments of present disclosure relate to
And tie up, by 2, the method and apparatus that (2D) contents are converted to 3-dimensional (3D) content in real time.
Background technology
The producer of consumer electronics device utilizes available limited storage space not affect process energy in trial
The problem that during power, necessary treatment effeciency is relevant.Especially, during two-dimensional content is converted to three-dimensional content in real time, consumer's electricity
Sub-device needs high disposal ability and big storage space.Accordingly, there exist efficient 2D to 3D contents conversion system
Demand.
By some of relatively more such present disclosure having and illustrating referring to the drawings in the remainder of the application
The system of aspect, other limitation and the shortcoming of conventional and traditional method will become for a person skilled in the art
Substantially.
Summary of the invention
For 2 dimension contents are converted in the method and apparatus of 3-dimensional content the most in the accompanying drawings at least one in real time
It is illustrated, and/or combines at least one in accompanying drawing and be described, more fully illustrate as in claim.
By coming together to read the detailed description below of present disclosure with accompanying drawing, it would be recognized that present disclosure
These and other feature and advantage, in the accompanying drawings, identical reference represents identical part all the time.
Accompanying drawing explanation
Fig. 1 is the block diagram of the network environment for real-time 2D to 3D conversion illustrating the embodiment according to present disclosure.
Fig. 2 is the block diagram calculating device of the embodiment according to present disclosure.
Fig. 3 illustrates the picture frame of the embodiment according to present disclosure.
Fig. 4 A to Fig. 4 C is to illustrate the exemplary plot according to the pixel column in the picture frame of the embodiment of present disclosure.
Fig. 5 is to illustrate changing for the real-time 2D to 3D of offer at calculating device of the embodiment according to present disclosure
The flow chart of illustrative steps.
Fig. 6 be illustrate another embodiment according to present disclosure for providing real-time 2D to 3D at device calculating
The flow chart of the illustrative steps of conversion.
Detailed description of the invention
Can be for providing 2 dimension contents to find various enforcement in the method changed in real time and/or equipment of 3-dimensional content
Mode.
The illustrative aspects of present disclosure can include the calculating device for Video processing.In an embodiment, calculate
Device may determine that average vertical speed and/or the Mean Horizontal Velocity of the pixel subset in picture frame.Based on pixel subset
In the pixel motion vector calculated, average vertical speed and the Mean Horizontal Velocity that are associated, it may be determined that pixel subset
In the depth value of each pixel.
Motion vector algorithm possibly cannot calculate the motion vector of the pixel in the region of the homogeneous in picture frame.For entering
The calculating of one step, it may be considered that the motion vector corresponding with this pixel in the region of homogeneous.In an embodiment, Ke Yiti
For the motion vector corresponding with picture frame as input.The motion vector corresponding with one or more pixels is probably the unknown
, or may be undefined in given input.There is the motion vector of the unknown and/or this of undefined motion vector
Pixel may not have predetermined motion vector, it is thereby possible to not included in for the pixel subset of depth calculation.
In an embodiment, based on the motion vector calculated being associated with the pixel in pixel subset, picture frame flat
All vertical pixel speed and average level pixel speed of picture frame, it may be determined that the degree of depth of each pixel in pixel subset
Value.Average vertical pixel speed and average level picture can be calculated based on the pixel in picture frame with predetermined motion vector
Element speed.Based on the depth value being associated with the pixel of picture frame, calculate device and can produce the stereo-picture for picture frame
Right.Stereo pairs can include left image and right image, and in left image, each pixel in picture frame can be with left shift
Specific value, and, in right image, each pixel in picture frame can offset to the right specific value.Specific value is permissible
Proportional to the depth value that each pixel in same picture frame is associated.In an embodiment, pixel subset may belong to picture frame
In a line or multirow pixel.
In an embodiment, picture frame can be two dimension (2D) picture frame.In an embodiment, picture frame can be to have to reduce
Two dimension (2D) picture frame of the pixel resolution of size.In an embodiment, calculate device to deposit from external device (ED) and/or this locality
One or two in reservoir receives picture frame.Calculate device and can include television set, smart phone, tablet PC, above-knee
One or more in type computer and/or desktop PC.External device (ED) can include video player, Set Top Box, individual
One or more in people's video recorder, video camera and/or Cloud Server.
Fig. 1 is the block diagram of the network environment for real-time 2D to 3D conversion illustrating the embodiment according to present disclosure.
With reference to Fig. 1, it is shown that network environment 100.Network environment 100 can include calculating device 102, one or more external device (ED) 104
(such as, video player 104a, Set Top Box 104b, personal video recorder 104c, Cloud Server 104d, video camera 104e) and
Communication network 106.While it is true, present disclosure is not limited to this, and can be not intended to scope of the present disclosure
In the case of use other external device (ED).
Calculate device 102 to may be configured to directly communicate or via communication network 106 with external device (ED) (such as 104a)
Suitable logic, circuit, interface and/or the code of communication.In an embodiment, calculating device 102 is possible can be to external device (ED)
(such as 104a) sends instruction and order, and/or receives instruction and order from this external device (ED).Calculate device 102 also may be used
Can play three-dimensional (3D) content.The example calculating device 102 can include, but not limited to television set, smart phone, put down
Plate computer, laptop computer and/or desktop PC.While it is true, present disclosure is not limited to this and permissible
Other calculating device is used in the case of being not intended to scope of the present disclosure.
External device (ED) (such as 104a) may be configured to and calculate device 102 and directly communicate or via communication network 106
Suitable logic, circuit, interface and/or the code of communication.In an embodiment, external device (ED) (such as 104a) is possible can be to meter
Calculate device 102 and send instruction and order, and/or receive instruction and order from this calculating device 102.In another embodiment
In, external device (ED) (such as 104a) can provide and can be calculated the media content that device 102 is asked.
Communication network 106 can include medium, by this medium, the calculating device 102 in network environment 100 and outside dress
Put (such as 104a) can be in communication with each other.Communication network 106 can be realized by one or more communication protocols, described communication
Agreement includes, but are not limited to, Wireless Fidelity (Wi-Fi), radio universal serial bus (WUSB), LAN (LAN), ZigBee,
Transmission control protocol and Internet Protocol (TCP/IP), Ethernet, cable network, satellite network, DLNA
(DLNA) home network and/or Bluetooth communication protocol.Various assemblies in network environment 100 can be according to various wired and wireless
Communication protocol (such as, TCP/IP, UDP (UDP), ZigBee, infrared (IR), IEEE802.11a,
IEEE802.11b, IEEE802.11g and/or IEEE802.11n communication protocol) it is connected with communication network 106.
When operation, calculate device 102 and external device (ED) (such as 104a) can be switched on, and can be via communication network
Network 106 activates the connection calculated between device 102 and external device (ED) (such as 104a).Calculating device 102 can be based on can be by
Calculate the request that device 102 transmits, receive media content from external device (ED) (such as 104a).Media content can include may make up
One or more picture frames of 2D content.Calculate device 102 and can calculate owning in the one or more picture frames received
The motion vector of pixel.Calculate device 102 and can carry out calculation of motion vectors based on one or more algorithms.Calculate device 102
The degree of depth being associated with one or more pixels can be determined based on the motion vector calculated.
Additionally, calculate device 102 can produce the stereo pairs corresponding to picture frame based on the depth map calculated.
Stereo pairs includes left image and right image.Stereo pairs forms the 2D picture frame transmitted by external device (ED) (such as 104a)
Corresponding 3D content.
In an embodiment, 3D content can shown with calculating on the display screen that device 102 is associated.In embodiment
In, calculating device 102 (such as, smart phone) the 3D content produced can be sent to any display device and (such as, monitor
Device) for display.
In an embodiment, 2D content can be following in one or more: removable in video player 104a
In dish in the film of storage, Set Top Box the 104b Live TV programs transmitted, the media that recorded by personal video recorder 104c
The media content hold, stored in Cloud Server 104d, the video segment (footage) etc. transmitted by video camera 104e.
Fig. 2 is the block diagram calculating device of the embodiment according to present disclosure.Fig. 2 is explained in conjunction with the element in Fig. 1.
With reference to Fig. 2, it is shown that calculate device 102.Calculate device 102 and can include, but not limited to processor 202, transceiver 204, storage
Device 206, communication interface 208 and input and output (I/O) device 210.Memory 206 can include, but not limited to camera application
212, operating system 214, motion vector estimator 216, depth estimator 218, view data 220, estimation data 222, solid
Image generator 224 and out of Memory 226.
Processor 202 can be communicatively coupled with memory 206, transceiver 204 and I/O device 210.Transceiver 204 can
To be communicatively coupled with communication network 106 via communication interface 208.
Processor 202 can include suitably patrolling of operable one group of instruction for performing to be stored in memory 206
Volume, circuit, interface and/or code.Processor can be realized based on one or more processor technology as known in the art
202.The example of processor 202 can be processor based on X86, risc processor, asic processor, cisc processor or appoint
What his processor.
Memory 206 can include operable for storing a group of suitable logic instructed, circuit, the interface received
And/or code.Memory 206 can be based on, but is not limited to, random access memory (RAM), read-only storage (ROM), hard disk
Driver (HDD), storage server and/or secure digital (SD) card realize.
Transceiver 204 can include operable for communicating with external device (ED) (such as 104a) via communication interface 208
Suitably logic, circuit, interface and/or code.In an embodiment, with external device (ED) (such as, transceiver 204 is operable is used for
104a) directly communicate.Transceiver 204 can realize the known skill for supporting the wired or wireless communication with communication network 106
Art.
In I/O device 210 can include can being operatively coupled to receive user's input and show 3D with processor 202
Suitable logic, circuit, interface and/or the code held.The example of input unit can include, but not limited to touch pad, track
Plate, sensor, camera and/or microphone.The example of output device can include, but not limited to liquid crystal display (LCD)
And/or light emitting diode (LED) display.While it is true, present disclosure is not limited to this, and the disclosure can be not intended to
Other I/O device 210 is used in the case of the scope of content.
When operation, 2D content can be sent to calculate device 102 by external device (ED) (such as, 104a).Processor 202 can
The 2D content transmitted by external device (ED) (such as, 104a) with reception.In an embodiment, processor 202 can connect from memory 206
Receive 2D content.Camera application 212 can include can being preferably carried out performing to calculate the various of device 102 by processor 202
Function and the programmed instruction of operation.Operating system 214 can control and coordinate to calculate rudimentary (low-level) merit of device 102
Energy.Motion vector estimator 216 can control and coordinated movement of various economic factors vector estimation procedure.It is every that this can include determining that with picture frame
The motion vector that individual pixel is corresponding, or receive the motion vector information corresponding with picture frame.In an embodiment, motion to
Amount estimator 216 may determine that the average motion vector of picture frame, and the picture frame subsequently for predetermined quantity uses phase
Same average motion vector.
Depth estimator 218 can control and coordinate depth estimation procedure, to determine and each pixel phase in picture frame
The depth value of association.View data 220 can include one or more picture frame.Estimate that data 222 can include for performing
Any kind of information of depth estimation procedure or data.Stereo-picture generator 224 can control and coordinate stereo-picture and produce
Raw process, to produce the stereo pairs corresponding with picture frame.Stereo-picture produces process can use depth estimation procedure
Result produce stereo pairs.Out of Memory 226 can include that other of the operation for calculating device 102 is suitably believed
Breath.In order to be more fully understood that present disclosure, deposit the example that figure 3 illustrates.
Fig. 3 illustrates the picture frame of the embodiment according to present disclosure.Fig. 3 is explained in conjunction with the element in Fig. 1 and Fig. 2.
With reference to Fig. 3, it is shown that picture frame 302.Picture frame 302 can include respectively in the direction illustrated by corresponding arrow D1, D2, D3 etc.
(hereinafter collectively referred to as object 306) and the backgrounds 304 such as the object 306a of upper movement, object 306b, object 306c.Pixel
Collection (such as, 308a, 308b, 308c) is corresponding with the pixel column in picture frame 302.Object 306a as shown in arrow D1 along X
Direction is moved.Object 306a only has speed along the X direction.Object 306b as shown in arrow D2 along X-Y plane diagonal
Ground is mobile.Object 306b have along the X direction with the speed of both Y-directions.Object 306c moves as shown in arrow D3 along the Y direction
Dynamic.Object 306c only has speed along the Y direction.Processor 202 can calculate the motion of the multiple pixels in picture frame 302
Vector.Additionally, processor 202 may determine that the depth value being associated with each pixel in multiple pixels.
In an embodiment, processor 202 can by once process from picture frame 302 pixel subset (such as,
308a, 308b, 308c etc.) perform identification by step.Pixel subset (such as, 308a) may belong in picture frame 302
A line or multirow.
In an embodiment, processor calculates the mean pixel speed of picture frame in the X direction (hereinafter referred to as
“averVX") and the mean pixel speed (hereinafter referred to as " averV of picture frame in the Y directionY”).Processor 202 can
Operation is used for motion vector based on each pixel in picture frame 302 and average pixel speed calculates in picture frame 302
The depth value of each pixel.In an embodiment, processor 202 can use following formula (1) calculate with in picture frame 302
The depth value that each pixel is corresponding:
Depth=abs (VX-averVX)+abs(VY-averVY).............(1)
Wherein, VXAnd VYIt is the speed in the x-direction and the z-direction of the pixel in picture frame 302 respectively, and averVXWith
averVYIt is mean pixel speed in the x-direction and the z-direction respectively.In an embodiment, motion vector algorithm possibly cannot be counted
Calculate the motion vector corresponding with one or more pixels.In this case, the motion vector of this pixel may not include
In the calculating of mean pixel speed.In an embodiment, the mean pixel speed calculated for depth value can be pixel subset
The mean pixel speed of (such as, 308a).Pixel subset (such as, 308a) can include the pixel that can calculate depth value.
Most of pixel with approximately uniform velocity vector is referred to as " background pixel ", and remaining pixel is referred to as
" foreground pixel ".When the averVx, averVy being used for by the average speed of background pixel in above formula (1) is (rather than flat
All frame speeds) time, improve beholder's perceived quality of depth map.
In an embodiment, the motion vector of some pixels in picture frame 302 is probably the unknown.This is likely to occur in line
Reason may lack or in faint region (such as, cloudless sky).Described algorithm can be by using by good from having
The region extrapolation degree of depth of good texture fills the human capabilitiy of the degree of depth in homogenous area to remove this defect.
In an embodiment, the average motion vector of previous picture frame can be applicable to current picture frame.Can use with
The average speed that previous picture frame is associated is to estimate the calculating of the depth value of current image frame.This can reduce and picture frame
Calculating needed for the estimation of 302 depth values being associated and time.Additionally, for film, average motion vector may be at frame
And seldom change between frame.Average motion vector may change between different plots significantly, but in this case,
Beholder may ignore a frame or the multiframe with the incorrect degree of depth, because human brain may require that at least 3 in film
Depth perception based on parallax is formed to 4 frames.
The stereo pairs corresponding with picture frame 302 can be produced.Stereo pairs can include two images, i.e.
Left image and right image.Can be become by the degree of depth that the pixel left shift in picture frame 302 is associated with each pixel
The particular value of ratio produces left image.Can be relevant to each pixel by the pixel in picture frame 302 is offset to the right
The particular value that the degree of depth of connection is proportional produces right image.Stereo pairs forms the 3D of the 2D picture frame received and represents.
Stereo pairs forms the 3D of the 2D picture frame received and represents.Stereo pairs can correspond to fill from by outside
Put the 3D content of the 2D content generation that (such as 104a) transmits.Furthermore, it is possible at the display screen being associated with calculating device 102
Upper display 3D content.The method probes into the Germicidal efficacy of the three-dimensional perception about the mankind.Specifically, use can only pin for the method
The region in picture frame 302 with good texture is defined the observation of depth map accurately.Additionally, the method employing beholder can
Can ignore when new film plot starts the observation of the jump of mean depth the most in a frame.
Fig. 4 A to Fig. 4 C is the exemplary plot of the pixel column in the picture frame 302 illustrating the embodiment according to present disclosure.
Fig. 4 A to Fig. 4 C is explained in conjunction with the element in Fig. 3.Fig. 4 A illustrates and the picture in the picture frame 302 of the location overlap of object 306a
Element row.Each pixel in pixel groups 404 corresponds to object 306a, and only has speed along the X direction.Fig. 4 B illustrate with
Pixel column in the picture frame 302 of the location overlap of object 306b.Each pixel in pixel groups 406 corresponds to object 306b,
And have along the X direction with the speed of Y-direction.Fig. 4 C illustrates and the picture in the picture frame 302 of the location overlap of object 306c
Element row.Each pixel in pixel groups 408 corresponds to object 306c, and only has speed along the Y direction.Pixel groups 402
In each pixel corresponding to background.
In an embodiment, can calculate relative with each pixel in picture frame 302 dividually with Y-direction for X-direction
The motion vector answered.The motion vector calculated provides the speed about each pixel in the X-direction in picture frame and Y-direction
Information.In an embodiment, the letter of the reliability of the possible motion vector that will not provide about each pixel of motion vector algorithm
Breath.This problem can be by calculating local texture, then using local texture to solve as motion vector reliability figure.Have
The pixel of the absolute value of the spatial gradient more than threshold value may be considered that have reliable motion vector.Processor 202 is permissible
The mean pixel speed in the X-direction of picture frame 302 and Y-direction is calculated based on the motion vector calculated.Base can be distinguished
In corresponding to the speed in the X-direction of each pixel in pixel column (such as, 308a) and Y-direction and the X-direction of picture frame 302
With the degree of depth that the mean pixel speed in Y-direction determines each pixel.The previous formula (1) described may be used to determine and picture
The depth value that each pixel in element row (such as, 308a) is associated.Only there is the pixel ginseng of the velocity vector of good definition
With average speed and the calculating of depth value.The degree of depth of the pixel with the speed of not good definition is designated as (seeming equal to zero
They are background pixels).When watching 3D film, the degree of depth is assigned to enter it from the veined nearest region of tool by human brain
These pixels of row interpolation.
In an embodiment, the operable average vertical speed being used for motion vector based on pixel, background pixel of processor 202
Degree and Mean Horizontal Velocity calculate the depth map of each pixel in picture frame 302.Processor 202 can produce based on depth map
The raw stereo pairs corresponding to picture frame 302.Stereo pairs can include left image and right image.Can be by by prospect
Pixel is moved to the left particular value to produce left image.Right image can be produced by the particular value that moved right by foreground pixel.
Particular value can be proportional to the depth value that each pixel in same foreground pixel is associated.
Stereo pairs including left image and right image forms the 3D content corresponding to picture frame 302.Standing of 3D content
Body effect can be displayed on the I/O device 210 calculating device 102.In an embodiment, calculating device 102 can be by 3D
Hold and be sent to and calculate other display device that device 102 is associated.Such as, device 102 (such as, smart phone) is calculated permissible
3D content is sent to display device (such as, monitor).Therefore, it is possible to use calculate device 102 to realize real-time 2D extremely
3D Content Transformation.
In an embodiment, proportionality coefficient can be used to percentage regulation allowance to adapt to the distance of beholder's off screen curtain.This can
To cause producing more preferable 3D content from corresponding 2D picture frame.
In an embodiment, processor 202 can be by picture frame 302 while calculation of motion vectors and/or depth map
The predefined factor of size reduction (such as, in each direction twice).When calculating with direct image to original size
When depth map compares, that the frame of minification is calculated, then application return to the depth perception of original size frame and be probably
Human eye cannot be distinguished by.While it is true, present disclosure is not limited to this, and can scope of the present disclosure being not intended to
In the case of use other the predefined factor.Additionally, for the calculating of motion vector, processor 202 can use image
The luminance component of the pixel in the picture frame of frame 302 or minification.In an embodiment, for the calculating of motion vector, process
Device 202 can use the luminance component of the pixel of the texture having in predefined scope in the picture frame of minification.
In an embodiment, processor 202 can store pixel subset (such as, 308a) in memory 206, in order to calculates
The degree of depth being associated with each pixel.Therefore, it can overcome the needs to full frame memory, because memory 206 can be one
Storing one row pixel at individual time point.It is thus possible to by the methods disclosed herein realize to memory use notable
Reduce.It addition, the pixel resolution of downscaled images frame and reduce the quantity of the pixel in a line and can reduce further and be used for
The memory requirement calculated.In an embodiment, processor 202 can be to the minification of the texture having in predefined scope
Picture frame in the luminance component of pixel operate, thus reduce further and calculate and memory requirement.
According to another embodiment, the calculating time can be reduced by a line or multirow are performed in parallel calculating.To the greatest extent
So, present disclosure is not limited to this to pipe, and can use in the case of being not intended to scope of the present disclosure for
Reduce other technology of calculating time.
Said method can be stored in and calculate on the chip that device 102 is associated, in order to provide 2D to 3D Content Transformation.
Additionally, the method described in present disclosure can be with motion vector calculation chip portfolio, in order to provide and can be used for providing 2D extremely
The self-contained unit of 3D Content Transformation.
Fig. 5 is to illustrate changing for the real-time 2D to 3D of offer at calculating device of the embodiment according to present disclosure
The flow chart of illustrative steps.With reference to Fig. 5, it is shown that method 500.Element in conjunction with Fig. 1, Fig. 2, Fig. 3 and Fig. 4 describes method
500。
Illustrative steps is started in step 502.In step 504, calculate device 102 and can receive by external device (ED) (such as,
2D content 104a) transmitted.Processor 202 can receive corresponding with the 2D content transmitted by external device (ED) (such as, 104a)
Picture frame 302.In step 506, processor 202 can would correspond to the picture frame 302 of 2D content and reduce the predefined factor,
To obtain the picture frame with relatively low pixel resolution.In step 508, processor 202 can calculate has the resolution of relatively low pixel
The motion vector of the pixel of the texture with more than threshold value in the picture frame of rate.In step 510, processor 202 can select
There is the first row pixel in the picture frame of relatively low pixel resolution.
In step 512, processor 202 can calculate the mean pixel speed of the pixel of the row of selection.In step 514, place
Reason device 202 the mean pixel speed of pixel based on the row selected can determine the degree of depth of each pixel in the row of selection
Value.In step 516, processor 202 checks that whether the row selected is last column of picture frame 302.Capable in selection is not
When rear a line, present treatment proceeds to step 518.In step 518, processor 202 selects next line and proceeds to step
512.When the row selected is last column, present treatment proceeds to step 520.In step 520, processor can produce
The stereo pairs corresponding with picture frame 302.Can come based on the depth value being associated with each pixel in picture frame 302
Produce stereo pairs.Method 500 terminates in step 522.
Fig. 6 be illustrate another embodiment according to present disclosure for providing real-time 2D to 3D at device calculating
The flow chart of the illustrative steps of conversion.With reference to Fig. 6, it is shown that method 600.Element in conjunction with Fig. 1, Fig. 2, Fig. 3 and Fig. 4 describes
Method 600.
Illustrative steps is started in step 602.In step 604, calculate device 102 and can receive by external device (ED) (such as,
2D content 104a) transmitted.Processor 202 can receive corresponding with the 2D content transmitted by external device (ED) (such as, 104a)
Picture frame 302.In step 606, processor 202 can would correspond to the picture frame 302 of 2D content and reduce the predefined factor,
To obtain the picture frame with relatively low pixel resolution.In step 608, processor 202 can calculate has the resolution of relatively low pixel
The motion vector of the pixel of the texture with more than a certain threshold value in the picture frame of rate.In step 610, processor 202 is permissible
Calculate the mean pixel speed of the picture frame received.
In step 612, processor 202 can select the first row pixel having in the picture frame of relatively low pixel resolution.
In step 614, processor 202 may determine that the depth value of each pixel in the row of selection.In step 616, processor 202 is examined
Whether the row looking into selection is last column of picture frame 302.Select capable be not last column when, present treatment advance
To step 618.In step 618, processor 202 selects next line and proceeds to step 614.It is last column at the row selected
Under situation, present treatment proceeds to step 620.In step 620, processor can produce the stereogram corresponding with picture frame 302
As right.Stereo pairs can be produced based on the depth value being associated with each pixel in picture frame 302.Method 600 exists
Step 622 terminates.
According to the embodiment of present disclosure, the equipment for Video processing can include calculating device 102 (Fig. 1), should
Calculate device 102 to be communicatively coupled with external device (ED) (such as, 104a, Fig. 1).Calculate device 102 and can include one
Or multiple processor and/or circuit, such as, operable being used for of processor 202 (Fig. 2) determines the pixel in picture frame 302 (Fig. 3)
The average vertical speed of subset (such as, 308a) and Mean Horizontal Velocity.One or more processors and/or the operable use of circuit
Come based on the motion vector calculated of each pixel in pixel subset (such as, 308a), the average vertical speed of pixel subset
The Mean Horizontal Velocity of degree and pixel subset determines the depth value of each pixel in pixel subset (such as, 308a).In reality
Executing in example, one or more processors and/or circuit are operable for based on each pixel in pixel subset (such as, 308a)
The motion vector calculated, the average vertical pixel speed of picture frame 302 and picture frame 302 average level pixel speed come
Determine the depth value of each pixel in pixel subset (such as, 308a).Can based on picture frame has predetermined motion to
The pixel of amount determines average vertical pixel speed and average horizontal pixel speed.One or more processors and/or circuit can
Operation is used for the depth value of determination based on each pixel in picture frame 302 and produces the stereo pairs of picture frame 302.
One or more processors and/or circuit, such as, processor 202, operable being used for produces image based on depth map
The stereo pairs of frame 302.One or more processors and/or circuit, such as, processor 202, operable being used for produces left figure
Picture and right image.Left image can be produced by each pixel in picture frame 302 is moved to the left particular value, and permissible
Right image is produced by the particular value that each pixel in picture frame 302 moved right.With each pixel in picture frame 302
Corresponding particular value can be proportional to the depth value that same pixel is associated.
One or more processors and/or circuit, such as, processor 202, operable for from external device (ED) 104 (Fig. 1)
And/or one or two reception picture frame 302 in local storage (such as, 204, Fig. 2).External device (ED) 104 can include
In video player 104a, Set Top Box 104b, personal video recorder 104c, Cloud Server 104d and/or video camera 104e
One or more.Calculate device 102 and can include television set, smart phone, tablet PC, laptop computer and/or table
One or more in laptop computer.
The other embodiments of present disclosure can provide non-transitory computer-readable medium and/or storage medium, and/or
Non-transient state machine readable media and/or storage medium, on it, storage has and has at least can performed by machine and/or computer
The machine code of individual code segment and/or computer program, so that this machine and/or computer perform to be included in and for video
The step calculated in device that the external device (ED) processed is communicatively coupled.These steps can include determining that the picture in picture frame
The average vertical speed of sub-prime collection and/or Mean Horizontal Velocity.These steps may include that based on each picture in pixel subset
The motion vector calculated, average vertical speed and the Mean Horizontal Velocity of element, determines the deep of each pixel in pixel subset
Angle value.Pixel subset includes the pixel with predetermined motion vector.
Therefore, present disclosure can realize with the combination of hardware or hardware and software.Present disclosure can be
At least one computer system realizes in the way of centralization, or can realize in a distributed fashion, in distribution
In the mode of formula, different elements can be expanded in the middle of the computer system of several interconnection.It is adapted for carrying out retouching herein
Any kind of computer system of the method stated or miscellaneous equipment are probably applicable.The combination of hardware and software can be tool
Having the general-purpose computing system of computer program, this computer program can control computer system when loaded and executed,
Computer system is made to realize method described herein.Present disclosure can be to include the integrated electricity also performing other function
The hardware of the part on road realizes.
Present disclosure can also be embedded in computer program, and this computer program includes realizing herein
Described in all of feature of embodiment of method, and be capable of these sides when being loaded in computer systems
Method.Computer program in the present context refers to be intended to the system so that having information processing capability directly or in following
One or two after perform specific function with any language, code or one group of instruction representing of mark: a) be converted to separately
A kind of language, code or mark；B) reproduce with different material forms.
Although describing present disclosure for various embodiments, it will be understood by those skilled in the art that not
In the case of deviation scope of the present disclosure, various amendment can be carried out, and each equivalent can be substituted.Furthermore it is possible to
In the case of without departing from scope of the present disclosure, carry out multiple amendment so that specific situation or material adapt in the disclosure
The teaching held.Therefore, present disclosure should be not limited to disclosed specific embodiment, and present disclosure will include falling
Enter all of embodiment in scope of the following claims.
It addition, the application also can be configured as follows:
(1) a kind of method for video processing, described method includes:
In calculating device:
The motion vector calculated based on each pixel in the pixel subset in picture frame, described picture frame average
The average level pixel speed of vertical pixel speed and described picture frame determines in the described pixel subset in described picture frame
The depth value of each pixel,
Wherein, described average vertical pixel speed is determined based on the pixel in described picture frame with predetermined motion vector
Degree and described average level pixel speed.
(2) a kind of method for video processing, described method includes:
In calculating device:
Determine average vertical speed and/or the Mean Horizontal Velocity of pixel subset in picture frame；And
The motion vector calculated based on each pixel in described pixel subset, described average vertical speed and described
Mean Horizontal Velocity determines the depth value of the described each pixel in described pixel subset,
Wherein, described pixel subset includes the pixel with predetermined motion vector.
(3) according to method described in (2), including based on the described each pixel in described picture frame described determine deep
Angle value produces the stereo pairs of described picture frame.
(4) according to the method described in (3), wherein, the stereo pairs of described generation includes:
Left image, wherein, the described each pixel left shift particular value in described picture frame；And
Right image, wherein, the described each pixel in described picture frame offsets to the right described particular value,
Wherein, the described depth value determined that described particular value is associated with the described each pixel in described picture frame
Proportional.
(5) according to the method described in (2), wherein, described pixel subset includes a line in described picture frame or multirow picture
Element.
(6) according to the method described in (2), wherein, calculate described in the described each pixel in described pixel subset
Motion vector includes described each pixel speed in X-direction and/or Y-direction.
(7) according to the method described in (2), wherein, described picture frame is two dimension 2D picture frame.
(8) according to the method described in (2), wherein, described picture frame is the two dimension of the pixel resolution with minification
2D picture frame.
(9) according to the method described in (2), receive including one or two from external device (ED) and/or local storage
Described picture frame.
(10) according to the method described in (2), wherein, described calculating device includes that television set, smart phone, flat board calculate
One or more in machine, laptop computer and/or desktop PC.
(11) according to the method described in (9), wherein, described external device (ED) includes video player, Set Top Box, individual video
One or more in logger, video camera and/or Cloud Server.
(12) a kind of calculating device for Video processing, described calculating device includes:
One or more processors and/or circuit, it is possible to operation is used for:
Determine average vertical speed and/or the Mean Horizontal Velocity of pixel subset in picture frame；And
The motion vector calculated based on each pixel in described pixel subset, described average vertical speed and described
Mean Horizontal Velocity determines the depth value of the described each pixel in described pixel subset,
Wherein, described pixel subset includes the pixel with predetermined motion vector.
(13) according to the calculating device described in (12), wherein, the one or more processor and/or circuit are operable to
The stereogram of described picture frame is produced for the described depth value determined based on the described each pixel in described picture frame
As right.
(14) according to the calculating device described in (13), wherein, the one or more processor and/or circuit are operable to
For producing described stereo pairs based on left image and right image,
Wherein, by the described each pixel left shift particular value in described picture frame is produced described left image,
Wherein, described right figure is produced by the described each pixel in described picture frame is offset to the right described particular value
Picture, and
Wherein, described particular value is proportional to the described depth value determined of the described each pixel in described picture frame.
(15) according to the calculating device described in (12), wherein, the one or more processor and/or circuit are operable to
For calculating depth map.
(16) according to calculating device described in (12), wherein, described pixel subset include a line in described picture frame or
Multirow pixel.
(17) according to the calculating device described in (12), wherein, the described meter of the described each pixel in described pixel subset
The motion vector calculated includes described each pixel speed in X-direction and/or Y-direction.
(18) according to the calculating device described in (12), wherein, the one or more processor and/or circuit are operable to
Described picture frame is received for one or two from external device (ED) and/or local storage.
(19) according to calculating device described in (18), wherein, described external device (ED) includes video player, Set Top Box, individual
One or more in people's video recorder, video camera and/or Cloud Server.
(20) according to the calculating device described in (12), wherein, described calculating device includes television set, smart phone, flat board
One or more in computer, laptop computer and/or desktop PC.
Claims (9)
1. a method for video processing, described method includes:
In calculating device:
The motion vector calculated based on each pixel in the pixel subset in picture frame, the average vertical of described picture frame
It is every that the average level pixel speed of pixel speed and described picture frame determines in the described pixel subset in described picture frame
The depth value of individual pixel,
Wherein, based on the pixel in described picture frame with predetermined motion vector determine described average vertical pixel speed and
Described average level pixel speed,
Wherein, the one-row pixels during described pixel subset includes described picture frame.
2. a method for video processing, described method includes:
In calculating device:
Determine average vertical speed and/or the Mean Horizontal Velocity of pixel subset in picture frame；And
The motion vector calculated based on each pixel in described pixel subset, described average vertical speed and described averagely
Horizontal velocity determines the depth value of the described each pixel in described pixel subset,
Wherein, described pixel subset includes the pixel with predetermined motion vector,
Wherein, the one-row pixels during described pixel subset includes described picture frame.
Method the most according to claim 2, described determines including based on the described each pixel in described picture frame
Depth value produces the stereo pairs of described picture frame.
Method the most according to claim 3, wherein, the stereo pairs of described generation includes:
Left image, wherein, the described each pixel left shift particular value in described picture frame；And
Right image, wherein, the described each pixel in described picture frame offsets to the right described particular value,
Wherein, the described depth value determined that described particular value is associated with the described each pixel in described picture frame becomes ratio
Example.
Method the most according to claim 2, wherein, calculates described in the described each pixel in described pixel subset
Motion vector includes described each pixel speed in X-direction and/or Y-direction.
Method the most according to claim 2, wherein, described picture frame is two dimension 2D picture frame.
Method the most according to claim 2, wherein, described picture frame is the two dimension of the pixel resolution with minification
2D picture frame.
Method the most according to claim 2, receives including one or two from external device (ED) and/or local storage
Described picture frame.
9., for a calculating device for Video processing, described calculating device includes:
One or more processors and/or circuit, it is possible to operation is used for:
Determine average vertical speed and/or the Mean Horizontal Velocity of pixel subset in picture frame；And
The motion vector calculated based on each pixel in described pixel subset, described average vertical speed and described averagely
Horizontal velocity determines the depth value of the described each pixel in described pixel subset,
Wherein, described pixel subset includes the pixel with predetermined motion vector,
Wherein, the one-row pixels during described pixel subset includes described picture frame.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US13/911,389 | 2013-06-06 | ||
| US13/911,389 US9483836B2 (en) | 2011-02-28 | 2013-06-06 | Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104243950A CN104243950A (en) | 2014-12-24 |
| CN104243950B true CN104243950B (en) | 2016-08-24 |
Family
ID=52144931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410240725.8A Active CN104243950B (en) | 2013-06-06 | 2014-06-03 | For the method and apparatus that 2 dimension contents are converted to 3-dimensional content in real time |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP5858254B2 (en) |
| CN (1) | CN104243950B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104615421A (en) * | 2014-12-30 | 2015-05-13 | 广州酷狗计算机科技有限公司 | Virtual gift display method and device |
| CN111647807B (en) * | 2020-05-18 | 2022-03-04 | 樟树市兴隆高新材料有限公司 | High-alloy die steel and preparation process thereof |
| CN115786816B (en) * | 2022-12-14 | 2024-03-01 | 河钢股份有限公司 | Controllable eutectic carbide high-strength high-toughness cold-working die steel and preparation method thereof |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001359119A (en) * | 2000-06-15 | 2001-12-26 | Toshiba Corp | Stereoscopic video image generating method |
| CN102724531A (en) * | 2012-06-05 | 2012-10-10 | 上海易维视科技有限公司 | Method and system for converting two-dimensional video into three-dimensional video |
| WO2013023325A1 (en) * | 2011-08-18 | 2013-02-21 | 北京世纪高蓝科技有限公司 | Method for converting 2d into 3d based on image motion information |
| CN103250184A (en) * | 2010-11-23 | 2013-08-14 | 高通股份有限公司 | Depth estimation based on global motion |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000261828A (en) * | 1999-03-04 | 2000-09-22 | Toshiba Corp | Stereoscopic video image generating method |
| JP3938122B2 (en) * | 2002-09-20 | 2007-06-27 | 日本電信電話株式会社 | Pseudo three-dimensional image generation apparatus, generation method, program therefor, and recording medium |
| JP2006014121A (en) * | 2004-06-29 | 2006-01-12 | Sanyo Electric Co Ltd | Image encoding method, image encoding apparatus, and imaging apparatus |
| CN102326394B (en) * | 2009-09-08 | 2014-06-11 | 株式会社东芝 | Image processing method and apparatus |
| US8537200B2 (en) * | 2009-10-23 | 2013-09-17 | Qualcomm Incorporated | Depth map generation techniques for conversion of 2D video data to 3D video data |
| KR20120049997A (en) * | 2010-11-10 | 2012-05-18 | 삼성전자주식회사 | Image process device, display apparatus and methods thereof |
-
2014
- 2014-06-02 JP JP2014113801A patent/JP5858254B2/en active Active
- 2014-06-03 CN CN201410240725.8A patent/CN104243950B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001359119A (en) * | 2000-06-15 | 2001-12-26 | Toshiba Corp | Stereoscopic video image generating method |
| CN103250184A (en) * | 2010-11-23 | 2013-08-14 | 高通股份有限公司 | Depth estimation based on global motion |
| WO2013023325A1 (en) * | 2011-08-18 | 2013-02-21 | 北京世纪高蓝科技有限公司 | Method for converting 2d into 3d based on image motion information |
| CN102724531A (en) * | 2012-06-05 | 2012-10-10 | 上海易维视科技有限公司 | Method and system for converting two-dimensional video into three-dimensional video |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104243950A (en) | 2014-12-24 |
| JP2014239436A (en) | 2014-12-18 |
| JP5858254B2 (en) | 2016-02-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6563453B2 (en) | Generation of a depth map for an input image using an exemplary approximate depth map associated with an exemplary similar image | |
| CN102741879B (en) | Method for generating depth maps from monocular images and systems using the same | |
| WO2019149206A1 (en) | Depth estimation method and apparatus, electronic device, program, and medium | |
| Karsch et al. | Depth extraction from video using non-parametric sampling | |
| US8644596B1 (en) | Conversion of monoscopic visual content using image-depth database | |
| US11138432B2 (en) | Visual feature tagging in multi-view interactive digital media representations | |
| US9483836B2 (en) | Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content | |
| KR20180029315A (en) | Image processing method and apparatus for projecting an image of virtual reality content | |
| CN110869980A (en) | System and method for distribution and presentation of content as a spherical video and 3D portfolio | |
| CN105847850A (en) | Panorama video real time playing method and apparatus | |
| CN105516579B (en) | A kind of image processing method, device and electronic equipment | |
| CN108833877B (en) | Image processing method and device, computer device and readable storage medium | |
| US8577202B2 (en) | Method for processing a video data set | |
| CN105204347A (en) | Method, device and system for smart home interaction based on augmented reality technologies | |
| CN109345581B (en) | Augmented reality method, device and system based on multi-view camera | |
| US10127714B1 (en) | Spherical three-dimensional video rendering for virtual reality | |
| DE112016002022T5 (en) | MANAGING FEATURE DATA FOR ENVIRONMENTAL IMAGING ON AN ELECTRONIC DEVICE | |
| CN104243950B (en) | For the method and apparatus that 2 dimension contents are converted to 3-dimensional content in real time | |
| CN113643342A (en) | Image processing method and device, electronic equipment and storage medium | |
| EP2574066A2 (en) | Method and apparatus for converting 2D content into 3D content | |
| CN111292425A (en) | View synthesis method based on monocular and binocular mixed data set | |
| CN108124148A (en) | A kind of method and device of the multiple view images of single view video conversion | |
| CN102780900B (en) | Image display method of multi-person multi-view stereoscopic display | |
| Seitner et al. | Trifocal system for high-quality inter-camera mapping and virtual view synthesis | |
| Nagai et al. | An on-site visual feedback method using bullet-time video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |