CN105611271A - Real-time stereo image generating system - Google Patents
Real-time stereo image generating system Download PDFInfo
- Publication number
- CN105611271A CN105611271A CN201510967607.1A CN201510967607A CN105611271A CN 105611271 A CN105611271 A CN 105611271A CN 201510967607 A CN201510967607 A CN 201510967607A CN 105611271 A CN105611271 A CN 105611271A
- Authority
- CN
- China
- Prior art keywords
- pixel
- unit
- stereo
- visual point
- point image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/128—Adjusting depth or disparity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention discloses a real-time stereo image generating system, and belongs to the technical field of image processing. The real-time stereo image generating system comprises a multi-view generating module, a caching module and a stereo image synthesizing module. The multi-view generating module is composed of N parallel depth-image-based rendering processing units, and used for generating N virtual view images according to original image and disparity map information. The data of the generated virtual view images is stored into N dual-port dual-cache units of the caching module. The stereo image synthesizing module reads the data of the virtual view images from the caching module, synthesizes the N virtual view images into a stereo image which is applied to a lenticular grating auto-stereoscopic display for displaying. According to the real-time stereo image generating system, the characteristic that FPGA supports parallel and pipeline processing is used, thus the generation speed of the stereo image is improved, and the real-time processing performance of the system is improved.
Description
Technical field
The invention belongs to technical field of image processing, more specifically, relate to a kind of real-time stereo images rawOne-tenth system.
Background technology
Nowadays 3D Video Applications has appeared in people's life, entertainment field, military field, doctorIts figure can be seen in treatment field etc. While watching general three-dimensional video-frequency, people need to wear red blue glassesOr the auxiliary equipment such as polarised light glasses, thereby limit applying of Stereoscopic Video Presentation technology. InstituteMeaning free 3 D display technology, refers to that beholder is under the condition without any need for auxiliary equipment, directlyObservation display screen is the relief a kind of technology of energy perception just, i.e. bore hole 3D technology. Free stereo is aobviousShow technology, can not only make beholder obtain third dimension, and be convenient to watch, its development prospect is wide,It is a current study hotspot.
The principle of the free 3 D display technology based on column mirror grating is to place a post in display frontMirror grating, the image that several are had to parallax is presented at display in staggered mode. When watchingPerson stands in certain viewing location, and the light that display epigraph pixel sends produces through column mirror gratingRefraction enters into human eye, and human eye can be observed the different image of two width like this. Column mirror grating is because of its solidEffective, good in optical property and low cost and other advantages, in free stereo demonstration field, application is more.
Auto-stereo display system mainly comprises collection, coding, the transmission of video information and shows that these are severalIndividual step. Existing auto-stereoscopic display needs the information of multiple viewpoints at display end, watch like thisPerson is watching in scope, and right and left eyes can receive two width and have the image of parallax, through the fusion of brainPerceive third dimension thereby process. The source of these many view information is problems to be solved, asFruit utilizes multiple cameras to carry out the collection of image, and information content will sharply increase like this, needs designGood encoding scheme, simultaneously high to transmission bandwidth requirement, the memory space of hardware is required greatly, thereforeThis scheme is generally inadvisable. Three-dimensional display system is high to requirement of real-time in addition, so that beholder's energySee continuous, smooth video.
Summary of the invention
For the demand occurring in above-mentioned background technology, how to complete in real time from 2D figure and generate 3D figure,The invention provides a kind of real-time stereo images generation system, comprise many viewpoints generation module, cache moduleWith stereo-picture synthesis module, wherein:
Described many viewpoints generation module comprise N parallel draw processing unit based on depth image, itsMiddle N=x2, x is greater than 1 positive integer, draws processing unit according to former described in each based on depth imageBeginning figure and disparity map generating virtual visual point image are also sent in described cache module, send instruction simultaneouslyThe signal that described drawing virtual view image completes is to described stereo-picture synthesis module;
Described cache module comprises N the two buffer units of parallel dual-port, receives respectively and store instituteState N the described virtual visual point image that many viewpoints generation module generates, wherein the two buffer memorys of each dual-portUnit comprises that two degree of depth are that the color of described virtual visual point image width, bit wide and described original graph is darkSpend consistent two-port RAM, described two two-port RAMs composition double-damping structure is realized ping-pong operation,To realize the stream treatment to described virtual visual point image; And
Described stereo-picture synthesis module comprises pixel counter and stereo-picture generation unit, described pictureElement counter receives after the signal that described instruction drawing virtual view image completes, count asLocation is input to described cache module, and described stereo-picture generation unit reads ground from described cache moduleThe virtual visual point image data that the memory cell that location is described counting is stored, by described N virtual lookingDot image synthesizes 1 stereo-picture and exports.
In general, the above technical scheme of conceiving by the present invention compared with prior art, hasFollowing beneficial effect:
Utilize DIBR technology to realize the drafting of many viewpoints according to original graph and disparity map information, for freely standingBody shows provides many view information, has reduced transmission bandwidth, memory space and the cost of system;
Make full use of the characteristic that FPGA supports parallel and stream treatment, the processing to pixel is adopted to hardwareAccelerate to process, greatly improve system processing speed, raising system is processed real-time;
The applicability of system is good, and the mode of native system generating virtual viewpoint does not rely on camera parameter, suitableExtensive by property.
Experimental result shows: the XC6SLX150 of the Spartan6 family model of producing in Xilinx companyFPGA upper, processing speed of the present invention is the software of the PC of the Duo i5 of Intel than processorJoin fast 8 times of speed.
Brief description of the drawings
Fig. 1 is the hardware block diagram of real-time stereo images generation system of the present invention;
Fig. 2 is the structured flowchart of DIBR processing unit of the present invention;
Fig. 3 is the theory diagram of the two buffer units of dual-port of the present invention;
Fig. 4 is the connection diagram of cache module of the present invention and stereo-picture synthesis module.
Detailed description of the invention
In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with accompanying drawingAnd embodiment, the present invention is further elaborated. Should be appreciated that described herein concreteEmbodiment only, in order to explain the present invention, is not intended to limit the present invention. In addition described,Involved technical characterictic in each embodiment of the present invention just can as long as do not form to conflict each otherMutually to combine.
The embodiment of the present invention utilizes VHDL in the Spartan6 of the Xilinx company XC6SLX150 of family typeNumber upper enforcement of FPGA, fully excavate FPGA concurrency, and incorporating pipeline technology and Streaming MediaKnow-why is applied to many viewpoints generations of parallax and calculates with stereo-picture synthetic.
Figure 1 shows that the hardware block diagram of real-time stereo images generation system of the present invention. As shown in Figure 1,Real-time stereo images generation system comprises that many viewpoints generation module, cache module and stereo-picture synthesize mouldPiece.
In embodiments of the present invention, many viewpoints generation module is by 9 parallel drawing based on depth image(Depth-Image-BasedRendering, hereinafter to be referred as DIBR) processing unit DIBR_V1～DIBR_V9 forms, and is 640 × 360 in order to generate respectively 9 resolution ratio according to original graph and disparity mapVirtual visual point image, send that to be used to indicate the virtual visual point image of filling through cavity raw simultaneously(wherein, DIBR_V1 sends hf_over1, DIBR_V2 to the signal hf_over1～hf_over9 having becomeSend hf_over2, by that analogy) to the pixel counter of stereo-picture synthesis module. Wherein, formerBeginning to scheme (size is 640 × 360, and color depth is 24, R8G8B8) is to take from binocular cameraIn the left and right view image collecting, appoint and get one; Disparity map (size is 640 × 360,64 gray scales)For being calculated through Stereo matching by original graph, its size is identical with original graph.
Cache module comprises the two buffer units 1～9 of 9 parallel dual-ports. DIBR processing unitThe every generation a line of DIBR_V1～DIBR_V9 virtual visual point image data just respectively correspondence be stored in buffer memory mouldIn the two buffer units 1～9 of dual-port of piece. Cache module is sent out received virtual visual point image dataDeliver to stereo-picture synthesis module.
Stereo-picture synthesis module comprises pixel counter and stereo-picture generation unit. Pixel counterReceive signal hf_over1～hf_over9, after 9 signals all arrive, pixel counter startsEach clock cycle of virtual visual point image clock signal increases by 1, while being increased to maximum, makes zero, and its changeChange scope is 0～640 × 360. Stereo-picture generation unit reads virtual visual point image number from cache moduleAccording to, the virtual visual point image that 9 color depths are 24, resolution ratio is 640 × 360 is synthesized to 1Resolution ratio is 1920 × 1080 stereo-picture (color depth is 24, R8G8B8). At thisIn bright embodiment, the virtual visual point image that is width × height for size, its composite diagram largeLittle is (3 × width) × (3 × height).
Figure 2 shows that the structured flowchart of DIBR processing unit of the present invention. As shown in Figure 2, each DIBRProcessing unit comprises viewpoint mapping block, empty tag cache module, empty viewpoint cache module and cavityFill up module.
Viewpoint mapping block comprises that coordinate computing unit, zbuf buffer unit and parallax search unit. ItsIn, coordinate computing unit is according to the gray value of certain pixel (u0, v0) in disparity map, according to followingEquivalence formula (1) calculates this coordinate figure (u0+dis × n, v0) corresponding in virtual visual point image,Complete line by line the mapping from reference view to virtual view, and representing that a line reference view is to virtualWhen completing, the mapping of viewpoint sends the row_over signal of having indicated to hole-filling module, to notifyHole-filling module can be carried out hole-filling to new data line.
Zbuf buffer unit by 2 capacity be 640, bit wide be 6 (disparity map gray scale 64 rank,6 bit representation disparity map gray scale figure places) single port RAM composition, for storing zbuffer arithmetic result.Coordinate computing unit calculates after its coordinate result according to certain pixel of disparity map, first reads and sentencesIn disconnected zbuf buffer unit, address is whether the unit of this coordinate result has existed parallax record, ifExist, the parallax parallax in zbuf buffer unit and current calculating being adopted compares, generalThe coordinate at the pixel place that parallax is less is sat as the source points of certain pixel in virtual visual point imageMark, and in zbuf buffer unit, in the unit of address for this coordinate result, record that calculating adoptsParallax value. This process of zbuffer algorithm can solve blocking in pixel mapping calculation process and askTopic.
It is 64 (consistent with the pixel grey scale progression 64 of disparity map) by a capacity that parallax is searched unitROM forms, and it comprises a parallax look-up table, for completing the calculating from parallax value to parallax function,Wherein parallax value is the gray scale value of each pixel in disparity map, and scope is 0～63; Deviant is for participating inVirtual view generates an intermediate object program of calculating, and is integer, and scope is-12～12. Wherein 0～63Linearity corresponds on-12～12, and the corresponding deviant that for example parallax value is 31 is 0. Due to skewValue and the proportional relation of the degree of depth, the embodiment of the present invention is put into parallax by deviant instead of depth value and is searchedIn table, can speed-up computation.
The coordinate computing unit of 9 DIBR processing units of many viewpoints generation module calculates respectively formerFour virtual visual point images on beginning figure, four, original graph left side virtual visual point image and the right, according to DIBRAlgorithm, is simplified equivalence formula (1):
VIR_n(u0+DEEP(DIS(u0,v0))×n,v0)＝ORI(u0,v0)(1)
Wherein, VIR_n represent n virtual visual point image picture element matrix (n=-4 ,-3 ..., 4),ORI (u0, v0) represents the picture element matrix of original graph; Original graph n=0, original graph adjacent left-hand m virtualVisual point image n=-1～-m, the adjacent right side y of original graph virtual visual point image n=1～m, m is integer,M=in the time that N is odd number (N-1)/2, m=N/2 in the time that N is even number, in embodiments of the present invention, originalThe 1st virtual visual point image n=-1 of figure adjacent left-hand, the 1st, the adjacent right side of original graph virtual view figurePicture n=1; The 2nd virtual visual point image n=-2 of original graph adjacent left-hand, the adjacent right side of original graph the 2ndIndividual virtual visual point image n=2; It is known by that analogy that in 9 virtual visual point images, the leftmost side is virtual looksThe n=-4 of dot image, the n=4 of rightmost side virtual visual point image; U0, v0 are respectively pixel in original graphThe transverse and longitudinal coordinate of point; DEEP represents parallax look-up table, and DIS represents disparity map, and DIS (u0, v0) represents to sitThe parallax value of punctuate (u0, v0) point, the depth value of DEEP (DIS (u0, v0)) denotation coordination point (u0, v0) point.For example, for leftmost side virtual visual point image, have:
VIR_-4(u0+DEEP(DIS(u0,v0))×(-4),v0)＝ORI(u0,v0)
In the picture element matrix of virtual visual point image, the value of some point, less than determining, is cavity, hasA little points have repeatedly value, are and block. Through calculating, 9 coordinate computing units have calculated respectively 9Individual in different points of view position with empty virtual visual point image data, and in empty tag cacheModule marks has gone out the some position, cavity in virtual visual point image.
Cavity tag cache module by two degree of depth be 640 (number of pixel per line in virtual visual point image),Bit wide is the two-port RAM of 1, i.e. RAM11 and RAM12 composition, for storing non-empty mark.In embodiments of the present invention, empty point coordinates place mark 1, non-empty some place mark 0.
Empty viewpoint cache module by two degree of depth be 640, bit wide be 24 (with original graph color depthTwo-port RAM unanimously), i.e. RAM21 and RAM22 composition, for the virtual view after Storage MappingView data. Empty viewpoint cache module adopts two RAM composition double-damping structures to realize ping-pong operation,To realize the flowing water access of data.
Hole-filling module comprises that unit is searched in cavity and unit is repaired in cavity. Cavity is searched unit and is passed throughRead RAM11 and the RAM12 of empty tag cache module, by from binocular camera shooting, collecting to a left sideIn right view the coordinate figure of non-empty pixel export to cavity repair unit, wherein signal Laddr be fromThe coordinate figure of the non-empty pixel in the left figure of binocular image that binocular camera shooting, collecting arrives, signalRaddr be from binocular camera shooting, collecting to the right figure of binocular image the coordinate of non-empty pixelValue. Cavity is repaired unit and is carried out hole-filling according to the coordinate figure of the left and right Tu Fei cavity pixel transmitting,And transmission is used to indicate signal hf_over1～hf_over9 that virtual visual point image cavity has been filled(the hole-filling module of DIBR_V1 sends hf_over1, and the hole-filling module of DIBR_V2 sendsHf_over2, by that analogy) to the pixel counter of stereo-picture synthesis module.
In embodiments of the present invention, viewpoint mapping block is with behavior unit's generating virtual visual point image data,Data line in every generating virtual visual point image, deposits the pixel value of the pixel correctly shining upon in void and looksIn point cache module, non-empty mark is deposited in empty tag cache module simultaneously. List is searched in cavityUnit is by reading the non-empty mark in empty tag cache module, by Tu Fei cavity, hole region left and rightThe coordinate figure of pixel is exported to cavity and is repaired unit, and cavity is repaired unit and carried out sky according to left and right coordinate figureHole is filled up. Data after hole-filling completes output in cache module stores, for generating virtualVisual point image.
Figure 3 shows that the theory diagram of the two buffer units of dual-port of the present invention. As shown in Figure 3, storagePart adopts two 24 two-port RAMs compositions, i.e. RAM0 and RAM1,4 of RAM0 and RAM1Individual FPDP is respectively IN0, OUT1, IN1, OUT0, and the number of memory cells of each RAM is 640.Ce signal is enable signal, and the data of DIBR processing unit output are through being subject to the selection of ce signal controllingDevice muxi can enter into the IN0 port of RAM0 or the IN1 port of RAM1; Meanwhile, be subject to ceThe selector muxo of signal controlling can by from the current data that receiving another RAM that data are differentOutput. When depositing data in to an input of a RAM, in another RAMThe data that deposit into can, as output, substitute and form ping-pong operation. In_addr is data input pinAddress wire, be connected to the address wire of input one side of RAM0, RAM1; Out_addr is that data are defeatedGo out the address wire of end, be connected to the address wire of output one side of RAM0, RAM1.
Figure 4 shows that the connection diagram of cache module of the present invention and stereo-picture synthesis module. Three-dimensionalImage synthesis unit comprises 1 pixel counter and multiple stereo-picture generation unit. Real in the present inventionExecute in example, stereo-picture synthesis module comprises 3 stereo-picture generation units, is greater than 3 stereogramsCan produce better stereoeffect as generation unit and still calculate too complexity, reduce real-time; FewIn the fast but stereoeffect of 3 stereo-picture generation unit computational speeds a little less than. Wherein each stereogramComprise synthetic controller, multiple data switch, MUX and output buffer memory as generation unit.
In embodiments of the present invention, the clock of pixel counter is in system clock or vision signalClock. After pixel counter receives these 9 signals of hf_over1～hf_over9, its countingPiex_num is in each clock cycle increase by 1 and transfer to synthetic controller, wherein counts piex_numRepresent the position ordinal number of current sub-pixel that will be synthetic. Synthetic controller is by the counting of pixel counterPiex_num is input to as address the synthetic table that synthetic controller comprises and inquires about.
Synthetic table is that 640 × 360 (in the same size with original graph and disparity map), width are by a degree of depthThe ROM of 9 (consistent with the number of virtual visual point image) position forms, and its content is by formulaDetermine, wherein x represents sub-pixel abscissa; Y represents the vertical seat of sub-pixelMark; A represents column mirror grating inclination angle; A represents the sub-pixel numbers that column mirror grating strides across; N represents voidIntend visual point image number. To a sub-picture from left to right, count from top to bottom each sub-pixel horizontal strokeOrdinate can form a pixel sequence number. In the time that pixel sequence number is inputted as address, to synthetic tableContent is searched, output view index ordinal number, and it represents that in stereo-picture, this pixel sequence number is correspondingSub-pixel is taken from the sequence number of which virtual visual point image.
View index ordinal number (scope is 1～9 in embodiments of the present invention) is by the line of fourBe input to MUX, MUX according to 9 enable signal sel1 of view index ordinal number output～Sel9, a data switch of the corresponding control of each enable signal.
Data switch 1～9 only has at most one to be enabled at synchronization, and the data switch being enabled is permittedPermitted the data of its input from the output of output line, and the output of the data switch that all the other do not enable presents heightResistance state. The counting piex_num of pixel counter is sent to each both-end in cache module by synthetic controllerThe address wire Out_addr of output port one side of the two buffer units of mouth is upper, in the two buffer units of dual-portAddress is that the data of the memory cell of piex_num can export data switch to. According to MUXGating signal is opened corresponding data switch, and these data switch output virtual visual point image data are to outputBuffer memory.
3 stereo-picture generation units interlock and synthesize three row data respectively, concurrent carrying out. Three-dimensionalWhen image generation unit 0 interlocks to the capable pixel data of y, stereo-picture generation unit 1 placeManage y+1 capable, it is capable that stereo-picture generation unit 2 is processed y+2. The image slices vegetarian refreshments of synthetic controllerFrequency is 3 times of pixel counter clock frequency, and the counting piex_num of each pixel counter holdsIn the continuous time, complete 3 pixels and staggered synthetic. When last 3 line interlacing has synthesized,Can obtain 1920 × 1080 pixel datas.
Output buffer memory is that a degree of depth is that 1920 × 1080 (stereogram sizes), width are 24 (solidsFigure color depth figure place, consistent with original graph) ROM of position, receive from data switch 1～9Virtual visual point image data. After data are write completely, be the data flow of the stereo-picture after synthesizing, pressAccording to the output of sequential array packages, can obtain real-time stereo-picture and show.
Those skilled in the art will readily understand, the foregoing is only preferred embodiment of the present invention,Not in order to limit the present invention, all any amendments of doing within the spirit and principles in the present invention, etc.With replacement and improvement etc., within all should being included in protection scope of the present invention.
Claims (10)
1. a real-time stereo images generation system, comprises many viewpoints generation module, cache module and verticalVolume image synthesis module, is characterized in that:
Described many viewpoints generation module comprise N parallel draw processing unit based on depth image, itsMiddle N=x2, x is greater than 1 positive integer, draws processing unit according to former described in each based on depth imageBeginning figure and disparity map generating virtual visual point image are also sent in described cache module, send instruction simultaneouslyThe signal that described drawing virtual view image completes is to described stereo-picture synthesis module;
Described cache module comprises N the two buffer units of parallel dual-port, receives respectively and store instituteState N the described virtual visual point image that many viewpoints generation module generates, wherein the two buffer memorys of each dual-portUnit comprises that two degree of depth are that the color of described virtual visual point image width, bit wide and described original graph is darkSpend consistent two-port RAM, described two two-port RAMs composition double-damping structure is realized ping-pong operation,To realize the stream treatment to described virtual visual point image; And
Described stereo-picture synthesis module comprises pixel counter and stereo-picture generation unit, described pictureElement counter receives after the signal that described instruction drawing virtual view image completes, count asLocation is input to described cache module, and described stereo-picture generation unit reads ground from described cache moduleThe virtual visual point image data that the memory cell that location is described counting is stored, by described N virtual lookingDot image synthesizes 1 stereo-picture and exports.
2. real-time stereo images generation system as claimed in claim 1, is characterized in that, described everyIndividual based on depth image draw processing unit comprise:
Viewpoint mapping block, comprises that coordinate computing unit, zbuf buffer unit and parallax search unit,According to each pixel, the gray value in described disparity map calculates described each picture to described coordinate computing unitThe coordinate figure of vegetarian refreshments correspondence in described virtual visual point image, completes line by line from reference view to virtualThe mapping of viewpoint; Described zbuf buffer unit is described virtual visual point image width by 2 degree of depth, positionWide is the single port RAM composition of described disparity map gray scale figure place, for storing zbuffer arithmetic result;Described parallax is searched unit by the capacity ROM structure consistent with the pixel grey scale progression of described disparity mapBecome, for completing the calculating from parallax value to parallax function;
Cavity tag cache module is that described virtual visual point image width, bit wide are 1 by two degree of depthThe two-port RAM composition of position, for storing non-empty mark;
Empty viewpoint cache module, by two degree of depth be described virtual visual point image width, bit wide and described inThe two-port RAM composition that original graph color depth is consistent, completes for storing described viewpoint mapping blockDescribed virtual visual point image data after mapping, wherein, described two two-port RAMs composition double bufferingStructure realizes ping-pong operation; And
Hole-filling module, comprises that unit is repaired in cavity and unit is searched in cavity, and list is searched in described cavityUnit reads the described non-empty mark of described empty tag cache module stores, and by described original graphThe coordinate figure of non-empty pixel is exported to described cavity and is repaired unit, and unit is repaired according to institute in described cavityState coordinate figure and carry out hole-filling.
3. real-time stereo images generation system as claimed in claim 2, is characterized in that, described seatDescribed in mark computing unit calculates according to the simplification equivalence formula obtaining based on depth image rendering algorithmVirtual visual point image, described simplification equivalence formula is:
VIR_n(u0+DEEP(DIS(u0,v0))×n,v0)＝ORI(u0,v0)
Wherein, VIR_n represents the picture element matrix of described virtual visual point image, described in ORI (u0, v0) representsThe picture element matrix of original graph; Described original graph n=0, m virtual view figure of described original graph adjacent left-handPicture n=-1～-m, the adjacent right side y of described original graph virtual visual point image n=1～m, m is integer,M=in the time that N is odd number (N-1)/2, m=N/2 in the time that N is even number; U0, v0 represent respectively described formerThe transverse and longitudinal coordinate of pixel in beginning figure; The parallax value of DIS (u0, v0) denotation coordination point (u0, v0) point,The depth value of DEEP (DIS (u0, v0)) denotation coordination point (u0, v0) point.
4. real-time stereo images generation system as claimed in claim 2 or claim 3, is characterized in that instituteStating zbuffer algorithm is to calculate from described disparity map pixel according to described coordinate computing unitCoordinate result, reads and judges that address in described zbuf buffer unit is that the unit of described coordinate result isThe no parallax record that existed, if existed, by the parallax in described zbuf buffer unit and currentCalculate the parallax adopting and compare, using pixel place coordinate less parallax as described virtualThe source points coordinate of certain pixel described in visual point image, and address is in described zbuf buffer unitIn the unit of described coordinate result, record the parallax value that calculating adopts.
5. real-time stereo images generation system as claimed in claim 2 or claim 3, is characterized in that instituteState viewpoint mapping block and generate described virtual visual point image data with behavior unit, every generation is described virtualData line in visual point image, deposits the pixel value of the each pixel correctly shining upon in described empty viewpoint slowIn storing module, described non-empty mark is deposited in described empty tag cache module simultaneously.
6. real-time stereo images generation system as claimed in claim 1, is characterized in that, described verticalVolume image synthesis module comprises:
Pixel counter, receives after the signal that N described instruction drawing virtual view image completes,Its counting is in each clock cycle increase by 1 and be sent to stereo-picture generation unit, wherein said countingRepresent the current position ordinal number of wanting synthon pixel; And
Multiple described stereo-picture generation units, each described stereo-picture generation unit comprises synthetic controlDevice processed, MUX, a N data switch and output buffer memory, described synthetic controller is by described pixelThe described counting input synthetic table that counter sends is inquired about, and described synthetic controller is also by described meterNumber is input to described cache module as address, and described synthetic table is according to described counting output view indexSequence number is to described MUX, and described MUX sends enable signal to a described N data and opensOne corresponding with described view index sequence number in the Central Shanxi Plain, the described data switch being enabled will be from described slowThe described virtual visual point image data that the memory cell that in storing module, reading address is described counting is storedBe sent to described output buffer memory, the data of described output buffer memory are exported according to sequential array packages after writing completely,Obtaining real-time described stereo-picture exports.
7. real-time stereo images generation system as claimed in claim 6, is characterized in that, described in closeBecome controller to send to the two buffer memory lists of each dual-port of described cache module using described counting as addressIn the address wire of output port one side of unit.
8. the real-time stereo images generation system as described in claim 6 or 7, is characterized in that, instituteStating synthetic table is that described virtual visual point image width × highly, width are the ROM of N position by a degree of depthForm, its content is by formulaDetermine, wherein x represents sub-pixel abscissa;Y represents described sub-pixel ordinate; A represents column mirror grating inclination angle; A represent described column mirror grating acrossThe sub-pixel numbers of crossing.
9. the real-time stereo images generation system as described in claim 6 or 7, is characterized in that, manyIndividual described stereo-picture generation unit is parallel respectively to interlock and synthesizes three row data, the first stereogramWhen the capable pixel data of y being interlocked as generation unit, the second stereo-picture generation unit processingY+1 is capable, and it is capable that the 3rd stereo-picture generation unit is processed y+2, by that analogy.
10. the real-time stereo images generation system as described in claim 6 or 7, is characterized in that,The image pixel dot frequency of described synthetic controller is k times of described pixel counter clock frequency, and k isThe number of described stereo-picture generation unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510967607.1A CN105611271A (en) | 2015-12-18 | 2015-12-18 | Real-time stereo image generating system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510967607.1A CN105611271A (en) | 2015-12-18 | 2015-12-18 | Real-time stereo image generating system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105611271A true CN105611271A (en) | 2016-05-25 |
Family
ID=55990773
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510967607.1A Pending CN105611271A (en) | 2015-12-18 | 2015-12-18 | Real-time stereo image generating system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105611271A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105872530A (en) * | 2016-05-31 | 2016-08-17 | 上海易维视科技股份有限公司 | Multi-view three-dimensional image synthesis method |
| CN105959676A (en) * | 2016-05-31 | 2016-09-21 | 上海易维视科技股份有限公司 | Naked-eye 3D display system capable of carrying out lateral and vertical display |
| CN106060522A (en) * | 2016-06-29 | 2016-10-26 | 努比亚技术有限公司 | Video image processing device and method |
| CN106228530A (en) * | 2016-06-12 | 2016-12-14 | 深圳超多维光电子有限公司 | A kind of stereography method, device and stereophotography equipment |
| CN106454320A (en) * | 2016-11-01 | 2017-02-22 | 珠海明医医疗科技有限公司 | Method and equipment for generating three-dimensional (3D) image at low delay |
| CN108986062A (en) * | 2018-07-23 | 2018-12-11 | Oppo（重庆）智能科技有限公司 | Image processing method and device, electronic device, storage medium and computer equipment |
| CN109739651A (en) * | 2019-01-08 | 2019-05-10 | 中国科学技术大学 | A kind of Stereo matching hardware structure of low consumption of resources |
| CN110033426A (en) * | 2018-01-12 | 2019-07-19 | 杭州海康威视数字技术股份有限公司 | A kind of device for being handled disparity estimation image |
| CN111325693A (en) * | 2020-02-24 | 2020-06-23 | 西安交通大学 | Large-scale panoramic viewpoint synthesis method based on single-viewpoint RGB-D image |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729919A (en) * | 2009-10-30 | 2010-06-09 | 无锡景象数字技术有限公司 | System for full-automatically converting planar video into stereoscopic video based on FPGA |
| CN102316354A (en) * | 2011-09-22 | 2012-01-11 | 冠捷显示科技（厦门）有限公司 | Parallelly processable multi-view image synthesis method in imaging technology |
| CN102572482A (en) * | 2012-01-06 | 2012-07-11 | 浙江大学 | 3D (three-dimensional) reconstruction method for stereo/multi-view videos based on FPGA (field programmable gata array) |
| CN102625127A (en) * | 2012-03-24 | 2012-08-01 | 山东大学 | Optimization method suitable for virtual viewpoint generation of 3D television |
| CN102930593A (en) * | 2012-09-28 | 2013-02-13 | 上海大学 | Real-time rendering method based on GPU (Graphics Processing Unit) in binocular system |
| CN104079913A (en) * | 2014-06-24 | 2014-10-01 | 重庆卓美华视光电有限公司 | Sub-pixel arrangement method and device for compatibility of raster stereoscopic displayer with 2D and 3D display modes |
| CN104506872A (en) * | 2014-11-26 | 2015-04-08 | 深圳凯澳斯科技有限公司 | Method and device for converting planar video into stereoscopic video |
| CN104796685A (en) * | 2015-03-31 | 2015-07-22 | 王子强 | Universal free three-dimensional display system |
| CN104822059A (en) * | 2015-04-23 | 2015-08-05 | 东南大学 | Virtual viewpoint synthesis method based on GPU acceleration |
| CN104980733A (en) * | 2015-06-18 | 2015-10-14 | 中央民族大学 | Glasses-free 3D display crosstalk test method and test image thereof |
| CN105049826A (en) * | 2015-07-23 | 2015-11-11 | 南京大学 | FPGA-based real-time stereoscopic video fusion conversion method |
-
2015
- 2015-12-18 CN CN201510967607.1A patent/CN105611271A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729919A (en) * | 2009-10-30 | 2010-06-09 | 无锡景象数字技术有限公司 | System for full-automatically converting planar video into stereoscopic video based on FPGA |
| CN102316354A (en) * | 2011-09-22 | 2012-01-11 | 冠捷显示科技（厦门）有限公司 | Parallelly processable multi-view image synthesis method in imaging technology |
| CN102572482A (en) * | 2012-01-06 | 2012-07-11 | 浙江大学 | 3D (three-dimensional) reconstruction method for stereo/multi-view videos based on FPGA (field programmable gata array) |
| CN102625127A (en) * | 2012-03-24 | 2012-08-01 | 山东大学 | Optimization method suitable for virtual viewpoint generation of 3D television |
| CN102930593A (en) * | 2012-09-28 | 2013-02-13 | 上海大学 | Real-time rendering method based on GPU (Graphics Processing Unit) in binocular system |
| CN104079913A (en) * | 2014-06-24 | 2014-10-01 | 重庆卓美华视光电有限公司 | Sub-pixel arrangement method and device for compatibility of raster stereoscopic displayer with 2D and 3D display modes |
| CN104506872A (en) * | 2014-11-26 | 2015-04-08 | 深圳凯澳斯科技有限公司 | Method and device for converting planar video into stereoscopic video |
| CN104796685A (en) * | 2015-03-31 | 2015-07-22 | 王子强 | Universal free three-dimensional display system |
| CN104822059A (en) * | 2015-04-23 | 2015-08-05 | 东南大学 | Virtual viewpoint synthesis method based on GPU acceleration |
| CN104980733A (en) * | 2015-06-18 | 2015-10-14 | 中央民族大学 | Glasses-free 3D display crosstalk test method and test image thereof |
| CN105049826A (en) * | 2015-07-23 | 2015-11-11 | 南京大学 | FPGA-based real-time stereoscopic video fusion conversion method |
Non-Patent Citations (1)
| Title |
|---|
| 郭浩: "基于FPGA的实时2D转3D系统研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105872530A (en) * | 2016-05-31 | 2016-08-17 | 上海易维视科技股份有限公司 | Multi-view three-dimensional image synthesis method |
| CN105959676A (en) * | 2016-05-31 | 2016-09-21 | 上海易维视科技股份有限公司 | Naked-eye 3D display system capable of carrying out lateral and vertical display |
| CN106228530A (en) * | 2016-06-12 | 2016-12-14 | 深圳超多维光电子有限公司 | A kind of stereography method, device and stereophotography equipment |
| CN106060522A (en) * | 2016-06-29 | 2016-10-26 | 努比亚技术有限公司 | Video image processing device and method |
| CN106454320A (en) * | 2016-11-01 | 2017-02-22 | 珠海明医医疗科技有限公司 | Method and equipment for generating three-dimensional (3D) image at low delay |
| CN110033426A (en) * | 2018-01-12 | 2019-07-19 | 杭州海康威视数字技术股份有限公司 | A kind of device for being handled disparity estimation image |
| CN110033426B (en) * | 2018-01-12 | 2021-07-09 | 杭州海康威视数字技术股份有限公司 | Device for processing parallax estimation image |
| CN108986062A (en) * | 2018-07-23 | 2018-12-11 | Oppo（重庆）智能科技有限公司 | Image processing method and device, electronic device, storage medium and computer equipment |
| CN109739651A (en) * | 2019-01-08 | 2019-05-10 | 中国科学技术大学 | A kind of Stereo matching hardware structure of low consumption of resources |
| CN109739651B (en) * | 2019-01-08 | 2023-06-16 | 中国科学技术大学 | Three-dimensional matching hardware architecture with low resource consumption |
| CN111325693A (en) * | 2020-02-24 | 2020-06-23 | 西安交通大学 | Large-scale panoramic viewpoint synthesis method based on single-viewpoint RGB-D image |
| CN111325693B (en) * | 2020-02-24 | 2022-07-12 | 西安交通大学 | Large-scale panoramic viewpoint synthesis method based on single viewpoint RGB-D image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105611271A (en) | Real-time stereo image generating system | |
| JP6563453B2 (en) | Generation of a depth map for an input image using an exemplary approximate depth map associated with an exemplary similar image | |
| Schmidt et al. | Multiviewpoint autostereoscopic dispays from 4D-Vision GmbH | |
| CN103813153B (en) | A kind of bore hole 3D multi-view image synthetic method based on weighted sum | |
| US20160156898A1 (en) | Parallel scaling engine for multi-view 3DTV display and method thereof | |
| US20140111627A1 (en) | Multi-viewpoint image generation device and multi-viewpoint image generation method | |
| MX2024004977A (en) | Multifocal plane based method to produce stereoscopic viewpoints in a dibr system (mfp-dibr). | |
| US20060171028A1 (en) | Device and method for display capable of stereoscopic vision | |
| US9154765B2 (en) | Image processing device and method, and stereoscopic image display device | |
| CN102572482A (en) | 3D (three-dimensional) reconstruction method for stereo/multi-view videos based on FPGA (field programmable gata array) | |
| CN101547376A (en) | Multi parallax image generation apparatus and method | |
| CN105611270B (en) | A kind of binocular vision auto-stereo display system | |
| JP6585938B2 (en) | Stereoscopic image depth conversion apparatus and program thereof | |
| CN103945205B (en) | The video process apparatus that compatible 2D and multiple views bore hole 3D shows and method | |
| CN104506872A (en) | Method and device for converting planar video into stereoscopic video | |
| Chen et al. | Real-time multi-view rendering architecture for autostereoscopic displays | |
| US10122987B2 (en) | 3D system including additional 2D to 3D conversion | |
| Park et al. | Viewpoint vector rendering for efficient elemental image generation | |
| US8976171B2 (en) | Depth estimation data generating apparatus, depth estimation data generating method, and depth estimation data generating program, and pseudo three-dimensional image generating apparatus, pseudo three-dimensional image generating method, and pseudo three-dimensional image generating program | |
| CN103945206A (en) | Three-dimensional picture synthesis system based on comparison between similar frames | |
| CN103260041A (en) | Multi-viewpoint automatic light-splitting three-dimensional display device of scene | |
| US20120163700A1 (en) | Image processing device and image processing method | |
| CN110149508A (en) | A kind of array of figure generation and complementing method based on one-dimensional integrated imaging system | |
| CN101908233A (en) | Method and system for producing plural viewpoint picture for three-dimensional image reconstruction | |
| Hyun et al. | Curved Projection Integral Imaging Using an Additional Large‐Aperture Convex Lens for Viewing Angle Improvement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160525 |