CN109151441A - The image treatment method and device of omni-directional video - Google Patents
The image treatment method and device of omni-directional video Download PDFInfo
- Publication number
- CN109151441A CN109151441A CN201810618462.8A CN201810618462A CN109151441A CN 109151441 A CN109151441 A CN 109151441A CN 201810618462 A CN201810618462 A CN 201810618462A CN 109151441 A CN109151441 A CN 109151441A
- Authority
- CN
- China
- Prior art keywords
- image
- mentioned
- mounted display
- side head
- head
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/339—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using spatial multiplexing
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
Abstract
The image processor and method of this transmission offer omni-directional video.The image treatment method of the disclosure includes: the step of obtaining input image from omnidirectional camera；The step of acquired above-mentioned input image is configured to bidimensional image array by the structure based on above-mentioned omnidirectional camera；The step of image output corresponding with the movement of user is determined in constructed above-mentioned bidimensional image array；And the step of above-mentioned image output determined by providing a user.
Description
Technical field
This disclosure relates to the image processor and method of omni-directional video.More specifically, this disclosure relates to provide anti-
Mirror the device and method of the rotation of viewer and the 360 degrees omnidirection video of moving movement.
Background technique
Virtual reality (virtual reality) service can be with real figure or computer graphical (CG, Computer
Graphics) form generates 360 degrees omnidirection video (or omni-directional video) to show in the wear-type as personal terminal
Device (HMD, Head Mounted Display) plays, it just develops towards the direction of can make to get drunk sense and presence maximization.
360 degree of videos (or omni-directional video) can make it required for the fixed view beyond two-dimensional video when user watches video
The video of the viewpoint viewing video in direction.
Fig. 1 is the camera structure and playing video format for illustrating 360 degree of videos for obtaining one embodiment of the disclosure
Figure.Different from common camera, the omnidirectional camera for obtaining 360 degree of videos is configured that can be in order to obtain from full side
Position subject reflection light and towards comprehensive on the basis of the central point of camera 110.From composition omnidirectional camera
Each camera obtain multiple images can be played in head-mounted display by image joint (stitching) step.Structure
Wait images that can indicate the shadow obtained from each camera for constituting omnidirectional camera at C1, C2, C3 ... of splicing image 120
Picture.
When watching the image obtained by 360 degree of previous cameras by head-mounted display, viewer's is solidifying
Apparent direction is consistent with the direction that multiple cameras are looked at straight, and only just may be viewed by the case where viewer is rotated by 360 ° corresponding
The image in direction.Also, since the direction of viewer is consistent with the direction of camera, thus when viewer to the left and right or up and down
When direction is mobile, the image that viewer sees does not change.
On the other hand, in display true to nature, one kind is needed with natural image in order to provide sense of getting drunk to viewer
According to the movement of user or the method for the position change parallax (parallax) and focus of eyes main view.Fig. 2 is for illustrating this
The figure of the six degree of freedom (DoF, Degrees of Freedom) of the head-mounted display of one embodiment is disclosed.To wear wear-type
Centered on the viewer of display, the first movement 210 can indicate bilateral reciprocation (the move side to of trunnion axis
Side), the second movement 220 can indicate that the rotary motion (look up and down) of trunnion axis, third movement 230 can indicate
Lower axle pumps (move up and down), and the 4th movement 240 can indicate the rotary motion (look of upper and lower axle
Side to side), the 5th movement 250 can indicate the front-rear reciprocation movement (move front to back) of antero posterior axis, and
6th movement 260 can indicate the rotary motion (tilt head side to side) of antero posterior axis.Under normal circumstances, in order to provide
The value of the parallax of wide scope, view (view) axis direction in Fig. 2 has to greatly.Also, it in order to continuously change focus, needs to regard
Point is spaced narrow image, needs a large amount of data volume or a large amount of operand thus, thus needs a kind of for having in display
The method that effect ground reproduces viewpoint image.
On the other hand, in order to play nature and 360 degree of videos for feeling high of getting drunk in head-mounted display, need to reproduce six certainly
By spending.That is, it needs to be played and the moving left and right, be rotated up and down, moving up and down of viewer, left and right by head-mounted display picture
The image that the movement in 6 directions such as rotation is correspondingly stared.Up to the present, it is shown for playing the wear-type of real figure image
Device be rotated up and down and rotate left and right by detecting based on movement come playing video, and moved left and right providing with viewer
And it moves up and down and related stares in terms of image that there are limitations.
Summary of the invention
The technical issues of disclosure, is, provides the image processor and method of omni-directional video.
Another technical problem of the disclosure is, is provided for reflecting the complete of the rotation of viewer and moving movement
The device and method of the 360 degree of videos in orientation.
Another technical problem of the disclosure is, is provided for the device and method of 360 degree of camera structures, above-mentioned
360 degree of camera structures provide 360 degree of videos of more natural six degree of freedom by head-mounted display.
The also another technical problem of the disclosure is, is provided for the device and method of image format, above-mentioned image
Format provides 360 degree of videos of more natural six degree of freedom by head-mounted display.
According to an embodiment of the disclosure, it is possible to provide image treatment method, above-mentioned image treatment method include: from full side
The step of position camera obtains input image；Structure based on above-mentioned omnidirectional camera is by acquired above-mentioned input image structure
The step of building as bidimensional image array；Determination is corresponding with the movement of user defeated in constructed above-mentioned bidimensional image array
The step of image out；And the step of above-mentioned image output determined by providing a user.
According to another embodiment of the disclosure, it is possible to provide image processor, above-mentioned image processor include: image
Receiving unit, for obtaining input image from omnidirectional camera；Control unit, the structure based on above-mentioned omnidirectional camera will be obtained
The above-mentioned input image taken is configured to bidimensional image array, and determining with user's in constructed above-mentioned bidimensional image array
Act corresponding image output；And output section, for providing a user identified above-mentioned image output.
It is only the exemplary embodiment party of the detailed description of following disclosure above in relation to the feature that the disclosure is briefly summarized
Formula, and not limit the scope of the present disclosure.
According to the disclosure, it is possible to provide the image processor and method of omni-directional video.
Also, according to the disclosure, it is possible to provide for providing the up and down/left rotation and right rotation for reflecting viewer and up and down/left and right
The device and method of the 360 degrees omnidirection video of moving movement.
Also, according to the disclosure, it is possible to provide for providing the device and method of 360 degree of camera structures, above-mentioned 360 degree are taken the photograph
As header structure provides 360 degree of videos of more natural six degree of freedom by head-mounted display.
Also, according to the disclosure, it is possible to provide for providing the device and method of image format, above-mentioned image format passes through head
Head mounted displays provide 360 degree of videos of more natural six degree of freedom.
Obtainable effect is not limited to effect mentioned above in the disclosure, the technical field of the invention it is general
Logical technical staff can be expressly understood that other silent effects from following record.
Detailed description of the invention
Fig. 1 is the figure for illustrating camera structure and playing video format for obtaining 360 degree of videos.
Fig. 2 is the figure for illustrating the six degree of freedom of the head-mounted display of one embodiment of the disclosure.
Fig. 3 is the figure for illustrating 360 degree of camera structures of the support six degree of freedom of one embodiment of the disclosure.
Fig. 4 is the image format for illustrating the generation support six degree of freedom of one embodiment of the disclosure, and aobvious in wear-type
Show that device plays the figure of the process of image format generated.
Fig. 5 is for illustrating that providing for one embodiment of the disclosure reflects the head-mounted display of the rotary motion of viewer
The figure of the process of image.
Fig. 6 is for illustrating that providing for one embodiment of the disclosure reflects the head-mounted display of the moving movement of viewer
The figure of the process of image.
Specific embodiment
The implementable numerous variations of the present invention, can have various embodiments, will illustrate specific embodiment in the accompanying drawings and carry out
It is described in detail.But this not indicates to limit the invention to particular implementation, and it includes of the invention for should be understood as
Having altered in thought and technical scope, equivalent technical solutions and replace technical solution.In the accompanying drawings, similar appended drawing reference
Indicate many-sided same or similar function.In order to clearly state, the form and dimension of multiple elements in attached drawing may be overstated
Greatly.To the described further below referring to the attached drawing shown using specific embodiment as illustrating of exemplary embodiment.Sufficiently in detail
Ground illustrates that those skilled in the art can be implemented in these embodiments.It should be appreciated that various embodiments mutually not
It is identical but without mutually exclusive.For example, in the case where not departing from thought and range of the invention, can lead to about an embodiment
It crosses other embodiments and realizes documented specific shape, structure and characteristic in this specification.And it is to be understood that not departing from reality
In the case where the thought and range of applying example, thus it is possible to vary the position of each structural element in each disclosed embodiment is matched
It sets.Therefore, aftermentioned detailed description is not be considered in a limiting sense, as long as explanation is proper, the range of exemplary embodiment
It is limited together with all ranges equivalent with the opinion of its claim by claim.
In the present invention, can be during illustrating various structures element using the first, second equal terms, but above-mentioned knot
Structure element should not be limited by above-mentioned term.Above-mentioned term is only with the purpose of one structural element of differentiation from other structures element
To use.For example, first structure element can be named as the second structure and want in the case where not departing from interest field of the invention
Element, similarly, the second structural element can also be named as first structure element.Term "and/or" includes multiple related items recorded
Some project in purpose combination or multiple related projects recorded.
When referring to a structural element with other structures element " being connected " or " being in contact ", it should be understood that can be direct
It is connected or is in contact with other structures element, can includes also other structures element in centre.On the contrary, when referring to that a structure is wanted
When element is with other structures element " being directly connected to " or " directly contacting ", this should be understood as that there is no other structures elements for centre.
The multiple component units occurred in the embodiment of the present invention are independently shown to indicate different feature functionalities, are not offered as
Each component units are formed by isolated hardware or a software configuration unit.That is, each component units be for ease of description and
It is enumerated in the form of each component units and includes, and at least two component units of each component units can be combined into one
A component units or a component units can be divided into multiple component units to execute function.Unless they deviate this hair
Bright essence, otherwise the combined embodiment and isolated embodiment of this each component units are also included within right of the invention
In range.
Term as used in the present invention is merely to illustrate specific embodiment, and is not intended to limit the present invention.As long as
Other meanings are not explicitly indicated that on context, singular expression just may include the expression of plural number.In the present invention " comprising " or
The term of " having " etc. is interpreted as specifying feature recorded in specification, number, step, movement, structural element, portion
The presence of part or their combination, and other one or more features, number, step, movement, knot are not excluded in advance
The presence or additional possibility of structure element, component or their combination.That is, being described as " comprising " specific structure in the present invention
The structure except corresponding construction is not precluded in content, but indicates that additional structure may include in implementation or the present invention of the invention
Technical idea in the range of.
Some structural elements of the invention do not execute the requisite item of basic function not instead of in the present invention, can be only
For proposing high performance optional element.In addition to for other than mentioning high performance structural element, the present invention can only to include realizing this
Component units necessary to the essence of invention realize, and other than the optional component units for being used simply to improve performance, only
Structure including necessary structural element also is included in interest field of the invention.
Hereinafter, illustrating embodiments of the present invention referring to attached drawing.In the process for the embodiment for illustrating this specification
In, when judge to related known structure or function illustrate be possible to obscure purport of the invention when, will omit to it
Detailed description, for the identical structural element in attached drawing use identical appended drawing reference, and omit to identical structural element
Repeated explanation.
Fig. 3 is the figure for illustrating 360 degree of camera structures of the support six degree of freedom of one embodiment of the disclosure.
In 360 degree of camera structures (or omnidirectional camera structure) of the support six degree of freedom of the disclosure, camera
The radius in face (camera plane) 310 can be greater than the radius of previous camera shooting head, can obtain to the left and right or upper and lower
Parallax (disparity) when to movement (translation).Also, camera shooting head 310 can have polygon (for example, five sides
Shape, hexagon, octagon etc.) structure.The multiple cameras for being arranged in each side of camera shooting head 310 can have to be taken the photograph with multiple views
As head (for example, trunnion axis or two-dimensional camera array (2D camera array)) similar structure, also, work as using above-mentioned more
A camera can obtain motion parallax (motion parallax) information in three-dimensional space photographs subject.Also,
Multiple cameras of same edge in the multiple sides for constituting camera shooting head 310 are with parallel axes (parallel vidicon (parallel
Camera)) or intersecting axle (prenex, convergence video camera (toe-in, convergence camera)) mode configures, so as to
Adjust (+) or (-) sense of depth.For example, referring to Fig. 3, positioned at a side 312 of the camera shooting head 310 as eight-sided formation
A1, A2, A3 camera can be in intersect to configure in the form of (cross) by the direction of optical axis respectively, be located at other multiple sides
Multiple cameras also can intersecting form it is similarly configured.Also, A1, A2, A3, B1, B2 ... wait multiple camera shootings
Head can be respectively relative to perspective plane 302 with visual angle corresponding with each camera (FoV, Field of View).
According to the disclosure, the rotary motion (rotation) of viewer can be by being respectively selected from composition camera shooting head 310
The combination of camera on multiple sides (A, B, C ...) show.For example, referring to Fig. 3, the omni-directional video of the disclosure is handled
Device can by from selected from constitute the A of camera shooting head 310 while A1, selected from B while the multiple of B1, C1 selected from the side C etc. take the photograph
As head obtain multiple images spliced, the image for reflecting the rotary motion 320 of viewer is provided.Also, viewer's
Moving movement (transition) can be by being located at multiple cameras when multiple one in for constituting camera shooting head 310
Movement is to show.For example, referring to Fig. 3, the omni-directional video processing unit of the disclosure can be by imaging head 310 to from composition
Multiple B1, B2 when being located at B in, B3 ... the multiple images for waiting multiple cameras to obtain are spliced, and are reflected to provide
The image of the moving movement 330 of viewer.
On the other hand, during image mosaic, the boundary part between splicing image may lose because of parallax
Very, the omni-directional video processing unit of the disclosure, which can be utilized, corrects above-mentioned distortion in same edge adjacent camera.For example, ginseng
According to Fig. 3, in the case where viewer is rotated, when splicing multiple images of A1, B1, C1 camera, in order to correct
The distortion occurred by the parallax of boundary part, using A3, B3, C3 camera image information.
Fig. 4 is the image format for illustrating the generation support six degree of freedom of one embodiment of the disclosure, and aobvious in wear-type
Show that device plays the figure of the process of image format generated.
Image receiving unit 410 can receive from the multiple images for supporting 360 degree of cameras of six degree of freedom to obtain.For example, branch
360 degree of camera structures for holding six degree of freedom can be the camera structure of Fig. 3.
It is two-dimentional battle array by six degree of freedom image reconstruction that control unit 420, which can be utilized by the received image of image receiving unit 410,
The image format of column form.For example, 360 degree of image formats 440 of the six degree of freedom of the disclosure is supported to can be configured such that two-dimensional array
Form, parallel axes 442 and vertical axis 444 can be to obtain to from multiple cameras on the multiple sides for being respectively selected from composition camera shooting head
Image that the multiple images taken are spliced and to from constitute camera shooting head it is multiple one is included in while multiple take the photograph
The image spliced as multiple images that head obtains.
For example, referring to Fig. 3 and Fig. 4, can show in parallel axes 442 to constituting the multiple of camera shooting head 310 from being respectively selected from
Side A, B, C ... A1, B1, C1 ... the image A1-B1-C1- that is spliced of multiple images for waiting cameras to obtain
D1-…….Also, can vertical axis 444 show to from be contained in building camera shooting head 310 multiple A in while A1,
A2, A3 ... the image A1-A2- that the multiple images for waiting multiple cameras to obtain are spliced in a manner of moving along a direction
A3-…….Therefore, the omni-directional video processing unit of the disclosure can by from support six degree of freedom 360 degree cameras acquisition
Multiple image reconstructions are the shadow of the two-dimensional array form constructed by multiple images of 444 the two axis of parallel axes 442 and vertical axis
Picture.
On the other hand, the camera quantity for imaging each side of head can determine number of views, the camera shooting on above-mentioned each side
Head quantity may be restricted because of physical limitation.Therefore, control unit 420 can be by implementing view interpolation (view
Interpolation viewpoint image) is synthesized, the interval between viewpoint and increasing can be shortened by the viewpoint image by synthesis
Add number of views.For example, referring to Fig. 4, the omni-directional video processing unit of the disclosure can splice image A1-B1-C1- first
First view interpolation image 456 is generated between the splicing of D1- ... 450 and second image A2-B2-C2-D2- ... 452.Also,
The omni-directional video processing unit of the disclosure can splice image in the second splicing image A2-B2-C2-D2- ... 452 and third
The second view interpolation image 458 is generated between A3-B3-C3-D3- ... 454.On the other hand, above-mentioned view interpolation method can wrap
Include the common image processing technique for view interpolation, it is contemplated that implementation environment produces an appropriate number of interpolation image.
For the image format from the received two-dimensional array form of control unit 420, output section 430 can be based on the dynamic of viewer
Make to export image corresponding with above-mentioned movement to head-mounted display picture.That is, output section 430 can provide a user left side
(Left) head-mounted display image and right side (Right) head-mounted display image, it is contemplated that binocular parallax, viewer it is dynamic
Make etc., left side head-mounted display image and right side head-mounted display image can be, splice image or view interpolation image.
On the other hand, the movement of viewer may include (upper and lower) rotation of (upper and lower) movement in left and right or left and right.
In above-mentioned multiple embodiments, output section 430 and control unit 420 can carry out in the same way.
Fig. 5 is for illustrating that providing for one embodiment of the disclosure reflects the head-mounted display of the rotary motion of viewer
The figure of the process of image.
The case where the viewer for watching 360 degree of images by head-mounted display controls or is rotated up and down movement
Under, the omni-directional video processing unit of the disclosure can make head-mounted display again in the image of two-dimensional array Morphology Remodeling
Existing image (that is, left side head-mounted display and right side head-mounted display) is along parallel axis direction (that is, stitching image
(stitched image)) it is mobile.Referring to Fig. 5, it is assumed that in the image that the forward direction viewer that viewer is rotated provides
For the first left side head-mounted display image 510 and the first right side head-mounted display image 512, wherein left side wear-type is aobvious
Show that device and right side head-mounted display can distinguish one that head-mounted display to the left and right exports in 360 degree of images of display splicing
Divide image.Wherein, left side (Left) head-mounted display and right side (Right) head-mounted display by binocular parallax for being mentioned
For three-dimensional sense, vertical direction multiple images spaced apart from each other along image format may be selected, with by camera combination come
Image corresponding with the interval for separating human eye interval is exported in the multiple splicing images obtained.First left side head-mounted display shadow
Picture 510 can be the image synthesized by view interpolation, and the first right side head-mounted display image 512 can be to pass through A2-B2-
C2- ... waits cameras combination come the image spliced, and however, it is not limited to this.When the rotary motion 530 of user occurs, this public affairs
The omni-directional video processing unit opened can be by showing the first left side head-mounted display image 510 and the first right side wear-type
Device image 512 is moved respectively in a manner of maintaining specified interval along parallel axis direction to provide the second left side to viewer and wear
Formula display image 520 and the second right side head-mounted display image 522.By providing the above-mentioned image moved horizontally, from sight
For in the person's of seeing position, the effect for the image rotation stared similar to viewer can get.On the other hand, in the complete of the disclosure
In orientation video process apparatus, can relative to the rotary motion of user, can also make the first left side head-mounted display image 510 and
First right side head-mounted display image 512 is to maintain the mobile respectively along vertical direction of specified interval.
Fig. 6 is for illustrating that providing for one embodiment of the disclosure reflects the head-mounted display of the moving movement of viewer
The figure of the process of image.
The case where the viewer for watching 360 degree of images by head-mounted display controls or moves up and down movement
Under, the omni-directional video processing unit of the disclosure can make head-mounted display again in the image of two-dimensional array Morphology Remodeling
Existing image (that is, image of left side head-mounted display and right side head-mounted display) is along vertical axis (that is, view
(views)) mobile.Referring to Fig. 6, it is assumed that on the left of the image that the forward direction viewer that viewer carries out moving movement provides is first
Head-mounted display image 610 and the first right side head-mounted display image 612, then the first left side head-mounted display image
The image that 610, which can be to pass through A1-B1-C1- ... waits cameras combination to splice, the first right side head-mounted display image
612 can be the image synthesized by view interpolation, and however, it is not limited to this.When the moving movement 640 of user occurs, this public affairs
The omni-directional video processing unit opened can be by showing the first left side head-mounted display image 610 and the first right side wear-type
Device image 612 is moved respectively in a manner of maintaining specified interval along vertical axis to provide the second left side to viewer and wear
Formula display image 620 and the second right side head-mounted display image 622.At this point, the second left side head-mounted display image 620
It can be the image synthesized by view interpolation, the second right side head-mounted display image 622 can be to pass through A3-B3-
C3- ... waits the combination of cameras come the image spliced, and however, it is not limited to this.By providing the above-mentioned shadow by vertically moving
Picture, from viewer's position for, can get the image stared similar to viewer along left and right or the effect moved up and down.
On the other hand, if by making the reproduction image of head-mounted display diagonally in two-dimensional array building image
Direction is mobile to be reproduced in head-mounted display, then can provide while reflecting the image of rotation and mobile effect.It is false referring to Fig. 6
Being located at viewer and carrying out the image that the forward direction viewer of moving movement provides is the first left side head-mounted display image 610 and the
One right side head-mounted display image 612, then the first left side head-mounted display image 610 can be to pass through A1-B1-
The image that C1- ... waits the combination of cameras to splice, the first right side head-mounted display image 612 can be slotting by view
It is worth the image of synthesis, however, it is not limited to this.When the moving movement 650 of user occurs, the omni-directional video processing of the disclosure
Device is by making the first left side head-mounted display image 610 and the first right side head-mounted display image 612 respectively to maintain
The mode of specified interval moves to viewer along parallel axes and vertical axis and provides head-mounted display shadow on the left of third
The head-mounted display image 632 on the right side of 630 and third.At this point, head-mounted display image 630 can be to pass through on the left of third
The image of view interpolation synthesis, third right side head-mounted display image 632, which can be to pass through A3-B3-C3- ... waits cameras
The image for combining to splice, however, it is not limited to this.
On the other hand, although the camera and its format that are configured at horizontal plane direction are illustrated, the disclosure
Camera structure and image format are not limited thereto, can also be individually along vertical plane Directional Extension.If with side as described above
Formula extension then can obtain and play comprehensive 360 degree of images that three-dimensional polygon form supports six degree of freedom.
According to the disclosure, it is possible to provide the image processor and method of omni-directional video.
According to the disclosure, it is possible to provide for obtaining and playing the device and method of complete three-dimensional 360 degree of videos.
Also, according to the disclosure, it is possible to provide for providing the device and method of 360 degrees omnidirection video, this 360 degree full sides
Digital video reflects/rotating left and right and upper and lower/up and down for viewer and moves left and right movement.
Also, according to the disclosure, it is possible to provide for providing the device and method of 360 degree of camera structures, this 360 degree camera shootings
Header structure provides 360 degree of videos of more natural six degree of freedom by head-mounted display.
Also, according to the disclosure, it is possible to provide for the device and method of image format, the image format is for passing through head
Head mounted displays provide 360 degree of videos of more natural six degree of freedom.
Also, according to the disclosure, other than previous up and down/left-right rotation, also head-mounted display is being worn
Viewer carry out up and down/move left and right movement in the case where, corresponding image is provided, so as to pass through virtual reality
Device provides complete and natural stereopsis.
In the above-described embodiments, method is described as a series of steps or units based on flow chart, but the present invention is not
It is limited to the sequence of these steps, certain steps can occur with step unlike those described above and sequence or simultaneously.As long as and
General technical staff of the technical field of the invention it will be understood that step described in flow chart be not it is exclusive,
It but may include other steps, or one or more steps in flow chart can be deleted without influencing the scope of the present invention.
Above-described embodiment includes the example of various ways.Although can not describe to possess and indicate all possible of various ways
Combination, but those skilled in the art can be appreciated that other combinations are also possible.Therefore, the present invention may include belonging to
In all substitutions in claims, modifications and changes.
The embodiment of present invention mentioned above can be referred to that can be performed program by various computer configuation elements
Form is enabled to realize, and is recorded in computer-readable recording medium.Above-mentioned computer-readable recording medium can be single
Solely or combination includes program instruction, data file, data structure etc..It is recorded in the journey of above-mentioned computer-readable recording medium
Sequence instruction can be for the instruction of the invention for specially designing and being formed, or can be can be by the unskilled labor of computer software fields
Make the instruction used well known to personnel.As the example of computer-readable recording medium, including hard disk, floppy disk and tape equimagnetic
Magnet-optical mediums (the magneto- such as optical recording media, the soft CD (floptical disk) such as property medium, CD-ROM, DVD
Optical media) and the hardware device for storing and executing program instructions and be specially constructed such as ROM, RAM, flash memory.As
The example of program instruction not only includes carrying out manufactured machine code by compiler, further includes using interpretive program etc. and by meter
Calculation machine is come the higher-level language code that runs.Above-mentioned hardware device may be configured as, can be in order to execute processing of the invention and with one
A above software module carrys out work, and, otherwise it can also be with.
More than, though the present invention is illustrated by the examples and drawings of the specific items such as specific structural element and restriction,
But this is provided to help to be more fully appreciated the present invention, and the present invention is not limited to the above embodiments, as long as
Those skilled in the art can carry out a variety of modification and deformation from above-mentioned record.
Therefore, technical idea of the invention should not be limited to above-described embodiment to provide, be not only claims, with power
Sharp claim is equal or all protection scopes of equivalent variations should belong to thought category of the invention.
Claims (20)
1. a kind of image treatment method characterized by comprising
The step of obtaining input image from omnidirectional camera；
The step of acquired above-mentioned input image is configured to bidimensional image array by the structure based on above-mentioned omnidirectional camera；
The step of image output corresponding with the movement of user is determined in constructed above-mentioned bidimensional image array；And
The step of above-mentioned image output determined by providing a user.
2. image treatment method according to claim 1, which is characterized in that in the structure of above-mentioned omnidirectional camera,
Structure configuration of the multiple single cameras of above-mentioned omnidirectional camera in the form of polygon is constituted, is located in above-mentioned polygon form
Same edge multiple single cameras with parallel axes (parallel vidicon (parallel camera)) or intersecting axle (prenex, Quan Ju
Degree video camera (toe-in, convergence camera)) mode configures.
3. image treatment method according to claim 2, which is characterized in that in above-mentioned bidimensional image array, and obtained
The related multiple splicing images of the above-mentioned input image taken are configured respectively along horizontal direction and vertical direction.
4. image treatment method according to claim 3, which is characterized in that
Splicing image along the configuration of above-mentioned horizontal direction is by distinguishing from multiple sides in above-mentioned polygon form
The image that multiple images that multiple cameras of selection obtain are spliced,
Along above-mentioned vertical direction configuration splicing image be by from be contained in be in above-mentioned polygon form multiple sides in
A side the image that is spliced of multiple images that obtains of multiple cameras.
5. image treatment method according to claim 4, which is characterized in that acquired above-mentioned input image to be configured to
The step of bidimensional image array, further includes the steps that utilizing above-mentioned multiple splicing video generation interpolation images.
6. image treatment method according to claim 5, which is characterized in that
Above-mentioned image output is provided to above-mentioned user by head-mounted display,
Above-mentioned image output includes that the left side head-mounted display image for the binocular parallax for reflecting above-mentioned user and right side are worn
Formula display image,
Above-mentioned left side head-mounted display image and right side head-mounted display image are respectively the one of above-mentioned bidimensional image array
Part splicing image.
7. image treatment method according to claim 6, which is characterized in that above-mentioned user movement be rotate left and right or
In the case where being rotated up and down movement, the step of determining above-mentioned image output includes making above-mentioned left side head-mounted display image and the right side
Side head-mounted display image determines that left side head-mounted display exports shadow respectively along the mobile defined distance of horizontal direction
The step of picture and right side head-mounted display image output.
8. image treatment method according to claim 6, which is characterized in that above-mentioned user movement be move left and right or
In the case where moving up and down movement, the step of determining above-mentioned image output includes making above-mentioned left side head-mounted display image and the right side
Side head-mounted display image determines that left side head-mounted display exports shadow respectively along the mobile defined distance of vertical direction
The step of picture and right side head-mounted display image output.
9. image treatment method according to claim 6, which is characterized in that above-mentioned user movement be move left and right and
Rotary motion or in the case where moving up and down with rotary motion, the step of determining above-mentioned image output include making above-mentioned left side head
Head mounted displays image and right side head-mounted display image determine a left side respectively along the mobile defined distance of diagonal
The step of side head-mounted display image output and right side head-mounted display image output.
10. a kind of image processor characterized by comprising
Image receiving unit, for obtaining input image from omnidirectional camera；
Acquired above-mentioned input image is configured to bidimensional image battle array by control unit, the structure based on above-mentioned omnidirectional camera
Column, and image output corresponding with the movement of user is determined in constructed above-mentioned bidimensional image array；And
Output section, for providing a user identified above-mentioned image output.
11. image processor according to claim 10, which is characterized in that in the structure of above-mentioned omnidirectional camera
In, structure configuration of the multiple single cameras of above-mentioned omnidirectional camera in the form of polygon is constituted, above-mentioned polygon shape is located at
Multiple single cameras of same edge in state with parallel axes (parallel vidicon (parallel camera)) or intersecting axle (it is prenex,
Convergence video camera (toe-in, convergence camera)) mode configures.
12. image processor according to claim 11, which is characterized in that in above-mentioned bidimensional image array, with institute
The related multiple splicing images of above-mentioned input image obtained are configured respectively along horizontal direction and vertical direction.
13. image processor according to claim 12, which is characterized in that
Splicing image along the configuration of above-mentioned horizontal direction is by distinguishing from multiple sides in above-mentioned polygon form
The image that multiple images that multiple cameras of selection obtain are spliced,
Along above-mentioned vertical direction configuration splicing image be by from be contained in be in above-mentioned polygon form multiple sides in
A side the image that is spliced of multiple images that obtains of multiple cameras.
14. image processor according to claim 13, which is characterized in that above-mentioned control unit utilizes above-mentioned multiple splicings
Video generation interpolation image.
15. image processor according to claim 14, which is characterized in that
Above-mentioned image output is provided to above-mentioned user by head-mounted display,
Above-mentioned image output includes that the left side head-mounted display image for the binocular parallax for reflecting above-mentioned user and right side are worn
Formula display image,
Above-mentioned left side head-mounted display image and right side head-mounted display image are respectively the one of above-mentioned bidimensional image array
Part splicing image.
16. image processor according to claim 15, which is characterized in that above-mentioned user movement be rotate left and right
Or in the case where being rotated up and down movement, above-mentioned control unit makes above-mentioned left side head-mounted display image and right side head-mounted display
Image determines left side head-mounted display image output and right side wear-type respectively along the mobile defined distance of horizontal direction
Display image output.
17. image processor according to claim 16, which is characterized in that above-mentioned user movement be move left and right
Or in the case where moving up and down movement, above-mentioned control unit makes above-mentioned left side head-mounted display image and right side head-mounted display
Image determines left side head-mounted display image output and right side wear-type respectively along the mobile defined distance of vertical direction
Display image output.
18. image processor according to claim 17, which is characterized in that above-mentioned user movement be move left and right
In the case where rotary motion or moving up and down with rotary motion, above-mentioned control unit makes above-mentioned left side head-mounted display image
And right side head-mounted display image determines left side head-mounted display respectively along the mobile defined distance of diagonal
Image output and right side head-mounted display image output.
19. a kind of omnidirectional camera works, for providing 360 degree of videos, which is characterized in that constitute above-mentioned comprehensive camera shooting
Structure configuration of the multiple single cameras of head in the form of polygon, the multiple of same edge in above-mentioned polygon form singly take the photograph
As head is configured in intersecting axle (prenex, convergence video camera (toe-in, convergence camera)) mode.
20. omnidirectional camera works according to claim 19, which is characterized in that
Above-mentioned multiple single cameras are configured as, in the case where user is rotated, using from above-mentioned polygon
Multiple images that multiple cameras for selecting respectively in multiple sides of form obtain provide image output, carry out in above-mentioned user
In the case where moving movement, obtained using from multiple cameras when being contained in multiple one in above-mentioned polygon form
Multiple images for taking provide image output.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR10-2017-0075803 | 2017-06-15 | ||
| KR20170075803 | 2017-06-15 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109151441A true CN109151441A (en) | 2019-01-04 |
Family
ID=64802076
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810618462.8A Pending CN109151441A (en) | 2017-06-15 | 2018-06-15 | The image treatment method and device of omni-directional video |
Country Status (2)
| Country | Link |
|---|---|
| KR (1) | KR20180136891A (en) |
| CN (1) | CN109151441A (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102617940B1 (en) * | 2019-01-24 | 2023-12-22 | 주식회사 케이티 | Apparatus for generating 360 degree image supporting 6-degrees of freedom using multi micro lens array light field camera |
| WO2021130355A1 (en) * | 2019-12-24 | 2021-07-01 | Koninklijke Kpn N.V. | Video processing device and manifest file for video streaming |
| KR102242710B1 (en) * | 2019-12-31 | 2021-04-22 | 서울과학기술대학교 산학협력단 | Apparatus for providing half free-viewpoint video |
| KR102393309B1 (en) * | 2022-01-10 | 2022-04-29 | 정장수 | VR production method for 3D experience education |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016191708A1 (en) * | 2015-05-27 | 2016-12-01 | Google Inc. | Camera rig and stereoscopic image capture |
| WO2017054925A1 (en) * | 2015-09-30 | 2017-04-06 | Calay Venture S.A.R.L. | Presence camera |
| CN106797459A (en) * | 2014-09-22 | 2017-05-31 | 三星电子株式会社 | The transmission of 3 D video |
| CN106796390A (en) * | 2014-10-07 | 2017-05-31 | 诺基亚技术有限公司 | For the camera apparatus with big visual field of three-dimensional imaging |
-
2018
- 2018-06-08 KR KR1020180065860A patent/KR20180136891A/en unknown
- 2018-06-15 CN CN201810618462.8A patent/CN109151441A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106797459A (en) * | 2014-09-22 | 2017-05-31 | 三星电子株式会社 | The transmission of 3 D video |
| CN106796390A (en) * | 2014-10-07 | 2017-05-31 | 诺基亚技术有限公司 | For the camera apparatus with big visual field of three-dimensional imaging |
| WO2016191708A1 (en) * | 2015-05-27 | 2016-12-01 | Google Inc. | Camera rig and stereoscopic image capture |
| WO2017054925A1 (en) * | 2015-09-30 | 2017-04-06 | Calay Venture S.A.R.L. | Presence camera |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20180136891A (en) | 2018-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109151441A (en) | The image treatment method and device of omni-directional video | |
| CN104252045B (en) | Stereoscopic viewer and the three-dimensional Theatre devices of individual with 4G communication systems | |
| KR100947366B1 (en) | 3D image display method and system thereof | |
| US20130321396A1 (en) | Multi-input free viewpoint video processing pipeline | |
| KR102491749B1 (en) | Panoramic 3D Imaging System | |
| KR20180111798A (en) | Adaptive stitching of frames in the panorama frame creation process | |
| CN108141578A (en) | Camera is presented | |
| JP2017532847A (en) | 3D recording and playback | |
| JPH10188034A (en) | Three-dimensional image generator and distance operating device | |
| KR20070007059A (en) | Multi-dimensional imaging apparatus, systems, and methods | |
| SG181708A1 (en) | A system and method for producing stereoscopic images | |
| CN105894567A (en) | Scaling pixel depth values of user-controlled virtual object in three-dimensional scene | |
| US11436698B2 (en) | Method of playing back image on display device and display device | |
| Luo et al. | Pepper's Cone: An Inexpensive Do-It-Yourself 3D Display | |
| KR101198557B1 (en) | 3D stereoscopic image and video that is responsive to viewing angle and position | |
| Bourke | Synthetic stereoscopic panoramic images | |
| Thatte et al. | Depth augmented stereo panorama for cinematic virtual reality with focus cues | |
| US10200675B1 (en) | Omni-directional stereo system | |
| CN111699460A (en) | Multi-view virtual reality user interface | |
| WO2019130183A1 (en) | Multi-camera display | |
| Bourke | Low Cost Projection Environment for Immersive Gaming. | |
| KR101425321B1 (en) | System for displaying 3D integrated image with adaptive lens array, and method for generating elemental image of adaptive lens array | |
| US20220232201A1 (en) | Image generation system and method | |
| Yan et al. | Research summary on light field display technology based on projection | |
| CN109151273A (en) | A kind of light fan stereo camera and stereoscopic measurement method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190104 |