CN109074235A - any view generation - Google Patents
any view generation Download PDFInfo
- Publication number
- CN109074235A CN109074235A CN201780019122.7A CN201780019122A CN109074235A CN 109074235 A CN109074235 A CN 109074235A CN 201780019122 A CN201780019122 A CN 201780019122A CN 109074235 A CN109074235 A CN 109074235A
- Authority
- CN
- China
- Prior art keywords
- image
- visual angle
- view
- transformed
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000007 visual effect Effects 0.000 claims abstract description 76
- 238000000034 method Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims 1
- 230000001052 transient effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 15
- 230000009466 transformation Effects 0.000 description 17
- 230000008569 process Effects 0.000 description 13
- 230000004044 response Effects 0.000 description 9
- 238000009877 rendering Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 6
- 230000002452 interceptive effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000003306 harvesting Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241001269238 Data Species 0.000 description 1
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
- G06T15/205—Image-based rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Processing Or Creating Images (AREA)
- Image Generation (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Processing (AREA)
Abstract
It discloses for the technology according to other multiple existing any views of view generation.In some embodiments, any view generation includes: first image of the retrieval with the first visual angle and the second image with the second visual angle；Each of the first image and second image are transformed into third visual angle；And by the way that at least some of the pixel from least some of transformed pixel of the first image and from the second transformed image to be combined, to generate the third image with the third visual angle.
Description
Background technique
Existing Rendering is faced with the compromise between quality and the competitive purpose of speed.High quality rendering requirements are significant
Process resource and the time.However, slow Rendering is unacceptable in many applications (such as, interactive application in real time).Compared with
Low quality but faster Rendering for this application typically for being advantageous.For example, rasterizing is generally by being used for phase
The real-time graph application of fast rendering is used, but is used by cost of quality.Therefore, it is necessary to not damage quality or speed significantly
The improved technology of degree.
Detailed description of the invention
Various embodiments of the present invention are disclosed in features as discussed above.
Fig. 1 is the high level block diagram for illustrating the embodiment of system of any view for generating scene.
Fig. 2 illustrates the example of database asset.
Fig. 3 is the flow chart for illustrating the embodiment of the process for generating visual angle.
Specific embodiment
The present invention can realize in many ways, including be implemented as: process；Device；System；Material composition；It is embodied in
Computer program product in computer-readable storage media；And/or processor is such as configured to execution and is stored in storage
The processor of the instruction provided on device and/or by memory, the memory are coupled to the processor.In the present specification, these
Any other form that implementation or the present invention can take can be referred to as technology.It generally, can be in the scope of the present invention
The order of the step of process disclosed in interior change.Unless otherwise stated, such as it is described as being configured to execute appointing
The processor of business or the component of memory etc may be implemented as: general component is temporarily configured at given time
Execute the task；Or specific component, it is manufactured into and executes the task.As used herein, term " processor " refers to quilt
It is configured to one or more equipment, circuit and/or the processing core of processing data (such as, computer program instructions).
Retouching in detail for one or more embodiments of the invention is provided together below along with the attached drawing of the diagram principle of the invention
It states.The present invention is described in conjunction with this embodiment, but the present invention is not limited to any embodiments.The scope of the present invention is only by claim
Limitation, and the present invention covers many alternatives, modification and equivalent program.It illustrates in the following description many specific thin
Section, in order to provide thorough understanding of the invention.These details are to provide for exemplary purposes, and the present invention can be
It is practiced in the case where without some or all of these details according to claim.For purposes of clarity, not in detail
Thin description known technologic material in correlative technology field of the present invention, so that the present invention is not obscured unnecessarily.
Disclose the technology of any view for generating scene.Normal form described herein makes low-down processing or calculating
Expense becomes required, while still providing fine definition output, so that effectively eliminating between rendering speed and quality has challenge
The compromise of property.Disclosed technology is outstanding for quickly generating high quality output about the application of interactive real-time graph
Its is useful.This application depends on operations described below: in response to and according to the interactive views presented or scene user's manipulation,
The output of preferably high quality is substantially presented immediately.
Fig. 1 is the high level block diagram for illustrating the embodiment of system 100 of any view for generating scene.As depicted
As, any view generation device 102 receives the request for any view as input 104, is based on existing database assets
106 to generate requested view, and provides view generated as output 108 in response to input request.In various realities
It applies in example, any view generation device 102 may include such as central processing unit (CPU) or graphics processing unit (GPU) etc
Processor.The discribed configuration of system 100 in Fig. 1 it is for purposes of explanation and provide.Generally, system 100
It may include the interconnecting member for any other proper number and/or configuration that described function is provided.For example, in other implementations
In example, any view generation device 102 may include different configuration of internal part 110-116, and any view generation device 102 can be with
Including multiple concurrent physicals and/or virtual processor, database 106 may include multiple networking data libraries or assets cloud etc..
Any view request 104 includes the request for the visual angle of scene.In some embodiments, scene is asked
The visual angle asked is already present in the asset database 106 at other visual angles or viewpoint including scene.In various embodiments
In, any view request 104 can be received from process or user.For example, can be in response to the scene that is presented or part thereof
User manipulates (user of the camera view of the scene such as, presented manipulates) and receives input 104 from user interface.As another
One example, can in response in virtual environment (such as, the virtual roaming (fly-through) of scene) movement or travel path
Specification and receive any view request 104.In some embodiments, any view of the possibility of demandable scene is at least partly
It is constrained.For example, user may not manipulate the camera view of the interactive scene presented to any random position, and
It is certain positioning or the visual angle for being constrained to scene.
Multiple views of each assets stored of the storage of database 106.In given situation, assets are standardized
The concrete scene being stored in database 106 is referred to as multiple views.In various embodiments, scene may include single object,
Multiple objects or abundant virtual environment.Specifically, the storage of database 106 is corresponding with the different perspectives of each assets or viewpoint
Multiple images.The image being stored in database 106 includes the rendering of high-quality photos or photo realistic.It can be offline
Capture or render this fine definition high-definition picture that is filled to database 106 during process, or from external source
Obtain this fine definition high-definition picture being filled to database 106.In some embodiments, with being stored in data
Each of library 106 image stores corresponding camera characteristics.That is, with each image storage such as relative position or
The camera attribute of positioning, orientation, rotation, depth information, focal length, aperture, level of zoom or the like.Further, it is also possible to storage
In the camera illumination information of the storage of each of database 106 image such as shutter speed and exposure etc.
In various embodiments, any number of different perspectives of assets can be stored in database 106, Fig. 2 figure
The example of database asset is shown.In the given example, 73 corresponding with around the different angle of chair object
View is captured or renders, and is stored in database 106.It can be for example by around chair rotating camera or before camera
Face rotating chair captures view.With each image storage relative object and camera position generated and orientation information.Fig. 2
Particularly illustrate the view of the scene including single object.Database 106 can also be stored including multiple objects or be enriched virtual
The specification of the scene of environment.In this case, in scene or three-dimensional space different location or positioning it is corresponding multiple
View is captured or renders, and is stored in database 106 together with corresponding camera information.Generally, it is stored in number
It may include two or three dimensions according to the image in library 106, and may include the picture that fixes of animation or video sequence
(still) or frame.
In response to the request for any view for not being the scene 104 being already present in database 106, any view
Generator 102 is according to other multiple existing requested any views of view generation of the scene being stored in database 106.?
In the example arrangement of Fig. 1, the asset management engine 110 of any view generation device 102 manages database 106.For example, asset management
Engine 110 can be in order to storing and the data in searching database 106.In response to being directed to the request of any view of scene 104,
Asset management engine 110 identifies and obtains other multiple existing views of scene from database 106.In some embodiments, it provides
Produce all existing views that supervisor engine 1 10 retrieves scene from database 106.Alternatively, asset management engine 110 can select
Select and retrieve the subset for example with requested any immediate existing view of view.In this case, asset management is drawn
110 subsets for being configured to intelligently select that the existing view of pixel can be harvested from it are held up, to generate requested any view.
In various embodiments, multiple existing views can be retrieved together by asset management engine 110, or as they are arbitrarily regarded
The other component of diagram generator 102 is needed and is retrieved when they are needed by the other component of any view generation device 102.
The visual angle for each the existing view retrieved by asset management engine 110 is by the visual angle of any view generation device 102
Transform engine 112 is transformed into the visual angle of requested any view.As previously indicated, accurate camera information is known and with depositing
Storage is stored in each of database 106 image.Therefore, change from existing view to the visual angle of requested any view
Become includes that simple geometry is mapped or converted.In various embodiments, view transformation engine 112 can use any one or more of
Appropriate mathematical technique is with the visual angle by the view transformation of existing view at any view.Requested view includes and appoints wherein
In the case where what existing different any view of view, the transformation of existing view to the visual angle of any view will include at least one
The pixel for not mapping or losing a bit, that is, at the angle or positioning introduced in any view being not present in existing view.
Pixel Information from the existing view individually through view transformation will not be able to fill all pictures of different views
Element.However, in many cases, requested any view can be formed from multiple existing view harvests through view transformation
Most of pixels (if not all pixels).The merging engine 114 of any view generation device 102 will come from multiple through regarding
The pixel of the existing view of angular transformation is combined, to generate requested any view.It is desirable that from existing view harvest group
At all pixels of any view.This can be it is possible, for example, if consider in the existing view of assets or filling for visual angle
The set of point multiplicity is available and/or if requested visual angle with existing visual angle is not too dissimilar.
Any proper technology can be used so that the pixel from multiple existing views through view transformation to be combined or
Merge, to generate requested any view.In one embodiment, from database 106 select and retrieve with it is requested
The existing view of any view immediate first, and by the first existing view transformation at the visual angle of requested any view.
Then the first existing view from this through view transformation harvests pixel, and using the pixel to fill in requested any view
Respective pixel.In order to fill requested any view not from the first existing available pixel of view, selected from database 106
Select and retrieve the second existing view including at least some of these residual pixels and by the second existing view transformation at institute
The visual angle of any view of request.Then the second existing view harvest from this through view transformation can not obtained from the first existing view
Pixel, and fill using the pixel respective pixel in requested any view.It can be for any number of additional
Existing view repeats the process, until all pixels of requested any view are filled and/or until all existing
View has been depleted or the existing view of specified threshold number is used.
In some embodiments, requested any view may include not from any existing available some pictures of view
Element.In this case, interpolation engine 116 is configured to fill any residual pixel of requested any view.Various
In embodiment, any one or more of appropriate interpositioning can be interpolated engine 116 and use in requested any view
Middle these unfilled pixels of generation.The example of adoptable interpositioning includes such as linear interpolation, arest neighbors interpolation.Picture
The interpolation of element introduces averaging or smoothing.Eral image quality may indistinctively be influenced by certain interpolation, but excessive
Interpolation may introduce unacceptable fuzzy.Therefore, it may be desirable that abstemiously use interpolation.It as previously indicated, if can be with
The all pixels of requested any view are obtained from existing view, then interpolation is avoided entirely.However, if requested
View of anticipating includes not from any existing available some pixels of view, then interpolation is introduced into.Generally, the measurement of required interpolation
Certainly in the visual angle of the number of available existing view, the diversity at the visual angle of existing view, and/or any view relative to existing
The visual angle of view has mostly different.
About example depicted in figure 2, the existing view of chair is stored as around 73 views of chair object
Figure.These multiple existing views with preferably minimum (if any) interpolation can be used to generate compared to being stored
Any of view is different or unique any views around chair object.However, generating and storing existing view
This exhaustive set may not be efficient or desired.In some cases, can replace generation and storage covering view
The existing view of the significant lesser number of the set of the abundant multiplicity at angle.For example, can be by 73 views of chair object
Weaken into the small set of a small number of views around chair object.
As previously mentioned, in some embodiments, demandable possible any view can be at least partly constrained.Example
Such as, user can be limited in order to avoid virtual camera associated with interactive scene is moved to certain positioning.About the given of Fig. 2
Example, it is demandable view arbitrarily may to be limited to around any position of chair object, but can not include such as chair
Any position below subobject, this is because there are insufficient pixel datas for the bottom of chair object.To institute
This constraint of any view allowed ensures that requested any view can be by any view generation device 102 according to existing
Data generate.
Any view generation device 102 in response to input any view request 104 and generate and export and is requested any
View 108.The resolution ratio or quality of any view 108 generated and the identical in quality or class for generating its existing view
Seemingly, this is because the pixel from those views be used to generate any view.Therefore, using the existing view of fine definition big
Fine definition is caused to export in most cases.It in some embodiments, will be generated with other existing views of association scene
Any view 108 is stored in database 106, and can then use any view generated 108 in response to being directed to
The further request of any view and any other view for generating scene.It include in database 106 in wherein input 104
Existing view request in the case where, do not need as described according to the requested view of other view generations；It takes
And instead of, requested view is retrieved via simple data library lookup and requested view is directly rendered as output 108.
In addition, any view generation device 102 may be configured to generate any overview using described technology.
That is, input 104 may include the request by multiple object compositions at single custom views.In this case, for
Each of multiple object executes above mentioned technology, and it includes being somebody's turn to do that above mentioned technology, which is combined to generate,
The single integration of multiple objects or overview.Specifically, the existing view of each of multiple object is by asset management
Engine 110 is selected and is retrieved from database 106, and existing view is transformed into the view of requested view by view transformation engine 112
Angle, the pixel from the existing view through view transformation use pair to fill requested overview by merging engine 114
Pixel is answered, and any residue in overview is not filled by pixel by 116 interpolation of interpolation engine.In some embodiments, institute
The overview of request may include the already existing visual angle for organizing integral one or more objects.In this feelings
Under condition, using the existing view of subject property corresponding with requested visual angle, directly to fill and pair in overview
As corresponding pixel, rather than first according to other existing requested visual angles of view generation of object.
As the example for any overview for including multiple objects, the chair object and independent shooting or wash with watercolours of Fig. 2 are considered
The desk object of dye.Disclosed technology can be used and come combined chair object and desk object, to generate all two objects
Single overview.Therefore, using disclosed technology, independently catching for each of multiple objects can consistently be combined
The image or view for obtaining or rendering include multiple object and the scene with desired visual angle to generate.As previously indicated, each
The depth information of a existing view is known.The view transformation of each existing view includes depth conversion, to allow this
Multiple objects are properly located in overview relative to each other.
Any overview is generated to be not limited to multiple single object compositions into custom views.But there are multiple objects
Or multiple scenes of multiple abundant virtual environments can be similarly combined into customization overview.For example, it may be possible to from difference
Content generates source and may have can be by compared with multiple virtual environments separately and independently generated at individual visual angle
It is combined into the overview with desired visual angle.Therefore, generally, any view generation device 102 may be configured to will include
Multiple independent assets of existing view that may be different are consistently combined or are reconciled into desired possible arbitrary visual angle
Overview.Perfect harmonious gained overview is generated, this is because all combined assets be normalized to it is identical
Visual angle.It can arbitrarily be regarded based on the existing view for the individual assets that can be able to generate overview to constrain the possibility of overview
Angle.
Fig. 3 is the flow chart for illustrating the embodiment of the process for generating visual angle.Process 300 can be for example by scheming
1 any view generation device 102 uses.In various embodiments, process 300 can be used to generate specified assets
Any view or any overview.
Process 300 starts from step 302 place, at step 302, receives the request for being directed to visual angle.In some implementations
In example, the request received at step 302 may include being directed to that existing to obtain visual angle different from specified scene any
The scene visual angle request.In this case, for example, visual angle request can be the institute in response to the scene
Requested change in the visual angle of the view of presentation and it is received.This change in visual angle can be by changing or manipulating
Virtual camera (such as, passing through rolling camera, change focal length, change level of zoom etc.) associated with the scene is come what is promoted.
Alternatively, in some embodiments, the request received at step 302 may include asking for any overview
It asks.As an example, this any overview request can be about the multiple standalone objects of permission by selection and offer institute
Select object come through integration, the application of overview through perspective correction it is received.
At step 304, requested any view is generated according to it from one or more associations asset data library searching
At least part of multiple conventional images at angle.It include for specified assets in the request wherein received at step 302
Visual angle request in the case where, multiple retrieved image can be associated with specified assets, and at it
In the request that step 302 place receives include for any overview request in the case where, multiple retrieved figure
Picture can be associated with multiple assets.
At step 306, each with different visual angles in the multiple conventional images retrieved at step 304 is showed
There is image to be transformed into the visual angle requested at step 302.Each of the conventional images of retrieval include at step 304
Associated Viewing-angle information.By camera characteristics associated with the image is generated, (such as relative positioning is taken at the visual angle of each image
To, rotation, angle, depth, focal length, aperture, level of zoom, illumination information etc.) definition.Since complete camera information is for each
It is known for a image, therefore the view transformation of step 306 includes simple mathematical operation.In some embodiments, step
306 also optionally include light transform, so that all images are consistently normalized to equating expections lighting condition.
At step 308, is had by the pixel filling harvested from the conventional images through view transformation and requested in step 302
Visual angle image at least partly.That is, use the pixel from multiple conventional images through perspective correction with
Generate the image with requested visual angle.
Whether at step 310, determining has the image generated of requested visual angle complete.If in step
Determining at 310 has the image generated of requested visual angle imperfect, then determines whether to appoint at step 312
What more conventional images, any residue that image generated can be excavated from more conventional images are not filled by pixel.If
Determined at step 312 more conventional images be it is available, then one or more additional conventional images are retrieved at step 314,
And process 300 continues at step 306.
If at step 310 determine have requested visual angle image generated it is imperfect and if
Step 312 place determine no more conventional images be it is available, then at step 316 interpolation image generated any residue
It is not filled by pixel.Any one or more of appropriate interpositioning can be used at step 316.
If determining the image generated with requested visual angle completely or in step at step 310
Any residue of interpolation is not filled by after pixel at 316, and output has the generated of requested visual angle at step 318
Image.Process 300 then terminates.
As described, disclosed technology can be used for generating visual angle based on other existing visual angles.It will
It is possible for not being normalized into Public anticipation visual angle compared with visual angle, this is because saving camera letter with each existing visual angle
Breath.The gained image with desired visual angle can be constructed according to pixel is excavated from the conventional images through view transformation.With use
It is not only quickly and almost instantaneous that disclosed technology generates the associated processing of visual angle, but also high quality is caused to export, from
And keep disclosed technology especially powerful for the application of interactive real-time graph.
Although previous embodiment is slightly described in detail for clarity of understanding, the present invention is not limited to institutes
The details of offer.In the presence of realization many alternatives of the invention.The disclosed embodiments are illustrative rather than restricted
's.
Claims (20)
1. a kind of system, comprising:
Processor is configured to:
Retrieve the first image with the first visual angle and the second image with the second visual angle；
Each of the first image and second image are transformed into third visual angle；And
By by the pixel from least some of transformed pixel of the first image and from the second transformed image
At least some of be combined, to generate the third image with the third visual angle；And
Memory is coupled to the processor and is configured to provide instruction to the processor.
2. the system as claimed in claim 1, wherein first visual angle, second visual angle and the third visual angle include
Different perspectives.
3. the system as claimed in claim 1, wherein the first image and second image include pre-existing image.
4. the system as claimed in claim 1, wherein the third image with the third visual angle is different according to having
What multiple pre-existing images at visual angle generated.
5. the system as claimed in claim 1, wherein the visual angle of each image is special by camera associated with the image is generated
Property definition.
6. system as claimed in claim 5, wherein the camera characteristics one or more of include following items: positioning,
Orientation, rotation, angle, depth, focal length, aperture and level of zoom.
7. the system as claimed in claim 1, wherein camera characteristics are with every in the first image and second image
One and store, and be known for each of the first image and second image.
8. the system as claimed in claim 1, wherein generating the one or more that third image includes: third image described in interpolation
Pixel.
9. the system as claimed in claim 1, wherein the first image, second image and the third image include tool
There is the image of the same scene of different perspectives.
10. system as claimed in claim 9, wherein the scene includes single object or multiple objects.
11. system as claimed in claim 9, wherein the scene includes virtual environment.
12. the system as claimed in claim 1, wherein the first image includes the first object, and second image includes the
Two objects, and the third image includes first object and second object.
13. the system as claimed in claim 1, wherein the first image includes the first scene, and second image includes the
Two scenes, and the third image includes the entirety of first scene and second scene.
14. system as claimed in claim 13, wherein each scene includes single object, multiple objects or virtual environment.
15. the system as claimed in claim 1, wherein the first image, second image and the third image include
Static image.
16. the system as claimed in claim 1, wherein the first image, second image and the third image include
The view of three-dimensional space.
17. the system as claimed in claim 1, wherein the first image, second image and the third image include
The frame of animation or video sequence.
18. the system as claimed in claim 1, wherein the processor is further configured to: retrieval has and the third
One or more additional images at the different visual angle in visual angle；One or more of additional images are transformed into the third view
Angle；And at least some pixels from transformed one or more additional images are used when generating the third image.
19. a kind of method, comprising:
Retrieve the first image with the first visual angle and the second image with the second visual angle；
Each of the first image and second image are transformed into third visual angle；And
By by the pixel from least some of transformed pixel of the first image and from the second transformed image
At least some of be combined, to generate the third image with the third visual angle.
20. a kind of computer program product is embodied in non-transient computer-readable storage media and including following for executing
The computer instruction of operation:
Retrieve the first image with the first visual angle and the second image with the second visual angle；
Each of the first image and second image are transformed into third visual angle；And
By by the pixel from least some of transformed pixel of the first image and from the second transformed image
At least some of be combined, to generate the third image with the third visual angle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210275459.7A CN114663571A (en) | 2016-03-25 | 2017-03-24 | Arbitrary view generation |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/081,553 US9996914B2 (en) | 2016-03-25 | 2016-03-25 | Arbitrary view generation |
| US15/081,553 | 2016-03-25 | ||
| PCT/US2017/024101 WO2017165818A1 (en) | 2016-03-25 | 2017-03-24 | Arbitrary view generation |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210275459.7A Division CN114663571A (en) | 2016-03-25 | 2017-03-24 | Arbitrary view generation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109074235A true CN109074235A (en) | 2018-12-21 |
| CN109074235B CN109074235B (en) | 2022-04-05 |
Family
ID=59898995
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210275459.7A Pending CN114663571A (en) | 2016-03-25 | 2017-03-24 | Arbitrary view generation |
| CN201780019122.7A Active CN109074235B (en) | 2016-03-25 | 2017-03-24 | Arbitrary view generation |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210275459.7A Pending CN114663571A (en) | 2016-03-25 | 2017-03-24 | Arbitrary view generation |
Country Status (7)
| Country | Link |
|---|---|
| US (4) | US9996914B2 (en) |
| EP (1) | EP3433720A4 (en) |
| JP (3) | JP2019516202A (en) |
| KR (5) | KR102143098B1 (en) |
| CN (2) | CN114663571A (en) |
| CA (1) | CA3018749C (en) |
| WO (1) | WO2017165818A1 (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9332285B1 (en) | 2014-05-28 | 2016-05-03 | Lucasfilm Entertainment Company Ltd. | Switching modes of a media content item |
| US9996914B2 (en) * | 2016-03-25 | 2018-06-12 | Outward, Inc. | Arbitrary view generation |
| US10163250B2 (en) * | 2016-03-25 | 2018-12-25 | Outward, Inc. | Arbitrary view generation |
| CN110148088B (en) * | 2018-03-14 | 2023-09-19 | 北京邮电大学 | Image processing method, image rain removing method, device, terminal and medium |
| KR102245931B1 (en) | 2018-10-22 | 2021-04-28 | 주식회사 엘지화학 | Preparing method of dimethylolbutanal and preperation method of trimethylolpropane using the same |
| GB2586985B (en) * | 2019-09-10 | 2023-04-05 | Hitomi Ltd | Signal delay measurement |
| JP7410289B2 (en) * | 2019-11-08 | 2024-01-09 | アウトワード・インコーポレーテッド | Generating arbitrary views |
| WO2022196036A1 (en) | 2021-03-17 | 2022-09-22 | 株式会社Ｊｖｃケンウッド | Conjugate formation method, substance of interest measurement method, and kit |
| WO2024048210A1 (en) * | 2022-09-02 | 2024-03-07 | 富士フイルム株式会社 | Viewpoint image generation device, viewpoint image generation method, and program |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6222947B1 (en) * | 1997-02-19 | 2001-04-24 | Canon Kabushiki Kaisha | Image editing apparatus and method and medium on which programs are recorded |
| US20060280368A1 (en) * | 2001-07-02 | 2006-12-14 | Photoinaphoto.Com, Inc. | System and method for discovering and categorizing attributes of a digital image |
| CN101246599A (en) * | 2007-01-26 | 2008-08-20 | 转换器有限公司 | Methodology for 3d scene reconstruction from 2d image sequences |
| US20120140027A1 (en) * | 2010-12-06 | 2012-06-07 | Teranex Systems Inc. | Pseudo-3d Forced Perspective Methods And Devices |
| CN101281640B (en) * | 2007-04-05 | 2012-06-13 | 奥多比公司 | Laying out multiple images |
| US20120314937A1 (en) * | 2010-02-23 | 2012-12-13 | Samsung Electronics Co., Ltd. | Method and apparatus for providing a multi-view still image service, and method and apparatus for receiving a multi-view still image service |
| US20140254908A1 (en) * | 2007-01-17 | 2014-09-11 | MediGuide, Ltd. | Method and a system for registering a 3d pre-acquired image coordinate system with a medical positioning system coordinate system and with a 2d image coordinate system |
| US20140267343A1 (en) * | 2013-03-14 | 2014-09-18 | Microsoft Corporation | Translated view navigation for visualizations |
| CN203870604U (en) * | 2014-03-28 | 2014-10-08 | 索尼公司 | Display device |
Family Cites Families (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000137815A (en) | 1998-11-02 | 2000-05-16 | Gen Tec:Kk | New viewpoint image generating method |
| US6377257B1 (en) | 1999-10-04 | 2002-04-23 | International Business Machines Corporation | Methods and apparatus for delivering 3D graphics in a networked environment |
| JP2001140097A (en) | 1999-11-18 | 2001-05-22 | Nippon Paint Co Ltd | Double-layer electrodeposition coating film and method for forming multilayer coating film including the coating film |
| JP4065488B2 (en) * | 2001-12-14 | 2008-03-26 | キヤノン株式会社 | 3D image generation apparatus, 3D image generation method, and storage medium |
| GB2413720B (en) | 2003-03-14 | 2006-08-02 | British Broadcasting Corp | Video processing |
| JP4052331B2 (en) * | 2003-06-20 | 2008-02-27 | 日本電信電話株式会社 | Virtual viewpoint image generation method, three-dimensional image display method and apparatus |
| EP1605408A1 (en) | 2004-06-11 | 2005-12-14 | Saab Ab | Image-based rendering (IBR) |
| WO2007099525A2 (en) | 2006-03-03 | 2007-09-07 | Medic Vision - Brain Technologies Ltd. | System and method of automatic prioritization and analysis of medical images |
| JP5011168B2 (en) * | 2008-03-04 | 2012-08-29 | 日本電信電話株式会社 | Virtual viewpoint image generation method, virtual viewpoint image generation apparatus, virtual viewpoint image generation program, and computer-readable recording medium recording the program |
| JP5222597B2 (en) | 2008-03-19 | 2013-06-26 | 三洋電機株式会社 | Image processing apparatus and method, driving support system, and vehicle |
| EP2359336A1 (en) | 2008-10-03 | 2011-08-24 | 3M Innovative Properties Company | Systems and methods for optimizing a scene |
| JP5015126B2 (en) * | 2008-12-09 | 2012-08-29 | 日本電信電話株式会社 | Image generation method, image authentication method, image generation apparatus, image authentication apparatus, program, and recording medium |
| US8867820B2 (en) | 2009-10-07 | 2014-10-21 | Microsoft Corporation | Systems and methods for removing a background of an image |
| JP2011091527A (en) | 2009-10-21 | 2011-05-06 | Panasonic Corp | Video conversion device and imaging apparatus |
| JP5444139B2 (en) | 2010-06-29 | 2014-03-19 | クラリオン株式会社 | Image calibration method and apparatus |
| US9177381B2 (en) | 2010-12-22 | 2015-11-03 | Nani Holdings IP, LLC | Depth estimate determination, systems and methods |
| KR20120137205A (en) * | 2011-06-10 | 2012-12-20 | 삼성전자주식회사 | Method and apparatus for image processing |
| US9451232B2 (en) | 2011-09-29 | 2016-09-20 | Dolby Laboratories Licensing Corporation | Representation and coding of multi-view images using tapestry encoding |
| KR101415147B1 (en) * | 2012-08-10 | 2014-07-08 | 광운대학교 산학협력단 | A Boundary Noise Removal and Hole Filling Method for Virtual Viewpoint Image Generation |
| US9558423B2 (en) | 2013-12-17 | 2017-01-31 | Canon Kabushiki Kaisha | Observer preference model |
| US10726593B2 (en) | 2015-09-22 | 2020-07-28 | Fyusion, Inc. | Artificially rendering images using viewpoint interpolation and extrapolation |
| GB2532075A (en) | 2014-11-10 | 2016-05-11 | Lego As | System and method for toy recognition and detection based on convolutional neural networks |
| US9916525B2 (en) | 2015-10-13 | 2018-03-13 | Siemens Healthcare Gmbh | Learning-based framework for personalized image quality evaluation and optimization |
| US10163249B2 (en) * | 2016-03-25 | 2018-12-25 | Outward, Inc. | Arbitrary view generation |
| US10163250B2 (en) * | 2016-03-25 | 2018-12-25 | Outward, Inc. | Arbitrary view generation |
| US9996914B2 (en) * | 2016-03-25 | 2018-06-12 | Outward, Inc. | Arbitrary view generation |
| US10163251B2 (en) * | 2016-03-25 | 2018-12-25 | Outward, Inc. | Arbitrary view generation |
-
2016
- 2016-03-25 US US15/081,553 patent/US9996914B2/en active Active
-
2017
- 2017-03-24 EP EP17771269.2A patent/EP3433720A4/en active Pending
- 2017-03-24 CN CN202210275459.7A patent/CN114663571A/en active Pending
- 2017-03-24 KR KR1020187027510A patent/KR102143098B1/en active IP Right Grant
- 2017-03-24 JP JP2019500759A patent/JP2019516202A/en active Pending
- 2017-03-24 WO PCT/US2017/024101 patent/WO2017165818A1/en active Application Filing
- 2017-03-24 CA CA3018749A patent/CA3018749C/en active Active
- 2017-03-24 CN CN201780019122.7A patent/CN109074235B/en active Active
- 2017-03-24 KR KR1020227013410A patent/KR20220054902A/en not_active Application Discontinuation
- 2017-03-24 KR KR1020217005383A patent/KR102324651B1/en active IP Right Grant
- 2017-03-24 KR KR1020207022557A patent/KR102221937B1/en active IP Right Grant
- 2017-03-24 KR KR1020217036135A patent/KR102530116B1/en active IP Right Grant
-
2018
- 2018-04-30 US US15/967,394 patent/US10748265B2/en active Active
-
2020
- 2020-06-30 US US16/917,688 patent/US11544829B2/en active Active
- 2020-11-26 JP JP2020195632A patent/JP7116142B2/en active Active
-
2022
- 2022-07-28 JP JP2022120065A patent/JP2022141937A/en active Pending
- 2022-11-29 US US18/071,474 patent/US20230088433A1/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6222947B1 (en) * | 1997-02-19 | 2001-04-24 | Canon Kabushiki Kaisha | Image editing apparatus and method and medium on which programs are recorded |
| US20060280368A1 (en) * | 2001-07-02 | 2006-12-14 | Photoinaphoto.Com, Inc. | System and method for discovering and categorizing attributes of a digital image |
| US20140254908A1 (en) * | 2007-01-17 | 2014-09-11 | MediGuide, Ltd. | Method and a system for registering a 3d pre-acquired image coordinate system with a medical positioning system coordinate system and with a 2d image coordinate system |
| CN101246599A (en) * | 2007-01-26 | 2008-08-20 | 转换器有限公司 | Methodology for 3d scene reconstruction from 2d image sequences |
| CN101281640B (en) * | 2007-04-05 | 2012-06-13 | 奥多比公司 | Laying out multiple images |
| US20120314937A1 (en) * | 2010-02-23 | 2012-12-13 | Samsung Electronics Co., Ltd. | Method and apparatus for providing a multi-view still image service, and method and apparatus for receiving a multi-view still image service |
| US20120140027A1 (en) * | 2010-12-06 | 2012-06-07 | Teranex Systems Inc. | Pseudo-3d Forced Perspective Methods And Devices |
| US20140267343A1 (en) * | 2013-03-14 | 2014-09-18 | Microsoft Corporation | Translated view navigation for visualizations |
| CN203870604U (en) * | 2014-03-28 | 2014-10-08 | 索尼公司 | Display device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114663571A (en) | 2022-06-24 |
| KR20220054902A (en) | 2022-05-03 |
| KR102324651B1 (en) | 2021-11-11 |
| KR102221937B1 (en) | 2021-03-04 |
| JP2019516202A (en) | 2019-06-13 |
| JP7116142B2 (en) | 2022-08-09 |
| JP2022141937A (en) | 2022-09-29 |
| EP3433720A1 (en) | 2019-01-30 |
| CA3018749A1 (en) | 2017-09-28 |
| KR102530116B1 (en) | 2023-05-10 |
| US9996914B2 (en) | 2018-06-12 |
| US11544829B2 (en) | 2023-01-03 |
| US20180247398A1 (en) | 2018-08-30 |
| US20170278228A1 (en) | 2017-09-28 |
| KR102143098B1 (en) | 2020-08-10 |
| KR20210137235A (en) | 2021-11-17 |
| JP2021044004A (en) | 2021-03-18 |
| KR20200096691A (en) | 2020-08-12 |
| CN109074235B (en) | 2022-04-05 |
| US20200334798A1 (en) | 2020-10-22 |
| KR20210024218A (en) | 2021-03-04 |
| EP3433720A4 (en) | 2019-08-28 |
| WO2017165818A1 (en) | 2017-09-28 |
| KR20180125970A (en) | 2018-11-26 |
| US10748265B2 (en) | 2020-08-18 |
| US20230088433A1 (en) | 2023-03-23 |
| CA3018749C (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109074235A (en) | any view generation | |
| US11875451B2 (en) | Arbitrary view generation | |
| US11676332B2 (en) | Arbitrary view generation | |
| US20210217225A1 (en) | Arbitrary view generation | |
| US11636578B1 (en) | Partial image completion | |
| US11972522B2 (en) | Arbitrary view generation | |
| JP2019144958A (en) | Image processing device, image processing method, and program | |
| EP4055567A1 (en) | Arbitrary view generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |