WO2021044441A1 - Interactive augmented reality (ar) based video creation from existing video - Google Patents
Interactive augmented reality (ar) based video creation from existing video Download PDFInfo
- Publication number
- WO2021044441A1 WO2021044441A1 PCT/IN2020/050769 IN2020050769W WO2021044441A1 WO 2021044441 A1 WO2021044441 A1 WO 2021044441A1 IN 2020050769 W IN2020050769 W IN 2020050769W WO 2021044441 A1 WO2021044441 A1 WO 2021044441A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- frames
- object clusters
- extracted
- user
- rendered
- Prior art date
Links
- 230000002452 interceptive effect Effects 0.000 title description 11
- 230000003190 augmentative effect Effects 0.000 title description 7
- 238000000034 method Methods 0.000 claims abstract description 69
- 238000009877 rendering Methods 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims description 111
- 230000015654 memory Effects 0.000 claims description 57
- 230000003993 interaction Effects 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 10
- 238000000605 extraction Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003068 static effect Effects 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 239000000872 buffer Substances 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000002159 nanocrystal Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23412—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
- H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
Definitions
- the disclosure relates to video content processing.
- the disclosure describes example techniques of processing video content to extract object clusters (e.g., important object clusters (IOCs)) from frames of video content in a way to maintain movement of the object clusters through the frames.
- object clusters e.g., important object clusters (IOCs)
- the extracted object clusters form a set of frames that capture the movement of the object clusters in the original (i.e., pre-extracted) frames of video content.
- the extracted object clusters may be less the content of the original frames of the video content.
- the extracted object clusters may be a video rendering of a real-life object or person.
- an object cluster may be an item or person that is recognized as a discrete item or a complete real-life representation of the person.
- the object cluster may be more than a shell or stick figure but rather the actual representation of the person including facial and body features.
- a server may extract and store the set of frames having the extracted object clusters for outputting to a computing device.
- the computing device may be configured to extract the object clusters to generate the set of frames having the extracted object clusters.
- the computing device may render one or more of the set of frames in a live camera feed of a device.
- the computing device may generate video content based on the rendered one or more frames and a user interacting with the extracted one or more object clusters.
- the computing device may display, as part of the live camera feed, the rendered one or more set of frames and the user interacting with the extracted one or more object clusters.
- the computing device may generate a recording of the rendered one or more set of frames and the user interacting with the extracted one or more object clusters.
- This disclosure describes example techniques rooted in technology that provide practical applications related to processing video content.
- the example techniques may be based on processing circuitry configured to extract the object clusters and computing devices configured to process and render the object clusters in a way that maintains the movement of the object clusters rather than using a static image of an object cluster.
- the disclosure describes a method of processing video content, the method comprising receiving a set of frames having extracted one or more object clusters, wherein the one or more object clusters are extracted from a plurality of frames, wherein at least one of the one or more object clusters is an object cluster with movement through the plurality of frames, rendering one or more of the set of frames in a live camera feed of a device, and generating video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters.
- the disclosure describes a device for processing video content, the device comprising a memory configured to store a set of frames having extracted one or more object clusters and processing circuitry, coupled to the memory, and configured to receive a set of frames having extracted one or more object clusters, wherein the one or more object clusters are extracted from a plurality of frames, wherein at least one of the one or more object clusters is an object cluster with movement through the plurality of frames, render one or more of the set of frames in a live camera feed of a device, and generate video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters.
- the disclosure describes a computer-readable storage medium storing instructions thereon that when executed cause one or more processors to receive a set of frames having extracted one or more object clusters, wherein the one or more object clusters are extracted from a plurality of frames, wherein at least one of the one or more object clusters is an object cluster with movement through the plurality of frames, render one or more of the set of frames in a live camera feed of a device, and generate video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters.
- FIG. 1 is a block diagram illustrating a system for processing video content in accordance with one or more example techniques described in this disclosure.
- FIG. 2 is a block diagram illustrating an example of a personal computing device configured to render object clusters in accordance with one or more example techniques described in this disclosure.
- FIG. 3 is a block diagram illustrating an example of processing circuitry configured to extract object clusters in accordance with one or more example techniques described in this disclosure.
- FIG. 4 is a flowchart illustrating an example of extracting object clusters in accordance with one or more example techniques described in this disclosure.
- FIGS. 5A-5C are conceptual diagrams illustrating example of extracting object clusters in accordance with one or more example techniques described in this disclosure.
- FIG. 6 is a flowchart illustrating an example of generating video content in accordance with one or more example techniques described in this disclosure.
- FIGS. 7A and 7B are conceptual diagrams illustrating example of generating video content in accordance with one or more example techniques described in this disclosure. DETAILED DESCRIPTION
- This disclosure describes example techniques that may provide interactive and real time video creation for digital users.
- the example techniques may be useable in a variety of ways and for different purposes. For ease of description only, the example techniques are described with respect to digital brands, but the techniques should not be considered limited to digital brands.
- a computing device may extract Important Object Clusters (“IOCs”) from frames of the video content.
- IOCs or object clusters more generally, include an item or person that is recognized as a discrete item or a complete real-life representation of the person.
- the object cluster may be more than a shell or stick figure but rather the actual representation of the person including facial and body features.
- the object clusters may be associated with movement through the frames of the video content.
- an object cluster may be a celebrity, and the frames of the video content may include a video sequence of the celebrity dancing.
- the computing device may extract the one or more object clusters from the video content to generate a set of frames having the extracted object clusters.
- the computing device may extract a celebrity dancing from campaign video footage of a brand to generate a set of frames having the celebrity dancing.
- the computing device may render the set of frames having the object clusters for display and/or recording.
- the receiving computing device may place the IOC (e.g., virtual IOC) for rendering into a live camera feed of the computing device (e.g., such as in an augmented reality (AR) environment in world space). This may allow the user of the receiving computing device to personalize videos in real-time with themselves interacting with the IOC.
- IOC e.g., virtual IOC
- AR augmented reality
- the digital consumption is slowly shifting to influencer-led social discovery where social influences push editorial/paid brand content to their followers.
- the example techniques described in this disclosure may allow for the influencer or celebrity to personalize their campaigns with their own life-size virtual renders (e.g., IOCs) that the users can interact with and drive organic video shares.
- the example techniques described in this disclosure may be usable for a variety of purposes.
- the example techniques may be applicable to two- dimensional (2D) video digital disclosure, ad and commerce markets, and provides a scalable approach to shift consumer experience from static passive viewership to interactive and personalized real-time user generated video engagement.
- the example techniques described in this disclosure may provide approaches to engage consumers where the consumers can create user generated videos (e.g., where the campaign content is built in for word-of- mouth awareness) and share to their social circles thereby driving valuable organic growth for the brand.
- the example techniques may be used for providing training or sselling product application demonstrations to the users with the help of interactive video content generated out of the IOCs.
- FIG. 1 is a block diagram illustrating a system 10 for processing video content in accordance with one or more example techniques described in this disclosure.
- system 10 includes personal computing device 12, network 14, and cloud 16.
- Cloud 16 includes network computing devices 18A-18N (collectively referred to as “network computing devices 18”).
- FIG. 1 illustrates an example system 10 that enables users to interact with object clusters of an existing video in a camera augmented reality (AR) friendly environment.
- AR augmented reality
- Examples of personal computing device 12 include mobile computing devices (e.g., tablets or smartphones), laptop or desktop computers, e-book readers, digital cameras, video gaming devices, and the like.
- a user of personal computing device 12 may access video content such as video content of an influencer or a brand.
- network computing devices 18A-18N in cloud 16 may store video content from different brands or influences for the user of personal computing device 12 to access and view.
- Cloud 16 represents a cloud infrastructure that supports multiple network computing devices 18 on which applications or operations requested by one or more users run.
- cloud 16 provides cloud computing for using network computing devices 18, hosted on network 14, to store, manage, and process data.
- Network 14 may transport data between network computing devices 18 of cloud 16 and personal computing device 12.
- network 14 may form part a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet.
- Network 14 may include routers, switches, base stations, or any other equipment that may be useful to facilitate data between personal computing device 12 and network computing devices 18.
- Examples of network computing devices 18 include server devices and/or storage devices. As one example, network computing devices 18 may be computing servers, but the example techniques are not so limited. Network computing devices 18 may be a combination of computing servers, web servers, database servers, and the like.
- processing circuitry of one or more of network computing devices 18 may execute an application that causes processing circuitry to extract object clusters from frames of video content stored on one or more network computing devices 18.
- one or more network computing devices 18 may output the frames of video content to personal computing device 12, and personal computing device 12 may be configured to extract object clusters from frames of video content.
- the example techniques described with respect to processing circuitry performing example techniques may be performed by processing circuitry of one or more of network computing devices 18, processing circuitry of personal computing device 12, or a combination of processing circuity of network computing devices 18 and personal computing device 12.
- some operations are described with respect to network computing devices 18, and some operations are described with respect to personal computing device 12.
- the example techniques should not be considered so limiting.
- processing circuitry may determine one or more object clusters (e.g., important object clusters (IOCs)) from a plurality of frames of video content (e.g., stored on network computing devices 18 or personal computing device 12). At least one of the one or more object clusters is an object cluster with movement through the plurality of frames (e.g., video content of a celebrity dancing).
- the processing circuitry may be configured to extract the determined one or more object clusters from the plurality of frames to generate a set of frames having extracted one or more object clusters.
- the processing circuitry may store the set of frames having the extracted one or more object clusters and output the set of frames having the extracted one or more object clusters (e.g., to personal computing device 12).
- processing circuitry may receive a set of frames having extracted one or more object clusters.
- the one or more object clusters are extracted from a plurality of frames, and at least one of the one or more object clusters is an object cluster with movement through the plurality of frames.
- the processing circuitry may render one or more of the set of frames in a live camera feed of a device and generate video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters.
- personal computing device 12 outputs video content that includes the user to network computing devices 18.
- Processing circuitry of network computing devices 18 may render one or more of the set of frames having the extracted one or more object clusters and generate the video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters.
- Processing circuitry of network computing devices 18 may output the generated video content back to personal computing device 12 for display to the user.
- personal computing device 12 may execute mobile Tenderer 44.
- network computing devices 18 and personal computing device 12 may perform the example techniques described in this disclosure.
- personal computing device 12 may request for the set of frames having the extracted one or more object clusters from network computing devices 18.
- the user of personal computing device 12, interacting with mobile renderer 44 may request for a particular object cluster from frames of video content (e.g., particular celebrity from a video).
- network computing devices 18 may have already stored the extracted one or more object clusters requested by the user via mobile renderer 44 (e.g., such as for frequently requested content).
- network computing devices 18 may dynamically generate the set of frames having the extracted one or more object clusters (e.g., where network computing devices 18 do not already store the extracted one or more object clusters).
- FIG. 2 is a block diagram illustrating an example of a personal computing device configured to render object clusters in accordance with one or more example techniques described in this disclosure.
- Examples of personal computing device 12 include a computer (e.g., personal computer, a desktop computer, or a laptop computer), a mobile device such as a tablet computer, a wireless communication device (such as, e.g., a mobile telephone, a cellular telephone, a satellite telephone, and/or a mobile telephone handset), a landline telephone, an Internet telephone, a handheld device such as a portable video game device or a personal digital assistant (PDA).
- Additional examples of person computing device 12 include a personal music player, a video player, a display device, a camera, a television, or any other type of device that processes and/or displays graphical data.
- personal computing device 12 includes lens 19 coupled to sensor 20, camera processor 22, a central processing unit (CPU) 24, a graphical processing unit (GPU) 28, memory controller 30 that provides access to system memory 32, user interface 34, display interface 36 that outputs signals that cause graphical data to be displayed on display 38, and video processor 39.
- Personal computing device 12 also includes transceiver 42, which may include wired or wireless communication links, to communicate with network 14 of FIG. 1.
- FIG. 2 illustrates lens 19 as part of the same device that includes GPU 28, the techniques described in this disclosure are not so limited.
- the shape and configuration of lens 19 is illustrated as one example, and should not be considered limiting.
- camera processor 22, CPU 24, GPU 28, display interface 36, and video processor 39 may be formed on a common integrated circuit (IC) chip.
- IC integrated circuit
- one or more of camera processor 22, CPU 24, GPU 28, display interface 36, and video processor 39 may be in separate IC chips.
- FIG. 2 may be formed as at least one of fixed- function or programmable circuitry such as in one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other equivalent integrated or discrete logic circuitry.
- ASICs application specific integrated circuits
- FPGAs field programmable gate arrays
- DSPs digital signal processors
- Bus 40 may be any of a variety of bus structures, such as a third generation bus (e.g., a HyperTransport bus or an InfiniBand bus), a second generation bus (e.g., an Advanced Graphics Port bus, a Peripheral Component Interconnect (PCI) Express bus, or an Advanced extensible Interface (AXI) bus) or another type of bus or device interconnect.
- a third generation bus e.g., a HyperTransport bus or an InfiniBand bus
- a second generation bus e.g., an Advanced Graphics Port bus, a Peripheral Component Interconnect (PCI) Express bus, or an Advanced extensible Interface (AXI) bus
- PCI Peripheral Component Interconnect
- AXI Advanced extensible Interface
- Camera processor 22 is configured to receive the electrical currents as sensor signals from respective pixels of sensor 20 and process the electrical currents to generate pixel data of images.
- camera processor 22 may be configured as a single-input-multiple- data (SIMD) architecture.
- SIMD single-input-multiple- data
- Camera processor 22 may perform the same operations on current received from each of the pixels on sensor 20.
- Each lane of the SIMD architecture may include an image pipeline.
- the image pipeline includes hardwire circuitry and/or programmable circuitry (e.g., at least one of fixed-function or programmable circuitry) to process the output of the pixels.
- Camera processor 22 outputs the resulting images (e.g., pixel values for each of the image pixels) to system memory 32 via memory controller 30.
- camera processor 22 may output the resulting images for display on display 38.
- CPU 24 may be a general-purpose or a special-purpose processor that controls operation of personal computing device 12.
- a user may provide input to personal computing device 12 to cause CPU 24 to execute one or more software applications.
- the software applications that execute on CPU 24 may include, for example, mobile Tenderer 44.
- CPU 24 is configured to execute mobile Tenderer 44.
- GPU 28 or other processing circuitry may be configured to execute mobile Tenderer 44.
- a user may provide input to personal computing device 12 via one or more input devices (not shown) such as a keyboard, a mouse, a microphone, touchscreen, a touch pad or another input device that is coupled to personal computing device 12 via user interface 34.
- input devices such as a keyboard, a mouse, a microphone, touchscreen, a touch pad or another input device that is coupled to personal computing device 12 via user interface 34.
- user interface 34 may be part of display 38.
- Display 38 may include a monitor, a television, a projection device, a liquid crystal display (LCD), a plasma display panel, a light emitting diode (LED) array, electronic paper, a surface-conduction electron-emitted display (SED), a laser television display, a nanocrystal display or another type of display unit.
- Display 38 may be integrated within personal computing device 12.
- display 38 may be a screen of a mobile telephone handset or a tablet computer.
- display 38 may be a stand-alone device coupled to personal computing device 12 via a wired or wireless communications link.
- display 38 may be a computer monitor or flat panel display connected to a personal computer via a cable or wireless link.
- Camera processor 22, CPU 24, GPU 28, and video processor 39 may store image data, and the like in respective buffers that are allocated within system memory 32.
- Display interface 36 may retrieve the data from system memory 32 and configure display 38 to display the image represented by the generated image data.
- display interface 36 may include a digital-to-analog converter (DAC) that is configured to convert the digital values retrieved from system memory 32 into an analog signal consumable by display 38.
- DAC digital-to-analog converter
- display interface 36 may pass the digital values directly to display 38 for processing.
- Memory controller 30 facilitates the transfer of data going into and out of system memory 32.
- memory controller 30 may receive memory read and write commands, and service such commands with respect to memory 32 in order to provide memory services for the components in personal computing device 12.
- Memory controller 30 is communicatively coupled to system memory 32.
- memory controller 30 is illustrated in the example of personal computing device 12 of FIG. 2 as being a processing circuit that is separate from both CPU 24 and system memory 32, in other examples, some or all of the functionality of memory controller 30 may be implemented on one or both of CPU 24 and system memory 32.
- System memory 32 may store program modules and/or instructions and/or data that are accessible by camera processor 22, CPU 24, GPU 28, and video processor 39.
- system memory 32 may store user applications (e.g., mobile Tenderer 44), resulting images from camera processor 22, rendered image content from GPU 28, rendered video content from video processor 39, etc.
- System memory 32 may additionally store information for use by and/or generated by other components of personal computing device 12.
- System memory 32 may include one or more volatile or non-volatile memories or storage devices, such as, for example, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, a magnetic data media or an optical storage media.
- RAM random access memory
- SRAM static RAM
- DRAM dynamic RAM
- ROM read-only memory
- EPROM erasable programmable ROM
- EEPROM electrically erasable programmable ROM
- flash memory a magnetic data media or an optical storage media.
- system memory 32 may include instructions that cause camera processor 22, CPU 24, GPU 28, display interface 36, and video processor 39 to perform the functions ascribed to these components in this disclosure. Accordingly, system memory 32 may be a computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors (e.g., camera processor 22, CPU 24, GPU 28, display interface 36, and video processor 39) to perform various functions.
- processors e.g., camera processor 22, CPU 24, GPU 28, display interface 36, and video processor 39
- system memory 32 is a non-transitory storage medium.
- the term “non-transitory” indicates that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term “non-transitory” should not be interpreted to mean that system memory 32 is non-movable or that its contents are static.
- system memory 32 may be removed from personal computing device 12, and moved to another device.
- memory, substantially similar to system memory 32 may be inserted into personal computing device 12.
- a non- transitory storage medium may store data that can, over time, change (e.g., in RAM).
- CPU 24 executes is mobile renderer 44.
- CPU 24 may cause transceiver 42 to output a request for a set of frames having extracted one or more object clusters.
- CPU 24 may receive the video content and perform the extraction of the one or more object clusters.
- Video processor 39 may be configured to reconstruct the set of frames having the one or more object clusters and store the result of the reconstruction as set of frames 45 in system memory 32.
- CPU 24 may cause display 38 to display one or more frames of set of frames 45, where set of frames 45 include the extracted one or more object clusters. In this way, video processor 39 may render one or more of the set of frames having the extracted one or more object clusters.
- CPU 24 may cause video processor 39 to render one or more of the set of frames having the extracted one or more object clusters in a live camera feed of personal computing device 12.
- GPU 28 may be configured to render one or more of the set of frames having the extracted one or more object clusters. For example, GPU 28 may retrieve set of frames 45 and render set of frames 45 for display. In such examples, CPU 24 may cause GPU 28 to render one or more of the set of frames having the extracted one or more object clusters in a live camera feed of personal computing device 12
- Personal computing device 12 may generate video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters.
- CPU 24 may cause display 38 to display, as part of the live camera feed of the device, the rendered one or more set of frames and the user interacting with the extracted one or more object clusters.
- CPU 24 may generate a recording of the rendered one or more set of frames and the user interacting with the extracted one or more object clusters for storage in system memory 32. In both examples, the result may be video where the user appears to be interacting with the one or more object clusters.
- the video content e.g., in live camera feed mode or as stored
- the object clusters moving in the AR environment but appear as real-life objects (although graphical objects are possible).
- the user would appear to be interacting with the object clusters.
- the example techniques may provide for a camera AR based interactive, personalized, and real-time video creation from existing videos.
- personal computing device 12 may generate information indicative of where the user is to be positioned for interaction with one or more object clusters.
- GPU 28 may generate image content that overlays an area where the user is to be positioned relative to the extracted one or more object clusters from the rendered one or more set of frames. Other ways of generating information indicative of where the user is to be positioned are possible.
- Personal computing device 12 may be considered as performing the following example operations.
- the list of frames (e.g., set of frames 45) that personal computing device 12 receives or generates contains the IOCs (possibly as manipulated by network computing devices 18), and can be placed in live camera feed (either in screen space or in world space depending on campaign theme) allowing the user to interact with the IOC.
- the user is first guided to position the camera and themselves relative to the IOC using an instruction overlay showing the floor space and where the user should stand relative to IOC. Once done, the user can record real-time videos with themselves interacting with the virtual IOCs placed next to them.
- additional visual effects can be built into the camera feed to drive awareness and association. Interaction support for AR friendly gestures etc. can be built in for ease of user.
- FIG. 3 is a block diagram illustrating an example of processing circuitry configured to extract object clusters in accordance with one or more example techniques described in this disclosure.
- FIG. 3 illustrates processing circuitry 46 and memory 48.
- Processing circuity 44 and memory 48 may be distributed across one or more network computing devices 18 within cloud 16.
- processing circuitry 46 and memory 48 may be the processing circuitry and memory of one of network computing devices 18.
- cloud 16 provides a computing infrastructure where operations are distributed across network computing devices 18. Accordingly, FIG. 3 may be considered as illustrating one or more network computing devices 18 that include processing circuitry 46 and memory 48.
- processing circuitry 46 is illustrated as executing extraction application 52.
- Transceiver 56 is circuitry with which processing circuitry 46 receives and transmits information.
- memory 48 is illustrated as storing plurality of frames 58 and set of frames having extracted one or more object clusters 60. However, it may be possible, based on control of cloud 16 or workload of network computing devices 18 that one or more of the applications shown as executing on processing circuitry 46 execute on different network computing devices 18. Also, it may be possible that different network computing devices 18 each store some of the information shown to be stored in memory 48.
- FIG. 3 captures these various examples where processing circuitry 46 and memory 48 are distributed among one or more network computing devices 18. As illustrated, processing circuitry 46 and memory 48 communicate via communication channel 50.
- Communication channel 50 may be a bus, similar to bus 40 of FIG. 1.
- communication channel 50 may be a packet- based network, such as a local area network, a wide-area network, or a global network such as the Internet.
- Processing circuitry 46 may be formed as at least one of fixed-function or programmable circuitry such as in one or more microprocessors, ASICs, FPGAs, DSPs, or other equivalent integrated or discrete logic circuitry.
- memory 48 may include one or more volatile or non-volatile memories or storage devices, such as, for example, RAM, SRAM, DRAM, ROM, EPROM, EEPROM, flash memory, a magnetic data media or an optical storage media.
- processing circuitry 46 may be configured to perform one or more example techniques described in this disclosure via fixed-function circuits, programmable circuits, or a combination thereof.
- Fixed- function circuits refer to circuits that provide particular functionality and are preset on the operations that can be performed.
- Programmable circuits refer to circuits that can programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For instance, programmable circuits may execute software or firmware that cause the programmable circuits to operate in the manner defined by instructions of the software or firmware.
- Fixed- function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations that the fixed-function circuits perform are generally immutable.
- the one or more of the units may be distinct circuit blocks (fixed-function or programmable), and in some examples, the one or more units may be integrated circuits.
- Processing circuitry 46 may include arithmetic logic units (ALUs), elementary function units (EFUs), digital circuits, analog circuits, and/or programmable cores, formed from programmable circuits.
- ALUs arithmetic logic units
- EFUs elementary function units
- digital circuits analog circuits
- programmable cores formed from programmable circuits.
- memory 48 may store the object code of the software that processing circuitry 46 receives and executes.
- Plurality of frames 58 may be configured to store complete video content (e.g., full sequence of pictures).
- Extraction application 52 may be configured to determine one or more object clusters from a plurality of frames 58 of the video content. As described, at least one of the one or more object clusters is an object cluster with movement through the plurality of frames 58. Extraction application 52 may extract the determined one or more object clusters from the plurality of frames to generate a set of frames having extracted one or more object clusters 60 for storage in memory 48.
- Transceiver 56 may be configured to output the set of frames having extracted one or more object clusters (e.g., to personal computing device 12).
- set of frames having extracted one or more object clusters 60 may have already been generated, such as where the video content is highly viewed.
- processing circuitry 46 may execute extraction application 52 where memory 48 does not already store the desired set of frames having extracted one or more object clusters 60.
- processing circuitry 46 may identify, with a convolutional neural network, the one or more object clusters and determine contours around the one or more identified object clusters. As one example, to identify the one or more object clusters, processing circuitry 46 may identify the one or more object clusters in an initial frame of the plurality of frames and propagate the identified one or more object clusters in the initial frame through remaining frames of the plurality of frames.
- processing circuitry 46 may determine contours around the one or more identified object clusters in the initial frame of the plurality of frames and determine contours around the one or more identified object clusters in the remaining frames of the plurality of frames based on the propagation of the identified one or more object clusters in the initial frame through the remaining frames of the plurality of frames.
- processing circuitry 46 may discard content other than the determined one or more object clusters from the plurality of frames to generate the set of frames having the extracted one or more object clusters. Moreover, in some examples, processing circuitry 46 may compress the set of frames having the extracted one or more object clusters. In such examples, outputting the set of frames includes transceiver 56 outputting the compressed set of frames.
- Processing circuitry 46 may be configured to perform the following example operations. Processing circuitry 46 extract out N number of frames from the input video’s segment of interest (e.g., extract N frames from video content to generate plurality of frames 58). In some examples, the N frames to extract may be pre-defined by the brand creative team in-line with their existing campaign theme.
- Processing circuitry 46 may process the initial frame through an Automated Segmentation Application which is a Convolutional Neural Network (CNN) trained to identify predefined objects and draw identification contours around the IOCs.
- CNN Convolutional Neural Network
- processing circuitry 46 fine-tune the detected and undetected IOCs (although such operations of fine-tuning are not necessary in all examples).
- the detected segmentation is propagated automatically to detect IOCs in the following frames through a CNN module and extract the IOCs out.
- the list of frames e.g., set of frames having extracted one or more object clusters 60
- processing circuitry 46 may apply video compression of frames to ensure responsive in-camera experience.
- the frames from typical high-resolution video shoots are large in size.
- the file download size may be optimized as much as possible, while ensuring the visual experience quality is not impacted significantly.
- Existing techniques to reduce the file size of these frames may be used. For example, by selectively decreasing the number of colors in the image, there may be fewer bytes that are required to be stored. While keep the visual effect to minimal, the file size may be lowered (-70% reduction) per frame.
- set of frames having extracted one or more object clusters 60 may be compressed frames.
- processing circuitry 46 may perform some image manipulation. For instance, the list of frames (e.g., set of frames having extracted one or more object clusters 60) may now contain only the raw IOCs. In some examples, depending on the context, processing circuitry 46 may use Deep Learning (Generative Adversarial Networks, “GAN”) to manipulate the frames to suite a requirement. For example, it may be possible to change the apparel worn by the celebrity in the input frame sequence (e.g., plurality of frames 58) to a live brand apparel (from a 2D apparel image) that the brand wants to promote as part of the campaign.
- GAN Deep Learning
- FIG. 4 is a flowchart illustrating an example of extracting object clusters in accordance with one or more example techniques described in this disclosure.
- FIG. 4 is described with respect to processing circuitry 46 of FIG. 3.
- the example techniques although described with respect to FIG. 3 may be performed on personal computing device 12 of FIG. 2, or via some combination of processing circuitry 46 of FIG. 3 and personal computing device 12.
- FIGS. 5A-5C are conceptual diagrams illustrating example of extracting object clusters in accordance with one or more example techniques described in this disclosure.
- Processing circuitry 46 may be configured to determine one or more object clusters from a plurality of frames of video content (70).
- FIG. 5A illustrates frames 76A-76D, which are examples of plurality of frames 58 of FIG. 3.
- Each of frames 76A-76D may include one or more object clusters.
- frame 76A includes object cluster 78A (e.g., a person) and object cluster 80A (e.g., a tree).
- object cluster 80A e.g., a tree
- each one of frames 76B-76D may similarly include respective object clusters (e.g., object cluster 78B-78D and object cluster 80B-80D).
- At least one of the one or more object clusters is an object cluster with movement through the plurality of frames 76A-76D.
- object cluster 80A-80D may be static through frames 76A-76D.
- the person identified by object cluster 78A may move between frames 76A-76D (e.g., by dancing). Accordingly, the position of object clusters 78A-78D may be different relative to one another (e.g., different positions in the dance).
- object clusters 78A-78D may represent an object cluster with movement through frames 76A-76D.
- FIG. 5 A illustrates two object clusters 78A and 80A, there may be many object clusters in frames 76A-76D, and processing circuitry 46 may determine any one or more of the object clusters.
- processing circuitry 46 may determine object clusters 78A and 80A.
- a developer may select object cluster 78A in frame 76A, and processing circuitry 46 may determine object cluster 78A based on developer selection.
- the information used to generate frames 76A-76D may include metadata that identifies the location of object cluster 78 A.
- processing circuitry 46 may identify, with a convolutional neural network, the one or more object clusters and determine contours around the one or more identified object clusters.
- processing circuitry 46 may identify the one or more object clusters (e.g., object cluster 78A) in initial frame 76A of the plurality of frames and propagate the identified one or more object clusters in the initial frame 76A through remaining frames 76B- 76D of the plurality of frames 76A-76D.
- processing circuitry 46 may determine contours around the one or more identified object clusters in frame 76A and determine contours around the one or more identified object clusters in the remaining frames 76B-76D based on the propagation of the identified one or more object clusters in the initial frame 76 A through the remaining frames 76B-76D.
- Processing circuitry 46 may extract determined object clusters from the plurality of frames to generate set of frames having extracted object clusters (72). As one example, processing circuitry 46 may execute extraction application 52 to extract the determined object clusters.
- processing circuitry 46 may determine a bounding box 82 around object cluster 78 A. Although not shown, there may be bounding box 82 around each of object clusters 78B-78D in frames 76A-76D. In examples where object cluster 78A is moving locations between frames 76A-76D, the location of bounding box 82 may similarly move to cover object clusters 78B-78D. [0079] Processing circuitry 46 may extract the image content within bounding box 82 to generate set of frames 84A-84D, of FIG. 5C, having extracted object clusters. Frames 84A- 84D are examples of set of frames having one or more object clusters 60 of FIG. 3.
- processing circuitry 46 may discard content other than the determined one or more object clusters from the plurality of frames 76A-76D to generate the set of frames 84A-84D having the extracted one or more object clusters.
- Processing circuitry 46 may output set of frames having extracted object clusters (74). For example, processing circuitry 46 may cause transceiver 56 to output frames 84A-84D for personal computing device 12 to eventually receive and process.
- FIG. 6 is a flowchart illustrating an example of generating video content in accordance with one or more example techniques described in this disclosure.
- FIG. 6 is described with respect to processing circuitry of personal computing device 12 of FIG. 2, where examples of the processing circuitry include one or more of CPU 24, GPU 28, and video processor 39, as well as other example components of personal computing device 12.
- the example techniques although described with respect to FIG. 2 may be performed by processing circuitry 26 of FIG. 3, or via some combination of processing circuitry 46 of FIG. 3 and personal computing device 12.
- FIGS. 7A and 7B FIGS.
- the processing circuitry of personal computing device 12 may receive a set of frames having extracted one or more object clusters (90).
- CPU 24 may receive frames 84A-84D of FIG. 5C that include object cluster 78 for storage in memory 32.
- frames 84A-84D may be an example of set of frames 45 of FIG. 2.
- the one or more clusters that are in the set of frames received by personal computing device 12 are extracted from a plurality of frames (e.g., frames 76A-76D of FIG. 5A).
- At least one of the one or more object clusters is an object cluster with movement through the plurality of frames.
- object cluster 78A may represent a person that is dancing, and therefore, there is movement of object cluster 78A through frames 76A-76D (e.g., position and/or location of object cluster 78A is different than position and/or location of object cluster 78B, which is different than position and/or location of object cluster 78C, and so forth).
- the processing circuitry of personal computing device 12 may render one or more of the set of frames in a live camera feed of personal computing device 12 (92).
- personal computing device 12 may be in a live camera feed mode where a user desires to take a video.
- FIG. 7A illustrates display 38, and display 38 may be displaying image content that is captured by sensor 20 and camera processor 22.
- video processor 39 and/or GPU 28 may render frames 84A-84D in the live camera feed.
- FIG. 7A illustrates object cluster 96 which is an example instance of one of object clusters 78A-78D.
- the processing circuitry may generate video content based on the rendered one or more set of frames and a user interacting with the extracted one or more object clusters (94). For example, as illustrated in FIG. 7B, the user of personal computing device 12 may ask a friend (e.g., person 98) to enter the live camera feed so that sensor 20 and camera processor 22 capture the friend (e.g., generate image content of the friend for display). As another example, the user of personal computing device 12 may put personal computing device 12 on a stand and enter the live camera feed. In either example, the user or the friend is an example of a user (e.g., person 98) that is interacting with the extracted one or more object clusters (e.g., object cluster 96). For instance, person 98 may dance with object cluster 96, may act like he or she is having a conversation with object cluster 96, and so forth.
- a friend e.g., person 98
- the processing circuitry may display, as part of the live camera feed, the rendered one or more set of frames and the user interacting with the extracted one or more object clusters (e.g., as illustrated in FIG. 7B).
- the processing circuitry may generate a recording of the rendered one or more set of frames and the user interacting with the extracted one or more object clusters.
- the processing circuitry may generate information indicative of where the user is to be positioned for interaction with the one or more object clusters.
- GPU 28 may render a graphical box 100. Person 98 may move within the live camera feed until person 98 is located within graphical box 100.
- graphical box 100 may be image content that overlays an area where person 98 is to be positioned relative to the extracted one or more object clusters 96 from the rendered one or more set of frames 84A-84D.
- the processing circuitry may generate video content based on the rendered one or more set of frames 84A-84D and the user (e.g., person 98) interacting with the extracted one or more object clusters (e.g., object cluster 96) at a position based on the generated information indicative of where the user is to be positioned (e.g., graphical box 100).
- the processing circuitry may render one or more of the set of frames in the live camera feed. However, to ensure that the rendering appears realistic, the processing circuitry may need to scale the one or more set of frames. For instance, without scaling the one or more object clusters may appear to be too big relative to person 98 or may appear further back or in front of person 98.
- the processing circuitry may determine at least one of a size and relative depth of the user (e.g., person 98) in the live camera feed (e.g., based on graphical box 100 as one example).
- the processing circuitry may scale the extracted one or more objects (e.g., object 96) based on at least one of the determined size and relative depth of the user to generate a scaled set of frames.
- the processing circuitry may render one or more of the scaled set of frames.
- the processing circuitry may scale.
- the processing circuitry may utilize plane tracking. In plane tracking, the floor in the user environment is detected from the camera feed and then the extracted object cluster is placed in life-size form on the detected floor.
- the processing circuitry may utilize world space scaling. In world space scaling, the extracted object cluster is placed in the user environment via the camera feed and the user has the ability to scale it appropriately in order to place it wherever they want.
- a user may desire to add animation.
- the user may wish to add animation that shows that the user and object cluster are dancing in the rain.
- the processing circuitry may render animation with one or more set of frames in the live camera feed.
- FIG. 7B illustrates animation 102, which is an animation rendered by GPU 28 of rain.
- the processing circuitry may generate the video content based on the rendered one or more frames, the rendered animation, and the user interacting with the extracted one or more object clusters.
- processors may generally refer to any of the foregoing logic circuitry, alone or in combination with other logic circuitry, or any other equivalent circuitry, and alone or in combination with other digital or analog circuitry.
- At least some of the functionality ascribed to the systems and devices described in this disclosure may be embodied as instructions on a computer-readable storage medium such as RAM, ROM, non-volatile random access memory (NVRAM), EEPROM, FLASH memory, magnetic media, optical media, or the like that is tangible.
- the computer-readable storage media may be referred to as non-transitory.
- a server, client computing device, or any other computing device may also contain a more portable removable memory type to enable easy data transfer or offline data analysis.
- the instructions may be executed to support one or more aspects of the functionality described in this disclosure.
- a computer-readable storage medium comprises non-transitory medium.
- the term “non-transitory” may indicate that the storage medium is not embodied in a carrier wave or a propagated signal.
- a non-transitory storage medium may store data that can, over time, change (e.g., in RAM or cache).
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Strategic Management (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Marketing (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Human Computer Interaction (AREA)
- Economics (AREA)
- General Business, Economics & Management (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IN201941035434 | 2019-09-03 | ||
IN201941035434 | 2019-09-03 | ||
US17/009,498 | 2020-09-01 | ||
US17/009,498 US11451721B2 (en) | 2019-09-03 | 2020-09-01 | Interactive augmented reality (AR) based video creation from existing video |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021044441A1 true WO2021044441A1 (en) | 2021-03-11 |
Family
ID=74852574
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IN2020/050769 WO2021044441A1 (en) | 2019-09-03 | 2020-09-03 | Interactive augmented reality (ar) based video creation from existing video |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2021044441A1 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103929653A (en) * | 2014-04-30 | 2014-07-16 | 成都理想境界科技有限公司 | Enhanced real video generator and player, generating method of generator and playing method of player |
US20160180590A1 (en) * | 2014-12-23 | 2016-06-23 | Lntel Corporation | Systems and methods for contextually augmented video creation and sharing |
US20170193686A1 (en) * | 2015-12-30 | 2017-07-06 | Daqri, Llc | 3d video reconstruction system |
US10297085B2 (en) * | 2016-09-28 | 2019-05-21 | Intel Corporation | Augmented reality creations with interactive behavior and modality assignments |
-
2020
- 2020-09-03 WO PCT/IN2020/050769 patent/WO2021044441A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103929653A (en) * | 2014-04-30 | 2014-07-16 | 成都理想境界科技有限公司 | Enhanced real video generator and player, generating method of generator and playing method of player |
US20160180590A1 (en) * | 2014-12-23 | 2016-06-23 | Lntel Corporation | Systems and methods for contextually augmented video creation and sharing |
US20170193686A1 (en) * | 2015-12-30 | 2017-07-06 | Daqri, Llc | 3d video reconstruction system |
US10297085B2 (en) * | 2016-09-28 | 2019-05-21 | Intel Corporation | Augmented reality creations with interactive behavior and modality assignments |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10016679B2 (en) | Multiple frame distributed rendering of interactive content | |
US11043031B2 (en) | Content display property management | |
WO2021047396A1 (en) | Image processing method and apparatus, electronic device and computer-readable storage medium | |
US11393176B2 (en) | Video tools for mobile rendered augmented reality game | |
KR102107709B1 (en) | Spatial transformer modules | |
EP3274966B1 (en) | Facilitating true three-dimensional virtual representation of real objects using dynamic three-dimensional shapes | |
CN110058685B (en) | Virtual object display method and device, electronic equipment and computer-readable storage medium | |
CN105051792B (en) | Equipment for using depth map and light source to synthesize enhancing 3D rendering | |
US10891781B2 (en) | Methods and systems for rendering frames based on virtual entity description frames | |
US9679416B2 (en) | Content creation tool | |
CN106575445B (en) | Fur avatar animation | |
EP2972950B1 (en) | Segmentation of content delivery | |
CN113228625A (en) | Video conference supporting composite video streams | |
CN111476851B (en) | Image processing method, device, electronic equipment and storage medium | |
US20170213394A1 (en) | Environmentally mapped virtualization mechanism | |
US11451721B2 (en) | Interactive augmented reality (AR) based video creation from existing video | |
US20160350955A1 (en) | Image processing method and device | |
US11961190B2 (en) | Content distribution system, content distribution method, and content distribution program | |
WO2021044441A1 (en) | Interactive augmented reality (ar) based video creation from existing video | |
US20240015263A1 (en) | Methods and apparatus to provide remote telepresence communication | |
CN114049857B (en) | Naked eye 3D electronic relic display stand displayed by Yu Wenbo | |
GB2573328A (en) | A method and apparatus for generating a composite image | |
US20180365268A1 (en) | Data structure, system and method for interactive media | |
KR20240115657A (en) | Electronic apparatus and control method thereof | |
CN111640179A (en) | Display method, device and equipment of pet model and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20861833 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20861833 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20861833 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.03.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20861833 Country of ref document: EP Kind code of ref document: A1 |