WO2018203115A1 - Génération de variations d'infographie à partir de formats de fichier intermédiaire à variabilité limitée, comprenant la génération de différents aspects de jeu ou résultats de jeu - Google Patents
Génération de variations d'infographie à partir de formats de fichier intermédiaire à variabilité limitée, comprenant la génération de différents aspects de jeu ou résultats de jeu Download PDFInfo
- Publication number
- WO2018203115A1 WO2018203115A1 PCT/IB2017/052820 IB2017052820W WO2018203115A1 WO 2018203115 A1 WO2018203115 A1 WO 2018203115A1 IB 2017052820 W IB2017052820 W IB 2017052820W WO 2018203115 A1 WO2018203115 A1 WO 2018203115A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- game
- video segment
- rendered video
- texture data
- chance
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 62
- 230000008569 process Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 abstract description 82
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 239000002131 composite material Substances 0.000 description 18
- 241000283086 Equidae Species 0.000 description 17
- 239000003086 colorant Substances 0.000 description 14
- 230000003068 static effect Effects 0.000 description 11
- 230000008901 benefit Effects 0.000 description 8
- 230000017531 blood circulation Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000009877 rendering Methods 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 6
- 238000007792 addition Methods 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000002059 diagnostic imaging Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 102100039169 [Pyruvate dehydrogenase [acetyl-transferring]]-phosphatase 1, mitochondrial Human genes 0.000 description 1
- 101710126534 [Pyruvate dehydrogenase [acetyl-transferring]]-phosphatase 1, mitochondrial Proteins 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- OEBRKCOSUFCWJD-UHFFFAOYSA-N dichlorvos Chemical compound COP(=O)(OC)OC=C(Cl)Cl OEBRKCOSUFCWJD-UHFFFAOYSA-N 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003116 impacting effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000004020 luminiscence type Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002250 progressing effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
- A63F13/35—Details of game servers
- A63F13/355—Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/25—Output arrangements for video game devices
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/53—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- A63F13/65—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- A63F13/655—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition by importing photos, e.g. of the player
Definitions
- the present invention relates generally to computer graphics, such as: video games, medical imaging, digital watermarking and the like. More particularly, this invention relates to methods for greatly enhancing the variability of computer graphics from limited sources utilizing minimal processing and bandwidth. Specifically, this innovation resolves the problem of the generation of a video file or stream in real time producing complex video output consisting of realistic models of three-dimensional (3D) objects with varying backdrop scenes.
- 3D three-dimensional
- Another example is for the photorealistic looking 3D graphics to be pre- rendered and downloaded to the consumer's device in advance of interactive play, thereby maintaining high levels of computational processing at a central site while at the same time reducing bandwidth requirements since the graphics could be downloaded in time multiples of their real time playback.
- the typical pre-rendered 3D graphics large file size severely restricts the variety of options available for real time game playback thereby significantly reducing consumer enjoyment of multiple plays since the video will be familiar after only a few viewings.
- pre-rendered graphics digital watermarking is well known in the art and is often utilized to ensure copyright protection as well as embedding other metadata.
- U.S. Patent Application Publication No. 2008/0012870 discloses analyzing each frame of a video stream and related color profile indicating parameters of the color space of the video stream and then converting each frame from a source color space to a working color space.
- the intent of this process is to convert in real time a video stream from one format to another— e.g., composite NTSC (National Television System Committee) format to HDTV (High-Definition Television) format (paragraph [0024] of "Gies”).
- NTSC National Television System Committee
- HDTV High-Definition Television
- Color space conversion is also taught in U.S. Patent No. 7,593,021 (Tynefield, Jr. et al). In this process, color data is converted from one space to another with a driver determining if a color conversion function will be enabled and if so either with a single linear conversion or flagged to perform a non-linear conversion. Again, while this process discloses efficient methods of altering a video stream or file, it does not address the core problems of source computational power and bandwidth tradeoffs nor how to maintain photorealistic looking graphics on consumer devices with variety and dynamic outcomes determined in near real time.
- U.S. Patent No. 7,038,690 discloses methods and systems for interfaces between video applications and consumer display screens that allow applications to intelligently use display resources of the consumer's device with a graphics arbiter providing display environment information to the video application, possibly transforming the video stream or file or allowing another application to transform the video stream or file. With "Wilt” the graphics arbiter also informs applications of the estimated time when the next frame will be displayed on the screen, thereby allowing applications to tailor output to the estimated display time, thus improving output quality while decreasing resource waste by avoiding the production of "extra" frames that will never be viewed.
- the variation is processed on the consumer's device thereby also reducing bandwidth requirements.
- real world, high-resolution, digital images taken from appropriate locations are saved as backdrop images and then modified by superimposing multiplicities of 3D virtual or real world images over the saved backdrop.
- the real-world backdrop images are formatted into a 3D world model with a moving scene produced (i.e., "backdrop scene") from the formatted 3D model, which is generic regardless of the 3D images superimposed over the backdrop scene.
- multiple backdrop scenes of the same location can also be recorded (e.g., varying weather conditions, different times of day, different types of scans of the same body) and formatted into associated 3D world models thereby increasing the variety.
- the various backdrop scenes can be optionally downloaded to the consumer or user's device in advance of a real time video feed, which would only contain superimposed multiplicities of 3D overlay images.
- these embodiments have the advantage of reducing both central site processing and bandwidth requirements by having the user or consumer's device perform all of the a priori backdrop scene processing.
- the 3D overlay images can be downloaded in advance with the 3D world models sans animation. This embodiment reduces real time bandwidth requirements by pre-downloading the backdrop and overlay models with only the animation (i.e., instructions for how the models will be displayed and interact) and other related formatting data transmitted in real time.
- the 3D images superimposed over the backdrop scene are rendered in significant detail such that their lighting and texture information is compatible to the underlying backdrop scene.
- the associated texture data of the virtual images are exported into a new file format, which corresponds precisely to the backdrop scene previously saved.
- These virtual image files are effectively video frames of Partially Rendered Pixels (PRP), lacking only texture
- the PRP data is either combined with the backdrop scene at the central site or the consumer or user's device. In either case, this final “assembly" stage combines both the backdrop scene and the PRP data blending the two with a combination of texture maps.
- this increase in variability can include a portion processed by the consumer's device thereby also reducing bandwidth requirements.
- optional elements of PRP texture data can be omitted or replaced in real time to add variation to the displayed video.
- complex shadow information may be added to PRP data to be compatible with nighttime backdrop scenes, but omitted during daytime scenes; or, portions of real world backdrop scenes may be overwritten with PRP data signage; or, digital watermark game features may be embedded in the Least Significant Bits (LSBs) of each PRP data video frame overwriting a portion of actual video data but not to the level where video quality is significantly degraded; or, blood flow portions of medical scan models may include PRP data thereby enabling false colors (e.g., Doppler blood flow) to be added in real time.
- LSBs Least Significant Bits
- PRP generated video texture data is stored in a custom format consisting of one or many files.
- the PRP video format of this embodiment includes variables that can be substituted for values in real time. When video playback is executed, the color variables are replaced in real time with a fixed value from an associated texture data table or array.
- this embodiment has the advantage of producing a potentially large variety of different appearing videos from a relatively small set of base videos with minimal processing by the device executing the display. These changes in appearance may be utilized to simply enhance visual variety or vary the apparent outcome of a game or appearance of the medical scan.
- the colors of the "silks" of virtual jockey and horses can be varied from race to race thereby altering the appearance of the virtual race outcome, or the color coded velocity mapping of Doppler blood flow in a patient's medical scan can be changed as new data becomes available.
- FIG. 1A is a swim lane flowchart of a first representative example of a real time pre-processing high quality graphics generator with distributed processing
- FIG. IB is a continuation swim lane flowchart of FIG. 1A;
- FIG. 2 A is a system architecture diagram corresponding to FIG. 1A;
- FIG. 2B is a system architecture diagram corresponding to FIG. IB;
- FIG. 3A is a representative example of a high-resolution, digital backdrop image taken from a real world race track
- FIG. 3B is a first representative example of a high-resolution, digital backdrop image taken from a real world race track of FIG. 3 A with multiple static 3D virtual images embedded in the background model;
- FIG. 3C is a second representative example of the high-resolution, digital backdrop image taken from a real world race track of FIG. 3 A and FIG. 3B with multiplicities of dynamic virtual images superimposed over the backdrop as an added layer;
- FIG. 3D is a third representative example of the high- resolution, digital backdrop image taken from a real world race track of FIG. 3 A with the static images of FIG. 3B and dynamic overlays of FIG. 3C with the addition of multiplicities of dynamic 3D PRP virtual animated characters or images superimposed over the backdrop without any reference to an associated PRP lookup table;
- FIG. 3E is a fourth representative example of the high-resolution, digital backdrop image taken from a real world race track of FIG. 3 A with the same multiplicities of 3D virtual animated images superimposed as FIG. 3D, but with the color and texture variables of the "silks" of the virtual jockeys and horses augmented with a PRP color and texture table reference;
- FIG. 3F is a fifth representative example of the high-resolution, digital backdrop composite image taken from a real world race track of FIG. 3 A with the dynamic overlays of FIG. 3C and the dynamic 3D PRP virtual animated images or characters
- FIG. 4A is a representative example of dynamic 3D PRP virtual animated characters with one texture data lookup table.
- FIG. 4B is a second representative example of the same set of dynamic 3D PRP virtual animated characters of FIG. 4A albeit with an alternative texture data lookup table.
- FIG. 5 is an illustrative example of the user interface enabling control of the video's appearances of FIGS. 3B through FIG. 3F;
- FIG. 6 is a code snippet example illustrating the PRP file format and types used in PRP lookup tables.
- a "group” can be defined as a collection of digital processing or scanning at a conceptual location.
- each "group” could be a separate geographical location hosting a defined set of digital processing functionality.
- a "group” can also be a logical collection of digital devices (e.g., servers, cameras, scanners) dispersed over multiple geographic locations with the shared functionality maintained by network connections.
- digital devices e.g., servers, cameras, scanners
- Partially Rendered Pixels or “PRP” refers to pixels typically imaged on the surface of computationally complex 3D skeletal elements with texture information saved in a separate file that can be referenced and executed in real time.
- texture data refers to any additional data necessary to render a PRP generated character or model in photorealistic or near photorealistic detail (e.g., color, shadow, multi-shadow, luminance, optional surface patterns).
- Coordinated groupings of "texture data” where multiplicities of different data (e.g., related colors, surface patterns) combine to create an overall effect (e.g., tattoo that appears on a 3D virtual image with the correct grouping of "texture data”) are referred to as a "set of texture data.”
- “Texture data” may also include metadata that is associated with a PRP generated character that controls some aspects of "game features” or "game appearance.” Similar to a "set of texture data,” a collection of metadata combined to create an overall effect is referred to as a "set of metadata.”
- Game features refers to components that affect the response, feel, operation, or configuration of a digital game file when executed. More specifically, “game appearance” is similar to “game features”, but as used herein specifically refers to components only altering a games' visual display. “Game features” may include altering visual (i.e., appearance) data, such as changing the color of "silks” on jockeys and horses in a virtual horse race to change the apparent outcome of a race. "Game features” also refer to metadata (e.g., sound, digital watermark, game flags, key words, description) that does not necessarily impact the visual appearance of the game. Texture data thus control or affect game features, some of which determine the visual appearance of the game and some of which determine non-visual aspects of the game.
- fully rendered video segment refers to a video animation as displayed to the human player or user depicting a complete game or analysis showing all the relative information necessary to determine the game outcome or to conduct a complete analysis.
- partially rendered video segment refers to a core or fundamental video animation that is incomplete and does not include all of the "texture data” (e.g., colors) necessary for a fully rendered video segment.
- both fully and partially rendered video segments include various animated “elements” (e.g., horses, boxers, athletes) that are comprised of computationally complex 3D skeletal characters with texture information saved in a separate file that can alter the 3D skeletal characters' appearance and can be optionally executed in real time.
- variable features describes features or aspects of animated elements that can be changed with minimal computational burden by changing the texture data PRP reference file.
- FIGS. 1A and IB taken together, illustrate one embodiment of generating high quality graphics in real time while reducing processing burdens utilizing a distributed processing system.
- this one embodiment of the invention is conceptually divided into four groups (i.e., Real World Scan 101, Offline Processing 102, Real Time Processing 150, and User Device 151) by the four "swim lane" columns as shown in the two figures. If a particular flowchart function appears completely within a swim lane, its functionality is limited to the data category of the associated swim lane— e.g., Real World Scene(s) 105 is/are exclusively within Real World Scan group 101.
- FIGS. 1 A and 2A as well as FIGS. IB and 2B, each pair taken together represent portions of the same embodiment's system level flowchart and associated hardware architecture diagrams.
- FIG. 1A swim lane flowchart 100 begins with recording Real World Scan 101 data 104 from various sources— e.g., horse track, boxing ring, soccer stadium, human body scans. This 3D recorded real world data is then saved as various scenes 105. The number of scenes recorded will vary depending on the application, with the intent to provide a seamless backdrop model suitable for superimposing multiplicities of 3D virtual images with the saved backdrop. Regardless of how the scenes were obtained, the resulting 3D backdrop model should be of sufficient complexity to allow dynamic selection of any portion of the 3D backdrop to permit animation superimposed on a smooth flowing backdrop.
- sources e.g., horse track, boxing ring, soccer stadium, human body scans.
- This 3D recorded real world data is then saved as various scenes 105.
- the number of scenes recorded will vary depending on the application, with the intent to provide a seamless backdrop model suitable for superimposing multiplicities of 3D virtual images with the saved backdrop. Regardless of how the scenes were obtained, the resulting 3D backdrop model should be of sufficient complexity to
- the real world scans or scenes can be replaced with virtual (i.e., computer generated) backdrop scenes separately generated 106.
- the Offline Processing 102 input buffer 107 receives the backdrop scenes or images along with the 3D overlay virtual image models 106 typically rendered as Partially Rendered Pixels (PRP).
- the reformatted overlay PRP data 108 and backdrop display data 109 are then saved for further processing when required by the Real Time Processing 150 group as illustrated in the continuation of this embodiment 100' in FIG. IB— see off-page connection "A" (110 and 110') for the display data 109 and off-page connection "B" (111 and 111 ') for the PRP data 108.
- the display data 109 is received in the continuation of the embodiment 100' (FIG. IB) via off-page connection "A" 110' with the PRP data 108 received via off-page connection "B" 111 '.
- the Real Time Processing 150 group's Input/Output (I/O) 153 processes the previously scanned or generated backdrop model and 3D virtual PRP images superimposed on the backdrop for either animated or on-demand freeze frame displays in compliance with the specifications 152.
- the resulting 3D PRP overlay images are generated in significant detail with (if appropriate) lighting and texture information selected to match the underlying backdrop scene. Additionally, the overlay 3D PRP images are generated to be compatible with the intended final application and associated backdrop.
- the overlay 3D virtual images will respond to either a game outcome as determined by a random process (e.g., a Random Number Generator— "RNG”, physical ball drop machine, thereby making the game a game of chance), or embed variables such that the overlay 3D virtual images will respond to player input (e.g., first person shooting video game where a player chooses a uniform; poker game where a player buy in or betting patterns alters the color of chips) in real time.
- a random process is determining the outcome of a game where various wagers were collected, for security reasons against insider fraud it is preferable that the random process be executed after the period for accepting wagers (i.e., the betting period) has expired.
- the overlay images may respond to scanned data from different sources—e.g., vector blood flow Doppler data from sonograms and CAT (Computerized Axial Tomography) scans.
- the backdrop and the overlay 3D PRP virtual images can be optionally downloaded to the User Device's 156 nonvolatile memory 157 for playback at a later date or alternatively sent directly to the animator 154 algorithm.
- the backdrop and the overlay 3D PRP virtual images can be optionally downloaded directly to the animator 154 algorithm or alternatively to a User Device 156 at a remote location.
- the remotely located User Device 156 is also interchangeably referred to herein as a "local computer device," which in one preferred embodiment is a mobile device.
- the User Device's 156 nonvolatile memory 157 will store the backdrop and the overlay 3D PRP virtual images for playback and display at a remote location later.
- the texture information is removed leaving only detailed 3D skeletal element's information about the PRP virtual model 108.
- the removed texture data is then preferably exported into a new file format, which corresponds precisely to the associated backdrop scene.
- These files are effectively multiplicities of video frames of PRP data - i.e., pixels lacking only source texture information, and possibly animation details.
- the colors of the "silks" of virtual jockey and horses can be varied from race to race thereby altering the virtual race outcome; or the color coded velocity mapping of Doppler blood flow model of a patient's medical scan can be changed as to only illustrate blood flows of a specified velocity range or blood containing a medical taggant (e.g., radioisotope) or some combination thereof; or digital watermark data can be embedded in each pixel's LSB, or sounds associated with a specific character can be embedded.
- the PRP formatted files can be saved at the Offline Processing 102 group's nonvolatile memory 108 or alternatively exported to the User Device 151 nonvolatile memory 157 for later playback.
- an animator algorithm 154 specifies where the overlay 3D virtual images will reside relative to the backdrop model and assembles, with optional animation, the 3D backdrop model with the overlay 3D virtual images.
- static signage may be superimposed over a portion of the backdrop model (e.g., 312 of FIG. 3B) as well as dynamically animated virtual figures (e.g., virtual horses and jockeys 337 of FIG. 3D).
- the output of the animator 154 (FIG. IB) is a customized and portable DLL (Dynamic Link Library) that can be implemented with the Real Time Processing 150 group formatting of the digital stream 155 or, alternatively, executed on the user display device 156.
- DLL Dynamic Link Library
- the animator 154 generated DLL will execute and display the composite images on the User Device 151 screen 156.
- the entire composite video stream can be downloaded from the Real Time processing group's 150 formatter 155 to the User Device 151 group's display screens 156 in real time or to the User Device 151 group's nonvolatile memory 157 for later playback without alteration.
- UDP/IP User Datagram Protocol/Internet Protocol
- UDP/IP User Datagram Protocol/Internet Protocol
- Downloading of digital data formatted as digital files could theoretically have the advantage of requiring lesser bandwidth due to transmission of the file in multiplies of real time playback time with actual playback triggered separately or having portions of the 3D images downloaded ahead of the real time broadcast with the composite image being generated in real time on the display 156 device.
- Examples of transmission formats suitable for digital data arranged as digital files would be TCP/IP ("Transmission Control Protocol/Internet Protocol") or UDP/IP.
- the distributed processing system 100 and 100' is inherently capable of multiplicities of methods of downloading composite video data from the Real Time Processing 150 group to the User Device 151 group depending on efficiency tradeoffs and timing requirements. Regardless of the method(s) employed to download the composite video data, ultimately the composite video data will appear on the screens of the User Device 151 group.
- the remotely located User Device 151 group similar to the User Device 156, is also
- Embodiments' 100 and 100' associated system architecture diagrams 200 and 200' of FIGS. 2A and 2B also include four groups (i.e., Real World Scan 201, Offline Processing 202, Real Time Processing 250, and User Device 251) visually differentiated by four swim lane columns in FIGS. 2A and 2B. Similar to the flowcharts, if a particular hardware function appears completely within a swim lane, its functionality is limited to the data category of the associated swim lane.
- FIG. 2A swim lane system architecture diagram 200 begins with recording real world data (204 and 204') from various sources— e.g., real world horse track video 204 or medical scan data 204'.
- This 3D real world data is applied to a local computing device's 220 I/O buffer (221 or 224) and ultimately saved into nonvolatile memory 205 preferably linked with metadata— e.g., Global Positioning System (“GPS”) data, human body anatomical plane coordinates.
- GPS Global Positioning System
- the collected real world data is structured by the Central Processing Unit (CPU) 222 and memory 223 into a 3D model where portions of the model can be randomly accessed with minimal processing delay.
- the real world scans or scenes can be replaced with virtual (i.e., computer generated) backdrop scenes 206 generated by the Offline Processing 202 group.
- the Offline Processing 202 group's server(s) 225, 225', 225", I/O buffer(s) 226 receive the backdrop scenes or images with the received data formatted in a manner compatible for processing with overlay images that are preferably in PRP format.
- Each Offline Processing 202 group's server e.g., 225, 225', 225" is configured with a CPU 227, memory 228, and optionally a Graphics Processing Unit (GPU) 229.
- GPU Graphics Processing Unit
- Offline Processing group's 202 server e.g., 225, 225', 225
- the nonvolatile memory for storing general display data 209 as well as PRP data 208.
- Both the stored PRP data 208 and the display data 209 are transmitted on demand to the Real Time Processing 250 group's at least one server (e.g., 253, 253', 253") when required.
- This transmission of data is achieved via a common high-speed interface between the Offline Processing 202 group's server(s) (e.g., 225, 225', 225') and the Real Time Processing 250 group's server(s)— e.g., 253, 253', 253".
- the exact type of communication interface is irrelevant to the invention so long as sufficient bandwidth is available to transmit all PRP data 208 and display data 209 within the time restrictions required. This interface is illustrated by off- page connection "A/B" 210 in FIG. 2A and 210' FIG. 2B.
- the PRP data 208 and the display data 209 are received by the Real Time Processing 250 group's server(s) (e.g., 253, 253', 253") I/O 254 with the partially completed data types optionally forwarded to the User Device 251 group's multiplicity of display devices— e.g., 280 (mobile device— e.g., smart phone or tablet), 281 (kiosk), 282 (laptop).
- server(s) e.g., 253, 253', 253
- I/O 254 with the partially completed data types optionally forwarded to the User Device 251 group's multiplicity of display devices— e.g., 280 (mobile device— e.g., smart phone or tablet), 281 (kiosk), 282 (laptop).
- this embodiment has the advantage of potential reduction in bandwidth requirements with the User Device 251 group's discrete I/O (284, 288, and 292) receiving the digital data files from the Real Time Processing 250 group's game server(s) or server(s) I/O 254 not necessarily in real time and storing the data in non-volatile memory (286, 290, and 294).
- the DLL file previously created by the animator 154 FIG. IB
- 2B associated User Device 251 group's CPUs (e.g., 285, 289, and 293) with the output displayed on the associated displays (e.g., 287, 291, and 295).
- This embodiment has the advantage of decreased network bandwidth at the expense of some local processing being executed by the user's local computing device.
- the associated reduction in bandwidth requirements readily accommodates mobile wireless networks (other than Local Area Networks— "LANs”) that typically have lower bandwidth availability.
- LANs Local Area Networks
- typically the processing is minimal, thereby making this embodiment compatible with most user local computing devices (e.g., 280, 281, 282) but is incompatible with televisions or monitors 283 that typically do not offer computing devices.
- the remotely located User Device 251, similar to the User Device 156, is also interchangeably referred to herein as a "local computer device," which in one preferred embodiment is a mobile device.
- the PRP data 208 and the display data 209 received by the Real Time Processing 250 group's I/O 254 is processed on the Real Time Processing 250 group's servers (e.g., 253, 253', 253") with the DLL executing locally on the CPU 255, memory 256, and optional GPU 257— though it should be noted that for simplicity and economy it is preferred that a GPU is typically not included.
- the completed video stream is downloaded to the User Device 251 group's multiplicity of display devices— e.g., 280, 281, 282, and 283 (television or display screen 297).
- This embodiment has the advantage of requiring minimal processing on the User Device 251 group's display devices (e.g., 280, 281, 282, and 283), thereby possibly expanding the types of devices capable of displaying the video with the possible disadvantage of requiring increased transmission bandwidth.
- the Real Time Processing 250 group transmitting partially or completely processed video file(s) is determined by the specifications 252 being implemented through a user interface (not shown in FIG. 2B) that directs the servers (e.g., 253, 253', 253").
- the specifications 252 stipulates the formatting of the transmission (e.g., VGA, SDI, DVB, UDP/IP, TCP/IP) from the Real Time Processing 250 group to the User Device 251 group's I/O ports (e.g., 284, 288, 292, and 296) with possibly different User Device 251 group's I/O portals requiring differently formatted data— e.g., TCP/IP being essentially incompatible with geosynchronous satellite communications portals due to the propagational delay (i.e., 270 milliseconds one-way and 540 milliseconds round trip) greatly slowing handshaking signals such that UDP/IP (i.e., no handshaking) or other formats are preferred.
- These six figures progress from a recorded horse race track real world scene 301 (FIG. 3A) forming a 3D background model, to the same scene 311 enhanced with 3D static virtual images 312 embedded in the background model (FIG. 3B), to virtually the same scene 321 further enhanced with an overlay layer that includes multiple dynamic images (323 through 326) providing an indication of how the virtual race is progressing (FIG. 3C).
- PRP formatted animated 3D virtual images of horses and jockeys 337 are added in another layer (FIG.
- FIG. 3E illustrates the same animated PRP formatted 3D virtual images of horses and jockeys 347 of FIG. 3D with a PRP color and texture lookup table added.
- FIG. 3F illustrates a composite scene 351 of the previous real world background model including dynamic overlays (353 through 356) as well as PRP formatted animated 3D virtual images of horses and jockeys 357 with associated PRP color and texture lookup tables referenced— i.e., an example displaying how the scene would be seen by the consumer.
- FIG. 3A illustrates one example of a backdrop 3D model 300 generated from real world scan data that is compatible with the previous embodiments of the invention.
- the backdrop scene 301 is of a deserted horse race track (i.e., no horses, people, and the like) with metadata (e.g., GPS coordinates) also embedded in it and the associated 3D model 300.
- the entire backdrop 3D model 300 is sans moving objects because the model is intended for a gaming application wherein virtual PRP horses and riders will be superimposed over the backdrop 3D model. Therefore, any moving objects recorded in the 3D backdrop model 300 could potentially interfere with superimposed virtual PRP horses and riders animation when the composite game video is generated.
- FIG. 3B shows the same 3D model 310 generated from real world scan data of FIG. 3 A, but with the addition of static virtual billboards 312 embedded in the 3D model 310 — FIG. 3B.
- These static digital images 312 (that were superimposed and are not part of the original background scan) have been added to the 3D model 310 such that the billboards will be included in the background when this scene 311 of the 3D model 310 is displayed during animation.
- the static billboards 312 assume their own coordinates within the 3D model and are consequently displayed whenever the same portion of the 3D model is displayed without any need for additional real time processing.
- static in this context is only referring to this particular example 310 of background virtual billboards 312.
- dynamic images e.g., people, cars, beating heart, billboards with embedded animation
- the significant concept is added images are embedded in the 3D model 310 itself thereby performing all associated computational processing in advance, thus eliminating the requirements for the processing when displaying in real time.
- the 3D backdrop model 310 including virtual billboard 322 is further enhanced with a first layer overlay 321 that includes multiple dynamic images (i.e., image content may change throughout the virtual race, but not necessarily the image positions relative to the viewing scene 321) that in this example are determined by the game progression and outcome (e.g., as controlled by a random process).
- the leaderboard 323 illustrates the "silks" as well as the numbers, odds, and names of the first, second, and third place horses in the virtual race.
- example 320 Also included in example 320 is a secondary board 325 displaying the "silks", numbers, and odds of the other horses in the virtual race. Additionally, there is a display overlay 324 listing the virtual racetrack name, time, and event identifier (ID). Finally, there is also a track position graphic 326 that provides a visual representation of the location on the virtual racetrack that the scene 321 is currently displaying.
- all of these dynamic images i.e., 323 through 326) are easily modified during the virtual race with very little computational impact due to their relative simple structure and graphics.
- the overlay example 320 images (323 through 326) can be added to the composite video in advance or during real time with virtually no impact on real time processing requirements either way.
- these dynamic image data (323 through 326) are formatted as fonts or glyphs.
- FIG. 3D 330 further enhances the example 320 of FIG. 3C by adding a second layer in which animated virtual PRP horses and jockeys 337 (eight illustrated in FIG. 3D) are superimposed over the backdrop 3D model creating a composite scene 331.
- animated virtual horses and jockeys 337 are shown as animated skeletal models in PRP format without any associated PRP color or texture lookup table reference.
- composite scene 331 the animated virtual horses and jockeys 337 appear to have form and shadows, but no color or texture.
- the computational intensive portions of the rendered virtual horses and jockeys animation 337 may be generated and saved ahead of real time generation and display.
- FIG. 3E provides an example 340 highlighting the change in appearance of the animated PRP horses and jockeys 347 by providing the same scene enhanced with only a PRP color and texture lookup table added— for emphasis, all static and dynamic overlays were omitted from the FIG. 3E illustration 341.
- the computationally intensive rendering e.g., virtual horses and jockeys 337— FIG. 3D
- the computationally intensive rendering may be executed in advance with a PRP lookup table referenced when the actual game video executes in real time.
- any conceivable type of display processor can easily accommodate the minimal additional processing required to access the PRP lookup table in real time.
- PRP lookup table(s) runtime additions are then combined with the relatively simple dynamic overlays (323 through 326), the virtual race results can be effectively changed in real time with very little additional real time processing— i.e., the vast majority of computational rendering is generated in advance leaving only the trivial computations of placing dynamic overlays (323 through 326) and attaching PRP lookup table(s) to the PRP virtual horses and jockeys animation 337 remaining for real time processing.
- a relatively small set of generic horse races can be rendered in high quality in advance with real time generation results varying depending on the game outcome as determined by a random process (e.g., RNG, physical ball drop machine).
- RNG random process
- relatively small numbers of preprocessed high quality rendered horse races enable exponential or even factorial numbers of variations in real time display, such that it becomes unlikely that a consumer would notice that the same preprocessed high quality rendering was displayed in real time more than once.
- FIG. 3F illustrates one possible example 350 of a game outcome scene 351 when executed in real time utilizing the preprocessed model 330 example of FIG. 3D.
- the addition of a PRP color and texture lookup table to the previously skeletal element's rendered animated virtual PRP horses and jockeys 357 allow for one possible outcome as displayed by the jockey and horses "silks" colors 357.
- each virtual PRP horse and jockey 357 would have their own individual references in the PRP lookup table resulting in a different appearance of the PRP horse and jockey 357 positions in the virtual race depending on the color and texture referenced in the PRP lookup table.
- the lead PRP virtual horse and jockey “silks” colors are coordinated with the leaderboard display 358 along with the second and third place PRP virtual horse and jockey “silks” colors 359 and 360 respectively.
- these results can be modified for another virtual race with minimal computational processing by simply changing the PRP lookup table and the coordinated leaderboard display 353 and 355 images without affecting the computationally intensive 3D background model or the PRP animated renderings.
- composite scenes 350 can be created with animated (e.g., 357) and overlays (353 through 356) in compliance with the specifications 152 and 252 (FIGS. IB and 2B respectively) in a composite real time video format.
- animated e.g., 357
- overlays 353 through 356
- the appearance and outcome of the resulting composite real time video typically being variable and related to additional input (e.g., RNG, ball drop machine, user decisions) in accordance with the specifications 152 and 252 (FIGS. IB and 2B respectively).
- the PRP skeletal models augmented with additional file texture data lookup table(s) also has utility in ways other than determining a game's outcome.
- PRP models with texture data lookup table(s) can be employed to increase the apparent variety of a fixed number of processor intensive photorealistic looking 3D graphics previously rendered.
- varying PRP texture data lookup table(s) can be optionally combined with static signs, simple overlays, and the like, to provide perceptions of real time generated videos that can be readily changed with very little additional real time processing required. Therefore, the appearance of a relatively small number of preprocessed high quality rendered videos can be expanded exponentially or even on a factorial scale in real time.
- FIGS. 4A and 4B One example of varying the appearance of a 3D video by changing only its PRP texture lookup table is provided in FIGS. 4A and 4B.
- a scene 400 from a photorealistic 3D graphics boxing match game is depicted with two computer generated characters— i.e., a virtual Mike Tyson 401 and a virtual opponent 402.
- both computer generated characters 401 and 402 were generated as PRP skeletal models with their associated texture data saved in a separate PRP texture lookup table providing color information.
- the PRP texture lookup table results (among other things) in black trunks 403 for the Mike Tyson character 401 and blue trunks 404 for the opponent 402.
- the opponent 402 is illustrated with clean (i.e., unmarked) arms and torso 405.
- the PRP skeletal models are computationally expensive to render with the colors added by the PRP texture table lookup typically requiring trivial processing.
- a scene 400' differing in appearance can be rendered with virtually no change in the real time processing requirements.
- the same PRP skeletal models (401 ' and 402') appear with different color trunks due to referencing a different PRP texture lookup table— i.e., the Mike Tyson character 401 ' now appears to be wearing blue trunks 403' with the opponent 402' wearing red trunks 404'.
- the opponent's 402' arm and torso now appear to include a tattoo 405'. This appearance of a tattoo 405' on the opponent 402' in scene 400' was accomplished by essentially highlighting partially rendered pixels present in both skeletal models 402 and 402' with a different color in scene 400' than in scene 400.
- the PRP color lookup table associated with scene 400 colored the potential tattoo pixels on the opponent's 402 arm and torso 405 in a pallet similar to the rest of the model's skin tone.
- scene 400' (FIG. 4B) different colors were referenced for the tattoo pixels on the opponent's 402' arm and torso 405', thereby providing a much greater contrast to the model's skin tone.
- the appearance of the scene 400 (FIG. 4A) can be varied substantially 400' with a simple substitution of a PRP color lookup table at a trivial run time computational cost.
- FIG. 5 provides an illustrative example 500 of one of the Real Time
- Operator interface 501 enabling control of the game video's appearance in the example of FIG. 3F.
- the operator interface 501 includes multiplicities of user selectable parameters that enable a specification to determine the configuration of the generated DLL and consequently the look, feel, and configuration of the game video.
- entry parameter 502 allows the operator to specify the time (i.e., "14:30" as illustrated) that the game video will run with entry parameter 503 specifying the type of event (i.e., "Horse Race" as illustrated).
- clickable tick mark 504 allows the automated system to select the rest of the parameters on a pseudorandom basis.
- Entry parameter 505 selects the 3D backdrop model to be utilized with parameter 506 impacting the lighting of the overlay 3D model virtual images as well as possibly overriding the backdrop selection.
- Entry parameter 507 specifies the "Field Size", which in the context of example 500 specifies that eight virtual horses and jockeys will be generated for the overlay 3D model as illustrated in the example 350 of FIG. 3F.
- grayed out entry parameter portions (508 through 510) can have the effect of changing the colors on PRP animated characters via changing the referenced PRP internal color table for the tournament 508 or teams 509 and 510.
- the "OK" button 511 initiates the video creation process thereby allow the specified parameters to drive the generation of the game video.
- the "Cancel” button 512 allows the user to exit this mode without initiating any video generation.
- example 500 illustrates just one possible configuration of an operator interface 501 suitable for specifying the generated DLL and consequently the look, feel, and configuration of the game video.
- the invention preferably includes various user interfaces to automate (and therefore standardize) the generation of the video and associated DLL in compliance with the specifications 152 and 252 (FIG. IB and FIG. 2B respectively.
- Partially Rendered Pixels are embodied in a type of digital image file format that includes the portion of the image information (e.g., location, form, structure, lighting) that is computationally expensive to calculate, reserving some details (e.g., color tables, LSB values, luminescence, grey scale) to be saved in an external PRP table.
- PRP animated 3D characters are initially designed in significant amounts of detail with lighting and texture information carefully matched to the underlying backdrop 3D model. When all of the PRP animated 3D characters' attributes have been determined, the texture information (e.g., lighting, shadow, animation) is removed from the PRP animated 3D
- the PRP file format snippet consists of a header 601 in which version, matrix dimensions, and number of frames are included and every pixel in the image is defined with respect to a line reference 602.
- Each pixel 603 in a line is defined as a two byte "X" coordinate 604 (the "Y" coordinate being defined by the line number) with another byte reserved for the PRP "type”.
- the PRP type 605 referencing the external PRP lookup table which can be any one of a multiplicity of different types— e.g., LSB; Red, Green, Blue, Alpha (RGBA); nUVA; grayscale; luminance; shadow; multi-shadow; audio; etc.
- the actual PRP lookup table structure has varying dimensions depending on the number of types specified.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Processing Or Creating Images (AREA)
Abstract
L'invention concerne un procédé et un système d'amélioration de la variabilité d'infographie haute qualité à partir de sources limitées au moyen d'un traitement et d'une largeur de bande minimaux en temps réel. Un modèle 3D d'arrière-plan sous-jacent est généré. Ensuite, des pixels partiellement rendus (PRP) de personnages 3D animés sont générés, lesquels comprennent des informations d'éclairage et de texture compatibles avec le modèle 3D d'arrière-plan sous-jacent. La texture et d'autres données auxiliaires sont supprimées ou omises pour les personnages 3D animés à PRP ne laissant qu'un modèle de squelette détaillé. La texture extraite et d'autres données auxiliaires sont placées dans une table de consultation PRP séparée. Un segment vidéo entièrement rendu est créé à partir des PRP et d'un ensemble sélectionné de texture et d'autres données auxiliaires.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/587,100 US10210700B2 (en) | 2017-05-04 | 2017-05-04 | Generation of variations in computer graphics from intermediate file formats of limited variability, including generation of different game outcomes |
US15/587,100 | 2017-05-04 | ||
US15/587,088 US10322339B2 (en) | 2017-05-04 | 2017-05-04 | Generation of variations in computer graphics from intermediate formats of limited variability, including generation of different game appearances |
US15/587,088 | 2017-05-04 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018203115A1 true WO2018203115A1 (fr) | 2018-11-08 |
Family
ID=59153228
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2017/052820 WO2018203115A1 (fr) | 2017-05-04 | 2017-05-12 | Génération de variations d'infographie à partir de formats de fichier intermédiaire à variabilité limitée, comprenant la génération de différents aspects de jeu ou résultats de jeu |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2018203115A1 (fr) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5860924A (en) | 1996-11-26 | 1999-01-19 | Advanced Technology Laboratories, Inc. | Three dimensional ultrasonic diagnostic image rendering from tissue and flow images |
JP3263201B2 (ja) * | 1993-09-21 | 2002-03-04 | 株式会社リコー | 画像処理装置 |
US7038690B2 (en) | 2001-03-23 | 2006-05-02 | Microsoft Corporation | Methods and systems for displaying animated graphics on a computing device |
US20080012870A1 (en) | 2005-04-25 | 2008-01-17 | Apple Inc. | Color correction of digital video images using a programmable graphics processing unit |
US7593021B1 (en) | 2004-09-13 | 2009-09-22 | Nvidia Corp. | Optional color space conversion |
US8147339B1 (en) * | 2007-12-15 | 2012-04-03 | Gaikai Inc. | Systems and methods of serving game video |
US8151199B2 (en) * | 2009-02-09 | 2012-04-03 | AltEgo, LLC | Computational delivery system for avatar and background game content |
-
2017
- 2017-05-12 WO PCT/IB2017/052820 patent/WO2018203115A1/fr active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3263201B2 (ja) * | 1993-09-21 | 2002-03-04 | 株式会社リコー | 画像処理装置 |
US5860924A (en) | 1996-11-26 | 1999-01-19 | Advanced Technology Laboratories, Inc. | Three dimensional ultrasonic diagnostic image rendering from tissue and flow images |
US7038690B2 (en) | 2001-03-23 | 2006-05-02 | Microsoft Corporation | Methods and systems for displaying animated graphics on a computing device |
US7593021B1 (en) | 2004-09-13 | 2009-09-22 | Nvidia Corp. | Optional color space conversion |
US20080012870A1 (en) | 2005-04-25 | 2008-01-17 | Apple Inc. | Color correction of digital video images using a programmable graphics processing unit |
US8147339B1 (en) * | 2007-12-15 | 2012-04-03 | Gaikai Inc. | Systems and methods of serving game video |
US8151199B2 (en) * | 2009-02-09 | 2012-04-03 | AltEgo, LLC | Computational delivery system for avatar and background game content |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10322339B2 (en) | Generation of variations in computer graphics from intermediate formats of limited variability, including generation of different game appearances | |
US10210700B2 (en) | Generation of variations in computer graphics from intermediate file formats of limited variability, including generation of different game outcomes | |
WO2021249414A1 (fr) | Procédé et système de traitement de données, dispositif associé et support de stockage | |
CN101512553B (zh) | 用于虚拟内容安置的系统和方法 | |
US11538213B2 (en) | Creating and distributing interactive addressable virtual content | |
CN104618779B (zh) | 移动装置上的补充视频内容 | |
US10582191B1 (en) | Dynamic angle viewing system | |
Dobbyn et al. | Geopostors: a real-time geometry/impostor crowd rendering system | |
CN110178370A (zh) | 使用用于立体渲染的光线步进和虚拟视图广播器进行这种渲染 | |
US7142209B2 (en) | Real-time rendering system and process for interactive viewpoint video that was generated using overlapping images of a scene captured from viewpoints forming a grid | |
EP0633533B1 (fr) | Procédé pour produir des données d'image représentant une image | |
US9665334B2 (en) | Rendering system, rendering server, control method thereof, program, and recording medium | |
CN103635939B (zh) | 用于虚拟环境的间接照亮过程 | |
JP2010532890A (ja) | アバターカスタマイズの装置及び方法 | |
US20070198939A1 (en) | System and method for the production of presentation content depicting a real world event | |
US20140302930A1 (en) | Rendering system, rendering server, control method thereof, program, and recording medium | |
US6326972B1 (en) | 3D stroke-based character modeling suitable for efficiently rendering large crowds | |
JP2017056193A (ja) | ブロードキャスタを有するリモートレンダリングサーバ | |
JP7194125B2 (ja) | バーチャル・リアリティ・メディア・コンテンツ内に含める目的での現実世界シーンのカスタマイズされるビューの仮想化プロジェクションを生成するための方法及びシステム | |
US20170061687A1 (en) | Video-based interactive viewing along a path in medical imaging | |
KR100610689B1 (ko) | 3차원 화면에 동영상을 삽입하는 방법 및 이를 위한 기록매체 | |
Nguyen et al. | Real-time 3D human capture system for mixed-reality art and entertainment | |
CN108043030B (zh) | 一种用真实图画构造互动游戏玩家角色的方法 | |
US20130336640A1 (en) | System and method for distributing computer generated 3d visual effects over a communications network | |
WO2018203115A1 (fr) | Génération de variations d'infographie à partir de formats de fichier intermédiaire à variabilité limitée, comprenant la génération de différents aspects de jeu ou résultats de jeu |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17732569 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 13/02/2020) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17732569 Country of ref document: EP Kind code of ref document: A1 |