Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T11/00—Two-dimensional [2D] image generation
  • G06T11/60—Creating or editing images; Combining images with text
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/12—Computing arrangements based on biological models using genetic models
  • G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/25—Fusion techniques
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/045—Combinations of networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0475—Generative networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/094—Adversarial learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T13/00—Animation
  • G06T13/20—Three-dimensional [3D] animation
  • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/047—Probabilistic or stochastic networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
  • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
  • G06N3/065—Analogue means

Definitions

• the present disclosure relates to an image generation method, an image generation apparatus, an image generation system and a program.
• sites or platforms for providing various tools are provided.
• some platforms are operated on the Internet for providing tools to enable users to generate digital images, such as an avatar image and a character image, by means of provided parts and to edit, release or exchange the generated digital images .
• One object of the present disclosure is to provide a novel image generation technique.
• one aspect of the present disclosure relates to an image generation method that includes acquiring first latent information of a first image and second latent information of a second image, fusing the first latent information and the second latent information to generate fusion latent information, and supplying the fusion latent information to a trained generative model to generate a fusion image.
• Fig. 1 is a schematic diagram that depicts an image generation operation according to one embodiment of the present disclosure.
• Fig. 2 is a diagram that depicts a digital image according to one embodiment of the present
• Fig. 3 is a diagram that depicts a digital image according to one embodiment of the present
• Fig. 4 is a diagram that depicts a digital image according to one embodiment of the present
• Fig. 5 is a schematic diagram that depicts an image generation apparatus according to one embodiment of the present disclosure.
• Fig. 6 is a block diagram that depicts a functional arrangement of an image generation apparatus according to one embodiment of the present disclosure.
• Fig. 7 is a diagram that depicts image information on a platform according to one embodiment of the present disclosure.
• Fig. 8 is a diagram that depicts image information on a platform according to one embodiment of the present disclosure.
• Fig. 9 is a diagram that depicts fusion of latent information according to one embodiment of the present disclosure.
• Fig. 10 is a diagram that depicts a fusion image according to one embodiment of the present
• Fig. 11 is a diagram that depicts fusion of latent information in accordance with a genetic operation according to one embodiment of the present disclosure.
• Figs. 12A to 12C are diagrams that depicts fusion of latent information according to another
• Figs. 13A to 13C are diagrams that depicts fusion of latent information according to another
• Fig. 14 is a flowchart that depicts an image generation operation according to one embodiment of the present disclosure.
• Fig. 15 is a flowchart that depicts a training operation in accordance with GANs (Generative Adversarial Networks) according to one embodiment of the present disclosure .
• GANs Geneative Adversarial Networks
• Fig. 16 is a schematic diagram that depicts an image generation operation according to another embodiment of the present disclosure.
• Fig. 17 is a flowchart that depicts an image generation operation according to another embodiment of the present disclosure.
• Fig. 18 is a schematic diagram that depicts an image generation system according to one embodiment of the present disclosure.
• Fig. 19 is a block diagram that depicts a hardware arrangement of an image generation apparatus, a training apparatus and a user apparatus according to one embodiment of the present disclosure.
• An image generation apparatus fuses latent information (for example, genetic codes, attributes or the like) of two images to be fused in accordance with a predetermined operation (for example, a genetic operation or the like) to generate fusion latent information and supplies the generated fusion latent information to a trained generative model to generate a fusion image of the two images.
• latent information for example, genetic codes, attributes or the like
• a predetermined operation for example, a genetic operation or the like
• a character image as illustrated in FIG. 1 is characterized by the latent information including a code (for example, a genetic code, which may be referred to as a noise) and an attribute (for example, a hair style, a hair color, an eye color, a skin color, an expression, an attachment such as glasses or a hat, or the like, all or a part of which may be used) .
• a code for example, a genetic code, which may be referred to as a noise
• an attribute for example, a hair style, a hair color, an eye color, a skin color, an expression, an attachment such as glasses or a hat, or the like, all or a part of which may be used.
• an image generation apparatus may fuse the latent
• GANs Geneative Adversarial Networks
• the image generation apparatus can generate a variety of unique fusion images with succession of codes and attributes of the latent information of both the images rather than simple combinations of parts of the incoming images. Also, totally harmonious fusion images can be generated with use of the generative model.
• FIG. 5 An image generation apparatus according to one embodiment of the present disclosure will be described with reference to FIGS. 5 to 11.
• FIG. 5 is a schematic diagram for illustrating an image generation apparatus according to one embodiment of the present disclosure.
• an image generation apparatus 100 has a trained generative model. Upon acquiring latent information of two to-be-fused images, the image generation apparatus 100 fuses the acquired latent information, in accordance with a predetermined operation, to generate fusion latent information. Then, the image generation apparatus 100 supplies the generated fusion latent information to the generative model to generate a fusion image.
• the image generation apparatus 100 may be for implementing a platform for image
• the image generation apparatus 100 acquires latent information of the user' s indicated two images and fuses the acquired latent information in accordance with a predetermined operation.
• the operation may be a
• a genetic operation such as crossover, mutation and selection, an arithmetic
• the image generation apparatus 100 may use the trained generative model to generate a fusion image from the fusion latent information and provide the generated fusion image on the platform.
• the trained generative model is provided in the image generation apparatus 100 in the illustrated embodiment, the present disclosure is not limited to it, and the trained generative model may be located at an external apparatus that is communicatively connected to the image generation apparatus 100, for example. In this case, the image generation apparatus 100 may send the fusion latent information to the
• FIG. 6 is a block diagram for illustrating a functional arrangement of an image generation apparatus according to one embodiment of the present disclosure.
• the image generation apparatus 100 includes a latent information acquisition unit 110, a latent information fusion unit 120 and a fusion image generation unit 130.
• the latent information acquisition unit 110 acquires respective latent information of two to-be-fused images.
• the to-be-fused image may be a bitmap image such as an avatar image or a character image and be associated with the latent information for
• the latent information used herein may be information that may be put into a latent variable and inferred from observation data (for example, image data) through a model or the like.
• the latent information may include a code and an attribute.
• the code may represent a characteristic (for example, a body skeleton, a shape of a facial part or the like) specific to an object of interest
• the attribute may represent a variable characteristic (for example, a hair style, an expression or the like) of the object. For example, as illustrated in FIGS.
• the latent information acquisition unit 110 acquires codes (which are illustrated as (Xcl, Xc2, ..., Xc(n-l), Xcn) and (Ycl, Yc2, ..., Yc(n-l), Yen) in the illustrated example) and attributes (which are illustrated as (Xal, Xa2, ..., Xa(m-l), Xam) and (Yal, Ya2, ..., Ya(m-l), Yam) in the illustrated example) associated with these two images and provides respective latent information of the images, which is composed of the acquired code and attributes, to the latent
• the latent information fusion unit 120 fuses the latent information of the two images to generate fusion latent information. For example, upon acquiring the codes (Xcl, Xc2, ..., Xc(n-l), Xcn) and (Ycl,
• the latent information fusion unit 120 performs a predetermined fusion operation on the acquired codes (Xcl, Xc2, ..., Xc(n-l), Xcn) and (Ycl, Yc2, Yc(n-l), Yen) and attributes (Xal, Xa2, ..., Xa(m-l), Xam) and (Yal,
• the fusion operation may be performed on a per-code basis and a per-attribute basis as illustrated. Alternatively, in other embodiments, the fusion operation may be performed on a per-latent information basis. Then, the latent information fusion unit 120 provides the generated fusion latent information to the fusion image generation unit 130. Also, the to-be-fused images may be unnecessarily two different images and may be the same image or three or more different images.
• the fusion image generation unit 130 supplies the fusion latent information to the trained generative model to generate a fusion image. For example, upon acquiring the fusion latent information (Zcl, Zc2, ..., Zc(n-l), Zen) and (Zal, Za2, ..., Za(m-l), Zam) from the latent information fusion unit 120, the fusion image generation unit 130 supplies the acquired fusion latent information to the trained generative model to generate the fusion image as illustrated in FIG. 10 as its output.
• the generated fusion image is associated with the fusion latent information composed of the code (Zcl, Zc2, ..., Zc(n-l), Zen) and the attribute (Zal, Za2, ..., Za(m-l), Zam) . Also, the fusion image may be displayed in
• images of ancestors of the fusion image may be also displayed in the form of a family tree.
• the trained generative model may be a generator trained in accordance with GANs
• a training apparatus 200 may pre-train a generator and a discriminator, which are implemented as neural networks, as the GANs and provide them to the image generation apparatus 100.
• a dataset of training images is provided to the training apparatus 200, and the training apparatus 200 supplies a code such as a random number to the generator to acquire an image as its output.
• the training apparatus 200 supplies either the image generated by the generator or a training image in the dataset to the discriminator to acquire, as its output, a discrimination result indicative of whether the incoming image is the image generated by the
• the training apparatus 200 updates parameters for the discriminator so that the discriminator may make correct discrimination. Also, the training apparatus 200 updates parameters for the generator so that the discriminator may make
• the training apparatus 200 provides the finally acquired generator to the image generation apparatus 100 as the trained generative model.
• a dataset of training images with attributes is provided to the training apparatus 200, and the training apparatus 200 supplies a code such as a vector of random numbers and a list of attributes to the generator to acquire an image as its output.
• the training apparatus 200 supplies either the image generated by the generator or a training image in the dataset to the discriminator to acquire as its output a discrimination result indicative of not only whether the incoming image is the image generated by the generator or the training image but also what the attributes are.
• the training apparatus 200 updates parameters for the discriminator so that the discriminator may make correct discrimination.
• the training apparatus 200 updates parameters for the generator so that the discriminator may make incorrect discrimination. If a predetermined termination condition such as completion of the above-stated
• the training apparatus 200 provides the
• the trained generative model according to the present disclosure is not limited to the above-stated generator or neural network trained in accordance with the GANs and may be any type of machine learning model trained in accordance with any other appropriate training scheme .
• the latent information fusion unit 120 may perform a genetic operation on the acquired latent information of the two images to generate the fusion latent information. For example, as
• the latent information fusion unit 120 may generate a fusion code and a fusion
• code Xc and attribute Xa in the to-be-fused latent information X are
• the latent information fusion unit 120 may perform crossover operations on elements of the codes Xc and Yc to generate code Zc "-0.2", “1.4", “0.4”, ... in the fusion latent information Z, as illustrated. Also, the latent information fusion unit 120 may perform
• the above-stated crossover operations are performed on a per-element basis, but the crossover operations according to the present embodiment are not limited to the above and may be
• the genetic operations according to the present embodiment may include, but not limited to the crossover, other genetic operations such as mutation and selection.
• the latent information fusion unit 120 may set values other than those for elements of the to-be-fused latent information X and Y to one or more elements of the fusion latent information Z .
• Introduction of the mutation to the fusion operation can increase variations of the generated fusion latent information.
• the latent information fusion unit 120 may generate the fusion latent information by averaging the acquired latent information of two images. For example, for elements of the latent information that are represented as numerical values, for example, "0.2" and "-0.2" as illustrated in FIG. 1, the corresponding elements of the fusion latent information may be determined as an average value of the corresponding elements of two images. In this case, for elements having alternative values (presence of
• the corresponding elements of the fusion latent information may be randomly determined as one of the two.
• the latent information fusion unit 120 may perform a logical
• the logical operation may include a logical OR operation, a logical AND
• the arithmetic operation may include addition, subtraction,
• the arithmetic operation may be performed on the latent information of the two images, and then the conversion or the fluctuation may be applied (for example, a geometric mean or an exponential mean) .
• the latent information fusion unit 120 may perform a combination of any one or more of the above-stated operations on the acquired latent information of the two images to generate the fusion latent information.
• the latent information may further include a sub attribute.
• the above-stated attribute (referred to as a main attribute hereinafter) is an attribute having a higher likelihood of its
• the sub attribute is an attribute that does not appear in the origin image but is likely to appear in the fusion image with a lower occurrence likelihood.
• the sub attribute may have similar contents (for example, a hair style, an expression or the like) .
• Either the main attribute or the sub attribute may be stochastically selected and used to generate the fusion latent information.
• the sub attribute may be also displayed on a user interface.
• latent information X and latent information Y of two origins are composed of code Xc, main attribute Xa_main and sub attribute Xa_sub and code Yc, main
• the main attribute Xa_main is selected for the latent information X
• the sub attribute Ya_sub is selected for the latent information Y.
• probability value at selection of the attributes may be set to different values for respective elements of the attributes. Also, either the main attribute or the sub attribute may be selected for respective elements of the attributes depending on the preset probability values.
• the fusion latent information Z may also include main attribute Za_main and sub attribute Za_sub.
• the latent information fusion unit 120 may form the sub attribute Za_sub from elements that have not been selected for the main
• the fusion latent information Z includes code Zc, main attribute Za_ ain and sub
• the generative model may be pre-trained such that the incoming main attribute Za_main can be weighted with a higher weight than the sub attribute Za_sub. Also, the main attribute and the sub attribute may be fused with weights and be supplied to the trained generative model. Alternatively, the fusion image generation unit 130 may generate the fusion image based on the code Zc and the main attribute Za__main without supplying the sub attribute Za_sub to the generative model .
• the introduction of the sub attribute can increase variations of the generated fusion latent information and can accordingly increase
• the number of the sub attributes may be arbitrarily selected.
• the latent information may further include a sub code.
• the above-stated code (referred to as a main code hereinafter) is a code having a higher likelihood of its occurrence in fusion images.
• the sub code is a code that does not appear in the origin image but is likely to appear in the fusion image with a lower occurrence likelihood.
• the sub code may be also displayed as a barcode or the like on a user interface .
• latent information X and latent information Y of two origins are composed of main code Xc_main, sub code Xc_sub. main attribute Xa_main and sub attribute Xa sub and main code Yc main, sub code Yc sub, main attribute Ya_main and sub attribute Ya_sub, respectively.
• the sub code Xc_sub is selected for the latent information X
• the main code Yc_main is selected for the latent information Y.
• a probability value at selection of the codes may be set to different values for different elements of the codes.
• either the main code or the sub code may be selected for different elements of the codes depending on the preset probability values .
• the fusion latent information Z may also include main code Zc_main and sub code Zc_sub.
• the latent information fusion unit 120 may form the sub code Zc__sub from elements that have not been selected for the main code Zc_main in the fusion latent information Z in the crossover.
• the generative model may be pre-trained such that the incoming main code Zc_main can be weighted with a higher weight than the sub code Zc_sub.
• the main code and the sub code may be fused with weights, and the fused codes may be supplied to the trained generative model.
• the fusion image generation unit 130 may generate the fusion image based on the main code Zc_main and the main attribute Za_main without supplying the sub code Zc_sub to the generative model.
• the introduction of the sub code can increase variations of the generated fusion latent information and can accordingly increase
• FIG. 14 is a flowchart for illustrating an image generation operation according to one embodiment of the present disclosure.
• the image generation operation may be executed by the image generation apparatus 100, particularly a processor in the image generation
• the latent information acquisition unit 110 acquires latent information of two to-be-fused origins.
• the latent information of these origins is possessed by a user on a platform, and when the user indicates images of the to-be-fused origins, the latent information
• acquisition unit 110 acquires the latent information of the indicated origins from the platform and supplies it to the latent information fusion unit 120.
• the latent information fusion unit 120 fuses the acquired latent information.
• the latent information fusion unit 120 may perform a genetic operation such as crossover on the acquired latent information to generate fusion latent information .
• the fusion image generation unit 130 supplies the acquired fusion latent information to a trained generative model.
• the generative model is pre- trained by the training apparatus 200 and may be a generator for GANs, for example.
• the trained generative model may be trained to generate images from the incoming latent information.
• the fusion image generation unit 130 generates a fusion image from the trained generative model.
• the generated fusion image is stored together with the latent information associated with the fusion image, that is, the incoming fusion latent information supplied to the trained generative model, in user's folder or the like on the platform.
• FIG. 15 is a flowchart for illustrating a training operation in accordance with GANs according to one embodiment of the present disclosure.
• the training operation is executed by the training apparatus 200, particularly a processor in the training apparatus 200, for a generator and a discriminator for GANs.
• a dataset of training images are provided to the training apparatus 200.
• the training apparatus 200 supplies latent information, which is composed of a code (noise) such as a vector of random numbers and a list of random attributes, to the generator and acquires an image as its output.
• latent information which is composed of a code (noise) such as a vector of random numbers and a list of random attributes, to the generator and acquires an image as its output.
• the training apparatus 200 supplies latent information, which is composed of a code (noise) such as a vector of random numbers and a list of random attributes, to the generator and acquires an image as its output.
• a code such as a vector of random numbers and a list of random attributes
• an attribute may be supplied to the discriminator and may be outputted as the
• the training apparatus 200 updates the parameters for the discriminator to cause the
• discriminator to yield correct discrimination results and updates the parameters for the generator to cause the discriminator to yield incorrect discrimination results.
• the training apparatus 200 determines whether a termination condition is satisfied, and if the termination condition is satisfied (S204: YES), the training apparatus 200 terminates the training
• the training apparatus 200 returns to step S201 and repeats the above- stated steps S201 to S204 for the next training data.
• the termination condition may be completion of the training operation for a predetermined number of incoming data.
• the training apparatus 200 provides the generator that has been acquired after completion of the training operation to the image generation apparatus 100 as the trained generative model .
• FIG. 16 is a schematic diagram for illustrating an image generation operation according to another embodiment of the present disclosure.
• the latent information is provided in the form of a code compliant with a blockchain, and as a result, the latent information can be exchanged among users via a blockchain based
• the latent information may be represented as a blockchain compliant code that is in compliance with any standard for
• the latent information is described in 320 bits.
• a code is described in 256 bits, and an attribute is described in 64 bits.
• the blockchain compliant code can represent the latent information with a smaller data amount, compared with a typical data amount (for example, 0.65 KB) of the latent information that is not represented in the form of the blockchain compliant code.
• compliant code may be defined, and the mutual
• the image generation apparatus 100 may use the latent information in the form of the blockchain compliant code that a user has purchased from another user together with user' s possessed another latent information to generate a fusion image from the purchased latent information and the possessed latent information.
• the blockchain compliant code is not limited to the above-stated Ethereum compliant code and may be codes in compliance with any other appropriate standard based on blockchain techniques.
• the terminology "latent information" may include not only the latent information that is not represented in the form of a blockchain compliant code but also the latent information that is represented in the form of the blockchain compliant code as described in the present embodiment.
• FIG. 17 is a flowchart for illustrating an image generation operation according to another
• the generation apparatus 100 particularly a processor in the image generation apparatus 100.
• the latent information acquisition unit 110 acquires
• blockchain compliant codes indicative of the latent information of two to-be-fused origins.
• the blockchain compliant codes may be acquired from other users via a blockchain based platform.
• some conversion operations of the latent information that is not represented in the form of a blockchain compliant code into the latent information that is represented in the form of the blockchain compliant code may be
• the latent information fusion unit 120 fuses the acquired blockchain compliant codes.
• the latent information fusion unit 120 may perform a genetic operation such as crossover on the acquired blockchain compliant codes to generate a fusion blockchain compliant code.
• the fusion operation (including main-sub selection) described with reference to FIGS. 5 to 14 for the case where the blockchain compliant code is not used may be applied to the fusion operation on the blockchain compliant codes .
• the latent information represented in the form of the blockchain compliant code may at least include any one of the sub attribute and the sub code.
• the latent information fusion unit 120 converts the acquired fusion blockchain
• correspondence information indicative of correspondence between elements of the blockchain compliant codes and elements of the latent information that is not
• the fusion image generation unit 130 supplies the acquired fusion latent information to the trained generative model.
• the generative model is pre-trained by the training apparatus 200 and may be a generator for GANs.
• the trained generative model is trained to generate images from the incoming latent information .
• the fusion image generation unit 130 generates a fusion image from the trained generative model.
• the generated fusion image may be stored together with the fusion latent information in user' s folder on the platform.
• the fusion operation may be performed on the converted latent information.
• an image generation system according to one embodiment of the present disclosure is described with reference to FIG. 18.
• all or a part of functions of the image generation apparatus 100 are provided as cloud services in the image generation system 10, and a user may generate and acquire fusion images via a user apparatus 300 such as a personal computer.
• the image generation apparatus 100 may be located at a site different from the user apparatus 300.
• the image generation system 10 includes the image generation apparatus 100 and the user apparatus 300 communicatively coupled with the image generation apparatus 100.
• the image generation apparatus 100 When the user apparatus 300 indicates to-be- fused images to the image generation apparatus 100 via a network, the image generation apparatus 100 fuses latent information of the to-be-fused images based on the received indication to generate fusion latent information. Then, the image generation apparatus 100 supplies the generated fusion latent information to a trained
• the generative model to generate a fusion image and sends the generated fusion image to the user apparatus 300.
• the user apparatus 300 acquires the generated fusion image from the image generation apparatus 100 via the network.
• the to-be-fused latent information indicated by the user may be the latent information that is
• the to-be-fused latent information is sent from the user apparatus 300 to the image generation apparatus 100 in FIG. 18, but information relating to two origin images indicated by the user may be sent from the user apparatus 300 to the image generation apparatus 100.
• the image generation apparatus 100 may acquire the to-be-fused latent information based on the user's indication.
• the present disclosure is not limited to it and may be applied to generation of any type of information that can be generated from the latent information, for example, a video, a sound or the like.
• the above-stated image generation technique may be used in games to generate new characters or the like .
• respective functions may be implemented in a circuit that is formed of an analog circuit, a digital circuit or an analog-digital mixture circuit. Also, a control circuit for controlling the respective functions may be provided.
• the circuits may be implemented in an ASIC (Application Specific Integrated Circuit), a FPGA (Field Programmable Gate Array) or the like.
• At least a part of the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 may be arranged with hardware items. Also, if they are arranged with software items, a CPU (Central Processing Unit) or the like may implement them through information processing of the software items. In the case where they are arranged with software items, programs for implementing the image generation apparatus 100, the training apparatus 200, the user apparatus 300 and functions of at least a portion thereof are stored in a storage medium and may be loaded into a computer for execution.
• a CPU Central Processing Unit
• the storage medium is not limited to a removable storage medium such as a magnetic disk (for example, a flexible disk) or an optical disk (for example, a CD-ROM or a DVD-ROM) and may be a fixed type of storage medium such as a SSD (Solid State Drive) using a hard disk device or a memory device.
• a removable storage medium such as a magnetic disk (for example, a flexible disk) or an optical disk (for example, a CD-ROM or a DVD-ROM)
• a fixed type of storage medium such as a SSD (Solid State Drive) using a hard disk device or a memory device.
• the information processing with software items may be some specific implementations using hardware resources.
• processing with software items may be
• Jobs may be executed with hardware resources. Jobs may be executed by using an accelerator such as a GPU (Graphics
• the computer can be embodied as the above
• the type of storage medium is not limited to any specific one.
• FIG. 19 is a block diagram for illustrating one exemplary hardware arrangement according to one
• Each of the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 can be implemented as a computing device including a processor 101, a main memory device 102, an auxiliary storage device 103, a network interface 104 and a device interface 105, which are coupled via a bus 106.
• each of the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 in FIG. 14 includes respective components singly, but the same component may be plurally provided. Also, although the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 are singly illustrated, software items may be installed in multiple computers, and each of the multiple image generation apparatuses 100, the multiple training
• each of the multiple image generation apparatuses 100, the multiple training apparatuses 200 and the user apparatuses 300 may communicate with each other via the network interface 104 or the like.
• the processor 101 is an electronic circuit (a processing circuit or a processing circuitry) including a controller and an arithmetic unit of the image generation apparatus 100, the training apparatus 200 and the user apparatus 300.
• the processor 101 performs arithmetic operations based on incoming data and programs from respective internal devices in the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 and supplies operation results and control signals to the respective internal devices or the like.
• the processor 101 runs operating systems (OS) , applications or the like in the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 to control respective components of the image generation apparatus 100, the training apparatus 200 and the user apparatus 300.
• OS operating systems
• the processor 101 is not particularly limited to any certain one and may be any other implementation that can perform the above
• the image generation apparatus 100, the training apparatus 200, the user apparatus 300 and respective components thereof may be implemented with the processor 101.
• the processing circuit may be one or more electric circuits disposed on a single chip or on two or more chips or devices. If the multiple electronic circuits are used, the respective electronic circuits may communicate with each other in a wired or wireless manner.
• the main memory device 102 is a memory device for storing various data and instructions for execution by the processor 101, and information stored in the main memory device 102 is directly read by the processor 101.
• the auxiliary storage device 103 includes storage devices other than the main memory device 102. Note that the memory device and the storage device mean arbitrary electronic parts capable of storing electronic
• the memory device may be any of a volatile memory and a non-volatile memory.
• the memory device for storing various data in the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 may be implemented with the main memory device 102 or the
• auxiliary storage device 103 for example. As one
• At least a portion of the memory device may be implemented in the main memory device 102 or the
• auxiliary storage device 103 As another example, if an accelerator is provided, at least a portion of the above- stated memory device may be implemented in memory device within the accelerator.
• the network interface 104 is an interface for connecting to the communication network 108 in a wired or wireless manner.
• the network interface 104 may be compliant with any of existing communication standards. Information may be exchanged with the external apparatus 109A communicatively coupled via the communication network 108.
• the external apparatus 109A may include a camera, a motion capture, an output device, an external sensor, an input device and so on, for example. Also, the external apparatus 109A may be an apparatus having a part of functions of components in the image generation apparatus 100, the training apparatus 200 and the user apparatus 300. Then, the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 may receive a part of processing results of the image
• the device interface 105 is an interface such as a USB (Universal Serial Bus) directly coupled with the external apparatus 109B.
• the external apparatus 109B may be an external storage medium or a storage device.
• the memory device may be implemented with the external apparatus 109B.
• the external apparatus 109B may be an output device.
• the output device may be a display device for displaying images or an output device for sounds or the like, for example.
• the output device may be, but not limited to, a LCD (Liquid Crystal Display) , a CRT (Cathode Ray Tube), a PDF (Plasma Display Panel), an organic EL ( ElectroLuminescence ) display, a speaker or the like.
• the external apparatus 109B may be an input device.
• the input device may include a device such as a keyboard, a mouse, a touch panel, a microphone or the like, and incoming information from these devices is provided to the image generation apparatus 100, the training apparatus 200 and the user apparatus 300.
• processors 101 Also, memory devices in the image
• the training apparatus 200 and the user apparatus 300 may be implemented with the main memory device 102 or the auxiliary storage device 103.
• the image generation apparatus 100, the training apparatus 200 and the user apparatus 300 may include one or more memory devices.
• the representation "at least one of a, b and c" may include not only
• the representation may cover arrangements including elements other than a, b and c such as the combination a-b-c-d.
• representation "at least one of a, b or c" may include not only combinations a, b, c, a-b, a-c, b-c and a-b-c but also combinations of a plurality of the same elements a-a, a-b-b, a-a-b-b-c-c or the like. Also, the
• representation may cover arrangements including elements other than a, b and c such as the combination a-b-c-d.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Life Sciences & Earth Sciences (AREA)
• Data Mining & Analysis (AREA)
• Health & Medical Sciences (AREA)
• Biophysics (AREA)
• General Engineering & Computer Science (AREA)
• Evolutionary Computation (AREA)
• Artificial Intelligence (AREA)
• Software Systems (AREA)
• Computational Linguistics (AREA)
• Mathematical Physics (AREA)
• Biomedical Technology (AREA)
• General Health & Medical Sciences (AREA)
• Molecular Biology (AREA)
• Computing Systems (AREA)
• Evolutionary Biology (AREA)
• Bioinformatics & Computational Biology (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physiology (AREA)
• Genetics & Genomics (AREA)
• Image Analysis (AREA)
• Processing Or Creating Images (AREA)
• Image Processing (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (1)

Family

ID=66820702

Family Applications (1)

Country Status (3)

Cited By (3)

Families Citing this family (6)

Citations (2)

Family Cites Families (22)

• 2019
  • 2019-03-18 WO PCT/US2019/022688 patent/WO2019118990A1/en not_active Ceased
  • 2019-03-18 JP JP2019536917A patent/JP7092769B2/ja active Active
• 2020
  • 2020-08-28 JP JP2020144866A patent/JP7300430B2/ja active Active
• 2021
  • 2021-09-08 US US17/447,081 patent/US12182911B2/en active Active
• 2023
  • 2023-06-19 JP JP2023099732A patent/JP7582592B2/ja active Active
• 2024
  • 2024-11-19 US US18/952,232 patent/US20250078367A1/en active Pending

Patent Citations (2)

Also Published As

Similar Documents

Legal Events