DE102022111402A1 - GENERATIVE DOMAIN ADAPTATION IN A NEURONAL NETWORK - Google Patents

Abstract

A system includes a computer that includes a processor and memory. The memory stores instructions executable by the processor to cause the processor to generate a low-level representation of the input source domain data; create an embedding of the input source domain data; generate a high-level feature representation of features of the input source domain data; generate output target domain data in the target domain containing semantics corresponding to the input source domain data by processing the high-level feature representation of the features of the input source domain data using a low-level domain decoder layer of the neural network that generates data from the target; and modify a loss function so that latent attributes corresponding to the embedding are selected from an equal probability distribution.

Description

FIELD OF TECHNOLOGY

The disclosure concerns neural networks in vehicles.

GENERAL STATE OF THE ART

Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e. H. the next hidden layer or the output layer. Each layer of the network produces an output from a received input according to current values of a respective set of weights.

SHORT PRESENTATION

Neural networks are used for many tasks that power autonomous vehicles. For example, a neural network may input image data acquired by a vehicle sensor to determine objects in an environment around a vehicle and use the data regarding the objects to determine a vehicle path over which the vehicle should operate. Neural networks can also be trained to determine commands spoken by an occupant of a vehicle and operate the vehicle based on the specific command. For example, spoken commands may include spoken phrases such as “drive,” “stop,” and “turn left.” Neural networks can also be trained to process video data, for example to determine a real-world position for a vehicle based on visual odometry. Visual odometry is a technique for determining a position of a vehicle based on processing a sequence of video images to determine a vehicle position based on changes in the positions of certain features in the sequence of video images. Features are arrangements of pixel values that can be determined in two or more video images. Neural networks can be trained to accomplish these tasks by collecting large amounts of training data that includes examples of input data and corresponding ground truth. The input data may be images of environments around vehicles that include objects such as other vehicles and pedestrians. In other examples, training data may include commands spoken by a variety of different people who have different vocal characteristics. Ground truth is data that corresponds to the correct output desired by a neural network and has been acquired from a source independent of the neural network. In the image data example, human observers can view the training images and determine the identity and location of objects in the image data. In the example of spoken commands, human listeners can listen to the spoken commands and determine the correct vehicle command that corresponds to the spoken command.

One problem with training data is that a large number, typically greater than 1000, of training examples may be required to train a neural network. Because each training example requires a corresponding ground truth, assembling training datasets can be very expensive and require many person-hours of human labor to implement. Additional neural networks can be trained to generate simulated training data including ground truth from a smaller number of real examples, thereby reducing the time and cost required to generate neural network training data sets. Training data sets generated in this way are only useful if the simulated training data exactly matches the real data used to generate the simulated training data. Techniques discussed in this paper improve the process of generating training data sets using neural networks by improving techniques for generating accurate simulated training data based on limited amounts of input real training data, thereby reducing the time and cost required to generate training data sets are required for training neural networks. Techniques described in this paper can improve the generation of training data sets by neural networks by improving the determination of loss functions. Loss functions are used in training neural networks by comparing generated results with ground truth to determine differences between the generated result and the corresponding ground truth.

A system includes a computer that includes a processor and memory. The memory stores instructions executable by the processor to cause the processor to generate a low-level representation of the input source domain data by processing source domain data using a low-level source domain encoder layer of a neural network corresponding to data from the source domain to generate a low-level representation of the input source domain data; generate an embedding of the input source domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; to generate a high-level feature representation of features of the input source domain data by processing the embedding of the input source domain image using a high-level decoder layer of the neural network shared by data from the source and target domains to produce a high-level feature representation of features generate the input source domain data; generate output target domain data in the target domain containing semantics corresponding to the input source domain data by processing the high-level feature representation of the features of the input source domain data using a low-level domain decoder layer of the neural network that generates data from the target; and modify a loss function so that latent attributes corresponding to the embedding are selected from an equal probability distribution.

In other features, the processor is further programmed to: modify the loss function by calculating a maximum mean deviation between a first latent attribute corresponding to a source domain and a second latent attribute corresponding to a target domain.

In other features, the processor is further programmed to: modify the loss function based on a prediction from a discriminator, the prediction indicating a domain corresponding to a latent attribute.

In other features, the discriminator includes one or more convolution layers, one or more batch normalization layers, and one or more rectified linear unit layers.

In other features, a final layer of the discriminator includes a softmax layer.

In other features, the discriminator produces a multidimensional vector representing the prediction.

In other features, the multidimensional vector includes a four-dimensional vector corresponding to four domains.

In other features, the multidimensional vector includes a two-dimensional vector corresponding to two domains.

In other features, a loss function for the discriminator includes: L _D = Z̃ _AA log D(Z _AA )+Z̃ _BB log D(Z _BB ) + Z̃ _AB log D(Z _AB )+Z̃ _BA log D(Z _BA ), where L _D is defined as the loss function, Z̃ _AA , Z̃ _BB , Z̃ _AB , Z̃ _BA are defined as labels for the corresponding domain, log D is defined as an estimate that the probability for the latent attribute corresponds to a specific domain, and Z _AA , Z _AB , Z _BA , Z _BB are defined as predicted domain outputs.

In other features, the processor is further programmed to: generate a low-level representation of the input target domain data by processing the input target domain data using a target domain low-level neural network encoder layer specific to data from the target domain; generating an embedding of the input target domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; generating a high-level feature representation of features of the input target domain data by processing the embedding of the input target domain image using the high-level decoder layer of the neural network shared by data from the source and target domains; and generating output source domain data from the source domain that includes semantics corresponding to the input target domain data by high processing the feature representation Level of features of the target source domain image using a low-level source domain decoder layer of the neural network specific to data from the source domain.

A method includes: generating a low-level representation of the input source domain data by processing source domain data using a low-level source domain encoder layer of a neural network corresponding to data from the source domain to generate a low-level representation of the input source domain data; generating an embedding of the input source domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; Generate a high-level feature representation of features of the input source domain data by processing the embedding of the input source domain image using a high-level decoder layer of the neural network shared by data from the source and target domains to produce a high-level feature representation of features of the input source domain data to create; generating output target domain data in the target domain that includes semantics corresponding to the input source domain data by processing the high-level feature representation of the features of the input source domain data using a low-level domain decoder layer of the neural network that generates data from the target; and generating output source domain data from the source domain that includes semantics corresponding to the input target domain data by processing the high-level feature representation of the features of the target source domain image using a low-level source domain decoder layer of the neural network specific to data from the source domain.

In other features, the method includes: modifying the loss function by calculating a maximum mean deviation between a first latent attribute corresponding to a source domain and a second latent attribute corresponding to a target domain.

In other features, the method includes: modifying the loss function based on a prediction from a discriminator, the prediction indicating a domain corresponding to a latent attribute.

In other features, the discriminator includes one or more convolution layers, one or more batch normalization layers, and one or more rectified linear unit layers.

In other features, a final layer of the discriminator includes a softmax layer.

In other features, the discriminator produces a multidimensional vector representing the prediction.

In other features, the multidimensional vector includes a four-dimensional vector corresponding to four domains.

In other features, the multidimensional vector includes a two-dimensional vector corresponding to two domains.

In other features, a loss function of the discriminator includes: L _D = Z̃ _AA log D(Z _AA )+Z̃ _BB log D(Z _BB ) + Z̃ _AB log D(Z _AB )+Z̃ _BA log D(Z _BA ), where L _D is defined as the loss function, Z̃ _AA , Z̃ _BB , Z̃ _AB , Z̃ _BA are defined as labels for the corresponding domain, log D is defined as an estimate that the probability for the latent attribute corresponds to a specific domain, and Z _AA , Z _AB , Z _BA , Z _BB are defined as predicted domain outputs.

In other features, the method includes generating a low-level representation of the input target domain data by processing the input target domain data using a target domain low-level neural network encoder layer specific to data from the target domain; generating an embedding of the input target domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; Generating a high-level feature representation of features of the input target domain data by processing the embedding of the input target domain image using the high-level decoder layer of the neural network Layer shared by data from the source and target domains; and generating output source domain data from the source domain that includes semantics corresponding to the input target domain data by processing the high-level feature representation of the features of the target source domain image using a low-level source domain decoder layer of the neural network specific to data from the source domain.

The present disclosure describes a domain adaptation network that can receive data, such as an image, from a source domain and transforms the data into data from a target domain that has similar semantics to the source domain data, e.g. B. semantic content is retained within an image. Semantics, in this context, refers to data, such as objects within an image, that should be maintained between images. Generally the source domain is different, e.g. B. a daily image or an image of a virtual environment, from the target domain, e.g. B. a night picture or a real picture. For example, a distribution of pixel values in images from the source domain is different from the distribution of pixel values in images from the target domain. Thus, images that have the same semantics may look different if one image comes from the source domain and the other image comes from the target domain. For example, the source domain may be images of a virtual environment that simulates a real environment and the target domain may be images of the real environment.

Source domain images may be images of a virtual environment that simulates a real environment with which an autonomous or semi-autonomous vehicle is intended to interact, while the target domain images may be images of the real environment as captured by the vehicle. During training, a loss function is used to update one or more weights of the domain adaptation network. As described in more detail herein, a loss function can be modified to select latent attributes of an embedding from an equal probability distribution to produce more realistic data in the target domain.

By converting source domain images into target domain images, the target domain images can be used to develop a control policy for the vehicle or while training a neural network used to select actions to be performed by the vehicle. Thus, performance of the vehicle in the real environment can be improved by exposing the neural network and/or the control policy to additional situations generated in a virtual environment.

BRIEF DESCRIPTION OF THE DRAWINGS

• 1 is a diagram of an example vehicle system.
• 2 is a diagram of an example server.
• 3 is a diagram of an exemplary domain communications network.
• 4 is another diagram of the example domain transfer network.
• 5 is another diagram of the example domain transfer network that includes a discriminator.
• The 6A until 6D are diagrams of an example discriminator.
• 7 is a diagram illustrating example layers of the discriminator.
• 8th is a diagram of an example deep neural network.
• 9 is a flowchart that illustrates an example process for generating target domain data from source domain data.
• 10 is a flowchart that illustrates an example process for generating source domain data from target domain data.

DETAILED DESCRIPTION

1 is a block diagram of an example domain communications network 300 for determining and transmitting a route recommendation for one or more vehicles. The domain communications network 300 includes a vehicle 105, which is a land vehicle such as a car, a truck, etc. The vehicle 105 includes a computer 110, vehicle sensors 115, actuators transmission elements 120 for actuating various vehicle components 125 and a vehicle communication module 130. The communication module 130 enables the computer 110 to communicate with a server 145 via a communication network 135. The domain communication network 300 also includes a roadside device 150, which can communicate with the server 145 and the vehicle 105 via the communication network 135, and validation devices 155.

The computer 110 includes a processor and memory. Memory includes one or more forms of computer-readable media and stores instructions executable by computer 110 to perform various operations, including those disclosed herein.

The computer 110 may operate a vehicle 105 in an autonomous, semi-autonomous mode, or a non-autonomous (manual) mode. For purposes of this disclosure, an autonomous mode is defined as one in which each of the propulsion, braking, and steering of the vehicle 105 is controlled by the computer 110; in a semi-autonomous mode, the computer 110 controls one or two of the propulsion, braking and steering of the vehicle 105; In a non-autonomous mode, a human driver controls each of the propulsion, braking and steering of the vehicle 105.

The computer 110 may include programming to operate one or more of brakes, propulsion (e.g., controlling acceleration in the vehicle by controlling one or more of an internal combustion engine, an electric motor, a hybrid motor, etc.), steering, climate control, interior and/or exterior lighting, etc. of the vehicle 105 and to determine if and when the computer 110 should control such operations instead of a human driver. Furthermore, the computer 110 may be programmed to determine whether and when a human driver should control such operations.

The computer 110 can have more than one processor, e.g. B. included in electronic controller units (ECUs) or the like, which are included in the vehicle 105 for monitoring and / or controlling various vehicle components 125, e.g. B. a powertrain control, a brake control, a steering control, etc., include or be communicatively coupled to these, e.g. B. via the communication module 130 of the vehicle 105, as described in more detail below. Furthermore, the computer 110 can communicate with a navigation system that uses the global positioning system (GPS) via the communication module 130 of the vehicle 105. As an example, the computer 110 may request and receive location data of the vehicle 105. The location data can be in a known form, e.g. B. in geo coordinates (latitude and longitude coordinates).

The computer 110 is generally adapted to communicate via the communication module 130 of the vehicle 105 and also via an internal wired and/or wireless network of the vehicle 105, e.g. B. a bus or the like in the vehicle 105, such as a Controller Area Network (CAN) or the like, and / or other wired and / or wireless mechanisms.

Via the communication network of the vehicle 105, the computer 110 may transmit messages to various devices in the vehicle 105 and/or receive messages from the various devices, e.g. B. vehicle sensors 115, actuators 120, vehicle components 125, a human machine interface (HMI), etc. Alternatively or additionally, the communication network of the vehicle 105 in cases where the computer 110 actually includes a variety of devices used to communicate between devices, shown in this disclosure as the computer 110. Further, as noted below, various controllers and/or vehicle sensors 115 may provide data to computer 110.

The vehicle sensors 115 may include a variety of devices known to provide data to the computer 110. For example, the vehicle sensors 115 may include a light detection and ranging (LIDAR) sensor(s) 115, etc. located on a top surface of the vehicle 105, behind a windshield of the vehicle 105 , around the vehicle 105, etc., and provides relative locations, sizes and shapes of objects and/or conditions surrounding the vehicle 105. As another example, one or more radar sensors 115 attached to bumpers of the vehicle 105 may provide data to provide and classify the speed of objects (possibly including second vehicles 106), etc., relative to the location of the vehicle 105. The vehicle sensors 115 may further include camera sensor(s) 115, which e.g. B. is directed forward, to the side, backwards, etc./ are and the data from a field of view inside and / or outside the vehicle 105 provides / provide.

The actuators 120 of the vehicle 105 are implemented via circuits, chips, electric motors, or other electronic and/or mechanical components that can actuate various vehicle subsystems according to appropriate control signals, as is known. The actuators 120 may be used to control components 125, including braking, acceleration, and steering of a vehicle 105.

In the context of the present disclosure, a vehicle component 125 is one or more hardware components designed to perform a mechanical or electromechanical function or operation - such as moving the vehicle 105, braking, or stopping the vehicle 105, steering the vehicle 105, etc. Non-limiting examples of the components 125 include a drive component (e.g., including an internal combustion engine and/or an electric motor, etc.), a transmission component, a steering component (e.g., may include one or more of a steering wheel, a steering rack, etc.), a braking component (as described below), a parking assistance component, an adaptive cruise control component, an adaptive steering component, a movable seat, etc.

Furthermore, the computer 110 may be configured to communicate with devices external to the vehicle 105, e.g., via a vehicle-to-vehicle communication module or interface 130. B. via vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2X) wireless communication with another vehicle, with a remote server 145 (typically via the Communication network 135) to communicate. The module 130 could include one or more mechanisms by which the computer 110 can communicate, including any desired combination of wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired Network topology (or network topologies when a variety of communication mechanisms are used). Example communications provided via module 130 include cellular, Bluetooth®, IEEE 802.11, dedicated short range communications (DSRC), and/or wide area networks (WAN), including the Internet, which provide data communications services.

The communications network 135 may be one or more of various wired or wireless communications mechanisms, including any desired combination of wired (e.g., cable and fiber optic) and/or wireless (e.g., cellular, wireless, satellite, microwave and radio frequency) communication mechanisms and any desired network topology (or topologies if multiple communication mechanisms are used). Example communication networks include wireless communication networks (e.g., using Bluetooth, Bluetooth Low Energy (BLE), IEEE 802.11, vehicle-to-vehicle (V2V), such as short-range communication (DSRC), etc.), local area networks - LAN) and/or wide area networks (WAN), including the Internet, that provide data communication services.

A computer 110 may receive and analyze data from the sensors 115 substantially continuously, periodically, and/or as directed by a server 145, etc. Furthermore, conventional object classification or object identification techniques can be used, e.g. B. in a computer 110 based on data from a LIDAR sensor 115, a camera sensor 115 etc. to determine an object type, e.g. E.g. vehicle, person, stone, pothole, bicycle, motorcycle, etc., as well as identifying physical characteristics of objects.

2 is a block diagram of an example server 145. The server 145 includes a computer 235 and a communications module 240. The computer 235 includes a processor and memory. Memory includes one or more forms of computer-readable media and stores instructions that can be executed by computer 235 to perform various operations, including those disclosed herein. The communication module 240 allows the computer 235 to communicate with other devices, such as the vehicle 105.

3 illustrates an example domain communications network 300, which may be implemented as one or more computer programs executable by computer 110 and/or server 145. The domain transfer network 300 is a system that converts input source domain data 302 into output target domain data 342 and, at least during training, input target domain nendata 304 into output source domain data 362. For example, the domain communications network 300 may contain a sequence of data in a source domain, e.g. B. daily data, received and a sequence of data in a target domain, e.g. B. output night data.

The domain transfer network 300 processes the source domain data 302 using one or more low-level source domain encoder layers 310 of the neural network specific to data from the source domain to produce a low-level representation 312 of the input source domain data. For example, the low-level source domain encoder layers 310 of the neural network are used when encoding data from the source domain and not when encoding data from the target domain. The low level representation 312 is the output of the last of the low level encoder layers.

The domain transfer network 300 then processes the low-level representation 312 using one or more high-level neural network encoder layers 320 shared by data from the source and destination domains to create an embedding 322 of the input source domain data 302 and the input destination domain data 304, respectively to create. That is, the high-level neural network encoder layers 320 are used to generate the embedding 322 based on the source domain data and target domain data. The embedding 322 may be a vector of probability distributions, where each probability distribution represents a latent attribute or attributes of the input data. In this context, a vector means an ordered collection of numerical values and a latent attribute is a feature within the input data. For example, a latent attribute for an input image of a person may be a feature representing an eye or a nose. In another example, a latent attribute for an input image of a vehicle may be a feature representing a tire, a bumper, or a vehicle body portion.

The domain transfer network 300 processes the embedding 322 of the input source domain data using one or more high-level decoder layers 330 of the neural network shared by data from the source and target domains to generate a high-level feature representation 332 of features of the input source domain data 302 . The high-level representation of a latent attribute is the output of the last of the high-level encoder layers 330.

The domain transfer network 300 then processes the high-level feature representation 332 of the features of the input source domain data using one or more low-level target domain decoder layers 340 of the neural network specific to generating data from the source domain to generate output target domain data 342 derived from the target domain but have similar semantics to the input source domain data 302. Similar semantics means that the target domain data 342 has regions of pixel values that correspond to the same objects as regions of pixel values in input source domain data 302. For example, the output target domain data 342 may have a distribution of pixel values that matches that of data from the target domain, but have similar semantics to the input source domain data 302, meaning that the output destination domain data 342 includes objects that are recognizable by a user as the same objects included in the input source domain data 302. For example, target domain data 342 may include a trailer that has an A-frame trailer tongue that corresponds to an A-frame trailer tongue that occurs in the input source domain data 302.

During training, the domain transfer network 300 may also generate output source domain data 362 from input target domain data 304, i.e. that is, to transform target domain data into source domain data that has similar semantics to the original target domain data.

In an exemplary implementation, the domain transfer network 300 processes the target domain data 304 using one or more low-level target domain encoder layers 350 of the neural network specific to data from the target domain to generate a low-level representation 352 of the input source domain data to convert input target domain data 304 . The low-level target domain encoder layers 350 of the neural network can only be used when encoding data from the target domain and not when encoding data from the source domain.

The domain transfer network 300 then processes the low-level representation 352 using the one or more high-level neural network encoder layers 320 shared by data from the source and target domains to generate an embedding 324 of the input target domain data 304.

The domain transfer network 300 processes the embedding 324 of the input target domain data using the one or more high-level decoder layers 330 of the neural network shared by data from the source and target domains to produce a high-level feature representation 334 of features of the target domain data 304 generate. Similar to embedding 322, embedding 324 may be a vector of probability distributions, where each probability distribution represents a latent attribute of the input data.

The domain transfer network 300 then processes the high-level feature representation 334 of the features of the input target domain data using one or more low-level source domain decoder layers 360 of the neural network specific to generating data from the source domain to generate output source domain data 362 derived from the Source domain but have similar semantics to the input target domain data 304. That is, the output source domain data 362 has distribution data values that are consistent with data from the source domain but have similar semantics to the input target domain data 304. For example, a source domain image that has one or more objects mapped therein may be generated for the target domain such that the generated target domain images appear to be from the target domain but retain the semantics of the corresponding source domain images, e.g. B. depict one or more objects. During training, the low-level target domain decoder layers 340 of the neural network are trained together with the target domain decoder layers 350 of the neural network and the source domain decoder layers 310 of the low-level neural network.

4 _illustrates an exemplary diagram of the domain communications network 300 that receives input data X _A , X _B and can generate output data X' _AA , X _{' BA ,} As shown, the target low-level domain encoder layers 350 of the neural network may receive an input X _A and the source domain encoder layers 310 of the low-level neural network may receive an input X _B. The domain transfer network 300 can perform inter- and intra-domain conversions because the low-level target domain encoder layers 350 of the neural network and the low-level source domain encoder layers 310 with the low-level source domain decoder layers 360 of the neural network and the low-level target domain decoder layers 340 of the neural network via shared high-level neural network encoder layers 320 and shared high-level neural network decoder layers 330. The domain communications network 300 is illustrated with shared high-level neural network encoder layers 320 and shared high-level network decoder layers 330 that share weights. The domain transfer network 300 may also operate with high-level neural network encoder layers 320 and high-level network decoder layers 330 that do not share weights.

The _input data a daily image, and the input data X _B may be data in the target domain, where the index "B" represents a second domain of the data, e.g. B. a night picture. The data X' _AA , X' _BA , X' _{AB ,} For example, if the input data is an image depicting one or more objects in the first domain, the domain communications network 300 may generate images X' _AA , X' _BA , X' _AB , X' _BB representing an image depicting the one or the multiple objects in the first domain (X' _AA ), e.g. B. during the day, a second domain (X' _BB ), e.g. B. at night, a third domain (X' _AB ), e.g. B. in the morning, and a fourth domain (X' _BB ), e.g. B. at dusk. The elements Z _AA , Z _AB , Z _BA , Z _BB represent the latent attribute or attributes generated by the high-level encoder layers 320 of the neural network.

During training, the domain transfer network 300 may modify a traditional loss function of the domain transfer network 300 so that latent attributes are selected from the same probability distribution. A traditional loss function can determine a loss value by calculating a probability of an outcome and ground truth data of the same corresponding probability distribution. Techniques described in this paper add the condition that latent attributes are selected from the same probability distribution. By selecting latent attributes from the same probability distribution, features within the input source domain data 304 can be represented in the output target domain data 342. For example, as discussed above, features may represent portions of objects depicted in an image.

$$L=LF+MMD\left(Z_{AA},Z_{AB}\right)$$

$$L=LF+MMD\left(Z_{AA},Z_{BA}\right)$$

$$L=LF+MMD\left(Z_{AB},Z_{BA}\right)$$

$$L=LF+MMD\left(Z_{AB},Z_{BB}\right)$$

$$L=LF+MMD\left(Z_{BA},Z_{BB}\right)$$

wobei L als eine Verlustfunktion des Domänenübertragungsnetzwerks 300 definiert ist, LF als eine herkömmliche Verlustfunktion definiert ist und MMD als die maximale mittlere Abweichung zwischen den latenten Attributen definiert ist. Während des Trainings kann die Verlustfunktion L verwendet werden, um eine oder mehrere Gewichtungen innerhalb des Domänenübertragungsnetzwerks 300 zu aktualisieren.The conventional loss function can be achieved by minimizing a conventional maximum mean discrepancy (MMD) between embeddings 322, 324, e.g. B. the latent attributes Z _AA , Z _AB , Z _BA , Z _BB , can be modified. The maximum mean deviation may be defined as a numerical difference between the embeddings 322, 324, such as the difference between a mean of a probability distribution from the embedding 322 and a mean of a probability distribution from the embedding 324. During training of the domain transfer network 300, the maximum mean deviation can be used to modify the traditional loss function to update one or more weights within the domain transmission network 300. Equations 1 to 6 illustrate the modification of the traditional loss function (LF) using the maximum mean deviation: $$L=LF+MMD\left(Z_{AA},Z_{bb}\right)$$

$$L=LF+MMD\left(Z_{AA},Z_{Ab}\right)$$

$$L=LF+MMD\left(Z_{AA},Z_{bA}\right)$$

$$L=LF+MMD\left(Z_{Ab},Z_{bA}\right)$$

$$L=LF+MMD\left(Z_{Ab},Z_{bb}\right)$$

$$L=LF+MMD\left(Z_{bA},Z_{bb}\right)$$

where L is defined as a loss function of the domain communication network 300, LF is defined as a conventional loss function, and MMD is defined as the maximum mean deviation between the latent attributes. During training, the loss function L may be used to update one or more weights within the domain transmission network 300.

As in 5 As shown, the domain communications network 300 may include a discriminator 502 in some implementations. An output of the discriminator 502 can be used to update the traditional loss function during training so that latent attributes are selected from the same probability distribution during data generation. The discriminator 502 can receive the latent attributes Z _AA , Z _AB , Z _BA , Z _BB , evaluate the latent attributes Z _AA , Z _AB , Z _BA , Z _BB , and generate a prediction indicating whether the latent attributes Z _AA , Z _AB , Z _BA , Z _BB correspond to a specific domain. In various implementations, the discriminator 502 may include one or more Convolution BatchNorm ReLU layers, as discussed below.

Like in the 6A until 6D As shown, the discriminator 502 may receive a latent attribute and output a multidimensional vector 602 that classifies the corresponding latent attribute to a particular domain. In the in the 6A until 6D In the illustrated examples, the multidimensional vector 602 includes a four-dimensional vector that includes a predictive label representing which domain the corresponding latent attribute comes from. For example, the discriminator 502 produces a "1" within the multidimensional vector 602 corresponding to the predicted domain and a "0" for all other domains. In some implementations, the multidimensional vector 602 may include a two-dimensional vector 602. In these implementations, the discriminator 502 produces a prediction between the first domain and the second domain.

wobei L_D als eine Verlustfunktion für den Diskriminator 502 definiert ist, Z̃_AA, Z̃_BB, Z̃_AB, Z̃_BA als Kennzeichnungen für die entsprechende Domäne definiert sind, log D als die Schätzung des Diskriminators 302 definiert ist, dass die Wahrscheinlichkeit für das latente Attribut einer spezifischen Domäne entspricht, und Z_AA, Z_AB, Z_BA, Z_BB als vorhergesagte Domänenausgaben aus dem Diskriminator 502 definiert sind, z. B. „0“ oder „1“. Gleichung 8 veranschaulicht eine Verlustfunktion für einen Diskriminator, die einen zweidimensionalen Vektor erzeugt: $$L_D={\tilde{Z}}_{AA}logD\left(Z_{AA}\right)+{\tilde{Z}}_{BB}logD\left(Z_{BB}\right)$$

wobei L_D als eine Verlustfunktion für den Diskriminator 502 definiert ist, Z̃_AA, Z̃_BB als Kennzeichnungen für die entsprechende Domäne definiert sind, log D als die Schätzung des Diskriminators 302 definiert ist, dass die Wahrscheinlichkeit für das latente Attribut einer spezifischen Domäne entspricht, und Z_AA, Z_BB als vorhergesagte Domänenausgaben aus dem Diskriminator 502 definiert sind, z. B. „0“ oder „1“. Die Verlustfunktion L_D kann verwendet werden, um Gewichtungen des Diskriminators 502 und/oder einer herkömmlichen Verlustfunktion des Domänenübertragungsnetzwerks 300 zu aktualisieren.Equation 7 illustrates a loss function for a discriminator 502 that produces a four-dimensional vector: $$L_D={\tilde{Z}}_{AA}lOGD\left(Z_{AA}\right)+{\tilde{Z}}_{bb}lOGD\left(Z_{bb}\right)+{\tilde{Z}}_{Ab}lOGD\left(Z_{Ab}\right)+{\tilde{Z}}_{bA}lOGD\left(Z_{bA}\right)$$

where L _D is defined as a loss function for the discriminator 502, Z̃ _AA , Z̃ _BB , Z̃ _AB , Z̃ _BA are defined as labels for the corresponding domain, log D is defined as the discriminator 302's estimate that the probability for the latent Attribute corresponds to a specific domain, and Z _AA , Z _AB , Z _BA , Z _BB are defined as predicted domain outputs from the discriminator 502, e.g. B. “0” or “1”. Equation 8 illustrates a loss function for a discriminator that produces a two-dimensional vector: $$L_D={\tilde{Z}}_{AA}lOGD\left(Z_{AA}\right)+{\tilde{Z}}_{bb}lOGD\left(Z_{bb}\right)$$

where L _D is defined as a loss function for the discriminator 502, Z̃ _AA , Z̃ _BB are defined as labels for the corresponding domain, log D is defined as the discriminator 302's estimate that the probability for the latent attribute corresponds to a specific domain, and Z _AA , Z _BB are defined as predicted domain outputs from the discriminator 502, e.g. B. “0” or “1”. The loss function L _D may be used to update weights of the discriminator 502 and/or a conventional loss function of the domain communications network 300.

7 is a block diagram illustrating an example of the discriminator 502. The discriminator architecture is assumed to be convolutional, using standard components internally (e.g. Conv-BatchNorm-ReLU). Conv-BatchNorm-ReLU is a neural network training technique that processes latent variables between convolutional layers of a neural network using batch normalization (BatchNorm) to avoid problems caused by highly varying input data values and rectified linear activation ( ReLU) is performed to accelerate convergence during training. The latent variables can be batched by storing and normalizing results from a series of training runs to give the batched latent variables a mean of zero and a standard deviation of one without changing output results. The normalized latent variables can then undergo ReLU by outputting positive values directly to the subsequent stage and converting negative values to zero.

The discriminator 502 may include several different types of layers based on connectivity and weight distribution. As in 7 As shown, the discriminator 502 may include one or more convolution layers (CONV) 702, one or more batch normalization layers (BatchNorm) 704, and one or more rectified linear unit (ReLU) layers 706. The convolution layers 702 may include one or more convolution filters that are applied to the latent attributes. The filtered latent attributes may be provided to the batch normalization layers 704 and the batch normalization layers 704 normalize the filtered latent attributes. The normalized filtered latent attributes may be provided to the rectified linear unit layers 706 and the rectified linear unit layers 706 include an activation function, e.g. B. a piecewise linear function that produces an output based on the normalized filtered latent attributes. The output of the rectified linear unit layers 706 may be provided as input to a softmax layer 708 to generate the predicted domain. The softmax layer 708 includes the last layer of the discriminator 502 and is an activation function that produces a probability of which domain the normalized filtered latent attributes belong to. While only a single convolution layer 702, batch normalization layer 704, rectified unit layer 706, and softmax layer 708 are shown, the discriminator 502 may include additional layers depending on the implementation of the discriminator 502.

A deep neural network trained using the techniques described in this paper can improve the training of deep neural networks by enabling one-shot or few-shot training. As discussed above, generating training datasets for a deep neural network may require acquiring thousands of example input images along with the corresponding ground truth. The training data set should contain a variety of examples of all different types of objects trained in examples of all environmental conditions expected during operation of the deep neural network. For example, a deep neural network can be trained to identify and locate vehicle trailers. During training, all types and configurations of trailers that may occur when operating a vehicle should be included in a training data set. Furthermore, each type and configuration of followers in the training data set should be included in each of the different environmental conditions that may occur when operating the deep neural network. Environmental conditions include weather and lighting conditions such as rain, snow, fog, bright sunlight, night, etc.

Acquiring a training data set that includes a variety of images containing all types and configurations of objects to be identified and located under all types of environmental conditions can be expensive and time-consuming. Techniques discussed in this paper can improve the training of deep neural networks by allowing the deep neural networks to be trained using a single input image and modifying the input image to produce different types of objects in different configurations under different environmental conditions simulate. For example, a new type or configuration of an object may be detected by a vehicle during operation. The single image of an object can be sent back to a server computer and used to retrain a deep neural network by simulating a variety of orientations and positions of the new object based on previously acquired training images that contain ground truth. Techniques described in this paper enable training a deep neural network using limited training data sets, saving time and money.

Techniques described in this paper can be applied to deep neural networks that process image data, video data, and human speech data. Images can be processed using convolutional neural networks. Convolutional neural networks include convolutional layers that encode images to form latent variables that can be decoded by convolutional layers that reconstruct the latent variables to form output image data. Video data and human speech data can be processed using recurrent neural networks. Recurrent neural networks include memory that stores results from a variety of previous decoding layers and previous encoding layers to combine with current encoding and decoding layers. In an example recurrent neural network that processes video data, the encoding and decoding layers may include convolutional layers. In an example recurrent neural network that processes human language, the encoding and decoding layers may be fully connected layers. Example convolutional neural networks or recurrent neural networks may be configured as domain transfer networks that include loss functions and discriminators as described above with respect to 3-7 described to enable convolutional neural networks or recurrent neural networks to process image data, video data or human speech data from a source domain to a target domain and thereby improve the ability to train neural networks by increasing the amount of training data without additional collection of real data and corresponding ground truth is required, saving time and money.

8th illustrates an example deep neural network (DNN) 800 that can perform the functions described above and in this document. For example, the domain communications network 300 may be implemented as one or more DNNs 800. For example, the DNN 800 may be a software program that can be loaded into memory and executed by a processor included in the computer 110 or the server 145. In an exemplary implementation, the DNN 800 may include, but is not limited to, a convolutional neural network (CNN), R-CNN (Regions with CNN Features), fast R-CNN, faster R-CNN, and recurrent neural networks (RNNs). The DNN 800 includes a plurality of nodes 805, and the nodes 805 are arranged such that the DNN 800 includes an input layer, one or more hidden layers, and an output layer. Each layer of the DNN 800 may include a plurality of nodes 805. While 8th illustrating three (3) hidden layers, it is understood that the DNN may include 800 additional or fewer hidden layers. The input and output layers may also include more than one (1) node 805.

The nodes 805 are sometimes referred to as artificial neurons 805 because they are designed to generate biological, e.g. B. to simulate human neurons. A set of inputs (represented by the arrows) to each neuron 805 are each multiplied by respective weights. The weighted inputs can then be summed in an input function to provide a net input, possibly adjusted for bias. The net input may then be provided to an activation function, which in turn provides an output to a connected neuron 805. The activation function may be a variety of suitable functions, typically selected based on empirical analysis. As shown by the arrows in 8th As illustrated, the outputs of neuron 805 may then be provided in a next layer for inclusion in a set of inputs to one or more neurons 805.

The DNN 800 can be trained to read data, e.g. B. from the CAN bus of the vehicle 105, sensors or another network, to accept as input and to generate a distribution of possible outputs based on the input. The DNN 800 can be trained using ground truth data, that is, data about a real-world condition or state. For example, the DNN 800 may be trained with ground truth data or updated with additional data by a processor of the server 145. The DNN 800 can be transmitted to the vehicle 105 via the network 135. For example, the weights may be initiated using a Gaussian distribution and a bias for each node 805 may be set to zero. Training the DNN 800 may include updating weights and biases through appropriate techniques, such as backpropagation with optimizations. Ground truth data may include, but is not limited to, data that specifies objects within data or data that specifies a physical parameter, e.g. B. Angle, speed, distance or angle of an object relative to another object.

9 is a flowchart of an example process 900 for generating target domain data from source domain data. Blocks of process 900 may be executed by a processor of computer 110 and/or server 145. The process 900 may begin at block 905 in which data, e.g. B. an image, received from the source domain. The source domain data is processed at block 910 using one or more low-level source domain encoder layers of the neural network specific to data from the source domain to produce a low-level representation of the input source domain data.

At block 915, the low-level representation is processed using one or more high-level encoder layers of the neural network shared by data from the source and target domains to create an embedding, e.g. B. latent attributes of the input target domain data. At block 920, the embeddings of the input source domain data are processed using one or more high-level decoder layers of the neural network shared by data from the source and target domains to produce a high-level feature representation of features of the input source domain data.

At block 925, the high-level feature representation of the features of the input source domain data is processed using one or more low-level target domain decoder layers of the neural network specific to generating data from the target domain to generate output target domain data derived from the target domain , but have similar semantics to the input source domain data.

At block 930, one or more weights of the domain communications network are updated based on a loss function. In an exemplary implementation, the maximum average deviation between different embeddings is calculated. In this implementation, the maximum mean deviation is used to modify a traditional loss function. In another exemplary implementation, the domain transfer network includes a discriminator, and the discriminator produces a prediction of which domain an embedding belongs to. The prediction can be compared to ground truth data and the loss function for the discriminator can be updated based on the comparison. Additionally or alternatively, weights of the domain transmission network may be updated based on the comparison. The domain transmission network can change its weights according to an update rule, e.g. B. an ADAM update rule, a stochastic gradient descent (SGD) update rule. Process 900 then ends.

10 is a flowchart of an example process 1000 for generating source domain data from target domain data. Blocks of process 1000 may be executed by a processor of computer 110 and/or server 145. The process 1000 may begin at block 1005 in which data from the target domain is received. The target domain data is processed at block 1010 using one or more low-level target domain encoder layers of the neural network specific to data from the target domain to produce a low-level representation of the input target domain data. At block 1015, the low-level representation is processed using the one or more high-level encoder layers of the neural network shared by data from the source and target domains to produce an embedding of the input target domain data.

At block 1020, the embedding of the input target domain data is processed using the one or more high-level decoder layers of the neural network shared by data from the source and target domains to produce a high-level feature representation of features of the input target domain data. At block 1025, the high-level feature representation of the features of the input target domain data is processed using one or more low-level source domain decoder layers of the neural network specific to generating data from the source domain to generate output source domain data derived from the source domain , but have similar semantics to the input source domain data.

At block 1030, one or more weights of the domain communications network are updated based on a loss function. In an exemplary implementation, the maximum average deviation between different embeddings is calculated. In this implementation, the maximum mean deviation is used to modify a traditional loss function. In another exemplary implementation, the domain transfer network includes a discriminator, and the discriminator produces a prediction of which domain an embedding belongs to. The prediction can be compared to ground truth data and the loss function for the discriminator can be updated based on the comparison. Additionally or alternatively, weights of the domain transmission network may be updated based on the comparison. The domain transmission network can change its weights according to an update rule, e.g. B. an ADAM update rule, a stochastic gradient descent (SGD) update rule.

In general, the computing systems and/or devices described may employ any of a variety of computer operating systems, including, but not limited to, versions and/or variants of the Ford Sync® application, the AppLink/Smart Device Link middleware, the Microsoft Automotive® operating system, the operating system Microsoft Windows®, the Unix operating system (e.g. the Solaris® operating system, distributed by Oracle Corporation of Redwood Shores, California), the AIX UNIX operating system, distributed by International Business Machines of Armonk, New York, the Linux operating system, the Operating systems Mac OSX and iOS, distributed by Apple Inc. in Cupertino, California, the BlackBerry OS, distributed by Blackberry, Ltd. in Waterloo, Canada, and the Android operating system, developed by Google, Inc. and the Open Handset Alliance, or the QNX® CAR Platform for Infotainment, offered by QNX Software Systems. Examples of computing devices include, but are not limited to, the following: a vehicle onboard computer, a computer workstation, a server, a desktop, notebook, laptop or handheld computer or other computing system and/or other computing device.

Computers and computing devices generally include computer-executable instructions, where the instructions may be executed by one or more computing devices, such as those listed above. Computer-executable instructions may be compiled or evaluated by computer programs created using a variety of programming languages and/or technologies, including, but not limited to, and either alone or in combination, Java™, C, C++, Matlab, Simulink, Stateflow, Visual Basic , Java Script, Perl, HTML, etc. Some of these applications can be compiled and executed on a virtual machine, such as the Java Virtual Machine, the Dalvik Virtual Machine or the like. In general, a processor (e.g. a microprocessor) receives instructions, e.g. B. from memory, a computer-readable medium, etc., and executes those instructions, thereby performing one or more processes, including one or more of the processes described in this document. Such instructions and other data may be stored and transmitted using a variety of computer-readable media. A file in a computing device is generally a collection of data stored on a computer-readable medium such as a storage medium, random access memory, etc.

Memory may include a computer-readable medium (also referred to as a processor-readable medium), which includes any non-transitory (e.g., tangible) medium that is involved in providing data (e.g., instructions) to be read by a computer ( B. by a processor of a computer) can be read. Such a medium may take many forms, including, but not limited to, non-volatile media and volatile media. Non-volatile media can include, for example, image and magnetic disks and other permanent storage devices. Volatile media may include, for example, dynamic random-access memory (DRAM), which is typically main memory. Such instructions may be transmitted through one or more transmission media, including coaxial cables, copper wire and fiber optics, including Wires that make up a system bus that is coupled to a processor of an ECU. Common forms of computer-readable media include, for example, the following: a floppy disk, a foil storage disk, a hard drive, a magnetic tape, any other magnetic medium, a CD-ROM, a DVD, any other optical medium, punched cards, punched tape, any other physical medium with hole patterns, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or any other memory cartridge or any other medium that can be read by a computer.

Databases, data repositories or other data stores described in this document may include various types of mechanisms for storing, accessing and retrieving various types of data, including a hierarchical database, a set of files in a file system, an application database in a user's own format, a relational database management system (RDBMS), etc. Each such data store is generally included in a computing device employing a computer operating system, such as one of those mentioned above, and is implemented in one or more of a variety of ways accessed via a network. A file system may be accessible from a computer operating system and may include files stored in various formats. An RDBMS generally uses the Structured Query Language (SQL) in addition to a language for creating, storing, editing, and executing stored procedures, such as the previously mentioned PL/SQL language.

In some examples, system elements may be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) stored on computer-readable media (e.g., disks, memories, etc.) associated therewith .) are saved. A computer program product may include such instructions stored on computer-readable media for performing the functions described in this document.

With respect to the media, processes, systems, procedures, heuristics, etc. described in this document, it is to be understood that although the steps of such processes, etc. have been described as occurring in a particular order, such processes may be implemented in such a way that the steps described are carried out in an order that differs from the order described in this document. It is further understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the descriptions of processes in this document are for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claims.

Accordingly, it is to be understood that the foregoing description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided will become apparent to those skilled in the art upon reading the foregoing description. The scope of the invention should be determined not with reference to the foregoing description, but rather with reference to the appended claims in conjunction with the full scope of equivalents to which such claims are entitled. It is expected and intended that there will be future developments in the prior art discussed in this document and that the systems and methods disclosed will be incorporated into such future embodiments. Overall, it is to be understood that the invention may be modified and varied and is limited solely by the following claims.

All expressions used in the claims are intended to have their general and ordinary meaning as understood by those skilled in the art, unless expressly stated otherwise in this document. In particular, the use of the singular articles, such as "a", "an", "the", "the", "that", etc., is to be interpreted as meaning that one or more of the listed elements are mentioned, unless a patent claim expressly states this states the opposite restriction.

According to the present invention, there is provided a system comprising a computer including a processor and a memory, the memory storing instructions executable by the processor to cause the processor to: generate a low-level representation of the Input source domain data by processing source domain data using a low-level source domain encoder layer of a neural network corresponding to data from the source domain to form a low-level representation of the input source domain to generate personal data; generating an embedding of the input source domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; Generate a high-level feature representation of features of the input source domain data by processing the embedding of the input source domain image using a high-level decoder layer of the neural network shared by data from the source and target domains to produce a high-level feature representation of features of the input source domain data to create; generating output target domain data in the target domain that includes semantics corresponding to the input source domain data by processing the high-level feature representation of the features of the input source domain data using a low-level domain decoder layer of the neural network that generates data from the target; and modifying a loss function so that latent attributes corresponding to the embedding are selected from an equal probability distribution.

According to one embodiment, the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in convolutional neural networks.

According to one embodiment, the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in recurrent neural networks.

According to one embodiment, the input source domain and the target domain include image data, video data, and human speech data.

According to one embodiment, the processor is further programmed to modify the loss function by calculating a maximum average deviation between a first latent attribute corresponding to a source domain and a second latent attribute corresponding to a target domain.

According to one embodiment, the processor is further programmed to modify the loss function based on a prediction from a discriminator, the prediction indicating a domain corresponding to a latent attribute.

According to one embodiment, the discriminator includes one or more convolution layers, one or more batch normalization layers, and one or more rectified linear unit layers.

According to one embodiment, a final layer of the discriminator comprises a softmax layer.

According to one embodiment, the discriminator generates a multidimensional vector representing the prediction.

According to one embodiment, the multidimensional vector includes a four-dimensional vector corresponding to four domains.

According to one embodiment, the multidimensional vector includes a two-dimensional vector corresponding to two domains.

According to one embodiment, a loss function for the discriminator comprises L _D = Z̃ _AA logD(Z _AA )+Z̃ _BB logD(Z _BB ) + Z̃ _ABl ogD (Z _AB ) +Z _BA logD (Z _BA ) _' where L _D is the loss function is defined, Z̃ _AA , Z̃ _BB , Z̃ _AB , Z̃ _BA are defined as labels for the corresponding domain, log D is defined as an estimate that the probability for the latent attribute corresponds to a specific domain, and Z _AA , Z _AB , Z _BA , Z _BB are defined as predicted domain outputs.

According to one embodiment, the processor is further programmed to generate a low-level representation of the input target domain data by processing the input target domain data using a low-level target domain encoder layer of a neural network specific to data from the target domain; an embedding of the input target domain data through processing generate the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; generate a high-level feature representation of features of the input target domain data by processing the embedding of the input target domain image using the high-level decoder layer of the neural network shared by data from the source and target domains; and generate output source domain data from the source domain containing semantics corresponding to the input target domain data by processing the high-level feature representation of the features of the target source domain image using a low-level source domain decoder layer of the neural network specific to data from the source domain.

According to the present invention, a method includes: generating a low-level representation of the input source domain data by processing source domain data using a low-level source domain encoder layer of a neural network corresponding to data from the source domain to generate a low-level representation of the input source domain data; generating an embedding of the input source domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; Generate a high-level feature representation of features of the input source domain data by processing the embedding of the input source domain image using a high-level decoder layer of the neural network shared by data from the source and target domains to produce a high-level feature representation of features of the input source domain data to create; generating output target domain data in the target domain that includes semantics corresponding to the input source domain data by processing the high-level feature representation of the features of the input source domain data using a low-level domain decoder layer of the neural network that generates data from the target; and modifying a loss function so that latent attributes corresponding to the embedding are selected from an equal probability distribution.

In one aspect of the invention, the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in convolutional neural networks.

In one aspect of the invention, the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in recurrent neural networks.

In one aspect of the invention, the input source domain and the target domain include image data, video data and human speech data.

In one aspect of the invention, the method includes modifying the loss function by calculating a maximum mean deviation between a first latent attribute corresponding to a source domain and a second latent attribute corresponding to a target domain.

In one aspect of the invention, the method includes modifying the loss function based on a prediction from a discriminator, the prediction indicating a domain corresponding to a latent attribute.

In one aspect of the invention, the discriminator includes one or more convolution layers, one or more batch normalization layers, and one or more rectified linear unit layers.

Claims (15)

A method comprising: generating a low-level representation of the input source domain data by processing source domain data using a low-level source domain encoder layer of a neural network corresponding to data from the source domain to generate a low-level representation of the input source domain data; generating an embedding of the input source domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; Generate a high-level feature representation of features of the input source domain data by processing the embedding of the input source domain image using a high-level decoder layer of the neural network shared by data from the source and target domains to produce a high-level feature representation of features of the input source domain data to create; generating output target domain data in the target domain that includes semantics corresponding to the input source domain data by processing the high-level feature representation of the features of the input source domain data using a low-level domain decoder layer of the neural network that generates data from the target; and modifying a loss function so that latent attributes corresponding to the embedding are selected from an equal probability distribution. Procedure according to Claim 1 , wherein the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in convolutional neural networks. Procedure according to Claim 1 , wherein the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in recurrent neural networks. Procedure according to Claim 1 , where the input source domain and the target domain include image data, video data and human language data. Procedure according to Claim 1 , wherein the processor is further programmed to modify the loss function by calculating a maximum mean deviation between a first latent attribute corresponding to a source domain and a second latent attribute corresponding to a target domain. Procedure according to Claim 1 , wherein the processor is further programmed to modify the loss function based on a prediction from a discriminator, the prediction indicating a domain corresponding to a latent attribute. Procedure according to Claim 6 , wherein the discriminator includes one or more convolution layers, one or more batch normalization layers, and one or more rectified linear unit layers. Procedure according to Claim 7 , where a final layer of the discriminator includes a softmax layer. Procedure according to Claim 6 , where the discriminator produces a multidimensional vector representing the prediction. Procedure according to Claim 9 , wherein the multidimensional vector includes a four-dimensional vector corresponding to four domains. Procedure according to Claim 9 , where the multidimensional vector comprises a two-dimensional vector corresponding to two domains. Procedure according to Claim 6 , where a loss function for the discriminator includes LD _D = Z̃ _AA log D(Z _AA )+Z̃ _BB log D(Z _BB ) + Z̃ _AB log D(Z _AB )+Z̃ _BA log D(Z _BA ), where L _D is defined as the loss function, Z̃ _AA , Z̃ _BB , Z̃ _AB , Z̃ _BA are defined as labels for the corresponding domain, log D is defined as an estimate that the probability for the latent attribute corresponds to a specific domain, and Z _AA , Z _AB , Z _BA , Z _BB are defined as predicted domain outputs. Procedure according to Claim 1 , wherein the processor is further defined to generate a low-level representation of the input target domain data by processing the input target domain data using a target domain encoder layer of a low-level neural network specific to data from the target domain; generate an embedding of the input target domain data by processing the low-level representation using a high-level encoder layer of the neural network shared by data from the source and target domains; generate a high-level feature representation of features of the input target domain data by processing the embedding of the input target domain image using the high-level decoder layer of the neural network shared by data from the source and target domains; and generate output source domain data from the source domain containing semantics corresponding to the input target domain data by processing the high-level feature representation of the features of the target source domain image using a low-level source domain decoder layer of the neural network specific to data from the source domain. Procedure according to Claim 13 , wherein the low-level neural network encoder layer, the high-level neural network encoder layer, the low-level neural network decoder layer, and the high-level neural network decoder layer are included in convolutional neural networks. System comprising a computer programmed to carry out the methods according to one of the Claims 1 - 14 to carry out.

Images