FR3104292A1 - Method of configuring an imaging device of a motor vehicle comprising an optical image capture device - Google Patents

Abstract

The method comprises the training of a neural network RNe1 associated with the optical device C1 comprising, for each index i: A) supply of N images IMn, i respectively to N neural networks RNen, including the target network RNe1 and N- 1 ancillary networks RNe2,…, RNeN, previously captured by N optical devices Cn, including a reference device C1 'and N-1 ancillary devices Cn mounted offset from C1'; B) coding of the image IMn, i by each network RNen, C) for each of the N-1 pairs of images (IM1, i, IMn, i) with n ranging from 2 to N, - calculation of the distance between the two codes CD1, i and CDn, i, - prediction of a class by binary classification of said distance, and - calculation of an error between the predicted class and a real class; D) adjustment of the connection weights between neurons of the networks RNe1 and RNen, as a function of said error. Figure to be published with the abstract: Fig. 2

Description

Method of configuring an imaging device of a motor vehicle comprising an optical image capture device

The present invention relates generally to a method of configuring an imaging device of a motor vehicle comprising an optical device C1 and an associated _{encoder neural network RNe 1.}

PRIOR ART

In the automotive field, more and more vehicles are now equipped with so-called "intelligent" cameras, capturing images of the environment around the vehicle and using deep neural networks for the detection and recognition of objects. . These neural networks are of the "deep" type: they have many layers of neurons, and are trained on millions of input-output pairs, intended for learning, in order to develop the performances deemed sufficient to carry out a task. desired, for example generic recognition of an object such as a pedestrian. Currently, generic object recognition uses so-called residual convolutional neural networks, such as ResNeXt101 (Xie, Girshick, Dollár, Tu, & He (2017) Aggregated Residual Transformations for Deep Neural Networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)).

These neural networks must be trained beforehand, during a learning phase. Typically, learning of smart camera neural networks is done in a supervised manner.

The idea of supervised learning of a neural network is to provide many training pairs, or input-output pairs, with each training pair containing known input data and known output data, and to adjust the weights of connections between neurons in order to minimize the expression of the error at the output of the neural network. In supervised learning, the neural network is thus trained and trained by providing pairs of matched input data and output data, with the aim of the neural network providing a desired output for a given input.

Supervised learning of a neural network is very powerful but also very expensive in human time to analyze, annotate and verify data. It requires the production of a gigantic database of known input-output pairs, analyzed and labeled (or tagged) by human experts. For example, training a neural network to detect vehicles requires millions of camera images on which vehicles have already been detected by human experts, and surrounded in the image by bounding boxes. This labeling work by human experts is expensive and, in this fully supervised approach, the quantity and quality of the labels (or labels) is decisive for the performance of the neural network.

Deep neural networks provide excellent performance in visual recognition. However, these performanes are limited due to the supervised training which introduces a bias towards a certain type of solution: the network seeks to identify the non-linear combinations of pixels, known under the name of “codes” or “features”, most useful for solving the task, without seeking to form a prior representation of the object that is independent of the task. As a result, deep neural networks are vulnerable to attacks by antagonistic or adverse images which would be harmless to the human eye, as explained in the document “Eykholt, Evtimov, Fernandes, Li, Rahmati, Xiao, Prakash, Kohno, & Song (2018) Robust Physical-World Attacks on Deep Learning Visual Classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ”. Thus, deep neural networks have difficulty adapting to new tasks, for which they have not been specifically trained, and are therefore generally not very versatile. In addition, this supervised training is also found to limit the performance of deep networks even on the target task of training, as explained in the document "Arandjelovic & Zisserman (2018) Objects that sound. European Conference on Computer Vision (ECCV) ”. A deep network previously trained in a self-supervised manner, before being trained on a visual classification task by conventional supervision, exceeds the performance of a network of the same size trained only by supervision. More precisely, the model described by Arandjelovic & Zisserman is composed of two networks (visual and auditory), and trained to determine whether audio and visual input data comes from the same video sequence or not. The labels (or tags) for this task are binary, so as to indicate a match or a non-match, and are constructed automatically from videos obtained from the "YouTube" site, without requiring human decision. This type of learning is said to be "self-supervised".

An aim of the invention is to configure an imaging device of a motor vehicle comprising a sensor and at least one associated encoder neural network, by training the encoder neural network, so as to develop codes, representative of scenes. detected, which are more robust.

With this aim and in a first aspect, the present invention relates to a method of configuring an imaging device of a motor vehicle comprising an optical device C1 for capturing images and an _{associated encoder neural network RNe 1} , comprising a training phase of the encoder neural network RNe ₁ comprising an iterative process which comprises, for each iteration of index i, the steps of:
A) supply of N images IM _{n, i} , n varying from 1 to N, respectively to N encoder neural networks RNe _n , including the neural network RNe ₁ associated with the optical device C1, as target network, and N- 1 neural networks RNe ₂ ,…, RNe _N , the N images having been captured beforehand by N optical image capture devices Cn mounted on an image acquisition vehicle, including a reference optical device, corresponding to the optical device C1, having a reference field of vision, and N-1 ancillary optical devices Cn, with n ranging from 2 to N, mounted on the acquisition vehicle offset from the reference optical device so as to have respective fields of vision exhibiting an overlap rate with the reference field of vision greater than 70%;
B) encoding of the image IM _{n, i} supplied in a descriptor code CD _{n, i} , by each encoder neural network RNe _n ,
C) for each of the N-1 pairs of images including image IM _{1, i} and image IM _{n, i} with n ranging from 2 to N,
- calculation of the distance between the two corresponding descriptor codes CD _{1, i} and CD _{n, i} ,
- prediction of a class, representative of correspondence or non-correspondence information of the images IM _{1, i} and IM _{n, i} of the pair, by binary classification of said distance, and
- calculation of an error between the predicted class and a real class given by a target label associated with said pair of images IM _{1, i} and IM _{n, i} and previously known,
D) a step of adjusting the connection weights between neurons of the encoder neural networks RNe ₁ and RNe _n , as a function of said error.

According to the invention, a target neural network, corresponding to the encoder neural network dedicated to the optical device of the vehicle (on board the vehicle), and one or more annex neural networks, dedicated to views different from that of the optical device of the vehicle. vehicle, are trained to detect the correspondence between views. The terms “different views” are understood to denote views taken by ancillary optical devices each having a field of view which is almost the same as that of the vehicle optical device, the rate of coverage of the fields of vision being greater than 70%, but having an image capture angle different from that of the optical device of the vehicle. The ancillary optical devices have offset positions (spatially and advantageously angularly) with respect to the optical device C1. The target neural network and the annexed neural networks are then trained to detect the correspondence or non-correspondence between views, that is to say whether the views taken correspond to the same scene or to different scenes. During step A, the images can be supplied N by N to the neural networks RNe ₁ to RNe _n or two by two (one image to the RNe ₁ network and one image to one of the RNe ₂ to RNe _N networks, the other networks being deactivated).

Advantageously, during the distance calculation step C), a Euclidean distance is calculated between the two descriptor codes CD _{1, i} and CD _{n, i}

The distance between the two codes CD _{1, i} and CD _{n, i} can be calculated by a distance neuron, which is connected to two binary output neurons, corresponding respectively to the two alternatives of the same scene (correspondence between views) and of scenes different (mismatch between views).

The configuration method advantageously comprises an operation of generating a training (or training) database comprising a step of capturing images by said N optical devices comprising the reference optical device, corresponding to the optical device C1 , and the N-1 ancillary optical devices Cn, with n ranging from 2 to N, mounted on the image acquisition vehicle.

The generation of the training database advantageously comprises the steps of:
- generation of positive training samples, each positive training sample comprising an image captured by the reference optical device and at least one image captured by one of the auxiliary optical devices C2 to CN, said images corresponding to the same scene;
- creation, for each positive training sample, of an associated label indicating that the images of said training sample correspond to the same scene,
said steps of generating positive training samples and creating associated labels being implemented using time stamp data of the images.

In a particular embodiment, the generation of the training database also comprises the steps of
- generation of negative training samples, each negative training sample comprising at least one image captured by the reference optical device and one image captured by one of the auxiliary optical devices C2 to CN, said images corresponding to scenes different;
- creation, for each negative training sample, of an associated label indicating that the images of said training sample correspond to different scenes,
said steps of generating negative training samples and creating associated labels being implemented using time stamp data of the images.

Advantageously, the imaging device of the motor vehicle comprises a decoder for performing a specific decoding task, and another training phase is provided during which said decoder is trained, in a supervised manner, on descriptor codes supplied by the device. previously trained target RNe1 encoder neural network.

The decoder may include a neural network to perform said specific decoding task.

The optical device C1 can be one of the types comprising a camera, a lidar and a radar, and the auxiliary optical devices C2 to CN can be of the same type as the optical device C1.

Another aspect of the invention relates to a system for configuring an imaging device of a motor vehicle comprising an optical device C1 for capturing images and an _{associated encoder neural network RNe 1} , comprising a drive system of the encoder neural network RNe ₁ by the implementation of an iterative process which comprises, for each iteration of index i:
A) the supply of N images IM _{n, i} , n varying from 1 to N, respectively to N encoder neural networks RNe _n , including the neural network RNe ₁ associated with the optical device C1, as target network, and N -1 networks of annexed neurons RNe ₂ ,…, RNe _N , the N images having been captured beforehand by N optical devices Cn for capturing images mounted on an image acquisition vehicle, including a reference optical device, corresponding to the optical device C1, having a reference field of vision, and N-1 ancillary optical devices Cn, with n ranging from 2 to N, mounted on the acquisition vehicle offset from the reference optical device so as to have respective fields of vision exhibiting an overlap rate with the reference field of vision greater than 70%;
B) the encoding of the image IM _{n, i} supplied in a descriptor code CD _{n, i} , by each encoder neural network RNe _n ,
C) for each of the N-1 pairs of images including image IM _{1, i} and image IM _{n, i} with n ranging from 2 to N,
- the calculation of the distance between the two corresponding descriptor codes CD _{1, i} and CD _{n, i} ,
- the prediction of a class, representative of correspondence or non-correspondence information of the images IM _{1, i} and IM _{n, i} of the pair, by binary classification of said distance, and
the calculation of an error between the predicted class and a real class given by a target label associated with said pair of images IM _{1, i} and IM _{n, i} and previously known,
D) the adjustment of the connection weights between neurons of the encoder neural networks RNe ₁ and RNe _n , as a function of said error.

The imaging device of the motor vehicle comprising a decoder for performing a specific decoding task, said system advantageously comprises another decoder drive system configured to drive said decoder, in a supervised manner, on descriptor codes supplied by the network of previously trained _{RNe 1} encoder neurons.

BRIEF DESCRIPTION OF THE FIGURES

Other characteristics and advantages of the present invention will emerge more clearly on reading the detailed description which will follow and which presents various embodiments of the invention given by way of non-limiting examples and illustrated by the appended figures in which :

represents an imaging device of a motor vehicle;

schematically represents a first phase of training a target encoder neural network of the imaging device of FIG. 1, using annex encoder neural networks;

schematically represents a second phase of training of a so-called “decoding” neural network intended for carrying out a specific decoding task;

represents a flowchart of the first training phase of FIG. 2; and

represents a flowchart of an initial step in generating a training database.

DETAILED DESCRIPTION

The present invention relates to a method and a system for configuring an imaging device 1 of a motor vehicle V.

The imaging device 1, shown in Figure 1, is on board the motor vehicle V after configuration. It comprises an optical image capture device C1 and an image processing module 20.

The optical device C1 can be an optical camera, for example a monocular optical camera. It comprises an image sensor 11 and an optical system 12. The image sensor 11 is for example a two-dimensional image sensor of the CCD type (standing for “Charge-Coupled Device” or “charge transfer device”. ”) Or CMOS (standing for“ Complementary Metal-Oxide Semiconductor ”). The optical system 12 is adapted to form an image of at least a portion of the external environment of the vehicle on the image sensor 11.

The processing module 20 comprises a computer 21, or a microprocessor, and a storage module 22 in which is implemented a software module for processing in real time the images captured by the optical image capture device C1. The processing module 20 performs visual recognition and performs at least one target task, for example identifying pedestrians. It could perform other target tasks related to visual recognition, for example vehicle identification, bicycle identification, identification of traffic signs, etc.

Functionally, the processing module 20 comprises a neural network, called an “encoder” and denoted “RNe₁ », Associated with the optical device C1. In operation, the RNe neural network₁receives as input images captured by optical device C1 and outputs corresponding codes, also called descriptor codes, representative of the environment of vehicle V or of at least part of this environment as seen by optical device C1 . The RNe neural network₁is here a convolutional neural network.

The processing module 20 also includes at least one DCD decoder specific to the target task (here a pedestrian identification task). In the exemplary embodiment described here, the DCD decoder comprises a decoding neural network, denoted "RNd".

The processing module 20 can comprise several decoders specific to several target tasks, for example identifying pedestrians, identifying vehicles, identifying bicycles, identifying traffic signs, etc. Each of these specific decoders is based on a network of decoding neurons and uses the codes supplied by the encoder neural network RNe ₁ associated with the optical device C1.

The method of configuring the imaging device 1, according to a particular embodiment, will now be described with reference to Figures 2 to 4.

Referring to Figure 5, the method of configuring the imaging device 1 comprises a prior operation E0 of generating a training database, also called a training database.

The generation of the learning or training database E0 comprises an initial step E01 of capturing images, followed by a step E02 of creating learning samples.

The image capture is carried out by N optical image capture devices (N being greater than or equal to 2), including a reference optical device, denoted C1 ′, corresponding to the optical device C1 (that is to say either the optical device C1, or a similar optical device) and N-1 auxiliary optical devices for capturing images C2,…, CN, all mounted on the same motor vehicle, during one or more driving sessions. The vehicle used to acquire images in order to constitute the training database is not necessarily the final vehicle V. It may be a specific vehicle for the acquisition of images. During each driving session, a large number of images are captured by the reference optical device and ancillary optical devices C2 to CN and stored with time stamp data indicating the times at which the images were captured. Typically, several tens of thousands of images (or views), advantageously coming from several driving sessions, are captured and stored.

During this initial image capture step E01, the optical reference device C1 ’is mounted on the vehicle in a reference position. This reference position advantageously corresponds to the position of the optical device C1 on the vehicle V. The N-1 ancillary optical devices C2,…, CN are mounted offset from the reference optical device. The offset of each of the ancillary optical devices with respect to the reference optical device is such that each of the respective N-1 fields of view of the N-1 ancillary optical devices C2 to CN has an overlap rate greater than 70% with the field of view. reference vision of the reference optical device. The N-1 ancillary optical devices C2 to CN can be spatially offset from the reference position of the reference optical device and have image capture angles different from that of the reference optical device, so that the fields of vision overlap widely (with a recovery rate greater than 70% between the reference field of vision and that of each of the ancillary devices C2 and CN). The respective offsets of the N-1 auxiliary optical devices with respect to the reference optical device are advantageously different from each other. The N optical devices are advantageously of the same type, here of the camera type. As a variant, the N optical devices can be of the lidar type or of the radar type. The images taken by the N optical devices do not necessarily have the same format and / or the same resolution. More generally, the images captured by the N optical devices can be decorrelated by varying one or more image and / or image capture characteristics (for example the brightness and / or the format), in order to to promote the production of high-level abstraction codes.

Step E01 of capturing and storing images is followed by step E02 of generating training samples, comprising positive samples and negative samples.

Each training sample comprises at least two images, including one image captured by the reference optical device C1 ′ and at least one other image captured by one of the N-1 ancillary optical devices Cn with n ranging from 2 to N .

Alternatively, each training sample could comprise N images, respectively captured by the N optical devices C1 ’and C2 to CN.

The training samples generated include positive samples, obtained by selecting images or views from the same scene. These images are selected on the basis of the time stamp data associated with the images, two different images being considered to correspond to the same scene if they were captured at the same instant or in a reduced time interval, preferably less than 50 milliseconds.

The samples generated may also include negative samples, obtained by selecting images or views from different scenes, i.e. captured at different times or not belonging to the predefined reduced time interval, on the base of the timestamp data associated with the images.

For each training sample, a label, also called a "label", classifying this sample is created and added or associated with the sample, during a step E03. This label classifies the sample (positive or negative). It corresponds to correspondence or non-correspondence information of the images of the sample. This match or non-match information is also called the "sample class". This is for example binary information having a value equal to 1 or 0. By convention, in the exemplary embodiment described here, the class “1” of a sample indicates that the images of the sample are views of the same scene, and the class "0" of a sample indicates the sample images are views of different scenes. Class 1 is therefore assigned to positive samples, and class 0 is assigned to negative samples.

The training samples and their associated labels are stored in the training database, during a step E04.

The training database generated therefore comprises positive training samples, each associated with a label indicating the real class of the sample obtained on the basis of the time-stamping data of the images (here class 1), and the samples negative learning, each associated with a label indicating the real class of the sample obtained on the basis of the timestamp data of the images (here class 0),

With reference to FIG. 4, the configuration method comprises a first phase of training Ph1 of the encoder neural network RNe ₁ associated with the optical device C1. The Ph1 training phase is an iterative learning process. We denote "i" an iteration index, initially equal to 1. At each iteration of index i, steps A, B, C and D, described below, are executed by a first drive system having access to the training database.

With reference to FIG. 2, the first training phase Ph1 uses N encoder neural networks RNe _n , including the neural network RNe ₁ associated with the optical device C1, as the target network, and N-1 networks of annexed neurons RNe ₂ ,…, RNe _N respectively associated with N-2 auxiliary optical devices C2 to CN, with respective positions offset with respect to the reference optical device C1 ′. The N neural networks RNe _n with n ranging from 1 to N, used during the training phase Ph1 to train the _{target encoder neural network RNe 1} , are dedicated to the processing of the images coming from the optical devices C1 to CN respectively.

In a particular exemplary embodiment, the networks RNe ₁ to RNe _N each have between five and ten processing levels. For example, the neural architecture of each of the networks RNe ₁ to RNe _N is based on the “AVEnet” network described in the document “Arandjelovic & Zisserman (2018) Objects that sound. European Conference on Computer Vision (ECCV) ”. According to the architecture presented in this document, each level of processing is itself composed of seven operations: convolution, normalization by batch or "batch", linear rectification, convolution, normalization by batch or "batch", linear rectification, and " pooling ”or“ pooling ”(downsampling by“ Max ”operation). In each network, the successive levels include an equal or increasing number of channels, starting from 64 channels for the first level and arriving at 512 channels for the last level. In these successive levels, each channel implements a 3x3 constant receptor field convolution kernel, with less than 2 units encroachment across the levels. The “pooling” operator uses a constant radius of 2x2 units for all levels except the last, whose radius must be adapted to the number of processing levels to obtain channels of dimension 1x1. The top of each convolutional network consists of two fully connected layers with ReLu activation functions.

During step A, the training database supplies at least N images IM _{n, i} , n varying from 1 to N, respectively to the N neural networks RNe ₁ to RNe _N. These images IM _{1, i} to IM _{N, i} come from the training samples previously captured by the N optical devices C1 ′, C2 to CN and then stored. Depending on whether the images come from positive or negative training samples, each of the images IM _{2 , i} to IM _{N, i} coming from the auxiliary optical devices C2 to CN is either in correspondence with the image IM _{1, i} coming from the device reference optics C1 '(ie each of the images IM _{2 images, i} to IM _{N, i} and the image IM _{1, i} correspond to two views of the same scene), or in non-correspondence with the image IM _{1 , i} (ie each of the image images IM _{2, i} to IM _{N, i} and the image IM _{1, i} correspond to two different scenes).

The N images IM _{1, i} to IM _{N, i} can be supplied N by N to the neural networks RNe ₁ to RNe _N. In other words, at each iteration i, the N images are supplied in parallel with the N neural networks RNe ₁ to RNe _N. As a variant, the N images can be provided two by two, that is to say to the network RNe ₁ and to one of the networks RNe ₂ to RNe _N , by deactivating the unused networks, so as to limit the interference during learning.

The next step B is a coding step during which each neural network RNe _n , with n ranging from 1 to N, calculates a code CD _{n, i} from the image IM _{n, i} received as input, and provides this CD code _{n, i} as output. The code CD _{n, i} is a descriptor code which corresponds to a vector type representation of the scene captured by the image IM _{n , i} .

Then, during step C, for each of the N-1 pairs of images including the image IM _{1, i} and one of the images IM _{2 , i} to IM _{N, i} , the first training system
- calculates the distance between the two corresponding codes CD _{1, i} and CD _{n, i} (n ranging from 2 to N), during a sub-step C-1,
- then, by binary classification of this calculated distance, predicts the class of this pair of images IM _{1, i} and IM _{n, i} (with n ranging from 2 to N) representative of a correspondence or non-correspondence information of the two images of the pair of images, during a sub-step C-2.

The distance between the two codes CD _{1, i} and CD _{n, i} is for example the Euclidean distance between two codes of vector type. It is advantageously calculated by a distance neuron.

The binary classification consists in assigning 0 to the pair of images (IM _{1, i;} IM _{n, i} ) either class 1 or class 0, as a predicted class, according to the calculated distance. For example, if the calculated distance is zero, the assigned predicted class is 1, and if the calculated distance is non-zero, the assigned predicted class is 0. To perform the classification, each distance neuron is connected to two neurons output, with “softmax” activation function, the two output neurons corresponding respectively to class 1 and class 0 (in other words to the two alternatives of the same scene and of two different scenes). The network thus learns to categorize the pair of images as identical / different, on the sole basis of the distance calculated between the two codes. These connections form a biased perceptron network, the weights of which are learned during training.

During a sub-step C-3, the first training system calculates a prediction error between the predicted class and the real class given by the label associated with the training sample from which the images IM _{1 originate, i} and IM _{n, i} .

The drive system checks the prediction error during a sub-step C-4. If the prediction error is significant, the method goes to step D. If the prediction error is insignificant, the method interrupts the loop (for the neural network concerned RNe _n ) to then go to the second phase training Ph2, described later.

The following step D is a step of adjusting or updating the connection weights between neurons of the encoder neural networks RNe ₁ and RNe _n , as a function of the prediction error calculated during step C. A such an operation of adjusting the connection weights of the neural networks is well known to those skilled in the art. The connection weights are adjusted so as to reduce the prediction error made by the neural network in its current state. For this, a gradient descent algorithm can be used. For example, we use the ADAM supervised gradient descent algorithm [ Kingma & Ba (2015) Adam: A Method for Stochastic Optimization. Proceedings of the International Conference for Learning Representations, San Diego (ICLR) ] with a weight decay parameter of 10 ^-5 and a learning rate parameter to be determined by grid search. A cost function, or error function, is used. For example, we use a function of the type "binary cross-entropy", or cross-entropy, with logits, defined by the following equation:

or
- y _i represents the class predicted during step C;
- t _i represents the real class given by the label associated with the training sample used.

The first training phase Ph1 (comprising steps A, B, C and D repeated iteratively) is implemented until the error functions no longer indicate a significant error, that is - that is, when until the prediction errors (substep C-3) satisfy a stop criterion, for all neural networks. In practice, we try to reduce the error as much as possible. A classic stopping criterion can be the following: no change at x decimal places of the error function over the last n evaluations, with for example x = 4 decimal places and n = 10 iterations. The weights of the connections between neurons of these encoder networks RNe ₁ to RNe _N are then fixed.

The first training phase Ph1 is followed by a second phase Ph2 for training the decoding neural network RNd of the DCD decoder. During this second training phase, Ph2, the decoder neural network RNd is trained, in a supervised manner, on codes supplied by the encoder neural network RNe ₁ previously trained during the first training phase Ph1 and whose connections between neurons are now frozen. FIG. 3 represents the architecture used during the second training phase. The representations obtained in the codes at the output of the encoder neural network RNe ₁ are used to train in a supervised manner the specific decoder neural network RNd according to the target task to be performed (for example pedestrian identification). The second training phase Ph2 is implemented by a second training system.

The neural network RNd is trained, in a known manner, for example by a supervised gradient descent algorithm such as ADAM. In an exemplary embodiment, the neural network RNd is a neural network of the multilayer perceptron type, provided with ReLu activation functions.

Alternatively, the DCD decoder can also be implemented by a vector support machine.

As previously, the DCD decoder is advantageously trained over a large volume of codes, obtained by presenting to the encoder neural network RNe ₁ a large number of images captured by a reference optical device which is either the optical device C1 or an optical device. similar. The acquisition of these images can be done during a driving session with an image acquisition vehicle equipped with the reference optical device. These images must be labeled by a human expert, depending on the task at hand.

The architecture as illustrated in FIG. 3, after training of the neural networks RNe ₁ and of the DCD decoder, corresponds to the final system on board the vehicle V.

The invention is of interest compared to fully supervised methods, either by reducing the learning base to obtain equivalent performance at a lower cost of labeling, or by keeping the same learning base but obtaining higher performance at the same cost. labeling.

The present invention also relates to a system for configuring the imaging device 1 of a motor vehicle, said imaging device comprising the optical image capture device C1, the _{associated encoder neural network RNe 1} and the specific DCD decoder. to a target task, configured to implement the configuration method described above.

This configuration system comprises a first system for training the encoder neural network RNe ₁ by implementing an iterative process which comprises, for each iteration of index i:
A) the supply of N images IM _{n, i} , n varying from 1 to N, respectively to N encoder neural networks RNe _n , including the neural network RNe ₁ associated with the optical device C1, as target network, and N -1 networks of annexed neurons RNe ₂ ,…, RNe _N , the N images having been previously captured by N optical image capture devices Cn, including a reference optical device corresponding to the optical device C1, having a field of view of reference, and N-1 ancillary optical devices Cn, with n ranging from 2 to N, mounted on the vehicle offset with respect to the reference optical device so as to have respective fields of vision exhibiting a rate of overlap with the field of reference vision greater than 70%;
B) the encoding of the image IM _{n, i} supplied in a descriptor code CD _{n, i} , by each encoder neural network RNe _n ,
C) for each of the N-1 pairs of images including image IM _{1, i} and image IM _{n, i} with n ranging from 2 to N,
- the calculation of the distance between the two corresponding descriptor codes CD _{1, i} and CD _{n, i} ,
- the prediction of a class, representative of correspondence or non-correspondence information of the images IM _{1, i} and IM _{n, i} of the pair, by binary classification of said distance, and
the calculation of an error between the predicted class and a real class given by a target label associated with said pair of images IM _{1, i} and IM _{n, i} and previously known;
D) the adjustment of the connection weights between neurons of the encoder neural networks RNe ₁ and RNe _n , as a function of said error,
as previously described.

The configuration system also includes a second training system configured to train the DCD decoder, in a supervised manner, on codes provided by the previously trained RNe1 encoder neural network.

The training of neural networks is carried out off-line, or "offline".

Although the objects of the present invention have been described with reference to specific examples, various obvious modifications and / or improvements could be made to the described embodiments without departing from the spirit and scope of the invention. .

The present invention also relates to a computer program comprising program code instructions for controlling the execution of the steps of the configuration method described above, when said program is executed on a computer.

Claims (10)

Method of configuring an imaging device (1) of a motor vehicle comprising an optical image capture device C1 and an _{associated encoder neural network RNe 1} , comprising a training phase of the encoder neural network RNe ₁ comprising an iterative process which comprises, for each iteration of index i, the steps of:
A) supply of N images IM _{n, i} , n varying from 1 to N, respectively to N encoder neural networks RNe _n , including the neural network RNe ₁ associated with the optical device C1, as target network, and N- 1 neural networks RNe ₂ ,…, RNe _N , the N images having been captured beforehand by N optical image capture devices Cn mounted on an image acquisition vehicle, including a reference optical device, corresponding to the optical device C1, having a reference field of vision, and N-1 ancillary optical devices Cn, with n ranging from 2 to N, mounted on the acquisition vehicle offset from the reference optical device so as to have respective fields of vision exhibiting an overlap rate with the reference field of vision greater than 70%;
B) encoding of the image IM _{n, i} supplied in a descriptor code CD _{n, i} , by each encoder neural network RNe _n ,
C) for each of the N-1 pairs of images including image IM _{1, i} and image IM _{n, i} with n ranging from 2 to N,
- calculation of the distance between the two corresponding descriptor codes CD _{1, i} and CD _{n, i} ,
- prediction of a class, representative of correspondence or non-correspondence information of the images IM _{1, i} and IM _{n, i} of the pair, by binary classification of said distance, and
calculation of an error between the predicted class and a real class given by a target label associated with said pair of images IM _{1, i} and IM _{n, i} and previously known;
D) adjustment of the connection weights between neurons of the encoder neural networks RNe ₁ and RNe _n , as a function of said error. Method according to Claim 1, characterized in that, during step C), a Euclidean distance is calculated between the two descriptor codes CD _{1, i} and CD _{n, i} . Method according to claim 1 or 2, characterized in that the distance between the two codes CD _{1, i} and CD _{n, i} is calculated by a distance neuron, which is connected to two binary output neurons, corresponding respectively to the two alternatives same scene and different scenes, the network learning to categorize the pair of images as identical / different, on the sole basis of the distance calculated between the two codes. Method according to one of claims 1 to 3, characterized in that it comprises the generation of a training database comprising a step of capturing images by said N optical devices Cn, including the reference optical device , corresponding to the optical device C1, and the N-1 ancillary optical devices Cn, with n ranging from 2 to N, during at least one driving session of the acquisition vehicle. Method according to Claim 4, characterized in that it comprises the steps of:
- generation of positive training samples, each positive training sample comprising at least one image captured by the reference optical device corresponding to the optical device C1 and an image captured by one of the auxiliary optical devices C2 to CN, said images corresponding to the same scene;
- creation, for each positive training sample, of an associated label indicating that the images of said training sample correspond to the same scene,
said steps of generating positive training samples and creating associated labels being implemented using time stamp data of the images. Method according to claim 4 or 5, characterized in that it comprises the steps of
- generation of negative training samples, each negative training sample comprising at least one image captured by the reference optical device corresponding to the optical device C1 and an image captured by one of the auxiliary optical devices C2 to CN, said images corresponding to different scenes;
- creation, for each negative training sample, of an associated label indicating that the images of said training sample correspond to different scenes,
said steps of generating negative training samples and creating associated labels being implemented using time stamp data of the images. Method according to one of claims 1 to 6, characterized in that the imaging device of the motor vehicle comprises a decoder for performing a specific decoding task, and another training phase is provided during which said decoder is driven. , in a supervised manner, on descriptor codes supplied by the target encoder neural network RNe1 previously trained. Method according to Claim 7, characterized in that the decoder comprises a neural network for performing said specific decoding task. Method according to one of claims 1 to 8, characterized in that the optical device C1 is of one of the types comprising a camera, a lidar and a radar, and the auxiliary optical devices C2 to CN are of the same type as the optical device C1. System for configuring an imaging device of a motor vehicle comprising an optical image capture device C1 and an _{associated encoder neural network RNe 1} , comprising a system for training the encoder neural network RNe ₁ by the implementation of an iterative process which includes, for each iteration of index i:
A) the supply of N images IM _{n, i} , n varying from 1 to N, respectively to N encoder neural networks RNe _n , including the neural network RNe ₁ associated with the optical device C1, as target network, and N -1 networks of annexed neurons RNe ₂ ,…, RNe _N , the N images having been previously captured by N optical image capture devices Cn, including a reference optical device corresponding to the optical device C1, having a field of view of reference, and N-1 ancillary optical devices Cn, with n ranging from 2 to N, mounted on the vehicle offset with respect to the reference optical device so as to have respective fields of vision exhibiting a rate of overlap with the field of reference vision greater than 70%;
B) the encoding of the image IM _{n, i} supplied in a descriptor code CD _{n, i} , by each encoder neural network RNe _n ,
C) for each of the N-1 pairs of images including image IM _{1, i} and image IM _{n, i} with n ranging from 2 to N,
- the calculation of the distance between the two corresponding descriptor codes CD _{1, i} and CD _{n, i} ,
- the prediction of a class, representative of correspondence or non-correspondence information of the images IM _{1, i} and IM _{n, i} of the pair, by binary classification of said distance, and
the calculation of an error between the predicted class and a real class given by a target label associated with said pair of images IM _{1, i} and IM _{n, i} and previously known.
D) the adjustment of the connection weights between neurons of the encoder neural networks RNe ₁ and RNe _n , as a function of said error.