Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/084—Backpropagation, e.g. using gradient descent
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/11—Region-based segmentation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/136—Segmentation; Edge detection involving thresholding
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/10—Segmentation; Edge detection
  • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/167—Position within a video image, e.g. region of interest [ROI]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/649—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding the transform being applied to non rectangular image segments
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/21—Server components or server architectures
  • H04N21/218—Source of audio or video content, e.g. local disk arrays
  • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/234345—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment

Definitions

• the present invention relates generally to the field of processing a plurality of data, such as for example 3D images, of the multi-view type of a scene acquired by a plurality of cameras or physiological data of a patient from a plurality of sensors.
• the invention relates in particular to the segmentation of this plurality of data before their coding and then their transmission via a communication network to a processing device.
• the SC scene is conventionally captured by a set of cameras, as illustrated in Figure 1.
• These cameras can be of 2D type (cameras Ci, C 2 ...C N , with N non-zero integer of Figure 1), that is to say that each of them captures a view from a point of view, or 360 type, that is to say they capture the entire scene at 360 degrees around the camera (camera C 360 in Figure 1), therefore from several different points of view.
• the cameras can be arranged in an arc, rectangle, or any other configuration that allows for good coverage of the scene.
• the MIV standard allows the transmission of videos suitable for immersive navigation.
• the encoder chooses portions of each view (patches) that it wishes to transmit in order to maximize the quality of view synthesis from these patches, while reducing the quantity of data to transmit.
• the patches are extracted from the views and gathered into one or more atlases, which are therefore images comprising an assembly of patches from different views.
• the patches are generally arranged in an atlas so as to fill it with in the most complete way possible. With each atlas an occupancy map is transmitted, which is an image of which each pixel can take a first or a second value, distinct from the first (for example corresponding to the color “white”.
• the MIV decoder can find the patches and arrange them in the view to which they belong. This view is then called “partial", since it does not contain all the pixel values of the original view as acquired by one of the cameras. However, if the encoder has efficiently selected the portions of views to transmit, they are sufficient to generate or synthesize any point of view of the scene. In this regard, view synthesis from decoded views is not specified by the MIV standard. It relies on occupancy maps to determine whether a pixel in a given view contains relevant information or not.
• a disadvantage of this method, and more generally of current view synthesis methods, is that they require all of the views acquired by the cameras. This represents a large quantity of data to be transmitted and then decoded, which poses a problem of complexity at the decoder level, particularly when it is embedded in a mobile terminal such as a smartphone. ”) or an augmented reality headset.
• the invention improves the situation.
• the invention responds to this need by proposing a method for segmenting a plurality of data acquired by sensors, called input data, said method comprising:
• weight values to apply to the plurality of input data before processing by at least one processing device configured to produce a processing result according to a criterion for optimizing a quality of the processing result of the input data, said weight values being determined as a function of said criterion and a criterion for optimizing a quantity of input data to be processed,
• segmentation information of said plurality of input data being valued at a first value or at a second value distinct from the first, as a function of said weights
• the subset of data to be processed comprising the data of the plurality of associated input data to segmentation information equal to the first value.
• the invention proposes a completely new and inventive approach to the segmentation of a plurality of input data before their processing by a given processing device, which consists of configuring weight values to be applied to the input data so that the quantity of data resulting from the segmentation and presented as input to the processing device, as well as the processing quality, are both optimized.
• the values of the weights are determined from the plurality of input data themselves, therefore specifically chosen for them.
• the criterion for optimizing a quantity of input data to be processed includes a minimization of a cumulative value of the weight values or of a number of weight values less than a given threshold. According to a variant, it includes a minimization of a cumulative value of the input data kept in the subset of data to be processed or of a number of these input data.
• the criterion for optimizing processing quality includes minimizing a quadratic error between a result obtained from the plurality of input data and a result obtained from the subset of data to be processed. or even maximization of a signal to noise ratio or PSNR (for “Peak Signal to Noise Ratio”, in English).
• the weight values of the configuration obtained are then used to determine one or more segmentation information of the plurality of input data.
• This segmentation information indicates:
• the segmentation information thus determined finally makes it possible to obtain the subset of data to be processed. In this way, only this subset of data processed can be coded and transmitted to a receiver integrating a similar processing device, in terms of structure and configuration.
• the invention does not simply propose to carry out an effective segmentation of the plurality of input data, but it also takes into account, when determining the values of the weights which carry out the segmentation, the impact of the values of the weights of the segmentation module, which carries out the segmentation, on the output of the processing device. It therefore does not consider a simple segmentation module independent of the processing device, but takes into account the combination of the two, and more precisely, their successive actions on the input data.
• the invention applies to any type of data acquired by any type of sensors.
• a plurality of sensors is arranged around a scene, an object or a subject...
• it is a plurality of cameras each having distinct points of view of the scene and configured to acquire a sequence of images or views of this scene.
• the processing device can in this case be a device for synthesizing additional views from the original views acquired by the plurality of cameras and the segmentation of the original views according to the invention makes it possible to retain only the data useful for the synthesis of an additional view and remove redundant data.
• the plurality of data consists of temporal sequences of physiological measurements of a patient captured by a plurality of sensors of various natures (ElectroCardioGramme (ECG), ElectroEncephaloGramme (EEG), scanner, magnetic resonance imaging (MRI), X-ray image, blood composition indicator, etc.).
• ECG ElectroCardioGramme
• EEG ElectroEncephaloGramme
• scanner magnetic resonance imaging
• MRI magnetic resonance imaging
• X-ray image X-ray image
• blood composition indicator etc.
• the processing device can be a diagnostic aid device.
• the determination comprises learning said weight values, from said plurality of input data, said learning being carried out by back-propagation of a gradient of a function of loss combining the two criteria.
• the determination of the weight values carrying out the segmentation of the input data implements a technique for learning an artificial intelligence module using the plurality of input data themselves .
• the artificial intelligence module which we will hereinafter refer to as the segmentation module, learns the best possible internal configuration of its weights, making it possible to optimize the quantity of data of the subset of data d input to be presented as input to the processing device and to optimize the quality of data processing.
• the invention goes against usual practices in terms of machine learning, in that it implements learning specific to a plurality of given input data. Thus it does not require the prior acquisition of a large labeled learning database, but only the plurality of current data to be processed.
• said processing device comprises weights, called processing weights, values of said processing weights have been previously determined as a function of the criterion for optimizing the quality of the processing result of the input data and the method further comprises the determination of modified values of said weights treatment.
• the processing device also implements an artificial intelligence technique, based on the application of weights to the input data.
• this processing device has been previously trained, in a conventional manner, from a labeled database, to process the plurality of input data so as to provide at output a result conforming to the optimization criterion of the quality of treatment.
• the learning according to the invention is a combined learning of the segmentation module and the processing device. It includes an update of the configuration of the processing device so that it can optimally process the subset of data resulting from the segmentation of the plurality of input data.
• the segmentation module and the processing device are each organized in layers comprising weights and the succession of their respective layers forms an artificial intelligence module, for example a neural network, trained to segment the data input and then process them optimally.
• an artificial intelligence module for example a neural network
• said plurality of input data comprises a plurality of views acquired by a plurality of cameras, said view comprises pixels, said weights are included in a plurality of layers, said layer is associated with said view and comprising said weight per pixel, and the segmentation information comprises a plurality of segmentation maps, said map being associated with said view.
• the invention applies particularly to input data of the multi-view video sequence type.
• a segmentation module structured in layers, with one layer per view and a weight per pixel, the weights of each layer making it possible to derive the segmentation information from the pixels of the associated view.
• the segmentation information can take the form of segmentation maps, of the same dimensions as the input images, and whose useful pixels are valued at a first value (for example example, corresponding to “white”) and non-useful pixels to a second value (for example, corresponding to “black”).
• the processing device is configured to synthesize an additional view associated with a given point of view and from at least one original view that it receives as input.
• the learning according to the invention can consist of presenting as input to the segmentation module, combined with the processing device, the plurality of views acquired except one which will be the one to be synthesized.
• the loss function can be calculated by comparing the synthesized view to the original view.
• this procedure will be repeated for each of the views acquired.
• said plurality of input data comprises a plurality of sequences of measurement data acquired by a plurality of sensors, said weights are included in a plurality of layers, said layer is associated with said sequence of measurement data and comprises said weight per measurement data and the segmentation information comprises a plurality of segmentation sequences, said segmentation sequence being associated with said sequence of measurement data.
• the plurality of data consists of temporal sequences of physiological measurements of a patient captured by a plurality of sensors of various types.
• the processing device may include one or more diagnostic aid devices.
• the invention also relates to a device for segmenting a plurality of data acquired by sensors, called input data, said device being configured to implement:
• weight values to be applied to the plurality of input data before processing by at least one processing device configured to produce a processing result according to a criterion for optimizing a quality of the data processing result d input, said weight values being determined as a function of said criterion and a criterion for optimizing a quantity of input data to be processed,
• segmentation information of said plurality of input data being valued at a first value or at a second value distinct from the first, as a function of said weights
• the subset of data to be processed comprising the data of the plurality of associated input data to segmentation information equal to the first value.
• the aforementioned device implements the segmentation method according to the invention in its different embodiments.
• said segmentation device is integrated into server equipment configured to receive the plurality of input data and further comprising the aforementioned device for processing the plurality of input data.
• the invention also relates to a method for coding a plurality of data acquired by sensors, called input data, comprising:
• segmentation information of the plurality of input data and a subset of data to be processed by applying said segmentation information to the plurality of input data said segmentation comprising the determination of values of weight to be applied to the plurality of input data before processing by at least one processing device previously configured to produce a processing result according to an optimization criterion of a quality of the processing, said weight values being determined as a function of said criterion and another criterion for optimizing a quantity of input data to be processed, said segmentation information of said data being valued at a first or at a second value distinct from the first, as a function of said weight values, the subset (USS) of data to be processed comprising the data of the plurality of input data associated with segmentation information equal to the first value, and
• the coded data contains all the information making it possible to reconstruct a plurality of segmented input data at the level of a receiver comprising a processing device similar to that implemented by the transmitter, in terms structure and configuration.
• the obtaining further comprises:
• weights being intended to be applied to the input data by the processing device, values of said weights previously determined according to said optimization criterion of a quality of the processing result of the input data and for processing the plurality of input data, having been modified according to said criteria, for processing the subset of data to be processed,
• the invention proposes to transmit in the coded data the modified weight values of the processing device on the transmitter side, with a view to updating the configuration of the processing device on the receiver side. This ensures that the processing device produces an optimal result within the meaning of the optimization criterion.
• the invention also relates to a device for coding a plurality of data acquired by sensors, called input data, configured to implement:
• segmentation information of the plurality of input data and a subset of data to be processed by applying said segmentation information to the plurality of input data said segmentation comprising the determination of values of weight to be applied to the plurality of input data before processing by at least one processing device (previously configured to produce a processing result according to a criterion for optimizing a quality of the processing, said weight values being determined as a function of said criterion and another criterion for optimizing a quantity of input data to be processed, said segmentation information of said data being valued at a first or a second value distinct from the first, depending on said weight values, the subset of data to be processed comprising the data of the plurality of input data associated with segmentation information equal to the first value, and - the coding of the segmentation information and the subset of data to be processed.
• the aforementioned device implements the coding method according to the invention in its different embodiments.
• said coding device is integrated into the aforementioned server equipment.
• the invention also relates to a method for decoding coded data, comprising:
• said coded data comprising segmentation information of a plurality of data acquired by sensors, called input data, a subset of data to be processed by a processing device configured to apply weights to the plurality of decoded input data and to produce a processing result according to a processing quality optimization criterion, said segmentation information of said input data being valued at a first value or to a second value distinct from the first, said subset of data to be processed having been obtained by applying said segmentation information to the plurality of input data, the subset of data to be processed comprising the data of the plurality of input data associated with said segmentation information equal to the first value, said coded data further comprising modified values of said weights, said modified values having been determined for the processing of the plurality of segmented input data, depending on the criterion for optimizing a quality of processing and a criterion for optimizing a quantity of data of the subset of data to be processed,
• the invention proposes to code the modified weight values of the processing device on the transmitter side, with a view to their transmission to a receiver and the updating of the configuration of the processing device on the receiver side. This ensures that the processing device produces an optimal result within the meaning of the optimization criterion.
• the invention also relates to a decoding device comprising:
• said coded data comprising segmentation information of a plurality of data acquired by sensors, called input data, a subset of data to be processed by a processing device configured to apply weights to the plurality of decoded input data and to produce a processing result according to a processing quality optimization criterion, said segmentation information of said input data being valued at a first value or to a second distinct value of the first, said subset of data to be processed having been obtained by applying said segmentation information to the plurality of input data, the subset of data to be processed comprising the data of the plurality of input data associated with said segmentation information equal to the first value, said coded data further comprising modified values of said weights, said modified values having been determined for the processing of the plurality of segmented input data, as a function of the optimization criterion of a quality of processing and a criterion for optimizing a quantity of data from the subset of data to be processed,
• the aforementioned device implements the decoding method according to the invention in its different embodiments.
• said decoding device is integrated into terminal equipment configured to receive the coded data and further comprising the aforementioned processing device.
• the invention also relates to a signal carrying coded data.
• Said coded data comprises segmentation information of a plurality of data acquired by sensors, called input data, a subset of data to be processed obtained by applying said segmentation information to said plurality of input data, said segmentation information of said input data being valued at a first value or at a second value distinct from the first, the subset of data to be processed comprising the data of the plurality of input data associated with information of segmentation equal to the first value, said subset of data to be processed being intended to be decoded, then used to reconstruct a plurality of decoded segmented input data from the decoded segmentation information, with a view to processing said plurality of input data segmented by a processing device, configured to produce a processing result according to a processing quality optimization criterion, by applying weights to the plurality of said input data.
• Said coded data further comprises modified values of said weights, said modified values having been determined for the processing of the plurality of segmented input data, according to the optimization criterion of a processing quality and a criterion of optimizing a quantity of data from the subset of data to be processed, and being intended to be used by said processing device to update said weights before processing the plurality of reconstructed decoded segmented input data.
• the invention also relates to a system comprising the segmentation device, the coding device, the decoding device and the processing device aforementioned.
• This system, the segmentation, coding and decoding devices according to the invention have at least the same advantages as those conferred by the aforementioned methods.
• the invention finally relates to computer program products comprising program code instructions for the respective implementation of the aforementioned segmentation, coding and decoding methods, when they are executed by a processor.
• Such a program can use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in n any other desirable shape.
• the invention also relates to at least one recording medium readable by a computer on which the computer programs comprising program code instructions for the execution of the steps of the methods according to the invention as described are recorded. above.
• Such a recording medium can be any entity or device capable of storing the program.
• the medium may comprise a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or even a magnetic recording means, for example a mobile medium (memory card) or a hard drive or SSD.
• such a recording medium may be a transmissible medium such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio or by other means, so that the computer programs it contains can be executed remotely.
• the programs according to the invention can in particular be downloaded onto a network, for example the Internet network.
• the recording medium(s) may be one or more integrated circuits in which each program is incorporated, the circuit(s) being adapted to execute or to be used in the execution of the aforementioned methods.
• the present technique is implemented by means of software and/or hardware components.
• module can correspond in this document to a software component as well as to a hardware component or to a set of hardware and software components.
• a software component corresponds to one or more computer programs, one or more subprograms of a program, or more generally to any element of a program or software capable of implementing a function or set of functions, as described below for the module concerned.
• Such a software component is executed by a data processor of a physical entity (terminal, server, gateway, set-top-box, router, etc.) and is capable of accessing the hardware resources of this physical entity (memories, recording media, communication buses, electronic input/output cards, user interfaces, etc.). Subsequently, by resources we mean all sets of hardware and/or software elements supporting a function or service, whether they are unitary or combined.
• a hardware component corresponds to any element of a hardware assembly capable of implementing a function or a set of functions, according to what is described below for the module concerned. It may be a programmable hardware component or one with an integrated processor for executing software, for example an integrated circuit, a smart card, a memory card, an electronic card for executing firmware ( “firmware” in English), etc.
• FIG 1 presents an example of arrangement of a plurality of cameras forming a system for acquiring a multi-view video of a scene, according to the prior art
• FIG. 1 schematically illustrates a plurality of images of the scene, captured by the plurality of cameras at a given time, according to the prior art
• FIG. 3 schematically illustrates a sequence of the plurality of images, captured by the plurality of cameras at several successive times and forming the original multi-view video, according to the prior art
• FIG 4 schematically illustrates a first example of architecture of a system according to one embodiment of the invention comprising a device for segmenting a plurality of input data, a coding device segmentation information and a subset of data to be processed resulting from the segmentation, a device for decoding coded data representative of the subset of data to be processed and associated segmentation information and a processing device decoded data, when the plurality of data is a multi-view video of the scene acquired by a plurality of cameras;
• FIG 5 schematically illustrates a second example of architecture of a system according to one embodiment of the invention comprising a device for segmenting a plurality of input data, a coding device segmentation information and a subset of data to be processed resulting from the segmentation, a device for decoding coded data representative of the subset of data to be processed and associated segmentation information and a processing device decoded data, when the plurality of data includes physiological measurements of a patient acquired by a plurality of sensors;
• FIG 6 describes in the form of a flowchart the steps of a process for segmenting a plurality of input data, according to an exemplary embodiment of the invention
• FIG 7 describes in the form of a flowchart the steps of a method of coding data resulting from the segmentation of the plurality of data coded according to one embodiment of the invention
• FIG 8 describes in the form of a flowchart the steps of a method of decoding data resulting from the segmentation of the plurality of data according to one embodiment of the invention
• FIG 9 details a first example of implementation of the aforementioned segmentation and coding methods when the plurality of data includes a multi-view video
• FIG 10 details a first example of implementation of the aforementioned decoding method when the plurality of input data includes a multi-view video
• FIG 11 details a second example of implementation of the aforementioned segmentation and coding methods when the plurality of input data comprises a plurality of sequences of physiological measurements of a patient;
• FIG 12 details a second example of implementation of the aforementioned decoding method when the plurality of input data comprises a plurality of sequences of physiological measurements of a patient;
• FIG. 13 describes an example of hardware structure of a device for segmenting a plurality of data according to the invention
• FIG 14 describes an example of hardware structure of a device for coding a plurality of data according to the invention.
• FIG 15 describes an example of hardware structure of a device for decoding data resulting from segmentation and coding according to the invention.
• the principle of the invention is based on the segmentation of a plurality of input data, acquired by sensors arranged around a scene, an object or a subject, with a view to their processing by a given processing device. Segmentation here refers to the fact of identifying among the plurality of input data those to be kept for a given subsequent processing and to bring them together into a subset of data to be processed.
• the decision to prune or keep the input data is made on the basis of two criteria, a criterion for optimizing a quantity of segmented data, that is to say kept in the subset of data to be processed, and a criterion for optimizing a quality of processing given from the subset of data to be processed.
• the segmentation is implemented by an artificial intelligence module placed upstream of the processing device and capable of being configured by machine learning (in English, “machine learning”).
• the structure of the automatic segmentation module comprises a plurality of weights intended to be applied to the plurality of input data and learning a configuration of the automatic segmentation module consists of the module learning the best possible internal configuration of its weight, making it possible both to minimize the quantity of data of the subset of input data to be presented as input to the processing device and to maximize the quality of data processing.
• this learning is on the one hand specific to the plurality of input data and on the other hand carried out by considering the result of the combination of the actions of the segmentation module and the processing device placed in downstream of the segmentation module, which therefore recovers as input the plurality of data weighted by the configuration weights of the automatic segmentation module, according to at least two criteria.
• the first performance criterion relating to the segmentation module is to minimize a quantity of data to be provided to the processing device, which amounts to pruning the plurality of input data as much as possible.
• the second performance criterion is linked to the processing device and requires maximizing a measure of quality of the processing carried out by the processing device.
• the configuration values of the automatic segmentation module are then used to determine segmentation information of the plurality of data and a subset of data to be processed is finally obtained by applying the segmentation information to the plurality of data d 'entrance.
• the invention thus makes it possible to reduce the quantity of data to be encoded and transmitted to remote receiver equipment carrying a processing device similar to those implemented by transmitter equipment, while preserving the quality of the processing of this data. .
• the invention applies to any type of input data and is independent of the artificial intelligence technique implemented by the automatic segmentation module, provided that it can be configured using a plurality of configuration weights associated with the plurality of input data.
• the invention finds a particular application in a free navigation system within a multi-view video, for example embedded in terminal equipment, for example of the mobile telephone or virtual reality headset type.
• the processing includes the synthesis of an additional view, desired by the user of the terminal equipment, from the segmented data.
• the invention is not limited to this use case, but can be applied to any other plurality of data acquired by sensors, such as for example sensors for physiological measurements of a patient.
• scene in the broad sense refers to any object, subject or plurality of objects or subjects in their environment.
• server equipment ES configured to receive input data acquired by a plurality of Cl, C2, ...CN sensors.
• the input data received are a plurality of video sequences taken from different points of view of a scene SC, also called multi-view video VMV.
• the server equipment ES comprises an E/R transmission-reception module, a segmentation device 100 according to the invention, a PROCI, SYNT1 processing device for the plurality of input data and a device 200 for coding segmented data according to the invention.
• Such server equipment ES is configured to transmit the coded data, for example in the form of a STR data stream or F D file, using its E/R module. It also includes a memory M ES in which it stores, for example, the coded data.
• the segmentation device 100 is configured to determine weight values to be applied to the plurality of input data before processing by at least one processing device (PROCI, SYNT1) configured to produce a result processing according to a criterion for maximizing a quality measure of the result of processing the input data, said weight values being determined as a function of said criterion and another criterion for minimizing a quantity of input data to process, determine segmentation information of said plurality of input data, said segmentation information of said data being valued at a first value or at a second value distinct from the first, depending on said weights, and obtain a sub -set of data to be processed by application of the determined segmentation information to said plurality of input data, the subset of data to be processed comprising the data of the plurality of input data associated with segmentation information equal to the first value.
• PROCI processing device
• the system S comprises an artificial intelligence module or segmentation module SEG1 comprising said weights and located upstream of the processing device PROCI, SYNT1.
• the determination comprises learning said weight values, from said plurality of input data, by back-propagation of a gradient of a loss function combining the two criteria.
• the module SEG1 is integrated into the device 100.
• the device 100 thus implements the method of segmenting a plurality of data representative of a scene according to the invention which will be detailed below in relation to Figure 6.
• the coding device 200 is configured to obtain segmentation information from the plurality of data and the subset of data to be processed produced by the segmentation device 100 and to code the subset of data to be processed and the segmentation information obtained.
• the device 100 is independent of device 200, but according to a variant (not shown), it is integrated into device 200.
• the device 200 thus implements the method of coding data representative of a scene according to the invention which will be detailed below in relation to Figure 7.
• the processing device PROCI, SYNT1 is configured to synthesize a view chosen by a user from the decoded input views, when this does not correspond to any of the views of the multi-view video which were transmitted in the STR, F D encoded data stream or file. It is understood that according to the invention, the processing device PROCI, SYNT1 implemented in the server equipment ES is only used for training purposes of the automatic segmentation module SEG1.
• the system S also includes terminal equipment UE, for example a smartphone or an augmented reality headset, for example of the head device or HMD (Head Mounted Device) type. carried by a user UT, remote from the server equipment ES and for example connected to it via a communications network RC.
• terminal equipment UE for example a smartphone or an augmented reality headset, for example of the head device or HMD (Head Mounted Device) type.
• HMD Head Mounted Device
• the terminal equipment UE comprises an E/R transmission/reception module, capable of receiving the stream or file of coded data STR, F D via the communication network RC, a device 300 for decoding the coded data received, a PROC2, SYNT2 processing device for segmented data, similar to that of the equipment ES, in terms of structure and configuration, and a memory M UE-
• the device 300 is configured to decode the coded data received and make them available to the processing device PROC2, SYNT2.
• the coded data comprises the segmentation information of the plurality of input data (here the multi-view video VMV) and the subset of data - to be processed, resulting from the application of the segmentation information determined at said plurality of input data, produced by the device 100 of the equipment ES, as previously described.
• the device 300 thus implements the decoding method according to the invention which will be detailed below in relation to Figure 8.
• the processing device PROC2, SYNT2 of the terminal equipment UE is therefore configured to synthesize the view chosen by a user from the other decoded views of the multi-view video, when this does not correspond to any of the views of the multi-view video transmitted in the stream or the encoded data file STR, F D . It is similar to the PROCI, SYNT1 device of the ES server equipment, in that it has the same structure and the same configuration.
• the PROCI, PROC2 processing devices also include an artificial intelligence module comprising a weight structure, the values of which are configured using a machine learning technique.
• the prior configuration of the two monitoring devices PROCI, PROC2 processing (ES and UE side) is automatically learned from the same input database, for example labeled and under the constraint of a criterion of maximizing a quality measure of the processing result.
• the memory M UE is for example used to store the additional view synthesized by the processing device PROC2, SYNT2.
• a system S' comprising server equipment ES' configured to receive input data acquired by a plurality of sensors SI, S2, ...SN, of various natures, for example ECG, EEG, scanner, MRI, blood composition analysis, radio, etc.
• a plurality of physiological sensors placed on or near a patient SB and the input data received are a plurality of sequences of physiological measurements acquired at different successive times by the plurality of sensors.
• the server equipment ES' has a structure similar to that of the server equipment ES previously described in relation to Figure 4.
• It comprises a device 100' for segmenting the plurality of physiological measurements according to the invention, a segmentation module SEG1' of the input data, configured by the device 100' to segment the input data according to the invention, a processing device PROCI,' DIAG1 of the data resulting from the segmentation by the module SEG1', a device 200' for coding segmented data according to the invention, an E/R transmission/reception module and an M ES- memory.
• the devices 100' and 200' are similar to the aforementioned devices 100 and 200.
• the device 100' thus implements the input data segmentation method (here, physiological measurements of the patient SB) according to the invention which will be detailed below in relation to Figure 6.
• the device 200' thus puts implement the input data coding method according to the invention which will be detailed below in relation to Figure 7.
• the processing device PROCI', DIAG1 is configured to process the physiological measurements provided by the sensors to carry out diagnostic assistance processing.
• this processing device PROCI', DIAG1 is only used here to configure the segmentation module SEG1' of the device 100' according to the invention.
• the system S' also includes terminal equipment UE', remote from the server equipment ES' and connected to it via the communication network RC.
• the equipment UE' has a structure similar to that of the terminal equipment UE previously described in relation to Figure 4. It comprises at least one decoding device 300' similar to the aforementioned decoding device 300 in relation to Figure 4, a processing device PROC2', DIAG2 previously configured to process the plurality of data from the sensors once decoded by the decoding device 300', an E/R transmission and reception module and a memory M UE-.
• the device 300' is similar to the aforementioned device 300 and implements the method of decoding data representative of a scene according to the invention which will be detailed below in relation with Figure 8.
• it further comprises a DISP display device, for example of the screen type, which is used by a DOC user to view the physiological measurements received and decoded by 300'.
• SEG1' is an artificial intelligence module capable of learning, from input data SQl(t), SQ2(t), ..., SQN(t), those which are useful to a PROCI processing device (resp. PROC1') located downstream (the output of the SEG1 module (resp. SEG1') is connected to the input of the PROCI processing device (resp. PROC1')) and previously configured to apply predetermined processing to the plurality of input data, and those which are not.
• the segmentation module SEG1 (resp. SEG1') is structured so as to apply weights to the plurality of input data, the value of which is determined during this learning.
• SEG1' includes as many weights as input data. These weights have for example floating values between 0 and 1 and their application (multiplication) to the value of an input data has the effect of keeping this data “on” or “off” before it is taken into account. accounted by the processing device PROCI (resp. PROC1').
• these weights are distributed in successive layers associated with each of the N input data sequences at each of the Nt instants of each sequence .
• the module SEG1 comprises N x Nt layers, with one layer per image or view and per time instant t, whose dimensions correspond respectively to those of the images of the N sequences .
• learning is done on the basis of a first performance criterion of the segmentation module SEG1 (resp. SEG1'), which is here a criterion for optimizing a quantity of data to be present to the processing device PROC1 (resp. PROC1'), but also on the basis of a second performance criterion, specific to the processing device PROCI (resp. PROC1') itself, which is an optimization criterion of 'quality of treatment.
• This taking into account the combination of the two successive treatments and the satisfaction of these two performance criteria make it possible to guarantee that the pruning of input data by the segmentation module SEG1 (resp. SEG1') is not done to the detriment of the performance of the processing device PROCI (resp. PROC1').
• the first criterion includes a minimization of an amplitude of the cumulative values of the weights of the layers of the segmentation module SEG1, SEG1'.
• it comprises a minimization of a number of weight values greater than a given threshold in said layers or a maximization of a number of weight values lower than said threshold.
• it includes a minimization of the entropy of the input data at the output of the segmentation module SEG1, SEG1', that is to say once the weight values have been applied to them. of said layers.
• it includes a minimization of an amplitude of the cumulative values of the input data kept in the subset of data to be processed.
• the first criterion includes a combination of several of these different criteria.
• segmentation information SGI of the input data is determined from the weight values determined for the module SEG1 (resp. SEG1').
• the corresponding SGI segmentation information is positioned at a first or a second value distinct from the first, depending on the value of the corresponding weight. For example, the first value is greater than the second value.
• the weight value is compared to a given threshold. When the weight value is greater than the threshold, the segmentation information is valued at the first value (which means “useful” therefore “on”), and at the second value (which means “useless” therefore “off”) otherwise .
• the segmentation information associated with this image includes a segmentation map of the same dimension WxH, therefore the pixels are valued at a gray level corresponding to “white” when they correspond to data decided to be “useful” and to a gray level corresponding to “black”, otherwise.
• the first value is equal to 1 and the second is zero.
• the SGI segmentation information is stored in a memory Ml.
• a subset of data to be processed USS is obtained by applying segmentation information to the plurality of input data SQ1, SQ2, ..., SQN and retaining only those which are associated with segmentation information valued at the first value.
• the subset of data to be processed USS is stored in the memory Ml and can be provided to an encoder.
• the PROCI processing device also comprises an artificial intelligence module having a layer structure comprising processing weights.
• an artificial intelligence module having a layer structure comprising processing weights.
• the processing weights of the processing device PROCI are fixed, that is to say their values are not modified by the determination 61 .
• determination 61 also applies to the processing device PROCI (resp. PROC1') and has the effect of modifying the values of its processing weights.
• the modified processing weight values MW are obtained at 64. They are supplied to an encoder or stored in the memory Ml.
• the device 200, 200' obtains N temporal sequences of input data SQl(t), SQ2(t), ..., SQN(t), with N non-zero integer, acquired by N sensors placed around the object, subject or scene of interest. At 71, it obtains a segmentation of this data carried out by the segmentation method according to the invention which has just been described in relation to Figure 6. It is assumed that the device 200, 200' obtains at least the segmentation information SGI and the subset of data to be processed USS provided by the segmentation method according to the invention. Advantageously, it also obtains modified weight values MW intended for the processing device PROCI, PROC1'.
• the device 100, 100' may or may not be integrated into the encoding device 200, 200'.
• the device 200, 200' directly implements the segmentation method according to the invention.
• it commands the segmentation of the input data to the device 100, 100' and receives the segmented data in response.
• the device 200, 200' encodes the data obtained SGI, USS in a conventional manner, known per se.
• the coded data is stored in memory M2 in a data file F D or transmitted in a data stream STR, to remote equipment, for example the terminal equipment UE, UE' of Figures 4 and 5.
• the coded data is obtained, for example from a memory M3 or else they are received via a communication network, such as for example the RC network of Figures 4 and 5.
• the coded data comprises at least the segmentation information SGI and the subset of data to be processed USS, the segmentation information SGI making it possible to identify the data to be processed of the subset USS in the N sequences of input data expected by the decoder.
• the coded data is decoded in a manner known per se.
• a subset of USSD decoded data to be processed and SGID decoded segmentation information are obtained.
• MWD decoded modified processing weight values intended for the processing device PROC2 are also obtained.
• the decoded segmentation information SGID and the subset of USSD decoded data to be processed are used to construct segmented decoded data sequences SQlDS(t), SQ2DS (t), ..., SQNDS(t) to be presented to the processing device PROC2, PROC2'. If modified MWD decoded weight values have been obtained, they are previously applied to a processing device PROC2, PROC2' of structure and initial configuration similar to the processing device PROCI, PROC1' on the encoder side. Once the configuration has been carried out, the segmented decoded data sequences SQlDS(t), SQ2DS(t), ..., SQNDS(t) are provided to the device PROC2, PROC2' for processing.
• the processing to be applied to the input data corresponds here to a view synthesis.
• it involves synthesizing an additional view associated with an additional point of view PVS distinct from the N points of view of the N cameras.
• the synthesis device in question is configured to take as input all or part of the N images Tl(t), T2(t), ..., TN(t) and their intrinsic parameters (focal length, sensor resolution, number of components colors, etc.) and extrinsic (position in space, orientation, etc.) of the N cameras, as well as the coordinates of the additional point of view PVS of the scene, and to produce as output the image or additional view TS .
• the synthesis device SYNT1, SYNT2 comprises a neural network of the IBRNet type.
• Such a network has a structure of M successive layers L'1, L'2, ..., L'M, with M non-zero integer, connected together so that the output of an upstream layer is presented as an input to the next downstream layer.
• Each layer has configuration weights WjL'i, with i integer between 1 and M and j integer between 1 and a number of processing weights associated with layer i, the values of which are determined during prior learning .
• the module automatic segmentation SEG1 also has the structure of a layered neural network, also called relevance layers, because the values of its weights are intended to ultimately indicate a level of relevance or usefulness of 'input data for subsequent processing by the SYNT1 synthesis device.
• the automatic segmentation module comprises as many layers LI, L2, ..., LN as there are input images Tl(t), T2(t), etc. ., TN(t) at a given time t.
• Tl input images
• T2(t) input images
• TN TN
• each layer Li has the same dimensions as the corresponding input image or view.
• the layer Li is also of size WxH, that is to say it includes WxH weights, each weight being a unique multiplier coefficient of the scalar value ( gray level) or vector (RGB or YUV triplet) associated with the corresponding pixel of the input image.
• the multiplier coefficient is a floating real.
• the weights of each Li layer take binary values. If the weight bit has the value 1, the associated pixel is transmitted without change to the input of the processing device SYNT1, otherwise the associated pixel is not transmitted to the processing device SYNT1 or it is transmitted with a predefined value (e.g. -1) to indicate that it is not useful.
• a predefined value e.g. -1
• the neural network of the segmentation module SEG1 is then subjected to learning, so as to assign adequate values to the weights of these different layers so that it executes the expected task (here, the synthesis of an additional view ) In an acceptable way.
• the neural network CRN1 combining the N layers LI, L2, ..., LN of the segmentation module SEG1 and the M layers L'1, L'2, ..., L'M of the processing device PROC1 which is subject to this learning.
• the first criterion concerns a loss linked to the task of the neural network of the synthesis device SYNT1, and measures a quality of the synthesized image
• the second concerns a loss linked to the segmentation of the input data by the neural network of the module SEG1 and measures the cumulative amplitude of the values of the weights of its layers Ll, ..., LN.
• the learning is similar to that described in the document by Wang et al. already cited. Briefly, this learning works as follows: for each value j in 1, ..., N, the combined network CRN1 is asked to generate the view Tj from the other views. Since we have the original view Tj associated with the point of view PVj, it is possible to evaluate the quality of the synthesis carried out by the device PROC1, for example by calculating a difference (for example, based on the mean square error) between the synthesized view TSj and the original view Tj . This difference is then used to calculate a gradient which is back-propagated to layer Lj of module SEG1 in a manner known per se. Note that when the synthesis device is not based on an artificial intelligence method, the calculated gradient is simply propagated in layer Lj.
• the global loss function to be minimized is the following (where
• A is a parameter set by the user which makes it possible to determine a choice between minimizing the size of the segmented areas (A. high) or the view synthesis performance (A. low). It is a positive real, which can be larger or smaller than 1.
• PLj designates a vector concatenation of the WxH weights of the N Li layers.
• the weights of the neural network of the synthesis device SYNT1 are fixed and are not modified by the learning of the module SEG1. In other words, only the weights of the layers LI, L2, ..., LN of the module SEG1 are optimized in the sense of the loss function.
• the weights of the layers of the network of the processing device SYNT1 may vary.
• they are optimized during the learning of the SEG1 module, which becomes a learning of the combination of the SEG1 module and the SYNT1 device.
• the modified values MWL'l, MWL'2, ..., MWL'M of the processing weights of the device SYNT1 are obtained and stored with a view to their transmission, for example to the terminal equipment UE of the figure 4.
• the segmentation information SGI of the input data Tl, T2, ..., TN is determined at 62.
• each layer LI, L2, ..., LN of the module SEG1 is transformed into a segmentation map SGI1, SGI2, ..., SGIN of the view Tl, T2, ..., associated TN.
• a segmentation map of a given image or view is a map having the same dimensions as this image, and whose elements or pixels are valued at a first value for example the value 1, meaning that the associated pixel in the image Tj is useful for the subsequent task of the processing device PROC1, the value 0 meaning that the associated pixel in the image Tj is not useful for this processing device.
• the determination of the corresponding value of the segmentation map is done by comparison of each weight at a given threshold, for example equal to 0.01. If the weight value is less than this threshold, then the corresponding pixel of the segmentation map is set to 0, otherwise this pixel is set to 1.
• the weight values are binary, they are directly copied into the corresponding SGIi segmentation map (the SGIi map is equal to the weight map of the Li layer).
• the views Tl, T2, ..., TN are then coded (72) by the coding device 200 using the segmentation cards SGI, SG2, ..., SGN.
• a method of coding a multi-view video is used, preferably the MIV standard.
• we can use other standards such as for example MV-HEVC or 3D HEVC.
• the atlases corresponding to the different moments of the video are then coded like conventional 2D videos, using for example the HEVC video coding standard.
• the IV standard indeed provides a syntax which allows the decoder to identify the patches inside the atlases and to rearrange them in the initial views. The views decoded then reconstructed on the decoder side are therefore partial.
• the division into patches is carried out, for each view Tj, on the basis of the segmentation map SGIj produced from the values assigned to the weights of the automatic segmentation module at the end of the learning (61).
• the atlases are filled with all the patches identified in all the views, the atlases are coded by a conventional encoder, for example conforming to the HEVC standard.
• the configuration of the neural network of the synthesis device SYNT1 is fixed.
• the configuration of the neural network of the synthesis device SYNT1 is modified by learning 61 and the modified values MWL'l, MWL'2,..., MWL'N of the weights of processing of the different layers L'1, L'2, ..., L'M of the neural network of the synthesis device SYNT1 which are also coded (for example with the MPEG NNR coding standard, from English "Neural Net Representation"), specified in part 17 of the MPEG-7 standard, for transmission to the corresponding SYNT2 synthesis device on the receiver side (UE).
• the MPEG NNR coding standard from English "Neural Net Representation”
• UE receiver side
• the coded data is transmitted to receiver equipment, for example the terminal equipment UE of FIG. 4, via the communication network RC, for example in the form of a coded data stream. STR or an F D encoded data file.
• the decoding device 300 which, according to this embodiment, performs decoding of the coded data relating to the complete or partial views Tl, T2, ..., TN in accordance with the MIV standard.
• the segmented decoded views T1DS, T2DS, ..., TNDS are reconstructed (82), either completely or partially (depending on the information contained in the patches).
• modified weight values MWL'l, MWL'2, ..., MWL'M are encoded, they are also decoded then used to update the configuration of the layers of the network of neurons of the SYNT2 processing device.
• the N decoded views T1DS, T2DS, ..., TNDS are presented to the neural network corresponding to the device PROC2.
• a user UT wishes to synthesize an additional view according to the additional point of view PVS.
• the segmented decoded T1DS, T2DS, ..., TNDS views are presented at the input of the synthesis device SYNT2 (certain layers of which L'i could have been updated with the modified values of weights MWL'l, MWL'2, ..., MWL'M decoded, according to the embodiment considered.
• the coordinates of the PVS point of view desired by the user UT are also entered.
• the SYNT2 device finally produces the view additional synthesized TSj'.
• these data are then processed by several neural networks RI, R2, ..., Rk, ..., RK, with K non-zero integer less than or equal to N, each specialized in a diagnostic or diagnostic assistance task.
• these networks are each configured to take as input a subset of the N input data T1(Y), . .., Ti(y), ..., TN(Y) and to produce diagnostic information Di(Y) (for example, presence of a tumor, suspicion of epilepsy, etc.) or a diagnostic aid information (for example, segmentation, that is to say a geographical delimitation of an area likely to contain a tumor, etc.).
• the subset of input data to be presented to each neural network Rk is selected by an algorithm external to the neural network considered and may include part of or all of the data Ti(Y).
• Rk neural networks adapted to the diagnosis of particular pathologies are given in the document by Zhang et al., entitled “Multi-channel deep convolutional neural networks for multi-classifying thyroid disease”, published by the site https://arxiv .org/abs/2203.03627, in March 2022, in the document Dehghani et al., entitled “Joint brain tumor segmentation from multi MR sequences through a deep convolutional neural network”, published by the site https://arxiv.org/abs /2203.03338, in March 2022, and in the document by Salafian et al., entitled “CNN-Aided Factor Graphs with Estimated Mutual Information Characteristics for Seizure Detection”, published by the site https://arxiv.org/abs/2203.05950, in March 2022.
• the processing to be applied to the input data of each of the networks Rk therefore corresponds here to diagnostic assistance processing.
• each processing device DIAG11, DIAG12, ..., DIAG1K comprises a layered neural network RI, R2, ..., RK.
• each of these networks Rk has a structure in successive layers, connected together so that the output of an upstream layer is presented as input to the following downstream layer.
• Each layer has configuration weights whose values are adjusted during prior learning.
• each segmentation module SEGlk' comprises as many layers LI, L2, ..., LN as input data intended for the network Rk.
• the segmentation layers can be shared so that the segmentation module SEG1 has one layer Li per type of input data Ti and the corresponding segmented data is sent to each of the Rk networks which need it.
• the network Rk which receives the data T3, T8 and T9, is connected as input to layers L3(y), L8(y) and L9(y).
• each layer Lik(y) of the module SEGlk' has the same size as the associated input data.
• the layer Li(y) is also of size H weight, each weight being a unique multiplier coefficient associated with a piece of data (this data having for example a scalar or vector value ).
• the weights of each layer Lik(y) of the segmentation module SEGlk' take binary values. If the weight bit has the value 1, the associated data or parameter is transmitted to the Rk network, otherwise it is not transmitted. Note that, in the latter case, a predefined value (for example equal to -1) is transmitted to the DIAGlk processing device.
• training is carried out on the networks combining a certain number of layers Li(Y) and the layers (not shown) of the networks Rk, using a loss function which combines the related loss to the task of the networks Rk (namely the quality of the diagnosis) and a loss representative of the cumulative amplitude of the weights of the layers Ci(Y).
• CRk the network obtained by combining the input layer(s) Li(Y) with the network Rk.
• the learning known in itself, works in the following way. For each value k in 1, ..., K, we will try to maximize the diagnostic capacity of the CRk network from the data. Since we have the original diagnosis Dk(y), it is possible to calculate a difference between this original diagnosis and the network output CRk (for example, the digital diagnostic error). This difference is used to cal- culate a gradient which is back-propagated in the weights of the combined network CRk to be optimized, according to the known formula.
• A is a parameter set by the user which makes it possible to determine a choice between minimizing the size of the segmented zones (A. high) or the diagnostic performance (A low) and APLi is a vector concatenation of the weights of K layers Li present in the neural network of the segmentation module SEGlk' at the input of the DIAGlk processing device. These are real positives that can be larger or smaller than 1.
• the weights of the network Rk contained in the neural network CRk are fixed during learning, only the weights of the layers Li of the network of the segmentation module SEGlk' are optimized in the sense of the loss function explained above.
• the weights of the layers of the network Rk are also modified by learning. Their modified values (not shown) are stored in memory.
• a segmentation map is a map of the same size as the data, containing for each parameter of the data the value 0 or 1, 0 meaning that the parameter associated in the data is not used for the subsequent diagnostic task, the value 1 meaning the opposite.
• the determination of the corresponding value of the segmentation map is done by comparing each weight to a given threshold, by example equal to 0.01. If the weight value is lower than this threshold, then the corresponding data in the segmentation map is set to 0, otherwise it is set to 1.
• the weight values are binary, they are directly copied into the corresponding SGIi segmentation map (the SGIi map is equal to the weight map of layer Ci).
• segmentation card for example in the form of a series of binary values indicating for each parameter or measurement of the Ti sequence considered whether it is coded or not.
• the encoded data is stored in memory and/or transmitted in a data file or a data stream.
• the segmented decoded data sequences T1DS, T2DS, ..., TNDS are then reconstructed respectively (in 82, by CONST1, CONST2, ..., CONSTK) for each of the diagnostic devices DIAG21, DIAG22, ... , DIAG2K, whether complete (all the parameters of the data sequence can be reconstructed) or partial (the transmitted parameters only allow part of the sequence to be reconstructed) from the decoded subset of data to be processed USSD and decoded segmentation cards SGD1, SGD2, ..., SGDK.
• the decoded data includes modified weight values for the Rk networks, these modified weight values are injected into the Rk networks concerned.
• a user here a practitioner wishes to carry out a diagnosis using a network Rk of Figure 11, for example the network R2, and some of the reconstructed data sequences, for example T3D, T8D and T9D.
• the segmented data T1DS, T2DS, ..., TNDS reconstructed during decoding (possibly partial) are entered into the layers Ci, themselves connected to the input of the network CRk of the DIAG2k device.
• the Rk network can then produce its diagnosis from the segmented data alone.
• a device 100 for segmenting a plurality of input data acquired by sensors comprising a module for determining weight values at applying to the plurality of input data before processing by at least one processing device configured to produce a processing result according to a criterion of maximizing a quality measure of the processing result of the input data, said weight values being determined as a function of said criterion and another criterion for minimizing a quantity of input data to be processed, a module for determining segmentation information of said plurality of input data, said segmentation information of said data being valued at a first value or at a second value distinct from the first, depending on said weights, and a module for obtaining a subset of data to be process by applying determined segmentation information to said plurality of input data, the subset of data to be processed comprising the data of the plurality of input data associated with segmentation information equal to the first value.
• module can correspond as well to a software component as to a hardware component or a set of hardware and software components, a software component itself corresponding to one or more programs or subprograms of computer or more generally any element of a program capable of implementing a function or a set of functions.
• such a device 100 comprises a random access memory 103 (for example a RAM memory), a processing unit 102 equipped for example with a processor, and controlled by a computer program Pgl, representative of the modules of the aforementioned device 100, stored in a read only memory 101 (for example a ROM memory or a hard disk).
• a computer program Pgl representative of the modules of the aforementioned device 100, stored in a read only memory 101 (for example a ROM memory or a hard disk).
• the code instructions of the computer program are for example loaded into the RAM 103 before being executed by the processor of the processing unit 102.
• the RAM 103 can also contain the segmentation information and the useful data subset.
• Figure 13 illustrates only one particular way, among several possible, of producing the device 100, 100' so that it carries out the steps of the segmentation process as detailed above, in relation to Figures 6, 9 and 11 in its different embodiments. Indeed, these steps can be carried out indifferently on a reprogrammable calculation machine (a PC computer, a DSP processor or a microcontroller) executing a program comprising a sequence of instructions, or on a dedicated calculation machine (for example a set of logic gates like an FPGA or an ASIC, or any other hardware module).
• a reprogrammable calculation machine a PC computer, a DSP processor or a microcontroller
• a program comprising a sequence of instructions
• a dedicated calculation machine for example a set of logic gates like an FPGA or an ASIC, or any other hardware module.
• the corresponding program (that is to say the sequence of instructions) can be stored in a removable storage medium (such as for example an SD card, a USB key, a CD-ROM or a DVD-ROM) or not, this storage medium being partially or totally readable by a computer or a processor.
• a removable storage medium such as for example an SD card, a USB key, a CD-ROM or a DVD-ROM
• FIG. 14 We also present, in relation to Figure 14, an example of hardware structure of a coding device 200, 200' of a plurality of input data according to the invention, comprising at least one module obtaining segmentation information of the plurality of input data and a subset of data to be processed by applying said segmentation information to the plurality of input data, said segmentation comprising determining weight values to be apply to the plurality of input data before processing by at least one processing device previously configured to produce a processing result according to a criterion for maximizing a processing quality measure, said weight values being determined in function of said criterion and another criterion of minimi- sation of a quantity of input data to be processed, said segmentation information of said data being valued at a first or a second value distinct from the first, depending on said weight values, the subset of data to be processing comprising the data of the plurality of input data associated with segmentation information equal to the first value, and a module for coding the segmentation information and the subset of data to be processed.
• module can correspond as well to a software component as to a hardware component or a set of hardware and software components, a software component itself corresponding to one or more programs or subprograms of computer or more generally any element of a program capable of implementing a function or a set of functions.
• such a device 200, 200' comprises a random access memory 203 (for example a RAM memory), a processing unit 202 equipped for example with a processor, and controlled by a computer program Pg2, representative segmentation and coding modules, stored in a read only memory 201 (for example a ROM memory or a hard disk).
• a computer program Pg2 representative segmentation and coding modules, stored in a read only memory 201 (for example a ROM memory or a hard disk).
• the code instructions of the computer program are for example loaded into the RAM 203 before being executed by the processor of the processing unit 202.
• the RAM 203 can also contain the coded information.
• Figure 14 illustrates only one particular way, among several possible, of producing the device 200, 200' so that it carries out the steps of the coding process as detailed above, in relation to Figures 7, 9 and 11 in its different embodiments. Indeed, these steps can be carried out indifferently on a reprogrammable calculation machine (a PC computer, a DSP processor or a microcontroller) executing a program comprising a sequence of instructions, or on a dedicated calculation machine (for example a set of logic gates like an FPGA or an ASIC, or any other hardware module).
• a reprogrammable calculation machine a PC computer, a DSP processor or a microcontroller
• a program comprising a sequence of instructions
• a dedicated calculation machine for example a set of logic gates like an FPGA or an ASIC, or any other hardware module.
• the corresponding program (that is to say the sequence of instructions) can be stored in a removable storage medium (such as such as an SD card, a USB key, a CD-ROM or a DVD-ROM) or not, this storage medium being partially or totally readable by a computer or a processor.
• a removable storage medium such as an SD card, a USB key, a CD-ROM or a DVD-ROM
• a device 300, 300' for decoding coded data comprising a module for decoding coded data comprising segmentation information of a plurality of data acquired by sensors, called input data, and a subset of data to be processed, said segmentation information of said input data being valued at a first value or at a second value distinct from the first, said subset of data to be processed having been obtained by applying said segmentation information to the plurality of input data, the subset of data to be processed comprising the data of the plurality of input data associated with said segmentation information equal to the first value, a module for constructing a plurality of segmented input data decoded from the subset of decoded data to be processed and decoded segmentation information, and a module for providing the plurality of decoded segmented input data to a processing device configured to produce a processing result based on a criterion for maximizing a processing quality measure.
• module can correspond as well to a software component as to a hardware component or a set of hardware and software components, a software component itself corresponding to one or more programs or subprograms of computer or more generally any element of a program capable of implementing a function or a set of functions.
• such a device 300, 300' comprises a random access memory 303 (for example a RAM memory), a processing unit 302 equipped for example with a processor, and controlled by a computer program Pg3, representative of the aforementioned modules, stored in a read only memory 301 (for example a ROM memory or a hard disk).
• a computer program Pg3 representative of the aforementioned modules, stored in a read only memory 301 (for example a ROM memory or a hard disk).
• the code instructions of the computer program are for example loaded into the RAM 303 before being executed by the processor of the processing unit 302.
• Figure 15 illustrates only one particular way, among several possible, of producing the device 300, 300' so that it carries out the steps of the decoding process as detailed above, in relation to Figures 8, 10 and 12 in its different embodiments. Indeed, these steps can be carried out indifferently on a reprogrammable calculation machine (a PC computer, a DSP processor or a microcontroller) executing a program comprising a sequence of instructions, or on a dedicated calculation machine (for example a set of logic gates like an FPGA or an ASIC, or any other hardware module).
• a reprogrammable calculation machine a PC computer, a DSP processor or a microcontroller
• a program comprising a sequence of instructions
• a dedicated calculation machine for example a set of logic gates like an FPGA or an ASIC, or any other hardware module.
• the corresponding program (that is to say the sequence of instructions) can be stored in a removable storage medium (such as such as an SD card, a USB key, a CD-ROM or a DVD-ROM) or not, this storage medium being partially or totally readable by a computer or a processor.
• a removable storage medium such as an SD card, a USB key, a CD-ROM or a DVD-ROM
• the invention which has just been described in its different embodiments has numerous advantages. It makes it possible to select, from among a plurality of data representative of a scene, an object or a subject, before their transmission in a communication network, those which will be really useful for the processing of this plurality of data by a device processing of receiving equipment. To this end, the invention uses machine learning techniques which it cleverly implements to specifically train, at the level of transmitting equipment, an automatic segmentation module to segment the plurality of input data so as to to minimize the quantity of data to be transmitted while maximizing the quality of processing.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Health & Medical Sciences (AREA)
• Artificial Intelligence (AREA)
• Computational Linguistics (AREA)
• Data Mining & Analysis (AREA)
• Evolutionary Computation (AREA)
• Biomedical Technology (AREA)
• Molecular Biology (AREA)
• Computing Systems (AREA)
• General Engineering & Computer Science (AREA)
• Biophysics (AREA)
• Mathematical Physics (AREA)
• Software Systems (AREA)
• Life Sciences & Earth Sciences (AREA)
• Health & Medical Sciences (AREA)
• Databases & Information Systems (AREA)
• Image Processing (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Compression Of Band Width Or Redundancy In Fax (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=82943302

Family Applications (1)

Country Status (5)

Family Cites Families (3)

• 2022
  • 2022-06-22 FR FR2206177A patent/FR3137240A1/fr not_active Withdrawn
• 2023
  • 2023-06-09 EP EP23733230.9A patent/EP4544779A1/de active Pending
  • 2023-06-09 CN CN202380061046.1A patent/CN119678500A/zh active Pending
  • 2023-06-09 WO PCT/EP2023/065446 patent/WO2023247208A1/fr not_active Ceased
  • 2023-06-09 US US18/876,914 patent/US20250380001A1/en active Pending

Also Published As

Similar Documents

Legal Events