WO2022034371A1 - Machine learning based system and method of detecting inauthentic content - Google Patents

Machine learning based system and method of detecting inauthentic content Download PDF

Info

Publication number
WO2022034371A1
WO2022034371A1 PCT/IB2020/061470 IB2020061470W WO2022034371A1 WO 2022034371 A1 WO2022034371 A1 WO 2022034371A1 IB 2020061470 W IB2020061470 W IB 2020061470W WO 2022034371 A1 WO2022034371 A1 WO 2022034371A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
metadata
inauthentic
available
resulting
Prior art date
Application number
PCT/IB2020/061470
Other languages
French (fr)
Inventor
Touradj Ebrahimi
Evgeniy UPENIK
Original Assignee
Quantum Integrity Sa
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Quantum Integrity Sa filed Critical Quantum Integrity Sa
Publication of WO2022034371A1 publication Critical patent/WO2022034371A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds

Definitions

  • the invention relates to detecting of authenticity or integrity of content using machine learning and artificial intelligence technologies.
  • inauthentic content image, video, sound, text
  • inauthentic content is used to dupe the observer in perceiving altered digital content. That comes about by way of “deep fakes” where the idea is to fool the observer (human or otherwise) while monitoring security control checkpoints, inventory departments, manufacturing plants, etc.
  • the observer is fooled when presented with inauthentic testimony of people, especially of famous or notable whether through video, imagery or sound, which never occurred.
  • “deep fakes” are often used in malicious attempts to alter public opinion on particular matters.
  • a method for, receiving data for training and/or for operational mode and separating metadata, if available, from content of the received data Transforming the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data. Injecting inauthentic content into the resulting plurality of the content and plurality of the metadata, if metadata is available. Outputting the processed plurality of the resulting content and metadata, if metadata is available.
  • a system that contains a processor device operatively coupled to a memory device, where the processor device is to, receive data for training and/or for operational mode and separate metadata, if available, from content of the received data. Transform the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data. Inject inauthentic content into the resulting plurality of the content and the metadata, if metadata is available. Output the processed plurality of the resulting content and metadata, if metadata is available.
  • a system that contains a processor device operatively coupled to a memory device, where the processor device is to, detect inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes. Fuse the resulting detection of the inauthentic content from the multiple detection schemes. Provide quantitative and/or descriptive output from the fused resulting detection.
  • a computer program product for detection of inauthentic content based on machine learning
  • the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device for, cause the computing device to perform the method of, receiving data for training and/or for operational mode and separating metadata, if available, from content of the received data.
  • Transforming the content and the metadata if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data.
  • a computer program product for detection of inauthentic content based on machine learning
  • the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device for, detecting inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes. Fusing the resulting detection of the inauthentic content from the multiple detection schemes. Providing quantitative and/or descriptive output from the fused resulting detection.
  • FIG. 1 shows a schematic view of the machine learning based system of detecting inauthentic content
  • FIG. 2 shows a block diagram of the machine learning based system of detecting inauthentic content specifically the environment of the creator
  • FIG. 3 shows a block diagram of the machine learning based system of detecting inauthentic content specifically the environment of the discriminator
  • FIG. 4 shows a schematic view of the machine learning based system’s electronic and processing circuits used for detecting inauthentic content
  • Fig. 5 shows a diagram of the machine learning based method of detecting inauthentic content specifically the environment of the method steps of the creator component; and [0019] Fig. 6 shows a diagram of the machine learning based method of detecting inauthentic content specifically the environment of the method steps of the discriminator component.
  • the present system and method are composed of metadata extractor, creator, discriminator and a performance assessor, where some or all of the sections may be implemented, rearranged, repeated in a segmented basis as explained in some embodiments. Moreover, some or all of these sections can be carried out on a distributed fashion such as on distributed neuro-processors, synchronous to one another, asynchronous, or using serial or parallel computation tasks.
  • Figure 1 the present machine learning based system of detecting inauthentic content is described in a schematic view from a high-level perspective. It is projected that this system and its ancillary components can be integrated into a computing device, although it is not limited in that configuration. It is also projected that the computational parts of the system, processed through the method steps and through a computer programmable code, may be carried out on a remote system or on a distributed system over a wired or wireless network. Moreover, in some embodiments, the method or process steps can be implemented on a distributed neuro- processors integrated with storage and memory that do not require executable code to operate. [0022] Here, Figure 1 exemplifies an embodiment of the machine learning based system of detecting inauthentic content in environment 100.
  • the system is fed data 102 for processing.
  • the data 102 can be an image, video or any type of content represented in any format, resolution or rate or any media.
  • the data 102 is then pre-processed through a metadata extractor 104 which separates the auxiliary information (metadata 105) from the primary content (content 103), resulting in information that is to be used for further processing.
  • This pre-processing stage can instead be integrated into the creator component 106 and/or the discriminator 112.
  • the metadata extractor 104 is incorporated into the system environment 100 at the pre-processing stage, although it is not limited to this constellation.
  • the metadata extractor 104 extracts the metadata 105 information, from the data 102, such as the date and time when data 102 was generated or captured.
  • Metadata 105 information extracted can be shutter speed, frame rate, exposure settings, Global Positioning System (GPS) coordinates, make and model of the device used to capture the data 102, the data 102 characteristics such as the format, size, error correcting protocol, encoding protocol, authorship and attribution information, watermarked or embedded information, or the like. In some embodiments, the metadata 105 information may not be present because certain types of data 102 may not have any to begin with. [0023] The metadata 105 information is then supplied to the creator component 106 and/or the discriminator component 112 depending on whether the switching component 108 directs data flow from the creator component 106 to the discriminator component 112.
  • GPS Global Positioning System
  • the metadata extractor 104 is also supplying the content 103 to the creator component 106 and can also supply the content 103 to the discriminator component 112 based on whether the switching component 108 is switched to receive input from the metadata extractor 104 and/or the creator component 105.
  • the switching component 108 has a data switch 109 which is controlled by the system 100 in order to direct content 103 and metadata 105 information into the discriminator component 112.
  • the data switch 109 can direct the output (processed information 107) of the creator component 106 into the discriminator component 112.
  • the system environment 100 switches component 108 to engage the data switch 109 to select as an input source, composed of processed information 107 outputted from the creator component 106 and flowing into the discriminator component 112.
  • the switching component 108 will engage the data switch 109 to select as an input source, composed of the content 103 and metadata 105 to be supplied to the discriminator component 112.
  • the content 103 is supplied to the discriminator component 112, especially in situations when there is no metadata 105 to be derived from the data 102.
  • the creator component 106 can still receive content 103 and metadata 105 for training, while at the same time, the discriminator component 112 is running in an operational mode to detect inauthentic content when the switching component 108 has engaged the data switch 109 such that the discriminator component’s 112 input source is composed of the content 103 and metadata 105.
  • the switching component 108 can select to receive content from both the creator component 106 and the discriminator 112 such that the discriminator component 112 receives both the processed information 107 and the content 103 and if available, the metadata 105.
  • the output 114 flow of the discriminator component 112 is provided as a final result and sometimes as a bias input to the performance assessor component 110.
  • the performance assessor component 110 operates as a watchdog to measure efficacy of the overall operation and supply a feedback loop 111 flow to the discriminator component 112 and/or the creator component 106 based on some measured parameters.
  • the performance assessor component 110 may be an optional component
  • the feedback loop 111 flow is intended to modify the weighted components of a normative neural network or a Machine Leaming/Artificial Intelligence (ML/AI) based learning algorithm.
  • ML/AI Machine Leaming/Artificial Intelligence
  • the performance assessor component 110 can modify the behavior of the discriminator component 112 and/or the creator component 106.
  • GAN Geneative Adversarial Network
  • a content supplying module is trying to fool the discriminating module. So, there is a constant competition between the two modules at this arms race. The better the discriminator is able to detect the false positives and inform the system (i.e. the creator module) to better modify and supply better information in order to try to fool the discriminator.
  • the performance assessor component 110 can affect the behaviors of both the discriminator component 112 and the creator component 106 in order to improve the performance of the system 100.
  • the system 100 can operate in a supervised, unsupervised or semi-supervised learning fashion.
  • other ML/AI architectures are contemplated that do not rely on the GAN methodology such that the performance assessor component 110 can be placed at any stage of the process.
  • the system 100 can include multiple GAN or non-GAN methodologies. Some of these methodologies can be training the system 100 while at the same time detecting inauthentic content during the operational mode.
  • some embodiments of the systems 100 can be running in the operational mode, either simultaneously or out of sync with one another and/or on a distributed network and/or in a distributed neuro-processors setup, in relation to training of the systems 100.
  • the multiple methodologies can operate in a parallel or serial manner, synchronously or asynchronously to one another or over a distributed network or on distributed neuro-processors.
  • the performance assessor component 110 can be based on either human participants or quantifiable metrics acceptable with the specific use case. It is envisioned that in one embodiment, the performance assessor component 110 is based on human intervention where individuals observe the data and label or grade objects detected in the data. In another embodiment, the quantifiable metrics can be based on the noise level of the data or even the “naturalness” aspect of the data. However, in other embodiments, the performance assessor component 110 can be based on a hybrid system of human intervention and quantifiable metrics. In yet another embodiment, the performance assessor component 110 is optional and no feedback loop 111 flow are sent to the creator 106 or discriminator component 112.
  • Figure 2 is a block diagram of the machine learning based system of detecting inauthentic content showing the environment of the creator 200.
  • the creator 200 uses the content 203 input and metadata 205 input and in some embodiments, passes through those inputs unto the augmentation component 206. Then after the content 203 and metadata 205 have gone through some transformation in the augmentation component 206, they are fed into a series of editor components 207. However, in some embodiments, the content 203 and the metadata 205 are first fed into the editor components 207 and then that output is passed unto the augmentation component 206. In other words, the editor components 207 and the augmentation component 206 can be in reverse positions within Figure 2.
  • the content 203 and the metadata 205 are first fed into the augmentation component 206, then they are outputted to a series of editor components 207 and then fed back into another augmentation component (not shown). If the augmentation component 206 is placed before the series of editor components 207, then the creator 200 and the overall system consumes more processing resources whereas when the augmentation component 206 is placed after the series of editors 207, the processing burden is lower as the calculations are simpler.
  • the metadata 205 is not present and thus it is not augmented or edited in the creator 200.
  • the augmentation component 206 creates multiples of the original data represented by the content 203 and the metadata 205. Each portion of data, whether an image, video sequence, sound or text, is transformed into a multiple in the order of 10, 100, 1000 and in some cases, even more.
  • the content 203 along with the metadata 205 is used to scale up/down, rotate, mirror based on the horizontal or the vertical axes, compress, project, convert the color space, change the contrast, e.g. gamma correction, tone mapping, crop, extract regions of interest and any other transformation in order to generate additional information during training.
  • the output of the augmentation component 206 is the transformed content 204. If the content 203 is of some other type, then germane transformation steps are carried out in the augmentation component 206 so that conforming data is generated for processing.
  • the transformed content 204 is then fed into a series of editor components 207 which can operate in an automatic, semiautomatic fashion using machines or with use of human intervention.
  • the editor components 207 can show indicators as to where the inauthentic content is generated and ask the user to confirm entry of that inauthentic content.
  • the editor components 207 are directly operated by a human operator by allowing them to inject additional inauthentic content into the system.
  • the editor components 207 can automatically generate -on a random basis- inauthentic content and insert it into the transformed content 204.
  • These editor components 207 can be off-the-shelf array of algorithms or an entirely new set of algorithms.
  • the series of editor components 207 can process a volume of data and can run in series or in parallel to one another. The output of each editor component 207 is then sent to the discriminator component 112 for processing.
  • a first augmentation component 206 is used as a preprocessing step of the creator 106 and the transformed content 204 is directly fed into the editor components 207. The output of those editor components 207 are again fed into another augmentation component 206 for further processing.
  • the editor components 207 modify the content 203 and metadata 205 first before the outputted content is sent off into an array of augmentation components 206 for final processing. This latter scenario can be used in applications that require conservation of processing power in order to reduce data flow fed into the editor components 207.
  • the augmentation components 206 can be applied at before and after the editor components 207. In any of the embodiments listed above, once the content 203 and metadata 205 are processed, the outputted data 209 is then fed into the discriminator 112.
  • the performance assessor component 110 has the feedback loop 211 flow biasing the creator 106 by changing the behavior of the augmentation component 206 and/or the behavior of the editor components 207.
  • the creator 200 may not be supplying the discriminator component 112 with optimal data in which case the parameters of the augmentation component 206 and/or the editor components 207 may be varied in order to increase the efficacy of the discriminator component 112 or the overall system 100.
  • Figure 3 is a block diagram of the machine learning based system of detecting inauthentic content showing the environment of the discriminator component 300.
  • the discriminator 300 receives the outputted data 209 as content 303 as well as metadata 305 streams if it is available.
  • the content 303 and in some circumstances, the metadata 305, are inputted into the preprocessor 307.
  • the content 303 is processed through a classifier component 309 and then the classified signal, which results into classified metadata 306, is supplied to the context extractor component 311.
  • the classifier component 309 may be a multiple of classifiers. The classifier component 309 therefore converts content 303 into a description of the content. If metadata 305 is available, it is also inputted into the context extractor component 311 in order to further contextualize the classified metadata 306 and provide a supplemental metadata
  • the classifier component 309 can discern whether an image is of a person or an animal and thus provide classified metadata 306 which states it is an image of humans or animals. Then that classified metadata 306 can further be refined by using the metadata 305 with the context extractor component 311 to determine where for example the image was taken by a particular camera, or within a specific time frame. Based on that refining, the supplemental metadata 310 can provide contextualized information to the controller 308. If, however, metadata 305 is not available, the content extractor 311 still processes the classified metadata 306 to further refine it and provide compatible information that is the supplemental metadata 310.
  • the combination of the classifier component 309 and the context extractor component 311 may be implemented by any number of classification algorithms employing ML/AI methodologies to that end. Detection of objects of germane content is not limited to visual objects but can be written information identified by the classifier component 309 and context extractor component
  • the supplemental metadata 310 is sent to the fusion component 315. Moreover, the supplemental metadata 310 of the preprocessor 307 is also fed into the controller component 308.
  • the controller component 308 is used to supply directive signal 312 to the fusion component 315 and in some embodiments, to the series of detector components 313.
  • the controller component 308 is a circuit or algorithm implemented to drive the detector components 313 and in order to impact the value assigned to the individual detector components 313 and ultimately the overall outcome.
  • the controller component 308 determines that the supplemental metadata 310 from the preprocessor 307 is of a type which includes head and shoulders of a person, then the controller component 308 indicates to the fusion component 315 -using directive signal 312- as to which detector components 313 to assign greater weight to, and in some circumstances, engages those detector components 313 that are better suited for processing head and shoulder type information.
  • the controller component 308 can direct any of the detector components 313 to engage and detect the inauthentic content.
  • the directive signal 312 thus operates as a control signal to engage those detector components 313 that are germane for the specific supplemental metadata 310 information extracted by the content extractor 311 as well as an informer to the fusion component 315 as to which detector components 313 to give more relevance to.
  • the controller component 308 only sends the directive signal 312 to the fusion component 315. Therefore, affecting the way the fusion component 315 weighs the output of each of the detector components 313.
  • the detector components 313 are also supplied with the content 303 which contains the authentic or inauthentic information for detection purposes.
  • the output of each of the detector components 313 -in the case when the controller component 308 is employed with a directive signal 312 to the selected detector components 313- is then supplied to the fusion component 315.
  • the fusion component 315 takes a multitude of information to fuse the data and supply the user with practical information.
  • the fusion component 315 uses both the supplemental metadata 310 from the pre-processor 307, the output from the detector components 313 and the controller component’s 308 directive signal 312 to fuse the data and output it to the user.
  • the fusion component 315 can employ a variety of methodologies to conduct the fusion of data.
  • the fusion component 315 can use a simple majority algorithm, can calculate a minimum, a maximum, median, average, the simple sum, employ a non-linear function, a statistical or regression method or even a learning-based method, such as a deep neural network (DNN), to fuse the data.
  • DNN deep neural network
  • the outputs) 317 of the fusion component 315 can be supplied in different categories.
  • One category of output 317 is the determination of whether content was detected to be inauthentic, whether no inauthentic content was detected or whether the system is unsure whether the content is inauthentic or authentic.
  • Another category of output 317 can be the probability that inauthentic content was detected, the probability that the content is authentic or the probability that the system does not know if the content is authentic or inauthentic. Because the fusion component 315 can be trained based on independent statistical methodologies, the probability of inauthentic, authentic, or undetermined values can add up to be greater or less than one.
  • Yet another output 317 category can be information pertaining to the segment of the content such as location, position, duration, region and shape of the inauthentic content.
  • the various outputs of the fusion component 315 can also be used as input into the performance assessor component 110 which can have a feedback loop 316 flow into the creator component 106 and discriminator component 300/112 to vary the final performance of the overall system.
  • the feedback loop 316 flow can be supplied to various components of the discriminator component 300/112.
  • the feedback loop 316 flow can input a signal into the pre-processor 307 to vary the behavior of the classifier component 309 and/or the context extractor component 311.
  • the feedback loop 316 flow can be inputted into the controller component 308 and/or the detector components 313 and/or the fusion component 315 to change the output 317 of the fusion component 315.
  • the performance assessor component 110 is not present and thus not feedback loop 316 flow exists to affect the behavior of any other component
  • FIG. 4 is an embodiment of a block diagram showing the system architecture of the machine learning based system and method of detecting inauthentic content in environment 400 that is referred to as a representation of a computational instance 410.
  • the processing device 401 can be a computer such as a server in a datacenter, a field programable gate array (FPGA), application specific integrated circuit (ASIC), a neuro-processor, a personal computer (PC), a tablet pc, a smart-phone or the like.
  • the processing device 401 carries on all the operations and computational aspects of the machine learning based system and method of detecting inauthentic content 400 using a processors) 406 and a memory 408.
  • the processors) 406 can be a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Visual Processing Unit (VPU), or a series of processors and/or microprocessors, but are not limited in this regard, connected in series or parallel to execute the functions relayed through the memory 408 which may house the software programs and/or sets of executable instructions.
  • the processors) 406 can be based on a distributed system such that the computational load is offloaded unto a cloud-based server 415.
  • the processor 406 and memory 408 are interconnected via bus lines or other intermediary connections.
  • the processors(s) 406 can also send control signals to the other components of the machine learning based system and method of detecting inauthentic content 400.
  • the memory 408 can be a conventional memory device such as RAM (Random Access Memory), ROM (Read Only Memory) or other volatile or non-volatile basis that is connected to the processors) 406.
  • the memory 408 includes one or more memory devices, each of which includes, or a plurality of which collectively include a computer readable storage medium.
  • the computer readable storage medium may include a read-only memory (ROM), a flash memory, a floppy disk, a hard disk, an optical disc, a flash disk, a flash drive, a tape, a database accessible from a network, and/or any storage medium with the same functionality that can be contemplated by persons of ordinary skill in the art to which this disclosure pertains.
  • the processing device 401 is connected to various other aspects of the machine learning based system and method of detecting inauthentic content 400.
  • the processing device 401 is connected to a communication module 404 which enables it to communicate with a network 402 or remote servers 415 on a wired or on a wireless basis.
  • communication module 404 can also communicate with a network 402 such as a cloud or a web, on a need to basis.
  • the computational instance 410 can process information and execute the methods of the system 100, as discussed below, based on one or more occurrences of computational instance 410.
  • the processing device 401 is also connected to a terminal 403 which allows the user to interact with the system 400 and view the process such as manipulating the operational instructions flowing in between the processors) 406, the network 402, the computational instance 410, the communication module 404, the remote server 415, and the memory 408. Moreover, the terminal 403 can be used to input additional data from an external source such as the remote server 415, the network 402 or another computational instance 410. The processing device 401 can use the terminal 403 which allows the user to see the operations and output of the machine learning based system and method of detecting inauthentic content 400.
  • the processing device 401 can take input signal from the terminal 403 by using voice commands, touchscreens, incorporation of a stylus pen, hand gestures or body “language.”
  • the processing device 401 can house some or all of the training data or the algorithms needed for recall.
  • the remote servers 415 also has a built-in database for recalling training data or algorithms.
  • the algorithm and training data can flow from the terminal 403, the remote server 415, the network 402 or another computational instance 410.
  • the processing device 401 can also have a storage 416 for long- or short-term storage of applications, intermediate data, database, look-up tables (LUTs), the operating system, any executable code, etc.
  • LUTs look-up tables
  • Figure 5 depicts an exemplary flowchart illustrating the machine learning based method of detecting inauthentic content in environment 500, specifically the environment of the method steps of the creator component This method 500 is described for any of the embodiments discussed above under Figure 1 and Figure 2.
  • the method begins the process of intaking data 102 at input II.
  • the method checks whether content 102 was inputted. If data 102 was inputted, then it proceeds to step S3. However, if at step S2, no data 102 was inputted, then it goes back to step SI to wait for input of data 102.
  • step S3 the method 500 determines whether the creator component 106 should prepare the data 102 for training the system or not. If it is chosen to prepare the data 102 for training the system, then it proceeds to step S4. If, however, it is chosen not to prepare the data 102 for training the system, the data 102 supplied is outputted at output Ol. In some embodiments, step S3 allows to both prepare the data 102 for training and also output it separately in output Ol for further processing.
  • step S4 the method 500 goes through a decision step within the metadata extractor 104 of deciding whether there is metadata in the data 102 which can be separated into the auxiliary information (metadata 105) from the primary content (content 103). If metadata in the data 102 is present, then in step S5, the metadata extractor 104 extracts the metadata 105 and the content 103. If, however, no metadata 105 is present, the data 102/content 103 is sent to the augmentation component 206 for further transformation of the data 102/content 103 in step S6.
  • the content 103 and the metadata 105 are transformed at step S6 as discussed above, to create plurality of the content 103 and sometimes the metadata 105.
  • the augmentation component 206 at step S6 can be a series of augmentation steps such that content 103 and metadata 105 is transformed multiple times. Therefore, this step can be iterated in a sequence to achieve a plurality of transformed content 103 and sometimes metadata 105.
  • the editor components 207 can inject the inauthentic content in an iterative fashion at step S7.
  • step S7 the plurality of the transformed content 103 and the available plurality of the transformed metadata 105 are inputted into the editor components 207 to inject inauthentic content into them and therefore output it to output Ol.
  • multiple instances of augmentation and editors in steps S6 and S7, respectively, can be operating to provide plurality of the transformed content 103 and metadata 105, if available, and inject the same with inauthentic content
  • step S8 if feedback loop 111/211 flow is available from the performance assessor component 110, then that feedback loop 111/211 flow is used to bias the augmentation component 206 and/or the editor components 207.
  • the performance assessor component 110 uses quantifiable and descriptive results outputted from the discriminator component 112 in order to affect the behavior of the creator component 106 and/or the discriminator component 112.
  • Figure 6 depicts an exemplary flowchart illustrating the machine learning based method of detecting inauthentic content in environment 600 specifically the environment of the method steps of the discriminator component.
  • This method 600 is described for any of the embodiments discussed above under Figure 1 and Figure 3.
  • the method begins the process of receiving input Ol that is coming from the creator component 106 and/or from the data 102.
  • the method 600 determines whether to engage the switch 109 and process in an operational mode or not If operational mode is selected, the data 102 is received unprocessed through the creator component 106 in order to separate the metadata in steps S2 and S3. However, if the discriminator component 112 is not placed in an operational mode, then the creator component’s 106 outputted information Ol is further processed through steps S4-S9.
  • the metadata extractor 104 is inputted into the metadata extractor 104 in S2. If metadata is detected in the data 102, the metadata extractor 104 extracts the metadata 105 at step S3. If, however no metadata is detected and extracted from the data 102, the information Ol is processed at step S4. As stated above, the metadata extractor 104 separates data 102 into the auxiliary information (metadata 105) from the primary content (content 103). At step S4, the method 600 proceeds into classifying the content 103 using the classifier(s) 309. The output of the classifiers) 309 indicated by 306 supplies contextual information about the content 103 which can be designated as metadata.
  • the metadata 105/305 is used by the context extractor 311 at step S5 to further refine the contextual information 306, which results in an output designated as supplemental metadata 310. If, however, no metadata 105/305 is available, the context extractor 311 at step S5, nonetheless refines the contextual information 306 to produce a supplemental metadata 310.
  • the supplemental metadata 310 is supplied both to the controller component 308 as well as to the fusion component 315.
  • the controller component 308 determines which of the detectors 313 to engage and supplies a directive signal 312 to the detector as well as to the fusion component 315.
  • the detector components 313 use inputted Ol which can be coming from the creator component 106 and/or the content 103 to detect inauthentic content.
  • the result of the detected inauthentic content is inputted into the fusion component 315 in step S8 to further process that information using the directive signal 312 and the supplemental metadata 310.
  • the final output from the fusion component 315 at output 02 is communicated to a user at terminal 403 or to a computational instance 410.
  • the performance assessor evaluates the output 02 and provides a feedback loop 316 flow to any of the components in the system such as the classifiers) 309, the context extractor 311, the controller 308, the detector components 313, the fusion component 315, the augmentation component 206 through 211, or the editor components 207 through 211, in order to affect their behavior.
  • method 600 can process first, before method 500, or vice versa. In some embodiments, methods 500 and 600 can be processed together. In some embodiments, the training aspect of the system and the operational mode of the system can be in process such that the creator component 106 and the discriminator component 112 are turned on. [0048] It should be noted that, in some embodiments, the method 500/600 may be implemented as a computer program.
  • the computer program When the computer program is executed by a computer, an electronic device, or the one or more processors 406 in Figure 4, carries on the method 500/600 as shown in Figures. 5-6.
  • the computer program can be stored in a non-transitory computer readable medium such as a ROM, a flash memory, a floppy disk, a hard disk, an optical disc, a flash disk, a flash drive, a tape, a database accessible from a network, or any storage medium with the same functionality that can be contemplated by persons of ordinary skill in the art to which this disclosure pertains.

Abstract

The machine learning based system and method of detecting inauthentic content incorporating a creator component, a discriminator component and possibly a performance assessor component. The creator component increases the volume of the inputted data through various transformations and also injects inauthentic content into the inputted data. The discriminator component in turn extracts contextual information and detects inauthentic content using multiple detector components. The output of the detector components is then fused to formulate a final decision about the authenticity of the inputted data. Then, that final output may be used by the performance assessor component to better train the overall system.

Description

Machine learning based system and method of detecting inauthentic content
PRIORITY CLAIMS AND CROSS-REFERENCE
TO RELATED APPLICATIONS
[0001] This application is related to and claims domestic priority benefits, per 35 USC §119(e), from U.S. Provisional Patent Application Ser. No. 63/063,956 filed on August 22, 2019, the entire contents, of the aforementioned applications, are expressly incorporated hereinto by reference.
BACKGROUND
[0002] The invention relates to detecting of authenticity or integrity of content using machine learning and artificial intelligence technologies.
[0003] It is reported that some jurisdictions are becoming aware of the prevalence of inauthentic image and/or video content being produced and consumed by the public. Some of that technological feats are an advancement of human knowledge and some may be detrimental to human endeavors.
[0004] In the former case, some of that technology is intended to help commercial activity such as when lifelike models are generated for fashion and photography use cases, therefore aiding in cutting down on costs. Other use cases allow consumers to insert realistic representation of objects in an image in order to better visualize the image.
[0005] In the latter case, however, inauthentic content (image, video, sound, text) is used to dupe the observer in perceiving altered digital content. That comes about by way of “deep fakes” where the idea is to fool the observer (human or otherwise) while monitoring security control checkpoints, inventory departments, manufacturing plants, etc. In other blatant cases, the observer is fooled when presented with inauthentic testimony of people, especially of famous or notable whether through video, imagery or sound, which never occurred. When disseminated through mass media, “deep fakes” are often used in malicious attempts to alter public opinion on particular matters.
[0006] In the latter situation, many governing bodies have taken steps to curb such misdeeds by way of legislation. For example, the State of California has enacted laws banning the use of materially deceptive media that are used to create fake pornography when consent of the participants is not available. Another law passed in the State of California aims to prohibit distribution of fabricated videos of a political candidate within 60 days of an election. These are just a few examples of how governments are aiming to solve these types of problems. Yet, there are others who are finding ways to curb these measures using technological means.
SUMMARY
[0007] In accordance with an aspect of the invention, there is provided a method for, receiving data for training and/or for operational mode and separating metadata, if available, from content of the received data. Transforming the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data. Injecting inauthentic content into the resulting plurality of the content and plurality of the metadata, if metadata is available. Outputting the processed plurality of the resulting content and metadata, if metadata is available. [0008] In accordance with another aspect of the invention, there is provided a method for, detecting inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes. Fusing the resulting detection of the inauthentic content from the multiple detection schemes. Providing quantitative and/or descriptive output from the fused resulting detection.
[0009] In accordance with an aspect of the invention, there is provided a system that contains a processor device operatively coupled to a memory device, where the processor device is to, receive data for training and/or for operational mode and separate metadata, if available, from content of the received data. Transform the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data. Inject inauthentic content into the resulting plurality of the content and the metadata, if metadata is available. Output the processed plurality of the resulting content and metadata, if metadata is available.
[0010] In accordance with another aspect of the invention, there is provided a system that contains a processor device operatively coupled to a memory device, where the processor device is to, detect inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes. Fuse the resulting detection of the inauthentic content from the multiple detection schemes. Provide quantitative and/or descriptive output from the fused resulting detection.
[0011] In accordance with an aspect of the invention, there is provided a computer program product for detection of inauthentic content based on machine learning, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device for, cause the computing device to perform the method of, receiving data for training and/or for operational mode and separating metadata, if available, from content of the received data. Transforming the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data. Injecting inauthentic content into the resulting plurality of the content and plurality of the metadata, if metadata is available. Outputting the processed plurality of the resulting content and metadata, if metadata is available.
[0012] In accordance with another aspect of the invention, there is provided a computer program product for detection of inauthentic content based on machine learning, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device for, detecting inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes. Fusing the resulting detection of the inauthentic content from the multiple detection schemes. Providing quantitative and/or descriptive output from the fused resulting detection.
DESCRIPTION OF THE DRAWINGS
[0013] Embodiments in accordance with the present invention are shown in the drawings and will be described below with reference to the figures, whereby elements having the same effect have been provided with the same reference numerals. The following is shown:
[0014] Fig. 1 shows a schematic view of the machine learning based system of detecting inauthentic content;
[0015] Fig. 2 shows a block diagram of the machine learning based system of detecting inauthentic content specifically the environment of the creator;
[0016] Fig. 3 shows a block diagram of the machine learning based system of detecting inauthentic content specifically the environment of the discriminator;
[0017] Fig. 4 shows a schematic view of the machine learning based system’s electronic and processing circuits used for detecting inauthentic content;
[0018] Fig. 5 shows a diagram of the machine learning based method of detecting inauthentic content specifically the environment of the method steps of the creator component; and [0019] Fig. 6 shows a diagram of the machine learning based method of detecting inauthentic content specifically the environment of the method steps of the discriminator component.
DESCRIPTION
[0020] The present system and method are composed of metadata extractor, creator, discriminator and a performance assessor, where some or all of the sections may be implemented, rearranged, repeated in a segmented basis as explained in some embodiments. Moreover, some or all of these sections can be carried out on a distributed fashion such as on distributed neuro-processors, synchronous to one another, asynchronous, or using serial or parallel computation tasks.
[0021] In Figure 1 , the present machine learning based system of detecting inauthentic content is described in a schematic view from a high-level perspective. It is projected that this system and its ancillary components can be integrated into a computing device, although it is not limited in that configuration. It is also projected that the computational parts of the system, processed through the method steps and through a computer programmable code, may be carried out on a remote system or on a distributed system over a wired or wireless network. Moreover, in some embodiments, the method or process steps can be implemented on a distributed neuro- processors integrated with storage and memory that do not require executable code to operate. [0022] Here, Figure 1 exemplifies an embodiment of the machine learning based system of detecting inauthentic content in environment 100. The system is fed data 102 for processing. The data 102 can be an image, video or any type of content represented in any format, resolution or rate or any media. The data 102 is then pre-processed through a metadata extractor 104 which separates the auxiliary information (metadata 105) from the primary content (content 103), resulting in information that is to be used for further processing. This pre-processing stage, can instead be integrated into the creator component 106 and/or the discriminator 112. In this embodiment however, the metadata extractor 104 is incorporated into the system environment 100 at the pre-processing stage, although it is not limited to this constellation. The metadata extractor 104, extracts the metadata 105 information, from the data 102, such as the date and time when data 102 was generated or captured. Other metadata 105 information extracted can be shutter speed, frame rate, exposure settings, Global Positioning System (GPS) coordinates, make and model of the device used to capture the data 102, the data 102 characteristics such as the format, size, error correcting protocol, encoding protocol, authorship and attribution information, watermarked or embedded information, or the like. In some embodiments, the metadata 105 information may not be present because certain types of data 102 may not have any to begin with. [0023] The metadata 105 information is then supplied to the creator component 106 and/or the discriminator component 112 depending on whether the switching component 108 directs data flow from the creator component 106 to the discriminator component 112. The metadata extractor 104 is also supplying the content 103 to the creator component 106 and can also supply the content 103 to the discriminator component 112 based on whether the switching component 108 is switched to receive input from the metadata extractor 104 and/or the creator component 105. The switching component 108 has a data switch 109 which is controlled by the system 100 in order to direct content 103 and metadata 105 information into the discriminator component 112. In other embodiments, the data switch 109 can direct the output (processed information 107) of the creator component 106 into the discriminator component 112. For example, during training, the system environment 100 switches component 108 to engage the data switch 109 to select as an input source, composed of processed information 107 outputted from the creator component 106 and flowing into the discriminator component 112. During operational mode, the switching component 108 will engage the data switch 109 to select as an input source, composed of the content 103 and metadata 105 to be supplied to the discriminator component 112. In some embodiments, only the content 103 is supplied to the discriminator component 112, especially in situations when there is no metadata 105 to be derived from the data 102. In other embodiments, the creator component 106 can still receive content 103 and metadata 105 for training, while at the same time, the discriminator component 112 is running in an operational mode to detect inauthentic content when the switching component 108 has engaged the data switch 109 such that the discriminator component’s 112 input source is composed of the content 103 and metadata 105. In which case, if there is feedback loop 111 flow from the performance assessor component 110, then that feedback loop 111 flow can also affect the behavior of the creator component 106 even though the creator component 106 is not supplying processed information 107 to the discriminator component 112. In some embodiments the switching component 108 can select to receive content from both the creator component 106 and the discriminator 112 such that the discriminator component 112 receives both the processed information 107 and the content 103 and if available, the metadata 105.
[0024] The output 114 flow of the discriminator component 112 is provided as a final result and sometimes as a bias input to the performance assessor component 110. The performance assessor component 110 operates as a watchdog to measure efficacy of the overall operation and supply a feedback loop 111 flow to the discriminator component 112 and/or the creator component 106 based on some measured parameters. The performance assessor component 110 may be an optional component The feedback loop 111 flow is intended to modify the weighted components of a normative neural network or a Machine Leaming/Artificial Intelligence (ML/AI) based learning algorithm. This is done in order to bias the system 100 to perform efficiently and provide accurate results by minimizing false positives and increasing true positives when detecting inauthentic content placed in the data 102 and/or inauthentic content placed in the processed information 107. Thus, the performance assessor component 110 can modify the behavior of the discriminator component 112 and/or the creator component 106. During training, a GAN (Generative Adversarial Network) paradigm is adopted where a content supplying module is trying to fool the discriminating module. So, there is a constant competition between the two modules at this arms race. The better the discriminator is able to detect the false positives and inform the system (i.e. the creator module) to better modify and supply better information in order to try to fool the discriminator. In any mode, the performance assessor component 110 can affect the behaviors of both the discriminator component 112 and the creator component 106 in order to improve the performance of the system 100. In this regard, the system 100 can operate in a supervised, unsupervised or semi-supervised learning fashion. However, other ML/AI architectures are contemplated that do not rely on the GAN methodology such that the performance assessor component 110 can be placed at any stage of the process. In other variations, the system 100 can include multiple GAN or non-GAN methodologies. Some of these methodologies can be training the system 100 while at the same time detecting inauthentic content during the operational mode. Moreover, some embodiments of the systems 100 can be running in the operational mode, either simultaneously or out of sync with one another and/or on a distributed network and/or in a distributed neuro-processors setup, in relation to training of the systems 100. The multiple methodologies can operate in a parallel or serial manner, synchronously or asynchronously to one another or over a distributed network or on distributed neuro-processors.
[0025] The performance assessor component 110 can be based on either human participants or quantifiable metrics acceptable with the specific use case. It is envisioned that in one embodiment, the performance assessor component 110 is based on human intervention where individuals observe the data and label or grade objects detected in the data. In another embodiment, the quantifiable metrics can be based on the noise level of the data or even the “naturalness” aspect of the data. However, in other embodiments, the performance assessor component 110 can be based on a hybrid system of human intervention and quantifiable metrics. In yet another embodiment, the performance assessor component 110 is optional and no feedback loop 111 flow are sent to the creator 106 or discriminator component 112.
[0026] Figure 2 is a block diagram of the machine learning based system of detecting inauthentic content showing the environment of the creator 200. The creator 200 uses the content 203 input and metadata 205 input and in some embodiments, passes through those inputs unto the augmentation component 206. Then after the content 203 and metadata 205 have gone through some transformation in the augmentation component 206, they are fed into a series of editor components 207. However, in some embodiments, the content 203 and the metadata 205 are first fed into the editor components 207 and then that output is passed unto the augmentation component 206. In other words, the editor components 207 and the augmentation component 206 can be in reverse positions within Figure 2. Yet in some embodiments, the content 203 and the metadata 205 are first fed into the augmentation component 206, then they are outputted to a series of editor components 207 and then fed back into another augmentation component (not shown). If the augmentation component 206 is placed before the series of editor components 207, then the creator 200 and the overall system consumes more processing resources whereas when the augmentation component 206 is placed after the series of editors 207, the processing burden is lower as the calculations are simpler. In some embodiments, the metadata 205 is not present and thus it is not augmented or edited in the creator 200.
[002η In some embodiments, the augmentation component 206 creates multiples of the original data represented by the content 203 and the metadata 205. Each portion of data, whether an image, video sequence, sound or text, is transformed into a multiple in the order of 10, 100, 1000 and in some cases, even more. In some variants, for instance when dealing with image and/or video, the content 203 along with the metadata 205 is used to scale up/down, rotate, mirror based on the horizontal or the vertical axes, compress, project, convert the color space, change the contrast, e.g. gamma correction, tone mapping, crop, extract regions of interest and any other transformation in order to generate additional information during training. Thus, the output of the augmentation component 206 is the transformed content 204. If the content 203 is of some other type, then germane transformation steps are carried out in the augmentation component 206 so that conforming data is generated for processing.
[0028] The transformed content 204, in some embodiments, is then fed into a series of editor components 207 which can operate in an automatic, semiautomatic fashion using machines or with use of human intervention. The editor components 207 can show indicators as to where the inauthentic content is generated and ask the user to confirm entry of that inauthentic content. In other embodiments, the editor components 207 are directly operated by a human operator by allowing them to inject additional inauthentic content into the system. In other embodiments, the editor components 207 can automatically generate -on a random basis- inauthentic content and insert it into the transformed content 204. These editor components 207 can be off-the-shelf array of algorithms or an entirely new set of algorithms. Regardless, the series of editor components 207 can process a volume of data and can run in series or in parallel to one another. The output of each editor component 207 is then sent to the discriminator component 112 for processing. [0029] In another embodiment, a first augmentation component 206 is used as a preprocessing step of the creator 106 and the transformed content 204 is directly fed into the editor components 207. The output of those editor components 207 are again fed into another augmentation component 206 for further processing. Yet in other embodiments, the editor components 207 modify the content 203 and metadata 205 first before the outputted content is sent off into an array of augmentation components 206 for final processing. This latter scenario can be used in applications that require conservation of processing power in order to reduce data flow fed into the editor components 207. Yet, in another embodiment, the augmentation components 206 can be applied at before and after the editor components 207. In any of the embodiments listed above, once the content 203 and metadata 205 are processed, the outputted data 209 is then fed into the discriminator 112.
[0030] Moreover, the performance assessor component 110 has the feedback loop 211 flow biasing the creator 106 by changing the behavior of the augmentation component 206 and/or the behavior of the editor components 207. As discussed above, the creator 200 may not be supplying the discriminator component 112 with optimal data in which case the parameters of the augmentation component 206 and/or the editor components 207 may be varied in order to increase the efficacy of the discriminator component 112 or the overall system 100.
[0031] Figure 3 is a block diagram of the machine learning based system of detecting inauthentic content showing the environment of the discriminator component 300. After the creator component 200 transforms and processes the content 203 and/or metadata 205 and has the outputted data 209 sent to the discriminator 300, the discriminator 300 receives the outputted data 209 as content 303 as well as metadata 305 streams if it is available.
[0032] The content 303 and in some circumstances, the metadata 305, are inputted into the preprocessor 307. The content 303 is processed through a classifier component 309 and then the classified signal, which results into classified metadata 306, is supplied to the context extractor component 311. In some embodiments, the classifier component 309 may be a multiple of classifiers. The classifier component 309 therefore converts content 303 into a description of the content. If metadata 305 is available, it is also inputted into the context extractor component 311 in order to further contextualize the classified metadata 306 and provide a supplemental metadata
310. For example, the classifier component 309 can discern whether an image is of a person or an animal and thus provide classified metadata 306 which states it is an image of humans or animals. Then that classified metadata 306 can further be refined by using the metadata 305 with the context extractor component 311 to determine where for example the image was taken by a particular camera, or within a specific time frame. Based on that refining, the supplemental metadata 310 can provide contextualized information to the controller 308. If, however, metadata 305 is not available, the content extractor 311 still processes the classified metadata 306 to further refine it and provide compatible information that is the supplemental metadata 310. The combination of the classifier component 309 and the context extractor component 311 may be implemented by any number of classification algorithms employing ML/AI methodologies to that end. Detection of objects of germane content is not limited to visual objects but can be written information identified by the classifier component 309 and context extractor component
311.
[0033] After the pre-processing step has ended in 307, the supplemental metadata 310 is sent to the fusion component 315. Moreover, the supplemental metadata 310 of the preprocessor 307 is also fed into the controller component 308. The controller component 308 is used to supply directive signal 312 to the fusion component 315 and in some embodiments, to the series of detector components 313. The controller component 308 is a circuit or algorithm implemented to drive the detector components 313 and in order to impact the value assigned to the individual detector components 313 and ultimately the overall outcome. For example, if the controller component 308 determines that the supplemental metadata 310 from the preprocessor 307 is of a type which includes head and shoulders of a person, then the controller component 308 indicates to the fusion component 315 -using directive signal 312- as to which detector components 313 to assign greater weight to, and in some circumstances, engages those detector components 313 that are better suited for processing head and shoulder type information. The controller component 308 can direct any of the detector components 313 to engage and detect the inauthentic content. The directive signal 312 thus operates as a control signal to engage those detector components 313 that are germane for the specific supplemental metadata 310 information extracted by the content extractor 311 as well as an informer to the fusion component 315 as to which detector components 313 to give more relevance to. In another embodiment, the controller component 308 only sends the directive signal 312 to the fusion component 315. Therefore, affecting the way the fusion component 315 weighs the output of each of the detector components 313.
[0034] The detector components 313 are also supplied with the content 303 which contains the authentic or inauthentic information for detection purposes. The output of each of the detector components 313 -in the case when the controller component 308 is employed with a directive signal 312 to the selected detector components 313- is then supplied to the fusion component 315.
[0035] The fusion component 315 takes a multitude of information to fuse the data and supply the user with practical information. The fusion component 315 uses both the supplemental metadata 310 from the pre-processor 307, the output from the detector components 313 and the controller component’s 308 directive signal 312 to fuse the data and output it to the user. The fusion component 315 can employ a variety of methodologies to conduct the fusion of data. The fusion component 315 can use a simple majority algorithm, can calculate a minimum, a maximum, median, average, the simple sum, employ a non-linear function, a statistical or regression method or even a learning-based method, such as a deep neural network (DNN), to fuse the data. The outputs) 317 of the fusion component 315 can be supplied in different categories. One category of output 317 is the determination of whether content was detected to be inauthentic, whether no inauthentic content was detected or whether the system is unsure whether the content is inauthentic or authentic. Another category of output 317 can be the probability that inauthentic content was detected, the probability that the content is authentic or the probability that the system does not know if the content is authentic or inauthentic. Because the fusion component 315 can be trained based on independent statistical methodologies, the probability of inauthentic, authentic, or undetermined values can add up to be greater or less than one. Yet another output 317 category can be information pertaining to the segment of the content such as location, position, duration, region and shape of the inauthentic content.
[0036] The various outputs of the fusion component 315 can also be used as input into the performance assessor component 110 which can have a feedback loop 316 flow into the creator component 106 and discriminator component 300/112 to vary the final performance of the overall system. Thus, the feedback loop 316 flow can be supplied to various components of the discriminator component 300/112. For example, the feedback loop 316 flow can input a signal into the pre-processor 307 to vary the behavior of the classifier component 309 and/or the context extractor component 311. In other embodiments, the feedback loop 316 flow can be inputted into the controller component 308 and/or the detector components 313 and/or the fusion component 315 to change the output 317 of the fusion component 315. In some embodiments, the performance assessor component 110 is not present and thus not feedback loop 316 flow exists to affect the behavior of any other component
[003η Figure 4 is an embodiment of a block diagram showing the system architecture of the machine learning based system and method of detecting inauthentic content in environment 400 that is referred to as a representation of a computational instance 410. The processing device 401 can be a computer such as a server in a datacenter, a field programable gate array (FPGA), application specific integrated circuit (ASIC), a neuro-processor, a personal computer (PC), a tablet pc, a smart-phone or the like. The processing device 401 carries on all the operations and computational aspects of the machine learning based system and method of detecting inauthentic content 400 using a processors) 406 and a memory 408. The processors) 406 can be a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Visual Processing Unit (VPU), or a series of processors and/or microprocessors, but are not limited in this regard, connected in series or parallel to execute the functions relayed through the memory 408 which may house the software programs and/or sets of executable instructions. The processors) 406 can be based on a distributed system such that the computational load is offloaded unto a cloud-based server 415. The processor 406 and memory 408 are interconnected via bus lines or other intermediary connections. The processors(s) 406 can also send control signals to the other components of the machine learning based system and method of detecting inauthentic content 400. The memory 408 can be a conventional memory device such as RAM (Random Access Memory), ROM (Read Only Memory) or other volatile or non-volatile basis that is connected to the processors) 406. The memory 408 includes one or more memory devices, each of which includes, or a plurality of which collectively include a computer readable storage medium. The computer readable storage medium may include a read-only memory (ROM), a flash memory, a floppy disk, a hard disk, an optical disc, a flash disk, a flash drive, a tape, a database accessible from a network, and/or any storage medium with the same functionality that can be contemplated by persons of ordinary skill in the art to which this disclosure pertains.
[0038] The processing device 401 is connected to various other aspects of the machine learning based system and method of detecting inauthentic content 400. For example, the processing device 401 is connected to a communication module 404 which enables it to communicate with a network 402 or remote servers 415 on a wired or on a wireless basis. Moreover, communication module 404 can also communicate with a network 402 such as a cloud or a web, on a need to basis. Thereby, receiving operational instructions and/or data from a source other than what is available to the processors) 406 and/or the memory 408 and/or from the terminal 403. Furthermore, the computational instance 410 can process information and execute the methods of the system 100, as discussed below, based on one or more occurrences of computational instance 410.
[0039] The processing device 401 is also connected to a terminal 403 which allows the user to interact with the system 400 and view the process such as manipulating the operational instructions flowing in between the processors) 406, the network 402, the computational instance 410, the communication module 404, the remote server 415, and the memory 408. Moreover, the terminal 403 can be used to input additional data from an external source such as the remote server 415, the network 402 or another computational instance 410. The processing device 401 can use the terminal 403 which allows the user to see the operations and output of the machine learning based system and method of detecting inauthentic content 400. The processing device 401 can take input signal from the terminal 403 by using voice commands, touchscreens, incorporation of a stylus pen, hand gestures or body “language.” The processing device 401 can house some or all of the training data or the algorithms needed for recall. In other embodiments, the remote servers 415 also has a built-in database for recalling training data or algorithms. In some embodiments, the algorithm and training data can flow from the terminal 403, the remote server 415, the network 402 or another computational instance 410. Furthermore, the processing device 401 can also have a storage 416 for long- or short-term storage of applications, intermediate data, database, look-up tables (LUTs), the operating system, any executable code, etc.
[0040] Figure 5 depicts an exemplary flowchart illustrating the machine learning based method of detecting inauthentic content in environment 500, specifically the environment of the method steps of the creator component This method 500 is described for any of the embodiments discussed above under Figure 1 and Figure 2. At step SI, the method begins the process of intaking data 102 at input II. At step S2, the method checks whether content 102 was inputted. If data 102 was inputted, then it proceeds to step S3. However, if at step S2, no data 102 was inputted, then it goes back to step SI to wait for input of data 102.
[0041] At step S3, the method 500 determines whether the creator component 106 should prepare the data 102 for training the system or not. If it is chosen to prepare the data 102 for training the system, then it proceeds to step S4. If, however, it is chosen not to prepare the data 102 for training the system, the data 102 supplied is outputted at output Ol. In some embodiments, step S3 allows to both prepare the data 102 for training and also output it separately in output Ol for further processing.
[0042] At step S4, the method 500 goes through a decision step within the metadata extractor 104 of deciding whether there is metadata in the data 102 which can be separated into the auxiliary information (metadata 105) from the primary content (content 103). If metadata in the data 102 is present, then in step S5, the metadata extractor 104 extracts the metadata 105 and the content 103. If, however, no metadata 105 is present, the data 102/content 103 is sent to the augmentation component 206 for further transformation of the data 102/content 103 in step S6.
In the case where metadata 105 is available, at step S5, the content 103 and the metadata 105, are transformed at step S6 as discussed above, to create plurality of the content 103 and sometimes the metadata 105. It is worthy to note, that the augmentation component 206 at step S6 can be a series of augmentation steps such that content 103 and metadata 105 is transformed multiple times. Therefore, this step can be iterated in a sequence to achieve a plurality of transformed content 103 and sometimes metadata 105. Likewise, the editor components 207 can inject the inauthentic content in an iterative fashion at step S7.
[0043] At step S7, the plurality of the transformed content 103 and the available plurality of the transformed metadata 105 are inputted into the editor components 207 to inject inauthentic content into them and therefore output it to output Ol. Note that multiple instances of augmentation and editors in steps S6 and S7, respectively, can be operating to provide plurality of the transformed content 103 and metadata 105, if available, and inject the same with inauthentic content At step S8, if feedback loop 111/211 flow is available from the performance assessor component 110, then that feedback loop 111/211 flow is used to bias the augmentation component 206 and/or the editor components 207. The performance assessor component 110 uses quantifiable and descriptive results outputted from the discriminator component 112 in order to affect the behavior of the creator component 106 and/or the discriminator component 112.
[0044] Figure 6 depicts an exemplary flowchart illustrating the machine learning based method of detecting inauthentic content in environment 600 specifically the environment of the method steps of the discriminator component. This method 600 is described for any of the embodiments discussed above under Figure 1 and Figure 3. At step SI, the method begins the process of receiving input Ol that is coming from the creator component 106 and/or from the data 102. At step SI, the method 600 determines whether to engage the switch 109 and process in an operational mode or not If operational mode is selected, the data 102 is received unprocessed through the creator component 106 in order to separate the metadata in steps S2 and S3. However, if the discriminator component 112 is not placed in an operational mode, then the creator component’s 106 outputted information Ol is further processed through steps S4-S9.
[0045] When operational mode is selected at step SI, the input Ol which comprises of data
102, is inputted into the metadata extractor 104 in S2. If metadata is detected in the data 102, the metadata extractor 104 extracts the metadata 105 at step S3. If, however no metadata is detected and extracted from the data 102, the information Ol is processed at step S4. As stated above, the metadata extractor 104 separates data 102 into the auxiliary information (metadata 105) from the primary content (content 103). At step S4, the method 600 proceeds into classifying the content 103 using the classifier(s) 309. The output of the classifiers) 309 indicated by 306 supplies contextual information about the content 103 which can be designated as metadata. Furthermore, if metadata 105 is available, the metadata 105/305 is used by the context extractor 311 at step S5 to further refine the contextual information 306, which results in an output designated as supplemental metadata 310. If, however, no metadata 105/305 is available, the context extractor 311 at step S5, nonetheless refines the contextual information 306 to produce a supplemental metadata 310. The supplemental metadata 310 is supplied both to the controller component 308 as well as to the fusion component 315.
[0046] At step S6, the controller component 308 determines which of the detectors 313 to engage and supplies a directive signal 312 to the detector as well as to the fusion component 315. At step S7, the detector components 313 use inputted Ol which can be coming from the creator component 106 and/or the content 103 to detect inauthentic content. The result of the detected inauthentic content is inputted into the fusion component 315 in step S8 to further process that information using the directive signal 312 and the supplemental metadata 310. The final output from the fusion component 315 at output 02 is communicated to a user at terminal 403 or to a computational instance 410. Furthermore, at step S9, the performance assessor evaluates the output 02 and provides a feedback loop 316 flow to any of the components in the system such as the classifiers) 309, the context extractor 311, the controller 308, the detector components 313, the fusion component 315, the augmentation component 206 through 211, or the editor components 207 through 211, in order to affect their behavior.
[0047] Details of methods 500/600 can be ascertained with reference to the paragraphs above, and a description in this regard will not be repeated herein. Any embodiment described herein incorporating methods 500/600 should not be taken as indication of any order or having a temporal component. In some embodiments, method 600 can process first, before method 500, or vice versa. In some embodiments, methods 500 and 600 can be processed together. In some embodiments, the training aspect of the system and the operational mode of the system can be in process such that the creator component 106 and the discriminator component 112 are turned on. [0048] It should be noted that, in some embodiments, the method 500/600 may be implemented as a computer program. When the computer program is executed by a computer, an electronic device, or the one or more processors 406 in Figure 4, carries on the method 500/600 as shown in Figures. 5-6. The computer program can be stored in a non-transitory computer readable medium such as a ROM, a flash memory, a floppy disk, a hard disk, an optical disc, a flash disk, a flash drive, a tape, a database accessible from a network, or any storage medium with the same functionality that can be contemplated by persons of ordinary skill in the art to which this disclosure pertains.
[0049] In addition, it should be noted that in the operations of the following method 500/600, no particular sequence is required unless otherwise specified. Moreover, any of the operations may also be performed simultaneously, or the execution times thereof may at least partially overlap. Furthermore, some of the components and sequences may be consolidated or subdivided in particular order and/or carried out on a hardware platform. The various components and subroutines can also be carried out on a distributed network.
[0050] Furthermore, the operations of the following method 500/600 may be added to, replaced, and/or eliminated as appropriate, in accordance with various embodiments of the present disclosure. [0051] Although the present disclosure has been described in considerable detail with reference to certain embodiments thereof, other embodiments are possible. Therefore, the scope of the appended claims should not be limited to the description of the embodiments contained herein.

Claims

CLAIMS What is claimed is:
1. A method of implementing a learning-based detection of inauthentic content, the method comprising the steps of: receiving data for training and/or for operational mode; separating metadata, if available, from content of the received data; transforming the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data; injecting inauthentic content into the resulting plurality of the content and plurality of the metadata, if metadata is available; and outputting the processed plurality of the resulting content and metadata, if metadata is available.
2. The method of claim 1, further comprising the steps of: detecting inauthentic content using the plurality of the resulting content and the metadata, if the metadata is available, by incorporating multiple detection schemes; fusing the resulting detection of the inauthentic content from the multiple detection schemes; and providing quantitative and/or descriptive output from the fused resulting detection.
3. The method of claim 2, further comprising the steps of: preprocessing the plurality of the resulting content by classifying the plurality of the resulting content; and further preprocessing the plurality of the resulting content by extracting context from the classified plurality of the resulting content and if metadata is available, additionally using the metadata to extract context.
4. The method of claim 2, further comprising the steps of: classifying each of the plurality of the resulting content using an iterative process and/or a series of classifiers; and further preprocessing the plurality of the resulting content by extracting context from the classified plurality of the resulting content and if metadata is available, additionally using the metadata to extract context.
5. The method of the claim 3, further comprising the steps of: controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the resulting detection of the inauthentic content from the multiple detection schemes; and/or controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
6. The method of the claims 4, further comprising the steps of: controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the resulting detection of the inauthentic content from the multiple detection schemes; and/or controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
7. The methods of any of the claims 1-6, further comprising the step of: using the provided quantitative and/or descriptive output to affect the behavior of the transformation of the content, and/or the injection of any inauthentic content and/or the preprocessing of the plurality of the resulting content, and/or detection of any inauthentic content and/or the controlling of the overall preprocessed information, and/or the fusion of any result of the detected inauthentic content
8. The method of implementing a learning-based detection of inauthentic content, the method comprising the steps of: detecting inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes; fusing the resulting detection of the inauthentic content from the multiple detection schemes; and providing quantitative and/or descriptive output from the fused resulting detection.
9. A method of claim 8, further comprising the steps of: preprocessing the content by classifying the content; and further preprocessing the content by extracting context from the classified content using the metadata, and if metadata is available, additionally using the metadata to extract context.
10. A method of claim 8, further comprising the steps of: preprocessing the content by classifying the content using an iterative process and/or a series of classifiers; and further preprocessing the content by extracting context from the classified content using the metadata, and if metadata is available, additionally using the metadata to extract context.
11. The method of claim 9, further comprising the step of: controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the detection of the inauthentic content from the multiple detection schemes; and controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
12. The method of claim 10, further comprising the step of: controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the detection of the inauthentic content from the multiple detection schemes; and controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
13. The method of any of the claims 8-12, further comprising the step of: using the provided quantitative and/or descriptive output to affect the behavior of the transformation of the content, and/or the preprocessing of the content and the metadata, if metadata is available, and/or detection of any inauthentic content and/or the controlling of the overall preprocessed information, and/or the fusion of any result of the detected inauthentic content
14. A learning-based system of detecting inauthentic content, comprising: a processor device operatively coupled to a memory device, the processor device being configured to: receive data for training and/or for operational mode; separate metadata, if available, from content of the received data; transform the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data; inject inauthentic content into the resulting plurality of the content and the metadata, if metadata is available; and output the processed plurality of the resulting content and metadata, if metadata is available.
15. The system of claim 14, wherein the processor device is further configured to: detect inauthentic content using the plurality of the resulting content and the metadata, if the metadata is available, by incorporating multiple detection schemes; fuse the resulting detection of the inauthentic content from the multiple detection schemes; and provide quantitative and/or descriptive output from the fused resulting detection.
16. The system of claim 15, wherein the processor device is further configured to: preprocess the plurality of the resulting content by classifying the plurality of the resulting content; and further preprocess the plurality of the resulting content by extracting context from the classified plurality of the resulting content and if metadata is available, additionally using the metadata to extract context
17. The system of claim 15, wherein the processor device is further configured to: classify each of the plurality of the resulting content using an iterative process and/or a series of classifiers; and further preprocess the plurality of the resulting content by extracting context from the classified plurality of the resulting content and if metadata is available, additionally using the metadata to extract context
18. The system of claim 16, wherein the processor device is further configured to: control the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the resulting detection of the inauthentic content from the multiple detection schemes; and/or control the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
19. The system of claim 17, wherein the processor device is further configured to: control the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the resulting detection of the inauthentic content from the multiple detection schemes; and/or control the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
20. The system of any of the claims 14-19, wherein the processor device is further configured to: use the provided quantitative and/or descriptive output to affect the behavior of the transformation of the content, and/or the injection of any inauthentic content and/or the preprocessing of the plurality of the resulting content, and/or detection of any inauthentic content and/or the controlling of the overall preprocessed information, and/or the fusion of any result of the detected inauthentic content.
21. A learning-based system of detecting inauthentic content, comprising: a processor device operatively coupled to a memory device, the processor device being configured to: detect inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes; fuse the resulting detection of the inauthentic content from the multiple detection schemes; and provide quantitative and/or descriptive output from the fused resulting detection.
22. The system of claim 21, wherein the processor device is further configured to: preprocess the content by classifying the content; and further preprocess the content by extracting context from the classified content using the metadata, and if metadata is available, additionally using the metadata to extract context.
23. The system of claim 21, wherein the processor device is further configured to: preprocess the content by classifying the content using an iterative process and/or a series of classifiers; and further preprocess the content by extracting context from the classified content using the metadata, and if metadata is available, additionally using the metadata to extract context.
24. The system of claim 22, wherein the processor device is further configured to: control the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the detection of the inauthentic content from the multiple detection schemes; and control the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
25. The system of claim 23, wherein the processor device is further configured to: control the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the detection of the inauthentic content from the multiple detection schemes; and control the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
26. The system of any of the claims 21-25, wherein the processor device is further configured to: use the provided quantitative and/or descriptive output to affect the behavior of the transformation of the content, and/or the preprocessing of the content and the metadata, if metadata is available, and/or detection of any inauthentic content and/or the controlling of the overall preprocessed information, and/or the fusion of any result of the detected inauthentic content
27. A computer program product for detection of inauthentic content based on machine learning, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to perform the method comprising: receiving data for training and/or for operational mode; separating metadata, if available, from content of the received data; transforming the content and the metadata, if the metadata is available, resulting in a plurality of content of the same semantic substance and metadata, if the metadata is available, on at least one instance of the received data; injecting inauthentic content into the resulting plurality of the content and plurality of the metadata, if metadata is available; and outputting the processed plurality of the resulting content and metadata, if metadata is available.
28. The product as recited in claim 27, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: detecting inauthentic content using the plurality of the resulting content and the metadata, if the metadata is available, by incorporating multiple detection schemes; fusing the resulting detection of the inauthentic content from the multiple detection schemes; and providing quantitative and/or descriptive output from the fused resulting detection.
29. The product as recited in claim 28, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: preprocessing the plurality of the resulting content by classifying the plurality of the resulting content; and further preprocessing the plurality of the resulting content by extracting context from the classified plurality of the resulting content and if metadata is available, additionally using the metadata to extract context.
30. The product as recited in claim 28, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: classifying each of the plurality of the resulting content using an iterative process and/or a series of classifiers; and further preprocessing the plurality of the resulting content by extracting context from the classified plurality of the resulting content and if metadata is available, additionally using the metadata to extract context.
31. The product as recited in claim 29, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the resulting detection of the inauthentic content from the multiple detection schemes; and/or controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
32. The product as recited in claim 30, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the resulting detection of the inauthentic content from the multiple detection schemes; and/or controlling the overall preprocessed information from the plurality of the resulting content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
33. The product as recited in of any of the claims 27-32, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: using the provided quantitative and/or descriptive output to affect the behavior of the transformation of the content, and/or the injection of any inauthentic content and/or the preprocessing of the plurality of the resulting content, and/or detection of any inauthentic content and/or the controlling of the overall preprocessed information, and/or the fusion of any result of the detected inauthentic content.
34. A computer program product for detection of inauthentic content based on machine learning, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to perform the method comprising: detecting inauthentic content using content and metadata, if the metadata is available, by incorporating multiple detection schemes; fusing the resulting detection of the inauthentic content from the multiple detection schemes; and providing quantitative and/or descriptive output from the fused resulting detection.
35. The product as recited in claim 34, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: preprocessing the content by classifying the content; and further preprocessing the content by extracting context from the classified content using the metadata, and if metadata is available, additionally using the metadata to extract context.
36. The product as recited in claim 34, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: preprocessing the content by classifying the content using an iterative process and/or a series of classifiers; and further preprocessing the content by extracting context from the classified content using the metadata, and if metadata is available, additionally using the metadata to extract context.
37. The product as recited in claim 35, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the detection of the inauthentic content from the multiple detection schemes; and controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
38. The product as recited in claim 36, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the fusing of the detection of the inauthentic content from the multiple detection schemes; and controlling the overall preprocessed information from the content and the metadata, if the metadata is available, as an input to affect the behavior of the detection of the inauthentic content from the multiple detection schemes.
39. The product as recited in of any of the claims 34-38, wherein the program instructions are further executable by a computing device to cause the computing device to perform the method comprising: using the provided quantitative and/or descriptive output to affect the behavior of the transformation of the content, and/or the preprocessing of the content and the metadata, if metadata is available, and/or detection of any inauthentic content and/or the controlling of the overall preprocessed information, and/or the fusion of any result of the detected inauthentic content
PCT/IB2020/061470 2020-08-10 2020-12-03 Machine learning based system and method of detecting inauthentic content WO2022034371A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063063956P 2020-08-10 2020-08-10
US63/063,956 2020-08-10

Publications (1)

Publication Number Publication Date
WO2022034371A1 true WO2022034371A1 (en) 2022-02-17

Family

ID=73835639

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2020/061470 WO2022034371A1 (en) 2020-08-10 2020-12-03 Machine learning based system and method of detecting inauthentic content

Country Status (1)

Country Link
WO (1) WO2022034371A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150139485A1 (en) * 2013-11-15 2015-05-21 Facebook, Inc. Pose-aligned networks for deep attribute modeling
US20190197331A1 (en) * 2017-12-21 2019-06-27 Samsung Electronics Co., Ltd. Liveness test method and apparatus
EP3525133A2 (en) * 2018-01-22 2019-08-14 Samsung Electronics Co., Ltd. Apparatus and method with liveness verification

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150139485A1 (en) * 2013-11-15 2015-05-21 Facebook, Inc. Pose-aligned networks for deep attribute modeling
US20190197331A1 (en) * 2017-12-21 2019-06-27 Samsung Electronics Co., Ltd. Liveness test method and apparatus
EP3525133A2 (en) * 2018-01-22 2019-08-14 Samsung Electronics Co., Ltd. Apparatus and method with liveness verification

Similar Documents

Publication Publication Date Title
US20200250304A1 (en) Detecting adversarial examples
CN111709408B (en) Image authenticity detection method and device
KR102097905B1 (en) Apparatus and method for recognizing one or more objects in images
US9906704B2 (en) Managing crowd sourced photography in a wireless network
US20180129742A1 (en) Natural language object tracking
KR102008290B1 (en) Method and apparatus for detecting action of object in viedio stream
JP7007829B2 (en) Information processing equipment, information processing methods and programs
CN104246656A (en) Automatic detection of suggested video edits
US20200349414A1 (en) Systems and methods for neuronal networks for associative gestalt learning
US11436436B2 (en) Data augmentation system, data augmentation method, and information storage medium
CN109522970B (en) Image classification method, device and system
CN112149651B (en) Facial expression recognition method, device and equipment based on deep learning
CN112766366A (en) Training method for resisting generation network and image processing method and device thereof
CN114549369A (en) Data restoration method and device, computer and readable storage medium
CN113705294A (en) Image identification method and device based on artificial intelligence
CN113553954A (en) Method and apparatus for training behavior recognition model, device, medium, and program product
CN114973349A (en) Face image processing method and training method of face image processing model
CN111435457B (en) Method for classifying acquisitions acquired by sensors
WO2022034371A1 (en) Machine learning based system and method of detecting inauthentic content
US20220366254A1 (en) Image Processing Model Training Method and Apparatus
CN110675312A (en) Image data processing method, image data processing device, computer equipment and storage medium
US20220114717A1 (en) Distortion-based filtering for image classification
KR102430742B1 (en) Apparatus and method for developing space analysis model based on data augmentation
CN115019218A (en) Image processing method and processor
CN116778212A (en) Image processing method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20824651

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20824651

Country of ref document: EP

Kind code of ref document: A1