TW202312105A - On-device image authentication - Google Patents

Abstract

Methods, systems, and apparatus, for on-device image authentication. One of the apparatus includes an image sensor configured to capture raw image data; a processing pipeline having one or more processing elements configured to generate processed image data from the raw image data captured by the image sensor, wherein the processed image data includes an encoded image generated by an encoder; a decoder configured to generate a decoded frame from the processed image data generated from the processing pipeline; and an image authentication module configured to generate an output in an authenticated image format, the output including the processed image data and a representation of a delta between the decoded frame of the processed image data and the raw image data.

Description

On-device image verification

This specification is about systems and devices for image verification.

Image verification is the application of trusted image encoding and decoding to determine whether an image in question is an accurate representation of the raw data captured by an imaging sensor (eg, a camera). Image authentication can be applied to a device or a system that receives and displays an image, such as a camera application in a mobile device, a property monitoring and security device, any social media application that displays an image, etc.

Image verification plays an important role in risk identification and fraud detection for various use cases. For example, when an image of a damaged car is sent to an insurance company during an insurance claim, the insurance company may need to determine whether the image of the damaged car is real. As another example, a bank needs to determine whether an image of a deposit check has been tampered with. A government may wish to know whether images circulated on social media are fake images spreading fake news. A user of a social media platform may wish to know whether an image is real or overly manipulated.

Some image verification techniques involve generating a verification image in a specialized image format that primarily focuses on proving that the original image data was captured by the camera sensor. The generated authentication image may comprise: (i) the original image data; (ii) a digital signature for a hash of the original image data; (iii) a processed image of the original image data in a compressed image format; and (iv) A digital signature for a hash of the processed image used for the original image data. An example of a proprietary image format includes the digital negative image format (DNG). Watermarking can also be used to establish the source of the image. For example, an image may contain a watermark identifying the creator or owner of the image.

One disadvantage of conventional image verification is that it can be spoofed. For example, if an image has two separate signatures for an original image and a processed image, one of the images and its corresponding signature can be maliciously modified or replaced.

In addition, a conventional verification image can have a large file size because all original data can be included in the verification image except for the processed version of the image data.

This specification describes techniques for on-device image authentication. The on-device image verification techniques described below use an output generated in a verified image format that includes a post-processed image and additional information that allows an image to represent a previous version of computer-reconstructed image data. The additional information may include a representation of a delta between a decoded frame of the processed image data and a previous version of the image data before processing was applied. The delta represents the difference between the previous version of the image and the decoded frame, and can be used by the image rendering computer to reconstruct the previous version of the image. In this specification, the previous version of the image data used to calculate the difference will be referred to as the original image data. Raw image data can be, but need not be, the first sensor data acquired by a camera before any processing takes place. Instead, raw image data means data from a previous stage of an image processing pipeline that can be used to compute a delta. Additionally, the term image data means any suitable data that can be used to generate an image representation, and thus includes actual image formats as well as other binary information or intermediate representations that can be used for the same purpose.

The subject matter described in this specification can be implemented in particular embodiments to realize one or more of the following advantages. Instead of generating two individual signatures for the raw image data and the processed image data in a conventional image verification format, a verified image is generated based on the entire content (i.e., both the processed image data and the representation of the delta) A digital signature of the format to allow verification of the output produced in the authenticated image format. That is, joint signing ensures that verification is based on everything in the output produced. The proven image format results in a smaller file size because one representation of a delta between one of the decoded frames of the processed image data and the original image data can be saved as part of the resulting output rather than the entire original image material. Thus, the authenticated image format reduces the costs associated with storing and transmitting the resulting output in the authenticated image format as compared to conventional image authentication formats.

After receiving a request to generate raw image data from input data in a validated image format, an image rendering computer may reconstruct the original image data from a representation of the difference between the decoded frames of the processed image data and the original image data. video material. The image presentation computer can present the processed image data and the original image through a user interface such that a user can compare the reconstructed original image data with the processed image data and the user can determine that the input data is in a validated image format Whether it is authentic, for example, whether the processed image data is a true representation of the original image data. In some embodiments, the image rendering computer or another computer can receive as input the processed image data and the original image, and can use, for example, a machine learning algorithm to generate a signal indicating whether the processed image data is a true representation of the original image data. - Likelihood score.

Furthermore, for some use cases, the raw image data obtained directly from the sensor may not be the best data for image verification. In many use cases, post-processing of the raw image data can help improve the image quality of the raw image data without compromising the realism of the image. For example, image enhancement (eg, denoising, motion correction, etc.) can help improve image quality. In some cases, multiple raw images may be used to create a processed image profile, such as high dynamic range (HDR) imaging, low light imaging, and the like. Therefore, the final post-processed image can be related to multiple original images instead of a single original image. Such post-processed images can still be considered real images even though they are different from the original image data. The authenticated image format may allow authentication of processed images generated from a single raw image or a sequence of multiple raw images.

Additionally, the authenticated image format may include a digital certificate comprising a public key of a public-private key pair of the device that generated the input data in the authenticated image format. The image presentation computer can verify that the device generating the input data is authentic by validating the public key in the digital certificate (eg, by validating the signature in the certificate using a certificate authority (CA) public key). In some embodiments, the image rendering computer can retrieve a device public key stored in a trusted server, and can use the retrieved device public key to verify the digital signature. This ensures that an untrusted device cannot generate an unauthorized public-private key pair, and cannot assert that an unauthorized public-private key pair corresponds to another trusted device.

The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims.

FIG. 1 is a diagram of an example system 100 for generating an output in a validated image format.

System 100 includes a sensor 102 , an image processing pipeline 106 , a decoder 120 , a differential encoder 124 and a verified image format generator 128 .

The sensor 102 can capture raw image data 104 . The sensor 102 can be a camera of the system 100, or any other kind of image capture sensor. The original image data 104 may include an original image or a plurality of original images at a sequence frame captured within a period of time. The original image data 104 may include a color image or a grayscale image. The raw image data 104 is not processed by any image post-processing method.

The system 100 can use an image processing pipeline 106 to perform a post-processing on the raw image data 104 . Image processing pipeline 106 may include a sequence of image processing steps and may produce processed image data 108 with improved image quality and or reduced file size. Each image processing step may use one or more image processing algorithms. In some embodiments, the image processing pipeline 106 may include steps, such as noise removal, motion correction, contrast enhancement, etc., that may produce an enhanced image that improves the appearance of the original image data. In some embodiments, the image processing pipeline 106 may include a step of generating an enhanced image from multiple original images at multiple frames, such as HDR/HDR+ imaging, low light imaging, and the like. In some embodiments, the image processing pipeline 106 may include a step of aligning multiple raw images at multiple frames and merging the multiple raw images into a single image, and the single image may be passed through the image processing pipeline 106 Additional steps further enhance and compress.

In some implementations, image processing pipeline 106 may include an encoding step that compresses raw image data 104 or enhances the image and produces processed image data 108 in a compressed image format (eg, JPEG, PNG, TIFF, etc.). Processed image data 108 may include a processed image generated from raw image data 104 or processed data in other formats. Processed image data 108 in a compressed image format may have a file size that is smaller than one of the original images in original image data 104 .

Some image processing pipelines 106 may generate processed image data 108 that is a true representation of original image data 104 . For example, an image processing pipeline 106 can use one or more of low light enhancement, HDR imaging, denoising, deblurring, etc. to generate the processed image data 108, and the processed image data 108 can still be regarded as the original image data One of 104 true representations.

However, image processing pipeline 106 may produce processed image data 108 that would not be considered a true representation of the original image. The image processing pipeline 106 can, for example, change the shape, size, position or texture of an object in the image, and/or can add or remove an object in the image. For example, a fake image may be generated by a mobile application that excessively modifies the content of the image, eg, a fake image of a car with modifications to a damaged portion of a car to be submitted to an insurance company. As another example, an image can be edited by a computer software such that the processed image contains one or more objects that were not part of the original image, for example, a fake image of a check for deposit at a bank with an altered bank account number .

The decoder 120 can decode the processed image data 108 and generate a decoded frame 122 . Decoder 120 may be deterministic and specific. Subsequently, the same decoder can be used in an image rendering computer to reconstruct the original image from the verified image. The decoded frame 122 includes image data in a format equivalent to the original image data 104 . For example, the decoded frame 122 may be an uncompressed video format. In some implementations, decoder 120 may be a JPEG image decoder that can generate a decoded image from a JPEG encoded image.

The differential encoder 124 may take as input the decoded frame 122 of the processed image data and the original image data 104 and may generate a difference 126 between the decoded frame 122 of the processed image data 108 and the original image data 104 One said. The differential encoder 124 may compute a difference (eg, a differential image) by subtracting the decoded frame 122 from one of the original images in the original image data 104 . The differential encoder 124 may generate a representation of the difference 126 based on a compression algorithm that measures differences between image frames. For example, the compression algorithm may be similar to a video compression algorithm that measures and compresses the differences between image frames in a video, such as MPEG-2, MPEG-4 part 10/H.264 or H.265 algorithms . The video compression algorithm may be adapted to measure and compress the difference between the decoded frame 122 and the original image data 104 .

In some implementations, when the raw image data 104 includes two or more raw image frames, the differential encoder 124 may compare the decoded frame with a selected raw image from one of the plurality of raw image frames. In some implementations, the differential encoder 124 may select an original image corresponding to the same time point as or a closest one to the time point of the decoded frame 122 .

The verified image format generator 128 may combine the processed image data 108 and the representation of the difference 126 between the decoded frames and the original image data into a single output image file 130 in a verified image format. Unlike conventional image verification methods that save the original image data 104 into an output image file, the system 100 can reduce the difference 126 between the decoded frame 122 of the processed image data 108 and the original image data 104 by a small Multiple compressed representations are included in the output image file 130 . Accordingly, the output image file 130 in the authenticated image format may have a size smaller than that of image data generated using a conventional image authentication method.

Upon request, an image rendering device may reconstruct the original image data from the output image file 130 received by the image rendering device. The reconstructed raw image data may be compared to the processed image data 108 contained in the output image file 130 in order to determine whether the processed image data 108 is a true representation of the reconstructed raw image data. More details on reconstructing and rendering an image in a validated image format are described below in conjunction with FIG. 3 .

In some implementations, the output image file may include a digital signature based on either the processed image data 108 and a representation of the difference 122 between the decoded frame 122 and the original image data 104 . The joint signature may be based on the entire content in the authenticated image format, ie on both the processed image data 108 and the representation of the delta 122, and may ensure the authenticity of the output image file 130. For example, a fraudulent modification can be applied to a digital signature that is part of a conventional verified image format without being recognized because the digital signature of a conventional verified image format is based only on the original image data or processed video material. Conversely, a fraudulent modification cannot be applied to a digital signature of a portion of the authenticated image format because the digital signature of the authenticated image format is based on the entire content in the authenticated image format. That is, joint signing ensures that everything in the generated output 130 is authentic.

2 is a diagram of an example computing device 200 that can generate a camera image in a validated image format. Computing device 200 includes a system-on-chip (SoC) 204 that can generate a verified image file (eg, output image file 130 in a verified image format). That is, generating the verification image file can be executed on the device through an SoC 204 . Figure 2 and its description use a SoC as an example device. However, computing device 200 may be any general-purpose processing device.

SoC 204 includes a camera control 208 that interacts with a camera sensor 202 of computing device 200 . The camera control unit 208 is connected to the central processing unit (CPU) 222 of the SoC through an SoC bus 234 . Camera controls 208 may communicate with camera driver 216 on CPU 222 . In some implementations, the SoC 204 can include a camera firmware 206 that can communicate with a camera driver 216 through a camera control 208 .

SoC 204 includes a ML accelerator 212 . ML accelerator 212 is connected to CPU 222 through SoC bus 234 . An ML driver 218 may perform post-processing on an image captured by the camera sensor 202 . The ML driver 218 may communicate with the ML accelerator 212 during an image post-processing procedure in order to accelerate one or more steps in the post-processing pipeline 106 . In some implementations, SoC 204 may include ML accelerator firmware 210 that may communicate with ML driver 218 through ML accelerator 212 .

SoC 204 includes a cryptographic engine 214, which may be a software or hardware module capable of generating a digital signature for a hash of data of interest. A cryptographic driver (cryptographic driver/crypto driver) 220 can communicate with the cryptographic engine 214 through the SoC bus 234 . The cryptographic driver 220 may control the cryptographic engine 214 to generate a digital signature of the entire content of the data in the authenticated image format, i.e., the processed image data 108 and the decoded frames 122 of the processed image data 108 and the original image data 104 A representation of the difference between 126.

The cryptographic engine 214 can access the hardware key 228 through the SoC bus 234 . The hardware key 228 may include a public-private key pair associated with the computing device 200 . The public-private key pair can indicate whether the computing device 200 is trusted. Cryptographic engine 214 may use a private key of hardware key 228 of computing device 200 to generate a digital signature of the representation of processed image data 108 and delta 126 . For example, cryptographic engine 214 may generate a digital signature by encrypting a hash value of the representation of processed image data 108 and difference 126 using a private key of hardware key 228 .

The SoC 204 includes a memory controller 224 that controls a memory device 230 in the computing device 200 . The memory controller 224 is connected to the CPU 222 through the SoC bus 234 . Memory device 230 may include both volatile and non-volatile memory, such as random access memory (RAM) and flash RAM. For example, after the camera sensor 202 captures the original image data 104 , the original image data 104 can be stored in a shared buffer in the memory device 230 . The CPU can then access the raw image data 104 stored in the memory device 230 and perform post-processing on the raw image data 104 using the ML accelerator 212 . A representation of processed image data 108 and delta 126 may also be stored in a shared buffer in memory device 230 controlled by memory control 224 .

The SoC 204 includes a storage controller 226 that controls the storage device 232 in the computing device 200 . The storage device 232 may include, for example, one or more of a hard disk device, an optical disk device, a solid-state memory device, and the like. The storage controller 226 is connected to the CPU 222 through the SoC bus 234 . After SoC 204 generates output image file 130 in a validated image format, output image file 130 may be saved to storage device 232 controlled by storage control 226 .

The following is an example procedure for generating a verified image file on the computing device 200 .

A camera application of the computing device 200 can trigger the camera sensor 202 to capture an image. The camera firmware 206 can communicate with the camera driver 216 on the CPU 222 to save the raw image data 104 to a shared buffer in the memory device 230 .

The camera application can invoke an image post-processing component in the SoC 204 to perform post-processing on the raw image data to generate processed image data, for example, use HDR+ post-processing to generate a processed image. For example, a camera application may invoke ML driver 218 on CPU 222 to perform post-processing on raw image data 104 to generate processed image data 108 . The CPU 222 can compare the processed image data 108 with the original image data 104 to generate a difference, ie, the difference between the decoded frame 122 of the processed image data 108 and the original image data 104 . In some implementations, the ML driver 218 can work with an ML accelerator firmware 210 to execute the image post-processing pipeline and generate the delta between two images.

The CPU 222 may perform a differential encoding using a differential encoder 209 to generate a compressed delta of the difference between the decoded frames of the original image data and the post-processed image data. For example, differential encoding may be entropy encoding, Huffman encoding, run-length encoding, and/or any other suitable technique. After generating the compression delta, the CPU 222 can attach the compression delta to the processed image data.

The camera application may invoke the cryptographic driver 220 to control the cryptographic engine 214 to generate a digital signature representing the processed image data 108 and a difference 126 between the decoded frame 122 of the processed image data 108 and the original image data 104. chapter. In some embodiments, the cryptographic engine 214 can access the hardware key 228 of the computing device, and can use the private key of the hardware key 228 to generate a digital signature.

The camera application can generate an output image file 130 in one of the validated image formats. Output image file 130 may include a representation of processed image data 108 and delta 126 . The output image file 130 may further include a digital signature generated based on the processed image data 108 and the representation of the delta 126 . The camera application can save the output image file 130 to the storage device 232 .

FIG. 3 is a diagram of an example system 300 for presenting an image in a validated image format. A program for rendering images in a validated image format may be implemented in an operating system of an image rendering computer, may be implemented as part of a standalone image viewing or editing application on a computing device, or may be implemented as part of web browser software implementation etc.

An image rendering computer 304 may receive input data 302 in a validated image format. The image presentation computer 304 may be a different computer than the computing device that generated the input data 302 . As depicted in FIG. 1 , the input data 302 in a verified image format may be the output image file 130 generated by an on-device image verification process. The input data 302 may include the processed image data 108 and a representation of a delta 126 between a decoded frame of the processed image data and the original image data.

Image rendering computer 304 may receive, for example, a request 306 from a user 316 of image rendering computer 304 to view an original image included in input data 302 .

For example, a vehicle owner may use a mobile device to capture an image of a damaged vehicle. Since the images are captured at night, the mobile device can perform low light correction and JPEG encoding, and can produce a processed encoded image of one of the original images of the damaged vehicle. The mobile device can generate an image file in a validated image format containing a low-light corrected processed and encoded image of the damaged vehicle and a representation of the difference between the processed image data and the original image captured by the camera . The vehicle owner may upload the image file in the authenticated image format to an insurance company's computer during a procedure of claiming damage to the vehicle. The insurance company's computer (ie, the image rendering computer 304) can receive the uploaded image file in a validated image format. An insurance agent 316 may send a request to the computer to view the original image contained in the uploaded image file in the authenticated image format. For example, insurance agent 316 may wish to visually compare the original and processed images to determine whether the post-processed image is a true representation of a damaged vehicle. In some embodiments, insurance agent 316 may send a request to the computer and instruct the computer to validate the digital signature contained in the uploaded image file.

After receiving a request 306 to view a raw image, the image rendering computer 304 may generate a processed image 308 and a raw image 310 from the input data 302 in a validated image format. Image rendering computer 304 may generate processed image 308 by reading the processed image contained in input data 302 . Image rendering computer 304 may generate a decoded image of processed image 308 by decoding the processed image using a decoder (eg, the same decoder 120 during generation of input data 302). Image rendering computer 304 may generate raw image 310 by combining the decoded image with a representation of delta 126 contained in input data 302 .

For example, a computer of an insurance company may use a JPEG decoder that decodes a JPEG image to generate a processed image of a damaged vehicle, eg, a JPEG encoded image of a damaged vehicle. The computer may generate an original image corresponding to the processed image by combining the representation of the delta 126 included in the input data 302 with the decoded image of the processed image 308 .

The image presentation computer 304 may include a display device 314 , such as a monitor of a desktop computer, a screen of a mobile device, a VR and/or an AR device, and the like. In some implementations, as shown in FIG. 3 , the image presentation computer 304 can present the processed image 308 and the original image 310 side by side on a display device 314 . In some implementations not currently shown in FIG. 3 , image rendering computer 304 may present a user interface element that enables a user 316 to switch between processed image 308 and original image 310 . In some implementations not presently shown in FIG. 3 , image rendering computer 304 may render a user interface element that displays a difference (eg, a differential image) between processed image 308 and original image 310 . For example, the difference between two images can be calculated by finding the difference between the pixels in the images and generating an image based on the difference between the pixels in the images.

In some embodiments, image rendering computer 304 may receive a request 306 to validate a digital signature included in input data 302 . In some embodiments, the image rendering computer 304 can verify the digital signature before generating the original image from the input data 302 . If the verification of the digital signature is successful, the image rendering computer 304 can proceed to reconstruct the original image from the input data 302 in a verified image format, and the image rendering computer 304 can notify the user 316 that the computing device that generated the input data 302 is A trusted device. If the verification of the digital signature is unsuccessful, the image rendering computer 304 may not continue to reconstruct the original image, and the image rendering computer 304 may notify the user 316 that the computing device that generated the input data 302 is not a trusted device.

In some embodiments, the image presentation computer 304 can display a device identification 312 on a display device 314 . The device identification 312 may be included in a digital certificate along with the device public key. The digital certificate can be signed by a certificate authority (CA) private key, for example by a sub-CA at the device manufacturer. The device identification 312 may include an identification of the device that generated the input data 302, or a verification result of a digital signature, eg, "device verified" or "device not verified". A user 316 can view the digital signature verification results included in the device identification 312 and can determine whether the device that generated the input data 302 is authentic.

In some implementations, image rendering computer 104 may include a general-purpose image reader not dedicated to processing input data 302 in a validated image format, eg, a JPEG decoder. The image rendering computer 104 can still decode and render the processed image, for example, render the processed JPEG image as if it were just a normal JPEG image. In some implementations, image rendering computer 104 may discard the remainder of the input data 302, eg, discard representations of differences between decoded frames of the processed image and the original image.

In some embodiments, image rendering computer 104 may receive input data that is not in a validated image format, eg, a post-processed image in JPEG format. The image rendering computer 104 can still generate post-processed images, such as by executing a JPEG decoding process, and display the post-processed images on the display device 314 without generating an original image 310 . Thus, the image rendering computer 304 is backward compatible to display input data that is not in one of the validated image formats.

Figure 4 is a flowchart of an example procedure for generating an output in a validated image format. For convenience, the process will be described as being performed by a system (eg, system 100 in FIG. 1 ) for generating an output image in an authenticated image format. The system may include components of a computing device 200 described with reference to FIG. 2, including one or more camera sensors, one or more SoCs, one or more memory devices, and one or more storage devices, or a combination thereof a certain combination.

The system captures raw image data through an image sensor (402). In some embodiments, the raw image data may include a single image of a single raw frame captured by a camera sensor. In some embodiments, the raw image data may include a plurality of images of a sequence of raw frames captured by a camera sensor. The system can perform an alignment and merging process to generate a single image as an original image representing what was captured by the camera.

The system generates processed image data from new image data captured by the image sensor by a processing pipeline having one or more processing elements (404). The processing pipeline may include one or more image processing routines or algorithms that produce processed image data with improved image quality. Processed image data may comprise a processed image or processed data in other formats generated from raw image data. In some implementations, the processed image may be in a compressed image format, eg, JPEG, PNG, TIFF, or another compressed image format.

The system generates a decoded frame from the processed image data generated by the processing pipeline (406). In some implementations, the decoded frame may be in an uncompressed video format. For example, when the processed image is in JPEG format, the system may use a JPEG image decoder to generate a decoded frame from the processed image.

The system generates an output in one of the validated image formats (408). A representation comprising the processed image data and a delta between the decoded frames of the processed image data and the original image data is output. That is, instead of saving the original image data, the system generates the difference between the decoded frames of the original image data and the processed image data. The system then decodes the differences in the output image in the authenticated image format so that an image reader can then correctly reconstruct the original image data upon request.

The system can reliably generate and encode raw image frames regardless of the number and type of processing steps in the processing pipeline used to generate the processed image data. An image reader configured to process the output file in the validated image format can then reconstruct and display the original image frame.

In some implementations, a representation of the difference between the decoded frame of the processed image data and the original image data may be generated based on a compression algorithm that measures the difference between the image frames. The compression algorithm efficiently handles differences between image frames, eg, differences with a large number of zeros. Differences between two image frames can be efficiently compressed when the two image frames are sufficiently similar to each other. For example, the system may generate output using a video encoding algorithm that encodes the differences between frames in a video, such as a video MPEG encoder.

In some implementations, the output may include a digital signature based on one of the processed image data and a representation of the difference between the decoded frame of the processed image data and the original image data. That is, the system can sign and verify both the processed and encoded frames and the difference information (ie, the difference between the decoded frames of the processed image data and the original image data) in the same output file.

In some embodiments, the system may generate a hash value based on the processed image data and a representation of the difference between the decoded frame of the processed image data and the original image data, and the system may generate a digital signature based on the hash value chapter. A hash value is a value of fixed length that uniquely identifies data. A hash value can represent a large amount of data into a much smaller value.

In some embodiments, the system can generate the digital signature by encrypting the hash value with a private key of a public-private key pair of the device. The system can access a hardware key associated with a computing device of the system, eg, hardware key 228 of computing device 200 in FIG. 2 . The public-private key pair can indicate whether the computing device is trusted. An image rendering computer can then obtain one of the public keys of the private key pair associated with the device and can verify the digital signature in the output file in authenticated image format in order to determine whether the device that generated the output file is authentic .

In some implementations, the output may comprise a digital certificate comprising a public key of a public-private key pair of the device. In some implementations, the output may further include a device identifier, expiration date, and a digital signature generated by a CA or a sub-CA. An image rendering computer can then use the public key to validate the digital certificate in the output file in the authenticated image format in order to determine whether the device generating the output file is authentic.

5 is a flowchart of an example procedure for reconstructing raw image data from input data in a validated image format. For convenience, the procedure will be described as being executed by a system for image presentation (eg, system 300 in FIG. 3 ). The system can include an image presentation computer or device that can be configured to parse input data in a validated image format and reconstruct original image data from the input data in a validated image format.

The system receives input data in a validated image format (502). The input data includes processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data. Processed image data may comprise a processed image or processed data in other formats generated from raw image data. For example, the input data may include a post-processed image in JPEG format and a delta between a decoded frame of the post-processed image and an original image captured by a camera sensor.

The system receives a request to generate one of raw image data from the input data (504). For example, the system may receive a request 306 from a user of the image presentation system to display the original image 310 and the processed image 308 so that the user can confirm the authenticity of the input data 302 .

The system reconstructs the original image data from the representation of the difference between the decoded frames of the processed image data and the original image data (506). In some implementations, reconstructing the original image data from a representation of the difference between the decoded frame of the processed image data and the original image data may include combining the processed image data with the decoded frame of the processed image data by The representation of the difference with the original image data to generate the original image data.

In some implementations, input data in a validated image format may include representations based on total content (i.e., both the processed image data and the difference between decoded frames of the processed image data and the original image data) Generate a digital signature. A digital signature is generated on all content to ensure the authenticity of the entered data. That is, a digital signature ensures that the representation of the processed image data and deltas cannot be modified after the digital signature has been computed.

In some embodiments, the digital signature included in the input data may be generated by: (i) a representation based on the processed image data and the difference between the decoded frames of the processed image data and the original image data generating a hash value; and (ii) encrypting the hash value using a private key of a public-private key pair associated with a device that generated the input data in the authenticated image format. In some implementations, the public-private key pair may be a hardware key associated with the device that generated the input data, eg, the hardware key 228 of the computing device 200 . A digital signature can indicate whether input data was generated by a trusted device authorized to access the private key. In some embodiments, the system can obtain the public key of one of the public-private key pair associated with the device that generated the input in the authenticated image format, and the system can use the public key to validate the digital signature in the input data .

In some implementations, the input data in authenticated image format can include a digital certificate that can be used to confirm whether the input data was generated by a trusted device. The system may validate the digital certificate in the input data before validating the digital signature in the input data in the authenticated image format, for example, confirming that the input data was generated by a trusted device. Although any device can provide a verified image with a valid signature, only a legitimate device can have its device public key authenticated, eg, by including the device public key in a valid digital certificate. An invalid device manufacturer cannot have its public key authenticated eg by a CA or a security server.

In response to a successful validation of one of the digital certificates, the system may extract the device public key from the digital certificate. The system can then proceed to confirm the validity of the input data in the authenticated image format by validating the digital signature using the device public key. If the input data is valid, the system can reconstruct the original image data from the input data in the verified image format, and display the reconstructed original image to the user, so that the user can determine whether the processed image is one of the reconstructed original images Accurate representation. In some embodiments, the system can provide the reconstructed original image and the processed image to a computer-implemented algorithm (e.g., a machine learning algorithm), and the algorithm can generate the difference between the reconstructed original image and the processed image. a matching score, and the matching score may represent a likelihood that the processed image is an accurate representation of one of the reconstructed original images.

In some implementations, the device public key may be provided in a digital certificate. The digital certificate can be authenticated by a certification authority (CA) and can be included in the input data in a certified image format. Digital credentials can be generated external to the device and can be deployed in the device at the time of manufacture. The device public key associated with one of the device's private keys can be validated by the CA. The system can validate digital certificates contained in input data using a CA root public key in order to ensure the authenticity of the device generating the input data in authenticated image format. For example, the system can obtain the digital certificate by accessing the metadata included in the input data, and the system can use a CA root public key to validate the digital certificate.

In some embodiments, the device public key may be stored in a trusted server or a trusted database accessible by the system, and the system may obtain the device public key stored in the trusted server, and may Use the device public key to confirm the digital signature in the input data. For example, the system can retrieve the device public key stored in a trusted server based on an identification (ID) provided in the data after inputting the data. The system can verify the digital signature using the device's public key retrieved from a trusted server.

In some implementations, the system can present the processed image data and the reconstructed original image data side-by-side on one of the system's display devices. In some implementations, the system may present a user interface element that toggles between processed image data and reconstructed raw image data. For example, a user of the system may use the user interface element to switch between processed images and reconstructed original images.

In some embodiments, the system or another system may take as input the processed image data and the reconstructed original image data and generate a prediction result indicating whether the processed image data is a true representation of the reconstructed original image data. In some embodiments, the system or another system may use a trained machine learning model to generate a prediction indicating the likelihood that the processed image data is a true representation of the reconstructed original image data.

Embodiments of the subject matter and actions and operations described in this specification can be implemented in digital electronic circuits, tangibly embodied computer software or firmware, computer hardware (including the structures disclosed in this specification and their structural equivalents) matter), or a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, a collection of computer program instructions encoded on a tangible, non-transitory storage medium for execution by, or to control the operation of, data processing equipment. One or more mods. Alternatively or in addition, program instructions may be encoded on an artificially generated propagated signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information for transmission to suitable receiver equipment to be executed by a data processing device. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more of them, or a part thereof. A computer storage medium is not a propagated signal.

A computer program (which may also be called or described as a program, software, a software application, an application, a module, a software module, an engine, a script, or code) may be in any form Programming languages (including compiled or interpreted languages or declarative or procedural languages) that can be deployed in any form, including as a stand-alone program or as a module, component, engine, subroutine, or suitable for use in a computing Other units that execute in an environment that may include one or more components interconnected by a data communication network in one or more locations.

A computer program may (but need not) correspond to a file in a file system. A computer program may be stored in a portion of a file that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or Stored in multiple collaborative files (for example, a file that stores one or more modules, subroutines, or portions of code).

To provide for interaction with a user, embodiments of the subject matter described in this specification may have a display device (e.g., an LCD (liquid crystal display) monitor for displaying information to the user) and an input A device by which a user can provide input to the computer, such as a keyboard and a pointing device (eg, a mouse, a trackball, or touch pad), implemented on a computer or configured to communicate with it. Other types of devices can also be used to provide interaction with a user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user can be any Formal reception, including acoustic, speech or tactile input. In addition, a computer may send files to and receive files from a device for use by a user (for example, by sending a web page in response to a request received from a web browser on a user's device) sent to the web browser, or interact with the user by interacting with an application running on a user device (eg, a smartphone or electronic tablet). Also, a computer can interact with a user by sending text messages or other forms of information to a personal device (eg, a smartphone running a messaging application) and then receiving response messages from the user.

Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a backend component (eg, as a data server), or that includes a middleware component (eg, as an application server ), or include a front-end component (e.g., a client device having a graphical user interface, a web browser, or an application program through which a user can implement one of the subject matter described in this specification Program Interaction) or any combination of one or more back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include a local area network (LAN) and a wide area network (WAN) (eg, the Internet).

The computing system may include clients and servers. A client and server are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML web page) to a user device, for example, for displaying the data to a user interacting with the device (which acts as a client) and receives User input from this user. Data generated at a user device (eg, as a result of a user interaction) may be received at a server from the device.

In addition to the embodiments described above, the following embodiments are also innovative:

Embodiment 1 is a kind of device, and it comprises: an image sensor configured to capture raw image data; A processing pipeline having one or more processing elements configured to generate processed image data from the raw image data captured by the image sensor, wherein the processed image data includes data generated by an encoder an encoded image; a decoder configured to generate a decoded frame from the processed image data produced by the processing pipeline; and An image verification module configured to generate an output in a verified image format, the output comprising the processed image data and a distance between the decoded frame of the processed image data and the original image data One of the deltas is indicated.

Embodiment 2 is the device of embodiment 1, wherein the original image data can be reconstructed based on the difference and the processed image data.

Embodiment 3 is the device of embodiment 1, wherein the output comprises one of the representation based on the processed image data and the difference between the decoded frame of the processed image data and the original image data Digital signature.

Embodiment 4 is the device of embodiment 3, wherein generating the digital signature includes: generating a hash value based on the processed image data and the representation of the difference between the decoded frame of the processed image data and the original image data; and The digital signature is generated based on the hash value.

Embodiment 5 is the device of any one of embodiments 3-4, wherein generating the digital signature comprises encrypting the hash value with a private key of a public-private key pair of the device.

Embodiment 6 is the device of embodiment 3, wherein the output includes a digital certificate, wherein the digital certificate includes a public key of a public-private key pair of the device.

Embodiment 7 is the device of embodiment 1, wherein the encoded image is in a compressed image format.

Embodiment 8 is the device of embodiment 7, wherein the decoded frame is in an uncompressed video format.

Embodiment 9 is the device of embodiment 1, wherein the difference between the decoded frame of the processed image data and the original image data is generated based on a compression algorithm that measures the difference between image frames It should be expressed.

Embodiment 10 is a computer-implemented method, which includes: Capture raw image data by an image sensor; generating processed image data from the raw image data captured by the image sensor by a processing pipeline having one or more processing elements, wherein the processed image data includes an encoded image generated by an encoder; generating a decoded frame from the processed image data generated by the processing pipeline; and An output in a validated image format is generated, the output including the processed image data and a representation of a delta between the decoded frame of the processed image data and the original image data.

Embodiment 11 is one or more non-transitory storage media encoded with instructions that, when executed by a computing device, cause the computing device to perform operations comprising: Capture raw image data by an image sensor; generating processed image data from the raw image data captured by the image sensor by a processing pipeline having one or more processing elements, wherein the processed image data includes an encoded image generated by an encoder; generating a decoded frame from the processed image data generated by the processing pipeline; and An output in a verified image format is generated, the output including the processed image data and a representation of a delta between the decoded frame of the processed image and the original image data.

Embodiment 12 is a computer-implemented method comprising: receiving, by an image rendering computer, input data in an authenticated image format, the input data comprising processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data; receiving a request to generate the raw image data from the input data; and The original image data is reconstructed from the representation of the difference between the decoded frame of the processed image data and the original image data.

Embodiment 13 is the method of embodiment 12, wherein reconstructing the original image data from the representation of the difference between the decoded frame of the processed image data and the original image data comprises combining the processed image data by data and the representation of the difference between the decoded frame of the processed image data and the original image data to generate the original image data.

Embodiment 14 is the method of embodiment 12, wherein the input data in the validated image format comprises based on the processed image data and the difference between the decoded frame of the processed image data and the original image data The representation of the quantity both produces a digital signature, and wherein the method includes: confirming the digital signature in the input data before reconstructing the original image data from the input data in the authenticated image format; and Responsive to a successful confirmation of a digital signature, reconstructing the original image data from the input data in the authenticated image format.

Embodiment 15 is the method of embodiment 14, wherein a hash value is hashed by using a private key of a public-private key pair associated with a device that generated the input data in the authenticated image format encryption to generate the digital signature, and confirming that the digital signature in the input data includes: obtain a public key of the public-private key pair associated with the device that generated the input data in the authenticated image format; and Using the public key to confirm the digital signature in the input data.

Embodiment 16 is the method of embodiment 15, wherein the public key is certified by a certification authority (CA), the input data in the authenticated image format includes a digital certificate, wherein the public key is obtained and the public key is used The key confirms that the digital signature in the input data includes: The digital certificate included in the input data is validated by the image presentation computer using a CA root public key; in response to a successful confirmation, the public key included in the digital certificate is obtained by the image rendering computer; and The digital signature in the input data is confirmed by the image presentation computer using the public key.

Embodiment 17 is the method of embodiment 15, wherein the public key is stored in a trusted server accessible by the image rendering computer, wherein the public key is obtained and the input data is validated using the public key The digital signature includes: Obtaining the public key stored in the trusted server through the image presentation computer; and The digital signature in the input data is confirmed by the image presentation computer using the public key.

Embodiment 18 is the method of embodiment 12, which further comprises: The processed image data and the original image data are presented side by side.

Embodiment 19 is the method of embodiment 12, which further comprises: Presenting a user interface element for switching between the processed image data and the raw image data.

Embodiment 20 is the method of embodiment 12, which further comprises: A user interface element displaying a difference between the processed image data and the original image is presented.

Embodiment 21 is one or more non-transitory storage media encoded with instructions that, when executed by a computing device, cause the computing device to perform operations comprising: receiving input data in a validated image format, the input data comprising processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data; receiving a request to generate the raw image data from the input data; and The original image data is reconstructed from the representation of the difference between the decoded frame of the processed image data and the original image data.

Embodiment 22 is a system comprising one or more computers and one or more storage devices storing instructions operable when executed by the one or more computers to cause the one or more computers to perform Do the following: receiving input data in a validated image format, the input data comprising processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data; receiving a request to generate the raw image data from the input data; and The original image data is reconstructed from the representation of the difference between the decoded frame of the processed image data and the original image data.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what is being claimed or may be claimed, but rather as descriptions that may be specific to particular embodiments of the invention. feature. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as functioning in particular combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be exempt from that combination and claims may relate to A subgroup or a variation of a subgroup.

Similarly, while operations are depicted in the drawings and in the claims recited in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order, or that all The illustrated operations are performed to achieve the desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system modules and components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a In a single software product or packaged in multiple software products.

Certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following invention claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the procedures depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

100: system 102: Sensor 104:Original image data 106: Image processing pipeline 108: Processed image data 120: decoder 122: decoded frame 124: Differential encoder 126: difference 128:Proven Image Format Generator 130: Output image file 200: computing device 202: Camera sensor 204: System-on-a-chip (SoC) 206:Camera Firmware 208: Camera controls 209: Differential encoder 210:ML Accelerator Firmware 212:ML Accelerator 214: Cipher engine 216: Camera driver 218: ML driver 220: password driver 222: Central Processing Unit (CPU) 224: Memory control part 226: storage control part 228: Hardware key 230: memory device 232: storage device 234: System single chip (SoC) bus 300: system 302: Input data 304: Image Presentation Computer 306: request 308: Processed image 310: Original image 312: Device identification 314: display device 316: User/Insurance Agent 402: Capture raw image data by an image sensor 404: Generate processed image data from new image data captured by the image sensor by a processing pipeline having one or more processing elements 406: Generate a decoded frame from the processed image data generated by the processing pipeline 408: Generate output in one of the validated image formats 502: Receive input data in a validated image format 504: Receive a request to generate one of the original image data from the input data 506: Reconstructing the original image data from the representation of the difference between the decoded frame of the processed image data and the original image data

Figure 1 is a diagram of an example system for generating an output in a validated image format.

2 is a diagram of an example computing device.

3 is a diagram of an example system for rendering an image in a validated image format.

Figure 4 is a flowchart of an example procedure for generating an output in a validated image format.

5 is a flowchart of an example procedure for reconstructing raw image data from input data in a validated image format.

In the various drawings, like reference numerals and names indicate like components.

100: system

102: Sensor

104:Original image data

106: Image processing pipeline

108: Processed image data

120: decoder

122: decoded frame

124: Differential encoder

126: difference

128:Proven Image Format Generator

130: Output image file

Claims (22)

A device comprising: an image sensor configured to capture raw image data; A processing pipeline having one or more processing elements configured to generate processed image data from the raw image data captured by the image sensor, wherein the processed image data includes data generated by an encoder an encoded image; a decoder configured to generate a decoded frame from the processed image data produced by the processing pipeline; and An image verification module configured to generate an output in a verified image format, the output comprising the processed image data and a distance between the decoded frame of the processed image data and the original image data One of the deltas is indicated. The device according to claim 1, wherein the original image data can be reconstructed based on the difference and the processed image data. The apparatus of claim 1, wherein the output includes a digital signature based on both the processed image data and the representation of the difference between the decoded frame of the processed image data and the original image data . As the device of claim 3, wherein generating the digital signature includes: generating a hash value based on the processed image data and the representation of the difference between the decoded frame of the processed image data and the original image data; and The digital signature is generated based on the hash value. The device according to any one of claims 3 to 4, wherein generating the digital signature includes encrypting the hash value using a private key of a public-private key pair of the device. The device of claim 3, wherein the output includes a digital certificate, wherein the digital certificate includes a public key of a public-private key pair of the device. The device according to claim 1, wherein the encoded image is in a compressed image format. The device according to claim 7, wherein the decoded frame is in an uncompressed video format. The device of claim 1, wherein the representation of the difference between the decoded frame of the processed image data and the original image data is generated based on a compression algorithm that measures differences between image frames . A computer-implemented method comprising: Capture raw image data by an image sensor; generating processed image data from the raw image data captured by the image sensor by a processing pipeline having one or more processing elements, wherein the processed image data includes an encoded image generated by an encoder; generating a decoded frame from the processed image data generated by the processing pipeline; and An output in a validated image format is generated, the output including the processed image data and a representation of a delta between the decoded frame of the processed image data and the original image data. One or more non-transitory storage media encoded with instructions that, when executed by a computing device, cause the computing device to perform operations including: Capture raw image data by an image sensor; generating processed image data from the raw image data captured by the image sensor by a processing pipeline having one or more processing elements, wherein the processed image data includes an encoded image generated by an encoder; generating a decoded frame from the processed image data generated by the processing pipeline; and An output in a verified image format is generated, the output including the processed image data and a representation of a delta between the decoded frame of the processed image and the original image data. A computer-implemented method comprising: receiving, by an image rendering computer, input data in an authenticated image format, the input data comprising processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data; receiving a request to generate the raw image data from the input data; and The original image data is reconstructed from the representation of the difference between the decoded frame of the processed image data and the original image data. The method of claim 12, wherein reconstructing the original image data from the representation of the difference between the decoded frame of the processed image data and the original image data comprises combining the processed image data with the The representation of the difference between the decoded frame of image data and the original image data is processed to generate the original image data. The method of claim 12, wherein the input data in the validated image format includes the processed image data based on the processed image data and the difference between the decoded frame of the processed image data and the original image data Indicates that both generate a digital signature, and wherein the method includes: confirming the digital signature in the input data before reconstructing the original image data from the input data in the authenticated image format; and Responsive to a successful confirmation of a digital signature, reconstructing the original image data from the input data in the authenticated image format. The method of claim 14, wherein a hash value is generated by encrypting a hash value using a private key of a public-private key pair associated with a device that generated the input data in the authenticated image format The digital signature, and which confirms that the digital signature in the input data includes: obtain a public key of the public-private key pair associated with the device that generated the input data in the authenticated image format; and Using the public key to confirm the digital signature in the input data. The method of claim 15, wherein the public key is certified by a certification authority (CA), the input data in the verified image format includes a digital certificate, wherein the public key is obtained and the public key is used to confirm the The digital signature included in the input data includes: The digital certificate included in the input data is validated by the image presentation computer using a CA root public key; in response to a successful confirmation, the public key included in the digital certificate is obtained by the image rendering computer; and The digital signature in the input data is confirmed by the image presentation computer using the public key. The method of claim 15, wherein the public key is stored in a trusted server accessible by the image rendering computer, wherein the public key is obtained and the digit in the input data is confirmed using the public key Signatures include: Obtaining the public key stored in the trusted server through the image presentation computer; and The digital signature in the input data is confirmed by the image presentation computer using the public key. The method of claim 12, further comprising: The processed image data and the original image data are presented side by side. The method of claim 12, further comprising: Presenting a user interface element for switching between the processed image data and the raw image data. The method of claim 12, further comprising: A user interface element displaying a difference between the processed image data and the original image is presented. One or more non-transitory storage media encoded with instructions that, when executed by a computing device, cause the computing device to perform operations including: receiving input data in a validated image format, the input data comprising processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data; receiving a request to generate the raw image data from the input data; and The original image data is reconstructed from the representation of the difference between the decoded frame of the processed image data and the original image data. A system comprising one or more computers and one or more storage devices storing instructions operable when executed by the one or more computers to cause the one or more computers to perform operations including: receiving input data in a validated image format, the input data comprising processed image data and a representation of a difference between a decoded frame of the processed image data and the original image data; receiving a request to generate the raw image data from the input data; and The original image data is reconstructed from the representation of the difference between the decoded frame of the processed image data and the original image data.

Images