EP4695706A1

EP4695706A1 - Secure metadata chain for on-device provenance of a media file

Info

Publication number: EP4695706A1
Application number: EP24745559.5A
Authority: EP
Inventors: Shu-Han Yu; Chuan-wen LAI; Chihchi CHENG; Madhu Sudan ATHREYA
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2024-06-26
Filing date: 2024-06-26
Publication date: 2026-02-18
Also published as: WO2026005775A1

Abstract

Methods and systems, including computer-readable media, are described for implementing a secure process history chain for on-device provenance of a media file using a hardware integrated circuit, such as a system-on-chip ("SoC") of a mobile computing device. The system obtains a media file, such as an image or video stream, and processes the media file using a processor block of the hardware circuit. The system generates a key that indicates a type of processing operation performed on the media file by the processor block and stores the key in a secure memory of a security subsystem on the hardware circuit. The system then generates process history information based on the key. The process history information indicates processing operations that were performed on the media file.

Description

SECURE METADATA CHAIN FOR ON-DEVICE PROVENANCE OF A MEDIA FILE

BACKGROUND

[0001] This specification generally relates to a metadata for a media file.

[0002] Modem computing devices can incorporate a wide variety of compute processing units that each offer different computing capabilities and trade-offs. Efficient execution of a given compute job often involves parsing computations into meaningful sub-tasks or workloads that are mapped to available processor cores of a computing system. The computations may be parsed and mapped based on suitability criteria, such as processor capability’, performance, and power.

[0003] Generally, this overall process of allocating portions of a computation to appropriate processor resources is referred to as heterogeneous computing. At least one processor core of the computing system can be an Intellectual Property block (“IP block"’) that executes a respective portion of a computational operation for different multimedia use cases. An example use case can involve processing image data captured by’ a camera of a mobile device (e g., a smartphone). Using heterogeneous computations to process the image data requires synchronization among processor cores that receive respective allocations of sub-tasks.

[0004] In the context of machine-learning, neural networks can generate an output from a received input in accordance with current values of a respective set of parameters. Some neural networks can be convolutional neural netw orks (CNNs) configured for image processing or recurrent neural netw orks (RNNs) configured for speech and language processing. Other neural networks can have a transformer network architecture that is based on attention mechanisms, which could augment recurrent and convolutional networks. Different types of neural network architectures can be used to perform a variety of tasks related to data classification and image recognition, augmentation, and/or enhancement, as well as predictions that involve data modeling.

[0005] A machine-learning (ML) model that is based on a trained neural network can be stored and implemented locally to compute inferences on an edge, mobile, or client computing device. Implementing ML models locally on an edge device presents various security' concerns and challenges. For example, the model binary and sensitive data processed by the model can be leaked and accessed by unauthorized users. Inference and self-learning capabilities of local ML models can be exploited to tamper with the authenticity of a media file or otherw ise abused by adversaries that gain unauthorized access to the mobile/edge device storing the models.

SUMMARY

[0006] This specification describes a system that implements specific hardware and software techniques to securely generate provenance information for a media file generated by a processor (or IP) block of the system. A media file such as an image or video stream that is generated using a camera of the system may undergo processing, for example, by an image signal processor (ISP) or machine-learning (ML) hardware accelerator of the system. The processing operation performed on the media file may alter the overall provenance or authenticity of the media file. The disclosed techniques can be used to securely generate a sequence or grouping of process history information that reliably and accurately indicates whether a processing operation performed on the media file altered the provenance or authenticity of the media file.

[0007] One aspect of the subject matter described in this specification can be embodied in a method performed on a hardware integrated circuit (IC) of computing device. The method includes obtaining a media file, processing the media file using a processor block of the hardware IC, and generating a key that indicates a ty pe of processing operation performed on the media file by the processor block. The method further includes storing the key in a secure memory of a security’ subsystem on the hardware IC; and generating, based on the key. process history information that indicates one or more processing operations that were performed on the media file.

[0008] These and other implementations can each optionally include one or more of the following features. For example, in some implementations, the method includes storing a processed media content identifier (ID) in the secure memory of the hardware IC. The media content ID represents a measure of integrity or authenticity of the media file after the media file is processed using the processor block. In some implementations, the method includes generating the processed media content ID based on at least one of a cyclic-redundancy- check (CRC) function, a hash function, or a hash-based message authentication code.

[0009] In some implementations, the method includes determining, for the key, an allocation of resources in the secure memory’ of the hardware IC. In some implementations, the method includes obtaining a media content ID derived from the media file; and verifying an authenticity or integrity of the media file based on the media content ID prior to processing the media file. [0010] In some implementations, the key is a second key, and the method further includes: obtaining a first key derived from the media file; and generating a security manifest based on: i) the first key, ii) the second key, and iii) the process history information. The method can further include: obtaining a first key associated with the media file; appending the second key to the first key to generate a key chain; and generating the security manifest based on the key chain. The processed media file can be stored in a region of memory that is outside of the security subsystem on the hardware IC.

[0011] Another aspect of the subject matter described in this specification can be embodied in a method performed on a hardware integrated circuit (IC) of computing device. The method includes obtaining a media file; generating, by a first processor block of the hardware IC: i) a first processed media file based on a first processing operation performed on the media file, ii) a first media ID based on the first processed media file, and iii) a first key that indicates the first processing operation. The method includes verifying, by a second processor block of the hardware IC, the first processed media file based on the first media ID; generating, by the second processor block of the hardware IC: i) a second processed media file based on a second processing operation performed on the first processed media file, and ii) a second key that indicates the second processing operation. The method also includes generating, based on the first key and the second key, a security' manifest indicating a process history of the media file.

[0012] These and other implementations can each optionally include one or more of the following features. For example, in some implementations, verifying the first processed media file based on the first media ID includes performing an intermediate provenance check on the first processed media file by determining a measure of consistency between the first media content ID and the first processed media file.

[0013] In some implementations, the method includes storing, based on a write request issued by the first processor block: i) the media ID and the first key in a secure memory of a security' subsystem on the hardware IC, and ii) the first processed media file in a region of memory that is outside of the security subsystem on the hardware IC. In some implementations, the method includes storing, based on a write request issued by the second processing block: i) the second key in the secure memory, and ii) the second processed media file in the region of memory that is outside of the security' subsystem on the hardware IC. [0014] In some implementations, the method includes retrieving the first key and the second key from secure memory prior to generating the security manifest. In some implementations, the method includes appending the second key to the first key to generate a key chain; storing the key chain in the secure memory; and generating, based on the key chain, a provenance tag that encodes the process history of the media file. In some implementations, the method includes retrieving the key chain from the secure memory prior to generating the security manifest. In some implementations, the method includes storing, by the first processor block, the first media ID in the secure memory; and retrieving, by the second processor block, the first media ID from the secure memory prior to verifying the first processed media file.

[0015] At least one of the first media ID or the second media ID can be generated based on at least one of a cyclic-redundancy-check (CRC) function, a hash function, or a hashbased message authentication code. In some implementations, the method includes determining, for the first key and the second key, an allocation of resources in the secure memory of the hardware IC. The first media ID can represent a measure of integrity or authenticity of the first processed media file.

[0016] Other implementations of this and other aspects include corresponding sy stems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices. A system of one or more computers can be so configured by virtue of software, firmware, hardware, or a combination of them installed on the system that in operation causes the system to perform the actions. One or more computer programs can be so configured by virtue of having instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.

[0017] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Fig. 1 is a block diagram of an example computing sy stem with a security subblock for generating a metadata chain and provenance information for a media file.

[0019] Fig. 2 is a first example process for generating a provenance tag using the system of Fig. 1

[0020] Fig. 3 is one portion of a second example process for generating a provenance tag using a first processor block in the system of Fig. 1.

[0021] Fig. 4 is another portion of the second example process for generating a provenance tag using a second processor block in the system of Fig. 1. [0022] Like reference numbers and designations in the various drawings indicate like elements.

DETAILED DESCRIPTION

[0023] Fig. 1 is a block diagram of an example computing system 100 (“system 100"’) with a security sub-block for generating a metadata chain and provenance tag (or signature) for a media file. System 100 can be (or include) a System-on-Chip (SoC) implemented as a hardware integrated circuit of the computing device. In some implementations, system 100 includes multiple SoCs and descriptions in this document that reference an SoC can apply equally to each of the multiple SoCs that may be included at system 100.

[0024] The media file can be an image, a collection of images (or image frames), video data, or a video stream. The media file can be generated locally at an example computing device, such as a mobile/client device, using a camera and one or more image sensors of the computing device. Some media files may be generated locally at system 100, whereas other media files are received from a source that is external to system 100, to the example computing device that includes system 100, or both. In some implementations, the provenance operations that are described as being performed on media files can be also performed on non-media file types, such as text documents, spreadsheets, portable document files (PDFs), or certain image file types that may be regarded as non-media file types.

[0025] System 100 includes a first processor block 102. a second processor block 104. and a memory device 106. In some implementations, system 100 includes n processor blocks, where n is an integer greater than or equal one. System 100 includes a central processing unit (“CPU”), a security resource manager (described below), and an IP/circuit block that includes multiple processor blocks, e.g., some (or all) of the n processor blocks. [0026] The CPU can be included among the n processor blocks and may be a general- purpose CPU (e.g., a single or multi-core CPU). In some implementations, the CPU is an application processor of system 100 that manages data flow operations for a media processing pipeline of system 100. The CPU can generate one or more indicators, such as an app-launch indicator or a function call that is triggered in response to executing or launching an application at a user device.

[0027] For example, the application can be a camera application that uses an imaging sensor to generate image data or a gaming application that requires substantial memory and graphics processing resources to render graphical content of the game. The CPU also generates one or more application values, such as pixel values or frame rate data. The application values may be associated with a function call, may be descriptive of an event that occurs during execution of the application, or both.

[0028] Memory device 106 can be a shared memory that is shared among one or more processors of system 100. In some implementations, memory device 106 is a system memory⁷, shared memory, or both. As indicated in the example of Fig. 1, memory⁷ device 106 can be a random access memory, such as a dynamic random-access memory (DRAM), a synchronous DRAM (SDRAM), or double data rate (DDR) SDRAM. In some implementations, aspects of memory⁷ device 106 are configured as a shared scratchpad memory' that supports parallel access of its memory⁷ resources by two or more processors (processor blocks) of system 100. Memory device 106 can also include various other types of memory, such as high bandwidth memory (HBM), narrow memory (e.g., for storing 8-bit values), wide memory (e.g., for storing 16-bit or 32-bit values), etc.

[0029] The system 100 includes a security⁷ subsystem 114 (“security block 114”) that includes secure memory resources. In some implementations, the secure memory⁷ resources of the security block 114 are provisioned from a subset of secure resources in the memory device 106. The subset of secure resources can exist entirely within the security subsystem of system 100 and is isolated from a separate and distinct region of the memory' device 106 that is outside of the security⁷ block 114. In some implementations, the subset of secure resources is configured for access only by a secure operating system (“secure OS”) of system 100.

[0030] The secure resources of the security block 114 include a subset of secure control status registers (CSRs). Each secure CSR can be used to store process history information for a corresponding processor block of the processing pipeline. Each secure CSR is used to securely capture/store and/or indicate data processing operations performed on a media file by one or more of the processor blocks 102, 104. For example, a first CSR can store a media identifier (ID) and a corresponding key for a first process history 108-1, whereas a second CSR can store a media ID and a corresponding key for a second process history 108-2. System 100 can include n CSRs for storing n sets of process history⁷ 108-n for n processor blocks.

[0031] System 100 includes a manifest generator 120 that generates a security manifest 130 based on a provenance tag 116. The provenance tag/signature 116 is used to indicate authenticity' of the media file following one or more processing operations performed on the media file. In some implementations, the provenance tag 116 indicates non-generative attributes of the media file. For example, the provenance tag 116 can indicate that no generative artificial intelligence (Al) (“genAI”) content has altered, or been added to, the media file, which would alter media file’s authenticity. Stated another way, the provenance tag 116 can be used to assess or determine whether a media file has undergone data processing operations that have materially altered or augmented the original sensor data (e.g., image sensors) that was used to generate the media file. The manifest generator 120 and security manifest 130 are described in more detail below with reference to Figs. 2-4.

[0032] The system 100 is configured to generate a media file thumbnail 118 that can be used to assess provenance of a processed media file. The media file thumbnail 118 can be stored in a secure memory resource of the security block 114 or at some other memory resource of system 100. For example, and as indicated at Fig. 1, the media file thumbnail 118 is metadata stored in the security block 114 and later used to generate a security' manifest that indicates a process history of the media file and/or modifications to the media file that affect provenance of the original image/video or other content of the media file.

[0033] In some implementations, the media file thumbnail 118 is a type of metadata or provenance asset that can be shown or displayed to a user to allow for a quick visual comparison to a final or intermediate version of a processed media file. The system 100 can use the media file thumbnail 118 to determine: i) whether content in a processed version of the media file is consistent with content of the original media file (e.g., provenance = “high” or “1”) or ii) whether content in a processed version of the media file is inconsistent with content of the original media file (e.g., provenance = “low” or “0”).

[0034] In examples the media file metadata 118 can be a thumbnail that has a data size that is smaller (e.g., substantially smaller) than the data size of the original media file from which the thumbnail is derived. For example, the media file metadata 118 can be a small thumbnail image/frame of the original media file generated from a downsampling operation performed on the original media file. Other sampling and/or processing operations may also be used to generate metadata of the original media file that is represented as a thumbnail version of the original media file.

[0035] As discussed above, the system 100 can be implemented as an integrated circuit, an SoC, or both. System 100 can be integrated in a computing device, such as any consumer electronic device or mobile/client device. For example, these devices can include items such as a smartphone, tablet, laptop, or a smartwatch and/or other wearable devices. The devices may also include other items such as an eNotebook, Netbook, smart speaker, or mobile computer. System 100 can also be implemented on (or as) an integrated circuit of a desktop computer, network server, or related cloud-based asset. [0036] Fig. 2 is an example process for generating a provenance tag for a media file. Process 200 can be implemented or executed using system 100 as well as other computing resources described in this document. Hence, descriptions of process 200 may reference the above-mentioned computing resources of system 100. In some examples, the steps or actions of process 200 are enabled by programmed software instructions, firmware instructions, or both. Each t pe of instruction may be stored in a non-transitory machine-readable storage device and is executable by one or more of the processors or other resources described in this document.

[0037] In some implementations, the steps of process 200 are performed at a hardware integrated circuit during processing operations for generating a machine-learning (ML) output or other data processing outputs, including an output for a neural network layer of a neural network that implements one or more ML models. For example, the output can be a portion of a computation for a ML task or inference workload to generate an image processing, speech processing, or image recognition output. As indicated above, a portion of the integrated circuit can include a special-purpose neural network processor or hardware ML accelerator configured to accelerate computations for generating different types of data processing outputs.

[0038] Referring again to process 200, the system 100 obtains a media file (202). The system 100 processes the media file using a processor block of the hardware integrated circuit (204). As indicated above, the system 100 can include multiple processor blocks 102, 106, that are individual IP devices such as processors, processor cores, or special-purpose processing devices. For example, system 100 can include an image signal processor (ISP), a tensor processing unit (TPU) (or ML hardware accelerator), and a graphics processing unit (GPU). For example, each of the ISP. TPU. and GPU can be a respective proprietary’ IP block (or IP device) of a particular entity or device manufacturer.

[0039] The system 100 generates a key that indicates a type of processing operation that was performed on the media file by the processor block (206). The key can be extended to media files such as small images (e.g.. 512 byte or IK) that are processed along a processing pipeline of system 100. The key can be also extended to other types of files that may be processed along the processing pipeline, including audio, video, text, and/or other types of data files. In some implementations, different keys are coded to a particular data processing operation, such as a scaling & warping operation, an ML inference operation, a generative Al operation, a resolution enhancement operation, or a data encoding (decoding) operation. The processor block can generate a key based on the type of operation performed on the media file.

[0040] The system 100 stores the key in a secure memory of a security subsystem on the hardware integrated circuit (208). In some implementations, the processor block also generates a processed media content identifier (ID) based on the pixel data or other media content of the media file that was accessed or processed by the processor block. The media content ID represents a measure of integrity or authenticity of the media file after the media file is processed using the processor block.

[0041] In some implementations, the processor block can pass data values for the pixel data or other media content to a checksum engine, encry ption engine, or hash generator. For example, system 100 (or the checksum engine) can pass the data values as inputs to a cyclic- redundancy-check (CRC) function applied by the checksum engine and generates the content ID as an output of the checksum engine. Alternatively, or additionally, the system 100 (or the hash generator) can pass the data values as inputs to a hash function applied by the hash generator to generate the content ID as an output of the hash generator. In some implementations, the hash generator generates the key based on a hash-based message authentication code. The system 100 can also use corresponding CRC, checksum, or hashing methods to generate a key to indicate a type of operation performed on the media file.

[0042] The processor block can also generate one or more write requests to store the key and the processed media content ID in the secure memory of system 100. In some implementations, each of the key and media content ID is stored in a CSR of security' block 114 that is included among a subset of secure resources that are allocated to the processor block to capture processor history data for that processor block.

[0043] The system 100 generates a provenance tag based on the key (210). The provenance tag encodes process history information about the media file, where the process history information indicates one or more data processing operations that were performed on the media file. For example, the process history information can be derived from the key, which can indicate a particular type of data processing operation, such as resolution enhancement or inference processing, performed on a media file.

[0044] The system 100 generates a security manifest based on the process history information encoded in (or by) the provenance tag. The security manifest indicates the one or more processing operations that were performed on the media file by each of the processor blocks that processed the media file. [0045] Fig. 3 is another example process 300 for generating a provenance tag for a media file. Like process 200. process 300 can be also implemented or executed using system 100 and other resources described herein. The steps or actions of process 300 are performed using programmed software instructions, firmware instructions, or both. One or more of the instructions may be stored in a non-transitory machine-readable storage device and are executable by one or more of the processors or other resources described in this document. In some implementations, the steps of process 300 are performed across one or more hardware integrated circuits of a computing device, such as a smartphone or tablet.

[0046] To process a media file, system 100 can initiate a heterogeneous compute operation where some (or all) of the multiple processor blocks, e.g., ISP, TPU, and GPU, are responsible for a specific data processing operation of a larger heterogeneous operation. For example, the media file can be an image and the heterogeneous compute can include: i) an inference operation to perform facial recognition on the image data and ii) an enhancement operation to enhance the resolution of the image.

[0047] Referring again to process 300, the system 100 obtains a media file (302). A first processor block of the system 100 generates a first processed media file 110-1 based on a first processing operation performed on the media file (304). For example, the first processing operation can be performed by or using an ML Accelerator or GPU IP block that represents the first processor block of system 100. In some implementations, the first processing operation is performed to modify or enhance resolution of content in an initial or intermediate version of the media file that is received, obtained, or requested by the first processor block . [0048] The first processor block generates a first media content ID based on the first processed media file 110-1 (306). In some implementations, the media file can be an image generated using one or more cameras and image sensors of an example computing device that includes system 100. The first processor block is configured to generate a processed media content identifier (ID) (e.g., an image ID) based on the pixel data or other media content of the media file that was accessed or processed by the processor block. For example, the image/content ID can be generated by using the resulting pixel and/or data values of the content in the media file after the first processing operation was performed to modify or enhance the resolution of content. This media ID represents a measure of integrity or authenticity of the media file after the media file is processed using the first processor block. [0049] The first processor block also generates a first key that indicates the first processing operation (308). The first processor block (e.g., the first processor block 102 shown in Fig. 1) can store both the first media content ID and the first key in the security sub- system 114 as process history-1 108-1. The first processor block 102 can also store the processed media file 110-1 in memory 106 for access by other processing blocks.

[0050] In some implementations, if a processing operation performed by the first processor block alters information in the image content of the media file, then the first processor block can determine that a content ID (IMID) is not consistent with the image content (e.g., provenance is altered) and set a value of a provenance flag to “0.” The system 100 can also determine whether the alternation to the image content is sufficiently relevant so as to warrant adjusting a value of the provenance flag to indicate a loss provenance or authenticity of the original media file.

[0051] As described above, keys are generated and securely stored in the security block 114 to indicate the process operations performed on the media file. In some implementations, each IP/processor block of system 100 can have one key or multiple sets of keys that it can use, reference, and/or select from to indicate a type of processing operation that is performed on a version of the media file that is received at the processor block. In some cases, the sophistication, complexity, and/or length of a particular key can depend on the extent to which the IP/processor block has modified the media file.

[0052] In some implementations, each IP/processor block 104, 106 may have one key that provides a discrete indication that a processing operation was performed on the media file, whereas in some other implementations each IP/processor block 104, 106 has multiple keys that each provide a granular indication of the type of operation that was performed on a media file.

[0053] Fig. 4 is another example process 400 for generating the provenance tag for the media file associated with process 300. Like process 300, process 400 can be also implemented or executed using system 100 and other resources described herein. The steps or actions of process 400 are performed using programmed software instructions, firmware instructions, or both. One or more of the instructions may be stored in a non-transitory machine-readable storage device and are executable by one or more of the processors or other resources described in this document.

[0054] In some implementations, processes 300, 400 are respective portions of a larger process flow that is performed to generate a provenance tag (e g., a single provenance tag) for a final processed version of the media file identified above with reference to process 300.

For example, process 300 can be performed using a first processor block of system 100, whereas process 400 can be performed using a second, different process block of system 100. Each of the first and second processor blocks can define a processing pipeline of system 100. [0055] Referring again to process 400, the second processor block of system 100 verifies the first processed media file based on the first media ID (402). In some implementations, the second processor block performs a content verification operation on the first processed media file as a way to verify some aspect of the media content of the first processed media file.

[0056] For example, the second processor block can perform an intermediate provenance check on the first processed media file by determining a measure of consistency between the first media content ID and the first processed media file. For example, the second processor block 104 (Fig. 1) can access the first media content ID included in the process history-1 108- 1 as well as the first processed media file 110-1 stored in memory 106 to perform an intermediate provenance check on the first processed media file 110-1 prior to generating its own processed version of the media file.

[0057] In some implementations, the second processor block 104 (Fig. 1) determines the measure of consistency by computing (e.g., locally at the processor block) a content ID based on pixel (and/or other) data values of the first processed media file 110-1 that is received at the second processor block from the memory device 106. The second processor block performs the intermediate provenance check by determining whether the content ID it computed for the first processed media file 110-1 is consistent with the first media content ID. In some implementations, the intermediate provenance check functions as an additional security check to determine whether the first processed media file 110-1 was tampered with or modified before, during, or after storage in memory device 106.

[0058] The second processor block generates a second processed media file based on a second processing operation performed on the first processed media file (404). For example, referring again to Fig. 1, the second processor block 104 can generate a second processed media file 110-2 based on a second processing operation performed on the first processed media file 110-1. In some implementations, the second processor block 104 is a specialpurpose neural network processor or ML accelerator, such as a TPU, and the second processing operation performed on the first processed media file 110-1 can be an ML inference operation to recognize/detect a specific type of content in the media file and/or to remove the recognized/detected content from the media file.

[0059] The second processor block generates a second key that indicates the second processing operation (406). For example, the second processor block 104 can generate a second key to indicate the second processing operation as an ML inference operation that detected and/or removed content from the media file. The second processor block 104 is configured to store the second key as process history -2 108-2 in the security sub-system 114, for example using a secure CSR of the security sub-system. The system 100 can also operate on the media file with additional processor blocks, such as the w-th processor block, and generate an w-th processed media file 110-n based on additional processing operations performed by the zi-th processor block.

[0060] The system 100 generates a security manifest based on the first key generated at process 300 and the second key (408). For example, referring to Fig. 1, the manifest generator 120 can generate a security manifest based on the fist key (e.g., process history-1 108-1) and the second key (e.g., the processed history-2 108-2). In some implementations, where the media file undergoes processing by n processor blocks, the manifest generator 120 can generate the manifest based on n keys (e.g., process history-1 108-1 - 108-n). The security manifest indicates a process history of the media file.

[0061] The system 100 can generates a single provenance tag based on the first and second keys and appends that provenance tag, along with process history data, to the security manifest to indicate provenance and process history of the media file. In some implementations, the security manifest and/or process history is configured to include, or provide indications of, a first processing operation (if performed) and a second processing operation (if performed) on the media file, including whether at least one or both operations altered the provenance of the original media file.

[0062] The manifest generator 120 can be implemented in hardware, software, or both. In some implementations, the manifest generator 120 retrieves a first key and/or a second key from a secure CSR or other secure register of the security sub-system 116 prior to generating the security manifest. The manifest generator 120 can generate the security manifest based on data processing operations represented by one or more of the first key and the second key. [0063] In some implementations, control logic of the security sub-system 1 16 can retrieve the first key and the second key from a secure memory resource, append the second key to the first key to generate a key chain, and generate a provenance tag and process history information based on operations represented by the key chain. The system 100 can also store the key chain in the secure memory. In some implementations, provenance tag is configured to encode the process history of the media file, for example, as a sequence of process opcodes.

[0064] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus.

[0065] Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0066] The term '‘computing system” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0067] A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0068] A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication netw ork.

[0069] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), or a GPGPU (General purpose graphics processing unit).

[0070] Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. Some elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few. [0071] Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0072] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by⁷ which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser. [0073] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.

[0074] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0075] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0076] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. [0077] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

What is claimed is:

1. A method performed on a hardware integrated circuit (IC) of computing device, the method comprising: obtaining a media file; processing the media file using a processor block of the hardware IC; generating a key that indicates a type of processing operation performed on the media file by the processor block; storing the key in a secure memory of a security subsystem on the hardware IC; and generating, based on the key, process history' information that indicates one or more processing operations that were performed on the media file.

2. The method of claim 1 , further comprising: storing a processed media content identifier (ID) in the secure memory of the hardware IC, wherein the media content ID represents a measure of integrity or authenticity of the media file after the media file is processed using the processor block.

3. The method of claim 2, further comprising: generating the processed media content ID based on at least one of a cyclic- redundancy-check (CRC) function, a hash function, or a hash-based message authentication code.

4. The method of claim 1 , further comprising: determining, for the key, an allocation of resources in the secure memory of the hardware IC.

5. The method of claim 1, further comprising: obtaining a media content ID derived from the media file; and verify ing an authenticity or integrity of the media file based on the media content ID prior to processing the media file.

6. The method of claim 1, wherein the key is a second key, and the method further comprises: obtaining a first key derived from the media file; and generating a security manifest based on: i) the first key, ii) the second key, and hi) the process history information.

7. The method of claim 1, wherein the key is a second key, and the method further comprises: obtaining a first key associated with the media file; appending the second key to the first key to generate a key chain; and generating the security manifest based on the key chain.

8. The method of claim 1, wherein the processed media file is stored in a region of memory that is outside of the security subsystem on the hardware IC.

9. A method performed on a hardware integrated circuit (IC) of a computing device, the method comprising: obtaining a media file; generating, by a first processor block of the hardware IC: i) a first processed media file based on a first processing operation performed on the media file, ii) a first media ID based on the first processed media file, and iii) a first key that indicates the first processing operation; verifying, by a second processor block of the hardware IC, the first processed media file based on the first media ID; generating, by the second processor block of the hardware IC: i) a second processed media file based on a second processing operation performed on the first processed media file, and ii) a second key that indicates the second processing operation; and generating, based on the first key and the second key, a security manifest indicating a process history of the media file.

10. The method of claim 9, wherein verifying the first processed media file based on the first media ID comprises: performing an intermediate provenance check on the first processed media file by determining a measure of consistency between the first media content ID and the first processed media file.

11. The method of claim 9, further comprising: storing, based on a write request issued by the first processor block: i) the media ID and the first key in a secure memory of a security subsystem on the hardware IC, and ii) the first processed media file in a region of memory' that is outside of the security subsystem on the hardware IC.

12. The method of claim 1 1, further comprising: storing, based on a write request issued by the second processing block: i) the second key in the secure memory, and ii) the second processed media file in the region of memory that is outside of the security subsystem on the hardware IC.

13. The method of claim 11, further comprising: retrieving the first key and the second key from secure memory prior to generating the security manifest.

14. The method of claim 9, further comprising: appending the second key to the first key to generate a key chain; storing the key chain in the secure memory; and generating, based on the key chain, a provenance tag that encodes the process history of the media file.

15. The method of claim 14, further comprising: retrieving the key chain from the secure memory prior to generating the security manifest.

16. The method of claim 9, further comprising: storing, by the first processor block, the first media ID in the secure memory; and retrieving, by the second processor block, the first media ID from the secure memory prior to verifying the first processed media file.

17. The method of claim 9, wherein at least one of the first media ID or the second media ID is generated based on at least one of a cyclic-redundancy-check (CRC) function, a hash function, or a hash-based message authentication code.

18. The method of claim 9, further comprising: determining, for the first key and the second key, an allocation of resources in the secure memory of the hardware IC.

19. The method of claim 9, wherein the first media ID represents a measure of integrity or authenticity of the first processed media file.

20. A system comprising: a processing device; a hardware integrated circuit (IC) that includes the processing device; and a non-transitory machine-readable storage device storing instructions that are executable by the processing device to cause performance of operations comprising: obtaining a media file; processing the media file using a processor block of the hardware IC; generating a key that indicates a type of processing operation performed on the media file by the processor block; storing the key in a secure memory of a security subsystem on the hardware IC; and generating, based on the key, process history information that indicates one or more processing operations that were performed on the media file.