WO2024040444A1

WO2024040444A1 - Data processing method and apparatus, and device, movable platform, unmanned aerial vehicle, storage medium and program product

Info

Publication number: WO2024040444A1
Application number: PCT/CN2022/114329
Authority: WO
Inventors: 程王钊; 周桂文; 田鹏
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2022-08-23
Filing date: 2022-08-23
Publication date: 2024-02-29

Abstract

Provided in the present application are a data processing method and apparatus, and a device, a movable platform, an unmanned aerial vehicle, a storage medium and a program product. The method is applied to an electronic device, and the electronic device locally stores a private key, a device certificate, and a verification certificate for verifying the device certificate, wherein the device certificate comprises a public key, and the private key and the public key correspond to each other. The method comprises: acquiring sensing data collected by an electronic device; and signing the sensing data by means of a private key to obtain signature information, wherein after a verification certificate verifies that a device certificate is legitimate, the signature information and a public key are used for determining whether the sensing data has been tampered with after being signed. The present application can solve the technical problem in the related art of verifying whether sensing data has been tampered with.

Description

Data processing methods, devices, equipment, movable platforms, drones, storage media and program products

Technical field

This application relates to the field of data processing technology, specifically, to a data processing method, device, equipment, movable platform, drone, storage medium and program product.

Background technique

Electronic devices collect sensing data during operation. How to verify whether the sensing data has been tampered with is a technical issue that has attracted much attention.

Some solutions are based on the server's online verification of whether the sensing data has been tampered with. The verification process of this type of scheme requires access to the server online to obtain a verification certificate, and based on the verification certificate returned by the server, connect to the server to check whether the issuing authority of the certificate is trustworthy, etc. to complete the identity verification. The entire verification process relies on the ability to connect online.

However, in some application scenarios, the network environment may be unstable, causing the verification process to be unstable or even impossible to complete. Moreover, in some specific scenarios, for example, the sensing data itself is confidential, and users want to complete the verification process of the sensing data offline.

Other solutions can be used in offline scenarios, such as adding watermarks or changing specific pixels in the image in the field of image processing. However, the security strength is weak and such solutions can be circumvented by removing anti-counterfeiting information. The image cannot be tampered with. aware.

Contents of the invention

In view of this, this application provides a data processing method, device, equipment, movable platform, drone, storage medium and program product to solve the technical problem of verifying whether data has been tampered with in related technologies.

In a first aspect, a data processing method is provided. The method is applied to an electronic device. The electronic device locally stores a private key, a device certificate and a verification certificate for verifying the device certificate. The device certificate includes a public key. , the private key and the public key correspond to each other, and the method includes:

Obtain sensor data collected by the electronic device;

Sign the sensing data with the private key to obtain signature information;

Wherein, after the verification certificate verifies that the device certificate is legitimate, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.

In a second aspect, a data processing method is provided, the method is applied to a verification device, and the method includes:

Obtain sensing data, signature information, device certificate and verification certificate from the electronic device, wherein the sensing data is collected by the electronic device, and the signature information is the electronic device using the private key stored locally on the electronic device. Obtained from the sensing data signature, the device certificate includes a public key corresponding to the private key, and the verification certificate is used to verify the device certificate;

Use the verification certificate to verify whether the device certificate is legal;

After the device certificate is verified to be legal, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.

In a third aspect, a data processing device is provided. The device includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the first Embodiments of data processing methods described in aspects.

A fourth aspect provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the first Embodiments of data processing methods described in aspects.

In a fifth aspect, a verification device is provided. The verification device further includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the first Embodiments of the data processing method described in the second aspect.

In a sixth aspect, a drone is provided, the drone comprising:

body;

A power system, located in the fuselage, is used to provide power for the drone;

And, a processor and a memory provided in the fuselage, the memory stores a computer program that can be executed by the processor, and when the processor executes the computer program, the first aspect or the second aspect is achieved. Embodiments of the data processing method described above.

In a seventh aspect, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed, the embodiment of the data processing method described in the first or second aspect is implemented.

An eighth aspect provides a computer program product, including a computer program that implements the data processing method embodiments described in the first or second aspect when executed by a processor.

Applying the solution provided by this application, the electronic device locally stores the private key, and the private key corresponds to the public key in the device certificate. Therefore, the signature information is obtained by signing the sensing data through the private key; because the electronic device also locally stores the Verify the verification certificate of the device certificate. Therefore, the verification certificate can be directly used to verify whether the device certificate is legitimate, and after the device certificate is legitimate, the signature information and public key are used to determine whether the sensing data has been tampered with after being signed. Therefore, this embodiment The solution does not need to be connected to the server for verification, does not need to rely on networking capabilities, and can verify whether the data has been tampered with offline. Electronic devices can work offline, and the verification process can also be performed offline, meeting users' needs for equipment to work offline. ; It realizes offline verification of whether data has been tampered with and ensures the security of sensor data. Moreover, this scheme is based on cryptographic characteristics. The signature process of sensing data is credible and irreversible. Legal signatures cannot be forged by attackers, and the security strength is high.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting any creative effort.

Figure 1A is a flow chart of a data processing method according to an embodiment of the present application.

Figure 1B is a verification schematic diagram of an embodiment of the present application.

Figure 1C is a schematic diagram of data processing according to an embodiment of the present application.

1D to 1F are respectively schematic diagrams of the user interface according to an embodiment of the present application.

Figure 1G is a schematic diagram of an electronic device and a remote control device according to an embodiment of the present application.

Figure 1H is a schematic diagram of the user interface of a remote control device according to an embodiment of the present application.

Figure 2A is a schematic diagram of a data processing method according to another embodiment of the present application.

Figure 2B is a schematic diagram of a drone transmitting files to a verification device according to an embodiment of the present application.

Figure 2C is a flow chart of another data processing method according to an embodiment of the present application.

Figure 2D is a schematic diagram of the processing of signature information and certificate chain information according to an embodiment of the present application.

Figure 2E is a flow chart of online verification according to an embodiment of the present application.

Figure 3 is a flow chart of a data processing method according to an embodiment of the present application.

Figures 4 and 5 are respectively hardware structure diagrams of a data processing device according to an embodiment of the present application.

Figure 6 is a hardware structure diagram of an electronic device according to an embodiment of the present application.

Figure 7 is a hardware structure diagram of a verification device according to an embodiment of the present application.

Figure 8 is a structural diagram of an unmanned aerial vehicle according to an embodiment of the present application.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the specification. As used in this specification and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this specification, the first information may also be called second information, and similarly, the second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."

In many application scenarios, users have the need to verify whether the sensor data collected by electronic devices during operation has been tampered with. For example, point cloud data collected by lidar, images or videos collected by camera equipment, etc. Taking drones as an example, drones will generate many types of data during operation, such as images, audio, point clouds or flight trajectory data. In some scenarios, users have the need to use images captured by drones for investigation and evidence collection. If these images are maliciously forged or tampered with, the validity of the forensic evidence will be challenged. At the same time, the copyright ownership of the image itself is also a matter of great concern.

Some image anti-counterfeiting technologies declare image ownership by adding watermarks to the original image. However, this technology will destroy the original image content and make it more obvious in appearance. Attackers can use watermark removal technology to remove the watermark.

Other similar technologies are steganographic watermarks: first, the image data is transformed from a time domain image into a frequency domain image through Fourier transform. After superimposing the watermark information on the image, the frequency domain image is converted into a time domain image. This ensures The movement of the original image content is small and difficult to detect. However, this solution can still be removed through simple watermark removal methods.

There are also some solutions that edit author information into specific pixels in the image by changing them. In this way, it is almost impossible to detect changes in the image visually, but it is still unavoidable to circumvent such protection schemes by deleting this information.

The above solutions are based on image processing, and they can all achieve the purpose of image anti-counterfeiting and declaration of image copyright. However, the security strength of these solutions is weak, and the effectiveness of such solutions can be reduced or eliminated by removing anti-counterfeiting information. . Moreover, such solutions cannot resist the problem of image tampering, that is, the image cannot be detected after being tampered with.

To solve the above problems, some solutions are based on techniques such as mathematics or cryptography. For example, some solutions prove that the file has not been tampered with by calculating the hash value (hash, also called hash) of the file. The main processes include:

For the image file file1, calculate its hash value hash1, and store the image file file1 and the hash value hash1;

The user gets the image file file2 and hash1, and calculates the hash value hash2 of the image file file2;

Compare hash1 and hash2. If they are equal, the image file file2 and image file file1 obtained by the user are the same file. Otherwise, the image file file2 obtained by the user is a tampered image file.

Other solutions are based on symmetric cryptographic algorithms and hash algorithms. They use the symmetric key held by the device to calculate the cmac (Cipher Block Chaining-Message Authentication Code) value of the file to prove that the file has not been tampered with. The general process Similar to the above, the specific process will not be described again. The difference in the process is that in the process of calculating cmac, the symmetric key key needs to be used, and the verification process also needs to use the key key.

Other solutions are file anti-counterfeiting signature schemes based on asymmetric cryptographic algorithms. The traditional PKI (Public Key Infrastructure) solution is mainly used for identity verification in online scenarios. For example, common browsers verify the identity of a website. The browser requests a certificate from the website to be visited and returns the certificate according to the website. certificate to check whether the issuing authority of the certificate is trustworthy, etc. In the browser scenario, the verifier (browser) must have the ability to connect to the Internet. Therefore, in traditional PKI system solutions, the verification process relies on online networking capabilities.

The above solutions have the following shortcomings:

With solutions based on image processing, tampering of images cannot be detected; on the other hand, such technical solutions can remove anti-counterfeiting information through technical means, thus removing the protective effect of such solutions.

To calculate the hash value of a file, you only need to know the hash algorithm, and the hash value can be calculated by anyone. Currently, the mainstream hash algorithm is public. An attacker can recalculate the hash value after tampering with the file to complete the attack.

During the verification process of the cmac-based verification algorithm, the verifier needs to use the key to complete the verification. The symmetric key key is private data. Before verifying whether the file has been tampered with, the verifier needs to obtain this key through a secure channel to complete the verification. If the key is lost, cmac can still be forged. The process of transferring the key to the verifier also greatly increases the possibility of key loss.

In the asymmetric key scheme, the verification process requires networking. Therefore, in some specific scenarios, such as when the data itself is confidential, users have offline requirements.

To sum up, in the current solution, it is either impossible to verify whether the photo has been forged or maliciously tampered through offline methods, or the solution has incomplete security functions and cannot detect whether it has been maliciously tampered with, or the security strength of the solution itself is too weak. It cannot meet the needs of protecting photo authentication or verifying whether it has been tampered with. However, these characteristics are all very important in industrial applications such as drones.

1) Offline verification of photos is very necessary in industrial application scenarios such as drones, because it is very common in many outdoor operation scenarios to not have a network. In addition, when power grid and other departments use electronic equipment to collect data such as images, the data itself has high confidentiality requirements. These data are not allowed to appear on the public network, and online data verification solutions cannot be used.

2) For verifying whether the image has been tampered with, in some scenarios, the image is an important evidence collection material. In this scenario, proving that the image has not been maliciously tampered with is an important basis for ensuring the legitimacy of the image as evidence collection material.

3) Image authentication scenarios are more common, and the issue of how to prove the copyright ownership of an image is also very common.

4) Reducing the complexity of the solution in actual application scenarios also needs to be considered.

Based on this, embodiments of this specification provide a data processing method that can verify offline whether the data has been tampered with. As shown in Figure 1A, it is a flow chart of a data processing method shown in this specification according to an exemplary embodiment. The method in this embodiment is applied to an electronic device, and the electronic device locally stores a private key, a device certificate and a method for Verify the verification certificate of the device certificate, the device certificate includes a public key, and the private key corresponds to the public key; the method may include the following steps:

In step 102, obtain sensing data collected by the electronic device;

In step 104, the sensing data is signed using the private key to obtain signature information.

Figure 1B is a schematic diagram of a verification process shown in this specification according to an exemplary embodiment. After the verification certificate verifies that the device certificate is legal, the signature information and the public key in the device certificate are To determine whether the sensing data has been tampered with after being signed.

In this embodiment, the electronic device locally stores a private key, and the private key corresponds to the public key in the device certificate. Therefore, the signature information is obtained by signing the sensing data through the private key; because the electronic device also locally stores a key for verifying the device. The verification certificate of the certificate, therefore, the verification certificate can be directly used to verify whether the device certificate is legal, and after the device certificate is legal, the signature information and public key are used to determine whether the sensing data has been tampered with after being signed, so the solution of this embodiment can Offline verification of whether the data has been tampered with, electronic equipment can work in an offline state, and the verification process can also be performed in an offline state, meeting the user's needs for the device to work offline; the security of the sensing data is also guaranteed in the offline state.

The solution of this embodiment can be applied to any electronic device. For example, the electronic device may include a shooting device, a movable platform, a lidar, a mobile terminal, an audio device or a computer device, etc.; wherein the movable platform may include Drones, robots, cars or automatic cleaning equipment, etc.; mobile terminals can include smartphones or wearable devices, etc.

Sensed data can be implemented in a variety of ways depending on how the electronic device is implemented. For example, the sensing data may include data collected by one or more sensing modules in the electronic device, so that the electronic device can verify whether any type of sensing data has been tampered with after being signed.

For example, the sensing module may include any of the following: a camera module, an audio module, a point cloud module or a mobile control module, so that the data collected by the above-mentioned sensing modules can be verified whether it has been tampered with after being signed.

For example, the sensing data may include any of the following: images, audio, point clouds, or movement trajectory data of the electronic device, so that the above types of sensing data can be verified whether they have been tampered with after being signed.

There are many ways to implement local storage of private keys, device certificates and verification certificates on electronic devices. For example, based on the security requirements of the private key, the electronic device can use a higher security method to store the private key. For example, the private key can be stored in an offline security chip in the electronic device; or the central processing unit CPU of the electronic device It can include a Trusted Execution Environment (TEE), which uses software and hardware methods to build a secure area TEE in the central processor to ensure that the programs and data loaded inside are protected in terms of confidentiality and integrity. The device certificate and verification certificate are public certificates, and their storage method can be different from the private key. For example, their storage location can be flexibly configured according to needs; of course, the storage method of the device certificate and verification certificate in the same way as the private key is also optional. , this embodiment does not limit this. Of course, those skilled in the art know that other storage methods may be used in practical applications, which are not limited in this embodiment.

For example, the private key, device certificate and verification certificate may be stored in the electronic device at the same time or at different times. For example, the three information may be stored in the electronic device before the electronic device leaves the factory. Or, it is stored in the electronic device by user operation. For example, the electronic device obtains and stores the information after communication connection with another device such as a server. When the user operates the storage, optionally, the security of the storage process can be ensured through various methods such as verifying the user's identity. Or, part of the three can be stored before the electronic device leaves the factory, and some can be stored in the electronic device by user operation, etc. For example, the private key can be stored before the electronic device leaves the factory, and the other two can be stored in the electronic device by user operation, etc. etc., this embodiment does not limit this.

In some examples, the device certificate carries device information of the electronic device. In actual applications, the device information may include a variety of information, which is not limited in this embodiment. For example, it may include information indicating the identity of the electronic device. For example, the information may include the identification code of the device, such as the MEID (Mobile Equipment Identifier, mobile equipment identification code) code of the mobile device or the IMEI (International Mobile Equipment Identity, International Mobile Equipment ID) code, etc., or it can also be the unique product identification code of the drone, etc. In actual applications, it can be configured according to the type of electronic equipment applied in the solution of this embodiment, and this embodiment does not limit this. In other examples, the device certificate carrying the device information of the electronic device may also include information representing the sensing module in the electronic device that collects sensing data, so that the identity of the sensing module can be determined. Based on this, this embodiment can ensure that the source of the sensing data is trustworthy. For example, since the device certificate of this embodiment carries the device information of the electronic device, and the device certificate includes a public key, the signature of the sensing data is made using the public key. The corresponding private key is issued, so when it is determined that the sensing data has not been tampered with, it can also be determined that the copyright ownership of the sensing data is credible; and, since this embodiment realizes the verification of whether the sensing data has been tampered with, It also makes it difficult to forge the copyright ownership of sensing data.

In this embodiment, the verification certificate can be used to verify the legitimacy of the device certificate. For example, the device certificate is signed by the verification certificate.

In some examples, the verification certificate may include a root certificate, which is a public key certificate belonging to a root certification authority (CA) that is trusted in a public key infrastructure. The starting point of the chain. Among them, the device certificate can be signed by the root certificate. Therefore, when validating a device certificate, it is the root certificate that validates the device certificate.

For example, the verification certificate may only include the root certificate, that is, the root certificate may issue device certificates for each electronic device. In order to ensure the security of the root certificate, in other examples, the verification certificate includes a root certificate and one or more levels of sub-certificates; among the one or more levels of sub-certificates, the highest level sub-certificate is represented by the root certificate. Issuance, each sub-certificate of other levels of sub-certificates is issued by the sub-certificate of the previous level, and the lowest-level sub-certificate is used to issue the device certificate. Similarly, when verifying the device certificate, the root certificate can be used to verify the highest level sub-certificate, and each level of sub-certificates in other levels is verified by the sub-certificate of the previous level. In this embodiment, the number of sub-certificates is not limited, and can be configured as needed in actual applications.

In some examples, the verification certificate is issued by a certificate authority (CA). The certificate authority may be the CA organization of the electronic equipment manufacturer, or it may be a third-party CA organization, etc., which is not limited in this embodiment. Based on this, the authority and fairness of the verification certificate can be guaranteed.

For example, the private key and the public key correspond to each other, and the private key and the public key may be a pair of asymmetric keys. This embodiment does not limit the number of private keys, device certificates, and verification certificates. The public key and private key in the device certificate correspond to each other, that is, the number of device certificates corresponds to the number of private keys. For example, if m private keys are stored, there are m device certificates; the verification certificate is used to verify the device certificate, and the verification certificate The number can be implemented in various ways. For example, m device certificates may be verified using the same verification certificate or different verification certificates. Persons skilled in the art clearly know that this can be implemented according to actual needs, and this embodiment does not limit this.

As in the aforementioned embodiments, there may be a variety of sensing modules in the electronic device, which can collect different types of sensing data, or the same sensing module can also collect different types of sensing data. For example, the electronic device can store private data locally. When there are multiple keys, each private key can be used to sign the sensing data collected by different sensing modules, or each private key can be used to sign different types of sensing data.

The process of private key signature can be implemented in a variety of ways. It can be to directly use the private key to encrypt the sensing data to obtain the signature information, or it can be combined with the hash value algorithm to perform the private key signature. For example, it can be to calculate the third number of the sensing data. After a hash value is generated, the private key is used to encrypt the first hash value to obtain the signature information; in other examples, the private key may also be used to encrypt the sensing data, and then the hash value is calculated for the encryption result as the signature information. Optionally, the hash value can be calculated using any hash algorithm. When calculating the hash value of the sensing data, it can be all the information of the sensing data, or it can be part of the information. For example, the sensing data is image data, including pixel information, and may also include shooting time information, resolution information or camera information, etc. All or part of the information may be selected to calculate the hash value, which is not limited in this embodiment. Optionally, the calculation of the hash value can be performed in the TEE.

Permissions can be set for the private key signing process to ensure the security of the private key. For example, the sensing module that collects the sensing data calls the private key to sign the sensing data to obtain the signature information. That is, only the sensing module in the electronic device can call the private key for data signature, but other modules cannot. transfer. Optionally, this can be achieved by setting a whitelist. Exemplarily, the sensing module that collects the sensing data can, after passing the verification of the trusted execution environment, call the private key to sign the sensing data to obtain the signature information, which can prevent other devices running in the electronic device from The module calls the private key.

For example, the private key is stored in the TEE, and the signing process is also executed in a trusted execution environment; the execution process can be that the requesting party calls the signature interface to initiate a signature application, and the requesting party can be a sensing module that collects sensing data. , the signature interface may be a module running in the electronic device implemented using the solution of this embodiment. The signature application can carry the requesting party's information and the indication information of the sensing data. The requesting party's information can include the requesting party's identification information, etc., and the indication information can include the address information of the sensing data in the memory, etc. The signature interface according to the signature application, TEE can be called to determine the requester's permissions based on the requester's information. If not, the call failure message can be returned. After the permissions are passed, the sensing data is read from the memory according to the indication information of the sensing data and the private key is used. Sign, and then return the signature information to the requester.

The process of verifying whether the sensing data has been tampered with corresponds to the above signature process. In this embodiment, whether the sensor data has been tampered with refers to whether the sensor data changes after it is signed. For example, if the signature process is to use the private key to encrypt the sensor data to obtain the signature information, the verification process is to use the public key to decrypt the signature information, and compare the decryption result with the sensor data to determine whether tampering has occurred. If the signature process is to calculate the first hash value of the sensing data and then use the private key to encrypt the first hash value to obtain the signature information, the verification process can be to calculate the second hash value of the sensing data and use the public key in the device certificate. The signature information is decrypted to obtain a first hash value; based on the first hash value and the second hash value, it is determined whether the sensing data has been tampered with after being signed. For example, if the first hash value and the second hash value are the same, it can be determined that the sensing data has not been tampered with after being signed; if they are different, it can be determined that tampering has occurred. If the signature process is to encrypt the sensing data with the private key, and then calculate the first hash value as the signature information for the encryption result, then the verification process is to use the public key to encrypt the sensing data, then calculate the second hash value for the encryption result, and compare Whether the first hash value and the second hash value are the same determines whether the sensing data has been tampered with after being signed.

In this embodiment, there can be multiple execution subjects for verifying whether the sensing data has been tampered with:

(1) Executed on the electronic device itself; in some examples, the method may also include: storing the sensing data, the signature information, the device certificate and the verification certificate; verifying using the verification certificate Whether the device certificate is legal; after the device certificate is verified to be legal, use the signature information and the public key to determine whether the sensing data has been tampered with after being signed.

The solution of this embodiment can be applied to the local end of the electronic device to verify whether the data has been tampered with. In this way, the sensing data does not need to be transmitted to other devices, ensuring the security of the sensing data and satisfying the user's offline requirements. need.

(2) Executed in the verification device; for example, in other examples, the method may further include: providing the sensing data, the signature information, the device certificate and the verification certificate to the verification device, to After the verification device uses the verification certificate to verify whether the device certificate is legal, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.

In this embodiment, the method provided by the electronic device to the verification device may include an offline method. As shown in Figure 1C, it is a schematic diagram of data processing shown in this specification according to an exemplary embodiment. In this embodiment, the verification of whether the data has been tampered may be performed in a verification device. For example, in this embodiment The electronic device can provide sensing data, signature information, device certificates and verification certificates to the offline verification device in an offline state. The verification device does not need to be connected to the Internet. It only needs to use the verification certificate provided by the electronic device to verify whether the device certificate is legal. If legal, the signature information and public key can be further used to determine whether the sensing data has been tampered with after being signed. Therefore, offline verification of the verification device is realized, preventing data leakage problems that may occur when the verification device is connected to the Internet, and meeting the user's offline verification needs.

Among them, the offline method can be that the electronic device and the verification device are connected through a data line, and the above four are transmitted through the data line. Or, in other examples, when the electronic device accesses the storage medium, the method further includes: storing the sensing data, the signature information, the device certificate and the verification certificate in Storage medium, so that after the storage medium is connected to the verification device, the verification device reads the sensing data, the signature information, the device certificate and the verification certificate from the storage medium . In this embodiment, storage media can be used for transmission, that is, the two are transmitted offline, which can ensure the security of sensing data, signature information, device certificates, and verification certificates. It is clear to those skilled in the art that the storage medium may include multiple types, which is not limited in this embodiment. For example, it may include any non-volatile memory, such as flash memory and so on. Examples include disks, SD cards or USB flash drives. In other examples, it is also optional that the electronic device is provided offline to another device in an offline state, and then the electronic device is provided offline to the verification device.

In other examples, the electronic device may send the sensing data, the signature information, the device certificate and the verification certificate online to the verification device. For example, the electronic device may directly send the above four information to the verification device. For example, the electronic device and the verification device may be connected through communication. The communication connection method may include a wired communication connection or a wireless communication connection, or may include a direct connection, or through other means. Indirect connection of equipment and other methods. For example, the electronic device and the verification device can be connected through Bluetooth, near field communication or the Internet. The electronic device sends sensing data, signature information, device certificates and verification certificates to the verification device, which is received by the verification device. In other examples, the transmission may also be indirect, for example, the electronic device provides the above four information to another electronic device, and then the electronic device provides the above four information to the verification device, etc.

Exemplarily, the method of this embodiment may also include any of the following steps: if the verification certificate verifies that the device certificate is illegal, output the first verification failure information; if it is determined that the sensing data has been tampered with after being signed, Outputs the second verification failure message.

Exemplarily, the first verification failure information and/or the second verification failure information is output in a user interface; when the verification process is executed by an electronic device, the user interface may include the user interface of the electronic device, and /or, user interfaces of other terminals communicatively connected to the electronic device. For example, when the verification process is executed by the verification device, the user interface may include the user interface of the verification device and/or the user interface of other terminals that are communicatively connected to the verification device.

Figure 1D and Figure 1E are respectively a schematic diagram of a user interface shown in this specification according to an exemplary embodiment. Figure 1D shows the verification result of the image AAAA. The first verification failure message is "device certificate not verified" as an example. ; Figure 1E shows the verification result of image AAAB, and the second verification failure message is "image has been tampered with" as an example. It is clear to those skilled in the art that the first verification failure information and/or the second verification failure information can be output in any manner, such as text, audio, or video, which is not limited in this embodiment.

Optionally, if it is determined that the sensing data has not been tampered with after being signed, verification passing information can be output. For example, as shown in Figure 1F, which is a schematic diagram of a user interface shown in this specification according to an exemplary embodiment, it shows prompt information that the image AAAC has passed the verification. The prompt information can be implemented in a variety of ways, such as Text, audio or video, etc., this embodiment does not limit this.

In this embodiment, in order to verify the legitimacy of the sensing data, additional signature information, device certificates and verification certificates need to be attached. In practical applications, there are many ways to store sensing data, signature information, device certificates, and verification certificates. For example, the above four can be stored independently. In order to reduce the complexity of data management, any two can be stored as one file, any three can be stored as one file and the other is stored independently, or four can be stored as one file, etc.; in other examples , the device certificate and the verification certificate can also be combined and stored, for example, the two are spliced together as one certificate chain information storage, etc. This embodiment does not limit this.

For example, the verification certificate and the sensing data can be stored in a file. Based on this, the sensing data and the verification certificate can be read from a file, which reduces the complexity of data management.

Exemplarily, at least one of the signature information and the device certificate may be stored in a file with the sensing data, and the sensing data, as well as the signature information and/or Device certificates, thus reducing data management complexity.

For example, an electronic device can collect multiple sensing data, each sensing data corresponding to signature information, for example, n sensing data, that is, there are n signature information; if each sensing data is signed by a private key , then n sensing data corresponds to the same device certificate and verification certificate. The storage method can be: n sensing data, n signature information, i device certificates and j verification certificates, where i and j are both positive integers. , for example, it can be 1 or other integer greater than 1. The specific number can be configured as needed. It can be seen that some storage methods need to store the correspondence between the sensing data and the other three.

Exemplarily, one implementation is that the sensing data, signature information, device certificate and verification certificate are stored in one file. In this way, there is no need to store the corresponding relationship between the above four, and the data management, verification process or transmission to the verification device are all done. Can reduce complexity.

In some examples, sensor data can be stored as raw information. In other examples, sensing data can be stored according to a storage format (encoding format) to reduce redundant information, achieve data compression, etc. Different types of sensing data have different storage formats, and the same type of sensing data can also have different storage formats. In actual applications, the required storage format can be selected according to needs. The sensing data is stored in a file according to the storage format. When any of the sensing data, signature information, device certificate, and verification certificate is stored in the file, it is also stored in the file according to the storage format. In other examples, not following the storage format is also optional. For example, the signature information can be a string. It is also optional to store the string of signature information in the name of the file, so that the sensor data and Corresponding signature information.

In some examples, the verification certificate and the sensing data are stored in a file, which may include: the verification certificate is stored in a free storage location of the sensing data; for example, when storing the sensing data, it may be determined that the sensing data is stored in a file. One or more free storage locations of the sensing data. The free storage location means that the original information of the sensing data is not stored in this location. Therefore, the free storage location is used to store the verification certificate, so that the two can be in one file to facilitate data storage. manage. In the same way, the signature information or the verification certificate can also be stored in the free storage location of the sensing data, or all three of the above can be stored in the free storage location. The free storage locations stored by the three can be the same or different. This can facilitate data management.

In some examples, one or more identifiers are used to indicate the signature information, the device certificate, and the verification certificate, so that any of the above three can be quickly and accurately obtained through the identifiers. For example, one identifier may be used to indicate the above three, or the three may use different identifiers respectively, or the signature information may use one identifier, the device certificate and the verification certificate may use one identifier, etc. This embodiment does not perform this limited.

In some examples, the identification may be determined based on the storage format of the sensing data. For example, when the sensing data is stored in a storage format, some storage formats stipulate the identification of various types of information in the sensing data. In this embodiment, the identification used for signature information, device certificates and verification certificates can be based on the sensing data. The storage format of the data is determined and does not affect the reading of the sensing data from the file. For example, identifications different from those specified in the storage format can be used to indicate the signature information, the device certificate, the verification certificate, etc., which can prevent File decoding error.

In some examples, the storage location of the verification certificate in the file may be determined based on the encoding format of the sensing data. The storage location of the signature information or verification certificate in the file may also be determined based on the encoding format of the sensing data. For example, some storage formats stipulate the storage location of various types of information in sensing data. According to the storage format, you can find a location that does not affect the original information of the sensing data to store any of the above three, thereby preventing file decoding. Something went wrong.

For example, the file may include multiple areas, some of which store sensing data. Any one of the above three storage areas is different from the area where the sensing data is stored. Take the sensor data as an image and the storage format as JPEG (Joint Photographic Experts Group) format as an example. Some JPEG storage formats are divided into areas APPx for storing various metadata of images. This format can be customized by the application. Create some APPx areas to store custom information. This embodiment can create one or more APPx areas for storing any of the above three.

For example, the storage location of the sensing data in the file is different from any of the above three storage locations. Taking RAW images based on the DNG format as an example, this storage format includes a variety of tags used to store image information, as well as tags used to store non-image information such as debug. You can use one or more tags to store any of the above three, such as DNGPrivateData, etc. The above embodiment uses two standard storage formats, JPEG and RAW, as examples. In other examples, the sensing data of some electronic devices is a storage format customized by the equipment manufacturer. Similarly, those skilled in the art know that in practical applications, it can The storage method of any of the above three items in the file is determined according to the storage format, such as storage location or identification, etc., which is not limited in this embodiment.

In practical applications, electronic devices may collect various types of sensing data and can be used in a variety of scenarios. In practical applications, it may be necessary to verify whether the data has been tampered with in some scenarios, and in some scenarios verification may not be required. Based on this, in some examples, the data processing method may be executed when the signature function of the electronic device is activated, thereby facilitating user use.

For example, the signature function may be automatically started by the electronic device. For example, the electronic device may be started after detecting that the preset startup conditions are met. Optionally, the preset startup conditions may be preconfigured by a technician or configured by the user. . In addition, the specific preset startup conditions can also be flexibly configured, for example, they can include conditions of the geographical location of the electronic device, conditions of the type of sensor data collected by the electronic device, and/or conditions of the time at which the sensor data is collected, etc. This embodiment is not limited to this.

In other examples, the process of activating the signature function of the electronic device may include: displaying the signature function activation object on a user interface; the user interface includes the user interface of the electronic device, and/or, and the electronic device. User interfaces of other terminals connected by device communication; in response to the signature function startup object being triggered by the user, the signature function is started. In this embodiment, the signature function may be initiated by the user, and the electronic device or other terminals connected to the electronic device may provide the user with the function of initiating the signature function. In some examples, the electronic device includes a display that can display the user interface. In other examples, the user interface may be displayed on the display of other terminals connected by the electronic device. This embodiment does not limit the other terminals connected by the electronic device. They may include shooting equipment, movable platforms, laser radars, and mobile terminals. , audio equipment or computer equipment, etc.

As shown in FIG. 1G , it is a schematic diagram of an electronic device 11 and a remote control device 12 according to an exemplary embodiment of this specification, in which the electronic device 11 is communicatively connected with the remote control device 12 . As shown in Figure 1H, it is a schematic diagram of a user interface of a remote control device according to an exemplary embodiment of this specification. The signature function activation object is specifically the "Confirm Open" button as an example. The user clicks this button to activate the camera. Signature function.

For example, the sensing data that needs to be verified as to whether it has been tampered may be of various types, and the user interface may also provide a function to enable the signature function for different types of sensing data, which is not limited in this embodiment.

Next, an embodiment will be used to illustrate.

Currently, there is no standard solution for verifying whether documents have been tampered with in fields such as cameras or drones. In a large number of industrial application scenarios and personal application scenarios, there is a need to verify whether the sensor data collected by electronic devices has been tampered with. For example, data such as images or videos collected by the drone's camera module, flight trajectory data collected by the drone's flight control module, point cloud data collected by the lidar mounted on the drone, etc., users have strong verification Whether the requirement has been tampered with.

In outdoor scenarios, it is very common for electronic devices to be unable to connect to the Internet. In some specific scenarios, data such as images, videos or point clouds themselves are sensitive and confidential information, and offline verification of whether the data has been tampered with is a necessary function. However, the traditional PKI system is mainly used by browsers to verify website identities. In the scenario of browser browsing web pages, the verifier must have the ability to connect to the Internet.

Therefore, this specification provides embodiments of offline verification of sensory data collected by electronic devices.

As the producer of data, electronic devices hold certificates issued by PKI. Among them, the PKI issues certificates and manages certificates to trusted electronic devices. Electronic devices can include: drones, action cameras, cameras or robots, etc. The electronic device includes a sensing module for collecting sensing data. For example, a camera module is used to capture images or videos. The electronic device can apply the solution of this embodiment, and the generated sensing data can be verified offline by the verification party. Of course, online verification is also possible.

Take images as an example. In the drone scenario, the working process of the drone's camera module is: collecting optical signals, converting the optical signals into electrical signals, and processing the electrical signals through the ISP circuit to generate image data and store it in the memory. . The operating system encodes and writes the image data in the memory to a storage medium such as a hard disk according to the preset image storage format, thereby generating an image file containing image information.

The certificate held by the electronic device includes: a certificate chain issued by PKI for the public and private key pair held by the electronic device; the certificate chain includes: the device certificate, and also includes the certificate of the first-level subordinate CA that issued the device certificate (i.e., the first-level sub-certificate ), and the root CA self-signed certificate (i.e. root certificate) that issues the first-level sub-certificate. This certificate chain provides complete device trust. There are three certificates in this embodiment, that is, the length of the certificate chain is three. In actual applications, the number of certificates can be adjusted as needed.

Optionally, the private key can be stored in the TEE of the electronic device to ensure the security of the private key. The certificate chain is public information that can prove the correspondence between the private key held by the device and the electronic device. Its storage location can be configured as needed.

As shown in FIG. 2A , it is a schematic diagram of data processing of the drone in this embodiment, which shows the process from image collection by the camera to file storage to the storage medium. As an example, the method of the aforementioned embodiment can be a data processing module running in an electronic device. During the operation of the electronic device, the camera of the device can call the TEE of the device after completing the conversion of optical signals into images in the memory; TEE After confirming that the calling permission is legal, you can read the image in the memory, use the private key to sign the data that needs to be verified, and generate signature information. After obtaining the signature information, the certificate chain can be obtained, and the sensor data, signature information and certificate chain can be stored as a file in the storage medium according to the storage format of the sensor data. The file can be verified by the verifier whether it has been tampered with after signing. That is to say, the signature process in this embodiment is executed before the data collected by the sensing module is stored as a file. The verification party can be a drone, or it can be a schematic diagram as shown in Figure 2B. The drone 21 can transmit the file to the verification device 22 for verification.

Among them, during the image generation process, the private key needs to be called, and permissions can be set for the caller of the private key. For example, the camera is only allowed to call during the process of taking pictures and generating image files, and other modules in the electronic device do not have the right to call this interface.

In this embodiment, the verifier is the object that verifies the legitimacy of the sensing data. The verifier can be the electronic equipment of the above-mentioned manufacturer, or other equipment, including but not limited to: drones, remote controls, cloud platforms, mobile applications or PC interactive software, offline software, or Online verification server, etc. After obtaining the file stored in the electronic device, the verifier can parse the signature information and certificate chain in the file, and verify whether the file has been forged or maliciously tampered with by verifying the legitimacy of the certificate chain and signature.

Next, the sensor data is specifically an image as an example for explanation.

After the electronic device obtains the signature information of the image, it needs to be stored as an image file.

(1) For image files in JPEG storage format, the generation method may be as follows.

According to the JPEG storage format, JPEG files can be simply divided into the following areas:

(1)Jpeg Head: It is a fixed 2byte 0xFFD8

(2)APPx: An area used to store various metadata of images, starting with marker 0xFFEx.

All APP areas are stored in a continuous buffer.

For example, there can be multiple APP1s, and different APP1s are distinguished by identifier_code, which is a string ending with '\0'. Each APP1 has a length bit of 2 bytes, which is used to mark the length of this APP area except the marker.

Take jpeg as an example. These two APP1s store exif and xmp respectively. There is a very small thumb nail in exif, and the thumb nail is also a complete photo.

APP2 stores screen image information.

In some examples, one or more APP areas can be created to store custom information according to the JPEG storage format. For example, APP7 can be created to store debug information, etc.

(3) Stream area: stores the streams of main, screen, etc., that is, the encoded image, until the end of the file.

As can be seen from the above embodiment, the stored image files in JPEG format are stored in the form of partitions, and each partition starts with a marker field:

Optionally, this embodiment can create two new APP areas in the original fields for placing signature information and certificate chains. The storage location of the signature information and certificate chains can be behind all APPs, that is, the newly created APP area. Located behind all APPs, for example:

1)marker=0xFFE7, identifier_code="SIGNATURE"; this area stores signature information

2) marker = 0xFFE7, identifier_code = "CERTIFICATION"; this area stores the certificate chain.

After adding this part of the field, the JPEG image file generated by the drone is the signed image file. At the same time, the image file also has a certificate chain. After the verifier obtains the image file, it can parse the file and obtain the signature information and certificate chain, thereby completing the verification of the image signature and proving whether the image has been tampered with.

It can be understood that in the JPEG encoding format, the marker field indicating the signature information and certificate chain can also choose other fields, and the identifier_code can also choose other fields, as long as it is not the same as the field used in the existing encoding standard. .

(2) For image files in RAW storage format, the generation method may be as follows.

Take RAW images based on the DNG format as an example. DNG is an extension of the Adobe format based on TIFF, with the purpose of unifying the format of RAW images. TIFF files consist of the following three basic members:

(1) IFH: image file header, which is the file header, contains the byte order and type of the file, and also contains the address of the first IFD.

(2)IFD: image file directory, that is, a directory. There can be multiple directories, organized in a linked list-like form. In addition to the cnt at the beginning of each directory, the remaining part is the IFD Entry, which is the tag.

(3)tag: also called IFD Entry. Each tag contains tag_id, data type, number of data, and value. For types whose data does not exceed 4 bytes, the value can be read directly; if the data length is greater than 4 bytes, the value indicates the address where the data is located.

Each tag will be marked with a piece of information, such as width, height, image address or other metadata. By identifying the tag, you can find and read the location of image information, thumbnails, etc., and you can also find customized information.

Based on this, according to the RAW storage format, it includes a tag called DNGPrivateData, which only saves 3a_debug_info. This embodiment can expand the tag based on the original data. For example, the signature information and certificate chain can be added after 3a_debug_info. For example, this tag can be expanded to:

The above sig and crt_list represent the signature information and the identity of the certificate chain respectively. It is understandable that the sig field can also choose other names, and the crt_list can also choose other fields, as long as they are not the same as the fields used in existing standards. In addition, the positions of these two fields can be stored in the DNGPrivateData segment or in other tags, which is not limited in this embodiment.

After adding the above two fields to the above tag, the RAW image file generated by the drone is the signed image file. At the same time, the image file also has a certificate chain. After the verifier obtains the image file, it can parse the file and obtain the signature information and certificate chain, thereby completing the verification of the image signature and proving whether the image has been tampered with.

It can be seen from the above embodiments that the electronic device can generate an image file with a signature and a certificate chain in JPEG format or RAW format through the above two methods. Since the image file file is not affected, it can be used in the same way as the normal JPEG format or RAW format. Use, store and transfer the same image files.

As shown in Figure 2C, it is a flow chart of another data processing shown in this specification according to an exemplary embodiment. When the electronic device is running, the data processing method of this embodiment involves a camera, a memory, a signature interface and a storage medium (Fig. (taking SD card as an example), including the following steps:

Step 201: The camera collects image data; the collected image data is written into the memory.

Step 202: The camera passes the memory address to the signature interface to request a signature.

Step 203: The signature interface accesses the image data according to the memory address.

Step 204: The signature interface generates signature information based on the image data.

Step 205: The signature interface returns signature information to the camera.

Step 206: The camera requests certificate chain information from the signature interface.

Step 207: The signature interface returns the certificate chain information to the camera.

Step 208: The camera writes the image with signature information and certificate chain information into the memory.

Step 209: Save the image with signature information and certificate chain information in the memory to the SD card.

In other examples, as shown in Figure 2D, the signature information sig and the certificate chain information crt_list can also be stored separately. For example, in the figure, the image file, signature information, and certificate chain information are stored independently for verification by the verifier.

Taking offline verification as an example, the verifier can obtain the image file file from the electronic device (or other declarer), parse the file, and obtain the signed data data, signature information sig, and certificate chain crt_list.

An offline verification program can be run in the verification device to verify whether the three certificates in the certificate chain crt_list are legal:

First, verify whether the root certificate in the file is legal. If it is not legal, it will directly return that the verification failed. For example, the root certificate is the starting point of the trust chain, and the obtained root certificate can be considered legitimate. Alternatively, the verification device can obtain the root certificate from the Certificate Authority in advance and store it locally, and compare whether the locally stored root certificate is consistent with the root certificate in the file. If they are consistent, it is determined to be legal.

If the root certificate is legal, continue to verify whether the first-level sub-certificate is legal, that is, use the root certificate to verify the first-level sub-certificate. If it is illegal, it will directly return that the verification failed.

If the first-level sub-certificate is legal, continue to verify whether the device certificate is legal, that is, use the first-level sub-certificate to verify the device certificate. If it is illegal, it will directly return that the verification failed.

If the device certificate verification is legal, verify the signature information sig. That is: obtain the public key key_pub in the device certificate; calculate the hash value hash for the data segment data, use key_pub to decrypt sig, and obtain hash’. If the hash is consistent with hash’, the verification is returned. Otherwise, verification failure is returned.

Through the above method, the verifier can verify whether the photo is forged or maliciously tampered with.

In other examples, after the verification requester obtains the photo file file from the device (or other declarer), parses the file, obtains the signature sig, and signs the device certificate chain crt_list, online verification is also optional. As shown in Figure 2E, it is a flow chart of online verification according to an exemplary embodiment of this specification, including:

Step 211: The verification device sends a verification request for the file to the security server. For example, the authentication device sends file, sig, crt_list to the security server.

Step 212: The security server requests the OCSP (Online Certificate Status Protocol) server to check whether the certificate in the certificate chain sent by the verification device has been revoked.

Step 213: OCSP checks whether the certificate in the certificate chain has been revoked.

Step 214: OCSP returns the inspection results. If it is revoked, it will directly return that the verification failed. Otherwise proceed to the next step.

Step 215: The security server verifies the file.

For example, the security server sequentially verifies whether the three certificates in the certificate chain crt_list are legal. It first verifies whether the root certificate is legal. If it is not legal, it directly returns the verification failure. Otherwise, continue to verify whether the first-level sub-certificate is legal. If it is not legal, it will directly return that the verification failed. Otherwise, continue to verify whether the device certificate is legal. If it is not legal, it will directly return that the verification failed. If it is legal, obtain the public key key_pub in the device certificate. Parse the file file, obtain the signed data segment data, calculate its hash value hash, use key_pub to decrypt sig, and obtain hash’. Depending on whether the hash is consistent, it is determined whether the verification is passed. If so, the verification passes, otherwise the verification fails.

Step 216: The security server returns the verification result to the verification device.

It can be seen from the above embodiment that in this embodiment, the drone camera module can generate a communication file carrying signature information and a certificate chain. Any verifier can use a single photo file to verify its ownership and whether it has been maliciously tampered with offline.

It can be seen that this embodiment has the following advantages:

Security: In this embodiment, legal signatures cannot be forged by attackers. This solution is based on cryptographic characteristics and has credible security strength. The signature process is initiated by the camera, and the authentication is completed by the device TEE, ensuring that the execution of the image signature process is trustworthy and It is irreversible and can meet the needs of anti-counterfeiting of image files and verification of whether they have been tampered with. It can prove to third parties that image files in JPEG, RAW and other formats have not been forged or illegally tampered with.

High flexibility: In this embodiment, the drone can independently complete the signature of the image. The verifier can independently complete the image signature verification without additional certificate transfer or public key query operations; in industry application scenarios, the verifier holds the root certificate and can complete the verification of the legality of the photo. The certificate is public information and can be obtained from public sources.

Ease of use: In this embodiment, encoding formats such as JEPG and RAW are used to encode the signature information and certificate chain into the original image file, which does not affect the normal reading of the file and does not add other files. The image file generated based on this embodiment is as easy to use as a general image file.

Reliability: In this embodiment, the signature private key is protected by TEE and does not need to be disseminated. The photo generation process is irreversible, reducing the overall risk of the solution. Reliability relies on mathematically verifiable strong security guarantees. Users can choose whether to With this feature turned on, the overall solution is reliable.

As shown in Figure 3, it is a flow chart of another data processing method shown in this specification according to an exemplary embodiment. This embodiment takes the verification device as an example. The method includes the following steps:

In step 302, sensing data, signature information, device certificate and verification certificate are obtained from the electronic device.

Wherein, the sensing data is collected by an electronic device, and the signature information is obtained by the electronic device signing the sensing data using a private key stored locally on the electronic device, and the device certificate includes the same information as the private key. Corresponding public keys, the verification certificate is used to verify the device certificate.

In step 304, use the verification certificate to verify whether the device certificate is legal.

In step 306, after the device certificate is verified to be legal, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.

In this embodiment, the verification device may directly obtain the sensing data, signature information, device certificate and verification certificate from the electronic device, or may obtain it indirectly, which is not limited in this embodiment.

In some examples, if the sensing data provided by the electronic device to the verification device has not been tampered with, the verification device can use the verification certificate to verify that the device certificate is legitimate, and then use the signature information and the public key to determine the received sensing data. Has not been tampered with since it was signed. If the electronic device has tampered with the sensing data after signing it, the sensing data received by the verification device is the tampered data. After the verification device uses the verification certificate to verify that the device certificate is legitimate, it can use the signature information and public key to determine the current The received sensing data is different from the sensing data of the electronic device signature, that is, the sensing data of the electronic device signature has been tampered with after being signed. For the specific implementation process, reference may be made to the foregoing embodiments, which will not be described again here.

In some examples, the verification certificate and the sensor data are stored in a file.

In some examples, at least one of the signature information and the device certificate is stored in a file with the sensing data.

In some examples, the storage of the verification certificate and the sensing data in a file includes:

The verification certificate is stored in a free storage location of the sensing data.

In some examples, the processor 301 executes at least one of the signature information and the device certificate, and stores the sensing data in a file, including:

At least one of the signature information and the device certificate is stored in a free storage location of the sensing data.

In some examples, one or more identifiers are used to indicate the signature information, the device certificate, and the verification certificate.

In some examples, the identification is determined based on the encoding format of the sensing data.

In some examples, the storage location of the verification certificate in the file is determined according to the storage format of the sensing data.

In some examples, the storage location of at least one of the signature information and the device certificate in the file is determined according to the storage format of the sensing data.

In some examples, the sensing data includes any of the following: images, audio, point clouds, or movement trajectory data of the electronic device.

In some examples, the verification certificate includes a root certificate and the device certificate is signed by a root certificate; or,

The verification certificate includes: a root certificate, and one or more levels of sub-certificates; among the one-level or multi-level sub-certificates, the highest level sub-certificate is issued by the root certificate, and each of the other levels of sub-certificates The subcertificate of is issued by the subcertificate of the previous level, and the lowest level subcertificate is used to issue the device certificate.

In some examples, the processor 301 executes that the device certificate is signed by a root certificate, and using the verification certificate to verify whether the device certificate is legal includes: using the root certificate to verify whether the device certificate is legal; or,

The verification certificate includes: a root certificate, and one or more levels of sub-certificates; among the one-level or multi-level sub-certificates, the highest level sub-certificate is verified by the root certificate to see whether it is legal, and each of the other levels of sub-certificates is The sub-certificate of one level is verified by the sub-certificate of the previous level to see whether it is legitimate, and the sub-certificate of the lowest level is used to verify whether the device certificate is legitimate.

In some examples, the verification certificate is issued by a certificate authority.

In some examples, the device certificate carries device information of the electronic device.

In some examples, the processor 301 also performs any of the following steps:

If the verification certificate verifies that the device certificate is illegal, output the first verification failure message;

If it is determined that the sensing data has been tampered with after being signed, a second verification failure message is output.

In some examples, the first verification failure information and/or the second verification failure information is output in a user interface; the user interface includes a user interface of the verification device, and/or is connected to the verification device. User interface of other terminals connected by communication.

In some examples, the processor 301 obtains sensing data, signature information, device certificates and verification certificates from the electronic device, including:

Obtain sensing data, signature information, device certificates and verification certificates sent online by electronic devices; and/or,

In the offline state, obtain sensing data, signature information, device certificates and verification certificates provided by the electronic device in the offline state.

In some examples, the processor 301 also performs the acquisition of sensing data, signature information, device certificates and verification certificates from the electronic device, including:

When connected to the storage medium, the sensing data, signature information, device certificate and verification certificate stored in the storage medium by the electronic device are read from the storage medium.

In some examples, the verification device includes:

Photography equipment, movable platforms, remote control equipment, mobile terminals, audio equipment, computer equipment or servers.

The above data processing method embodiments can be implemented by software, or can be implemented by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running them through the data processing processor where it is located. From a hardware level, as shown in Figure 4, it is a hardware structure diagram of the data processing device 400 that implements the data processing method of this embodiment. In addition to the processor 401 and the memory 402 shown in Figure 4, the embodiment The data processing equipment used to implement this data processing method usually may also include other hardware according to the actual functions of the data processing equipment, which will not be described again.

In this embodiment, the data processing device locally stores a private key, a device certificate, and a verification certificate for verifying the device certificate. The device certificate includes a public key, and the private key corresponds to the public key. The processor 401 implements the following steps when executing the computer program:

Obtain sensor data collected by the electronic device;

Sign the sensing data with the private key to obtain signature information;

In some examples, the processor 401 also performs:

store the sensing data, the signature information, the device certificate and the verification certificate;

In some examples, the processor 401 also performs:

Provide the sensing data, the signature information, the device certificate and the verification certificate to the verification device, so that the verification device uses the verification certificate to verify whether the device certificate is legal, and then uses the signature information and the public key to determine whether the sensing data has been tampered with after being signed.

In some examples, the verification certificate and the sensing data are stored in a file.

In some examples, the processor 401 executes at least one of the signature information and the device certificate, and stores the sensing data in a file, including:

In some examples, the processor 401 executes the step of signing the sensing data using the private key to obtain signature information, including:

The sensing module that collects the sensing data calls the private key to sign the sensing data to obtain signature information.

In some examples, the electronic device includes a trusted execution environment and the private key is stored in the trusted execution environment.

In some examples, the sensing module that collects the sensing data calls the private key to sign the sensing data to obtain signature information, including:

After the sensing module that collects the sensing data passes the verification of the trusted execution environment, it calls the private key to sign the sensing data to obtain the signature information.

In some examples, the sensing data includes data collected by one or more sensing modules in the electronic device.

In some examples, the sensing module includes any of the following: a camera module, an audio module, a point cloud module, or a mobile control module.

In some examples, the processor 401 also performs any of the following steps:

If the verification certificate verifies that the device certificate is illegal, output the first verification failure information;

In some examples, the first verification failure information and/or the second verification failure information is output in a user interface; the user interface includes a user interface of the electronic device, and/or is connected to the electronic device. User interface of other terminals connected by communication.

In some examples, the processor 401 performs providing the sensing data, the signature information, the device certificate, and the verification certificate to a verification device, including:

Send the sensing data, the signature information, the device certificate and the verification certificate online to the verification device; and/or,

In an offline state, the sensing data, the signature information, the device certificate and the verification certificate are provided to the verification device in the offline state.

In some examples, when the electronic device accesses the storage medium, the processor 401 also executes:

The sensing data, the signature information, the device certificate and the verification certificate are stored in a storage medium, so that after the storage medium is connected to the verification device for communication connection, the verification device obtains the data from the verification device. The storage medium reads the sensing data, the signature information, the device certificate and the verification certificate.

In some examples, the data processing apparatus is executed when the signature function of the electronic device is activated.

In some examples, the processor 401 performs a process of initiating the signature function of the electronic device, including:

Display the signature function activation object on the user interface; the user interface includes the user interface of the electronic device, and/or the user interface of other terminals that are communicatively connected to the electronic device;

In response to the signature function activation object being triggered by the user, the signature function is activated.

In some examples, the electronic device includes any of the following:

Photography equipment, movable platforms, lidar, mobile terminals, audio equipment or computer equipment.

As shown in Figure 5, it is another hardware structure diagram of a data processing device 400 for implementing the data processing method of this embodiment. In addition to the processor 501 and the memory 502 shown in Figure 5, the other hardware components used in the embodiment are used to implement the data processing method. The data processing equipment of this data processing method usually can also include other hardware according to the actual functions of the data processing equipment, which will not be described again.

In this embodiment, the processor 501 implements the following steps when executing the computer program:

Obtain sensing data, signature information, device certificates, and verification certificates from electronic devices. Wherein, the sensing data is collected by an electronic device, and the signature information is obtained by the electronic device signing the sensing data using a private key stored locally on the electronic device, and the device certificate includes the same information as the private key. Corresponding public keys, the verification certificate is used to verify the device certificate.

Use the verification certificate to verify whether the device certificate is legal.

In some examples, at least one of the signature information and the device certificate is stored in a file with the sensing data, including:

In some examples, the device certificate is issued by a root certificate, and using the verification certificate to verify whether the device certificate is legal includes: using the root certificate to verify whether the device certificate is legal; or,

In some examples, the processor 501 implements the following steps when executing the computer program:

In some examples, obtaining sensing data, signature information, device certificates and verification certificates from electronic devices includes:

After communicating with the electronic device, obtain the sensing data, signature information, device certificate and verification certificate sent by the electronic device; and/or,

After communication connection with the storage medium, the sensing data, signature information, device certificate and verification certificate stored in the storage medium by the electronic device are read from the storage medium.

In some examples, the verification device includes:

As shown in Figure 6, this embodiment also provides an electronic device. The electronic device includes a processor 61, a memory 62, and a computer program stored on the memory and executable by the processor. The processor executes The computer program implements an embodiment of the aforementioned data processing method.

As shown in Figure 7, this embodiment also provides a verification device. The verification device also includes a processor 71, a memory 71, and a computer program stored on the memory and executable by the processor. The processor The aforementioned embodiments of the data processing method are implemented when the computer program is executed.

As shown in Figure 8, this embodiment also provides a drone 80, which includes:

Body 81;

A power system 811 is provided in the fuselage 81 and is used to provide power for the drone;

And, a processor 812 and a memory 813 are provided in the body 81. The memory stores a computer program that can be executed by the processor. When the processor executes the computer program, the aforementioned data processing method is implemented. Example.

This embodiment also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed, the embodiment of the foregoing data processing method is implemented.

This embodiment also provides a computer program product, including a computer program that implements the foregoing embodiment of the data processing method when executed by a processor.

Embodiments of the present description may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Storage media available for computers include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device.

The steps of the various methods above are divided just for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process, are within the scope of protection of this application. Additionally, the processes depicted in the figures do not necessarily require the specific order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain implementations.

The description of "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of this specification. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

As for the device embodiment, since it basically corresponds to the method embodiment, please refer to the partial description of the method embodiment for relevant details. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. The terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article or apparatus including a list of elements includes not only those elements but also others not expressly listed elements, or elements inherent to such process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The methods and devices provided by the embodiments of the present invention have been introduced in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present invention. The description of the above embodiments is only used to help understand the method and its implementation of the present invention. Core idea; at the same time, for those of ordinary skill in the art, there will be changes in the specific implementation and application scope based on the idea of the present invention. In summary, the content of this description should not be understood as a limitation of the present invention. .

Other embodiments of the present disclosure will readily occur to those skilled in the art, upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any modifications, uses or adaptations of this specification that follow the general principles of this specification and include common knowledge or common technical means in the technical field that are not applied in this specification. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

Claims

A data processing method, the method is applied to an electronic device, characterized in that the electronic device locally stores a private key, a device certificate and a verification certificate for verifying the device certificate, and the device certificate includes a public key, The private key and the public key correspond to each other, and the method includes:

Obtain sensor data collected by the electronic device;

Sign the sensing data with the private key to obtain signature information;

Wherein, after the verification certificate verifies that the device certificate is legitimate, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.
The method of claim 1, further comprising:

store the sensing data, the signature information, the device certificate and the verification certificate;

Use the verification certificate to verify whether the device certificate is legal;

After the device certificate is verified to be legal, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.
The method of claim 1, further comprising:

Provide the sensing data, the signature information, the device certificate and the verification certificate to the verification device, so that the verification device uses the verification certificate to verify whether the device certificate is legal, and then uses the signature information and the public key to determine whether the sensing data has been tampered with after being signed.
The method of claim 1, wherein the verification certificate and the sensing data are stored in a file.
The method according to claim 1 or 4, characterized in that at least one of the signature information and the device certificate is stored in a file with the sensing data.
The method according to claim 4, wherein the storage of the verification certificate and the sensing data in one file includes:

The verification certificate is stored in a free storage location of the sensing data.
The method according to claim 5 or 6, characterized in that at least one of the signature information and the device certificate is stored in a file with the sensing data, including:

At least one of the signature information and the device certificate is stored in a free storage location of the sensing data.
The method according to claim 1, characterized in that one or more identifiers are used to indicate the signature information, the device certificate and the verification certificate.
The method according to claim 8, characterized in that the identification is determined according to the encoding format of the sensing data.
The method according to claim 4, characterized in that the storage location of the verification certificate in the file is determined according to the storage format of the sensing data.
The method according to claim 5 or 10, characterized in that the storage location of at least one of the signature information and the device certificate in the file is determined according to the storage format of the sensing data.
The method according to claim 1, wherein the step of signing the sensing data using the private key to obtain the signature information includes:

The sensing module that collects the sensing data calls the private key to sign the sensing data to obtain signature information.
The method according to claim 1 or 12, characterized in that the electronic device includes a trusted execution environment, and the private key is stored in the trusted execution environment.
The method according to claim 13, characterized in that the sensing module that collects the sensing data calls the private key to sign the sensing data to obtain signature information, including:

After the sensing module that collects the sensing data passes the verification of the trusted execution environment, it calls the private key to sign the sensing data to obtain the signature information.
The method of claim 1, wherein the sensing data includes data collected by one or more sensing modules in the electronic device.
The method of claim 1, wherein the sensing data includes any of the following: images, audio, point clouds, or movement trajectory data of the electronic device.
The method according to claim 12, 14 or 15, characterized in that the sensing module includes any of the following: a camera module, an audio module, a point cloud module or a movement control module.
The method of claim 1, wherein the verification certificate includes a root certificate, and the device certificate is issued by a root certificate; or,

The verification certificate includes: a root certificate, and one or more levels of sub-certificates; among the one-level or multi-level sub-certificates, the highest level sub-certificate is issued by the root certificate, and each of the other levels of sub-certificates The subcertificate of is issued by the subcertificate of the previous level, and the lowest level subcertificate is used to issue the device certificate.
The method of claim 1, wherein the verification certificate is issued by a certificate authority.
The method of claim 1, wherein the device certificate carries device information of the electronic device.
The method according to any one of claims 1 to 20, characterized in that the method further includes any of the following steps:

If the verification certificate verifies that the device certificate is illegal, output the first verification failure message;

If it is determined that the sensing data has been tampered with after being signed, second verification failure information is output.
The method of claim 21, wherein the first verification failure information and/or the second verification failure information is output in a user interface; the user interface includes a user interface of the electronic device, and /or, user interfaces of other terminals communicatively connected to the electronic device.
The method of claim 3, wherein providing the sensing data, the signature information, the device certificate and the verification certificate to a verification device includes:

Send the sensing data, the signature information, the device certificate and the verification certificate online to the verification device; and/or,

In an offline state, the sensing data, the signature information, the device certificate and the verification certificate are provided to the verification device in the offline state.
The method according to claim 1, 3 or 23, characterized in that when the electronic device accesses the storage medium, the method further includes:

The sensing data, the signature information, the device certificate and the verification certificate are stored in a storage medium, so that after the storage medium is accessed to the verification device, the verification device obtains data from the storage medium. The medium reads the sensing data, the signature information, the device certificate and the verification certificate.
The method according to claim 1, characterized in that the data processing method is executed when the signature function of the electronic device is activated.
The method according to claim 25, characterized in that the process of activating the signature function of the electronic device includes:

Display the signature function activation object on the user interface; the user interface includes the user interface of the electronic device, and/or the user interface of other terminals that are communicatively connected to the electronic device;

In response to the signature function activation object being triggered by the user, the signature function is activated.
The method according to claim 1, wherein the electronic device includes any of the following:

Photography equipment, movable platforms, lidar, mobile terminals, audio equipment or computer equipment.
A data processing method, characterized in that the method is applied to verification equipment, and the method includes:

Obtain sensing data, signature information, device certificate and verification certificate from the electronic device, wherein the sensing data is collected by the electronic device, and the signature information is the electronic device using the private key stored locally on the electronic device. Obtained from the sensing data signature, the device certificate includes a public key corresponding to the private key, and the verification certificate is used to verify the device certificate;

Use the verification certificate to verify whether the device certificate is legal;

After the device certificate is verified to be legal, the signature information and the public key are used to determine whether the sensing data has been tampered with after being signed.
The method of claim 28, wherein the verification certificate and the sensing data are stored in a file.
The method according to claim 28 or 29, characterized in that at least one of the signature information and the device certificate is stored in a file with the sensing data.
The method according to claim 29, wherein the storage of the verification certificate and the sensing data in one file includes:

The verification certificate is stored in a free storage location of the sensing data.
The method according to claim 30 or 31, characterized in that at least one of the signature information and the device certificate is stored in a file with the sensing data, including:

At least one of the signature information and the device certificate is stored in a free storage location of the sensing data.
The method according to claim 28, characterized in that one or more identifiers are used to indicate the signature information, the device certificate and the verification certificate.
The method according to claim 33, characterized in that the identification is determined according to the encoding format of the sensing data.
The method according to claim 29, characterized in that the storage location of the verification certificate in the file is determined according to the storage format of the sensing data.
The method according to claim 30 or 35, characterized in that the storage location of at least one of the signature information and the device certificate in the file is determined according to the storage format of the sensing data.
The method of claim 28, wherein the sensing data includes any of the following: images, audio, point clouds, or movement trajectory data of the electronic device.
The method of claim 28, wherein the verification certificate includes a root certificate, and the device certificate is issued by a root certificate; or,

The verification certificate includes: a root certificate, and one or more levels of sub-certificates; among the one-level or multi-level sub-certificates, the highest level sub-certificate is issued by the root certificate, and each of the other levels of sub-certificates The subcertificate of is issued by the subcertificate of the previous level, and the lowest level subcertificate is used to issue the device certificate.
The method of claim 38, wherein the device certificate is issued by a root certificate, and using the verification certificate to verify whether the device certificate is legal includes: using the root certificate to verify whether the device certificate is legal. legal; or,

The verification certificate includes: root certificate, and one or more levels of sub-certificates;

Among the one-level or multi-level sub-certificates, the highest-level sub-certificate is verified by the root certificate to see whether it is legal. The sub-certificates of each level among other levels of sub-certificates are verified by the sub-certificate of the previous level. The sub-certificate is used to verify whether the device certificate is legitimate.
The method according to claim 28, characterized in that the verification certificate is issued by a certificate authority center.
The method of claim 28, wherein the device certificate carries device information of the electronic device.
The method according to claim 28, characterized in that the method further includes any of the following steps:

If the verification certificate verifies that the device certificate is illegal, output the first verification failure message;

If it is determined that the sensing data has been tampered with after being signed, a second verification failure message is output.
The method of claim 42, wherein the first verification failure information and/or the second verification failure information is output in a user interface; the user interface includes a user interface of the verification device, and /or, user interfaces of other terminals communicatively connected to the verification device.
The method according to claim 28, characterized in that said obtaining sensing data, signature information, device certificate and verification certificate from the electronic device includes:

Obtain sensing data, signature information, device certificates and verification certificates sent online by electronic devices; and/or, in an offline state, obtain sensing data, signature information, device certificates and verification certificates provided by electronic devices in an offline state .
The method according to claim 28 or 44, characterized in that the obtaining sensing data, signature information, device certificate and verification certificate from the electronic device includes:

When the storage medium is accessed, the sensing data, signature information, device certificate and verification certificate stored in the storage medium by the electronic device are read from the storage medium.
The method according to claim 28, characterized in that the verification device includes:

Photography equipment, movable platforms, remote control equipment, mobile terminals, audio equipment, computer equipment or servers.
A data processing device, characterized in that the device includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, claim 1 is realized. to any of the methods described in 46.
An electronic device, characterized in that the electronic device includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, claim 1 is realized. to any of the methods described in 27.
A verification device, characterized in that the verification device further includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the claims are realized. Any of the methods described in 28 to 46.
An unmanned aerial vehicle, characterized in that the unmanned aerial vehicle includes:

body;

A power system, located in the fuselage, used to provide power for the drone;

And, a processor and a memory provided in the fuselage, the memory stores a computer program that can be executed by the processor, and when the processor executes the computer program, any one of claims 1 to 46 is realized. method described.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed, the method of any one of claims 1 to 46 is implemented.
A computer program product, characterized by comprising a computer program that implements the method of any one of claims 1 to 46 when executed by a processor.