WO2021082463A1

WO2021082463A1 - Data processing method and apparatus, electronic device and storage medium

Info

Publication number: WO2021082463A1
Application number: PCT/CN2020/095955
Authority: WO
Inventors: 王若晖; 胡颖凡
Original assignee: 深圳市商汤科技有限公司
Priority date: 2019-10-28
Filing date: 2020-06-12
Publication date: 2021-05-06
Also published as: KR20210096230A; TW202117707A; JP2022510660A; TWI755890B; CN110569329A; CN110569329B

Abstract

A data processing method and apparatus, an electronic device and a storage medium. The method comprises: determining a sample list comprising a plurality of samples according to the fields of the samples and parameter values corresponding to the fields (S11); determining description information corresponding to the sample list according to the data types of the samples in the sample list (S12); and determining a sample set according to the sample list and the description information (S13).

Description

Data processing method and device, electronic equipment and storage medium

Cross-references to related applications

This application is filed based on the Chinese patent application with the application number 201911031910.5 and the filing date on October 28, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into this application.

Technical field

In machine learning, especially when multiple data sets need to be processed, the difficulty of reading data will increase. In related technologies, there is no unified way for the storage of data sets, and the storage of data sets is usually to adapt to the structure of the data set itself. For example, the data storage method of the Da taFrame type in the pandas library of the python language is powerful, but the supported data types are limited, and it is mainly for professional users. In addition, the read operation mainly focuses on the data itself.

Summary of the invention

The embodiments of the present disclosure propose a data processing method and device, electronic equipment, and storage medium.

According to an aspect of the embodiments of the present disclosure, there is provided a data processing method, including: determining a sample list including a plurality of samples according to a field of a sample and a parameter value corresponding to the field; The data type is used to determine the description information corresponding to the sample list; and the sample set is determined according to the sample list and the description information.

According to the data processing method of the embodiment of the present disclosure, the sample set used to store each sample and the corresponding description information can be determined, multiple types of samples can be stored in a unified manner, and the sample set includes description information describing the data type, It is convenient to store and read sample data, can support multiple types of data storage and read, and is easy to understand.

In a possible implementation manner, each sample has a sample index, and the determining a sample list including multiple samples according to the field of the sample and the parameter value corresponding to the field includes: determining the field as all the samples. The column coordinates of the sample list, the sample index is determined as the row coordinates of the sample list; the parameter value corresponding to the field is objectified to obtain the object corresponding to the sample; the data in the object Add to the storage area corresponding to the column coordinates and the row coordinates to obtain the sample list. In this way, the parameter values and/or annotations of the samples can be objectified, and samples of multiple data types can be stored through the sample list, which is conducive to the rapid storage and reading of samples, and reduces the resource occupation during storage and reading. , And can support multiple types of data storage, adapt to the storage of more complex types of data sets.

In a possible implementation manner, the description information includes description information of how to read the sample, and the description information corresponding to the sample list is determined according to the data type of the sample in the sample list, It includes: determining a reading method for reading the sample according to the data type of the sample in the sample list; obtaining the description information according to the data type of the sample and the corresponding reading method. In this way, a variety of sample reading methods can be determined, which is conducive to supporting multiple types of data storage and access, and adapts to the storage and access of more complex types of data sets.

In a possible implementation manner, the description information further includes description information of a storage location of the sample, and the storage location includes a storage path of the sample set and an index position of the sample in the sample set. In this way, the precise location of the sample can be determined, making it easy to read the sample.

In a possible implementation manner, the method further includes: determining index range information used to classify the types of samples in the sample set; generating information related to the sample set based on the description information of the samples in the sample set and the index range information. The description information corresponding to the set. In this way, the index range of the sample can be determined, which is convenient for searching the sample.

In a possible implementation manner, the method further includes: reading parameter values of samples in the sample list according to the description information and the sample list.

In a possible implementation manner, the reading the parameter values of the samples in the sample list according to the description information and the sample list includes: obtaining objects in the sample list; according to the sample index, from Among the acquired objects, the first object of the first sample corresponding to the sample index is obtained; the data type of the first sample and the corresponding reading method are determined according to the description information; according to the reading In this way, the parameter value of the first sample is read from the first object.

In this way, the resource consumption when reading samples can be reduced, and the sample data can be stored in a unified way, and the sample data type can be stored using the sample description information, so that the data reading method can be easily obtained according to the description information, which is easy to read The samples of multiple data types are conducive to quick reading of the samples.

In a possible implementation manner, the reading the parameter values of the samples in the sample list according to the description information and the sample list includes: obtaining a sample index; according to the sample index, from the sample The parameter value of the sample corresponding to the sample index is obtained from the list.

In a possible implementation, the reading the parameter values of the samples in the sample list according to the description information and the sample list includes: obtaining the object corresponding to the target field in the sample list; The description information determines the data type and reading method of the second sample corresponding to the acquired object; according to the reading method, the parameter value of the second sample is read from the acquired object. In this way, the parameter value of one or more samples in the field is read by accessing the field, which improves the flexibility of sample reading and facilitates the rapid reading of samples.

In a possible implementation manner, the reading the parameter values of the samples in the sample list according to the description information and the sample list includes: obtaining the field name of the target field; according to the field name, from Obtain the parameter value of the target field corresponding to the field name from the sample list.

In a possible implementation manner, the method further includes: performing at least one operation of query, modification, and deletion processing on the parameter value of the read sample.

According to an aspect of the embodiments of the present disclosure, there is provided a data processing device, including: a first determining part configured to determine a sample list including a plurality of samples according to a field of the sample and a parameter value corresponding to the field; The second determining part is configured to determine the description information corresponding to the sample list according to the data type of the samples in the sample list; the third determining part is configured to determine according to the sample list and the description information Sample set.

In a possible implementation manner, each sample has a sample index, and the first determining part is further configured to: determine the field as the column coordinates of the sample list, and determine the sample index as the The row coordinates of the sample list; object the parameter values corresponding to the fields to obtain the object corresponding to the sample; add the data in the object to the storage area corresponding to the column coordinates and the row coordinates , To obtain the sample list.

In a possible implementation manner, the description information includes description information of how to read the sample, and the second determining part is further configured to: determine a sample for reading according to the data type of the sample in the sample list. The method of reading; the description information is obtained according to the data type of the sample and the corresponding reading method.

In a possible implementation manner, the description information further includes description information of a storage location of the sample, and the storage location includes a storage path of the sample set and an index position of the sample in the sample set.

In a possible implementation manner, the device further includes: a fourth determining part configured to determine index range information used to classify samples in the sample set; and the generating part configured to determine the index range information of the samples in the sample set according to The description information and the index range information generate description information corresponding to the sample set.

In a possible implementation manner, the device further includes: a reading part configured to read parameter values of samples in the sample list according to the description information and the sample list.

In a possible implementation manner, the reading part is further configured to: obtain objects in the sample list; according to the sample index, obtain the first object corresponding to the sample index from the obtained objects. The first object of this book; the data type of the first sample and the corresponding reading method are determined according to the description information; the first sample is read from the first object according to the reading method The parameter value.

In a possible implementation manner, the reading part is further configured to: obtain a sample index; according to the sample index, obtain the parameter value of the sample corresponding to the sample index from the sample list.

In a possible implementation manner, the reading part is further configured to: obtain the object corresponding to the target field in the sample list; determine the data type and the data type of the second sample corresponding to the obtained object according to the description information. Reading mode; according to the reading mode, the parameter value of the second sample is read from the obtained object.

In a possible implementation manner, the reading part is further configured to: obtain the field name of the target field; according to the field name, obtain the parameter value of the target field corresponding to the field name from the sample list.

In a possible implementation manner, the device further includes: an operation part configured to perform at least one of query, modification, and deletion of the parameter value of the read sample.

According to an aspect of the embodiments of the present disclosure, there is provided an electronic device including: a processor; a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the above-mentioned data processing method.

According to an aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the foregoing data processing method is implemented.

According to an aspect of the embodiments of the present disclosure, there is provided a computer program, including computer-readable code, when the computer-readable code runs in an electronic device, the processor in the electronic device executes the above-mentioned data processing method.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the embodiments of the present disclosure.

According to the following detailed description of the exemplary embodiments with reference to the accompanying drawings, other features and aspects of the embodiments of the present disclosure will become clear.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments in accordance with the present disclosure, and are used together with the specification to describe the technical solutions of the embodiments of the present disclosure.

Fig. 1 shows a flowchart of data processing according to an embodiment of the present disclosure;

Fig. 2 shows a schematic diagram of an exemplary sample list according to an embodiment of the present disclosure;

Fig. 3 shows an application schematic diagram of an exemplary storage method according to an embodiment of the present disclosure;

Fig. 4 shows a block diagram of a data processing device according to an embodiment of the present disclosure;

Figure 5 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

Fig. 6 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings indicate elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, unless otherwise noted, the drawings are not necessarily drawn to scale.

The dedicated word "exemplary" here means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" need not be construed as being superior or better than other embodiments.

The term "and/or" in this article is only an association relationship that describes associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations. In addition, the term "at least one" in this document means any one or any combination of at least two of the multiple, for example, including at least one of A, B, and C, may mean including A, Any one or more elements selected in the set formed by B and C.

In addition, in order to better illustrate the embodiments of the present disclosure, numerous specific details are given in the following specific embodiments. Those skilled in the art should understand that the embodiments of the present disclosure can also be implemented without certain specific details. In some instances, the methods, means, elements, and circuits well-known to those skilled in the art have not been described in detail, so as to highlight the gist of the embodiments of the present disclosure.

Fig. 1 shows a flowchart of a data processing method according to an embodiment of the present disclosure. As shown in Fig. 1, the method includes:

S11: Determine a sample list including multiple samples according to the fields of the samples and the parameter values corresponding to the fields;

S12: Determine description information corresponding to the sample list according to the data type of the sample in the sample list;

S13: Determine a sample set according to the sample list and the description information.

In a possible implementation, the execution subject of the data processing method may be a terminal device or a server or other processing equipment, where the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, or a terminal. , Cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. In some possible implementation manners, the method may be implemented by a processor invoking computer-readable instructions stored in a memory or executed by a server. For example, data in the server may be stored by the storage method.

In a possible implementation manner, the data type of the sample may include multiple types, such as image, video, and/or ordinary numerical value or text data, etc. The embodiment of the present disclosure does not limit the data type of the sample. The sample may include one or more fields, and each field may correspond to a certain feature or a certain type of characteristic of the sample. For example, a certain sample is the profile data of a certain person, and the fields of the sample may include the height and weight of the person , Education, nationality, etc., and the sample can have data corresponding to each field. The embodiments of the present disclosure do not impose restrictions on the fields.

In a possible implementation manner, in S11, the multiple samples may be stored in a datatable, and the storage in a list manner is convenient for operations such as data addition, deletion, and modification.

In a possible implementation manner, each sample has a sample index, and the sample index may include the number or name corresponding to the sample, S11 may include: determining the field as the column coordinates of the sample list, The sample index is determined as the row coordinates of the sample list; the parameter values of the sample are objectified to obtain the object corresponding to the sample; the object is added to the column coordinates and the row coordinates. The storage area corresponding to the coordinates is used to obtain the sample list.

FIG. 2 shows a schematic diagram of an exemplary sample list according to an embodiment of the present disclosure. As shown in FIG. 2, the list may use sample indexes as row coordinates and fields as column coordinates. The fields of each sample can be summarized as the column coordinates of the sample list. The sample can be a tuple, such as a two-dimensional tuple, and a data group composed of data in multiple fields. For example, a sample (for example, sample 1) is the profile data of a certain person, and the fields of the sample may include the person's height (for example, field 1), weight (for example, field 2), education (for example, field 3), etc., The other sample is a face image of a person. The fields of the sample can include the person’s identity information, etc., and the fields of each sample can be summarized. The column coordinates of the sample list can include height, weight, education, nationality, and identity. Information and other fields. Each field corresponds to a column of data in the sample list. In some rows of the column of data, the data can be empty. For example, the list of samples includes the "nationality" field. In a sample of a person’s profile data, this field can contain data, such as "China", "United States", etc., while in a face image sample, this field There may be no data, that is, in the sample list, the data is empty in the row where the face image sample is located and the area corresponding to the column where the "nationality" field is located.

In a possible implementation manner, the sample index of the sample can be determined as the row coordinates of the sample list. For example, each sample can have a sample index, for example, sample 1, sample 2, sample 3, etc., and the sample index can be Determined as the row coordinates of the sample list.

In a possible implementation, the samples may be training samples used when training a neural network, each sample may have a label, and the label may be data describing a field of the sample. For example, the sample may include a face image sample. When performing identity verification through a neural network, the neural network compares each face image sample with a reference image (for example, a face image of a certain target person), and determines the person The similarity between the face image sample and the reference image. When the similarity is greater than or equal to the similarity threshold (for example, 0.5), it can be determined that the face image sample has a high similarity with the reference image and the identity verification is passed; otherwise, the identity verification fails. When training the neural network, it is necessary to label each face image sample to provide parameters for the loss function of the neural network. For example, if a face image sample is the face image of the target person, it can be marked as 1, which means that the face in the face image sample is the face of the target person. If the face image sample is not the target person's face The face image can be marked as 0, which means that the face in the face image sample is not the face of the target person. The sample may have parameter values and annotations of each field. For example, the field of a certain face image sample is the identity information (for example, name or number, etc.) of the target object in the image, which is marked as 0 or 1.

In a possible implementation, the label and parameter value of each field in the sample can be objectified. For example, a storage area corresponding to the sample can be set up. For example, a storage space can be set up in the memory, and the label of each field in the sample can be set up. The sum parameter value is stored in the storage area, and the object corresponding to the sample is obtained for easy storage and reading. Among them, the storage area storing the data (label and parameter value) of the field is the object.

In a possible implementation, the data in the object corresponding to the sample can be added to the sample list, or the object can be associated with the sample list to initialize the sample list. For example, the storage area in which the data of the field is stored may correspond to the storage area in the sample list (for example, a certain cell in the sample list). In the example, the data in the object corresponding to sample 1 can be added to the first row of the sample list, that is, the data of each field in the object corresponding to sample 1 can be added to the corresponding field in the first row In the storage area, for example, add height data in the storage area where the row coordinates are the first row and the column coordinates are the "height" field, and the weight data is added in the row coordinates of the first row and the column coordinates are the "weight" field. In the storage area. Each sample can be objectified in the above-mentioned manner, and the data stored in the object can be filled into the storage area of the sample list to obtain a sample list storing multiple samples.

In a possible implementation, the sample list can be initialized according to the sample index. For example, the data of each field of one sample can be added to the corresponding storage area, and then the data of each field of the next sample can be added, that is, Add the data of each sample according to the row direction in the list to initialize the sample list.

In a possible implementation, the sample list can be initialized according to the fields, that is, all the data of a field can be added to the corresponding storage area, and then the data of the next field can be added, that is, according to the column direction in the list Add data to initialize the sample list.

In a possible implementation, you can use a two-dimensional tuple to define the character field name, and the numerical list or array of the sample corresponding to the field, or use the keyword parameter method, or the above two mixed methods to initialize List of samples.

In this way, the parameter values and/or annotations of the samples can be objectified, and samples of multiple data types can be stored through the sample list, which is conducive to the rapid storage and reading of samples and reduces the resource occupation during storage and reading. , And can support multiple types of data storage, adapt to the storage of more complex types of data sets.

In a possible implementation, in S12, the sample can include multiple data types, and each type of sample can have a specific storage and reading method. When multiple samples are stored, the description information can be used for each sample. Description for easy access (such as storage or reading) of multiple types of samples. In a possible implementation manner, the sample may include a video, the description information may include a description of how the video is stored or read, the sample may also include an image, and the description information may also include the storage of the image. Or read the description of the way. The description information may be description information in a metadata (Meta Data) format, for example, dictionary type metadata, which is convenient for improving the data query speed. The embodiment of the present disclosure does not limit the type of the description information.

In a possible implementation manner, the description information includes the description information of the reading method of the sample, and S12 may include: determining the reading method for reading each sample according to the data type of the sample in the sample list ; According to the data type of the sample and the corresponding reading mode, the description information is obtained.

In a possible implementation manner, the description information may be information describing how the sample is read. The reading method of each sample can be judged according to the data type of the sample. For example, if the data type of a sample is an image, the sample can be read by the image reading function, and the data type of a sample is video, then The sample can be read with the video reading function, and the reading method of each sample can be determined separately. The data type of each sample can also be summarized. For example, the sample set includes multiple samples. There are three types of data for the multiple samples, namely image, video, and general data. The image type can be determined by the image reading function. The sample of the video type is read through the video reading function, and the sample of the common data type is read through the data reading function. The embodiment of the present disclosure does not limit the data type and reading method of the sample.

In a possible implementation manner, the description information can be obtained according to the data type of the sample and the corresponding reading method. In a possible implementation manner, the description information can be configured as dictionary data, for example, a dictionary The key of is the data type, and the value of the dictionary is the reading method corresponding to the data type. For example, the key of the dictionary is the image, and the value of the dictionary is the image reading function. The embodiment of the present disclosure does not limit the data type of the description information.

In a possible implementation, the description information may include description information of the storage location of the sample, and the storage location includes the storage path of the sample set and the index position of the sample in the sample set. For example, the storage path of the sample set is that the sample set is stored in the "sample set" folder on the C drive of the computer.

In a possible implementation, the index position of the sample in the sample set is the specific position of the sample. In a possible implementation manner, the data of each sample may be stored in the sample set in the form of an object, that is, occupy a certain storage area in the sample set. The sample set may store each sample in the form of a list, the data of the sample may occupy a certain row in the sample set, and the index position may indicate the row number. For example, the first sample is located in the first row of the list of the sample set, The two samples are located in the second row of the sample set list,...; the field information of the sample can include the storage path of the sample set and the index position of each sample. The specific location of each sample can be determined according to the field information, for example, the first The 2 samples are located in the second row of the sample set list in the "sample set" folder in the C drive of the computer. For another example, some samples may only have certain fields, or multiple samples belong to the same type or the same sample file. The index position may be the row and column range of the sample in the sample set, or the row and column range of multiple samples. The embodiment of the present disclosure does not limit the storage location.

In a possible implementation manner, the sample set may include samples of multiple data types, such as images, videos, ordinary numbers and/or text data, etc., and the storage format of the samples may include a character-separated value file (Comma-Separated Value File). Values, CSV) format, hierarchical data format 5 (Hierarchical Data Format 5, HDF5), etc. The index position of the sample in the sample set may be the position range of the same type of sample generated after the same type of sample is summarized. For example, a sample file stored in the CSV format will store the index position of the CSV file in the description information.

In a possible implementation manner, the method further includes: determining index range information used to classify the types of samples in the sample set; generating and according to the description information of the samples in the sample set and the index range information of the sample categories. Descriptive information corresponding to the sample set.

In a possible implementation manner, the description information corresponding to the sample set further includes index range information for dividing the type of each sample in the sample set. The process of determining the description information corresponding to the sample set in the method further includes: determining index range information used to classify the categories of each sample in the sample set; according to the description information of the samples in the sample set and the index range information of the samples, Generate description information corresponding to the sample set. For example, it can be classified according to the label of the sample. For example, the sample set can be divided into a training set, a validation set, and a test set. The training set includes 5000 samples, of which 3000 samples are labeled as 1, and 2000 samples are labeled as 0. Or, the labels may not be distinguished, and the classification may be directly based on the number and index of the samples in the sample set. For example, the samples in the sample set may be training samples used when training a neural network, and the training samples may be divided into multiple categories. For example, Divide 40% of the samples into the training set, divide 30% of the samples into the test set, and divide 30% of the samples into the validation set. Or it can be divided according to the index of each sample, and the index range of each category can be determined. For example, the sample set stores each sample in the form of a list, and 10,000 samples are stored in the list. The 1st-4000th sample in the list can be divided into the training set, and the 4001th-4th sample in the list can be divided into training set. The 7000 samples are divided into the test set, and the 7001th-10000th samples are divided into the verification set. Wherein, the description information may include index range information (for example, 1st-4000th, 4001th-7000th, 7001th-10000th, etc.) for dividing sample categories. The embodiment of the present disclosure does not limit the division method and the index range.

In addition to the sample subsets of the above-mentioned training set, test set, and validation set, the sample set may also include other forms of data subsets, such as image subsets and video subsets. The description information of the sample set includes not only the sample subsets of the above-mentioned training set, test set, and validation set and the corresponding index range description information, but also the description information of other forms of data subsets in the sample set.

In this way, a variety of sample reading methods can be determined, which is conducive to supporting multiple types of data storage and access, and adapts to the storage and access of more complex types of data sets.

In a possible implementation manner, in S13, the sample set (dataset) can be obtained according to the sample list and the description information corresponding to the sample list. In a possible implementation manner, the sample list and the sample list can be combined The corresponding description information is written into the sample set to obtain the file of the sample set, or the description information corresponding to the sample list and the sample list are packaged to obtain the sample set. The embodiment of the present disclosure does not limit the way of obtaining the sample set.

In a possible implementation, the sample set includes a sample list storing multiple samples, and also includes description information that records the reading method of each sample, so as to facilitate the storage and reading of samples of multiple data types. Samples that store multiple data types provide the basis. The description information can record the data type of each sample, and the reading method and other information. Based on the description information, the sample or the data of the sample in the sample set can be read. The embodiments of the present disclosure do not limit the content of the description information.

In a possible implementation manner, the sample in the sample list can be read in a suitable manner according to the description of the sample reading method in the description information, and the reading process may include: obtaining objects in the sample list According to the sample index, from the acquired objects, obtain the first object of the first sample corresponding to the sample index; determine the data type of the first sample and the corresponding reading method according to the description information ; According to the reading method, the parameter value of the first sample is read from the first object.

In a possible implementation manner, the sample list includes a plurality of samples, and the samples are stored in the sample list in an object format. When reading a sample, each object in the sample list can be obtained (for example, by accessing the'samples' attribute, that is, accessing the sample object, all objects in the list containing all samples can be obtained). Further, the sample index in the sample list can be accessed, and the first sample to be read is selected in the sample index, that is, the first object corresponding to the first sample is selected, for example, the sample is selected through the sample index In the list, the row where the first sample is located.

In a possible implementation manner, the description information may include the reading method of each sample, and the corresponding reading method may be selected in the description information according to the data type of the first sample, for example, the first sample The data type of is image, you can select the image reading function in the description information to read the first sample, and the data type of the first sample is video, you can select the video reading function in the description information to read the first sample this.

In a possible implementation manner, the parameter value of the first sample can be read in the first object through a reading method corresponding to the data type of the first sample. For example, the first sample is information data of a certain person, the first sample may include the person's height data, weight data, educational background data, nationality data, etc. The data can be read through the data reading function. In the first object, the data of the first sample is extracted, and the parameter values of the first sample are read through the data reading function. In a possible implementation, the parameter value of one or more fields of the first sample data can also be read. For example, the height data and weight data in the data can be selected and read through the data read function. Take the height data and weight data. There is no need to read all the data of the first sample, which improves the flexibility of sample reading.

In a possible implementation manner, according to the description information and the sample list, reading the parameter values of the samples in the sample list includes: obtaining a sample index; according to the sample index, from the sample list Obtain the parameter value of the sample corresponding to the sample index.

In a possible implementation manner, the sample index can be used to access the sample list, read the sample corresponding to the sample index, and obtain the parameter value of the sample. For example, when the sample data needs to be read, a read instruction input by the user or the sample acquisition device can be received. The read instruction carries the sample index of the sample to be read. The sample index value can be set to 100, , You can get the sample in the 100th row of the list, that is, get the sample in the 100th row in the list, and read the parameter value of the sample according to the reading method of the sample. For example, the parameter value of the sample can be read according to the reading method in the description information, or the data type of the sample can be determined according to the suffix of the sample, and then the corresponding reading method can be selected to read the sample to obtain the sample The parameter value.

In a possible implementation manner, it is also possible to read multiple parameter values corresponding to a certain field or the parameter value of the field of a certain sample by accessing the fields in the sample list.

In a possible implementation manner, according to the description information and the sample list, reading the parameter values of the samples in the sample list includes: obtaining the object corresponding to the target field in the sample list; according to the description The information determines the data type and reading method of the second sample corresponding to the acquired object; according to the reading method, the parameter value of the second sample is read from the acquired object.

In a possible implementation, the target field in the sample list can be accessed to obtain the object corresponding to the target field. For example, by accessing the'fields' attribute, that is, accessing the field, you can get all the objects that contain all the sample data fields, that is , To obtain the storage space (such as the field list) where the parameters of multiple samples under the field are located. For example, the field list can include "age", "height" and "nationality", etc., and can also be accessed by keywords in the field To access the corresponding field, for example, you can access the parameter value in the "Nationality" field through the keyword "nationality", or you can directly access the field name or keyword to access the parameter value in the field without accessing the field list. For example, you can Access the parameter value in the field "nationality" directly through the field name "nationality". In a possible implementation manner, each field corresponds to a column of data in the sample list, and in some areas of the column of data, the data may be empty. For example, in the list of samples, the "nationality" field is included. In a sample of a person's profile data, this field can contain data, such as "China" or "United States", while in a face image sample, this field There is no data, that is, in the list, in the row where the face image sample is located, and the area corresponding to the column where the "nationality" field is located, the data is empty. When obtaining the object corresponding to the target field, the empty area may be ignored or not. The embodiment of the present disclosure does not limit whether to ignore. When obtaining the object corresponding to the target field, all the objects in the column of the target field can be obtained, that is, all the storage space for storing the data of the column.

In a possible implementation manner, the reading method of the second sample to be read can be determined according to the description information, and the target field of the second sample can also be determined in the object corresponding to the target field according to the index of the second sample. The storage space of the parameter value, that is, the object of the parameter value of the target field of the second sample.

In a possible implementation manner, in the object of the parameter value of the target field of the second sample, the parameter value of the target field of the second sample may be read through the reading method. For example, the second sample is a person's profile data, and the second sample may include the person's height data, weight data, education data, nationality data, and so on. Among the objects of all parameter values corresponding to the "height" field, the object of the second sample in this field can be determined, and the parameter values in the object of the second sample can be read through the reading method corresponding to the data data. In a possible implementation, the height data of the second sample in the object of the field is read through the data reading function. For example, when reading the data, you can first access the description information to query the reading method of the field, for example, "Height All parameter values in the "field are floating-point data, and then according to the reading method of floating-point data in the description information, the parameter value of the second sample in this field is read from the object of the second sample.

In a possible implementation manner, according to the description information and the sample list, reading the parameter values of the samples in the sample list includes: obtaining the field name of the target field; according to the field name, from the sample list To obtain the parameter value of the target field corresponding to the field name.

In a possible implementation manner, the field name of the target field can be used to access the target field, and then obtain the parameter value of the target field. For example, when a user or a device needs to read sample data, the storage device or processing device can receive a read instruction. The read instruction includes the field name of the target field of the sample to be read. The target field can be set to "height", When reading, the parameter value of the "height" field can be accessed through the field name "height". For example, the reading method corresponding to the data type of the field in the description information can be used to read the parameter value of the field, or the data type of the parameter of the field can be determined according to the suffix of the parameter of the field, and then the corresponding reading can be determined. Use the method to read the parameter value of the field. Further, any parameter value under the field can also be read, for example, the parameter value stored in the storage location with the index value of 100 under the "height" field can be obtained.

In this way, the parameter value of one or more samples in the field is read by accessing the field, which improves the flexibility of sample reading and facilitates the rapid reading of samples.

In the case of sample data reading, including in the case of reading samples or field parameter values, the storage device or data processing device will receive a read instruction, and the read instruction will include the sample data information that needs to be read, such as sample Index and field name, etc., according to the read instruction, the data type, storage location and corresponding reading method of the sample will be obtained from the description information of the sample set, so that the obtained reading method can be used to read the required data in the storage location Sample data. Therefore, a sample set containing description information can conveniently store and read various types of sample data, and the description information can facilitate users to understand and view the data stored in the sample set, which is suitable for scenarios for beginners.

In a possible implementation manner, the method further includes: performing at least one operation of query, modification, and deletion processing on the parameter value of the read sample. In a possible implementation manner, after reading the parameter value in an object, at least one of processing such as query, modification, and deletion may be performed on the parameter value. For example, the parameter value in the object can be queried, and the parameter value can be copied. Or, the parameter value can be modified, and the modified parameter value can be saved in the object. Alternatively, the parameter value in the object can be deleted, so that the data in the object is empty. The embodiment of the present disclosure does not limit the processing manner of the parameter value.

Fig. 3 shows an application schematic diagram of an exemplary storage method according to an embodiment of the present disclosure. As shown in Fig. 3, multiple samples can be stored in a sample list, and the data types of the multiple samples can include image, audio, Video and/or general data (for example, numerical data, categorical data, and character data), etc. Each sample can occupy one row of storage area in the sample list. Each sample can include one or more fields. For example, a sample is the profile data of a certain person. The fields of the sample can include the person’s height, weight, education and nationality, etc. The sample can have data corresponding to each field .

In a possible implementation manner, the fields of each sample can be summarized as the column coordinates of the sample list, and the sample index of the sample is determined as the row coordinate of the sample list to obtain an empty sample list.

In a possible implementation, the label and parameter value of the sample can be objectified, that is, a storage area corresponding to each parameter value of the sample is set up, and the label and parameter value of the sample are stored in the storage area to obtain The object corresponding to the sample is added, and the label and parameter values in the object are added to the sample list to obtain a sample list storing multiple samples.

In a possible implementation, the sample can include multiple data types, and each type of sample can have a specific storage and reading method. When multiple samples are stored, the description information in the metadata format can be used for each sample. Description, in order to store or read multiple types of samples.

In a possible implementation, the data type of each sample in the sample list can be summarized, and the reading method of each data type can be judged. There are three data types for multiple samples in the sample list, respectively: For images, videos and general data, it can be determined that the image type samples are read through the image reading function, the video type samples are read through the video reading function, and the common data type samples are read through the data reading function. The description information is configured as dictionary data. For example, the key of the dictionary is the data type, and the value of the dictionary is the reading mode corresponding to the data type.

In a possible implementation manner, the sample list and the description information corresponding to the sample list may be written into the sample set to obtain the file of the sample set. And by accessing the sample index or field to read the sample or a parameter value in the sample.

In a possible implementation manner, each object in the sample list can be obtained, and the sample index in the sample list can be accessed to select the first sample to be read, that is, the first sample corresponding to the first sample is selected The first object. Further, the corresponding reading method can be selected in the description information. For example, the data type of the first sample is an image, and the image reading function in the description information can be selected to read the first sample. The first sample can be read through the reading method, or only the parameter value of one or more fields in the first sample can be read.

In a possible implementation manner, the target field in the sample list can be accessed, and all objects corresponding to the target field can be obtained (the empty area in the target field can be ignored). The parameter value of the second sample in the target field can be read among all objects corresponding to the target field. In a possible implementation manner, the data type of the second sample can be determined, and the reading method corresponding to the data type of the second sample can be determined in the description information. Further, among the objects of all parameter values of the target field, the object of the second sample in the field can be determined, and the second sample can be read in the object according to the reading method corresponding to the data type of the second sample. The parameter value of the field.

In a possible implementation, the storage method can be used to store multiple types of samples. For example, in the field of computer education, using this storage method can store multiple types of samples according to a unified storage method. , And the program interface for storage and reading is convenient and concise. The storage and reading methods of various samples are unified and easy to understand, which is convenient for students to learn and operate.

It can be understood that the various method embodiments mentioned in the embodiments of the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, the embodiments of the present disclosure will not be repeated.

In addition, the embodiments of the present disclosure also provide data processing apparatuses, electronic equipment, computer-readable storage media, and programs. All of the above can be used to implement any image processing method provided by the embodiments of the present disclosure. For the corresponding technical solutions and descriptions, refer to the method Part of the corresponding records will not be repeated here.

Those skilled in the art can understand that in the above-mentioned methods of the specific implementation, the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possibility. The inner logic is determined.

Fig. 4 shows a block diagram of a data processing device according to an embodiment of the present disclosure. As shown in Fig. 4, the device includes: a first determining part 11 configured to according to a field of a sample and a parameter value corresponding to the field, Determine a sample list including a plurality of samples; the second determining part 12 is configured to determine the description information corresponding to the sample list according to the data type of the samples in the sample list; the third determining part 13 is configured to determine the description information corresponding to the sample list according to The sample list and the description information determine the sample set.

In a possible implementation manner, each sample has a sample index, and the first determining part is further configured to: determine the field as the column coordinates of the sample list, and determine the sample index of the sample as The row coordinates of the sample list; objectize the parameter value corresponding to the field to obtain the object corresponding to the sample; add the data in the object to the corresponding column coordinates and the row coordinates Storage area to obtain the sample list.

In a possible implementation manner, the device further includes: a fourth determining part configured to determine index range information used to classify samples in the sample set; and the generating part configured to determine the index range information of the samples in the sample set according to The description information and the index range information of the sample category generate description information corresponding to the sample set.

In some embodiments, the functions or parts included in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. For specific implementation, refer to the description of the above method embodiments. For brevity, here No longer.

In the embodiments of the present disclosure and other embodiments, "parts" may be parts of circuits, parts of processors, parts of programs or software, etc., of course, may also be units, modules, or non-modular.

The embodiments of the present disclosure also provide a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above-mentioned method when executed by a processor. The computer-readable storage medium may be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory configured to store executable instructions of the processor; wherein the processor is configured as the aforementioned method.

The electronic device can be provided as a terminal, server or other form of device.

Fig. 5 is a block diagram showing an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminals.

5, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, and a sensor component 814 , And communication component 816.

The processing component 802 generally controls the overall operations of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the foregoing method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations in the electronic device 800. Examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable and Programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power supply component 806 provides power for various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure related to the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module. The above-mentioned peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: home button, volume button, start button, and lock button.

The sensor component 814 includes one or more sensors for providing the electronic device 800 with various aspects of state evaluation. For example, the sensor component 814 can detect the on/off status of the electronic device 800 and the relative positioning of the components. For example, the component is the display and the keypad of the electronic device 800. The sensor component 814 can also detect the electronic device 800 or the electronic device 800. The position of the component changes, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects when there is no physical contact. The sensor component 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field-available A programmable gate array (FPGA), controller, microcontroller, microprocessor, or other electronic components are implemented to implement the above methods.

In an exemplary embodiment, a computer-readable storage medium is also provided, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to complete the foregoing method.

The embodiments of the present disclosure also provide a computer program product, including computer-readable code, and when the computer-readable code runs on a device, a processor in the device executes instructions for implementing the method provided in any of the above embodiments.

The computer program product can be specifically implemented by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc. Wait.

Fig. 6 is a block diagram showing an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. 6, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by the memory 1932, for storing instructions executable by the processing component 1922, such as application programs. The application program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to the network, and an input output (I/O) interface 1958 . The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to complete the foregoing method.

The present disclosure may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the embodiments of the present disclosure.

The computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon The protruding structure in the hole card or the groove, and any suitable combination of the above. The computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .

The computer program instructions used to perform the operations of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or one or more programming instructions. Source code or object code written in any combination of languages, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server carried out. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network-including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect to the user's computer) connection). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions. The computer-readable program instructions are executed to realize various aspects of the present disclosure.

Here, various aspects of the embodiments of the present disclosure are described with reference to flowcharts and/or block diagrams of methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions on a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , So that the instructions executed on the computer, other programmable data processing apparatus, or other equipment realize the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions, and operations of the system, method, and computer program product according to multiple embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function. Executable instructions. In some alternative implementations, the functions marked in the block may also occur in a different order than the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions.

The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Without departing from the scope and spirit of the described embodiments, many modifications and changes are obvious to those of ordinary skill in the art. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements of the technologies in the market, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Industrial applicability

The embodiments of the present disclosure relate to a data processing method and device, electronic equipment, and storage medium. The method includes: determining a sample list including a plurality of samples according to a field of a sample and a parameter value corresponding to the field; The data type of the samples in the sample list is determined, and the description information corresponding to the sample list is determined; and the sample set is determined according to the sample list and the description information. According to the data processing method of the embodiment of the present disclosure, the sample set used to store each sample and the corresponding description information can be determined, multiple types of samples can be stored in a unified manner, and the sample set includes description information describing the data type, It is convenient to store and read sample data, can support multiple types of data storage and read, and is easy to understand.

Claims

A data processing method, including:

Determine a sample list including multiple samples according to the fields of the samples and the parameter values corresponding to the fields;

Determine the description information corresponding to the sample list according to the data type of the sample in the sample list;

According to the sample list and the description information, a sample set is determined.
The method according to claim 1, wherein each sample has a sample index, and determining a sample list including a plurality of samples according to a field of the sample and a parameter value corresponding to the field includes:

Determining the field as the column coordinates of the sample list, and determining the sample index as the row coordinates of the sample list;

Objectify the parameter value corresponding to the field to obtain the object corresponding to the sample;

The data in the object is added to the storage area corresponding to the column coordinates and the row coordinates to obtain the sample list.
The method according to claim 1 or 2, wherein the description information includes description information of how the sample is read,

The determining the description information corresponding to the sample list according to the data type of the sample in the sample list includes:

According to the data type of the sample in the sample list, determine the reading method used to read the sample;

The description information is obtained according to the data type of the sample and the corresponding reading mode.
The method according to any one of claims 1 to 3, wherein the description information further includes description information of the storage location of the sample, and the storage location includes the storage path of the sample set and the location of the sample in the sample set. Index position.
The method according to any one of claims 1-4, wherein the method further comprises:

Determine the index range information used to classify the types of samples in the sample set;

According to the description information of the samples in the sample set and the index range information of the sample category, the description information corresponding to the sample set is generated.
The method according to any one of claims 1-5, wherein the method further comprises:

According to the description information and the sample list, read the parameter values of the samples in the sample list.
The method according to claim 6, wherein said reading parameter values of samples in said sample list according to said description information and said sample list comprises:

Obtaining objects in the sample list;

According to the sample index, obtain the first object of the first sample corresponding to the sample index from the acquired objects;

Determine the data type of the first sample and the corresponding reading method according to the description information;

According to the reading method, the parameter value of the first sample is read from the first object.
The method according to claim 6, wherein said reading parameter values of samples in said sample list according to said description information and said sample list comprises:

Get sample index;

According to the sample index, the parameter value of the sample corresponding to the sample index is obtained from the sample list.
The method according to claim 6, wherein said reading parameter values of samples in said sample list according to said description information and said sample list comprises:

Obtaining the object corresponding to the target field in the sample list;

Determine the data type and reading mode of the second sample corresponding to the acquired object according to the description information;

According to the reading method, the parameter value of the second sample is read from the acquired object.
The method according to claim 6, wherein said reading parameter values of samples in said sample list according to said description information and said sample list comprises:

Get the field name of the target field;

According to the field name, the parameter value of the target field corresponding to the field name is obtained from the sample list.
The method according to any one of claims 6-10, wherein the method further comprises:

At least one operation of query, modification and deletion processing is performed on the parameter value of the read sample.
A data processing device includes:

The first determining part is configured to determine a sample list including multiple samples according to the fields of the samples and the parameter values corresponding to the fields;

The second determining part is configured to determine the description information corresponding to the sample list according to the data type of the sample in the sample list;

The third determining part is configured to determine a sample set according to the sample list and the description information.
The apparatus according to claim 12, wherein each sample has a sample index and the first determining part is further configured to:

Determining the field as the column coordinates of the sample list, and determining the sample index as the row coordinates of the sample list;

Objectify the parameter value corresponding to the field to obtain the object corresponding to the sample;

The data in the object is added to the storage area corresponding to the column coordinates and the row coordinates to obtain the sample list.
The device according to claim 12 or 13, wherein the description information includes description information of how to read the sample, and the second determining part is further configured to:

According to the data type of the sample in the sample list, determine the reading method used to read the sample;

The description information is obtained according to the data type of the sample and the corresponding reading mode.
The device according to any one of claims 12-14, wherein the description information further includes description information of the storage location of the sample, and the storage location includes the storage path of the sample set and the location of the sample in the sample set. Index position.
The device according to any one of claims 12-15, wherein the method further comprises:

The fourth determining part is configured to determine index range information used to classify the types of samples in the sample set;

The generating part is configured to generate description information corresponding to the sample set according to the description information of the samples in the sample set and the index range information of the sample category.
The device according to any one of claims 12-16, wherein the method further comprises:

The reading part is configured to read the parameter values of the samples in the sample list according to the description information and the sample list.
The device according to claim 17, wherein the reading part is further configured to:

Obtaining objects in the sample list;

According to the sample index, obtain the first object of the first sample corresponding to the sample index from the acquired objects;

Determine the data type of the first sample and the corresponding reading method according to the description information;

According to the reading method, the parameter value of the first sample is read from the first object.
The device according to claim 17, wherein the reading part is further configured to:

Get sample index;

According to the sample index, the parameter value of the sample corresponding to the sample index is obtained from the sample list.
The device according to claim 17, wherein the reading part is further configured to:

Obtaining the object corresponding to the target field in the sample list;

Determine the data type and reading mode of the second sample corresponding to the acquired object according to the description information;

According to the reading method, the parameter value of the second sample is read from the acquired object.
The device according to claim 17, wherein the reading part is further configured to:

Get the field name of the target field;

According to the field name, the parameter value of the target field corresponding to the field name is obtained from the sample list.
The device according to any one of claims 17-21, wherein the device further comprises:

The operation part is configured to perform at least one operation of query, modification and deletion of the parameter value of the read sample.
An electronic device including:

processor;

A memory configured to store executable instructions of the processor;

Wherein, the processor is configured to call instructions stored in the memory to execute the method according to any one of claims 1 to 11.
A computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.
A computer program comprising computer readable code, when the computer readable code runs in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 11.