TWI739633B - Storage and reading method, electronic equipment and computer-readable storage medium - Google Patents

Abstract

The embodiment of the disclosure relates to a storage and reading method, an electronic device and a computer-readable storage medium. The storage method comprises: determining the description information corresponding to the sample set according to the label of the sample set; the field information corresponding to the sample set is determined according to the sample field and storage location in the sample set, and the configuration file of the sample set is determined according to the description information and field information.

Description

Storage and reading method, electronic equipment and computer readable storage medium

The present invention relates to the field of computer technology, and relates to a storage and reading method, an electronic device and a computer-readable storage medium.

In machine learning, especially when multiple data sets need to be processed, the data set storage method is extremely important. In related technologies, the storage of data sets is usually to adapt to the structure of the data set itself, for example, CSV (Comma-Separated Values), HDF5 (Hierarchical Data Format Version 5, hierarchical data format fifth edition) , File storage (image and audio) and other data storage methods.

The embodiments of the present invention provide a storage and reading method, an electronic device, and a computer-readable storage medium.

According to one aspect of the embodiments of the present invention, a storage method is provided, which includes: determining the description information corresponding to the sample set according to the labels of the samples in the sample set; Field information corresponding to the sample set; determine the configuration file of the sample set according to the description information and the field information.

According to the reading method of the embodiment of the present invention, the description information of the sample set and the field information of the sample can be determined, and the configuration file can be determined based on the description information and the field information, and the configuration file can be generated. The sample collection is suitable for the storage of more complex types of sample collections. According to the configuration file, the information of each type of sample data in the sample collection can be obtained, which is convenient for reading the sample data.

In a possible implementation manner, the description information includes description information of the label of the sample, wherein, according to the label of the sample in the sample set, determining the description information corresponding to the sample set includes: according to the label of the sample in the sample set Annotate, obtain an annotation list, the annotation list includes the type of the annotation of the sample; add a description to the annotation in the annotation list, and obtain the description information of the annotation of the sample. In this way, the description information of the label of the sample can be obtained for the label of the sample, so that the meaning of the label of each sample is unified, and it is convenient to store and read the sample.

In a possible implementation manner, the sample set includes samples of at least one data type.

In a possible implementation manner, the field information further includes sample reading method information, wherein the method further includes: determining the corresponding reading method for each data type according to the data type of the sample in the sample set; According to the data type of the sample and the corresponding reading mode, information about the reading mode of the sample is obtained.

In this way, a variety of sample reading methods can be determined, which is beneficial to support multiple types of data storage, adapt to the storage of more complex types of data sets, and at the same time facilitate the acquisition of suitable reading methods according to the configuration file when reading samples. .

In a possible implementation manner, the determining the description information corresponding to the sample set according to the labels of the samples in the sample set further includes: determining index range information used to classify the types of the samples in the sample set; The description information of the sample and the index range information of the sample category generate description information corresponding to the sample set.

In this way, when reading the sample, it is convenient to find the target sample in a specific index range, and the reading efficiency is improved.

In a possible implementation, the field information includes a field list of multiple fields of the sample, wherein the field information corresponding to the sample set is determined according to the fields and storage locations of the samples in the sample set. The method includes: performing summary processing on the fields of a plurality of samples in the sample set to obtain the field list.

In this way, each field can be summarized into a field list, which is easy to read.

In a possible implementation, the storage location includes a storage path of the sample set and an index position of the sample in the sample set. In this way, the accurate storage location of each sample can be obtained, which is easy to read.

In a possible implementation manner, the field information includes instruction information indicating whether to allow multiple samples to be read.

In this way, it is possible to indicate whether multiple samples are allowed to be read at one time, facilitating the reading of samples.

According to one aspect of the embodiments of the present invention, there is provided a reading method, including: obtaining the storage location and target field of the target sample in the sample set according to a configuration file corresponding to the sample set; and reading the storage location according to the storage location. The target parameter value of the target field of the target sample; wherein the configuration file is determined based on the description information of the sample set and the field information of the sample.

In a possible implementation manner, the method further includes: obtaining a reading manner of the target sample according to the configuration file.

In a possible implementation manner, reading the target parameter value of the target field of the target sample according to the storage location includes: obtaining the storage path of the sample set and the location of the target sample according to the storage location of the target sample The index position of the sample set; query the target sample according to the storage path of the sample set and the index position of the target sample in the sample set; read the target sample according to the reading method of the target sample to obtain the target sample The parameter value of one or more fields of the target sample; among the parameter values of one or more fields of the target sample, the target parameter value is determined.

In a possible implementation manner, reading the target parameter value of the target field of the target sample according to the storage location includes: obtaining the storage path of the sample set and the location of the target sample according to the storage location of the target sample The index position of the sample set; according to the storage path of the sample set and the target field, the storage position of the parameter value of the target field of the multiple samples in the sample set is obtained; according to the index position of the target sample in the sample set, Among the storage locations of the parameter values of the target field of the multiple samples, the storage location of the parameter values of the target field of the target sample is determined; according to the reading method of the target sample, in the target field of the target sample The target parameter value is read from the parameter value storage location.

In a possible implementation manner, the configuration file includes instruction information indicating whether to allow reading of multiple samples, and reading the target parameter value of the target field of the target sample according to the storage location includes: obtaining all The preset reading mode of the target sample; according to the preset reading mode and the instruction information, the target parameter value is read in the storage location of the parameter value of the target field of the target sample.

In a possible implementation manner, reading the target parameter value in the storage location of the parameter value of the target field of the target sample according to the preset reading mode and the instruction information includes: When the preset reading mode does not match the instruction information, the reading method corresponding to the instruction information is used to read the parameter value of the target field of the target sample.

According to an aspect of the embodiments of the present invention, there is provided a storage device, including: The first determining part is configured to determine the description information corresponding to the sample set according to the label of the sample in the sample set; The second determining part is configured to determine the field information corresponding to the sample set according to the column and storage location of the sample in the sample set; The third determining part is configured to determine the configuration file of the sample set based on the description information and the field information.

In a possible implementation, the description information includes the description information of the label of the sample: Wherein, the first determining part is further configured as: Obtaining a label list according to the labels of the samples in the sample set, the label list including the label categories of the samples; Add a description to the label in the label list to obtain the description information of the label of the sample.

In a possible implementation manner, the sample set includes samples of at least one data type.

In a possible implementation manner, the field information also includes information on how to read the sample; Wherein, the device further includes: The fourth determining part is configured to determine the corresponding reading method for each data type according to the data type of the samples in the sample set; The first obtaining part is configured to obtain the reading mode information of the sample according to the data type of the sample and the corresponding reading mode.

In a possible implementation manner, the first determining part is further configured to: Determine the index range information used to classify the types of samples in the sample set; According to the description information of the samples in the sample set and the index range information of the sample category, the description information corresponding to the sample set is generated.

In a possible implementation manner, the field information includes a field list of multiple fields of the sample; Wherein, the second determining part is further configured as: The fields of the multiple samples in the sample set are summarized to obtain the field list.

In a possible implementation, the storage location includes a storage path of the sample set and an index position of the sample in the sample set.

In a possible implementation manner, the field information includes instruction information indicating whether to allow multiple samples to be read.

According to an aspect of the embodiments of the present invention, there is provided a reading device, including: The first obtaining part is configured to obtain the storage location and target field of the target sample in the sample set according to the configuration file corresponding to the sample set; The first reading part is configured to read the target parameter value of the target field of the target sample according to the storage location; Wherein, the configuration file is determined based on the description information of the sample set and the field information of the sample.

In a possible implementation manner, the device further includes: According to the profile, the reading mode of the target sample is obtained.

In a possible implementation manner, the first reading part is further configured to: Obtaining the storage path of the sample set and the index position of the target sample in the sample set according to the storage location of the target sample; Query the target sample according to the storage path of the sample set and the index position of the target sample in the sample set; According to the reading method of the target sample, the target sample is read to obtain the parameter value of one or more fields of the target sample; Determine the target parameter value among the parameter values of one or more fields of the target sample.

In a possible implementation manner, the first reading part is further configured to: Obtaining the storage path of the sample set and the index position of the target sample in the sample set according to the storage location of the target sample; Acquiring, according to the storage path of the sample set and the target field, the storage locations of the parameter values of the target fields of the multiple samples in the sample set; According to the index position of the target sample in the sample set, among the storage positions of the parameter values of the target field of the multiple samples, determine the storage position of the parameter value of the target field of the target sample; According to the reading mode of the target sample, the target parameter value is read in the storage location of the parameter value of the target field of the target sample.

In a possible implementation manner, the configuration file includes instruction information indicating whether to allow multiple samples to be read; The first reading part is further configured to: Acquiring the preset reading mode of the target sample; According to the preset reading mode and the instruction information, the target parameter value is read in the storage location of the parameter value of the target field of the target sample.

In a possible implementation manner, the first reading part is further configured to: In the case that the preset reading mode does not match the instruction information, the reading method corresponding to the instruction information is used to read the parameter value of the target field of the target sample.

According to an aspect of the embodiments of the present invention, there is provided an electronic device, including: processor; Memory used to store executable instructions of the processor; Wherein, the processor is configured to execute the above-mentioned storing and/or reading method.

According to an aspect of the embodiments of the present invention, there is provided a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions implement the above-mentioned storing and/or reading methods when executed by a processor.

According to one aspect of the embodiments of the present invention, there is provided a computer program, including computer-readable code. When the computer-readable code is run in an electronic device, a processor in the electronic device executes for realizing the aforementioned storage and / Or the method of reading.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the embodiments of the present invention.

According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the embodiments of the present invention will become clear.

Various exemplary embodiments, features, and aspects of embodiments of the present invention will be described in detail below with reference to the drawings. The same reference numerals in the drawings indicate components with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, unless otherwise noted, the drawings are not necessarily drawn to scale.

The dedicated word "exemplary" here means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" need not be construed as being superior or better than other embodiments.

The term "and/or" in this article is only an association relationship describing related objects, which means that there can be three relationships. For example, A and/or B can mean: A alone exists, A and B exist at the same time, and B exists alone. three conditions. In addition, the term "at least one" herein means any one or any combination of at least two of the multiple, for example, including at least one of A, B, and C, and may mean including those made from A, B, and C Any one or more elements selected in the set.

In addition, in order to better illustrate the embodiments of the present invention, numerous details are given in the following embodiments. Those skilled in the art should understand that the embodiments of the present invention can also be implemented without certain details. In some instances, the methods, means, components, and circuits well-known to those skilled in the art have not been described in detail, so as to highlight the gist of the embodiments of the present invention.

Fig. 1 shows a flowchart of a storage method according to an embodiment of the present invention. As shown in Fig. 1, the method includes: In step S11, the description information corresponding to the sample set is determined according to the labels of the samples in the sample set; In step S12, the column information corresponding to the sample set is determined according to the column and storage location of the sample in the sample set; In step S13, the configuration file of the sample set is determined according to the description information and the field information.

According to the reading method of the embodiment of the present invention, the description information of the sample set and the field information of the sample can be determined, and the configuration file is determined based on the description information and the field information, and the configuration file is generated. It can support samples of multiple data types to be stored in A sample set is suitable for the storage of more complex types of sample sets. According to the configuration file, the information of each type of sample data in the sample set can be obtained, which is convenient for reading the sample data.

In a possible implementation, the execution subject of the storage method may be a terminal device or a server or other processing equipment, where the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, Terminals, cellular phones, wireless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. In some possible implementations, the method can be implemented by a processor calling computer-readable instructions stored in memory or executed by a server. For example, the data in the server can be executed by the storage method. store.

In a possible implementation, the sample set may include multiple samples, the data types of the samples may include images, videos, digital or text data, etc., and the storage format of the samples may include CSV (Comma-Separated Values, Character-separated value file), HDF5 (Hierarchical Data Format 5, hierarchical data format 5), etc. The embodiment of the present invention does not limit the data type and storage format of the sample. The sample set may store each sample in the form of a list, that is, each sample can occupy a row of storage area in the list, and the row of storage area may include one or more fields of the sample. Alternatively, each sample can be stored in the form of a dictionary, an array, etc. The embodiment of the present invention does not limit the storage method. When the sample set is saved, the configuration file corresponding to the sample set can be generated based on the sample set description and other information input by the user, or the configuration file corresponding to the sample set can be automatically generated based on the sample information of the stored sample set.

Fig. 2 shows a schematic diagram of a list according to an embodiment of the present invention. As shown in Fig. 2, the list may take the index of the sample as the row coordinates and the column as the column coordinates. For example, a sample (such as sample 1) is a person’s data, and the sample’s fields can include the person’s height (for example, field 1), weight (for example, field 2), and academic qualifications (for example, field 3) Etc., and the sample can have data corresponding to each field. The embodiment of the present invention does not limit the field.

In a possible implementation manner, the sample set may include a csv file, and the csv file may store information such as numeric data, categorical data, character data, and data path in the form of a table. The sample set may include audio, video, image and other audiovisual image files, for example, it may include AU files, MP4 files, JPEG files, etc. The sample set may also include HDF5 files, which can record audio, video, image, etc. files File format. The sample set may also include a txt file, and the txt file may store character data and the like. The sample set may include at least one of the aforementioned types of files, and the samples in the sample set are data in the included files. In an example, the sample set may include multiple types of subsets. For example, a certain subset is a csv file that includes multiple numerical data samples, and the other subset is a subset of image samples. In the example, each subset may store samples in the form of a list. For example, the csv file includes a list composed of multiple numerical data samples, and the subset composed of image samples includes a list composed of multiple image samples. It is also possible to summarize the lists of multiple subsets to obtain a list of all samples in the stored sample set. The embodiment of the present invention does not limit the form of the list in the sample collection.

In a possible implementation, the sample set can include samples of multiple data types, and each type of sample can have a specific storage and reading method. When storing multiple samples in the sample set, you can pass the column corresponding to the sample set. The bit information describes the sample set to facilitate storage or reading of multiple types of samples. In the example, the sample set includes multiple videos, and the field information may include a description of how the videos are stored or read.

In a possible implementation manner, the samples in the sample set may be training samples used in training the neural network, and the sample set may have descriptive information describing the annotations. The description information may be description information in the metadata meta format, and the embodiment of the present invention does not limit the type of the description information. In the example, each sample may have a label, for example, a label used to indicate information such as the type, attribute, or probability of the sample. For example, the sample may include a face image sample. When performing identity verification through a neural network, the neural network combines each face image sample with a reference image (for example, a face image of a certain target person). Make a comparison and determine the similarity between the face image sample and the reference image. When the similarity is greater than or equal to the similarity threshold (for example, 0.5), it can be determined that the face image sample has a high similarity with the reference image, and the identity verification passes; otherwise, the identity verification fails. When training the neural network, it is necessary to label each face image sample to provide parameters for the loss function of the neural network. For example, if a face image sample is the face image of the target person, it can be marked as 1, which means that the face in the face image sample is the face of the target person. If the face image sample is not The face image of the target person can be marked as 0, which means that the face in the face image sample is not the face of the target person.

In a possible implementation manner, the description information includes description information of the label of the sample, wherein step S11 may include: obtaining a label list according to the label of the sample in the sample set, the label list including the label of the sample Category: add a description to the label in the label list to obtain the description information of the label of the sample.

In a possible implementation, the description information of the label of the sample may be information describing the label. The labeling of each sample can be summarized to obtain a labeling list. In an example, the sample may include a face image sample, each face image sample may be marked as 0 or 1, and a sample marked as 0 may indicate that the face in the face image sample is not of the target person Face, the sample marked as 1 may indicate that the face in the face image sample is the face of the target person. The labels can be summarized. For example, labels 0 and 1 can be summarized into a label list, that is, the label list includes two columns, one column is 0 and the other column is 1. The embodiment of the present invention does not limit the type of the annotation and the type of the annotation list. Of course, in other embodiments, the label names, that is, the label descriptions may also be summarized to generate a list (label_names) of the label names of each sample included in the sample set.

In a possible implementation, you can add a description to each label in the label list. For example, you can add a description of "the face in the face image sample is not the face of the target person" to the column labeled 0. Add the description of “the face in the face image sample is the face of the target person” to the column labeled 1, and after adding the description for each label, the label description information (label_imap) of the sample can be obtained. The above labeling and description are only examples, and the embodiment of the present invention does not limit the labeling type and description.

In a possible implementation manner, in the process of storing samples in the sample set, the labels of the samples are all described in a unified way, for example, the column marked as 0 is added "face in the face image sample" The description of the face that is not the target person", the description of "the face in the face image sample is the face of the target person" is added to the column marked with 1, which is not interchangeable. Various types of information can be in a unified way Add a description.

In a possible implementation, the description information of the label of the sample can be configured as dictionary-type data, that is, the key of the dictionary is the label, and the value of the dictionary is the description of the label, and the description information of the label of the sample is described in the embodiment of the present invention. There is no restriction on the type of data.

In this way, the description information of the label of the sample can be obtained for the label of the sample, so that the meaning of the label of each sample is unified, and it is convenient to store and read the sample.

In a possible implementation manner, the description information further includes index range information used to classify the types of samples in the sample set. The process of determining description information by the method further includes: determining index range information used to classify the categories of each sample in the sample set; generating and matching the sample set based on the description information of the samples in the sample set and the index range information of the sample categories. Corresponding description information. For example, it can be classified according to the label of the sample. For example, the sample set can be divided into a training set, a validation set, and a test set. The training set includes 5000 samples, of which 3000 samples are labeled as 1, and 2000 samples are labeled as 0. Alternatively, the labels may not be distinguished, and the classification may be directly based on the number and index of the samples in the sample set. For example, the samples in the sample set may be training samples used when training a neural network, and the training samples may be divided into multiple categories, for example, 40% of the samples can be divided into the training set, 30% of the samples can be divided into the test set, and 30% of the samples can be divided into the validation set. Or it can be divided according to the index of each sample, and the index range of each category can be determined. For example, the sample set stores each sample in the form of a list, and 10,000 samples are stored in the list. The 1st-4000th sample in the list can be divided into the training set, and the 4001th-1st sample in the list can be divided into training sets. The 7000 samples are divided into the test set, and the 7001th-10000th samples are divided into the verification set. Among them, the description information may include index range information (for example, 1st-4000th, 4001th-7000th, 7001th-10000th, etc.) for dividing sample categories. The embodiment of the present invention does not limit the division method and the index range.

In addition to the sample subsets of the above-mentioned training set, test set, and validation set, the sample set may also include other forms of data subsets, such as image subsets, video subsets, and so on. The description information of the sample set can include the description information of other forms of data subsets, such as image subsets, video subsets, etc., in addition to the sample subset description information of the three sample categories of training set, test set, and validation set. .

In this way, when reading the sample, it is convenient to find the target sample in a specific index range, and the reading efficiency is improved.

The description information includes but is not limited to the above description information, and can also include other types of description information, for example, profile information about the sample, attribute information, space occupation information, etc. The embodiment of the present invention does not limit the information included in the description information.

In a possible implementation manner, in step S12, field information (fields) of the sample set can be determined according to the fields and storage locations of the samples. For example, the field information of each sample file can be determined according to the sample field and storage location of each sample file (such as CSV file, HDF5 file) in the sample set, and then the field information of the entire sample set can be obtained. Or according to the fields and storage locations of samples of various types (such as pictures, videos, texts, etc.) in the sample set, determine the field information of each type of sample, and then obtain the field information of the entire sample set. Of course, the field information of the entire sample set can also be obtained according to the fields and storage locations of all samples in the sample set. Each sample in the sample set can include one or more fields. For example, a sample is a person’s data, the sample’s fields can include the person’s height, weight, education, nationality, etc., and the other sample is a certain For an individual's face image, the sample field may include the person's identity information, etc. The embodiment of the present invention does not limit the field.

In a possible implementation manner, the field information includes a field list (name) of a plurality of fields of a sample, wherein step S12 may include: performing a summary processing on the fields of each sample in the sample set to obtain The list of said fields. For example, you can summarize all the fields in the field list, for example, you can fill in fields such as the person's height, weight, education, nationality, etc. into the field list, and fill in fields such as the identity information of the face image. A list of bits, the list of fields may represent the fields of multiple samples in the sample set. For example, the sample set includes multiple sample files or samples of multiple types, each sample file or each type of sample can correspond to a field list, and each field list has corresponding sample files or multiple samples of corresponding types Included fields. Or, you can use a field list to include the fields of all samples in the sample set. Each field corresponds to a row of data in the list of samples. In some areas of the row of data, the data can be empty. For example, in the list of samples, the "nationality" field is included. In a sample of a person's data, this field can contain data, such as "China", "United States", etc., while in the face image sample , There is no data in this field, that is, in the list, the data is empty in the row where the face image sample is located and the area corresponding to the row where the "nationality" field is located.

In a possible implementation, the field information may include a record of the storage location of the sample, and the storage location includes the storage path of the sample set and the index position of the sample in the sample set. For example, the storage path of the sample set is that the sample set is stored in the "sample set" folder in the C slot of the computer.

In a possible implementation, the index position of the sample in the sample set is the specific position of the sample. In the example, the data of each sample can be stored in the sample set in the form of an object, that is, occupy a certain storage area in the sample set. The sample set can store each sample in the form of a list, the data of the sample can occupy a certain row in the sample set, and the index position can represent the row number, for example, the first sample is located in the first row of the sample set list, The 2 samples are located in the second row of the sample set list... The field information can include the storage path of the sample set and the index position of each sample. The specific location of each sample can be determined according to the field information, for example, the first The 2 samples are located in the second row of the sample set list in the "sample set" folder in the C slot of the computer. For another example, some samples may only have certain fields, or multiple samples belong to the same type or the same sample file. The index position may be the row and column range of the sample in the sample set, or the row and column range of multiple samples. The embodiment of the present invention does not limit the storage location.

In a possible implementation manner, the sample set includes samples of at least one data type. For example, samples of CSV, HDF5, txt, and other image types. The field information also includes the reading method information of each sample, wherein the method further includes: determining the reading method for reading the sample according to the data type of the sample in the sample set; and according to the data type of the sample and Corresponding to the reading method, obtain the reading method information of the sample.

In a possible implementation manner, the reading mode information may be information describing the reading mode of the sample. The reading method of the sample can be judged according to the data type of the sample. For example, if the data type of a sample is an image, the sample can be read by the image reading function, and the data type of a sample is video. Then the sample can be read by the video reading function, and the reading method of each sample can be determined separately. The data type of each sample can also be summarized. For example, the sample file included in the sample set or the sample set includes multiple samples. There are three types of data for the multiple samples, which are images, videos, and general data, which can be determined Read the sample of the image type through the image reading function, read the sample of the video type through the video reading function, read the sample of the common data type through the data reading function, then the reading method of the sample set or sample file The information may include the above-mentioned data types and corresponding reading methods. The embodiment of the present invention does not limit the data type and reading method of the sample.

In a possible implementation manner, the reading method information (proc) can be obtained according to the data type of the sample and the corresponding reading method. In the example, the reading method information can be configured as dictionary data. For example, the key of the dictionary is the data type, and the value of the dictionary is the reading method corresponding to the data type. For example, the key of the dictionary is the image and the value of the dictionary. It is the image reading function. The embodiment of the present invention does not limit the data type of the reading mode information.

In a possible implementation manner, the field information may not include the reading method information. When reading the sample, the reading method can be determined according to the information such as the suffix of the sample file. For example, the suffix of a sample file is txt. You can select the text opening method to read the sample. If the end code of a sample file is jpg, you can select the image opening method to read the sample. The embodiment of the present invention does not limit the tail code and the opening method of the sample.

In this way, a variety of sample reading methods can be determined, which is beneficial to support multiple types of data storage, adapt to the storage of more complex types of data sets, and at the same time facilitate the acquisition of suitable reading methods according to the configuration file when reading samples. .

In a possible implementation manner, the field information further includes instruction information (fullonly) indicating whether multiple samples are allowed to be read, and the instruction information may indicate whether the data of multiple samples can be read at one time. For example, if the instruction information indicates that the data of all samples in the sample set or sample file can be read at one time, in the case of reading data, all samples in the sample set or sample file can be read, for example, HDF5 file Supports reading the samples in the entire file at one time. If the instruction information indicates that the data of the sample set or the sample in the sample file cannot be read at one time, in the case of reading the data, the sample set or all the samples in the sample file cannot be read at one time, and it supports reading at any time, such as , Only one sample or part of the sample data can be read at a time.

In a possible implementation, in step S13, the configuration file (index.yml) of the sample set can be determined based on the description information and the field information. In the example, the content of the description information and the field information can be written In the configuration file, or package the description information and the field information to obtain the configuration file, the embodiment of the present invention does not limit the way of obtaining the configuration file.

In a possible implementation, the configuration file may be a file that records the storage methods of multiple samples, that is, the configuration file includes information such as the fields, labels, storage locations, and reading methods of each sample, and various data types Samples can be stored in the sample collection, and information such as fields, annotations, storage locations, and reading methods are saved in the configuration file for easy reading. The configuration file provides a basis for storing and reading samples of multiple data types .

Fig. 3 shows a flowchart of a reading method according to an embodiment of the present invention. As shown in Fig. 3, the method includes: In step S14, obtain the storage location and target field of the target sample in the sample set according to the configuration file corresponding to the sample set; In step S15, the target parameter value of the target field of the target sample is read according to the storage location.

Wherein, the configuration file is determined based on the description information of the sample set and the field information of the sample.

According to the reading method of the embodiment of the present invention, the storage location and target field of the target sample can be determined according to the configuration file, and the target parameters of the target sample can be read. Various types of samples can be read in the sample set according to the configuration file. , So that different types of samples can be read through the configuration file, no need to configure the program interface for each type of sample, which facilitates the storage and reading of sample data, and improves the reading efficiency. Based on this configuration file, it can support the storage and reading of sample files of multiple data types (for example, CSV, HDF5, samples of other file (image, audio) types), adapting to the reading of samples in more complex types of sample sets .

In a possible implementation manner, the configuration file may be a file used to describe the samples in the sample set, and the configuration file may be used to read the samples in the sample set. The configuration file may include description information and field information corresponding to the sample set. In the case of reading a sample, the sample can be read based on the description information and field information. There can be samples of multiple data types in the sample set, such as CSV, HDF5, other files (image and audio) and other types of samples. The descriptive information may be descriptive information marked on the sample, and may include a marked list of each sample in the sample set, marked description information, and index range information used to classify each sample in the sample set. The field information can be the information used to describe the sample. The field information can record the field list, data type, reading method, storage path, index location, etc. of a variety of samples. It can read a certain Samples, such as target samples. Samples of each data type can be read in the above-mentioned manner, so multiple types of data can be read.

In a possible implementation manner, the sample in the sample set or the column data of the sample can be read based on the configuration file. In step S14, the sample set can store each sample in the form of a list, and the storage location of the target sample to be read and the target field to be read can be obtained from the field information of the configuration file. And in step S15, the target parameter value of the target field of the target sample is read.

In a possible implementation manner, in the case of reading the target sample, the reading method of the target sample can be determined according to the name or end code of the sample file to which the target sample belongs. For example, if the end code of a sample file is txt, then You can choose the text opening method to read the sample. If the end code of a sample file is jpg, you can choose the picture opening method to read the sample. Alternatively, the field information of the configuration file may include a sample, a sample file, or a reading method of a certain data type, and the method further includes: obtaining a reading method of a target sample according to the configuration file. That is, in the field information of the configuration file, the reading method information of the sample can be obtained, for example, the reading method of the target sample can be directly obtained, or the corresponding reading method can be obtained according to the sample file (such as CSV or HDF5, etc.) to which the target sample belongs. Alternatively, the corresponding reading method can be obtained according to the data type (such as image, text, etc.) to which the target sample belongs, and then the target sample can be read through the reading method.

In a possible implementation manner, in step S14, the target sample can be found according to the storage location of the target sample, and the target sample and the target field of the target sample can be read according to the reading method of the target sample. It can be any one or more of the multiple fields of the target sample, or all the fields of the target sample. Step S14 may include: obtaining the storage path of the sample set and the index position of the target sample in the sample set according to the storage location of the target sample; querying all data according to the storage path of the sample set and the index position of the target sample in the sample set The target sample; according to the reading method of the target sample, the target sample is read to obtain the parameter value of one or more fields of the target sample; the parameter value of one or more fields of the target sample , Determine the target parameter value.

In a possible implementation, the storage location of the target sample can be determined. For example, according to the storage path of the sample set recorded in the field information and the index position of the target sample in the sample list, the precise location of the target sample can be determined, and Query the target sample by precise location. For example, the target sample is sample 2. According to the storage location of the sample, the sample can be found in the second row of the sample set list in the "sample set" folder in the C slot of the computer.

In a possible implementation manner, the target sample can be read by the target sample reading method recorded in the reading mode information. For example, if the target sample is text data, the target sample can be read by the text data reading function. The parameter value of one or more fields of the target sample can be obtained. Further, the target parameter value of the target field can be determined from the parameter values of one or more fields. For example, the parameter values of the height, weight, education, nationality and other fields of the sample 2 can be read, where the target field is the height, and the parameter value of the height field can be determined from the above parameter values.

In a possible implementation manner, in step S14, a certain field (target field) may be obtained from the field information of the configuration file, and the parameter values of the target fields of multiple samples may be obtained in the list. Further, the parameter value of the target field of the target sample can be obtained. Step S14 may include: obtaining the storage path of the sample set and the index position of the target sample in the sample set according to the storage location of the target sample; obtaining multiple sample sets according to the storage path of the sample set and the target field The storage position of the parameter value of the target field of the sample; according to the index position of the target sample in the sample set, the parameter value of the target field of the target sample is determined in the storage position of the parameter value of the target field of the multiple samples The storage location of; according to the reading method of the target sample, read the target parameter value in the storage location of the parameter value of the target field of the target sample.

In a possible implementation, the storage location of the target sample can be determined based on the field information. For example, the target sample can be determined based on the storage path of the sample set recorded in the field information and the index position of the target sample in the sample list. Precise location. For example, the storage path of the sample set is the sample set in the "sample set" folder in the C slot of the computer, and the index position of the target sample is the second row of the sample set list.

In a possible implementation manner, the target field can be selected in the field information, and the storage location of the parameter value of the target field can be obtained according to the storage path and the target field of the sample set. For example, you can obtain the storage location of the parameter value of the height field in the sample set in the "sample set" folder in the C slot of the computer. And according to the index position of the target sample in the sample set, determine the storage location of the target field of the target sample, for example, determine the storage location of the height field of sample 2 from the storage locations of multiple parameter values of the height field, namely , The storage location of the second row in the storage location of multiple parameter values in the height field. Further, the reading method of the target sample can be obtained in the configuration file, and the target parameter value can be read, for example, the target parameter value in the storage location can be read through a data reading function.

In a possible implementation manner, the target samples can also be read according to the annotations. For example, in the training process of the neural network, the samples of each annotation type can be read according to a preset ratio, for example, a predetermined number of samples can be read , Of which 50% are samples labeled 1, and 50% are samples labeled 0. Or you can read samples based on the index range information. For example, in the case of training, you can read the samples in the training set, in the case of testing, you can read the samples in the test set, and in the case of verification, you can read the samples in the verification set. sample. The embodiment of the present invention does not limit the reading mode.

In a possible implementation manner, the configuration file includes instruction information indicating whether to allow reading of multiple samples, and reading the target parameter value of the target field of the target sample according to the storage location includes: obtaining all The preset reading mode of the target sample; according to the preset reading mode and the instruction information, the target parameter value is read in the storage location of the parameter value of the target field of the target sample. In the example, in the case of reading samples, the user can set a preset reading mode, which can be read at any time for each sample, or all samples can be read at once. For example, for samples of images, videos, texts, etc., the user can set it to be read as needed, that is, one or more samples can be read separately. It is also possible to read multiple samples in sequence, or read all samples at once.

In a possible implementation manner, reading the target parameter value in the storage location of the parameter value of the target field of the target sample according to the preset reading mode and the instruction information includes: When the preset reading mode does not match the instruction information, the reading method corresponding to the instruction information is used to read the parameter value of the target field of the target sample. Some data format samples only support reading all samples at one time. For example, HDF5 format samples can only read all samples at one time. At this time, the instruction information of the sample file is the instruction information that allows multiple samples to be read (such as Supports reading the instruction information of all samples at one time). If the default reading mode is read as you want, the default reading mode does not match the instruction information, and you cannot use the default reading mode to read samples in HDF5 format. Therefore, according to the instruction information of this type of sample, all HDF5 format samples can be read at one time, and then the target parameter value of the target field of the target sample can be queried among all the samples read. Similarly, when some data format samples do not support reading all samples at once, the instruction information can be set to the instruction information that does not allow reading multiple samples (if not supporting reading all samples at once), if The default reading mode is to read all samples at once. The default reading mode does not match the instruction information, and the default reading mode cannot be used to read the samples. Therefore, you can follow the instruction information of this type of sample. Read the sample data.

According to the reading method of the embodiment of the present invention, the description information of the label of the sample can be obtained for the label of the sample, so that the meaning of the label of each sample is unified, which is convenient for the storage and reading of the sample, and the identification information of multiple types of samples can be determined. The reading method is conducive to supporting multiple types of data storage and adapting to the storage of more complex types of data sets. Further, the configuration file is determined based on the description information and field information, and various types of samples can be read in the sample set according to the configuration file, so that different types of samples can be obtained and read through the configuration file. The program interface is configured for each type of sample to facilitate the storage and reading of sample data. It can support multiple types of data storage and adapt to the storage of more complex types of data sets.

Figure 4 shows a schematic diagram of the application of the reading method according to an embodiment of the present invention. As shown in Figure 4, the sample set can include multiple samples, and the data types of the samples can include images, audio, video, and/or common data (for example, , Numeric data, categorical data, character data), etc. The sample set can store each sample in the form of a list, that is, each sample can occupy a row of storage areas in the list of samples. Each sample can include one or more fields. For example, a sample is a person’s data. The sample’s fields can include the person’s height, weight, education, nationality, etc., and the sample can have various fields. Corresponding information.

In a possible implementation, the samples in the sample set may include multiple data types, each type of sample may have a specific storage and reading method, and each sample may have a corresponding label. The configuration file (index.yml) may include description information, for example, description information in a meta format of metadata, and the description information may record the label of each sample and the description of the label.

In the example, the label description information (label_imap) of the sample can be dictionary data, the key of the dictionary is the label, and the value of the dictionary is the description of the label. For example, the key of the dictionary is 0, and the value of the dictionary is the object in the sample. It is not the target object, the key of the dictionary is 1, and the value of the dictionary is the value of the dictionary. The object in the sample is the target object.

In the example, the description information also includes index range information used to classify the types of samples in the sample set (such as sample subsets). For example, if the samples in the sample set are training samples used when training a neural network, you can add The training samples are divided into multiple categories. For example, 40% of the samples can be divided into the training set, 30% of the samples can be divided into the test set, and 30% of the samples can be divided into the validation set. The index range information determines the index range of the training set samples, the test set samples, and the validation set samples in the sample list.

In a possible implementation manner, the configuration file may include field information, and the field information includes a field list (name) of multiple fields of the sample, and the storage location of the sample. It can be used to read the data of the sample through the field and storage location.

In the example, the field information can also include the reading method information of the sample. The reading method information of the sample (proc) can be dictionary data. The key of the dictionary is the data type of the sample, and the value of the dictionary corresponds to the data type. The reading method, for example, the key of the dictionary is an image, the value of the dictionary is an image reading function, the key of the dictionary is a video, and the value of the dictionary is a video reading function, etc.

In the example, the fields of each sample can be summarized. For example, a sample is the data of a person, and the fields of the sample can include the person's height, weight, education and nationality; the other sample is the person The face image sample of, the fields of the face image sample can include the person’s identity information (for example, name, number, etc.), and all the above fields can be summarized to obtain a list of fields, for example, a list of fields It includes fields such as height, weight, education, nationality, and identity information.

In an example, the field information can also include the storage location of the sample, which can determine the storage location of each sample. The storage location can include the storage path of the sample set and the index position of the sample in the sample list, for example, the sample set’s index position. The storage path is the "sample set" folder in slot C, and the index position of the target sample to be queried is the 2nd row in the list of samples, then the storage location of the sample is in the "sample set" folder in slot C Line 2 of the list of sample sets.

In the example, the field information also includes instruction information (fullonly) indicating whether to allow all samples to be read. The instruction information can indicate whether or not to read the data of all samples at once. For example, if the instruction information indicates that the data of all samples can be read at one time, in the case of reading the data, all the samples in the sample set can be read, for example, the entire sample list can be read. If the instruction information indicates that the data of all samples cannot be read at one time, it is not possible to read all the samples at one time when the data is read, for example, the data of only one sample can be read at a time, or the data of a part of the sample can be read at a time .

In a possible implementation manner, the sample or the data of the sample in the sample set can be read based on the configuration file. For example, the index of the target sample to be read in the sample list can be obtained in the field information of the configuration file, and the reading method of the target sample can be obtained in the description file of the configuration file, according to the target sample in the list The index finds the storage location of the target sample, and reads the target sample or one or more parameter values of the target sample according to the reading method of the target sample. For another example, you can obtain a certain field in the field information, and obtain the storage location of the field data of multiple samples in the list (the samples whose field is empty can be ignored), and according to the target sample in the sample The index in the list of the target sample can query the storage location of the data in the field of the target sample, and read the data in the storage location of the field of the target sample according to the reading method of the target sample.

In a possible implementation, the storage and reading method can be used to store multiple types of samples. For example, in the field of computer education, using this storage method can store multiple types of samples according to a unified storage method. , Easy to store and read, easy for students to operate. The storage method is not limited to the programming language environment, and different types of samples can be stored in a unified manner using configuration files.

In the following, an exemplary application of the embodiment of the present invention in an actual application scenario will be described.

In machine learning, especially when multiple data sets need to be processed, the data set storage method is extremely important, and different data sets need to be organized according to the same agreement. There is no general agreement or rule for the storage of data sets at present, and the storage of data sets is usually to adapt to the structure of the data set itself and cannot be adapted to each other. Although the existing data storage methods such as CSV, HDF5, and file storage (image and audio) are powerful, the types of data supported are limited and cannot meet the storage requirements of various data in the data set.

In machine learning, data sets are becoming more and more extensive. We need to find a suitable data set storage protocol to store most of the data sets that are very different. Metadata describes complex data sets, so that the overall data framework is clearer and is more important for data collection. The description of various materials is also more complete.

Therefore, the embodiment of the present invention provides a data set storage protocol, which aims to organize machine learning-related data sets in the same way as possible, covering the metadata of the data set and various data file information. At the same time, this protocol can It is transplanted to the data set storage of different bottom layers to adapt to multiple language environments.

In a possible implementation manner, the embodiment of the present invention includes: 1. This agreement provides a data set storage solution through the configuration file. The agreement includes metadata and data set field information. 2. Metadata includes data description, subset range, data dictionary and other information. 3. The data set field includes information on various data files covered by the data set.

In a possible implementation manner, the embodiment of the present invention includes: Each data set contains a configuration file (index.yml), which contains metadata information (meta) and data set field information (fields) of various data in the data set.

Metadata (meta) can contain multiple pieces of information used to describe the data set, usually including (but not limited to) the following fields: 1. A list of label names of each sample (label_names), that is, a list of all label names; 2. The description information (label_imap) of the label of the sample, that is, a dictionary of one-to-one correspondence between the label name and the digital label; 3. The index range information (subsets) of each sample type, that is, for the machine learning model, the data set needs to be divided into training set, test set and validation set, including the index range of the corresponding data. Other forms of data subsets can also be stored.

The data set fields (fields) contain the following information: 1. Field list (name), that is, the field name of the data in the data set. 2. Index location (file), that is, the file path where the data is located and the corresponding row and column range. 3. Reading method information (proc), that is, data reading method (text, image, etc.) and data type. 4. Instruction information (fullonly), that is, whether to support reading all data at once.

The program can extract the required information from the corresponding file based on the metadata and data set field information of the configuration file.

In a possible implementation manner, the various types of stored data files in the embodiment of the present invention include: 1. csv file: data stored in a table. Usually can include numeric data, categorical data, character data, and other data paths and other information. 2. HDF5 file: Integrate all image data, audio data, etc. in the data set to provide support for fullonly. 3. txt file: a file for storing character data. 4. Other image files: According to different data types (audio, video, image, etc.), different data files (au files, MP4 files, JPEG files, etc.) are usually generated.

Compared with the disadvantage that there is no general data set storage protocol in the prior art, the embodiment of the present invention enables most machine learning data sets to be stored in a more uniform manner.

Compared with the prior art that is generally limited to storage in a subset language environment, the embodiment of the present invention can be stored in different bottom layers without being restricted by the language environment.

To sum up, the embodiments of the present invention provide a way to gather data of different storage methods together to form a data set, and is not restricted by specific programming languages and environments.

It can be understood that the various method embodiments mentioned in the embodiments of the present invention can be combined with each other to form a combined embodiment without violating the principle and logic. The length is limited, and the embodiments of the present invention will not be repeated.

In addition, the embodiments of the present invention also provide storage and reading devices, electronic equipment, computer-readable storage media, and programs. All of the above can be used to implement any of the storage and reading methods provided in the embodiments of the present invention. The corresponding technical solutions and descriptions And refer to the corresponding records in the method section, so I won’t repeat them here.

Those skilled in the art can understand that in the above-mentioned method of implementation, the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process. The execution order of each step should be based on its function and possible internal logic. Sure.

FIG. 5 shows a block diagram of a storage device according to an embodiment of the present invention. As shown in FIG. 5, the device includes: The first determining part 11 is configured to determine the description information corresponding to the sample set according to the labels of the samples in the sample set; The second determining part 12 is configured to determine the field information corresponding to the sample set according to the fields and storage locations of the samples in the sample set; The third determining part 13 is configured to determine the configuration file of the sample set based on the description information and the field information.

In a possible implementation manner, the description information includes description information of the label of the sample; Wherein, the first determining part is further configured as: Obtaining a label list according to the labels of the samples in the sample set, the label list including the label categories of the samples; Add a description to the label in the label list to obtain the description information of the label of the sample.

In a possible implementation manner, the sample set includes samples of at least one data type.

In a possible implementation manner, the field information also includes information on how to read the sample; Wherein, the device further includes: The fourth determining part is configured to determine the corresponding reading method for each data type according to the data type of the samples in the sample set; The first obtaining part is configured to obtain the reading mode information of the sample according to the data type of the sample and the corresponding reading mode.

In a possible implementation manner, the first determining part is further configured to: Determine the index range information used to classify the types of samples in the sample set; According to the description information of the samples in the sample set and the index range information of the sample category, the description information corresponding to the sample set is generated.

In a possible implementation manner, the field information includes a field list of multiple fields of the sample; Wherein, the second determining part is further configured as: A summary process is performed on the fields of each sample in the sample set to obtain the field list.

In a possible implementation, the storage location includes a storage path of the sample set and an index position of the sample in the sample set.

In a possible implementation manner, the field information includes instruction information indicating whether to allow multiple samples to be read.

Fig. 6 shows a block diagram of a reading device according to an embodiment of the present invention. As shown in Fig. 6, the device includes: The first obtaining part 14 is configured to obtain the storage location and target field of the target sample in the sample set according to the configuration file corresponding to the sample set; The first reading part 15 is configured to read the target parameter value of the target field of the target sample according to the storage location; Wherein, the configuration file is determined based on the description information of the sample set and the field information of the sample.

In a possible implementation manner, the device further includes: According to the profile, the reading mode of the target sample is obtained.

In a possible implementation manner, the first reading part is further configured to: Obtaining the storage path of the sample set and the index position of the target sample in the sample set according to the storage location of the target sample; Query the target sample according to the storage path of the sample set and the index position of the target sample in the sample set; According to the reading method of the target sample, the target sample is read to obtain the parameter value of one or more fields of the target sample; Determine the target parameter value among the parameter values of one or more fields of the target sample.

In a possible implementation manner, the first reading part is further configured to: Obtaining the storage path of the sample set and the index position of the target sample in the sample set according to the storage location of the target sample; Acquiring, according to the storage path of the sample set and the target field, the storage locations of the parameter values of the target fields of the multiple samples in the sample set; According to the index position of the target sample in the sample set, among the storage positions of the parameter values of the target field of the multiple samples, determine the storage position of the parameter value of the target field of the target sample; According to the reading mode of the target sample, the target parameter value is read in the storage location of the parameter value of the target field of the target sample.

In a possible implementation manner, the configuration file includes instruction information indicating whether to allow multiple samples to be read; The first reading part is further configured to: Acquiring the preset reading mode of the target sample; According to the preset reading mode and the instruction information, the target parameter value is read in the storage location of the parameter value of the target field of the target sample.

In a possible implementation manner, the first reading part is further configured to: In the case that the preset reading mode does not match the instruction information, the reading method corresponding to the instruction information is used to read the parameter value of the target field of the target sample.

In some embodiments, the functions or parts included in the apparatus provided by the embodiments of the present invention can be used to execute the methods described in the above method embodiments. For implementation, refer to the description of the above method embodiments. For brevity, this is not Go into details again.

An embodiment of the present invention also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions are executed by a processor to implement the above method. The computer-readable storage medium may be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.

An embodiment of the present invention also provides an electronic device, including: a processor; a memory configured to store executable instructions of the processor; wherein the processor is configured as the aforementioned method.

Electronic devices can be provided as terminals, servers, or other types of devices.

Fig. 7 is a block diagram showing an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and other terminals.

7, the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, and an input/output (Input/Output, I/O) interface 812 , The sensor component 814, and the communication component 816.

The processing component 802 generally controls the overall operations of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the foregoing method. In addition, the processing component 802 may include one or more parts to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia part to facilitate the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions of any application or method configured to operate on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be realized by any type of volatile or non-volatile storage device or their combination, such as static random-access memory (Static Random-Access Memory, SRAM), electrically erasable, programmable read-only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM) , Read-Only Memory (Read-Only Memory, ROM), magnetic memory, flash memory, floppy disk or CD-ROM.

The power supply component 806 provides power for various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with the generation, management, and distribution of power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or sliding action, but also detect the duration and pressure related to the touch or sliding operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC). When the electronic device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 further includes a speaker configured to output audio signals.

The I/O interface 812 provides an interface between the processing component 802 and the peripheral interface part. The peripheral interface part may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: home button, volume button, start button, and lock button.

The sensor component 814 includes one or more sensors, and is configured to provide the electronic device 800 with various aspects of state assessment. For example, the sensor component 814 can detect the on/off state of the electronic device 800 and the relative positioning of the components. For example, the component is the display and the keypad of the electronic device 800. The sensor component 814 can also detect the electronic device 800 or The position of a component of the electronic device 800 changes, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 may also include a light sensor, such as a CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge-coupled Device) image sensor, which is configured to perform imaging Used in the application. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access wireless networks based on communication standards, such as Wi-Fi, 2G (2-Generation wireless telephone technology, second-generation mobile communication technology) or 3G (3-Generation wireless telephone technology, third-generation mobile communication) Technology), or a combination of them. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) part to facilitate short-range communication. For example, the NFC part can be based on radio frequency identification (Radio Frequency Identification, RFID) technology, infrared data association (Infrared Data Association, IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (Bluetooth, BT) technology and other Technology to achieve.

In an exemplary embodiment, the electronic device 800 may be used by one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processing (Digital Signal Processing, DSP), and digital signal processing equipment (Digital Signal Processing). device, DSPD), programmable logic device (programmable logic device, PLD), field programmable gate array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor or other electronic components to achieve于Execute the above method.

In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to complete the above method.

The embodiment of the present invention also provides a computer program product, including computer readable code. When the computer readable code runs on the device, the processor in the device executes instructions for implementing the method provided in any of the above embodiments. .

The computer program product can be realized through hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium. In another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and so on.

Fig. 8 is a block diagram showing an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. 8, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, configured to store instructions that can be executed by the processing component 1922, such as an application program . The application program stored in the memory 1932 may include one or more parts each corresponding to a set of commands. In addition, the processing component 1922 is configured to execute instructions to perform the above-described methods.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input and output (I/O) Interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

The embodiments of the present invention may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling the processor to implement various aspects of the embodiments of the present invention.

The computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples of computer-readable storage media (non-exhaustive list) include: portable computer disks, hard disks, random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), Erasable programmable read-only memory (Electrical Programmable Read Only Memory, EPROM or flash memory), static random access memory (Static Random Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punched cards with instructions stored on them or convex grooves From the structure, and any suitable combination of the above. The computer-readable storage medium used here is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or passing through Electrical signals transmitted by wires.

The computer-readable program instructions described here can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network Device. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. The network interface card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for computer-readable storage in each computing/processing device Medium.

The computer program instructions used to perform the operations of the embodiments of the present invention may be assembly instructions, instruction set architecture (Industry Standard Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, or one or more Source code or object code written in any combination of multiple programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as "C" language or Similar programming language. Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or completely remotely executed. Run on the end computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network-including local area network (LAN) or wide area network (Wide Area Network, WAN)-or, it can be connected To an external computer (for example, using an Internet service provider to connect via the Internet). In some embodiments, the electronic circuit is personalized by using the status information of computer-readable program instructions, such as programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (programmable logic array, PLA), the electronic circuit can execute computer-readable program instructions to realize various aspects of the embodiments of the present invention.

Here, various aspects of the embodiments of the present invention are described with reference to the flowcharts and/or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each block of the flowchart and/or block diagram and the combination of each block in the flowchart and/or block diagram can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to the processors of general-purpose computers, special-purpose computers, or other programmable data processing devices, thereby producing a machine that allows these instructions to be executed by the processors of the computer or other programmable data processing devices In the case of, a device that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make the computer, programmable data processing device and/or other equipment work in a specific manner, so that the computer-readable medium storing the instructions is It includes an article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowchart and/or block diagram.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to generate a computer The process of implementation enables instructions executed on a computer, other programmable data processing device, or other equipment to implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions, and operations of systems, methods, and computer program products according to multiple embodiments of the embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction includes one or more logic for implementing the specified Executable instructions for the function. In some alternative implementations, the functions marked in the block may also occur in a different order than the order marked in the drawings. For example, two consecutive blocks can actually be executed basically in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions. It can be realized, or it can be realized by a combination of dedicated hardware and computer instructions.

The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Without departing from the scope and spirit of the illustrated embodiments, many modifications and changes are obvious to those of ordinary skill in the art. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements in the market of the various embodiments, or to enable other ordinary skilled in the art to understand the various embodiments disclosed herein.

11: The first determination part 12: The second certain part 13: The third determination part 14: The first acquisition part 15: The first reading part 800: electronic equipment 802: Processing component 804: memory 806: Power Components 808: Multimedia components 810: Audio component 812: input/output interface 814: Sensor component 816: Communication Components 820: processor 1900: electronic equipment 1922: processing components 1926: power supply components 1932: memory 1950: network interface 1958: Input and output interface S11~S15: steps

The drawings here are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments in accordance with the present invention, and are used together with the specification to illustrate the technical solutions of the embodiments of the present invention. Fig. 1 shows a flowchart of a storage method according to an embodiment of the present invention; Figure 2 shows a schematic diagram of a list according to an embodiment of the present invention; Fig. 3 shows a flowchart of a reading method according to an embodiment of the present invention; FIG. 4 shows a schematic diagram of the application of the reading method according to an embodiment of the present invention; Figure 5 shows a block diagram of a storage device according to an embodiment of the present invention; Fig. 6 shows a block diagram of a reading device according to an embodiment of the present invention; FIG. 7 shows a block diagram of an electronic device according to an embodiment of the present invention; Fig. 8 shows a block diagram of an electronic device according to an embodiment of the present invention.

S11~S13: steps

Claims (16)

A storage method includes: determining the description information corresponding to the sample set according to the labels of the samples in the sample set; determining the field information corresponding to the sample set according to the fields and storage locations of the samples in the sample set; The description information and the field information determine the configuration file of the sample set. The method according to claim 1, wherein the description information includes the description information of the label of the sample; wherein, determining the description information corresponding to the sample set according to the label of the sample in the sample set includes: according to the sample Collect the labels of the samples to obtain a label list, which includes the label categories of the samples; add a description to the labels in the label list to obtain the description information of the labels of the sample. The method according to claim 1 or 2, wherein the sample set includes samples of at least one data type. The method according to claim 3, wherein the field information further includes sample reading method information: wherein, the method further includes: determining the corresponding reading method for each data type according to the data type of the sample in the sample set Acquisition method; according to the data type of the sample and the corresponding reading method, the Information on how to read the sample. The method according to claim 1 or 2, wherein the determining the description information corresponding to the sample set according to the labels of the samples in the sample set further includes: determining index range information used to classify the samples in the sample set; According to the description information of the samples in the sample set and the index range information of the sample category, the description information corresponding to the sample set is generated. The method according to claim 1 or 2, wherein the field information includes a field list of multiple fields of the sample; wherein, according to the field and storage location of the sample in the sample set, it is determined to correspond to the sample set The field information of includes: summarizing the fields of multiple samples in the sample set to obtain the field list. The method according to claim 1 or 2, wherein the storage location includes a storage path of the sample set and an index position of the sample in the sample set. The method according to claim 1 or 2, wherein the field information includes instruction information indicating whether to allow multiple samples to be read. A reading method includes: obtaining the storage location and target field of a target sample in the sample set according to a configuration file corresponding to the sample set; and reading the target parameter of the target field of the target sample according to the storage location Value; wherein the configuration file is based on the description information and field data of the sample set News is confirmed. The method according to claim 9, further comprising: obtaining a reading mode of the target sample according to the configuration file. The method according to claim 10, wherein, according to the storage location, reading the target parameter value of the target field of the target sample includes: obtaining the sample set according to the storage location of the target sample According to the storage path of the sample set and the index position of the target sample in the sample set; query the target sample according to the storage path of the sample set and the index position of the target sample in the sample set; according to the reading method of the target sample , Reading the target sample to obtain parameter values of one or more fields of the target sample; determining the target parameter value among the parameter values of one or more fields of the target sample. The method according to claim 10, wherein, according to the storage location, reading the target parameter value of the target field of the target sample includes: obtaining the sample set according to the storage location of the target sample According to the storage path of the sample set and the index position of the target sample in the sample set; according to the storage path of the sample set and the target field, obtain the storage location of the parameter value of the target field of the multiple samples in the sample set; according to The index position of the target sample in the sample set, among the storage positions of the parameter values of the target field of the multiple samples, determine the storage position of the parameter values of the target field of the target sample; According to the reading mode of the target sample, the target parameter value is read in the storage location of the parameter value of the target field of the target sample. The method according to any one of claims 9 to 12, wherein the configuration file includes instruction information indicating whether to allow reading of multiple samples; according to the storage location, the target column of the target sample is read The target parameter value of the target sample includes: obtaining the preset reading mode of the target sample; according to the preset reading mode and the indication information, the parameter value in the target field of the target sample The target parameter value is read from the storage location. The method according to claim 13, wherein, according to the preset reading mode and the instruction information, the target sample is read in the storage location of the parameter value of the target field The target parameter value includes: in the case where the preset reading mode does not match the indication information, using the reading method corresponding to the indication information to read the target field of the target sample The parameter value. An electronic device comprising: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to call the instructions stored in the memory to execute any one of request items 1 to 6 The method described. A computer-readable storage medium on which a computer program is stored Formula instructions, when the computer program instructions are executed by the processor, the method described in any one of request items 1 to 14 is implemented.

Images