CN115185453A - Data processing method and device, electronic equipment and storage medium - Google Patents

Data processing method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115185453A
CN115185453A CN202210610620.1A CN202210610620A CN115185453A CN 115185453 A CN115185453 A CN 115185453A CN 202210610620 A CN202210610620 A CN 202210610620A CN 115185453 A CN115185453 A CN 115185453A
Authority
CN
China
Prior art keywords
target
data
parameter
storage
storage location
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210610620.1A
Other languages
Chinese (zh)
Inventor
张回归
刘鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dajia Internet Information Technology Co Ltd
Original Assignee
Beijing Dajia Internet Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dajia Internet Information Technology Co Ltd filed Critical Beijing Dajia Internet Information Technology Co Ltd
Priority to CN202210610620.1A priority Critical patent/CN115185453A/en
Publication of CN115185453A publication Critical patent/CN115185453A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022—Mechanisms to release resources

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present disclosure relates to a data processing method, an apparatus, an electronic device, and a storage medium, including: the method comprises the steps of obtaining a data reading instruction, wherein the data reading instruction comprises original operation parameters, determining storage position information of target data from a target storage position group based on data corresponding to parameter positions and the original operation parameters, determining a target file based on the storage position information, and reading the target data from the target file, wherein the target storage position group is one of a plurality of storage position groups which are isolated from each other in operation. According to the embodiment of the application, the storage position information is stored by operating the target storage position group in the plurality of storage position groups which are isolated from each other, so that the influence of other data processing is avoided, and the process of data processing is accelerated.

Description

Data processing method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.
Background
With the rapid development of the current mobile internet, a huge amount of data needs to be read, stored, updated, deleted, backed up, integrated, etc. continuously.
Taking data reading as an example, since a user wants to read data as quickly as possible, in some existing technologies, a storage location of the data is usually obtained first, and then the data is read from an area corresponding to the storage location. However, during the reading process of the storage location, the current read operation is often affected by other processing operations, so that the multi-thread processing becomes a single-thread processing, and the data processing process becomes slow.
Disclosure of Invention
The present disclosure provides a data processing method, an apparatus, an electronic device and a storage medium, and the technical scheme of the present disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a data processing method, including:
acquiring a data reading instruction; the data read instruction includes original operating parameters;
determining storage position information of the target data from the target storage position group based on the data corresponding to the parameter positions and the original operation parameters; the target storage location group is one of a plurality of storage location groups which are mutually isolated in operation;
determining a target file based on the storage location information;
target data is read from the target file.
In some possible embodiments, determining storage location information of the target data from the target storage location group based on the data corresponding to the parameter location and the original operating parameter includes:
determining a parameter simplification mode corresponding to the data reading instruction;
processing the original operation parameters based on a parameter simplification mode to obtain target operation parameters;
and reading the storage position information of the target data from the target storage position group based on the data corresponding to the parameter position and the target operation parameter.
In some possible embodiments, reading storage location information of the target data from the target storage location group based on the data corresponding to the parameter location and the target operating parameter includes:
determining a sequence of target storage locations in the set of target storage locations based on the target operating parameters; the target set of storage locations comprises at least one sequence of storage locations; the target sequence of storage locations is one of the at least one sequence of storage locations;
locking the target storage location sequence;
reading storage position information of target data in the target storage position sequence based on the data corresponding to the parameter positions;
unlocking the target sequence of storage locations.
In some possible embodiments, the method further comprises:
acquiring a data writing instruction; the data writing instruction comprises original operating parameters and target data;
allocating a target file for storing data for the target data, and storing the target data and the original operating parameters;
determining storage position information corresponding to a target file;
determining a target set of storage locations from the plurality of sets of storage locations based on the original operating parameters; the operations between the plurality of storage location groups are isolated from each other;
and storing the original operating parameters and the storage position information in a target storage position group to obtain data corresponding to the parameter positions.
In some possible embodiments, determining a target storage location group from a plurality of storage location groups based on the original operating parameters, storing the original operating parameters and the storage location information in the target storage location group, and obtaining data corresponding to the parameter locations includes:
acquiring a parameter simplification instruction;
determining a parameter simplification mode based on the parameter simplification instruction;
processing the original operation parameters based on a parameter simplification mode to obtain target operation parameters;
determining a target set of storage locations from the plurality of sets of storage locations based on the target operating parameter;
and storing the target operating parameters and the storage position information in a target storage position group to obtain data corresponding to the parameter positions.
In some possible embodiments, processing the original operating parameters based on a parameter reduction scheme to obtain target operating parameters, and determining a target storage location group from the plurality of storage location groups based on the target operating parameters includes:
processing the original operation parameters based on a parameter simplification mode to obtain transition operation parameters;
determining a target set of storage locations from the plurality of sets of storage locations based on the transient operation parameters;
determining preset parameters corresponding to the target storage position group;
and carrying out differentiation processing on the transitional operation parameters and the preset parameters to obtain target operation parameters.
In some possible embodiments, determining a target storage location group from the plurality of storage location groups based on the transient operational parameters, storing the target operational parameters and the storage location information in the target storage location group, and obtaining data corresponding to the parameter locations comprises:
determining a sequence of target storage locations from at least one sequence of storage locations in the set of target storage locations based on the transient operational parameter;
locking the target storage location group;
storing the target operation parameters and the storage position information in a target storage position sequence to obtain data corresponding to the parameter positions;
unlocking the target storage location group.
In some possible embodiments, the method further comprises:
acquiring a data integration instruction; the data integration instruction comprises a file to be integrated;
traversing the file to be integrated to obtain data corresponding to a plurality of parameter positions;
determining data corresponding to effective parameter positions in the data corresponding to the parameter positions from the plurality of storage position groups;
acquiring data corresponding to the effective parameter position from the file to be integrated;
distributing an integrated file for data corresponding to the effective parameter position;
and storing the data corresponding to the effective parameter positions and the original operating parameters in the data corresponding to the effective parameter positions in the integrated file.
In some possible embodiments, after storing the data corresponding to the valid parameter position and the original operating parameter in the data corresponding to the valid parameter position in the integrated file, the method further includes:
determining storage position information of the integrated file;
determining a consolidated set of storage locations from the plurality of sets of storage locations based on the original operating parameters in the data corresponding to the valid parameter locations;
and storing the original operating parameters and the storage position information of the integrated file in an integrated storage position group to obtain data corresponding to the integrated parameter position.
In some possible embodiments, traversing the file to be integrated to obtain data corresponding to a plurality of parameter positions includes:
determining the read demand resource and/or the write demand resource of the current time;
and traversing the file to be integrated to obtain data corresponding to a plurality of parameter positions if the read demand resource and/or the write demand resource is less than or equal to the resource threshold.
According to a second aspect of an embodiment of the present disclosure, there is provided a data processing apparatus including:
an instruction fetch module configured to execute a fetch data read instruction; the data read instruction includes original operating parameters;
a position determination module configured to perform determining storage position information of the target data from the target storage position group based on the data corresponding to the parameter position and the original operation parameter; the target storage location group is one of a plurality of storage location groups which are mutually isolated in operation;
a file determination module configured to perform determining a target file based on the storage location information;
a reading module configured to perform reading target data from a target file.
In some possible embodiments, the position determination module is configured to perform:
determining a parameter simplification mode corresponding to the data reading instruction;
processing the original operation parameters based on a parameter simplification mode to obtain target operation parameters;
and reading the storage position information of the target data from the target storage position group based on the data corresponding to the parameter position and the target operation parameter.
In some possible embodiments, the position determination module is configured to perform:
determining a sequence of target storage locations in the set of target storage locations based on the target operating parameters; the target set of storage locations comprises at least one sequence of storage locations; the target sequence of storage locations is one of the at least one sequence of storage locations;
locking the target storage location sequence;
reading storage position information of the target data in the target storage position sequence based on the data corresponding to the parameter positions;
unlocking the target sequence of storage locations.
In some possible embodiments, the apparatus further comprises:
an instruction fetch module configured to execute a fetch data write instruction; the data writing instruction comprises original operation parameters and target data;
the writing module is configured to execute allocation of a target file of storage data for the target data, and store the target data and the original operating parameters;
the position determining module is configured to determine storage position information corresponding to the target file; determining a target set of storage locations from the plurality of sets of storage locations based on the original operating parameters; the operations between the plurality of storage location groups are isolated from each other;
and the writing module is configured to store the original operation parameters and the storage position information in the target storage position group to obtain data corresponding to the parameter positions.
In some possible embodiments, the location determination module is configured to execute the get parameter reduction instruction; determining a parameter simplification mode based on the parameter simplification instruction; processing the original operation parameters based on a parameter simplification mode to obtain target operation parameters; determining a target storage location group from the plurality of storage location groups based on the target operating parameter;
and the writing module is configured to store the target operating parameters and the storage location information in the target storage location group to obtain data corresponding to the parameter locations.
In some possible embodiments, the position determination module is configured to perform:
processing the original operation parameters based on a parameter simplification mode to obtain transition operation parameters;
determining a target storage location group from the plurality of storage location groups based on the transient operational parameter;
determining preset parameters corresponding to the target storage position group;
and carrying out differentiation processing on the transitional operation parameters and the preset parameters to obtain target operation parameters.
In some possible embodiments, the apparatus further comprises:
a location determination module configured to perform determining a target sequence of storage locations from at least one sequence of storage locations in the target set of storage locations based on the transient operational parameters;
a lock module configured to perform locking a target set of storage locations;
the writing module is configured to store the target operation parameters and the storage position information in a target storage position sequence to obtain data corresponding to the parameter positions;
a lock module configured to perform unlocking the target storage location group.
In some possible embodiments, the apparatus further comprises:
the instruction acquisition module is configured to execute the acquired data integration instruction; the data integration instruction comprises a file to be integrated;
the data traversing module is configured to execute traversing of the file to be integrated to obtain data corresponding to a plurality of parameter positions;
the data determining module is configured to determine data corresponding to effective parameter positions in the data corresponding to the parameter positions from a plurality of storage position groups; acquiring data corresponding to the effective parameter position from the file to be integrated;
the file distribution module is configured to distribute integrated files for data corresponding to the effective parameter positions;
and the writing module is configured to execute the step of storing the data corresponding to the effective parameter position and the original operation parameters in the data corresponding to the effective parameter position in the integrated file.
In some of the possible embodiments of the present invention,
a location determination module configured to perform determining storage location information of the integrated file; determining a consolidated set of storage locations from the plurality of sets of storage locations based on the original operating parameters in the data corresponding to the valid parameter locations;
and the writing module is configured to store the original operating parameters and the storage position information of the integrated file in the integrated storage position group to obtain data corresponding to the integrated parameter position.
In some possible embodiments, the data traversal module is configured to perform:
determining the read demand resource and/or the write demand resource of the current time;
and traversing the file to be integrated to obtain data corresponding to a plurality of parameter positions if the read demand resource and/or the write demand resource is less than or equal to the resource threshold.
According to a third aspect of an embodiment of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method of any one of the first aspects as described above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of the first aspects of the embodiments of the present disclosure.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising a computer program, the computer program being stored in a readable storage medium, from which at least one processor of a computer device reads and executes the computer program, causing the computer device to perform the method of any one of the first aspects of embodiments of the present disclosure.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the method comprises the steps of obtaining a data reading instruction, wherein the data reading instruction comprises original operation parameters, determining storage position information of target data from a target storage position group based on data corresponding to parameter positions and the original operation parameters, determining a target file based on the storage position information, and reading the target data from the target file, wherein the target storage position group is one of a plurality of storage position groups which are isolated from each other in operation. According to the embodiment of the application, the storage position information is stored by operating the target storage position group in the plurality of storage position groups which are isolated from each other, so that the influence of other data processing is avoided, and the process of data processing is accelerated.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a schematic diagram illustrating an application environment in accordance with an illustrative embodiment;
FIG. 2 is a flow diagram illustrating a method of data processing in accordance with an exemplary embodiment;
FIG. 3 is a flow chart illustrating a method of writing data in accordance with an exemplary embodiment;
FIG. 4 is a flow chart illustrating a method of writing data in accordance with an exemplary embodiment;
FIG. 5 is a flow chart illustrating a method of writing data in accordance with an exemplary embodiment;
FIG. 6 is a flow chart illustrating a method of data merging according to an exemplary embodiment;
FIG. 7 is a block diagram illustrating a data process in accordance with an exemplary embodiment;
FIG. 8 is a block diagram illustrating a data processing apparatus in accordance with an exemplary embodiment;
FIG. 9 is a block diagram illustrating an electronic device for data processing in accordance with an exemplary embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for presentation, analyzed data, etc.) referred to in the present disclosure are information and data authorized by the user or sufficiently authorized by each party.
Referring to fig. 1, fig. 1 is a schematic diagram illustrating an application environment of a data processing method according to an exemplary embodiment, and as shown in fig. 1, the application environment may include a data transmission device 01 and a data processing device 02.
In this embodiment, the data processing device 02 may obtain a data reading instruction, where the data reading instruction includes an original operation parameter, and determine storage location information of target data from a target storage location group based on data corresponding to a parameter location and the original operation parameter, where the target storage location group is one of a plurality of storage location groups isolated from each other in operation, determine a target file based on the storage location information, and read the target data from the target file.
Optionally, the data processing device 02 may include, but is not limited to, a smartphone, a desktop computer, a tablet computer, a laptop computer, a smart speaker, a digital assistant, an Augmented Reality (AR)/Virtual Reality (VR) device, a smart wearable device, and the like. Software running on the device, such as applications, applets, etc., is also possible. Optionally, the operating system running on the device may include, but is not limited to, an android system, an IOS system, linux, windows, unix, and the like. The data processing device 02 may include not only an independent physical server, but also a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like. The operating system running on the server may include, but is not limited to, an android system, an IOS system, linux, windows, unix, and the like.
Optionally, the data transmission device 01 may include an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), and a big data and artificial intelligence platform. The operating system running on the server may include, but is not limited to, an android system, an IOS system, linux, windows, unix, and the like. Data transmission device 01 may include, but is not limited to, smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, augmented Reality (AR)/Virtual Reality (VR) devices, smart wearable devices, and the like. Software running on the device, such as applications, applets, etc., is also possible. Optionally, the operating system running on the device may include, but is not limited to, an android system, an IOS system, linux, windows, unix, and the like.
The data transmission device 01 and the data processing device 02 may be connected by a wireless link or a wired link.
In addition, it should be noted that fig. 1 shows only one application environment of the data processing method provided by the present disclosure, and in practical applications, other application environments may also be included.
Fig. 2 is a flowchart illustrating a data processing method according to an exemplary embodiment, where as shown in fig. 2, the data processing method may be applied to a server or a client, and includes the following steps:
in step S201, a data read instruction is acquired; the data read instruction includes original operating parameters.
In the embodiment of the application, the device may obtain the data reading instruction, and obtain the target data from the target file of the stored data based on the original operation parameter included in the data reading instruction. Optionally, the target data may also need to be written before the device can provide the function of reading data.
An embodiment of the present application further provides a data writing method, and fig. 3 is a flowchart illustrating a data writing method according to an exemplary embodiment, and as shown in fig. 3, the method includes:
in step S301, a data write instruction is acquired; the data write instruction includes original operating parameters and target data.
In the embodiment of the application, the device can receive the data writing instruction sent by the data transmission device, and analyze the data writing instruction to obtain the original operation parameters and the target data.
Alternatively, the target data may be any form of data, such as text data, pictures, tables. The original operating parameter may be index data used to locate the target data in the data storage area.
Optionally, if the target data may be text data, a picture, or a table, the original operation parameter may be identification data of the text data, the picture, or the table. For example, the original operation data may be the name of a picture, such as the serial number "2021-XXXXYUyt", and the original operation data may be the name of a table, such as the text "XXX table".
In step S303, a target file for storing data is allocated for the target data, and the target data and the original operation parameters are stored.
In an alternative embodiment, if the device is a database-type server, the device may allocate a target file for storing data in the database for the target data, and store the target data and the original operating parameters in the target file. Alternatively, the device may allocate an area in the target file for storing data in the database for the target data and store the target data and the original operating parameters in the area.
In another alternative embodiment, if the device is a client of data storage and the data is stored in a normal disk or a solid-state disk, the device may allocate a target file for storing the data to the target data in the solid-state disk, and store the target data and the original operating parameters in the target file.
In step S305, storage location information corresponding to the target file is determined.
In this embodiment of the application, after the device stores the target data and the original operating parameters in the target file, the storage location information corresponding to the target file may also be obtained, where the storage location information may be a name of the target file or an identifier of a storage area in the target file. Alternatively, the storage location information may also be referred to as path information leading to the target file or a storage area in the target file. The storage location information is used to enable the device to determine the storage location of the target data in the database, the ordinary disk or the solid-state disk based on the storage location information directly, so that the target data can be read from the storage location directly.
In step S307, a target storage location group is determined from the plurality of storage location groups based on the original operating parameters; operations between multiple groups of storage locations are isolated from each other.
In an embodiment of the present application, a device may determine a target storage location group from a plurality of storage location groups based on original operating parameters.
Optionally, the device may perform a hash operation on the original operation parameter, and determine a target storage location group corresponding to the original operation parameter, where the target storage location group is used to store the original operation parameter and storage location information corresponding to the target file, so that when a subsequent user reads target data, the storage location information may be obtained from the target storage location group.
Optionally, the target storage location group may be located in a local memory of the device, and the target storage location group may be one of a plurality of storage location groups constructed in the local memory. Optionally, the operations of the storage location groups in the plurality of storage location groups are isolated from each other, so that different storage location groups do not interfere with each other when the original operation parameters and the storage location information are written in, and thus, the multi-thread write operation is realized.
In step S309, the original operation parameters and the storage location information are stored in the target storage location group, and data corresponding to the parameter locations are obtained.
In this embodiment of the application, the device may store the original operating parameter and the storage location information in the target storage location group, to obtain data corresponding to the parameter location of the target data.
Optionally, the device may perform data splicing on the original operation parameter and the storage location information to obtain data stored in the target storage location group and corresponding to the parameter location of the target data.
Optionally, the device may perform data splicing on the original operation parameters and the storage location information, and perform data conversion on the spliced data according to a preset data conversion rule to obtain data corresponding to the parameter location of the target data, which is stored in the target storage location group.
The data corresponding to the parameter location is used to determine storage location information from the data corresponding to the parameter location based on the original operating parameter after the original operating parameter is obtained. Or, the data corresponding to the parameter position is used for determining the storage position information from the data obtained by inverse conversion of the data corresponding to the parameter position based on the original operation parameter after the original operation parameter is obtained.
In some possible embodiments, a raw operation parameter may generally correspond to a specific target data, however, in an actual application process, such as a graph storage application scenario, a batch of target data, such as 20 graphs, may be obtained as desired. In the 20 graphs, the original operating parameters corresponding to each graph are different, but the original operating parameters all include the same parameters, for example, all include the same attribute parameter, so when the 20 graphs are obtained, a part of parameters including the attribute parameters can be selected from the original operating parameters, and the part of parameters can be read as a basis to obtain the 20 graphs.
Based on this, an embodiment of the present application further provides a data writing method, and fig. 4 is a flowchart illustrating a data writing method according to an exemplary embodiment, which replaces steps S307 and S309 in fig. 3, as shown in fig. 4, including:
in step S401, a parameter reduction instruction is acquired.
In the embodiment of the application, the original operation parameters can be obtained by analyzing the data writing instruction, and meanwhile, the parameter simplifying instruction can be obtained. Alternatively, the parameter reduction instruction may be included in the data write instruction, i.e., the parameter reduction instruction may be user-defined. Optionally, the parameter reduction instruction may be automatically generated by the device based on historical experience.
In step S403, a parameter reduction manner is determined based on the parameter reduction instruction.
Optionally, the device may determine the parameter reduction mode based on the parameter reduction instruction. For example, the device may parse the parameter reduction instruction to obtain the parameter reduction manner, for example, may obtain a parameter reduction description language "obtain a first X-bit character in the original operation parameter," where "X" is a settable numerical value. Alternatively, an equation such as "X =3" may also be derived, which schematically acquires the first 3-bit character in the original operating parameters. An equation like "X =3,y =4" can also be derived, which schematically obtains the first 3-bit character and the last four-bit character in the original operating parameters.
In step S405, the original operation parameters are processed based on the parameter simplification method to obtain the target operation parameters.
Optionally, the device may perform truncation processing on the original operation parameter based on a parameter simplification mode, and obtain a part of parameters to obtain the target operation parameter. The partial parameter may be a prefix parameter or a suffix parameter in the original operation parameters.
In step S407, a target storage location group is determined from the plurality of storage location groups based on the target operating parameter.
Optionally, the device may perform a hash operation on the target operation parameter, determine a target storage location group corresponding to the target operation parameter from the multiple storage location groups, where the target storage location group is used to store the target operation parameter and storage location information corresponding to the target file, so that when a subsequent user reads the target data, the storage location information may be obtained from the target storage location group.
Optionally, the target storage location group may be located in a local memory of the device, and the target storage location group may be one of a plurality of storage location groups constructed in the local memory. Optionally, the operations of the storage location groups in the plurality of storage location groups are isolated from each other, so that different storage location groups do not interfere with each other when the original operation parameters and the storage location information are written in, and thus, the multi-thread write operation is realized.
In step S409, the target operating parameters and the storage location information are stored in the target storage location group, resulting in data corresponding to the parameter locations.
In this embodiment of the application, the device may store the target operating parameter and the storage location information in the target storage location group, and obtain data corresponding to the parameter location of the target data.
Therefore, the hash operation is performed on the target operation parameters obtained by processing the original operation parameters, and the target storage position group is obtained, so that the memory occupation during operation can be effectively reduced, the subsequent reading or scanning performance is improved, and the reading or scanning efficiency is improved.
In the above embodiment, the device performs the hash operation on the target operation parameter, and determines the target storage location group corresponding to the target operation parameter, which may be specifically represented by that the device performs the hash operation on the target operation parameter to obtain a hash value of the target operation parameter, and then determines the target storage location group corresponding to the hash value based on a preset algorithm. Therefore, the subsequent equipment can store the hash value and the storage position information of the target operation parameter in the target storage position group to obtain data corresponding to the parameter position. However, the storage of data corresponding to the parameter location is also very memory consuming based on the amount of data stored, and in view of memory, the present application provides a way to reduce memory usage.
Fig. 5 is a flowchart illustrating a data writing method according to an exemplary embodiment, as shown in fig. 5, including:
in step S501, the original operation parameters are processed based on the parameter simplification method to obtain transient operation parameters.
Optionally, the device may determine the parameter simplification mode based on the parameter simplification instruction, for example, the device may analyze the parameter simplification mode from the parameter simplification instruction. For example, a simplified description language "obtain the first X-bit character" of the original operation parameter "may be obtained, where" X "is a settable value. Alternatively, an equation such as "X =3" may also be derived, which schematically obtains the first 3-bit character in the original operating parameters. An equation like "X =3,y =4" can also be derived, which schematically obtains the first 3-bit character and the last 4-bit character in the original operating parameters.
Optionally, the device may perform truncation processing on the original operation parameter based on a parameter simplification manner, and obtain a partial parameter to obtain a transition operation parameter, where the partial parameter may be a prefix parameter or a suffix parameter in the original operation parameter.
In step S503, a target storage location group is determined from the plurality of storage location groups based on the transient operation parameters.
Optionally, the device may perform hash operation on the transient operation parameter to obtain a hash value of the transient operation parameter, and then determine, based on a preset algorithm, a target storage location group corresponding to the hash value.
In step S505, preset parameters corresponding to the target storage location group are determined.
In the embodiment of the application, the device may read the preset parameter corresponding to the target storage location group.
Alternatively, the preset parameters may be customized for each storage location group.
Alternatively, the predetermined parameter of the set of storage locations may be the transitional operating parameter first stored in the set of storage locations. For example, assuming that the first transient operation parameter is "0000000", the transient operation parameter "0000000" is taken as the preset parameter.
In step S507, the transitional operation parameters and the preset parameters are differentiated to obtain target operation parameters.
In the embodiment of the application, the device can perform differentiation processing on the transitional operation parameters and the preset parameters to obtain the target operation parameters. Continuing with the above embodiment, the target operating parameter corresponding to the first transitional operating parameter is "-", and assuming that the other transitional operating parameter is "0000022", the corresponding target operating parameter is "22" which is the difference from the preset parameter "0000000".
Therefore, the storage of certain operation parameters is reduced from the original storage of 7 characters, such as 7 characters, to 1 or 2 characters, the memory space is saved, and the availability of the storage space is improved.
In an alternative embodiment, as the data amount increases, each of the target storage location groups may include one or more storage location sequences, and the data corresponding to the parameter location may be stored in a certain storage location sequence in a certain storage location group. Based on this, the device may determine a target storage location sequence from at least one storage location sequence in the target storage location group based on the transient/original operation parameters, store the target operation parameters (or the original operation parameters, or the transient operation parameters) and the storage location information in the target storage location sequence, and obtain data corresponding to the parameter locations.
During the storage process, in order to protect the write operation in the storage location group from being affected by other operations (such as update operation, integration operation), the device may lock the target storage location group before storing the target operation parameter (or the original operation parameter, or the transition operation parameter) and the storage location information in the target storage location sequence and obtaining the data corresponding to the parameter location. And after the target operating parameters and the storage location information are stored in the target storage location sequence and data corresponding to the parameter locations are obtained, the target storage location group is unlocked. Therefore, the write operation between different storage position groups can be synchronously carried out, and the write operation between one storage position group can be prevented from being influenced.
In conclusion, the present application completes the write process of the target data.
In step S203, determining storage location information of the target data from the target storage location group based on the data corresponding to the parameter location and the original operation parameter; the target storage location group is one of a plurality of storage location groups that are operationally isolated from each other.
According to the method and the device, after the data reading instruction is obtained, the original operation parameters can be obtained through analysis.
In an alternative embodiment, if the data corresponding to the parameter location is the original operating parameter during the writing process, the device may determine the target storage location group from the plurality of storage location groups by using the original operating parameter, and determine the storage location information corresponding to the original operating parameter based on the data corresponding to the parameter location.
In another alternative embodiment, if during the writing process, the data corresponding to the parameter position is the target operating parameter. The device may determine a parameter simplification manner corresponding to the data read instruction based on the rule, and process the original operation parameter based on the parameter simplification manner to obtain the target operation parameter. Subsequently, the device may read storage location information of the target data from the target storage location group based on the data corresponding to the parameter location and the target operating parameter.
In another optional embodiment, if in the writing process, the data corresponding to the parameter position is the target operating parameter, and the target operating parameter is processed by the preset parameter. The device may determine a parameter simplification manner corresponding to the data read instruction based on the rule, process the original operation parameter based on the parameter simplification manner to obtain a transition operation parameter, determine a target storage location group from the plurality of storage location groups based on the transition operation parameter, obtain a preset parameter of the target storage location group, and determine the target operation parameter by a difference between the transition operation parameter and the preset parameter. Subsequently, the device may read storage location information of the target data from the target storage location group based on the data corresponding to the parameter location and the target operating parameter.
In an alternative embodiment, as the data amount increases, each of the target storage location groups may include one or more storage location sequences, and the data corresponding to the parameter location may be stored in a certain storage location sequence in a certain storage location group. Therefore, the device can determine a target storage location sequence in the target storage location group based on the target operation parameter, and read storage location information of target data in the target storage location sequence based on data corresponding to the parameter location.
In order to protect the read operation in the storage location sequence from other operations (such as update operation and integration operation) during the reading process, the device may lock the target storage location sequence before reading the storage location information of the target data in the target storage location sequence based on the data corresponding to the parameter location, and unlock the target storage location sequence after reading the storage location information of the target data in the target storage location sequence based on the data corresponding to the parameter location. Therefore, the reading operation between different storage position groups and different storage position sequences can be synchronously carried out, and the writing sequence operation between one storage position sequence can not be influenced.
In step S205, the target file is determined based on the storage location information.
In step S207, target data is read from the target file.
As already mentioned above, the storage location information may be path information to the target file or a storage area in the target file. Therefore, the device can determine the target file by storing the position information, and can read the target data from the target file.
If the target data is read before, the target data can be stored in the local memory, and when the target data is read again, the target data can be directly read from the local memory without acquiring the target data from the target file based on the determined storage position information.
In conclusion, the present application completes the read process of the target data.
In the embodiment of the application, a data deleting process can be further included. Taking the original operating parameters as an example, in an alternative embodiment, the device may obtain a data deletion instruction, which may include the original operating parameters. The device may determine a sequence of target storage locations in the set of target storage locations based on the original operating parameters, and determine storage location information for the target data from the sequence of target storage locations based on the data corresponding to the parameter locations and the original operating parameters. Then, the device may determine a target file from the storage location information and delete target data from the target file.
Optionally, after the target data is deleted, the data corresponding to the parameter position of the target data in the target storage position sequence may also be deleted, and the memory in the target storage position sequence is released, so that new data may be stored.
Although the target data is deleted in this way, if the delete command is too concentrated, a large amount of delete work is generated, and other read-write operations are affected.
Fig. 6 is a flow chart illustrating a data merging method according to an example embodiment, as shown in fig. 6, including:
in step S601, a data integration instruction is acquired; the data integration instruction comprises files to be integrated.
Taking the original operating parameters as an example, in an alternative embodiment, the device may obtain a data deletion instruction, which may include the original operating parameters. The device may determine a sequence of target storage locations in the set of target storage locations based on the original operating parameters, and determine data corresponding to parameter locations corresponding to the original operating data from the sequence of target storage locations according to the original operating parameters. At this time, the device does not delete the target data from the target file found by storing the location information, but integrates the data uniformly in a period of time.
In the integration process, the device background of the embodiment of the present application may generate a data integration instruction, where the data integration instruction may include one file to be integrated or multiple files to be integrated.
In step S602, the file to be integrated is traversed to obtain data corresponding to a plurality of parameter positions.
The file to be integrated can be traversed to obtain data corresponding to the parameter position corresponding to the stored data, wherein the data comprises the original operation parameters (transition operation parameters or target operation parameters) and the storage position information of each piece of stored data.
In step S603, data corresponding to the valid parameter position among the data corresponding to the plurality of parameter positions is determined from the plurality of storage position groups.
Alternatively, the device may determine the storage location group or the storage location sequence for storing the data corresponding to the parameter location based on the original operating parameter (the transient operating parameter or the target operating parameter) in the data corresponding to the parameter location corresponding to the stored data. And determining whether the data corresponding to the parameter positions corresponding to the stored data obtained by traversing still exist in the storage position group or the storage position sequence, if so, that the data corresponding to the plurality of parameter positions are all valid, that is, the stored data are not deleted by the user. If the data does not exist, namely the data corresponding to the parameter positions are invalid, the device receives a corresponding deleting instruction and only waits for the data to be deleted from the file for storing the data subsequently. If the part exists, the part does not exist, namely the data part corresponding to the plurality of parameter positions is valid, and the part is invalid, namely the device receives a deletion instruction corresponding to the part of data.
Based on this, the device may determine data corresponding to the valid parameter location from among the data corresponding to the plurality of parameter locations from the plurality of storage location groups.
In step S604, data corresponding to the valid parameter position is obtained from the file to be integrated.
In step S605, an integrated file is allocated to the data corresponding to the valid parameter position.
In step S606, the data corresponding to the valid parameter position and the original operation parameters in the data corresponding to the valid parameter position are stored in the integrated file.
Then, the device may obtain data corresponding to the valid parameter position from the file to be integrated, and copy the data that is not required to be deleted and the original operation data corresponding to the data into a new file, that is, an integrated file.
In step S607, the storage location information of the integrated file is determined.
In step S608, an integrated storage location group is determined from the plurality of storage location groups based on the original operating parameters in the data corresponding to the valid parameter locations.
In step S609, the original operation parameters and the storage location information of the integrated file are stored in the integrated storage location group, so as to obtain data corresponding to the integrated parameter location.
Subsequently, the device may determine storage location information of the integrated file, determine an integrated storage location group from the plurality of storage location groups based on the original operation parameter in the data corresponding to the valid parameter location, and store the original operation parameter and the storage location information of the integrated file in the integrated storage location group to obtain data corresponding to the integrated parameter location.
That is to say, in the embodiment of the present application, there is a time interval from when a deletion instruction of certain data is received to when the data is deleted from a file in a database, a normal disk, or a solid-state disk. This is because it is considered that, if data is deleted in a file in a database, a normal disk, or a solid-state disk based on a deletion instruction every time a deletion instruction of data is issued, it is necessary to make the device free more resources to perform the deletion operation, which affects other read/write operations. Based on the method, the device can integrate the effective data of the file to be integrated based on the generated integration thread after accumulating deletion instructions of more data through a time interval, wherein the integration of the effective data comprises the deletion of the deleted data and the integration of the rest data, so that the workload of the device can be reduced.
Alternatively, the device may set the integration process to a time period during which the reading operation is less, such as midnight.
Optionally, after receiving the data integration instruction, the device may determine the read demand resource and/or the write demand resource at the current time, and if the read demand resource and/or the write demand resource is less than or equal to a resource threshold, traverse the file to be integrated to obtain data corresponding to the multiple parameter positions. Therefore, the influence of the integration task generated by the device layer on the read-write operation of the user can be reduced.
Fig. 7 is a block diagram illustrating a data processing system according to an exemplary embodiment, as shown in fig. 7, including an upper layer application and an operating system, wherein the upper layer application may be configured as a user side, and the operating system side includes a local memory and a solid-state disk. The device can acquire various instructions sent by the upper layer application, including a data reading instruction, a data writing instruction, a data updating instruction, a data deleting instruction and a data backup instruction. The data reading instruction may include an instruction to read a batch and an instruction to read one, and the data writing instruction may include an instruction to write a batch and an instruction to write one.
The storage location group may be located in a local memory of the device, and the local memory may include a plurality of storage location groups, each storage location group being expandable to a plurality of storage location sequences based on a data volume requirement. Each storage position group can have its own data lock, and each storage position sequence can also be provided with its own data lock for locking in a write operation or a read operation without being affected by other operations. Because the storage position groups are mutually isolated in the local memory, the multithread operation can be realized, and the high bandwidth performance of the solid-state disk is fully utilized.
The file storage area is a disk area including a normal disk and a solid-state disk. The disk area may include a data manager, a file read engine, a file write engine, other hardware engines, and various files. Among other things, a data manager may be used to manage data and files. The file reading engine may read target data in the target file corresponding to the location storage information by using the file reading engine after receiving the data reading operation. The file write engine may be after receiving the data read operation. And writing the target data into the target file corresponding to the storage information of the position by using a file writing engine.
Optionally, the other hardware engine may be an engine that improves performance of operations such as reading and writing, for example, in a write operation, the engine that can improve a write speed may be called, and target data is quickly written into a target file.
Optionally, the data storage files in the various files may include files storing different data structures, and the record file is used for recording update and deletion records of all the files, so that data can be restored to a consistent state when recovery due to a failure is performed subsequently. The last recorded file pointing file is used to indicate which of the last recorded files is.
FIG. 8 is a block diagram illustrating a data processing device according to an example embodiment. Referring to fig. 8, the apparatus includes:
an instruction fetch module 801 configured to execute a fetch data read instruction; the data read instruction includes original operating parameters;
a location determination module 802 configured to perform determining storage location information of target data from a target storage location group based on data corresponding to the parameter locations and the original operating parameters; the target storage location group is one storage location group in a plurality of storage location groups which are mutually isolated in operation;
a file determination module 803 configured to perform determining a target file based on the storage location information;
a read module 804 configured to perform reading target data from a target file.
In some possible embodiments, the position determination module is configured to perform:
determining a parameter simplification mode corresponding to the data reading instruction;
processing the original operation parameters based on a parameter simplification mode to obtain target operation parameters;
and reading the storage position information of the target data from the target storage position group based on the data corresponding to the parameter position and the target operation parameter.
In some possible embodiments, the position determination module is configured to perform:
determining a sequence of target storage locations in the set of target storage locations based on the target operating parameters; the target set of storage locations comprises at least one sequence of storage locations; the target sequence of storage locations is one of the at least one sequence of storage locations;
locking the target storage location sequence;
reading storage position information of the target data in the target storage position sequence based on the data corresponding to the parameter positions;
unlocking the target storage location sequence.
In some possible embodiments, the apparatus further comprises:
an instruction fetch module configured to execute a fetch data write instruction; the data writing instruction comprises original operation parameters and target data;
the writing module is configured to execute the allocation of a target file of the storage data to the target data and store the target data and the original operation parameters;
the position determining module is configured to determine storage position information corresponding to the target file; determining a target set of storage locations from the plurality of sets of storage locations based on the original operating parameters; the operations between the plurality of storage location groups are isolated from each other;
and the writing module is configured to store the original operation parameters and the storage position information in the target storage position group to obtain data corresponding to the parameter positions.
In some possible embodiments, the location determination module is configured to execute the get parameter reduction instruction; determining a parameter simplification mode based on the parameter simplification instruction; processing the original operation parameters based on a parameter simplification mode to obtain target operation parameters; determining a target storage location group from the plurality of storage location groups based on the target operating parameter;
and the writing module is configured to store the target operating parameters and the storage location information in the target storage location group to obtain data corresponding to the parameter locations.
In some possible embodiments, the position determination module is configured to perform:
processing the original operation parameters based on a parameter simplification mode to obtain transition operation parameters;
determining a target storage location group from the plurality of storage location groups based on the transient operational parameter;
determining preset parameters corresponding to the target storage position group;
and carrying out differentiation processing on the transitional operation parameters and the preset parameters to obtain target operation parameters.
In some possible embodiments, the apparatus further comprises:
a location determination module configured to perform determining a target sequence of storage locations from at least one sequence of storage locations in the target set of storage locations based on the transient operational parameter;
a lock module configured to perform locking a target set of storage locations;
the writing module is configured to store the target operation parameters and the storage position information in a target storage position sequence to obtain data corresponding to the parameter positions;
a lock module configured to perform unlocking the target storage location group.
In some possible embodiments, the apparatus further comprises:
the instruction acquisition module is configured to execute an acquisition data integration instruction; the data integration instruction comprises files to be integrated;
the data traversing module is configured to execute traversing of the file to be integrated to obtain data corresponding to a plurality of parameter positions;
the data determining module is configured to determine data corresponding to effective parameter positions in the data corresponding to the parameter positions from a plurality of storage position groups; acquiring data corresponding to the effective parameter position from the file to be integrated;
the file distribution module is configured to distribute the integrated file for the data corresponding to the effective parameter position;
and the writing module is configured to execute the step of storing the data corresponding to the effective parameter position and the original operation parameters in the data corresponding to the effective parameter position in the integrated file.
In some of the possible embodiments of the present invention,
a location determination module configured to perform determining storage location information of the integrated file; determining a consolidated set of storage locations from the plurality of sets of storage locations based on the original operating parameters in the data corresponding to the valid parameter locations;
and the writing module is configured to store the original operating parameters and the storage position information of the integrated file in the integrated storage position group to obtain data corresponding to the integrated parameter position.
In some possible embodiments, the data traversal module is configured to perform:
determining the read demand resource and/or the write demand resource of the current time;
and traversing the file to be integrated to obtain data corresponding to a plurality of parameter positions if the read demand resource and/or the write demand resource is less than or equal to the resource threshold.
With regard to the apparatus in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be described in detail here.
Fig. 9 is a block diagram illustrating an apparatus 2000 for data processing in accordance with an example embodiment. For example, the apparatus 2000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 9, the apparatus 2000 may include one or more of the following components: a processing component 2002, a memory 2004, a power component 2006, a multimedia component 2008, an audio component 2010, an input/output (I/O) interface 2012, a sensor component 2014, and a communications component 2016.
The processing component 2002 generally controls the overall operation of the device 2000, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 2002 may include one or more processors 2020 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 2002 can include one or more modules that facilitate interaction between the processing component 2002 and other components. For example, the processing component 2002 may include a multimedia module to facilitate interaction between the multimedia component 2008 and the processing component 2002.
The memory 2004 is configured to store various types of data to support operation at the device 2000. Examples of such data include instructions for any application or method operating on device 2000, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 2004 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 2006 provides power to the various components of the device 2000. The power supply components 2006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 2000.
The multimedia component 2008 includes a screen providing an output interface between the device 2000 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 2008 includes a front camera and/or a rear camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 2000 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
Audio component 2010 is configured to output and/or input audio signals. For example, audio component 2010 includes a Microphone (MIC) configured to receive external audio signals when apparatus 2000 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 2004 or transmitted via the communication component 2016. In some embodiments, audio assembly 2010 also includes a speaker for outputting audio signals.
The I/O interface 2012 provides an interface between the processing component 2002 and peripheral interface modules, which can be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 2014 includes one or more sensors for providing various aspects of state assessment for the device 2000. For example, sensor assembly 2014 may detect an open/closed state of device 2000, a relative positioning of components, such as a display and keypad of apparatus 2000, a change in position of apparatus 2000 or a component of apparatus 2000, the presence or absence of user contact with apparatus 2000, an orientation or acceleration/deceleration of apparatus 2000, and a change in temperature of apparatus 2000. The sensor assembly 2014 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 2014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 2014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 2016 is configured to facilitate wired or wireless communication between the apparatus 2000 and other devices. The apparatus 2000 may access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 2016 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 2016 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 2000 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components for performing the above-described methods.
In an exemplary embodiment, a storage medium comprising instructions, such as the memory 2004 comprising instructions, executable by the processor 2020 of the apparatus 2000 to perform the above-described method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Claims (10)

1. A data processing method, comprising:
acquiring a data reading instruction; the data read instruction comprises original operating parameters;
determining storage position information of target data from a target storage position group based on data corresponding to the parameter positions and the original operation parameters; the target storage location group is one of a plurality of storage location groups which are mutually isolated in operation;
determining a target file based on the storage location information;
and reading the target data from the target file.
2. The data processing method of claim 1, wherein determining storage location information of target data from a target storage location group based on the data corresponding to the parameter location and the original operating parameter comprises:
determining a parameter simplification mode corresponding to the data reading instruction;
processing the original operation parameters based on the parameter simplification mode to obtain target operation parameters;
and reading the storage position information of the target data from a target storage position group based on the data corresponding to the parameter position and the target operation parameter.
3. The data processing method according to claim 2, wherein the reading of the storage location information of the target data from the target storage location group based on the data corresponding to the parameter location and the target operating parameter comprises:
determining a target sequence of storage locations in the target set of storage locations based on the target operating parameters; the set of target storage locations comprises at least one sequence of storage locations; the target sequence of storage locations is one of the at least one sequence of storage locations;
locking the target sequence of storage locations;
reading storage position information of the target data in the target storage position sequence based on the data corresponding to the parameter positions;
unlocking the target sequence of storage locations.
4. The data processing method of claim 1, wherein the method further comprises:
acquiring a data writing instruction; the data write instruction comprises the original operating parameters and the target data;
distributing the target file for storing data for the target data, and storing the target data and the original operating parameters;
determining storage position information corresponding to the target file;
determining the target set of storage locations from the plurality of sets of storage locations based on the original operating parameters; the operations between the plurality of storage location groups are isolated from each other;
and storing the original operating parameters and the storage position information in the target storage position group to obtain data corresponding to the parameter positions.
5. The data processing method of claim 4, wherein the determining the target storage location group from the plurality of storage location groups based on the original operating parameters, storing the original operating parameters and the storage location information in the target storage location group, and obtaining data corresponding to the parameter locations comprises:
acquiring a parameter simplification instruction;
determining the parameter simplification mode based on the parameter simplification instruction;
processing the original operation parameters based on the parameter simplification mode to obtain target operation parameters;
determining the target set of storage locations from the plurality of sets of storage locations based on the target operating parameter;
and storing the target operating parameters and the storage position information in the target storage position group to obtain data corresponding to the parameter positions.
6. The data processing method of claim 5, wherein the processing the original operating parameters based on the parameter reduction scheme to obtain target operating parameters, and the determining the target storage location group from the plurality of storage location groups based on the target operating parameters comprises:
processing the original operation parameters based on the parameter simplification mode to obtain transition operation parameters;
determining the target set of storage locations from the plurality of sets of storage locations based on the transient operational parameters;
determining preset parameters corresponding to the target storage position group;
and carrying out differentiation processing on the transitional operation parameters and the preset parameters to obtain the target operation parameters.
7. A data processing apparatus, comprising:
an instruction obtaining module configured to execute a obtain data read instruction; the data read instruction comprises original operating parameters;
a position determination module configured to perform determining storage position information of target data from a target storage position group based on data corresponding to parameter positions and the original operation parameters; the target storage location group is one of a plurality of storage location groups which are mutually isolated in operation;
a file determination module configured to perform determining a target file based on the storage location information;
a reading module configured to perform reading the target data from the target file.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the data processing method of any one of claims 1 to 6.
9. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of any of claims 1 to 6.
10. A computer program product, characterized in that the computer program product comprises a computer program, which is stored in a readable storage medium, from which at least one processor of a computer device reads and executes the computer program, causing the computer device to perform the data processing method according to any one of claims 1 to 6.
CN202210610620.1A 2022-05-31 2022-05-31 Data processing method and device, electronic equipment and storage medium Pending CN115185453A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210610620.1A CN115185453A (en) 2022-05-31 2022-05-31 Data processing method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210610620.1A CN115185453A (en) 2022-05-31 2022-05-31 Data processing method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115185453A true CN115185453A (en) 2022-10-14

Family

ID=83514245

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210610620.1A Pending CN115185453A (en) 2022-05-31 2022-05-31 Data processing method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115185453A (en)

Similar Documents

Publication Publication Date Title
CN112003893B (en) Resource downloading method and device
CN105094904A (en) Application program installation method and device
CN111258952B (en) Data storage control method, device and storage medium
US11494117B2 (en) Method and system for data processing
CN113094378A (en) Data processing method and device, electronic equipment and storage medium
CN114428589B (en) Data processing method and device, electronic equipment and storage medium
CN111857806A (en) Permission configuration file updating method and device, electronic equipment and storage medium
CN113596175A (en) Scene synchronization method, terminal, server and system
CN115794853B (en) Updating method and device of government affair data resource catalog, electronic equipment and medium
CN111382161A (en) State data processing method and device, electronic equipment and storage medium
CN109522286B (en) Processing method and device of file system
CN114528037B (en) Application starting method and device, electronic equipment and storage medium
CN115185453A (en) Data processing method and device, electronic equipment and storage medium
CN115016944A (en) Process access method and device and electronic equipment
CN114003558A (en) Data archiving method, device, equipment and storage medium
CN110995767B (en) Request processing method and device
CN108509641B (en) File backup method, device, server and system
CN114239531A (en) Template recommendation method and device, electronic equipment and storage medium
CN109032686B (en) Data processing method and device
CN107257384B (en) Service state monitoring method and device
CN112819061A (en) Password information identification method, device, equipment and storage medium
CN114385373A (en) Data sharing method, data sharing device and computer readable storage medium
CN108419119B (en) Cache data clearing method and device
CN111625329A (en) Task allocation method and device, electronic equipment, server and storage medium
CN113378022A (en) In-station search platform, search method and related device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination