CN117931910A - Data storage method, device, equipment and storage medium - Google Patents
Data storage method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117931910A CN117931910A CN202311835254.0A CN202311835254A CN117931910A CN 117931910 A CN117931910 A CN 117931910A CN 202311835254 A CN202311835254 A CN 202311835254A CN 117931910 A CN117931910 A CN 117931910A
- Authority
- CN
- China
- Prior art keywords
- data
- attribute
- type
- data object
- original
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000013499 data model Methods 0.000 claims abstract description 18
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 24
- 238000004891 communication Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application belongs to the field of data processing, and relates to a data storage method, which comprises the steps of obtaining original data from a preset data model; creating an original data object corresponding to the original data; extracting data attributes corresponding to the original data objects; judging whether the data attribute belongs to a preset attribute type or not; if the data belongs to the attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object, wherein the updated data object does not belong to the preset attribute type; if the data attribute does not belong to the attribute type, converting the data attribute and the update data attribute corresponding to the update data object into a target data type corresponding to the preset attribute type; and storing the target data object corresponding to the target data type into the target area. The application also provides a data storage device, computer equipment and a storage medium. The application can improve the efficiency and accuracy of data storage.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a data storage method, apparatus, device, and storage medium.
Background
The data storage means that the data is converted into a target format and stored in a designated position;
Traditional data storage is to manually define and create objects and attributes of XML format data and store the XML format data into a target database according to the objects and attributes.
However, the method needs to manually define and create the data objects and the data attributes, the manual definition and creation of the data objects and the attributes is complex and easy to make mistakes, and secondly, only simple iterative processing can be carried out on the target data before storage, so that the complex data processing requirements cannot be met, and the efficiency and the accuracy of data storage are low.
Disclosure of Invention
The invention provides a data storage method, a device, equipment and a storage medium, and mainly aims to improve the efficiency and accuracy of data storage.
In order to solve the above technical problems, an embodiment of the present application provides a data storage method, which adopts the following technical schemes:
Acquiring original data from a preset data model;
creating an original data object corresponding to the original data;
Extracting data attributes corresponding to the original data objects;
judging whether the data attribute belongs to a preset attribute type or not;
If the data attribute belongs to the preset attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object, wherein the updated data object does not belong to the preset attribute type;
If the data attribute does not belong to the preset attribute type, converting the data attribute and the updated data attribute corresponding to the updated data object into a target data type corresponding to the preset attribute type;
And storing the target data object corresponding to the target data type into a target area.
Further, the creating the original data object of the original data includes:
Acquiring a data identifier corresponding to the original data;
Identifying data attributes and data types corresponding to the data identifiers;
Abstracting the data attributes and the data types into a spatial data structure;
And creating an original data object corresponding to the data identifier according to the space data structure, the data attribute and the data type.
Further, the merging the original data object with the sub data object to obtain an updated data object includes:
identifying the association relationship between the original data object and the sub data object;
Generating an object structure of the original data object nested with the child data object according to the association relation;
and merging the original data object and the sub-original data object according to the object structure to obtain the updated data object.
Further, the converting the data attribute and the update data attribute corresponding to the update data object into the target data type corresponding to the preset attribute type includes:
acquiring metadata information of the data attribute and the updated data attribute, and identifying original data types corresponding to the data attribute and the updated data attribute according to the metadata information;
acquiring a preset type mapping table, and searching a target data type corresponding to the original data type in the type mapping table;
And converting the original data type into the target data type through a preset conversion function.
Further, the storing the target data object corresponding to the target data type in the target area includes:
carrying out data serialization operation on each data attribute and data attribute value corresponding to the target data object to obtain a JSON key value pair;
acquiring an object structure of the target data object, and performing data structuring operation on the JSON key value pair according to the object structure to obtain a JSON data structure;
And acquiring a target storage position of the JSON data structure, and storing the JSON data structure into the target area according to the target storage position.
Further, the determining whether the data attribute belongs to a preset attribute type includes:
Acquiring an attribute type corresponding to the data attribute;
Identifying whether the attribute type belongs to the preset attribute type;
When the data attribute belongs to the preset attribute type, taking the data attribute as a sub data object, combining the original data object with the sub data object, and obtaining an updated data object comprises the following specific steps:
When the data type belongs to the preset attribute type, determining that the data attribute belongs to the preset attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object;
when the data attribute does not belong to the preset attribute type, the specific step of converting the data attribute into a target data type comprises the following steps:
and when the data type does not belong to the preset attribute type, determining that the data attribute does not belong to the preset attribute type, and converting the data attribute into a target data type.
In order to solve the above technical problems, the embodiment of the present application further provides a data storage device, which adopts the following technical schemes:
The acquisition module is used for acquiring the original data from a preset data model;
the creation module is used for creating an original data object of the original data;
The extraction module is used for extracting the data attribute corresponding to the original data object;
The judging module is used for judging whether the data attribute belongs to a preset attribute type or not;
the merging module is used for taking the data attribute as a sub-data object and merging the original data object with the sub-data object to obtain an updated data object if the data attribute belongs to the preset attribute type, wherein the updated data object does not belong to the preset attribute type;
The conversion module is used for converting the data attribute and the updated data attribute corresponding to the updated data object into a target data type corresponding to the preset attribute type if the data attribute does not belong to the preset attribute type; and
And the storage module is used for storing the target data object corresponding to the target data type into a target area.
In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical schemes:
A memory storing at least one computer program; and
And a processor executing the computer program stored in the memory to realize the data storage.
In order to solve the above technical problems, an embodiment of the present application further provides a computer readable storage medium, which adopts the following technical schemes:
the computer readable storage medium has stored therein at least one computer program that is executed by a processor in an electronic device to effect the data storage described above.
Compared with the prior art, the application has the following main beneficial effects:
In the embodiment of the application, the original data object of the original data is created by acquiring the original data from the preset data model, and the data attribute corresponding to the original data object is extracted, so that the automatic creation of the data object and the data attribute can be realized, and the error probability is reduced; secondly, whether the data attribute belongs to a preset attribute type or not is judged, the data object can be updated under the condition that the data attribute does not belong to the preset attribute type, and the data type is converted under the condition that the data object does not belong to the preset attribute type, so that the storage requirement is better met, the problem that only simple iterative processing can be carried out on target data before storage is solved, and the processing capacity of the data before storage is enhanced; finally, the efficiency and accuracy of data storage are improved by storing the target data object corresponding to the target data type into the target area. Therefore, the data storage method, the device, the equipment and the storage medium can improve the efficiency and the accuracy of data storage.
Drawings
In order to more clearly illustrate the solution of the present application, a brief description will be given below of the drawings required for the description of the embodiments of the present application, it being apparent that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without the exercise of inventive effort for a person of ordinary skill in the art.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a data storage method according to the present application;
FIG. 3 is a block diagram of one embodiment of a fabric side of the data storage system according to the present application;
FIG. 4 is a schematic structural diagram of one embodiment of a computer device in accordance with the present application.
Detailed Description
The method for determining a data format provided in the embodiments of the present application is applied to a data processing system, and unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application pertains; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the description of the drawings above are intended to cover a non-exclusive inclusion. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to make the person skilled in the art better understand the solution of the present application, the technical solution of the embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social networking platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the data storage method provided by the embodiment of the present application is generally executed by a server/terminal device, and accordingly, the data storage system is generally disposed in the server/terminal device.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow chart of one embodiment of a data storage method according to the present application is shown. The data storage method comprises the following steps:
s210, acquiring original data from a preset data model.
In the embodiment of the present application, the preset data model may be a data structure model in various forms, for example, a database table, an XML file, etc.
In the embodiment of the present application, the original data refers to a data set that needs to be stored in a target area, and the original data includes all data in a database table or an XML file. For example, if the data model is a database table, the original data includes data of all columns in the database table, each column corresponds to an attribute, and the type of the attribute value is the data type of the column; if the data model is an XML file, the original data contains all elements and attributes in the XML file, each element or attribute corresponds to an attribute, and the type of the attribute value corresponds to the content type of the element or attribute.
In an optional embodiment of the present application, the original data is also data to be stored in an actual application scenario, and may be obtained by querying from an enterprise database through a predefined SQL query statement, and in order to obtain accurate original data, the database table in the enterprise database may have a situation that multiple users access or operate the data concurrently, and the concurrency control and transaction management technologies of the database may be used to assist in obtaining the original data.
The concurrency control is a technology for processing the same data accessed by a plurality of users by the database management system in time, and can prevent the operation of one user from affecting the operation of other users, thereby ensuring the consistency of the data; the transaction management technology is a technology that a database management system takes user operation as a whole, and can ensure that the acquired original data is all successful or all failed, thereby ensuring the integrity of the data.
Specifically, in the embodiment of the application, the database can be queried through the SQL query statement to obtain the original data, each row in the original data corresponds to one record in the database, and each column corresponds to one attribute of the record. For example, there is a user database table, where three fields stored in the table are "name", "age" and "address", respectively, and the original data of Zhang three, 30, xx province xx city can be returned by querying the information of a user, and each row corresponds to a user, and each column corresponds to a user attribute.
S220, creating an original data object of the original data.
In the embodiment of the present application, the original data object refers to a dynamic data structure describing data attributes and data types, and the structure and specific content of the original data object are determined by a data model. For example, if the data model is a database table, the data of each column in the database table is taken as an object, and the original data object includes the objects of all columns of the database table; if the data model is an XML file, the original data object takes each element and the corresponding attribute in the XML file as an object.
As one embodiment of the present application, the creating the original data object of the original data includes:
Acquiring a data identifier corresponding to the original data;
Identifying data attributes and data types corresponding to the data identifiers;
Abstracting the data attributes and the data types into a spatial data structure;
And creating an original data object corresponding to the data identifier according to the space data structure, the data attribute and the data type.
The data identifier corresponding to the original data can be a primary key for identifying the original data, the number of the data identifiers depends on the number of the data attributes of the original data, and if the original data is a database table, the data attributes of each column of the database table have corresponding data identifiers; if the original data is an XML file, each element and corresponding attribute of the XML file corresponds to different data identifications.
In one embodiment of the present application, the spatial data structure is a structural relationship expressing data attributes and data types, wherein the data attributes are a data field representing data characteristics of original data,
For example, if a column name of a column of data in the database table a is an insurance type, the corresponding data attribute is an "insurance type" field; the data types are types describing the values of the data attributes, and include basic data types (such as integers, floating point numbers, character strings and the like), composite data types (such as groups, lists, dictionaries or objects and the like), custom data types and the like. For example, a database table a has a column name of a column of data as an insurance class, and a value corresponding to an attribute of the insurance class is a car risk, a medical risk, an accident risk, etc., and a data type corresponding to the value is a character string type.
In the embodiment of the application, the original data object corresponding to the data identifier is created according to the spatial data structure, the data attribute and the data type, so that the automatic creation of the data object can be realized, the manual definition and creation of the data object and the attribute are not needed, the manual error rate is reduced, and the data processing efficiency is improved.
S230, extracting the data attribute corresponding to the original data object.
In the embodiment of the present application, the data attribute may first identify a plurality of data fields from the specific content by identifying the specific content included in the original data object, so as to generate the data attribute corresponding to each data field.
The specific content refers to database table content or XML file content of the original data.
S240, judging whether the data attribute belongs to a preset attribute type.
In the embodiment of the present application, the preset attribute type may be determined based on an actual service scenario, that is, the preset attribute type may be a collection type or an object type, where the data attributes belonging to the collection type and the object type are independent and include other data attributes. For example, there is a user information table including a user name attribute, a user age attribute, a user address attribute, and the like, and the user address attribute also includes a street attribute and a city attribute.
As an embodiment of the present application, the determining whether the data attribute belongs to a preset attribute type includes:
Acquiring an attribute type corresponding to the data attribute;
Identifying whether the attribute type belongs to the preset attribute type;
When the data attribute belongs to the preset attribute type, taking the data attribute as a sub data object, combining the original data object with the sub data object, and obtaining an updated data object comprises the following specific steps:
When the data type belongs to the preset attribute type, determining that the data attribute belongs to the preset attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object;
when the data attribute does not belong to the preset attribute type, the specific step of converting the data attribute into a target data type comprises the following steps:
and when the data type does not belong to the preset attribute type, determining that the data attribute does not belong to the preset attribute type, and converting the data attribute into a target data type.
The attribute types comprise collection types or object types and general types, and the general types comprise foreign key types, default value types, annotation types and the like, wherein the foreign key types are used for establishing association relations among data attributes; default value type is set data default value; the annotation type is used to add descriptive information about the data attributes.
In the embodiment of the application, whether the attribute type belongs to the preset attribute type is identified, so that the association between the original data objects with the association relationship can be conveniently carried out, the nested relationship in the original data objects can be accurately expressed, and the accuracy of the subsequent data storage is improved.
S250, if the data attribute belongs to the preset attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object, wherein the updated data object does not belong to the preset attribute type.
In the embodiment of the application, if the data attribute belongs to the preset attribute type, the data attribute is represented to contain other attributes, and the original data object is combined with the child data object to obtain the updated data object, so that the data object with the association relationship can be combined, the nested structure expression of the data object is realized, the problem that only simple iterative processing can be carried out on target data before storage is solved, and the subsequent storage requirement is better met.
In the embodiment of the application, the data sub-data object refers to creating a new sub-data model for the data attribute belonging to the preset attribute type, and using the object created for the sub-data model as the sub-data object, wherein the sub-data object is also a dynamic sub-data structure for describing the data attribute and the data type.
In the embodiment of the present application, the update data object includes a dynamic data structure associated with the original data object and the child data object.
As one embodiment of the present application, the merging the original data object with the child data object to obtain an updated data object includes:
identifying the association relationship between the original data object and the sub data object;
Generating an object structure of the original data object nested with the child data object according to the association relation;
and merging the original data object and the sub-original data object according to the object structure to obtain the updated data object.
The association relation indicates that the original data object is associated with the sub data object by identifying the data attribute of the sub data object if the data attribute exists in the attribute of the original data object. For example, the original data object is a user data object, and the user data object includes a user name attribute, a user age attribute, a user address attribute, and the like, and the word data object includes a user address attribute, a street attribute, and a city attribute, and then the sub data object is associated with the original data object through the user address attribute.
In one embodiment of the present application, a child data object is marked with a child attribute of the original data object during the process of generating the data object, thereby forming a nested object structure. For example, the user address attributes of the child data objects are marked as original data objects, thereby forming a nested user data object structure.
In the embodiment of the application, the object structure of the sub data object nested by the original data object is generated according to the association relation, so that more complex data structures such as nested objects can be processed, and meanwhile, the flexible processing of different data structures is realized, and the efficiency and the flexibility of data processing are improved.
And S260, if the data attribute does not belong to the preset attribute type, converting the data attribute and the updated data attribute corresponding to the updated data object into a target data type corresponding to the preset attribute type.
In the embodiment of the application, if the data attribute does not belong to the preset attribute type, the data structure of the data object is determined, and the data attribute corresponding to the updated data object are converted into the target data type corresponding to the preset attribute type, so that the problem that only simple iterative processing can be performed on the target data before storage is solved, the processing capability of the data before storage is enhanced, and the efficiency and the accuracy of data storage are improved conveniently.
In the embodiment of the application, the target data type refers to SQL data type.
As one embodiment of the present application, the converting the data attribute and the update data attribute corresponding to the update data object into the target data type corresponding to the preset attribute type includes:
acquiring metadata information of the data attribute and the updated data attribute, and identifying original data types corresponding to the data attribute and the updated data attribute according to the metadata information;
acquiring a preset type mapping table, and searching a target data type corresponding to the original data type in the type mapping table;
And converting the original data type into the target data type through a preset conversion function.
Wherein the metadata information is information describing data to provide detailed information about the data in detail, including the length, type, source, etc. of the data; in the process of identifying the original data type, the metadata information is analyzed first, and the data type corresponding to the data attribute is identified through data abstraction and data encapsulation operation according to the metadata information. For example, there is a column of data attribute "user age", and the data type of "user age" is identified as an integer type by reading the specific data of the column.
In the embodiment of the application, the concrete data type is abstracted into a general data type by a data abstraction principle, so that various different data types can be processed by a program; meanwhile, the data packaging principle (namely, a method for packaging data and operating the data together) is utilized, so that the original data type corresponding to the data attribute can be obtained more safely.
In an embodiment of the present application, the type mapping table includes mapping relationships between all original data types and corresponding SQL types. For example, the original data type is an Integer (intelger), and the corresponding SQL type is INT; the original data type is a String (String), and the corresponding SQL type is VARCHAR; the original data type is floating point number (Float), and the corresponding SQL type is Float; the original data type is Boolean (Boolean), and the corresponding SQL type is Boolean.
In the embodiment of the present application, the conversion function is used to convert data from one type to another type, and the conversion function may be an int () function in Python or an intel.
Wherein the int () function in Python or the inter. For example, there is a "user age" attribute, the corresponding original data type is an integer, the target data type corresponding to the integer is an INT type, and then the integer data type is directly mapped to the INT data type through an nt () function or an intel. The purpose of the adapter function is to enable two incompatible interfaces to work together, e.g. with a "user age" attribute, the corresponding original data type is a string, but it is stored as an INT type in the database, the original data type is converted from a string to an integer by the adapter function, and the string type is mapped to the corresponding INT type.
In the embodiment of the application, the original data type is converted into the target data type through the preset conversion function, so that the correct mapping and conversion of the data type can be ensured, and the correctness and consistency of the converted data are ensured.
And S270, storing the target data object corresponding to the target data type into a target area.
In the embodiment of the present application, the target area may be a database or a file.
As one embodiment of the present application, the storing the target data object corresponding to the target data type in the target area includes:
carrying out data serialization operation on each data attribute and data attribute value corresponding to the target data object to obtain a JSON key value pair;
acquiring an object structure of the target data object, and performing data structuring operation on the JSON key value pair according to the object structure to obtain a JSON data structure;
And acquiring a target storage position of the JSON data structure, and storing the JSON data structure into the target area according to the target storage position.
All data attributes and corresponding data attribute values corresponding to the target data object are traversed, each data attribute and corresponding data attribute value are converted into a JSON format, and then the data attributes and corresponding data attribute values in the JSON format are converted into JSON key value pairs. For example, the target data object has a data attribute named "name" with a data attribute value of "John", and the data attribute is converted to a JSON key value pair "name" John "by data serialization.
In an embodiment of the present application, the data structuring means that the JSON key value pair is organized into a corresponding JSON data structure according to the object structure of the target data object, and if the object structure is a nested structure, the corresponding JSON data structure is also a nested structure. For example, if the target data object includes three data attributes of "name", "age" and "address", where "address" is a sub data object including two data attributes of "street" and "city", during the data structuring process, JSON key value pairs corresponding to "name" and "address" are obtained during the structuring process, then "street" and "city" are also converted into JSON key value pairs, and finally these JSON key value pairs are organized into a nested JSON data structure.
Further, in the embodiment of the present application, a storage path of the JSON data structure is obtained, a storage position is determined according to the storage path, and the JSON data structure is stored in the target area according to the storage position.
In the embodiment of the application, the JSON key value pair is subjected to data structuring operation according to the object structure to obtain the JSON data structure, so that the JSON data structure can also completely express the object structure of the target data object when the target data object is converted into the JSON data structure, the consistency of data is ensured, the processing of complex data structures such as nested objects and sets is realized, and the flexibility and the efficiency of data processing are improved.
In the embodiment of the application, the JSON data structure is stored in the target area according to the target storage position, and the JSON data is selected as the main storage format, so that the data in the JSON format has good universality and interoperability, can be seamlessly exchanged between different systems and platforms, and improves the usability of the stored data.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
In the embodiment of the application, the original data object of the original data is created by acquiring the original data from the preset data model, and the data attribute corresponding to the original data object is extracted, so that the automatic creation of the data object and the data attribute can be realized, and the error probability is reduced; secondly, whether the data attribute belongs to a preset attribute type or not is judged, the data object can be updated under the condition that the data attribute does not belong to the preset attribute type, and the data type is converted under the condition that the data object does not belong to the preset attribute type, so that the storage requirement is better met, the problem that only simple iterative processing can be carried out on target data before storage is solved, and the processing capacity of the data before storage is enhanced; finally, the efficiency and accuracy of data storage are improved by storing the target data object corresponding to the target data type into the target area. Therefore, the data storage method provided by the embodiment of the application can improve the efficiency and the accuracy of data storage.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored in a computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
With further reference to fig. 3, as an implementation of the method shown in fig. 2 described above, the present application provides an embodiment of a data storage device 310, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic apparatuses.
The embodiment of the invention provides a data storage system, which comprises;
An obtaining module 311, configured to obtain raw data from a preset data model;
a creation module 312, configured to create an original data object of the original data;
An extracting module 313, configured to extract a data attribute corresponding to the original data object;
A judging module 314, configured to judge whether the data attribute belongs to a preset attribute type;
A merging module 315, configured to, if the data attribute belongs to the preset attribute type, take the data attribute as a child data object, and merge the original data object with the child data object to obtain an updated data object, where the updated data object does not belong to the preset attribute type;
A conversion module 316, configured to convert the data attribute and the updated data attribute corresponding to the updated data object into a target data type corresponding to the preset attribute type if the data attribute does not belong to the preset attribute type; and
And a storage module 317, configured to store the target data object corresponding to the target data type into a target area.
The creation module comprises:
the identification sub-module is used for identifying the data attribute and the data type corresponding to the data identifier;
An abstract sub-module for abstracting the data attributes and the data types into a spatial data structure;
and the creation sub-module is used for creating an original data object corresponding to the data identifier according to the space data structure, the data attribute and the data type.
The merging module comprises:
a relationship identification sub-module, configured to identify an association relationship between the original data object and the sub-data object;
The generation sub-module is used for generating an object structure of the original data object nested with the sub-data object according to the association relation;
and the merging sub-module is used for merging the original data object and the sub-original data object according to the object structure to obtain the updated data object.
The conversion module includes:
The updating sub-module is used for acquiring the data attribute and the metadata information of the updated data attribute, and identifying the data attribute and the original data type corresponding to the updated data attribute according to the metadata information;
the searching sub-module is used for acquiring a preset type mapping table, and searching a target data type corresponding to the original data type in the type mapping table;
and the conversion sub-module is used for converting the original data type into the target data type through a preset conversion function.
The memory module includes:
The serialization submodule is used for carrying out data serialization operation on each data attribute and data attribute value corresponding to the target data object to obtain a JSON key value pair;
the structuring sub-module is used for obtaining an object structure of the target data object, and carrying out data structuring operation on the JSON key value pair according to the object structure to obtain a JSON data structure;
and the storage sub-module is used for acquiring a target storage position of the JSON data structure and storing the JSON data structure into the target area according to the target storage position.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
In the embodiment of the application, the original data object of the original data is created by acquiring the original data from the preset data model, and the data attribute corresponding to the original data object is extracted, so that the automatic creation of the data object and the data attribute can be realized, and the error probability is reduced; secondly, whether the data attribute belongs to a preset attribute type or not is judged, the data object can be updated under the condition that the data attribute does not belong to the preset attribute type, and the data type is converted under the condition that the data object does not belong to the preset attribute type, so that the storage requirement is better met, the problem that only simple iterative processing can be carried out on target data before storage is solved, and the processing capacity of the data before storage is enhanced; finally, the efficiency and accuracy of data storage are improved by storing the target data object corresponding to the target data type into the target area. Therefore, the data storage device provided by the embodiment of the application can improve the efficiency and the accuracy of data storage.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It should be noted that only computer device 4 having components 41-43 is shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and its hardware includes, but is not limited to, a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), a Programmable gate array (Field-Programmable GATE ARRAY, FPGA), a digital Processor (DIGITAL SIGNAL Processor, DSP), an embedded device, and the like.
The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 41 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used for storing an operating system and various types of application software installed on the computer device 4, such as program codes of a data storage method, and the like. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.
The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute the program code stored in the memory 41 or process data, for example, execute the program code of the data storage method.
The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.
The present application also provides another embodiment, namely, a computer-readable storage medium storing the data storage method program, where the data storage method program is executable by at least one processor, so that the at least one processor performs the steps of the data storage method as described above.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general purpose hardware online platform, and of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
The application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
It is apparent that the above-described embodiments are only some embodiments of the present application, but not all embodiments, and the preferred embodiments of the present application are shown in the drawings, which do not limit the scope of the patent claims. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a thorough and complete understanding of the present disclosure. Although the application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing description, or equivalents may be substituted for elements thereof. All equivalent structures made by the content of the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the scope of the application.
Claims (10)
1. A method of data storage comprising the steps of:
Acquiring original data from a preset data model;
creating an original data object corresponding to the original data;
Extracting data attributes corresponding to the original data objects;
judging whether the data attribute belongs to a preset attribute type or not;
If the data attribute belongs to the preset attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object, wherein the updated data object does not belong to the preset attribute type;
If the data attribute does not belong to the preset attribute type, converting the data attribute and the updated data attribute corresponding to the updated data object into a target data type corresponding to the preset attribute type;
And storing the target data object corresponding to the target data type into a target area.
2. The data storage method of claim 1, wherein the creating the original data object of the original data comprises:
Acquiring a data identifier corresponding to the original data;
Identifying data attributes and data types corresponding to the data identifiers;
Abstracting the data attributes and the data types into a spatial data structure;
And creating an original data object corresponding to the data identifier according to the space data structure, the data attribute and the data type.
3. The data storage method according to claim 1, wherein said merging said original data object with said child data object to obtain an updated data object comprises:
identifying the association relationship between the original data object and the sub data object;
Generating an object structure of the original data object nested with the child data object according to the association relation;
and merging the original data object and the sub-original data object according to the object structure to obtain the updated data object.
4. The data storage method according to claim 1, wherein said converting the data attribute and the update data attribute corresponding to the update data object into the target data type corresponding to the preset attribute type, comprises:
acquiring metadata information of the data attribute and the updated data attribute, and identifying original data types corresponding to the data attribute and the updated data attribute according to the metadata information;
acquiring a preset type mapping table, and searching a target data type corresponding to the original data type in the type mapping table;
And converting the original data type into the target data type through a preset conversion function.
5. The data storage method according to any one of claims 1 to 4, wherein storing the target data object corresponding to the target data type in the target area includes:
carrying out data serialization operation on each data attribute and data attribute value corresponding to the target data object to obtain a JSON key value pair;
acquiring an object structure of the target data object, and performing data structuring operation on the JSON key value pair according to the object structure to obtain a JSON data structure;
And acquiring a target storage position of the JSON data structure, and storing the JSON data structure into the target area according to the target storage position.
6. The method according to any one of claims 1-4, wherein the determining whether the data attribute is of a preset attribute type comprises:
Acquiring an attribute type corresponding to the data attribute;
Identifying whether the attribute type belongs to the preset attribute type;
When the data attribute belongs to the preset attribute type, taking the data attribute as a sub data object, combining the original data object with the sub data object, and obtaining an updated data object comprises the following specific steps:
When the data type belongs to the preset attribute type, determining that the data attribute belongs to the preset attribute type, taking the data attribute as a sub-data object, and combining the original data object with the sub-data object to obtain an updated data object;
when the data attribute does not belong to the preset attribute type, the specific step of converting the data attribute into a target data type comprises the following steps:
and when the data type does not belong to the preset attribute type, determining that the data attribute does not belong to the preset attribute type, and converting the data attribute into a target data type.
7. A data storage device, comprising:
The acquisition module is used for acquiring the original data from a preset data model;
the creation module is used for creating an original data object of the original data;
The extraction module is used for extracting the data attribute corresponding to the original data object;
The judging module is used for judging whether the data attribute belongs to a preset attribute type or not;
the merging module is used for taking the data attribute as a sub-data object and merging the original data object with the sub-data object to obtain an updated data object if the data attribute belongs to the preset attribute type, wherein the updated data object does not belong to the preset attribute type;
The conversion module is used for converting the data attribute and the updated data attribute corresponding to the updated data object into a target data type corresponding to the preset attribute type if the data attribute does not belong to the preset attribute type; and
And the storage module is used for storing the target data object corresponding to the target data type into a target area.
8. The data storage device of claim 7, wherein the creation module comprises:
the identification sub-module is used for identifying the data attribute and the data type corresponding to the data identifier;
An abstract sub-module for abstracting the data attributes and the data types into a spatial data structure;
and the creation sub-module is used for creating an original data object corresponding to the data identifier according to the space data structure, the data attribute and the data type.
9. A computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the data storage method of any of claims 1 to 6 when the computer program is executed.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the data storage method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311835254.0A CN117931910A (en) | 2023-12-27 | 2023-12-27 | Data storage method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311835254.0A CN117931910A (en) | 2023-12-27 | 2023-12-27 | Data storage method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117931910A true CN117931910A (en) | 2024-04-26 |
Family
ID=90762184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311835254.0A Pending CN117931910A (en) | 2023-12-27 | 2023-12-27 | Data storage method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117931910A (en) |
-
2023
- 2023-12-27 CN CN202311835254.0A patent/CN117931910A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112671734B (en) | Message processing method for multiple data sources and related equipment thereof | |
| CN112860662B (en) | Automatic production data blood relationship establishment method, device, computer equipment and storage medium | |
| CN113626223A (en) | Interface calling method and device | |
| WO2023134134A1 (en) | Method and apparatus for generating association viewing model, and computer device and storage medium | |
| CN116644213A (en) | XML file reading method, device, equipment and storage medium | |
| CN113010542B (en) | Service data processing method, device, computer equipment and storage medium | |
| CN112765270A (en) | Block chain data processing method and device, computer equipment and medium | |
| CN116974929A (en) | Automatic test tool construction method, automatic test method and related equipment thereof | |
| CN111444235A (en) | Django-based data serialization method and device, computer equipment and storage medium | |
| CN109189728B (en) | Intelligent hardware device, magnetic disk data processing method and storage medium | |
| CN115756692A (en) | Method for automatically combining and displaying pages based on style attributes and related equipment thereof | |
| CN115238009A (en) | Metadata management method, device and equipment based on blood vessel margin analysis and storage medium | |
| CN117931910A (en) | Data storage method, device, equipment and storage medium | |
| CN113239670A (en) | Method and device for uploading service template, computer equipment and storage medium | |
| CN117033249B (en) | Test case generation method and device, computer equipment and storage medium | |
| CN116975080A (en) | Data batch change processing method, device, equipment and storage medium | |
| CN113806372B (en) | New data information construction method, device, computer equipment and storage medium | |
| CN113254826B (en) | Dump file processing method and device | |
| CN118095232A (en) | Form data importing method, device, equipment and storage medium | |
| CN117492752A (en) | Page dynamic configuration method and device, computer equipment and storage medium | |
| CN117171172A (en) | Form processing method, device, equipment and storage medium | |
| CN117033249A (en) | Test case generation method and device, computer equipment and storage medium | |
| CN116775649A (en) | Data classified storage method and device, computer equipment and storage medium | |
| CN115858639A (en) | Method, system and storage medium for dynamically generating financial statement | |
| CN117667668A (en) | Application analysis method, device, equipment and storage medium based on application upgrading |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |