CN110928876A - Credit data storage method and device - Google Patents
Credit data storage method and device Download PDFInfo
- Publication number
- CN110928876A CN110928876A CN201911097504.9A CN201911097504A CN110928876A CN 110928876 A CN110928876 A CN 110928876A CN 201911097504 A CN201911097504 A CN 201911097504A CN 110928876 A CN110928876 A CN 110928876A
- Authority
- CN
- China
- Prior art keywords
- data
- credit
- storing
- json
- credit data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Abstract
The invention provides a method and a device for storing credit data, wherein the method comprises the following steps: receiving credit data of a JSON structure; carrying out structure splitting on credit worthiness data of the JSON structure to obtain credit worthiness data of a non-nested structure; associating the credit data without the nested structure according to the JSON structure by utilizing an external key; and storing the associated credit data. The credit standing data storage method and device provided by the invention can efficiently and massively store the credit standing data of the user, effectively reduce the payment cost of a company, have higher safety and flexible service expansibility and system stability, and bring better experience to the user.
Description
Technical Field
The invention relates to the field of data storage, in particular to a credit worthiness data storage method and a credit worthiness data storage device.
Background
In order to better serve users and control risks, many enterprises need to cooperate with external data sources to solve the defects of the enterprises themselves in other aspects, but when the enterprises are combined with the external data sources, credit data returned by the data sources need to be stored, the content volume of the credit data is large, and the increasing speed is difficult to expect. Conventional relational databases are no longer suitable for storing such credit worthiness data.
Therefore, at the present stage, a storage method capable of storing user credit data in a mass manner is needed, so that the payment cost of a company is reduced, and the security of the data can be guaranteed.
Disclosure of Invention
In the present phase, when an enterprise is combined with an external data source, the credit data returned by the data source needs to be stored, but the content volume of the credit data is large, and the increasing speed is difficult to expect.
In a first aspect, the present invention provides a method for storing credit data, the method comprising:
receiving credit data of a JSON structure;
carrying out structure splitting on credit worthiness data of the JSON structure to obtain credit worthiness data of a non-nested structure;
associating the credit data without the nested structure according to the JSON structure by utilizing an external key;
and storing the associated credit data.
Further, storing the associated credit worthiness data further comprises:
carrying out serialization processing on the associated credit data;
and storing the processed credit data.
Further, storing the processed credit data includes:
and storing the processed credit data to an Amazon file service system.
Further, storing the processed credit data to the amazon file service system includes:
and storing the processed data to an Amazon file service system according to a JSON format or a Protosteff format.
In a second aspect, the present invention provides a credit data storage device, including:
the receiving module is used for receiving credit worthiness data of the JSON structure;
the splitting module is used for carrying out structural splitting on credit worthiness data of the JSON structure to obtain the credit worthiness data without a nested structure;
the association module is used for associating the credit worthiness data without the nested structure according to the JSON structure by utilizing an external key;
and the storage module is used for storing the associated credit data.
Further, the storage module further comprises:
the serialization processing unit is used for carrying out serialization processing on the associated credit data;
and the storage unit is used for storing the processed credit data.
Further, the memory cell includes:
and storing the processed credit data into an amazon file service system unit.
Further, the store-to-amazon file service system unit includes:
and selecting a storage format unit for storing the processed data to the Amazon file service system according to a JSON format or a Protosteff format.
In a third aspect, the present invention provides an electronic device, comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor implements the steps of the method for storing credit data provided in the first aspect when executing the program.
In a fourth aspect, the present invention provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of credit worthiness data storage provided by the first aspect.
The credit standing data storage method and device provided by the invention can efficiently and massively store the credit standing data of the user, effectively reduce the payment cost of a company, have higher safety and flexible service expansibility and system stability, and bring better experience to the user.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for storing credit data according to an embodiment of the present invention;
FIG. 2 is a block diagram of a credit data storage apparatus according to an embodiment of the present invention;
fig. 3 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to better serve users and control risks and increase enterprise competitiveness, each wind control enterprise in the current stage can choose to introduce an external credit data source to expand a data source sample set, but after the external credit data is added, how to store a large amount of external credit data and extract the external credit data in time becomes a difficult problem to be solved in the current stage. In order to solve the above problem, an embodiment of the present invention provides a method for storing credit data, as shown in fig. 1, the method includes:
step S101, receiving credit worthiness data of a JSON structure;
step S102, carrying out structure splitting on credit worthiness data of the JSON structure to obtain credit worthiness data without a nested structure;
step S103, associating the credit worthiness data without the nested structure according to a JSON structure by using an external key;
and step S104, storing the associated credit data.
Specifically, the received external credit data is basically a JSON structure, each data can be given a unique key value to be stored, however, the data storage has a trouble of data analysis caused by future analysis modeling, so the credit data of the JSON structure can be split firstly, that is, the nested JSON structure can be split into the non-nested structure, and meanwhile, an external key is added to correlate the original nested structure. That is, even if the nested part is subjected to non-nested splitting, the non-nested part needs to be associated to ensure the integrity of the data. For example, the data structure returned by the data source is of the form:
the intelligent structure splitting splits the original structure into the following forms:
the source code shows that the original structured data is divided into new data structures, nested structures are not contained in the new data structures, and foreign keys are added to correlate the relationships among the new data structures. Where "p _ id" may represent a foreign key, serving as the id value of association A: "a1": and "a2": respectively.
And finally storing the credit worthiness data without the nested structure in a file form.
The credit worthiness data storage method provided by the embodiment of the invention can efficiently store user credit worthiness data in a large amount, effectively reduce the payment cost of a company, has higher safety and flexible service expansibility and system stability, and brings better experience to users.
Based on the content of the above embodiments, as an alternative embodiment: storing the associated credit data further comprises:
carrying out serialization processing on the associated credit data;
and storing the processed credit data.
Specifically, after splitting the JSON structure of credit worthiness data, the split parts need to be re-associated, that is, even if the nested parts are split without nesting, the nested parts need to be associated to ensure the integrity of the data.
And carrying out serialization processing on the associated credit worthiness data, wherein the serialization is a process of converting the state information of the object into a form which can be stored or transmitted. And then storing the processed credit data.
Based on the content of the above embodiments, as an alternative embodiment: storing the processed credit data comprises:
and storing the processed credit data to an Amazon file service system.
Specifically, in the embodiment of the present invention, the file is stored in the amazon file service system, and compared with the conventional relational database, the system has: good expandability, usability and durability; abundant, economical and efficient storage classes; good safety, compliance and auditing function; low cost and the like.
The embodiment of the invention adopts the amazon file service system to store the external credit data, so that the stored data has good expansibility and persistence and is beneficial to later retrieval and calling.
Based on the content of the above embodiments, as an alternative embodiment: storing the processed credit data to the amazon file service system comprises the following steps:
and storing the processed data to an Amazon file service system according to a JSON format or a Protosteff format.
Specifically, JSON (JSON Object Notation) is a lightweight data exchange format. It stores and represents data in a text format that is completely independent of the programming language, based on a subset of ECMAScript (js specification set by the european computer association). The ProtoStuff is an open-source JAVA language-based serialization library, wherein the serialization format supported by the ProtoStuff comprises: protobuf, protostuff, graph.
According to another aspect of the present invention, an apparatus for providing credit worthiness data storage according to an embodiment of the present invention is provided, and referring to fig. 2, fig. 2 is a block diagram of the apparatus for providing credit worthiness data storage according to an embodiment of the present invention. The device is used for completing the credit data storage provided by the embodiment of the invention in the embodiments. Therefore, the description and definition in the method for storing credit data provided by the embodiment of the present invention in the foregoing embodiments may be used for understanding each execution module in the embodiment of the present invention.
The device includes:
a receiving module 201, configured to receive credit worthiness data of a JSON structure;
the splitting module 202 is used for performing structure splitting on credit worthiness data of the JSON structure to obtain the credit worthiness data of a non-nested structure;
the association module 203 is used for associating the credit data without the nested structure according to the JSON structure by using an external key;
the storage module 204 is configured to store the associated credit data.
Specifically, the specific process of each module in the apparatus of this embodiment to implement its function may refer to the related description in the corresponding method embodiment, and is not described herein again.
The credit worthiness data storage device provided by the embodiment of the invention can efficiently store user credit worthiness data in a large amount, effectively reduce the payment cost of a company, has higher safety and flexible service expansibility and system stability, and brings better experience to users.
Based on the content of the above embodiments, as an alternative embodiment: the memory module further includes:
the serialization processing unit is used for carrying out serialization processing on the associated credit data;
and the storage unit is used for storing the processed credit data.
Specifically, the specific process of each module in the apparatus of this embodiment to implement its function may refer to the related description in the corresponding method embodiment, and is not described herein again.
Based on the content of the above embodiments, as an alternative embodiment: the memory cell includes:
and storing the processed credit data into an amazon file service system unit.
Specifically, the specific process of each module in the apparatus of this embodiment to implement its function may refer to the related description in the corresponding method embodiment, and is not described herein again.
The embodiment of the invention adopts the amazon file service system to store the external credit data, so that the stored data has good expansibility and persistence and is beneficial to later retrieval and calling.
Based on the content of the above embodiments, as an alternative embodiment: the store-to-amazon file service system unit includes:
and selecting a storage format unit for storing the processed data to the Amazon file service system according to a JSON format or a Protosteff format.
Specifically, the specific process of each module in the apparatus of this embodiment to implement its function may refer to the related description in the corresponding method embodiment, and is not described herein again.
Fig. 3 is a block diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 3, the electronic device includes: a processor 301, a memory 302, and a bus 303;
the processor 301 and the memory 302 respectively complete communication with each other through a bus 303; the processor 301 is configured to call the program instructions in the memory 302 to execute the method for storing credit data provided by the above embodiment, for example, including: receiving credit data of a JSON structure; carrying out structure splitting on credit worthiness data of the JSON structure to obtain credit worthiness data of a non-nested structure; associating the credit data without the nested structure according to the JSON structure by utilizing an external key; and storing the associated credit data.
Embodiments of the present invention provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of a method of credit worthiness data storage. Examples include: receiving credit data of a JSON structure; carrying out structure splitting on credit worthiness data of the JSON structure to obtain credit worthiness data of a non-nested structure; associating the credit data without the nested structure according to the JSON structure by utilizing an external key; and storing the associated credit data.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.
Finally, the principle and the implementation of the present invention are explained by applying the specific embodiments in the present invention, and the above description of the embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method of credit worthiness data storage, the method comprising:
receiving credit data of a JSON structure;
carrying out structure splitting on credit worthiness data of the JSON structure to obtain credit worthiness data of a non-nested structure;
the credit worthiness data of the non-nested structure is associated according to a JSON structure by utilizing an external key;
and storing the associated credit data.
2. The method of claim 1, wherein storing the associated credit worthiness data further comprises:
carrying out serialization processing on the associated credit data;
and storing the processed credit data.
3. The method of claim 2, wherein storing the processed credit data comprises:
and storing the processed credit data to an Amazon file service system.
4. The method of claim 3, wherein storing the processed credit data to an Amazon File services System comprises:
and storing the processed data to an Amazon file service system according to a JSON format or a Protosteff format.
5. An apparatus for credit data storage, the apparatus comprising:
the receiving module is used for receiving credit worthiness data of the JSON structure;
the splitting module is used for carrying out structural splitting on the credit worthiness data of the JSON structure to obtain the credit worthiness data without a nested structure;
the association module is used for associating the credit worthiness data of the non-nested structure according to a JSON structure by utilizing an external key;
and the storage module is used for storing the associated credit data.
6. The apparatus of claim 5, wherein the storage module further comprises:
the serialization processing unit is used for carrying out serialization processing on the associated credit data;
and the storage unit is used for storing the processed credit data.
7. The apparatus of claim 6, wherein the storage unit comprises:
and the data is stored to an amazon file service system unit and used for storing the processed credit data to an amazon file service system.
8. The apparatus of claim 7, wherein the store-to-amazon file service system unit comprises:
and selecting a storage format unit for storing the processed data to the Amazon file service system according to a JSON format or a Protosteff format.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of credit data storage according to any of claims 1 to 4 are implemented when the program is executed by the processor.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of credit data storage according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911097504.9A CN110928876A (en) | 2019-11-08 | 2019-11-08 | Credit data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911097504.9A CN110928876A (en) | 2019-11-08 | 2019-11-08 | Credit data storage method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110928876A true CN110928876A (en) | 2020-03-27 |
Family
ID=69852673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911097504.9A Pending CN110928876A (en) | 2019-11-08 | 2019-11-08 | Credit data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110928876A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113037848A (en) * | 2021-03-18 | 2021-06-25 | 上海哔哩哔哩科技有限公司 | File uploading method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104160394A (en) * | 2011-12-23 | 2014-11-19 | 阿米亚托股份有限公司 | Scalable analysis platform for semi-structured data |
| US20150378994A1 (en) * | 2014-06-26 | 2015-12-31 | International Business Machines Corporation | Self-documentation for representational state transfer (rest) application programming interface (api) |
| CN109800363A (en) * | 2018-12-13 | 2019-05-24 | 深圳壹账通智能科技有限公司 | Construct method, apparatus, equipment and the storage medium of standing relational network |
-
2019
- 2019-11-08 CN CN201911097504.9A patent/CN110928876A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104160394A (en) * | 2011-12-23 | 2014-11-19 | 阿米亚托股份有限公司 | Scalable analysis platform for semi-structured data |
| US20150378994A1 (en) * | 2014-06-26 | 2015-12-31 | International Business Machines Corporation | Self-documentation for representational state transfer (rest) application programming interface (api) |
| CN109800363A (en) * | 2018-12-13 | 2019-05-24 | 深圳壹账通智能科技有限公司 | Construct method, apparatus, equipment and the storage medium of standing relational network |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113037848A (en) * | 2021-03-18 | 2021-06-25 | 上海哔哩哔哩科技有限公司 | File uploading method and system |
| CN113037848B (en) * | 2021-03-18 | 2024-03-15 | 上海哔哩哔哩科技有限公司 | File uploading method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10547747B1 (en) | Configurable natural language contact flow | |
| US20180365254A1 (en) | Method and apparatus for processing information flow data | |
| US20130262461A1 (en) | Method and System for Centralized Issue Tracking | |
| CN101373432B (en) | Method and system for predicting component system performance based on intermediate part | |
| CN110781180B (en) | Data screening method and data screening device | |
| CN105824974A (en) | Method and system for analyzing and processing data | |
| CN113835692A (en) | Dictionary data processing method and device, electronic equipment and computer storage medium | |
| CN111158800B (en) | Method and device for constructing task DAG based on mapping relation | |
| CN114416868B (en) | Data synchronization method, device, equipment and storage medium | |
| CN111046640A (en) | Dynamic generation method and device of certification document | |
| CN110928876A (en) | Credit data storage method and device | |
| CN109165180A (en) | A kind of method and system improving source of houses broker operating efficiency | |
| CN117314139A (en) | Modeling method and device for business process, terminal equipment and storage medium | |
| CN110263104A (en) | JSON character string processing method and device | |
| CN111142871A (en) | Front-end page development system, method, equipment and medium | |
| CN106599244B (en) | General original log cleaning device and method | |
| CN114968917A (en) | Method and device for rapidly importing file data | |
| CN116263717A (en) | Order service processing method and device based on event | |
| US10922312B2 (en) | Optimization of data processing job execution using hash trees | |
| CN108460077B (en) | Index analysis method, system and computer readable storage medium | |
| CN107247621B (en) | Scheduling method and device for batch processing tasks | |
| CN114116877B (en) | Data conversion method, device, computer equipment and medium | |
| CN109815123A (en) | Interface testing case script classification method, device, electronic equipment and medium | |
| US20220398524A1 (en) | Consumer interaction agent management platform | |
| US10387554B1 (en) | Applying matching data transformation information based on a user's editing of data within a document |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200327 |
|
| RJ01 | Rejection of invention patent application after publication |