CN110554877A - JSON data analysis method, device, equipment and storage medium - Google Patents

JSON data analysis method, device, equipment and storage medium Download PDF

Info

Publication number
CN110554877A
CN110554877A CN201910837322.4A CN201910837322A CN110554877A CN 110554877 A CN110554877 A CN 110554877A CN 201910837322 A CN201910837322 A CN 201910837322A CN 110554877 A CN110554877 A CN 110554877A
Authority
CN
China
Prior art keywords
data
iterator
json
preset
matched
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910837322.4A
Other languages
Chinese (zh)
Inventor
程捷
罗俊
刘林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Bo Hongyuan Data Polytron Technologies Inc
Original Assignee
Beijing Bo Hongyuan Data Polytron Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Bo Hongyuan Data Polytron Technologies Inc filed Critical Beijing Bo Hongyuan Data Polytron Technologies Inc
Priority to CN201910837322.4A priority Critical patent/CN110554877A/en
Publication of CN110554877A publication Critical patent/CN110554877A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/40Transformation of program code
    • G06F8/41Compilation
    • G06F8/42Syntactic analysis
    • G06F8/427Parsing

Abstract

the embodiment of the invention discloses a JSON data analysis method, a device, equipment and a storage medium, wherein the method comprises the following steps: analyzing the data stream of the JSON data to generate character string data; adopting a first preset iterator matched with the data structure of the JSON data to analyze the character string data to obtain analyzed data; wherein the analytic data comprises an object and/or an array; and filtering the objects in the analysis data by adopting a second preset iterator, and screening to obtain object data matched with the target key. The technical scheme of the embodiment of the invention realizes the simplification of the process of streaming JSON data analysis, reduces the complexity of codes and improves the programming efficiency.

Description

JSON data analysis method, device, equipment and storage medium
Technical Field
the embodiment of the invention relates to a communication technology, in particular to a JSON data analysis method, a JSON data analysis device, JSON data analysis equipment and a storage medium.
Background
In the data query Application Programming Interface (API) protocol, we generally prefer to use JS Object Notation (JSON) format as the transport protocol.
when data is transmitted through a network, a sender serializes JSON data into a binary form and transmits the binary form to a receiver through the network, the receiver receives the data completely, then deserializes the data into JSON objects through an analysis tool, and then the data is obtained. In the prior art, the JSON data is parsed by streaming data JSON. The third party toolkit also provides some objects for streaming parsing JSON, such as JsonParser class provided by jackson package, JSONReader class provided by fastjson package, and the like, and the streaming parsing mode needs to be transmitted as input stream.
however, in the streaming data parsing manner, since parsing needs to be performed on different JSON structures, multiple layers of nesting may exist in the JSON structures, which requires higher encoding capability, and written parsing code statements have many branches, which makes the code more complex.
disclosure of Invention
Embodiments of the present invention provide a method, an apparatus, a device, and a storage medium for JSON data parsing, so as to simplify a process of streaming JSON data parsing and reduce complexity of a code.
In a first aspect, an embodiment of the present invention provides a JSON data analysis method, including:
analyzing the data stream of the JSON data to generate character string data;
adopting a first preset iterator matched with the data structure of the JSON data to analyze the character string data to obtain analyzed data; wherein the analytic data comprises an object and/or an array;
and filtering the objects in the analysis data by adopting a second preset iterator, and screening to obtain object data matched with the target key.
In a second aspect, an embodiment of the present invention further provides a JSON data parsing apparatus, including:
The character string data generation module is used for analyzing the data stream of the JSON data to generate character string data;
The analysis data obtaining module is used for adopting a first preset iterator matched with the data structure of the JSON data to analyze the character string data to obtain analysis data; wherein the analytic data comprises an object and/or an array;
and the object data screening module is used for filtering the objects in the analysis data by adopting a second preset iterator, and screening to obtain object data matched with the target key.
In a third aspect, an embodiment of the present invention further provides an apparatus, where the apparatus includes:
One or more processors;
A memory for storing one or more programs;
When executed by the one or more processors, cause the one or more processors to implement a JSON data parsing method as provided in any embodiment of the invention.
In a fourth aspect, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are used to perform the JSON data parsing method provided in any embodiment of the present invention.
according to the embodiment of the invention, the JSON data is analyzed by using the iterator, and the object obtained by analysis is filtered, so that the problems of complex codes and low analysis efficiency in the process of analyzing the JSON data in a streaming manner are solved, the process of analyzing the JSON data in the streaming manner is simplified, the complexity of the codes is reduced, and the programming efficiency effect is improved.
drawings
fig. 1 is a flowchart of a JSON data parsing method in a first embodiment of the present invention;
fig. 2 is a flowchart of a JSON data parsing method in the second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a JSON data parsing device in a third embodiment of the present invention;
Fig. 4 is a schematic structural diagram of an apparatus in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
example one
fig. 1 is a flowchart of a JSON data parsing method according to an embodiment of the present invention, where the present embodiment is applicable to a case of performing stream parsing on JSON data, and the method may be executed by a JSON data parsing device, where the device may be implemented by hardware and/or software, and may be generally integrated in a node, and the method specifically includes the following steps:
Step 110, analyzing the data stream of the JSON data to generate character string data;
The data stream of the JSON data is analyzed, and the input stream needs to be transmitted into the node. Optionally, a preset third-party toolkit is used for analyzing the data stream to generate the character string data. For example, a stream parsing tool Parser may be used to parse the data stream of the JSON data to obtain a character string in the JSON data.
Step 120, analyzing the character string data by adopting a first preset iterator matched with the data structure of the JSON data to obtain analyzed data;
Wherein the parsed data includes objects and/or arrays. The iterator for analyzing the JSON data structure can be constructed in advance, optionally, the data structure of an object and/or an array in the JSON data is determined according to a preset communication protocol before the data stream of the JSON data is analyzed to generate character string data; according to the data structure, a first preset iterator matched with the data structure is constructed; and constructing a second preset iterator for screening the object data of the target key according to the target key set. The predetermined communication protocol is predefined, and therefore, the data structure in the JSON data is also predetermined. The data structure in the JSON data to be analyzed can be obtained in advance, for example, whether the object and the array in the JSON data are nested or not and how the object and the array are nested, so that the object and the array in the JSON data can be analyzed by constructing a first preset iterator matched with the data structure.
and step 130, filtering the objects in the analyzed data by using a second preset iterator, and screening to obtain object data matched with the target key.
For the key value pairs which are particularly interesting to the user, a target key set can be established in advance, and the selected target keys are stored in a preset set. Thus, the object returned by the first preset iterator can be input into the second preset iterator, the object is filtered, and the key value pair matched with the target key is screened out.
According to the technical scheme of the embodiment, the JSON data is analyzed by using the iterator, and the object obtained by analyzing is filtered, so that the problems of complex codes and low analysis efficiency in the process of analyzing the JSON data in a streaming manner are solved, the process of analyzing the JSON data in the streaming manner is simplified, the complexity of the codes is reduced, and the programming efficiency effect is improved.
Example two
fig. 2 is a flowchart of a JSON data parsing method according to a second embodiment of the present invention, which is further detailed based on the foregoing technical solution, where the method specifically includes:
Step 210, analyzing the data stream of the JSON data to generate character string data;
step 220, analyzing the character string data by adopting an object iterator and/or an array iterator in the first preset iterator to obtain an object and/or an array in the analyzed data;
The first preset iterator may be a combination of the object iterator and the array iterator. The object iterator is used for analyzing an object structure in JSON data, input parameters are analysis tool objects, a hasNext method judges whether a current token is an object end symbol "}", and a next method returns a key value pair Entry object. The array iterator is used for analyzing an array structure in JSON data, an input parameter is an analysis tool Object, the hasNext method judges whether the current token is an array end symbol' ], and the next method returns an Object type Object.
step 230, inputting the obtained object into a second preset iterator, and matching the object with a target key in a target key set;
And step 240, determining the object successfully matched as the object data matched with the target key.
and if the key in the next element is true, judging whether the key is contained in the set, and returning the Entry object meeting the screening condition by the next method.
the input parameters and output results of the iterator used in this embodiment are shown in table 1.
TABLE 1
According to the technical scheme of the embodiment, whether the reading of the object and the array needing to be analyzed is finished is determined through a hasNest method, when the reading is not finished, the analysis result is returned, and the filtering iterator is utilized to realize the screening to obtain the key value pair of interest.
EXAMPLE III
fig. 3 is a schematic structural diagram of a JSON data analysis device according to a third embodiment of the present invention, where the device specifically includes:
the character string data generation module 310 is configured to parse a data stream of the JSON data to generate character string data;
The analysis data obtaining module 320 is configured to analyze the character string data by using a first preset iterator matched with the data structure of the JSON data to obtain analysis data; the analysis data comprises an object and/or an array;
and the object data screening module 330 is configured to filter objects in the analysis data by using a second preset iterator, and screen the objects to obtain object data matched with the target key.
according to the technical scheme of the embodiment, the JSON data is analyzed by using the iterator, and the object obtained by analyzing is filtered, so that the problems of complex codes and low analysis efficiency in the process of analyzing the JSON data in a streaming manner are solved, the process of analyzing the JSON data in the streaming manner is simplified, the complexity of the codes is reduced, and the programming efficiency effect is improved.
Optionally, the character string data generating module 310 is specifically configured to:
And analyzing the data stream by adopting a preset third-party tool kit to generate character string data.
Optionally, the JSON data analysis device further includes:
the data structure determining module is used for determining the data structure of an object and/or an array in the JSON data according to a preset communication protocol before analyzing the data stream of the JSON data and generating character string data;
The first preset iterator construction module is used for constructing a first preset iterator matched with the data structure according to the data structure;
and the second preset iterator constructing module is used for constructing a second preset iterator for screening the object data of the target key according to the target key set.
Optionally, the analysis data obtaining module 320 is specifically configured to:
and analyzing the character string data by adopting an object iterator and/or an array iterator in the first preset iterator to obtain an object and/or an array in the analyzed data.
Optionally, the object data filtering module 330 is specifically configured to:
Inputting the obtained object into a second preset iterator, and matching the obtained object with a target key in a target key set;
And determining the object successfully matched as the object data matched with the target key.
the JSON data analysis device provided by the embodiment of the invention can execute the JSON data analysis method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
example four
fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention, as shown in fig. 4, the apparatus includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processors 410 in the device may be one or more, and one processor 410 is taken as an example in fig. 4; the processor 410, the memory 420, the input device 430 and the output device 440 in the apparatus may be connected by a bus or other means, for example, in fig. 4.
the memory 420 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the JSON data parsing method in the embodiments of the present invention (for example, the character string data generating module 310, the parsed data obtaining module 320, and the object data filtering module 330 in the JSON data parsing apparatus). The processor 410 executes various functional applications of the device and data processing by executing software programs, instructions, and modules stored in the memory 420, that is, implements the JSON data parsing method described above.
The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 420 may further include memory located remotely from processor 410, which may be connected to devices through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
the input means 430 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the apparatus. The output device 440 may include a display device such as a display screen.
EXAMPLE five
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, where the computer-executable instructions are executed by a computer processor to perform a JSON data parsing method, and the method includes:
Analyzing the data stream of the JSON data to generate character string data;
Adopting a first preset iterator matched with the data structure of the JSON data to analyze the character string data to obtain analyzed data; wherein the analytic data comprises an object and/or an array;
And filtering the objects in the analysis data by adopting a second preset iterator, and screening to obtain object data matched with the target key.
of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also execute related operations in the JSON data parsing method provided by any embodiment of the present invention.
from the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiments of the JSON data parsing apparatus, the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (10)

1. A JSON data analysis method is characterized by comprising the following steps:
Analyzing the data stream of the JSON data to generate character string data;
adopting a first preset iterator matched with the data structure of the JSON data to analyze the character string data to obtain analyzed data; wherein the analytic data comprises an object and/or an array;
And filtering the objects in the analysis data by adopting a second preset iterator, and screening to obtain object data matched with the target key.
2. The method for parsing JSON data according to claim 1, wherein the parsing the data stream of JSON data to generate character string data comprises:
And analyzing the data stream by adopting a preset third-party tool kit to generate the character string data.
3. the JSON data parsing method according to claim 1, wherein before parsing the data stream of JSON data to generate string data, the method further comprises:
Determining a data structure of an object and/or an array in the JSON data according to a preset communication protocol;
according to the data structure, constructing the first preset iterator matched with the data structure;
and constructing the second preset iterator for screening the object data of the target key according to the target key set.
4. the JSON data parsing method according to claim 3, wherein the parsing the character string data by using a first preset iterator matched with a data structure of the JSON data to obtain parsed data comprises:
And analyzing the character string data by adopting an object iterator and/or an array iterator in the first preset iterator to obtain the object and/or the array in the analyzed data.
5. the JSON data analysis method according to claim 3, wherein the filtering the objects in the analysis data by using a second preset iterator to obtain object data matched with the target key by screening comprises:
inputting the obtained object into the second preset iterator to be matched with the target key in the target key set;
And determining the object successfully matched as the object data matched with the target key.
6. A JSON data analysis device is characterized by comprising:
The character string data generation module is used for analyzing the data stream of the JSON data to generate character string data;
the analysis data obtaining module is used for adopting a first preset iterator matched with the data structure of the JSON data to analyze the character string data to obtain analysis data; wherein the analytic data comprises an object and/or an array;
And the object data screening module is used for filtering the objects in the analysis data by adopting a second preset iterator, and screening to obtain object data matched with the target key.
7. the JSON data parsing device according to claim 6, further comprising:
The data structure determining module is used for determining the data structure of an object and/or an array in the JSON data according to a preset communication protocol before the data stream of the JSON data is analyzed to generate character string data;
The first preset iterator construction module is used for constructing the first preset iterator matched with the data structure according to the data structure;
and the second preset iterator constructing module is used for constructing the second preset iterator for screening the object data of the target key according to the target key set.
8. The JSON data parsing device according to claim 7, wherein the object data filtering module is specifically configured to:
Inputting the obtained object into the second preset iterator to be matched with the target key in the target key set;
And determining the object successfully matched as the object data matched with the target key.
9. An apparatus, characterized in that the apparatus comprises:
One or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the JSON data parsing method of any one of claims 1-5.
10. a storage medium containing computer executable instructions for performing the JSON data parsing method of any one of claims 1-5 when executed by a computer processor.
CN201910837322.4A 2019-09-05 2019-09-05 JSON data analysis method, device, equipment and storage medium Pending CN110554877A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910837322.4A CN110554877A (en) 2019-09-05 2019-09-05 JSON data analysis method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910837322.4A CN110554877A (en) 2019-09-05 2019-09-05 JSON data analysis method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN110554877A true CN110554877A (en) 2019-12-10

Family

ID=68739196

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910837322.4A Pending CN110554877A (en) 2019-09-05 2019-09-05 JSON data analysis method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN110554877A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113676437A (en) * 2020-05-14 2021-11-19 中国移动通信集团云南有限公司 Parameter analysis method, parameter acquisition method, parameter setting method and device
WO2022127259A1 (en) * 2020-12-16 2022-06-23 北京锐安科技有限公司 Data cleaning method, apparatus and device, and storage medium
CN116610679A (en) * 2023-07-17 2023-08-18 金锐同创(北京)科技股份有限公司 json data analysis method, json data analysis device, computer equipment and computer medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9460198B1 (en) * 2012-07-26 2016-10-04 Google Inc. Process for serializing and deserializing data described by a schema
CN106933916A (en) * 2015-12-31 2017-07-07 北京国双科技有限公司 The processing method and processing device of JSON character strings
CN108268592A (en) * 2017-12-08 2018-07-10 中国平安财产保险股份有限公司 Screening technique, device, server and storage medium based on JSON data
CN109189395A (en) * 2018-08-14 2019-01-11 北京搜狐新媒体信息技术有限公司 A kind of data analysis method and device
CN109558128A (en) * 2018-10-25 2019-04-02 平安科技(深圳)有限公司 Json data analysis method, device and computer readable storage medium
CN109684607A (en) * 2017-10-19 2019-04-26 腾讯科技(深圳)有限公司 JSON data analysis method, device, computer equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9460198B1 (en) * 2012-07-26 2016-10-04 Google Inc. Process for serializing and deserializing data described by a schema
CN106933916A (en) * 2015-12-31 2017-07-07 北京国双科技有限公司 The processing method and processing device of JSON character strings
CN109684607A (en) * 2017-10-19 2019-04-26 腾讯科技(深圳)有限公司 JSON data analysis method, device, computer equipment and storage medium
CN108268592A (en) * 2017-12-08 2018-07-10 中国平安财产保险股份有限公司 Screening technique, device, server and storage medium based on JSON data
CN109189395A (en) * 2018-08-14 2019-01-11 北京搜狐新媒体信息技术有限公司 A kind of data analysis method and device
CN109558128A (en) * 2018-10-25 2019-04-02 平安科技(深圳)有限公司 Json data analysis method, device and computer readable storage medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113676437A (en) * 2020-05-14 2021-11-19 中国移动通信集团云南有限公司 Parameter analysis method, parameter acquisition method, parameter setting method and device
CN113676437B (en) * 2020-05-14 2023-08-18 中国移动通信集团云南有限公司 Parameter acquisition method, parameter setting method and device
WO2022127259A1 (en) * 2020-12-16 2022-06-23 北京锐安科技有限公司 Data cleaning method, apparatus and device, and storage medium
CN116610679A (en) * 2023-07-17 2023-08-18 金锐同创(北京)科技股份有限公司 json data analysis method, json data analysis device, computer equipment and computer medium
CN116610679B (en) * 2023-07-17 2023-10-17 金锐同创(北京)科技股份有限公司 json data analysis method, json data analysis device, computer equipment and computer medium

Similar Documents

Publication Publication Date Title
US11146286B2 (en) Compression of JavaScript object notation data using structure information
CN110554877A (en) JSON data analysis method, device, equipment and storage medium
CN112154420B (en) Automatic intelligent cloud service testing tool
CN110445860B (en) Message sending method, device, terminal equipment and storage medium
CN108038212B (en) Data interaction method, device, system, equipment and storage medium
US10754628B2 (en) Extracting web API endpoint data from source code to identify potential security threats
CN111354099B (en) Data processing method, data processing device and diagnostic equipment
CN111090417A (en) Binary file analysis method, device, equipment and medium
CN110727417A (en) Data processing method and device
CN110704714A (en) Method and device for quickly indexing data of pcap file
CN111353036B (en) Rule file generation method, device, equipment and readable storage medium
CN112084179A (en) Data processing method, device, equipment and storage medium
CN113885880A (en) Multi-rule matching method, device and equipment
CN112069305B (en) Data screening method and device and electronic equipment
CN101442539B (en) Method and apparatus for implementing field filtration
CN108243015B (en) Call bill information extraction method, call bill server and network management server
CN108494752A (en) A kind of analysis method and device of agreement
US9235639B2 (en) Filter regular expression
CN113703996B (en) Access control method, equipment and medium based on user and YANG model grouping
CN114489594A (en) Method for constructing command parser, method and device for processing command parsing
US8549494B2 (en) Techniques for parsing electronic files
CN115048913A (en) Command processing method and device and electronic equipment
CN104753934B (en) By the method that the more communication party's data stream separations of unknown protocol are Point-to-Point Data stream
CN110597517A (en) Method and device for dynamically analyzing mat file
CN110569673A (en) Data file processing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20191210

RJ01 Rejection of invention patent application after publication