CN110795464A - Method, device, terminal and storage medium for checking field of object marker data - Google Patents
Method, device, terminal and storage medium for checking field of object marker data Download PDFInfo
- Publication number
- CN110795464A CN110795464A CN201910803083.0A CN201910803083A CN110795464A CN 110795464 A CN110795464 A CN 110795464A CN 201910803083 A CN201910803083 A CN 201910803083A CN 110795464 A CN110795464 A CN 110795464A
- Authority
- CN
- China
- Prior art keywords
- data
- field
- data type
- type
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Probability & Statistics with Applications (AREA)
- Remote Sensing (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a field verification method, a field verification device, a field verification terminal and a storage medium for object marking data. The method comprises the following steps: counting the field data type of the data to be processed to obtain a statistical result; determining the target data type of each field of the data to be processed by utilizing a field data type selection strategy according to the statistical result; and synthesizing the fields into result data, wherein the data type of each field of the result data is the target data type, and the data verification of the object mark deep into the field can be performed, so that the data accuracy is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a terminal, and a storage medium for field verification of object marker data.
Background
Development personnel generally develop according to the protocol and the document communicated before in the data transmission process, sample data is provided at the upstream, and the development personnel develop according to the sample data at the downstream. During data transmission, checksum verification is required to ensure data correctness.
Generally, data type verification and confirmation are mainly carried out by depending on documents in the data transmission process, in order to guarantee the correctness of data transmission, data verification is carried out on data to judge whether the data is correct, and when data errors are found, the data are timely found and intervened. The common data check methods include parity check, CRC check, LRC check, gray code check, sum check, and xor check.
Data verification is usually performed based on the entire data content to determine whether the content of the entire data meets expectations. For unstructured data such as Object markup (JSON), sometimes a data receiver needs to perform verification more deeply to be valid, and the existing data verification method cannot accurately achieve verification of Object markup data.
Disclosure of Invention
The application provides a field verification method, a field verification device, a terminal and a storage medium for object marked data, which can carry out deep field object marked data verification and improve data accuracy.
In a first aspect, a field checking method for object marker data is provided, including:
counting the field data type of the data to be processed to obtain a statistical result;
determining the target data type of each field of the data to be processed by utilizing a field data type selection strategy according to the statistical result;
and synthesizing the fields into result data, wherein the data type of each field of the result data is the target data type.
In a second aspect, an apparatus for checking a field of object tag data is provided, which includes a statistics module, a checking module, and a generation module, wherein:
the statistical module is used for counting the field data type of the data to be processed to obtain a statistical result;
the checking module is used for determining the target data type of each field of the data to be processed by utilizing a field data type selection strategy according to the statistical result;
and the generating module is used for synthesizing the fields into result data, and the data type of each field of the result data is the target data type.
In a third aspect, an embodiment of the present application provides a terminal, including an input device and an output device, further including:
a processor adapted to implement one or more instructions; and the number of the first and second groups,
a computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and to perform a method of field verification of object marking data as described in the first aspect.
In a fourth aspect, embodiments of the present application provide a computer storage medium storing one or more instructions adapted to be loaded by a processor and to perform the steps of the first aspect and any possible implementation manner thereof.
According to the method and the device, the field data type of the data to be processed is counted to obtain a statistical result, then the field data type selection strategy is used for determining the target data type of each field of the data to be processed according to the statistical result, and the fields are combined into the result data, wherein the data type of each field of the result data is the target data type.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or the background art of the present application, the drawings required to be used in the embodiments or the background art of the present application will be described below.
Fig. 1 is a schematic flowchart of a field verification method for object marker data according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of another field checking method for object marker data according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a field checking apparatus for object tagged data according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The embodiments of the present application will be described below with reference to the drawings.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a field verification method for object tag data according to an embodiment of the present disclosure. As shown in fig. 1, the method includes:
101. and counting the field data type of the data to be processed to obtain a statistical result.
The execution subject in the embodiments of the present application may be a data processing apparatus, and in particular, the data processing apparatus may be a terminal, which may also be referred to as a terminal device, including but not limited to other portable devices such as a mobile phone, a laptop computer, or a tablet computer having a touch-sensitive surface (e.g., a touch screen display and/or a touch pad). It should also be understood that in some embodiments, the device is not a portable communication device, but is a desktop computer having a touch-sensitive surface (e.g., a touch screen display and/or touchpad).
Wherein, the field name of the data can refer to the identification of each column in the two-dimensional table taking the relational model as the data structure. The data needs to be stored to the terminal according to a certain structure and a certain organization format. The relational model is currently the widest in practice, and requires that data be stored in a two-dimensional table containing a limited number of different rows and specific relationships. It is understood that each column in the two-dimensional table is a field, the field name is used for representing the field, each field contains information of a special subject, and the field can be specified by a user and follows a certain naming rule in different systems.
For example, in the "address book" database, "name" and "contact" are attributes common to all rows in the table, so these columns are referred to as the "name" field and the "contact" field.
The definition of a data type in a data structure is the sum of a set of qualitatively identical values and a set of operations defined on this set of values. Variables are where values are stored, and the data type of the variable determines how the bits representing these values are stored in the memory of the computer. A variable may also be specified in its data type when declared.
The field data type referred to in the embodiments of the present application refers to a string data type of a field, which may include a single character or a variable type of a string.
Specifically, the to-be-processed data may be data described by a field, and the field description of the to-be-processed data may adopt a nested description, for example:
wherein, the data content describes personal information with a name of Lixianming, and the personal information sequentially comprises the following names: plum, Xiaoming, gender: female, age: 19. hometown: saving parts: shanxi, City: xian.
The field checking device of the object marking data can read the data to be processed in batch to count the field data type in the data to be processed. The data content and the data type of each field in the data to be processed can be obtained, integrated statistics is carried out, and the obtained statistical result can be in a form of a statistical table.
In an optional implementation manner, the field path, the data content, and the data type of each field in the data to be processed may be obtained, and then the occurrence number of each data type in each field is counted, so as to obtain a corresponding relationship between the field path, the data type, and the occurrence number of the data type of each field.
Wherein the data content and the data type may be represented in a key-value pair format.
The field path may indicate a path for searching or storing a field, and may be specifically represented by an identifier like a field name, and the field path may generally locate the position of the field data in the data structure.
Through the data reading and statistics, the corresponding relation between the field path and the data type of each field and the occurrence frequency of the data type can be obtained. Further optionally, a correspondence table of the field path, the data type, the occurrence number of the data type, and the field data example of each field may be generated.
Step 102 may be performed after the statistics are completed.
102. And determining the target data type of each field of the data to be processed by using a field data type selection strategy according to the statistical result.
The field data type selection strategy aims to select the data type suitable for the data of each field according to the output of field data type statistics, and the effect of field data type disambiguation is achieved. Because one field in the data may have errors in the production process, different data types may exist in the same field, and if the field is not detected, the resolution of a downstream user of the data may fail, or even the service may be down.
In order to check the data field more deeply, the field data type selection strategy can be set, and the data types of different field paths are detected to ensure that each field path has a unique and correct data type.
For example, in an alternative embodiment, the data type with the largest number of occurrences in each field may be determined as the target data type of the field. The target data type may also be determined according to a preset corresponding relationship between the field path and the data type, that is, when it is detected that the target data type is inconsistent with the preset corresponding relationship, the target data type may be corrected.
103. And synthesizing the fields into result data, wherein the data type of each field of the result data is the target data type.
After the data types of the fields are determined, the data can be synthesized according to the determined data types of the fields to obtain the result data. The data format desired by the user can be merged through the above-mentioned composition operation, and optionally, data in a JSON, JCE, or Go language structure format can be generated according to the application scenario requirements, which is not limited in the embodiment of the present application.
Object tagging (JSON) is a lightweight data exchange format that stores and represents data in a text format that is completely independent of programming languages. The compact and clear hierarchy makes JSON an ideal data exchange language. The network transmission method is easy to read and write by people, is easy to analyze and generate by machines, and effectively improves the network transmission efficiency.
Any supported type can be represented by JSON, such as strings, numbers, objects, arrays, and the like. But objects and arrays are two types that are more specific and commonly used, where objects are represented as key-value pairs, data is separated by commas, curly brackets hold objects, and square brackets hold arrays. In this embodiment of the present application, an example of the foregoing nesting description of the to-be-processed data is JSON representation, which is not described herein again.
The JCE protocol related in the embodiment of the application is a protocol coding library self-developed by Tencent corporation, the JCE Struct is a data structure type in the JCE protocol, and the Go Struct is a data structure type in Golang language.
Generally, data type verification and confirmation are mainly carried out by depending on documents in the data transmission process, in order to guarantee the correctness of data transmission, data verification is carried out on data to judge whether the data is correct, and when data errors are found, the data are timely found and intervened. The common data check methods include parity check, CRC check, LRC check, gray code check, sum check, and xor check. However, these verification techniques are based on the overall data content to verify whether the overall data content meets expectations. For unstructured data such as JSON, sometimes a data receiver needs to perform verification more deeply to be effective, and the existing data verification method cannot meet the requirement.
According to the embodiment of the application, the field data type of the data to be processed is counted to obtain the statistical result, then the field data type selection strategy is used for determining the target data type of each field of the data to be processed according to the statistical result, and the fields are combined into the result data, wherein the data type of each field of the result data is the target data type.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating another method for checking a field of object tag data according to an embodiment of the present application, where the method includes:
201. and acquiring a field path, data content and a data type of each field in the data to be processed, wherein the data content and the data type are expressed in a key-value pair format.
The Key-value database is a database for storing data by Key-value pairs, wherein the Key is a Key and the value is a value, each Key corresponds to a unique value, and the value can be taken according to the Key, so that the Key-value database has extremely high concurrent read-write performance.
Nested descriptions may be employed for field descriptions of data to be processed. Reference may be made to the nesting description in step 101 of the embodiment shown in fig. 1 for example, and details are not repeated here.
202. And counting the occurrence times of each data type in each field to obtain the corresponding relation between the field path and the data type of each field and the occurrence times of the data types.
Specifically, the step may generate a corresponding relationship between the field path and the data type and the occurrence number of the data type, and optionally, a data example of the field may also be given. By way of example, the results obtained may be as shown in table 1 below, where table 1 is a field data type statistics table, which may correspond to the field statistics of the nested description in the embodiment shown in fig. 1. The field path and type describe each field path and data type, and are divided by semicolons, if the same field has a plurality of data types, a multi-line record can be generated, the field data type statistics describes the occurrence times of the data type of the corresponding field, and the field data example is the data example of the field.
TABLE 1
Step 201 and step 202 may also refer to the related detailed description in step 101 in the embodiment shown in fig. 1, and are not described herein again.
203. And acquiring the field path and the data type of the first field in each field.
After the statistics are carried out, the appropriate data type of the data of each field can be selected according to the output of the field data type statistics and a certain strategy. The first field may be any field in the data to be processed, and the fields in the data to be processed may be sequentially read and processed.
204. If the first field comprises at least two data types, detecting whether combinable data types exist in the at least two data types.
In this embodiment of the present application, only one field, that is, the first field, is taken as an example for description, and if the data types in the first field are at least two, it may be detected whether there are combinable data types in the at least two data types, for example, a float data type and an int data type may be combinable into a float. If the first field includes a data type, the data type of the field can be continuously detected for the next line of data without executing merging processing.
205. The mergeable data types described above are merged.
If there are combinable data types in the at least two data types, the combinable data types can be combined, and if the combinable data types are not combinable, the data type detection of the field can be continuously carried out on the data of the next row.
Step 206 may be executed after the steps 204 and 205 are executed for each field.
206. And determining the data type with the largest occurrence frequency in each field as the target data type of the field.
After the detection, statistical information of each data type can be obtained, including the occurrence number of each data type, and the data type with the largest occurrence number can be determined as the target data type of the field.
For example, reference may be made to a field data type disambiguation algorithm shown in table 2, which corresponds to the method described in the above step 204-step 206, and in particular, the type disambiguation corresponding to the field data type statistics shown in table 1 may be implemented by an algorithm similar to table 2:
TABLE 2
And (3) outputting: field path, data type after disambiguation, number of occurrences, sample data
It can be seen that the input for field data type disambiguation may include the output of the field data type statistical stage, the statistics of the occurrence times of each output data type may be the statistical result after disambiguation, and the sample data of the corresponding field may be displayed, that is, the field data example may clearly and definitely display the data content, which is convenient for query, audit, improvement, and the like. The appropriate data type of each field can be determined by the above method, and step 207 is performed.
207. And synthesizing the result data of the fields according to the field paths.
Step 207 may refer to the detailed description in step 103 in the embodiment shown in fig. 1, and is not described herein again.
208. And generating a corresponding relation table of the field path, the data type, the occurrence number of the data type and the field data example of each field.
The data to be processed in the embodiment of the present application may be Point of interest (POI) in a geographic information system, and in the geographic information system, one POI may be one house, one shop, one mailbox, one bus station, and the like. For the map industry, a point of interest often contains dozens or even hundreds of fields, and it is very difficult to find a piece of data covering all the fields. Meanwhile, the same field may have different data types due to errors in the production process, and if the field is not detected, the analysis of a downstream data user may fail, or even the service may be down.
By the field verification method of the object marker data in the embodiment of the application, deep field data field verification can be performed on JSON data in batches, and an accurate data type is selected for disambiguation of the data type under the condition that multiple data types exist in the same field. Meanwhile, according to the embodiment of the application, a full-size data template covering all fields can be generated according to the JSON data in batches and used by a downstream user.
After the step 208, a data template of the point of interest data may be generated, where the data template includes the point of interest data identifier, field paths of all fields in the correspondence table, a data type, and the field data example. In an optional implementation manner, different POI may allocate different POI identifiers to distinguish, and the data field check processing may be performed on the data of each POI, so that the data accuracy is improved, and meanwhile, a data template corresponding to the POI may be obtained, which is convenient for a downstream user to use.
The field verification method of the object marking data in the embodiment of the application can be applied to data protocol customization of data upstream and downstream in the data circulation process and data verification of a data receiver according to a protocol.
In the embodiment of the present application, by obtaining a field path, data content, and a data type of each field in the to-be-processed data, where the data content and the data type are represented in a key-value pair format, counting the occurrence frequency of each data type in each field, obtaining a corresponding relationship between the field path and the data type of each field and the occurrence frequency of the data type, obtaining a field path and a data type of a first field in each field, if the first field includes at least two data types, detecting whether a combinable data type exists in the at least two data types, combining the combinable data types, then determining the data type with the largest occurrence frequency in each field as a target data type of the field, and combining the fields into the result data according to the field path, compared with a general verification mode based on the whole data content, the method can carry out deep field object marking data verification and improve the data accuracy; and generating a corresponding relation table of the field path, the data type, the occurrence times of the data type and the field data example of each field to obtain a processing template covering a plurality of fields for downstream development and use according to the sample data.
Based on the above description of the field verification method embodiment of the object tagged data, the embodiment of the present application further discloses a field verification apparatus of the object tagged data, which can execute the method shown in fig. 1 and/or fig. 2.
Referring to fig. 3, the field verification apparatus 300 for object marker data includes: a statistics module 310, a verification module 320, and a generation module 330, wherein:
the statistical module 310 is configured to count field data types of the data to be processed to obtain a statistical result;
the checking module 320 is configured to determine a target data type of each field of the to-be-processed data according to the statistical result by using a field data type selection policy;
the generating module 330 is configured to synthesize the fields into result data, where a data type of each field of the result data is the target data type.
In a possible implementation manner, the statistical module 310 is specifically configured to:
acquiring a field path, data content and a data type of each field in the data to be processed, wherein the data content and the data type are represented in a key-value pair format;
and counting the occurrence times of each data type in each field to obtain the corresponding relation among the field path of each field, the data type and the occurrence times of the data type.
In a possible implementation manner, the verification module 320 is specifically configured to:
and determining the data type with the most occurrence times in each field as the target data type of the field.
In one possible implementation manner, the check module 320 includes a merging unit 321, where:
the checking module 320 is specifically configured to determine the data type with the largest occurrence number in each field, and obtain a field path and a data type of a first field in each field before the data type is used as a target data type of the field;
the merging unit 321 is configured to:
if the first field comprises at least two data types, detecting whether combinable data types exist in the at least two data types or not;
if so, merging the mergeable data types.
In a possible implementation manner, the generating module 330 is specifically configured to:
and synthesizing the result data of the fields according to the field paths.
In a possible implementation manner, the generating module 330 is further configured to generate a table of correspondence between the field path, the data type, the occurrence number of the data type, and a field data example of each field after the obtaining of the correspondence between the field path, the data type, and the occurrence number of the data type of each field.
Optionally, the data to be processed is interest point data in a geographic information system; the generating module 330 is further configured to:
and generating a data template of the point of interest data, wherein the data template comprises the point of interest data identification, field paths of all fields in the corresponding relation table, a data type and the field data example.
According to an embodiment of the present application, the steps involved in the methods shown in fig. 1 and fig. 2 may be performed by the modules in the field verification apparatus 300 of the object marker data shown in fig. 3.
According to another embodiment of the present application, the modules in the field verification apparatus 300 for object marking data shown in fig. 3 may be respectively or entirely combined into one or several additional modules to form the additional modules, or some module(s) may be further split into multiple functionally smaller modules to form the additional modules, which may achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The modules are divided based on logic functions, and in practical application, the functions of one module can be realized by a plurality of modules, or the functions of a plurality of modules can be realized by one module. In other embodiments of the present application, the terminal-based terminal may also include other modules, and in practical applications, these functions may also be implemented by the assistance of other modules, and may be implemented by cooperation of a plurality of modules.
According to another embodiment of the present application, the field verification apparatus 300 of the object marker data as shown in fig. 3 may be constructed by running a computer program (including program code) capable of executing the steps involved in the corresponding method as shown in fig. 1 and/or fig. 2 on a general-purpose computing device, such as a computer, including a processing element and a storage element, such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and the like, and a field verification method of the object marker data of the embodiment of the present application may be implemented. The computer program may be recorded on a computer-readable recording medium, for example, and loaded into and executed by the computing apparatus via the computer-readable recording medium.
The field verification apparatus 300 for object tagged data in the embodiment of the present application may obtain a statistical result by counting field data types of data to be processed, determine a target data type of each field of the data to be processed by using a field data type selection policy according to the statistical result, and synthesize the fields into result data, where the data type of each field of the result data is the target data type, and compared with a general verification method based on the whole data content, the field verification apparatus may perform object tagged data verification deep into a field, and improve data accuracy.
Based on the description of the method embodiment and the device embodiment, the embodiment of the application also provides a terminal. Referring to fig. 4, the terminal 400 includes at least a processor 401, an input device 402, an output device 403, and a computer storage medium 404. The processor 401, input device 402, output device 403, and computer storage medium 404 within the terminal may be connected by a bus or other means.
A computer storage medium 404 may be stored in the memory of the terminal, said computer storage medium 404 being adapted to store a computer program comprising program instructions, said processor 401 being adapted to execute said program instructions stored by said computer storage medium 404. The processor 401 (or CPU) is a computing core and a control core of the terminal, and is adapted to implement one or more instructions, and in particular, is adapted to load and execute the one or more instructions so as to implement a corresponding method flow or a corresponding function; in one embodiment, the processor 401 described above in this embodiment of the present application may be configured to perform a series of processes, including: and acquiring the generated long link to be processed, coding the long link to be processed according to a link coding format to acquire binary coding information, converting the coding information into a character string link of a target system, adding a first identifier corresponding to the first type link into the character string link to acquire a target short link, and the like.
An embodiment of the present application further provides a computer storage medium (Memory), where the computer storage medium is a Memory device in a terminal and is used to store programs and data. It is understood that the computer storage medium herein may include a built-in storage medium in the terminal, and may also include an extended storage medium supported by the terminal. The computer storage medium provides a storage space that stores an operating system of the terminal. Also stored in this memory space are one or more instructions, which may be one or more computer programs (including program code), suitable for loading and execution by processor 401. Note that the computer storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory; and optionally at least one computer storage medium located remotely from the processor.
In one embodiment, one or more instructions stored in a computer storage medium may be loaded and executed by processor 401 to perform the corresponding steps of the method in the above-described embodiments; in particular implementations, one or more instructions in the computer storage medium may be loaded and executed by processor 401 to perform any of the steps in fig. 1 and 2.
The terminal 400 of the embodiment of the application can count the field data type of the data to be processed to obtain a statistical result, then, according to the statistical result, the field data type selection strategy is used for determining the target data type of each field of the data to be processed, and the fields are combined into the result data, wherein the data type of each field of the result data is the target data type.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the division of the module is only one logical division, and other divisions may be possible in actual implementation, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not performed. The shown or discussed mutual coupling, direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some interfaces, and may be in an electrical, mechanical or other form.
Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are wholly or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)), or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a read-only memory (ROM), or a Random Access Memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape, a magnetic disk, or an optical medium, such as a Digital Versatile Disk (DVD), or a semiconductor medium, such as a Solid State Disk (SSD).
Claims (10)
1. A field checking method for object marking data is characterized by comprising the following steps:
counting the field data type of the data to be processed to obtain a statistical result;
determining the target data type of each field of the data to be processed by utilizing a field data type selection strategy according to the statistical result;
and synthesizing the fields into result data, wherein the data type of each field of the result data is the target data type.
2. The method according to claim 1, wherein the counting field data types of the data to be processed, and obtaining the statistical result comprises:
acquiring a field path, data content and a data type of each field in the data to be processed, wherein the data content and the data type are represented in a key-value pair format;
and counting the occurrence times of each data type in each field to obtain the corresponding relation among the field path of each field, the data type and the occurrence times of the data type.
3. The method according to claim 2, wherein the determining, according to the statistical result, the target data type of each field of the data to be processed by using a field data type selection policy comprises:
and determining the data type with the most occurrence times in each field as the target data type of the field.
4. The method according to claim 3, wherein before determining the data type with the largest number of occurrences in each field as the target data type for the field, the method comprises:
acquiring a field path and a data type of a first field in each field;
if the first field comprises at least two data types, detecting whether combinable data types exist in the at least two data types or not;
if so, merging the mergeable data types.
5. The method of claim 4, wherein the synthesizing the respective fields into result data comprises:
and synthesizing the result data of the fields according to the field paths.
6. The method according to any one of claims 2-5, wherein after obtaining the correspondence between the field path and the data type of each field and the occurrence number of the data type, the method further comprises:
and generating a corresponding relation table of the field path, the data type, the occurrence times of the data type and the field data example of each field.
7. The method according to claim 6, wherein the data to be processed is point-of-interest data in a geographic information system, and after generating the correspondence table of the field path, the data type, the occurrence number of the data type and the field data example of each field, the method further comprises:
and generating a data template of the point of interest data, wherein the data template comprises the point of interest data identification, field paths of all fields in the corresponding relation table, a data type and the field data example.
8. An apparatus for checking a field of object tag data, comprising: statistics module, check module and generation module, wherein:
the statistical module is used for counting the field data type of the data to be processed to obtain a statistical result;
the checking module is used for determining the target data type of each field of the data to be processed by utilizing a field data type selection strategy according to the statistical result;
and the generating module is used for synthesizing the fields into result data, and the data type of each field of the result data is the target data type.
9. A terminal comprising an input device and an output device, further comprising:
a processor adapted to implement one or more instructions; and the number of the first and second groups,
a computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and to perform a field verification method of object marking data according to any of claims 1-7.
10. A computer-readable storage medium having stored thereon one or more instructions adapted to be loaded by a processor and to perform a method of field verification of object marking data according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910803083.0A CN110795464B (en) | 2019-08-28 | 2019-08-28 | Method, device, terminal and storage medium for checking field of object marker data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910803083.0A CN110795464B (en) | 2019-08-28 | 2019-08-28 | Method, device, terminal and storage medium for checking field of object marker data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110795464A true CN110795464A (en) | 2020-02-14 |
| CN110795464B CN110795464B (en) | 2022-03-04 |
Family
ID=69427482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910803083.0A Active CN110795464B (en) | 2019-08-28 | 2019-08-28 | Method, device, terminal and storage medium for checking field of object marker data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110795464B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112333186A (en) * | 2020-11-03 | 2021-02-05 | 平安普惠企业管理有限公司 | Data communication method, device, equipment and storage medium |
| CN113642309A (en) * | 2020-04-27 | 2021-11-12 | 北京国双科技有限公司 | Data comparison method and related device |
| CN114547033A (en) * | 2022-02-22 | 2022-05-27 | 苏州浪潮智能科技有限公司 | Method, system, equipment and storage medium for managing key product data |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070255847A1 (en) * | 2006-04-27 | 2007-11-01 | Finisar Corporation | Systems and methods for preparing network data for analysis |
| US20120107779A1 (en) * | 2010-10-29 | 2012-05-03 | Halton P Karl | Object-field-based mathematics system |
| US8447755B1 (en) * | 2009-09-29 | 2013-05-21 | Aquire Solutions, Inc. | Systems and methods of analyzing changes and data between hierarchies |
| CN104731976A (en) * | 2015-04-14 | 2015-06-24 | 海量云图（北京）数据技术有限公司 | Method for finding and sorting private data in data table |
| US20160188747A1 (en) * | 2014-12-30 | 2016-06-30 | Raymond Cypher | Computer Implemented Systems and Methods for Processing Semi-Structured Documents |
| US9760571B1 (en) * | 2013-07-23 | 2017-09-12 | jSonar Inc. | Tabular DB interface for unstructured data |
| CN108256074A (en) * | 2018-01-17 | 2018-07-06 | 链家网（北京）科技有限公司 | Method, apparatus, electronic equipment and the storage medium of checking treatment |
| CN108415994A (en) * | 2018-02-13 | 2018-08-17 | 北京天元创新科技有限公司 | A kind of Network Management System report form generation method, device and equipment |
-
2019
- 2019-08-28 CN CN201910803083.0A patent/CN110795464B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070255847A1 (en) * | 2006-04-27 | 2007-11-01 | Finisar Corporation | Systems and methods for preparing network data for analysis |
| US8447755B1 (en) * | 2009-09-29 | 2013-05-21 | Aquire Solutions, Inc. | Systems and methods of analyzing changes and data between hierarchies |
| US20120107779A1 (en) * | 2010-10-29 | 2012-05-03 | Halton P Karl | Object-field-based mathematics system |
| US9760571B1 (en) * | 2013-07-23 | 2017-09-12 | jSonar Inc. | Tabular DB interface for unstructured data |
| US20160188747A1 (en) * | 2014-12-30 | 2016-06-30 | Raymond Cypher | Computer Implemented Systems and Methods for Processing Semi-Structured Documents |
| CN104731976A (en) * | 2015-04-14 | 2015-06-24 | 海量云图（北京）数据技术有限公司 | Method for finding and sorting private data in data table |
| CN108256074A (en) * | 2018-01-17 | 2018-07-06 | 链家网（北京）科技有限公司 | Method, apparatus, electronic equipment and the storage medium of checking treatment |
| CN108415994A (en) * | 2018-02-13 | 2018-08-17 | 北京天元创新科技有限公司 | A kind of Network Management System report form generation method, device and equipment |
Non-Patent Citations (4)
| Title |
|---|
| ZJG_JAVA: "【JAVA】使用JSON Schema校验JSON数据是否合规", 《HTTPS://BLOG.CSDN.NET/ZJG379569986/ARTICLE/DETAILS/84634803?TDSOURCETAG=S_PCTIM_AIOMSG》 * |
| 曹家锋: "数据库信息模型定义及测试系统的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 熊小华等: "数据编辑界面中数据自动校验的设计与实现", 《江西师范大学学报(自然科学版)》 * |
| 胡志伟: "基于Hadoop的分布式数据检测系统的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113642309A (en) * | 2020-04-27 | 2021-11-12 | 北京国双科技有限公司 | Data comparison method and related device |
| CN112333186A (en) * | 2020-11-03 | 2021-02-05 | 平安普惠企业管理有限公司 | Data communication method, device, equipment and storage medium |
| CN112333186B (en) * | 2020-11-03 | 2022-11-29 | 平安普惠企业管理有限公司 | Data communication method, device, equipment and storage medium |
| CN114547033A (en) * | 2022-02-22 | 2022-05-27 | 苏州浪潮智能科技有限公司 | Method, system, equipment and storage medium for managing key product data |
| CN114547033B (en) * | 2022-02-22 | 2024-01-16 | 苏州浪潮智能科技有限公司 | Method, system, equipment and storage medium for managing key product data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110795464B (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110795464B (en) | Method, device, terminal and storage medium for checking field of object marker data | |
| CN108334609B (en) | Method, device, equipment and storage medium for realizing JSON format data access in Oracle | |
| CN109271611B (en) | Data verification method and device and electronic equipment | |
| CN111209736A (en) | Text file analysis method and device, computer equipment and storage medium | |
| CN110688111A (en) | Configuration method, device, server and storage medium of business process | |
| CN111159040A (en) | Test data generation method, device, equipment and storage medium | |
| CN117493309A (en) | Standard model generation method, device, equipment and storage medium | |
| CN109324838B (en) | Execution method and execution device of single chip microcomputer program and terminal | |
| CN112463785B (en) | Data quality monitoring method and device, electronic equipment and storage medium | |
| CN115309554A (en) | Debugging method, debugging system and computing device for application running based on compatible layer | |
| CN111143398B (en) | Extra-large set query method and device based on extended SQL function | |
| CN112631852A (en) | Macro checking method, macro checking device, electronic equipment and computer readable storage medium | |
| CN112307029A (en) | Bill data storage and bill generation method, device, server and storage medium | |
| CN111125147B (en) | Extra-large set analysis method and device based on extended pre-calculation model and SQL function | |
| CN109284268A (en) | A kind of method, system and the electronic equipment of fast resolving log | |
| CN110704690B (en) | Data processing method, device and storage medium | |
| CN113986305B (en) | B/S model upgrade detection method, device, equipment and storage medium | |
| CN111125264B (en) | Extra-large set analysis method and device based on extended OLAP model | |
| CN111680112B (en) | Data analysis method and device | |
| CN109857816B (en) | Test sample selection method and device, storage medium and electronic equipment | |
| CN117555955A (en) | Data conversion method, data conversion device, computer device, and storage medium | |
| CN112860784A (en) | Blood margin analysis method, blood margin analysis device, electronic equipment and computer-readable storage medium | |
| CN114443634A (en) | Data quality checking method, device, equipment and storage medium | |
| CN115238664A (en) | Method, device, equipment and medium for batch import of form data | |
| CN115827677A (en) | Database operation method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40022037 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |