WO2021103409A1

WO2021103409A1 - Data generation method and apparatus, electronic device and storage medium

Info

Publication number: WO2021103409A1
Application number: PCT/CN2020/086799
Authority: WO
Inventors: 吴名先
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2019-11-26
Filing date: 2020-04-24
Publication date: 2021-06-03
Also published as: CN111159386A

Abstract

Provided by the present application are a data generation method and apparatus, an electronic device and a storage medium. The data generation method is capable of: acquiring a trigger interface of the data generation instruction when a data generation instruction is received, identifying all text on the trigger interface, extracting property data of data to be generated, encapsulating the property data to obtain a target character string, parsing the target character string to obtain a target parameter, further parsing the target parameter to obtain a target type, retrieving from an HBASE library a row key generation policy corresponding to the target type so as to generate a target row key of the target parameter in the HBASE, acquiring a target column name from the target parameter, generating an intermediate data file according to the target column name and the target row key, and writing the intermediate data file in the HBASE library to obtain the data to be generated. The method can process data on the basis of an HBASE library, and generate a large amount of data, and can also improve the data generation efficiency.

Description

Data generation method, device, electronic equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on November 26, 2019, the application number is 201911174761.8, and the invention title is "a data generation method, device, electronic equipment, and storage medium". The entire content of the application is approved The reference is incorporated in this application.

Technical field

This application relates to the technical field of data processing in artificial intelligence, and in particular to a data generation method, device, electronic equipment, and storage medium.

Background technique

In the big data test project, a large amount of test data is often required. However, the inventor realized that in the existing technical solutions, the amount of data generated based on the traditional database is not enough to support the big data test project. A large number of algorithms are run at the bottom of the traditional database, resulting in low efficiency of data generation. Therefore, how to efficiently generate massive amounts of data has become an urgent problem to be solved.

Summary of the invention

In view of the above, it is necessary to provide a data generation method, device, electronic equipment, and storage medium that can generate a large amount of data based on the HBASE library, and can also improve the efficiency of data generation.

A data generation method, the method includes:

When receiving the data generation instruction, obtain the trigger interface of the data generation instruction;

Using text recognition technology to recognize all text on the trigger interface;

Extract at least one keyword from all the texts to obtain the attribute data of the data to be generated;

Use JSON technology to encapsulate the attribute data to obtain the target string;

Analyze the target string using JSON technology to obtain target parameters;

Analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter;

Retrieve the row key generation strategy corresponding to the target type from the HBASE library;

Generating the target row key of the target parameter in the HBASE library based on the row key generation strategy;

Obtain the target column name from the target parameter;

Generate an intermediate data file according to the target column name and the target row key;

A calculation engine is used to write the intermediate data file into the HBASE library to obtain the data to be generated.

The present application provides a data generation device, the device includes:

The acquiring unit is configured to acquire the trigger interface of the data generating instruction when the data generating instruction is received;

The recognition unit is used to recognize all the text on the trigger interface by using text recognition technology;

The extraction unit is used to extract at least one keyword from all the texts to obtain the attribute data of the data to be generated;

The encapsulation unit is used to encapsulate the attribute data using JSON technology to obtain the target string;

The parsing unit is used to parse the target string using JSON technology to obtain target parameters;

The parsing unit is further configured to analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter;

The retrieval unit is used to retrieve the row key generation strategy corresponding to the target type from the HBASE library;

A generating unit, configured to generate a target row key of the target parameter in the HBASE library based on the row key generation strategy;

The obtaining unit is further configured to obtain the target column name from the target parameter;

The generating unit is further configured to generate an intermediate data file according to the target column name and the target row key;

The writing unit is configured to use a calculation engine to write the intermediate data file into the HBASE library to obtain the data to be generated.

This application provides an electronic device, which includes:

Memory, storing at least one instruction; and

The processor executes the instructions stored in the memory to implement the data generation method.

A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the data generation method.

It can be seen from the above technical solutions that when a data generation instruction is received, this application can obtain the trigger interface of the data generation instruction, use text recognition technology to recognize all the text on the trigger interface, and extract all the text from the trigger interface. At least one keyword is used to obtain the attribute data of the data to be generated, the attribute data is encapsulated using JSON technology to obtain the target string, the target string is parsed using JSON technology to obtain the target parameters, and the shallow semantic analysis method is used to parse the data. The target parameter, the target type of the target parameter is obtained, the row key generation strategy corresponding to the target type is retrieved from the HBASE library, and the target parameter in the HBASE library is generated based on the row key generation strategy Target row key, obtain the target column name from the target parameter, generate an intermediate data file according to the target column name and the target row key, and write the intermediate data file into the HBASE library using a calculation engine, Obtaining the data to be generated can not only generate a large amount of data based on the HBASE library, but also improve the efficiency of data generation.

Description of the drawings

Fig. 1 is a flowchart of a preferred embodiment of the data generation method of the present application.

Fig. 2 is a functional module diagram of a preferred embodiment of the data generating device of the present application.

FIG. 3 is a schematic structural diagram of an electronic device implementing a preferred embodiment of the data generation method of the present application.

Detailed ways

As shown in Fig. 1, it is a flowchart of a preferred embodiment of the data generation method of the present application. According to different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

The data generation method is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. Its hardware includes but is not limited to Microprocessor, Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), Digital Processor (Digital Signal Processor, DSP), embedded equipment, etc.

The electronic device may be any electronic product that can interact with a user with a human machine, for example, a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television ( Internet Protocol Television, IPTV), smart wearable devices, etc.

The electronic device may also include a network device and/or user equipment. Wherein, the network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

The network where the electronic device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), etc.

S10: When a data generation instruction is received, a trigger interface of the data generation instruction is acquired.

In at least one embodiment of the present application, the data generation instruction may be triggered by the user, or it may be automatically triggered when certain conditions are met, which is not limited in the present application.

Wherein, the meeting certain conditions includes, but is not limited to: meeting a preset time and so on.

The preset time may include a certain time point, or include a time period, etc., for example: the preset time may be 7 o'clock in the morning every day.

In at least one embodiment of the present application, the trigger interface is composed of an input text box and a submit button, and is mainly used to collect information. According to user requirements, the input text box may have various forms, wherein the content of the input text box may include, but is not limited to: text fields, characters, and the like.

S11: Recognize all the text on the trigger interface by using text recognition technology.

In at least one embodiment of the present application, all the characters include, but are not limited to: Chinese, characters, numbers, etc.

In at least one embodiment of the present application, the electronic device uses text recognition technology to recognize the trigger interface to obtain all the text on the trigger interface.

Wherein, the trigger interface includes an input text box and a submit button.

S12: Extract at least one keyword from all the texts to obtain attribute data of the data to be generated.

In at least one embodiment of the present application, the attribute data may include, but is not limited to: the number of rows, table names, column names, file names, and so on.

In at least one embodiment of the present application, the electronic device extracts at least one keyword from all the words, and obtains the attribute data of the data to be generated includes:

The electronic device uses a Chinese keyword extraction algorithm based on high-dimensional clustering technology to extract at least one keyword from all the texts to obtain attribute data of the data to be generated.

Specifically, the electronic device performs fast word segmentation on all the characters according to a pre-configured target dictionary to obtain the first word segmentation, calculates the target word frequency of the first word segmentation, and determines the first word segmentation whose target word frequency is greater than the preset word frequency Is an initial keyword, and the electronic device trims the initial keyword according to a preset small dictionary to obtain a final keyword, and determines the final keyword as the attribute data of the data to be generated.

Wherein, the target dictionary may include common keywords.

Further, the small dictionary may include, but is not limited to function words, stop words, etc.

By trimming the initial keywords, the attribute data of the data to be generated can be determined accurately and quickly.

S13. Use JSON (JavaScript Object Notation, JS object notation) technology to encapsulate the attribute data to obtain a target string.

In at least one embodiment of the present application, the target string is encapsulated by the attribute data, and its expression form is {tag: attribute data}.

In at least one embodiment of the present application, the electronic device adopts JSON technology to encapsulate the attribute data to obtain the target string including:

The electronic device retrieves the target tag corresponding to each attribute data in the attribute data from the HBASE library, and further, the electronic device uses JSON technology to encapsulate each attribute data and the target tag corresponding to each attribute data , Get the target string.

Among them, the JSON technology is a lightweight data exchange format. The JSON technology uses a text format completely independent of programming languages to store and represent data, which is easy to read and write, and is also easy to parse and generate by machines, and effectively improves network transmission efficiency.

Further, the HBASE library is a distributed, column-oriented open source database.

For example: attribute data 1 is a name, attribute data 2 is a gender, the electronic device retrieves the target tag corresponding to the name from the HBASE library as A, retrieves the target tag corresponding to the gender as B, and encapsulates the name and A , Obtain the target string of the attribute data 1 as {A: name}, encapsulate gender and B, and obtain the target string of the attribute data 2 as {B: gender}

Through the foregoing implementation manner, the attribute data is encapsulated into the target character string, so that the attribute data can be transmitted to the back-end server for data processing along with the target character string.

S14: Analyze the target string using JSON technology to obtain target parameters.

In at least one embodiment of the present application, the target parameter includes, but is not limited to: table name, row number, column name, file name, and so on.

In at least one embodiment of the present application, before using JSON technology to parse the target string to obtain target parameters, the method further includes:

The electronic device uses ajax (Asynchronous Javascript And XML) technology to transmit the target character string to the back-end server of the electronic device.

Among them, the ajax technology is a technology for creating better, faster and more interactive Web applications.

Specifically, the electronic device adds the target character string to the post request, and uses the ajax technology to transmit the post request to the back-end server.

Through the foregoing implementation manners, the ajax technology is used to receive the target character string from the front-end page of the electronic device, and data can be exchanged with the back-end server without reloading the front-end page, thereby improving efficiency.

In at least one embodiment of the present application, the use of JSON technology to parse the target string to obtain target parameters includes:

The electronic device acquires all target tags in the target character string, and further, the electronic device extracts data corresponding to each target tag in all target tags as the target parameter.

For example: the target character string of the attribute data 1 is {A: name}, the target character string of the attribute data 2 is {B: gender}, and the electronic device obtains the target tag A, The target tag B of the attribute data 2 extracts the name in the target string {A: name} according to the target tag A as a target parameter, and extracts the target character according to the target tag B The gender in the string {B: gender} is used as the target parameter.

Through the foregoing implementation manners, the JSON technology can be used to quickly and accurately deserialize the target string into the target parameters required by the backend.

S15: Analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter.

In at least one embodiment of the present application, the electronic device adopts a shallow semantic analysis method to analyze the semantic relationship between the target parameters to obtain the application scenario of the target parameter. Further, the electronic device is based on The application scenario determines the target type of the target parameter.

Wherein, the target type may include, but is not limited to: transaction data type, statistical data type, general data type, and so on.

S16: Retrieve a row key generation strategy corresponding to the target type from the HBASE library.

In at least one embodiment of the present application, the row key generation strategy includes, but is not limited to, one or a combination of the following:

(1) When the target type is a transaction data type, the row key generation strategy is to increment by a time stamp.

(2) When the target type is a general data type, the row key generation strategy is self-increment in sequence.

S17: Generate a target row key of the target parameter in the HBASE library based on the row key generation strategy.

In at least one embodiment of the present application, the electronic device obtains the number of rows from the target parameter, and further, the electronic device imports the row key generation strategy into the HBASE library, according to the number of rows And the row key generation strategy to generate the target row key.

Wherein, the number of the target row keys is equal to the number of rows.

Through the foregoing implementation manner, the target row key can be quickly generated without manual operation.

S18: Obtain a target column name from the target parameter.

In at least one embodiment of the present application, the target column name refers to the name of each column of data, and the target column name is stored in the target parameter obtained by JSON technology analysis. Further, the target column name may be Including, but not limited to: name, gender, ID number, date of birth, etc.

In at least one embodiment of the present application, the obtaining the target list name from the target parameter includes:

The electronic device matches the target parameter with the column name stored in the HBASE library, and further, the electronic device confirms the target parameter matching the column name as the target column name.

Through the foregoing implementation manners, the target column name can be obtained, which facilitates subsequent generation of an intermediate data file based on the target column name.

S19: Generate an intermediate data file according to the target column name and the target row key.

In at least one embodiment of the present application, the intermediate data file is a set of sentences generated in the HBASE library, and at least one piece of data with a fixed format can be generated through the intermediate data file.

In at least one embodiment of the present application, the generating an intermediate data file according to the target column name and the target row key includes:

The electronic device retrieves the target data corresponding to the target column name from the HBASE library, determines the target location according to the target column name and the target row key, and writes the target data to the The target location, the intermediate data file is obtained.

Wherein, the writing the target data into the target location to obtain the intermediate data file specifically includes:

The electronic device writes the target data into the target location through the statement put<table>,<rowkey>,<family:column>,<value>, repeats the above steps, until the target location is written in all For the target data, all the statements executed above are assembled to generate the intermediate data file.

The amount of data stored in the HBASE library is large, but the data is messy. Through this embodiment, the target data corresponding to the target column name can be retrieved from the HBASE library according to user needs, and the intermediate The data file, in view of this, since the data is written corresponding to the target location, the data generated by the intermediate data file will all have a fixed format.

In at least one embodiment of the present application, after the intermediate data file is generated, the method further includes:

The electronic device sends the intermediate data file to the intermediate data layer, and uses the intermediate data layer to save the intermediate data file.

Wherein, the intermediate data layer is a directory dedicated to storing intermediate files in the Linux system.

Through the foregoing implementation manners, the intermediate data file is saved, which can avoid the loss of the intermediate data file due to errors in the process of generating the data to be generated, thereby affecting the efficiency of data generation.

S20: Use a calculation engine to write the intermediate data file into the HBASE library to obtain the data to be generated.

In at least one embodiment of the present application, the data to be generated is a set of data with a fixed format and generated based on the HBASE library.

In at least one embodiment of the present application, the using a computing engine to write the intermediate data file into the HBASE library to obtain the data to be generated includes:

According to the file size of the intermediate data file, the electronic device divides the intermediate data file into at least one data block, calculates the number of the at least one data block, and obtains the target number. Further, the electronic device An HBASE table is created in the HBASE library, the HBASE table is partitioned using the target number as the number of partitions, and the calculation engine is used to write the at least one data block into the partitioned HBASE table to obtain the Data to be generated.

Through the foregoing implementation manner, due to the use of the calculation engine, the intermediate data file can be quickly written into the HBASE library, which improves the data generation rate.

In at least one embodiment of the present application, after the data to be generated is obtained, the method further includes:

The electronic device obtains the table name and generation time of the data to be generated, and according to the table name and the generation time, the electronic device generates prompt information, and further, the electronic device sends the prompt information to a designated The contact's terminal device.

The prompt information may include, but is not limited to: the name of the table of the data to be generated, the number of rows of the data to be generated, the generation time of the data to be generated, and the like.

Further, the designated contact person may be the trigger of the data generation instruction.

Through the foregoing implementation manners, after the data to be generated is generated, the designated contact person can be reminded to check the data.

The electronic device uses symmetric encryption technology to encrypt the data to be generated to obtain a ciphertext, and when it is detected that the ciphertext is successfully decrypted, it displays the data to be generated.

Through the foregoing implementation manners, it is possible to prevent the data to be generated from being tampered with, and to improve data security.

As shown in Figure 2, it is a functional module diagram of a preferred embodiment of the data generating device of the present application. The data generation device 11 includes an acquisition unit 110, an identification unit 111, an extraction unit 112, an encapsulation unit 113, an analysis unit 114, an retrieval unit 115, a generation unit 116, a writing unit 117, a sending unit 118, an encryption unit 119, and a display unit. The unit 120, the transmission unit 121, and the storage unit 122. The module/unit referred to in this application refers to a series of computer program segments that can be executed by the processor 13 and can complete fixed functions, and are stored in the memory 12. In this embodiment, the functions of each module/unit will be described in detail in subsequent embodiments.

When receiving the data generation instruction, the acquiring unit 110 acquires the trigger interface of the data generation instruction.

The recognition unit 111 uses text recognition technology to recognize all text on the trigger interface.

In at least one embodiment of the present application, the recognition unit 111 uses text recognition technology to recognize the trigger interface to obtain all the text on the trigger interface.

Wherein, the trigger interface includes an input text box and a submit button.

The extracting unit 112 extracts at least one keyword from all the texts to obtain the attribute data of the data to be generated.

In at least one embodiment of the present application, the extracting unit 112 extracts at least one keyword from all the words, and obtaining the attribute data of the data to be generated includes:

The extraction unit 112 uses a Chinese keyword extraction algorithm based on high-dimensional clustering technology to extract at least one keyword from all the texts to obtain attribute data of the data to be generated.

Specifically, the extraction unit 112 performs fast word segmentation of all the texts according to a pre-configured target dictionary to obtain the first word segmentation, calculates the target word frequency of the first word segmentation, and calculates the target word frequency of the first word segmentation whose frequency is greater than the preset word frequency Determined as an initial keyword, the extraction unit 112 trims the initial keyword according to a preset small dictionary to obtain a final keyword, and determines the final keyword as the attribute data of the data to be generated.

Wherein, the target dictionary may include common keywords.

The encapsulation unit 113 uses JSON (JavaScript Object Notation, JS object notation) technology to encapsulate the attribute data to obtain the target string.

In at least one embodiment of the present application, the encapsulation unit 113 adopts JSON technology to encapsulate the attribute data to obtain the target string including:

The encapsulation unit 113 retrieves the target tag corresponding to each attribute data in the attribute data from the HBASE library. Further, the encapsulation unit 113 uses JSON technology to encapsulate each attribute data and each attribute data corresponding to it. The target tag is used to obtain the target string.

For example: the attribute data 1 is the name, the attribute data 2 is the gender, the encapsulation unit 113 retrieves the target tag corresponding to the name from the HBASE library as A, retrieves the target tag corresponding to the gender as B, and compares the name with A. Encapsulate, obtain the target string of the attribute data 1 as {A: name}, encapsulate the gender and B, obtain the target string of the attribute data 2 as {B: gender}

The parsing unit 114 uses JSON technology to parse the target string to obtain target parameters.

The transmission unit 121 uses ajax (Asynchronous Javascript And XML) technology to transmit the target character string to the back-end server of the electronic device.

Specifically, the transmission unit 121 adds the target character string to the post request, and uses the ajax technology to transmit the post request to the back-end server.

Through the foregoing implementation manners, the ajax technology is used to receive the target string from the front-end page of the electronic device, and data can be exchanged with the back-end server without reloading the front-end page, thereby improving efficiency.

In at least one embodiment of the present application, the parsing unit 114 uses JSON technology to parse the target string to obtain target parameters including:

The parsing unit 114 obtains all target tags in the target string, and further, the parsing unit 114 extracts data corresponding to each target tag in all the target tags as the target parameter.

For example: the target character string of the attribute data 1 is {A: name}, the target character string of the attribute data 2 is {B: gender}, and the parsing unit 114 obtains the target tag A of the attribute data 1 , The target tag B of the attribute data 2 extracts the name in the target string {A: name} according to the target tag A as a target parameter, and extracts the target according to the target tag B The gender in the string {B: gender} is used as the target parameter.

The parsing unit 114 uses a shallow semantic analysis method to analyze the target parameter to obtain the target type of the target parameter.

In at least one embodiment of the present application, the parsing unit 114 adopts a shallow semantic analysis method to analyze the semantic relationship between the target parameters to obtain the application scenario of the target parameters. Further, the parsing unit 114 114 Determine the target type of the target parameter according to the application scenario.

The retrieval unit 115 retrieves the row key generation strategy corresponding to the target type from the HBASE library.

The generating unit 116 generates the target row key of the target parameter in the HBASE library based on the row key generation strategy.

In at least one embodiment of the present application, the generating unit 116 obtains the number of rows from the target parameter. Further, the generating unit 116 imports the row key generation strategy into the HBASE library, according to the The number of rows and the row key generation strategy are used to generate the target row key.

Wherein, the number of the target row keys is equal to the number of rows.

The obtaining unit 110 obtains the target column name from the target parameter.

In at least one embodiment of the present application, the obtaining unit 110 obtaining the target list name from the target parameter includes:

The acquiring unit 110 matches the target parameter with the column name stored in the HBASE library, and further, the acquiring unit 110 confirms the target parameter matching the column name as the target column name.

The generating unit 116 generates an intermediate data file according to the target column name and the target row key.

In at least one embodiment of the present application, the generating unit 116 generating an intermediate data file according to the target column name and the target row key includes:

The generating unit 116 retrieves the target data corresponding to the target column name from the HBASE library, determines the target position according to the target column name and the target row key, and writes the target data to the target data. The target location is described, and the intermediate data file is obtained.

Wherein, the generating unit 116 writes the target data into the target location, and obtaining the intermediate data file specifically includes:

The generating unit 116 writes the target data into the target location through the statement put<table>,<rowkey>,<family:column>,<value>, repeats the above steps, until the target locations are all written For the target data, all the statements executed above are assembled to generate the intermediate data file.

The sending unit 118 sends the intermediate data file to the intermediate data layer, and further, the saving unit 122 uses the intermediate data layer to save the intermediate data file.

The writing unit 117 uses a calculation engine to write the intermediate data file into the HBASE library to obtain the data to be generated.

In at least one embodiment of the present application, the writing unit 117 uses a calculation engine to write the intermediate data file into the HBASE library, and obtaining the data to be generated includes:

According to the file size of the intermediate data file, the writing unit 117 divides the intermediate data file into at least one data block, calculates the number of the at least one data block, and obtains the target number. Further, the The writing unit 117 creates an HBASE table in the HBASE library, partitions the HBASE table using the target number as the number of partitions, and uses a calculation engine to write the at least one data block into the partitioned HBASE table , To obtain the data to be generated.

The obtaining unit 110 obtains the table name and generation time of the data to be generated, and according to the table name and the generation time, the generation unit 116 generates prompt information, and further, the sending unit 118 transmits the prompt Message to the terminal device of the designated contact.

The encryption unit 119 uses symmetric encryption technology to encrypt the data to be generated to obtain a ciphertext. Further, when it is detected that the ciphertext is successfully decrypted, the display unit 120 displays the data to be generated.

As shown in Figure 3, it is a schematic diagram of the structure of an electronic device implementing a preferred embodiment of the data generation method of the present application.

In an embodiment of the present application, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and a computer program stored in the memory 12 and running on the processor 13, such as Data generation program based on HBASE library.

Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. It may include more or less components than those shown in the figure, or a combination of certain components, or different components. Components, for example, the electronic device 1 may also include an input/output device, a network access device, a bus, and the like.

The processor 13 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor 13 is the computing core and control center of the electronic device 1 and connects the entire electronic device with various interfaces and lines. Each part of 1, and executes the operating system of the electronic device 1, and various installed applications, program codes, etc.

The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the application program to implement the steps in the foregoing data generation method embodiments, such as steps S10, S11, S12, S13, S14, S15, S16, S17, S18, S19, and S19 shown in FIG. S20.

Alternatively, when the processor 13 executes the computer program, the function of each module/unit in the foregoing device embodiments is implemented, for example: when a data generation instruction is received, the trigger interface of the data generation instruction is acquired; text recognition is adopted Technology recognizes all the text on the trigger interface; extracts at least one keyword from all the text to obtain the attribute data of the data to be generated; encapsulates the attribute data with JSON technology to obtain the target string; uses JSON technology to parse the data The target character string is used to obtain the target parameter; the shallow semantic analysis method is used to parse the target parameter to obtain the target type of the target parameter; the row key generation strategy corresponding to the target type is retrieved from the HBASE library; based on The row key generation strategy generates the target row key of the target parameter in the HBASE library; obtains the target column name from the target parameter; generates an intermediate data file according to the target column name and the target row key ; Use a calculation engine to write the intermediate data file into the HBASE library to obtain the data to be generated.

Exemplarily, the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 12 and executed by the processor 13 to complete the present invention. Application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program can be divided into an acquisition unit 110, an identification unit 111, an extraction unit 112, an encapsulation unit 113, an analysis unit 114, an retrieval unit 115, a generation unit 116, a writing unit 117, a sending unit 118, and an encryption unit. 119, a display unit 120, a transmission unit 121, and a storage unit 122.

The memory 12 may be used to store the computer program and/or module. The processor 13 runs or executes the computer program and/or module stored in the memory 12 and calls data stored in the memory 12, The various functions of the electronic device 1 are realized. The memory 12 may mainly include a storage program area and a storage data area. The storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may Stores data, etc. created based on the use of electronic devices. In addition, the memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) Card, Flash Card, at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device.

The memory 12 may be an external memory and/or an internal memory of the electronic device 1. Further, the memory 12 may be a circuit with a storage function without a physical form in an integrated circuit, such as RAM (Random-Access Memory, random access memory), FIFO (First In First Out), and so on. Alternatively, the memory 12 may also be a memory in a physical form, such as a memory stick, a TF card (Trans-flash Card), and so on.

If the integrated module/unit of the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium, which may be non-volatile. , It can also be volatile. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments.

Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

With reference to FIG. 1, the memory 12 in the electronic device 1 stores multiple instructions to implement a data generation method, and the processor 13 can execute the multiple instructions to implement: when a data generation instruction is received, Acquire the trigger interface of the data generation instruction; use text recognition technology to identify all the text on the trigger interface; extract at least one keyword from all the text to obtain the attribute data of the data to be generated; use JSON technology to encapsulate the Attribute data to obtain the target string; use JSON technology to parse the target string to obtain target parameters; use shallow semantic analysis methods to parse the target parameters to obtain the target type of the target parameters; retrieve from the HBASE library The row key generation strategy corresponding to the target type; generate the target row key of the target parameter in the HBASE library based on the row key generation strategy; obtain the target column name from the target parameter; according to the target The column name and the target row key are used to generate an intermediate data file; the calculation engine is used to write the intermediate data file into the HBASE library to obtain the data to be generated.

Specifically, for the specific implementation method of the above-mentioned instructions by the processor 13, reference may be made to the description of the relevant steps in the embodiment corresponding to FIG. 1, which will not be repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements are made without departing from the spirit and scope of the technical solution of the present invention.

Claims

A data generation method, wherein the method includes:

When receiving the data generation instruction, obtain the trigger interface of the data generation instruction;

Using text recognition technology to recognize all text on the trigger interface;

Extract at least one keyword from all the texts to obtain the attribute data of the data to be generated;

Use JSON technology to encapsulate the attribute data to obtain the target string;

Analyze the target string using JSON technology to obtain target parameters;

Analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter;

Retrieve the row key generation strategy corresponding to the target type from the HBASE library;

Generating the target row key of the target parameter in the HBASE library based on the row key generation strategy;

Obtain the target column name from the target parameter;

Generate an intermediate data file according to the target column name and the target row key;

A calculation engine is used to write the intermediate data file into the HBASE library to obtain the data to be generated.
8. The data generating method according to claim 1, wherein said encapsulating said attribute data using JSON technology to obtain the target character string comprises:

Retrieve the target tag corresponding to each attribute data in the attribute data from the HBASE library;

The JSON technology is used to encapsulate each attribute data and the target tag corresponding to each attribute data to obtain the target string.
8. The data generation method according to claim 1, wherein said parsing said target string using JSON technology to obtain target parameters comprises:

Acquiring all target tags in the target string;

The data corresponding to each target tag in all the target tags is extracted as the target parameter.
5. The data generation method according to claim 1, wherein said generating an intermediate data file according to said target column name and said target row key comprises:

Call out the target data corresponding to the target column name from the HBASE library;

Determine the target position according to the target column name and the target row key;

The target data is written into the target location to obtain the intermediate data file.
8. The data generation method according to claim 1, wherein said using a calculation engine to write said intermediate data file into said HBASE library to obtain said to-be-generated data comprises:

Dividing the intermediate data file into at least one data block according to the file size of the intermediate data file;

Calculate the number of the at least one data block to obtain the target number;

Creating an HBASE table in the HBASE library;

Partition the HBASE table by using the target number as the number of partitions;

The calculation engine is used to write the at least one data block into the partitioned HBASE table to obtain the data to be generated.
5. The data generating method according to claim 1, wherein after obtaining the data to be generated, the method further comprises:

Acquiring the table name and generation time of the data to be generated;

Generate prompt information according to the table name and the generation time;

Send the prompt information to the terminal device of the designated contact.
5. The data generating method according to claim 1, wherein after obtaining the data to be generated, the method further comprises:

Encrypt the data to be generated by using a symmetric encryption technology to obtain a ciphertext;

When it is detected that the ciphertext is successfully decrypted, the data to be generated is displayed.
A data generating device, wherein the device includes:

The acquiring unit is configured to acquire the trigger interface of the data generating instruction when the data generating instruction is received;

The recognition unit is used to recognize all the text on the trigger interface by using text recognition technology;

The extraction unit is used to extract at least one keyword from all the texts to obtain the attribute data of the data to be generated;

The encapsulation unit is used to encapsulate the attribute data using JSON technology to obtain the target string;

The parsing unit is used to parse the target string using JSON technology to obtain target parameters;

The parsing unit is further configured to analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter;

The retrieval unit is used to retrieve the row key generation strategy corresponding to the target type from the HBASE library;

Generating unit, configured to generate the target parameter based on the row key generation strategy in the

The target row key in the HBASE library;

The obtaining unit is further configured to obtain the target column name from the target parameter;

The generating unit is further configured to generate an intermediate data file according to the target column name and the target row key;

The writing unit is configured to use a calculation engine to write the intermediate data file into the HBASE library to obtain the data to be generated.
The data generating device according to claim 8, wherein the encapsulation unit is further configured to retrieve the target tag corresponding to each attribute data in the attribute data from the HBASE library; and encapsulate each attribute using JSON technology Data and the target tag corresponding to each attribute data to obtain the target string.
An electronic device, wherein the electronic device includes:

Memory, storing at least one instruction; and

The processor executes the instructions stored in the memory to implement the data generation method:

Wherein, the data generation method includes:

When receiving the data generation instruction, obtain the trigger interface of the data generation instruction;

Using text recognition technology to recognize all text on the trigger interface;

Extract at least one keyword from all the texts to obtain the attribute data of the data to be generated;

Use JSON technology to encapsulate the attribute data to obtain the target string;

Analyze the target string using JSON technology to obtain target parameters;

Analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter;

Retrieve the row key generation strategy corresponding to the target type from the HBASE library;

Generating the target row key of the target parameter in the HBASE library based on the row key generation strategy;

Obtain the target column name from the target parameter;

Generate an intermediate data file according to the target column name and the target row key;

A calculation engine is used to write the intermediate data file into the HBASE library to obtain the data to be generated.
10. The electronic device of claim 10, wherein the step of encapsulating the attribute data using JSON technology to obtain the target character string comprises:

Retrieve the target tag corresponding to each attribute data in the attribute data from the HBASE library;

The JSON technology is used to encapsulate each attribute data and the target tag corresponding to each attribute data to obtain the target string.
10. The electronic device of claim 10, wherein the step of parsing the target string using JSON technology to obtain target parameters comprises:

Acquiring all target tags in the target string;

The data corresponding to each target tag in all the target tags is extracted as the target parameter.
10. The electronic device of claim 10, wherein the step of generating an intermediate data file according to the target column name and the target row key comprises:

Call out the target data corresponding to the target column name from the HBASE library;

Determine the target position according to the target column name and the target row key;

The target data is written into the target location to obtain the intermediate data file.
10. The electronic device of claim 10, wherein the step of using a calculation engine to write the intermediate data file into the HBASE library to obtain the data to be generated comprises:

Dividing the intermediate data file into at least one data block according to the file size of the intermediate data file;

Calculate the number of the at least one data block to obtain the target number;

Creating an HBASE table in the HBASE library;

Partition the HBASE table by using the target number as the number of partitions;

The calculation engine is used to write the at least one data block into the partitioned HBASE table to obtain the data to be generated.
10. The electronic device of claim 10, wherein, after obtaining the data to be generated, the method further comprises:

Acquiring the table name and generation time of the data to be generated;

Generate prompt information according to the table name and the generation time;

Send the prompt information to the terminal device of the designated contact.
A computer-readable storage medium, wherein at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in an electronic device to implement a data generation method, wherein the data generation method It includes the following steps:

When receiving the data generation instruction, obtain the trigger interface of the data generation instruction;

Using text recognition technology to recognize all text on the trigger interface;

Extract at least one keyword from all the texts to obtain the attribute data of the data to be generated;

Use JSON technology to encapsulate the attribute data to obtain the target string;

Analyze the target string using JSON technology to obtain target parameters;

Analyze the target parameter using a shallow semantic analysis method to obtain the target type of the target parameter;

Retrieve the row key generation strategy corresponding to the target type from the HBASE library;

Generating the target row key of the target parameter in the HBASE library based on the row key generation strategy;

Obtain the target column name from the target parameter;

Generate an intermediate data file according to the target column name and the target row key;

A calculation engine is used to write the intermediate data file into the HBASE library to obtain the data to be generated.
15. The computer-readable storage medium of claim 16, wherein the step of using JSON technology to encapsulate the attribute data to obtain the target character string comprises:

Retrieve the target tag corresponding to each attribute data in the attribute data from the HBASE library;

The JSON technology is used to encapsulate each attribute data and the target tag corresponding to each attribute data to obtain the target string.
15. The computer-readable storage medium of claim 16, wherein the step of parsing the target string using JSON technology to obtain target parameters comprises:

Acquiring all target tags in the target string;

The data corresponding to each target tag in all the target tags is extracted as the target parameter.
15. The computer-readable storage medium of claim 16, wherein the step of generating an intermediate data file according to the target column name and the target row key comprises:

Call out the target data corresponding to the target column name from the HBASE library;

Determine the target position according to the target column name and the target row key;

The target data is written into the target location to obtain the intermediate data file.
16. The computer-readable storage medium according to claim 16, wherein the step of using a calculation engine to write the intermediate data file into the HBASE library, and obtaining the data to be generated comprises:

Dividing the intermediate data file into at least one data block according to the file size of the intermediate data file;

Calculate the number of the at least one data block to obtain the target number;

Creating an HBASE table in the HBASE library;

Partition the HBASE table by using the target number as the number of partitions;

The calculation engine is used to write the at least one data block into the partitioned HBASE table to obtain the data to be generated.