CN112667659A - Feature processing method and related equipment - Google Patents
Feature processing method and related equipment Download PDFInfo
- Publication number
- CN112667659A CN112667659A CN202011540114.7A CN202011540114A CN112667659A CN 112667659 A CN112667659 A CN 112667659A CN 202011540114 A CN202011540114 A CN 202011540114A CN 112667659 A CN112667659 A CN 112667659A
- Authority
- CN
- China
- Prior art keywords
- processed
- data
- determining
- parameter
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 152
- 238000000034 method Methods 0.000 claims abstract description 17
- 230000015654 memory Effects 0.000 claims description 23
- 238000003754 machining Methods 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to data processing and provides a feature processing method and related equipment. The method can determine a data source to be processed according to a characteristic processing request when the characteristic processing request is received; acquiring data to be processed from the data source to be processed, and writing the data to be processed into a message queue; extracting parameters to be processed in the data to be processed; determining a demand parameter according to the characteristic processing request, and determining a code template according to the demand parameter; generating a structured query statement according to the parameter to be processed and the code template; and processing the data to be processed by utilizing the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature. The invention can improve the processing efficiency of the characteristics. In addition, the invention also relates to a block chain technology, and the target characteristics can be stored in the block chain.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a feature processing method and related equipment.
Background
In the field of big data, feature engineering refers to screening better data features from original data by a series of engineering modes to improve the training effect of a model. In the actual feature processing process, a requirement engineer usually sets up a requirement, and then a developer develops a new feature according to the requirement, however, the inventor realizes that the method requires the developer to spend a lot of time writing and debugging programs, so that the derivative feature cannot be generated according to the rapid change of business logic.
Disclosure of Invention
In view of the above, it is desirable to provide a feature processing method and related apparatus, which can improve the processing efficiency of features.
In one aspect, the present invention provides a feature processing method, including:
when a characteristic machining request is received, determining a data source to be machined according to the characteristic machining request;
acquiring data to be processed from the data source to be processed, and writing the data to be processed into a message queue;
extracting parameters to be processed in the data to be processed;
determining a demand parameter according to the characteristic processing request, and determining a code template according to the demand parameter;
generating a structured query statement according to the parameter to be processed and the code template;
and processing the data to be processed by utilizing the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature.
According to a preferred embodiment of the present invention, the generating a structured query statement according to the parameter to be processed and the code template includes:
determining the parameter quantity of the parameters to be processed and determining the filling quantity of filling positions on the code template;
when the parameter number is larger than the filling number, determining a first difference value between the parameter number and the filling number, expanding the filling position on the code template according to the first difference value to obtain an expanded template, and writing the parameter to be processed into the expanded template to obtain the structured query statement; or
When the parameter number is smaller than the filling number, determining a second difference value between the parameter number and the filling number, deleting the filling position on the code template according to the second difference value to obtain a shrinkage limiting template, and writing the parameter to be processed into the shrinkage limiting template to obtain the structured query statement; or
And when the number of the parameters is equal to the filling number, writing the parameters to be processed into the code template to obtain the structured query statement.
According to a preferred embodiment of the present invention, the processing the data to be processed by using the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature includes:
acquiring all requests in the message queue, wherein all the requests comprise the characteristic processing request;
determining the request grades of all the requests, and sequencing all the requests according to the sequence of the request grades from large to small to obtain a request queue;
determining the priority of the characteristic processing request according to the position of the characteristic processing request in the request queue;
and when the request higher than the priority is detected to complete the response, processing the data to be processed by using the structured query statement to obtain the target characteristic.
According to a preferred embodiment of the present invention, the acquiring data to be processed from the data source to be processed includes:
determining a triggering user of the characteristic machining request;
acquiring a user identification code of the trigger user, and determining a user role of the trigger user according to the user identification code;
determining a data table corresponding to the user role from the data source to be processed;
and extracting all data in the data table to obtain the data to be processed.
According to the preferred embodiment of the present invention, the writing the data to be processed into the message queue includes:
determining the data volume of the data to be processed, and determining a processing thread according to the data volume;
determining a data format of the data to be processed;
detecting whether the data format is a preset format or not;
and when the data format is not the preset format, converting the data to be processed into the data with the preset format by using the processing thread, and writing the converted data into the message queue.
According to a preferred embodiment of the present invention, the determining a data source to be processed according to the feature processing request includes:
analyzing the message of the characteristic processing request to obtain message information carried by the message;
acquiring a first preset label from a configuration label library, wherein the first preset label is used for indicating a system identifier;
acquiring information corresponding to the first preset label from the message information as a system number;
determining a service system according to the system code;
and determining a database in the service system as the data source to be processed.
According to a preferred embodiment of the present invention, the determining a requirement parameter according to the feature processing request and determining a code template according to the requirement parameter includes:
acquiring a second preset label from the configuration label library, wherein the second preset label is used for indicating a demand;
acquiring information corresponding to the second preset label from the message information as the demand parameter;
and acquiring a template corresponding to the demand parameter from a preset template library as the code template.
In another aspect, the present invention further provides a feature processing apparatus, including:
the determining unit is used for determining a data source to be processed according to the characteristic processing request when the characteristic processing request is received;
the execution unit is used for acquiring the data to be processed from the data source to be processed and writing the data to be processed into a message queue;
the extraction unit is used for extracting the parameters to be processed in the data to be processed;
the determining unit is further used for determining a demand parameter according to the characteristic processing request and determining a code template according to the demand parameter;
the generating unit is used for generating a structured query statement according to the parameter to be processed and the code template;
and the processing unit is used for processing the data to be processed by utilizing the structured query statement to obtain the target characteristics based on the priority of the characteristic processing request in the message queue.
In another aspect, the present invention further provides an electronic device, including:
a memory storing computer readable instructions; and
a processor executing computer readable instructions stored in the memory to implement the feature processing method.
In another aspect, the present invention further provides a computer-readable storage medium, in which computer-readable instructions are stored, and the computer-readable instructions are executed by a processor in an electronic device to implement the feature processing method.
According to the technical scheme, when a characteristic processing request is received, a data source to be processed is determined according to the characteristic processing request, a data source where data to be processed is located can be determined from the characteristic processing request, the data to be processed can be rapidly acquired through the data source, the data to be processed is acquired from the data source to be processed and is written into a message queue, parameters to be processed in the data to be processed are extracted, required parameters are determined according to the characteristic processing request, a code template is determined according to the required parameters, and the code template is determined, so that the generation efficiency of the structured query statement can be improved, the structured query statement is generated according to the parameters to be processed and the code template, and processing the data to be processed by using the structured query statement based on the priority of the feature processing request in the message queue to obtain a target feature, and processing the data to be processed by using the priority, so that the influence of other requests on the feature processing request can be avoided, and the smooth generation of the target feature is ensured. According to the method and the device, the required parameters in the feature processing request are determined, the code module corresponding to the required parameters can be further determined, the structural query statement can be rapidly generated by utilizing the parameters to be processed and the code template, and the processing efficiency of the target feature can be improved due to the fact that developers do not need to manually write and debug programs, so that the timeliness of feature processing is improved, meanwhile, the range of applicable crowds of the feature processing mode can be improved, in addition, the data to be processed is written into the message queue, the data to be processed can be processed based on the priority of the feature processing request, the influence of other requests on the feature processing request is avoided, and the smooth generation of the target feature is ensured.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the feature creation method of the present invention.
FIG. 2 is a flow diagram of an embodiment of the present invention for generating a structured query statement.
FIG. 3 is a flow diagram of one embodiment of generating target features of the present invention.
FIG. 4 is a functional block diagram of a preferred embodiment of the feature creation apparatus of the present invention.
Fig. 5 is a schematic structural diagram of an electronic device implementing a feature processing method according to a preferred embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
FIG. 1 is a flow chart of a preferred embodiment of the feature creation method of the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.
The feature processing method is applied to one or more electronic devices, which are devices capable of automatically performing numerical calculation and/or information processing according to computer readable instructions set or stored in advance, and the hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), a smart wearable device, and the like.
The electronic device may include a network device and/or a user device. Wherein the network device includes, but is not limited to, a single network electronic device, an electronic device group consisting of a plurality of network electronic devices, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network electronic devices.
The network in which the electronic device is located includes, but is not limited to: the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
And S10, when the characteristic processing request is received, determining a data source to be processed according to the characteristic processing request.
In at least one embodiment of the invention, the feature processing request may be triggered by a data analyst. The information carried in the feature processing request includes, but is not limited to: system number, demand parameters, etc.
The data source to be processed can be a database in any business system.
In at least one embodiment of the present invention, the electronic device determining the data source to be processed according to the characteristic processing request includes:
analyzing the message of the characteristic processing request to obtain message information carried by the message;
acquiring a first preset label from a configuration label library, wherein the first preset label is used for indicating a system identifier;
acquiring information corresponding to the first preset label from the message information as a system number;
determining a service system according to the system code;
and determining a database in the service system as the data source to be processed.
Wherein, a plurality of predefined tags are stored in the configuration tag library.
The system code is capable of uniquely identifying the business system.
The message information can be quickly acquired by analyzing the message of the characteristic machining request, the system number can be accurately extracted from the message information through the first preset label, and the service system can be uniquely identified through the system number, so that the data source to be machined can be quickly determined according to the system number.
And S11, acquiring the data to be processed from the data source to be processed, and writing the data to be processed into a message queue.
In at least one embodiment of the present invention, the data to be processed refers to data that needs to be processed for a feature.
The message queue may be Kafka, which is a high throughput distributed publish-subscribe messaging system.
In at least one embodiment of the present invention, the electronic device acquiring the data to be processed from the data source to be processed includes:
determining a triggering user of the characteristic machining request;
acquiring a user identification code of the trigger user, and determining a user role of the trigger user according to the user identification code;
determining a data table corresponding to the user role from the data source to be processed;
and extracting all data in the data table to obtain the data to be processed.
The user identification code is a code that can uniquely identify the trigger user, and the user identification code may be a job number of the trigger user or an identity card of the trigger user.
The user role refers to a role that the triggering user acts, for example, the user role may be a job of the triggering user.
The data table may record sales volume of a certain brand of mobile phone.
By determining the user role of the trigger user, the object to be processed in the characteristic processing request can be quickly determined according to the user role.
In at least one embodiment of the present invention, the electronic device writing the data to be processed into a message queue includes:
determining the data volume of the data to be processed, and determining a processing thread according to the data volume;
determining a data format of the data to be processed;
detecting whether the data format is a preset format or not;
and when the data format is not the preset format, converting the data to be processed into the data with the preset format by using the processing thread, and writing the converted data into the message queue.
The preset format can be set according to an application scene.
By the embodiment, the format of the data written into the message queue can be ensured to meet the preset format, so that the characteristic processing of the data can be facilitated subsequently, and in addition, the processing thread is determined according to the data quantity, so that the data writing efficiency can be improved.
And S12, extracting the parameters to be processed in the data to be processed.
In at least one embodiment of the present invention, the parameter to be processed refers to a field that needs to be processed, for example, the parameter to be processed may be a sales volume, a price, or the like.
In at least one embodiment of the present invention, the extracting, by the electronic device, the to-be-processed parameter in the to-be-processed data includes:
extracting all fields in the data table;
and determining all the fields as the parameters to be processed.
Through the embodiment, the data table stores the fields corresponding to the parameters to be processed, so that the parameters to be processed can be quickly determined.
And S13, determining a demand parameter according to the characteristic processing request, and determining a code template according to the demand parameter.
In at least one embodiment of the present invention, the requirement parameter is determined by the triggering user, and the requirement parameter may be an average value or a sum.
The code template refers to a code statement corresponding to the requirement parameter.
In at least one embodiment of the present invention, the determining, by the electronic device, a requirement parameter according to the feature processing request, and determining a code template according to the requirement parameter includes:
acquiring a second preset label from the configuration label library, wherein the second preset label is used for indicating a demand;
acquiring information corresponding to the second preset label from the message information as the demand parameter;
and acquiring a template corresponding to the demand parameter from a preset template library as the code template.
And a plurality of templates are stored in the preset module plate.
Through the implementation mode, the code template corresponding to the requirement parameter can be determined quickly.
And S14, generating a structured query statement according to the parameters to be processed and the code template.
Referring to FIG. 2, FIG. 2 is a flow diagram of one embodiment of the present invention for generating a structured query statement. In at least one embodiment of the present invention, the generating, by the electronic device, a structured query statement according to the parameter to be processed and the code template includes:
s140, determining the parameter number of the parameters to be processed, and determining the filling number of the filling positions on the code template.
The parameter number refers to the number of the parameters to be processed.
The filling quantity refers to the quantity of positions on the code template where filling parameters can be carried out.
S141, when the parameter number is larger than the filling number, determining a first difference value between the parameter number and the filling number, expanding the filling position on the code template according to the first difference value to obtain an expanded template, and writing the parameter to be processed into the expanded template to obtain the structured query statement.
For example, the number of the parameters is 8, the number of the padding is 5, and the first difference is calculated to be 3, so that 3 positions where padding parameters can be performed are expanded on the code template.
And S142, when the parameter number is smaller than the filling number, determining a second difference value between the parameter number and the filling number, deleting the filling position on the code template according to the second difference value to obtain a shrinkage limiting template, and writing the parameter to be processed into the shrinkage limiting template to obtain the structured query statement.
S143, when the number of the parameters is equal to the filling number, writing the parameters to be processed into the code template to obtain the structured query statement.
By the implementation mode, the code template can be adjusted according to the parameters to be processed, the applicability of the code template is improved, in addition, the parameters to be processed are written into the code template, and the generation efficiency of the structured query statement can be improved because the code template part does not need to be written.
And S15, processing the data to be processed by using the structured query statement to obtain the target characteristics based on the priority of the characteristic processing request in the message queue.
It is emphasized that the target feature may also be stored in a node of a blockchain in order to further ensure privacy and security of the target feature.
In at least one embodiment of the present invention, the target feature is determined according to the parameter to be processed, the data to be processed, and the requirement parameter, such as: the target characteristic may be a total amount of cell phone sales in a certain area.
Referring to fig. 3, fig. 3 is a flow chart of one embodiment of generating target features of the present invention. In at least one embodiment of the present invention, the processing, by the electronic device, the data to be processed by using the structured query statement based on the priority of the feature processing request in the message queue, and obtaining the target feature includes:
s150, all requests in the message queue are obtained, and the all requests comprise the characteristic processing request.
The all requests refer to requests that are not processed in the message queue.
S151, determining the request grades of all the requests, and sequencing all the requests according to the sequence of the request grades from big to small to obtain a request queue.
The request level refers to the urgency of handling each request.
S152, determining the priority of the characteristic processing request according to the position of the characteristic processing request in the request queue.
And S153, when the request higher than the priority is detected to complete the response, processing the data to be processed by using the structured query statement to obtain the target feature.
With the above-described embodiment, it is possible to avoid the influence of other requests on the feature processing request, and to ensure smooth generation of the target feature.
In at least one embodiment of the invention, after obtaining the target feature, the method further comprises:
acquiring a request number of the characteristic processing request;
generating prompt information according to the request number and the target characteristics;
encrypting the prompt message by adopting a symmetric encryption algorithm to obtain a ciphertext;
and sending the ciphertext to the terminal equipment of the trigger user.
Through the embodiment, the prompt message can be generated in time after the target feature is generated, and then the trigger user can be informed to receive the prompt message in time.
According to the technical scheme, when a characteristic processing request is received, a data source to be processed is determined according to the characteristic processing request, a data source where data to be processed is located can be determined from the characteristic processing request, the data to be processed can be rapidly acquired through the data source, the data to be processed is acquired from the data source to be processed and is written into a message queue, parameters to be processed in the data to be processed are extracted, required parameters are determined according to the characteristic processing request, a code template is determined according to the required parameters, and the code template is determined, so that the generation efficiency of the structured query statement can be improved, the structured query statement is generated according to the parameters to be processed and the code template, and processing the data to be processed by using the structured query statement based on the priority of the feature processing request in the message queue to obtain a target feature, and processing the data to be processed by using the priority, so that the influence of other requests on the feature processing request can be avoided, and the smooth generation of the target feature is ensured. According to the method and the device, the required parameters in the feature processing request are determined, the code module corresponding to the required parameters can be further determined, the structural query statement can be rapidly generated by utilizing the parameters to be processed and the code template, and the processing efficiency of the target feature can be improved due to the fact that developers do not need to manually write and debug programs, so that the timeliness of feature processing is improved, meanwhile, the range of applicable crowds of the feature processing mode can be improved, in addition, the data to be processed is written into the message queue, the data to be processed can be processed based on the priority of the feature processing request, the influence of other requests on the feature processing request is avoided, and the smooth generation of the target feature is ensured.
Fig. 4 is a functional block diagram of a preferred embodiment of the feature processing apparatus of the present invention. The feature processing device 11 includes a determination unit 110, an execution unit 111, an extraction unit 112, a generation unit 113, a processing unit 114, an acquisition unit 115, an encryption unit 116, and a transmission unit 117. The module/unit referred to herein is a series of computer readable instruction segments that can be accessed by the processor 13 and perform a fixed function and that are stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.
When a feature processing request is received, the determining unit 110 determines a data source to be processed according to the feature processing request.
In at least one embodiment of the invention, the feature processing request may be triggered by a data analyst. The information carried in the feature processing request includes, but is not limited to: system number, demand parameters, etc.
The data source to be processed can be a database in any business system.
In at least one embodiment of the present invention, the determining unit 110 determines the data source to be processed according to the characteristic processing request, including:
analyzing the message of the characteristic processing request to obtain message information carried by the message;
acquiring a first preset label from a configuration label library, wherein the first preset label is used for indicating a system identifier;
acquiring information corresponding to the first preset label from the message information as a system number;
determining a service system according to the system code;
and determining a database in the service system as the data source to be processed.
Wherein, a plurality of predefined tags are stored in the configuration tag library.
The system code is capable of uniquely identifying the business system.
The message information can be quickly acquired by analyzing the message of the characteristic machining request, the system number can be accurately extracted from the message information through the first preset label, and the service system can be uniquely identified through the system number, so that the data source to be machined can be quickly determined according to the system number.
The execution unit 111 obtains the data to be processed from the data source to be processed, and writes the data to be processed into the message queue.
In at least one embodiment of the present invention, the data to be processed refers to data that needs to be processed for a feature.
The message queue may be Kafka, which is a high throughput distributed publish-subscribe messaging system.
In at least one embodiment of the present invention, the acquiring, by the execution unit 111, data to be processed from the data source to be processed includes:
determining a triggering user of the characteristic machining request;
acquiring a user identification code of the trigger user, and determining a user role of the trigger user according to the user identification code;
determining a data table corresponding to the user role from the data source to be processed;
and extracting all data in the data table to obtain the data to be processed.
The user identification code is a code that can uniquely identify the trigger user, and the user identification code may be a job number of the trigger user or an identity card of the trigger user.
The user role refers to a role that the triggering user acts, for example, the user role may be a job of the triggering user.
The data table may record sales volume of a certain brand of mobile phone.
By determining the user role of the trigger user, the object to be processed in the characteristic processing request can be quickly determined according to the user role.
In at least one embodiment of the present invention, the writing, by the execution unit 111, the data to be processed into the message queue includes:
determining the data volume of the data to be processed, and determining a processing thread according to the data volume;
determining a data format of the data to be processed;
detecting whether the data format is a preset format or not;
and when the data format is not the preset format, converting the data to be processed into the data with the preset format by using the processing thread, and writing the converted data into the message queue.
The preset format can be set according to an application scene.
By the embodiment, the format of the data written into the message queue can be ensured to meet the preset format, so that the characteristic processing of the data can be facilitated subsequently, and in addition, the processing thread is determined according to the data quantity, so that the data writing efficiency can be improved.
The extraction unit 112 extracts a parameter to be processed in the data to be processed.
In at least one embodiment of the present invention, the parameter to be processed refers to a field that needs to be processed, for example, the parameter to be processed may be a sales volume, a price, or the like.
In at least one embodiment of the present invention, the extracting unit 112 extracts the parameter to be processed in the data to be processed, including:
extracting all fields in the data table;
and determining all the fields as the parameters to be processed.
Through the embodiment, the data table stores the fields corresponding to the parameters to be processed, so that the parameters to be processed can be quickly determined.
The determining unit 110 determines a requirement parameter according to the feature processing request, and determines a code template according to the requirement parameter.
In at least one embodiment of the present invention, the requirement parameter is determined by the triggering user, and the requirement parameter may be an average value or a sum.
The code template refers to a code statement corresponding to the requirement parameter.
In at least one embodiment of the present invention, the determining unit 110 determines a requirement parameter according to the feature processing request, and determining a code template according to the requirement parameter includes:
acquiring a second preset label from the configuration label library, wherein the second preset label is used for indicating a demand;
acquiring information corresponding to the second preset label from the message information as the demand parameter;
and acquiring a template corresponding to the demand parameter from a preset template library as the code template.
And a plurality of templates are stored in the preset module plate.
Through the implementation mode, the code template corresponding to the requirement parameter can be determined quickly.
The generating unit 113 generates a structured query statement according to the parameter to be processed and the code template.
In at least one embodiment of the present invention, the generating unit 113 generates the structured query statement according to the parameter to be processed and the code template, including:
and determining the parameter quantity of the parameters to be processed and determining the filling quantity of the filling positions on the code template.
The parameter number refers to the number of the parameters to be processed.
The filling quantity refers to the quantity of positions on the code template where filling parameters can be carried out.
And when the parameter quantity is greater than the filling quantity, determining a first difference value between the parameter quantity and the filling quantity, expanding the filling position on the code template according to the first difference value to obtain an expanded template, and writing the parameters to be processed into the expanded template to obtain the structured query statement.
For example, the number of the parameters is 8, the number of the padding is 5, and the first difference is calculated to be 3, so that 3 positions where padding parameters can be performed are expanded on the code template.
And when the parameter number is smaller than the filling number, determining a second difference value between the parameter number and the filling number, deleting the filling position on the code template according to the second difference value to obtain a shrinkage limiting template, and writing the parameter to be processed into the shrinkage limiting template to obtain the structured query statement.
And when the number of the parameters is equal to the filling number, writing the parameters to be processed into the code template to obtain the structured query statement.
By the implementation mode, the code template can be adjusted according to the parameters to be processed, the applicability of the code template is improved, in addition, the parameters to be processed are written into the code template, and the generation efficiency of the structured query statement can be improved because the code template part does not need to be written.
The processing unit 114 processes the data to be processed by using the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature.
It is emphasized that the target feature may also be stored in a node of a blockchain in order to further ensure privacy and security of the target feature.
In at least one embodiment of the present invention, the target feature is determined according to the parameter to be processed, the data to be processed, and the requirement parameter, such as: the target characteristic may be a total amount of cell phone sales in a certain area.
In at least one embodiment of the present invention, the processing unit 114, based on the priority of the feature processing request in the message queue, processing the data to be processed by using the structured query statement to obtain the target feature includes:
and acquiring all requests in the message queue, wherein all the requests comprise the characteristic processing request.
The all requests refer to requests that are not processed in the message queue.
And determining the request grades of all the requests, and sequencing all the requests according to the sequence of the request grades from large to small to obtain a request queue.
The request level refers to the urgency of handling each request.
And determining the priority of the characteristic processing request according to the position of the characteristic processing request in the request queue.
And when the request higher than the priority is detected to complete the response, processing the data to be processed by using the structured query statement to obtain the target characteristic.
With the above-described embodiment, it is possible to avoid the influence of other requests on the feature processing request, and to ensure smooth generation of the target feature.
In at least one embodiment of the present invention, after obtaining the target feature, the obtaining unit 115 obtains a request number of the feature processing request;
the generating unit 113 generates prompt information according to the request number and the target feature;
the encryption unit 116 encrypts the prompt message by using a symmetric encryption algorithm to obtain a ciphertext;
the transmitting unit 117 transmits the ciphertext to the terminal device of the trigger user.
Through the embodiment, the prompt message can be generated in time after the target feature is generated, and then the trigger user can be informed to receive the prompt message in time.
According to the technical scheme, when a characteristic processing request is received, a data source to be processed is determined according to the characteristic processing request, a data source where data to be processed is located can be determined from the characteristic processing request, the data to be processed can be rapidly acquired through the data source, the data to be processed is acquired from the data source to be processed and is written into a message queue, parameters to be processed in the data to be processed are extracted, required parameters are determined according to the characteristic processing request, a code template is determined according to the required parameters, and the code template is determined, so that the generation efficiency of the structured query statement can be improved, the structured query statement is generated according to the parameters to be processed and the code template, and processing the data to be processed by using the structured query statement based on the priority of the feature processing request in the message queue to obtain a target feature, and processing the data to be processed by using the priority, so that the influence of other requests on the feature processing request can be avoided, and the smooth generation of the target feature is ensured. According to the method and the device, the required parameters in the feature processing request are determined, the code module corresponding to the required parameters can be further determined, the structural query statement can be rapidly generated by utilizing the parameters to be processed and the code template, and the processing efficiency of the target feature can be improved due to the fact that developers do not need to manually write and debug programs, so that the timeliness of feature processing is improved, meanwhile, the range of applicable crowds of the feature processing mode can be improved, in addition, the data to be processed is written into the message queue, the data to be processed can be processed based on the priority of the feature processing request, the influence of other requests on the feature processing request is avoided, and the smooth generation of the target feature is ensured.
Fig. 5 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention.
In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer readable instructions, such as a feature processing program, stored in the memory 12 and executable on the processor 13.
It will be appreciated by a person skilled in the art that the schematic diagram is only an example of the electronic device 1 and does not constitute a limitation of the electronic device 1, and that it may comprise more or less components than shown, or some components may be combined, or different components, e.g. the electronic device 1 may further comprise an input output device, a network access device, a bus, etc.
The Processor 13 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. The processor 13 is an operation core and a control center of the electronic device 1, and is connected to each part of the whole electronic device 1 by various interfaces and lines, and executes an operating system of the electronic device 1 and various installed application programs, program codes, and the like.
Illustratively, the computer readable instructions may be partitioned into one or more modules/units that are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules/units may be a series of computer readable instruction segments capable of performing specific functions, which are used for describing the execution process of the computer readable instructions in the electronic device 1. For example, the computer-readable instructions may be divided into a determination unit 110, an execution unit 111, an extraction unit 112, a generation unit 113, a processing unit 114, an acquisition unit 115, an encryption unit 116, and a transmission unit 117.
The memory 12 may be used for storing the computer readable instructions and/or modules, and the processor 13 implements various functions of the electronic device 1 by executing or executing the computer readable instructions and/or modules stored in the memory 12 and invoking data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. The memory 12 may include non-volatile and volatile memories, such as: a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other storage device.
The memory 12 may be an external memory and/or an internal memory of the electronic device 1. Further, the memory 12 may be a memory having a physical form, such as a memory stick, a TF Card (Trans-flash Card), or the like.
The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the above embodiments may be implemented by hardware that is configured to be instructed by computer readable instructions, which may be stored in a computer readable storage medium, and when the computer readable instructions are executed by a processor, the steps of the method embodiments may be implemented.
Wherein the computer readable instructions comprise computer readable instruction code which may be in source code form, object code form, an executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying said computer readable instruction code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM).
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
With reference to fig. 1, the memory 12 in the electronic device 1 stores computer-readable instructions to implement a feature processing method, and the processor 13 can execute the computer-readable instructions to implement:
when a characteristic machining request is received, determining a data source to be machined according to the characteristic machining request;
acquiring data to be processed from the data source to be processed, and writing the data to be processed into a message queue;
extracting parameters to be processed in the data to be processed;
determining a demand parameter according to the characteristic processing request, and determining a code template according to the demand parameter;
generating a structured query statement according to the parameter to be processed and the code template;
and processing the data to be processed by utilizing the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature.
Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the computer readable instructions, which is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The computer readable storage medium has computer readable instructions stored thereon, wherein the computer readable instructions when executed by the processor 13 are configured to implement the steps of:
when a characteristic machining request is received, determining a data source to be machined according to the characteristic machining request;
acquiring data to be processed from the data source to be processed, and writing the data to be processed into a message queue;
extracting parameters to be processed in the data to be processed;
determining a demand parameter according to the characteristic processing request, and determining a code template according to the demand parameter;
generating a structured query statement according to the parameter to be processed and the code template;
and processing the data to be processed by utilizing the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. The plurality of units or devices may also be implemented by one unit or device through software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A feature processing method, characterized by comprising:
when a characteristic machining request is received, determining a data source to be machined according to the characteristic machining request;
acquiring data to be processed from the data source to be processed, and writing the data to be processed into a message queue;
extracting parameters to be processed in the data to be processed;
determining a demand parameter according to the characteristic processing request, and determining a code template according to the demand parameter;
generating a structured query statement according to the parameter to be processed and the code template;
and processing the data to be processed by utilizing the structured query statement based on the priority of the feature processing request in the message queue to obtain the target feature.
2. The feature processing method of claim 1, wherein the generating a structured query statement according to the parameter to be processed and the code template comprises:
determining the parameter quantity of the parameters to be processed and determining the filling quantity of filling positions on the code template;
when the parameter number is larger than the filling number, determining a first difference value between the parameter number and the filling number, expanding the filling position on the code template according to the first difference value to obtain an expanded template, and writing the parameter to be processed into the expanded template to obtain the structured query statement; or
When the parameter number is smaller than the filling number, determining a second difference value between the parameter number and the filling number, deleting the filling position on the code template according to the second difference value to obtain a shrinkage limiting template, and writing the parameter to be processed into the shrinkage limiting template to obtain the structured query statement; or
And when the number of the parameters is equal to the filling number, writing the parameters to be processed into the code template to obtain the structured query statement.
3. The feature processing method of claim 1, wherein processing the data to be processed using the structured query statement based on the priority of the feature processing request in the message queue to obtain a target feature comprises:
acquiring all requests in the message queue, wherein all the requests comprise the characteristic processing request;
determining the request grades of all the requests, and sequencing all the requests according to the sequence of the request grades from large to small to obtain a request queue;
determining the priority of the characteristic processing request according to the position of the characteristic processing request in the request queue;
and when the request higher than the priority is detected to complete the response, processing the data to be processed by using the structured query statement to obtain the target characteristic.
4. The feature processing method of claim 1, wherein said obtaining data to be processed from said data source to be processed comprises:
determining a triggering user of the characteristic machining request;
acquiring a user identification code of the trigger user, and determining a user role of the trigger user according to the user identification code;
determining a data table corresponding to the user role from the data source to be processed;
and extracting all data in the data table to obtain the data to be processed.
5. The feature elaboration method according to claim 1, characterized in that said writing the data to be elaborated into a message queue comprises:
determining the data volume of the data to be processed, and determining a processing thread according to the data volume;
determining a data format of the data to be processed;
detecting whether the data format is a preset format or not;
and when the data format is not the preset format, converting the data to be processed into the data with the preset format by using the processing thread, and writing the converted data into the message queue.
6. The feature processing method of claim 1, wherein said determining a source of data to be processed based on the feature processing request comprises:
analyzing the message of the characteristic processing request to obtain message information carried by the message;
acquiring a first preset label from a configuration label library, wherein the first preset label is used for indicating a system identifier;
acquiring information corresponding to the first preset label from the message information as a system number;
determining a service system according to the system code;
and determining a database in the service system as the data source to be processed.
7. The feature processing method of claim 6, wherein determining a demand parameter from the feature processing request and determining a code template from the demand parameter comprises:
acquiring a second preset label from the configuration label library, wherein the second preset label is used for indicating a demand;
acquiring information corresponding to the second preset label from the message information as the demand parameter;
and acquiring a template corresponding to the demand parameter from a preset template library as the code template.
8. A feature machining apparatus, characterized by comprising:
the determining unit is used for determining a data source to be processed according to the characteristic processing request when the characteristic processing request is received;
the execution unit is used for acquiring the data to be processed from the data source to be processed and writing the data to be processed into a message queue;
the extraction unit is used for extracting the parameters to be processed in the data to be processed;
the determining unit is further used for determining a demand parameter according to the characteristic processing request and determining a code template according to the demand parameter;
the generating unit is used for generating a structured query statement according to the parameter to be processed and the code template;
and the processing unit is used for processing the data to be processed by utilizing the structured query statement to obtain the target characteristics based on the priority of the characteristic processing request in the message queue.
9. An electronic device, characterized in that the electronic device comprises:
a memory storing computer readable instructions; and
a processor executing computer readable instructions stored in the memory to implement the feature processing method of any one of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium stores therein computer-readable instructions to be executed by a processor in an electronic device to implement the feature processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011540114.7A CN112667659B (en) | 2020-12-23 | 2020-12-23 | Feature processing method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011540114.7A CN112667659B (en) | 2020-12-23 | 2020-12-23 | Feature processing method and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112667659A true CN112667659A (en) | 2021-04-16 |
| CN112667659B CN112667659B (en) | 2024-04-02 |
Family
ID=75408912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011540114.7A Active CN112667659B (en) | 2020-12-23 | 2020-12-23 | Feature processing method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112667659B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080319959A1 (en) * | 2007-06-22 | 2008-12-25 | International Business Machines Corporation | Generating information on database queries in source code into object code compiled from the source code |
| US20090125892A1 (en) * | 2005-11-18 | 2009-05-14 | Robert Arthur Crewdson | Computer Software Development System and Method |
| CN103064950A (en) * | 2012-12-27 | 2013-04-24 | 北京思特奇信息技术股份有限公司 | Database client construction method and database client system |
| CN106293664A (en) * | 2015-05-27 | 2017-01-04 | 交通银行股份有限公司 | Code generating method and device |
| CN107480280A (en) * | 2017-08-22 | 2017-12-15 | 金蝶软件（中国）有限公司 | The method and relevant device of a kind of data processing |
| US20180113681A1 (en) * | 2016-10-25 | 2018-04-26 | Paypal, Inc. | Matching Programming Variables Across Different Data Domains |
-
2020
- 2020-12-23 CN CN202011540114.7A patent/CN112667659B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090125892A1 (en) * | 2005-11-18 | 2009-05-14 | Robert Arthur Crewdson | Computer Software Development System and Method |
| US20080319959A1 (en) * | 2007-06-22 | 2008-12-25 | International Business Machines Corporation | Generating information on database queries in source code into object code compiled from the source code |
| CN103064950A (en) * | 2012-12-27 | 2013-04-24 | 北京思特奇信息技术股份有限公司 | Database client construction method and database client system |
| CN106293664A (en) * | 2015-05-27 | 2017-01-04 | 交通银行股份有限公司 | Code generating method and device |
| US20180113681A1 (en) * | 2016-10-25 | 2018-04-26 | Paypal, Inc. | Matching Programming Variables Across Different Data Domains |
| CN107480280A (en) * | 2017-08-22 | 2017-12-15 | 金蝶软件（中国）有限公司 | The method and relevant device of a kind of data processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112667659B (en) | 2024-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111797351A (en) | Page data management method and device, electronic equipment and medium | |
| CN112632163B (en) | Big data report export method and related equipment | |
| CN112948418A (en) | Dynamic query method, device, equipment and storage medium | |
| CN112711398A (en) | Method, device and equipment for generating buried point file and storage medium | |
| CN112433705A (en) | Script generation method and device, electronic equipment and storage medium | |
| CN116360769A (en) | Code generation method, device, equipment and storage medium | |
| CN113536770B (en) | Text analysis method, device and equipment based on artificial intelligence and storage medium | |
| CN114372060A (en) | Data storage method, device, equipment and storage medium | |
| CN114510487A (en) | Data table merging method, device, equipment and storage medium | |
| CN114418398A (en) | Scene task development method, device, equipment and storage medium | |
| CN112947911A (en) | Interface script generation method, device, equipment and storage medium | |
| CN112784566A (en) | Document generation method, device, equipment and storage medium | |
| CN112181485A (en) | Script execution method and device, electronic equipment and storage medium | |
| CN112632098A (en) | Dynamic generation method of structured query statement and related equipment | |
| CN111680483A (en) | Document template updating method and device, electronic equipment and medium | |
| CN112199483A (en) | Information input assisting method and device, electronic equipment and storage medium | |
| CN112667659B (en) | Feature processing method and related equipment | |
| CN114692204A (en) | Data query method, device, equipment and storage medium | |
| CN112817742B (en) | Data migration method, device, equipment and storage medium | |
| CN114942749A (en) | Development method, device and equipment of approval system and storage medium | |
| CN112308440B (en) | Work order processing method, device, computer equipment and computer readable storage medium | |
| CN114238296A (en) | Product index data display method, device, equipment and storage medium | |
| CN114610386A (en) | Interaction method, device, equipment and storage medium of H5 and application program | |
| CN112434237A (en) | Page loading method and device, electronic equipment and storage medium | |
| CN113283677A (en) | Index data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |