CN111460773A - Data processing method, device and system, electronic equipment and storage medium - Google Patents
Data processing method, device and system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111460773A CN111460773A CN202010238784.7A CN202010238784A CN111460773A CN 111460773 A CN111460773 A CN 111460773A CN 202010238784 A CN202010238784 A CN 202010238784A CN 111460773 A CN111460773 A CN 111460773A
- Authority
- CN
- China
- Prior art keywords
- data
- determining
- target
- service type
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application relates to a data processing method, a device, a system, an electronic device and a storage medium, wherein the method comprises the following steps: determining a target service type; determining a target template according to the target service type; acquiring first data to be imported and data configuration information; and acquiring second data corresponding to the data configuration information from the first data, and importing the second data into the target template in batch to obtain a data report. The data processing method provided by the embodiment of the application determines the target template according to the target service type, and realizes that the associated template can be selected according to different service types, so that the speed and the efficiency of importing data are improved. Meanwhile, according to the data report obtained after the data are imported in batch, the data which are successfully imported, the data which are failed to import and the failure reason can be determined, and the follow-up processing is facilitated.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to a data processing method, apparatus, system, electronic device, and storage medium.
Background
At present, a service system batch import operation is generally implemented by a service system developer according to a specific service scenario, for example: the code writing can realize the operations of uploading, reading, checking and database storage. However, in the process of implementing the invention, the inventor finds that the following problems still exist:
when a scene is changed, the workload of developers is large, corresponding codes for reading the content of files and codes for connecting, communicating, operating and the like with a data storage facility need to be written for files of different formats, a large number of codes need to be written for each batch import function, and in addition, service logic codes are mixed with the codes for reading the files and operating the data storage facility, and the maintenance is not easy.
Disclosure of Invention
In order to solve the technical problems described above or at least partially solve the technical problems, the present application provides a data processing method, apparatus, system, electronic device, and storage medium.
In a first aspect, an embodiment of the present application provides a data processing method, including:
determining a target service type;
determining a target template according to the target service type;
acquiring first data to be imported and data configuration information;
and acquiring second data corresponding to the data configuration information from the first data, and importing the second data into the target template in batch to obtain a data report.
Optionally, the determining the target service type includes:
detecting a trigger operation acting on a preset control;
generating a service type option according to the triggering operation;
and determining the target service type according to the selected operation acting on the service type option.
Optionally, the method further includes:
determining a link address corresponding to the first data;
and storing the link address in a cloud server.
Optionally, the obtaining data configuration information includes:
receiving input information;
when the input information meets a preset condition, taking the input information as the data configuration information;
or, the acquiring data configuration information includes:
acquiring real-time memory information and presetting an optimal occupation ratio;
and calculating according to the real-time memory information and the optimal memory ratio according to a preset rule to obtain the data configuration information.
Optionally, the importing the first data into the target template in batch to obtain a data report includes:
acquiring a data item in the target template;
determining third data associated with the data item from the second data;
and importing the third data into the target template to obtain the data report.
Optionally, the importing the third data into the target template to obtain the data report includes:
when the third data meet preset conditions, importing the third data into the target template to obtain the data report;
the third data satisfies a preset condition, including:
data in the third data is valid data;
and the file format of the third data is a specified format.
In a second aspect, an embodiment of the present application further provides a data processing apparatus, including:
the first determining module is used for determining the target service type;
the second determining module is used for determining a target template according to the target service type;
the acquisition module is used for acquiring first data to be imported and data configuration information;
and the processing module is used for acquiring second data corresponding to the data configuration information from the first data, and importing the second data into the target template in batches to obtain a data report.
In a third aspect, an embodiment of the present application further provides a data processing system, including: a data processing device and a cloud server;
the data processing device is used for determining a target service type, determining a target template according to the target service type, acquiring first data to be imported, importing the first data into the target template in batches to obtain a data report, and sending a link address corresponding to the first data to the cloud server;
the cloud server is used for storing the link address.
In a fourth aspect, the present application provides an electronic device, comprising: the system comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the above method steps when executing the computer program.
In a fifth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the above-mentioned method steps.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: the data processing method provided by the embodiment determines the target template according to the target service type, and realizes that the associated template can be selected according to different service types, so that the speed and the efficiency of importing data are improved. Meanwhile, according to the data report obtained after the data are imported in batch, the data which are successfully imported, the data which are failed to import and the failure reason can be determined, and the follow-up processing is facilitated.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a selection interface of service types provided in an embodiment of the present application;
FIG. 3 is a target template interface provided by an embodiment of the present application;
fig. 4 is a data upload interface provided in an embodiment of the present application;
fig. 5 is a flowchart of a data processing method according to another embodiment of the present application;
fig. 6 is a block diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 7 is a block diagram of a data processing system provided by an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The application provides a data processing method, a device, a system, an electronic device and a storage medium. The method provided by the embodiment of the invention can be applied to any required electronic equipment, for example, the electronic equipment can be electronic equipment such as a server and a terminal, and the method is not particularly limited herein, and is hereinafter simply referred to as electronic equipment for convenience in description.
First, a data processing method provided in an embodiment of the present invention is described below.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S11, determining the target service type;
in this embodiment, the target service type is determined, and the determination is specifically realized by the following means: and detecting a trigger operation acting on a preset control, generating a service type option according to the trigger operation, and determining a target service type according to the selected operation acting on the service type option.
Step S12, determining a target template according to the target service type;
the data processing method provided by the embodiment determines the target template according to the target service type, and realizes that the associated template can be selected according to different service types, so that the speed and the efficiency of importing data are improved. Meanwhile, according to the data report obtained after the data are imported in batch, the data which are successfully imported, the data which are failed to import and the failure reason can be determined, and the follow-up processing is facilitated.
Taking an automobile operation system as an example, fig. 2 is a service type selection interface provided in an embodiment of the present application, as shown in fig. 2, when a trigger operation based on a preset control by a user is detected, where the trigger operation includes but is not limited to clicking or pulling, a service type option is generated according to the trigger operation, and the service type option includes: a vehicle model library, an SPU (Standard Product Unit), etc., then determine a target service type according to the selected operation acting on the service type option, and obtain a corresponding template according to the target service type (as shown in fig. 3). Or determining the input keyword and the target service type associated with the keyword, and determining the target template according to the target service type associated with the keyword.
It will be appreciated that the templates for different traffic types may be set in the database. By setting the keywords, the keywords include data names, data types and data sources. The keyword template contains a plurality of ordered keywords, and the keywords are input into the database in advance by developers and used as the basis for screening data. For example, one index data includes keywords such as an index source, an index code, and an index name, where the data name corresponds to the index name, the data type corresponds to the index code, and the data source corresponds to the index source. The data name is used as a keyword to identify the target data for the first time, the data type can improve the verification accuracy of the target data, and the data source can facilitate manual checking of the reliability of the target data.
Step S13, acquiring first data to be imported and data configuration information;
as shown in fig. 4, a trigger operation acting on a "browse import data file" button in the current interface is detected, a local file list is displayed according to the trigger operation, a selection operation acting on at least one local file in the file list is detected, a file to be imported is determined according to the selection operation, and data in the file to be imported is used as first data.
Meanwhile, after the first data are obtained, a link address corresponding to the first data is determined, and the link address is stored in the cloud server. In this embodiment, the link address is stored in the cloud server, so that when subsequently performing data check on the imported data, the original data, that is, the first data, can be checked through the link address. And the memory space of the cloud server can be reduced through the storage mode of the link address.
In this embodiment, before importing the first data into the target template, configuring the number of pieces of imported data may also be implemented, specifically, acquiring data configuration information, and acquiring second data corresponding to the data configuration information from the first data. The data configuration information is configured manually by a developer, and as one example, by accepting input information, when the number of pieces of data in the input information is less than or equal to the number of pieces of data in the first data, the input information is used as the data configuration information. The input information may be manually input by a user, and the data configuration information includes: configuration key, title, type, number of pieces of imported data, classification, and the like.
Taking an automobile operation system as an example, data items of a template of a vehicle model library are stored in a five-level relation, and data in first data are as follows:
1. the manufacturer is a popular automobile, the brand is Audi, the vehicle series is Audi A4L, the annual fee is 2013, and the vehicle type is 2013 Audi A4L 30TFSI manual comfortable type.
2. The manufacturer is an auto-comfortable type of Oddi, Auddi brand, Auddi A4L vehicle series, year 2013 and vehicle type 2013 Auddi A4L 30 TFSI.
3. The manufacturer is an Audi automobile, the brand is Audi, the vehicle series is Audi A4L, the annual fee is 2013, and the vehicle type is the Audi A4L 30TFSI automatic technical type of 2013.
4. The manufacturer is a popular automobile, the brand is Audi, the vehicle series is Audi A4L, the annual fee is 2013, and the vehicle type is 2013 and the Audi A4L 30TFSI automatic standard type.
5. The manufacturer is an Audi automobile, the brand is Audi, the vehicle series is Audi A4L, the annual fee is 2013, and the vehicle type is the automatic luxury Audi A4L 30TFSI model of 2013.
And determining that the number of the batch imported data is 3 according to the data configuration information, and using the data with the number of pieces being 1-3 in the first data as second data.
The number of the first data is configured, so that the configuration can be performed by combining the service requirement and the performance of the server, and the condition that the number of the imported data exceeds the pressure of the server to cause the server to be down or the number of the imported data is too small to meet the service requirement is prevented.
In this embodiment, besides manually configuring information, an embodiment of the present application further provides a method for acquiring data configuration information.
Specifically, after a data import request is received, determining first data to be imported, and acquiring current real-time memory information of the system, wherein the memory real-time information at least carries the size of a physical memory, real-time memory and real-time memory ratio, the real-time memory is the memory occupied by the current physical memory and virtual memory of the system, and the real-time memory ratio is the ratio of the current process of the system to all the memories;
then obtaining the optimal occupation ratio preset by the system, and obtaining the optimal import number corresponding to the optimal occupation ratio through a preset rule according to the current real-time memory information of the system; and then obtaining fourth data corresponding to the optimal import number in batches from the first data. The method comprises the steps of obtaining the optimal number of the import pieces corresponding to the optimal occupation ratio through a preset rule, determining the current bearing capacity of the system according to the optimal occupation ratio, and determining the optimal number of the import pieces according to the bearing capacity.
For example: the preset optimal occupation ratio of the system is 80%, and when the current real-time memory information of the system is acquired: the physical memory size is 4096MB, the real-time occupancy is 2880MB, and the real-time occupancy ratio is 70.31%, then the optimal occupancy ratio (80%) is greater than the real-time occupancy ratio (70.31%), then the current load-carrying capacity of the system is: 4096MB by 80% -2880MB by 396.8 MB.
Then, the size of each piece of data in the first data is obtained, such as: the first data includes: 200 pieces of data, each piece of data is 2.5MB, and the optimal number of leading-in pieces is calculated according to the bearing capacity determination and the size of each piece of data as follows: 158 pieces. 158 pieces of data are sequentially selected as the second data according to the data numbers in the first data.
It should be noted that, before the system acquires the data to be imported each time, the usage of the current memory of the system is detected in advance, so as to acquire the real-time import data corresponding to the current memory. For example, the physical memory size of the system is 512MB, the optimal memory ratio is 80%, the memory ratio of the system at this time is 50%, the current load capacity of the system is 153.6MB, and the obtained import data is 153.6 MB; when the next memory ratio of the system is 60%, the next bearing capacity of the system is 102.4MB, and the obtained import data is 102.4MB, it should be understood that the example of obtaining real-time import data in batches is only for convenience of understanding, that is, the real-time import data obtained in batches can be dynamically changed according to the current memory ratio of the system.
In addition, after the steps of acquiring the historical average import quantity within a preset time period and an import record meeting the historical average import quantity in the historical import task, acquiring the high-efficiency import quantity with the highest import efficiency and the high-efficiency occupation quantity corresponding to the high-efficiency import quantity in the import record, and acquiring the optimal import quantity corresponding to the optimal occupation ratio through a preset rule according to the current real-time memory information of the system, the following steps are required:
and judging whether the optimal occupied storage corresponding to the size of the physical memory is larger than the efficient occupied storage, if not, starting the virtual memory, and importing the real-time import data acquired each time into the system database in batches through the size of the physical memory and the virtual memory.
The embodiment can acquire the current memory use condition of the system through the acquired real-time memory information; acquiring the optimal occupation ratio preset by the system; according to the current real-time memory information of the system, the optimal number of imports corresponding to the optimal occupation ratio is obtained through a preset rule, and the optimal number of imports is calculated through the preset optimal occupation ratio, so that the program application can keep an efficient and smooth running state in the data importing process of the system, and the number of imports is maximized; acquiring real-time import data corresponding to the optimal import number in batches from the data to be imported; and leading the real-time lead-in data acquired each time into a system database in batches.
The real-time import data corresponding to the optimal import number are imported into the system database in batches, so that the import operation of mass data is realized, the data import time is greatly shortened when the system imports large-scale data, the data import efficiency is improved, and meanwhile, the system memory is kept in the state of the optimal storage ratio, so that the application program can be operated efficiently and smoothly, and the problem that the data cannot be imported into the system due to insufficient memory is avoided.
And step S14, acquiring second data corresponding to the data configuration information from the first data, and importing the second data into the target template in batch to obtain a data report.
In this embodiment, after receiving the data import request and determining second data corresponding to the data configuration information from the first data, the data item in the target template is obtained, third data associated with the data item is determined from the second data, and the third data is imported into the target template to obtain the data report. As in the above example, the data items include: manufacturer, vehicle series, brand, year, model. Or, when the target template is an SPU template, the data item includes: merchant number, merchant name, vehicle type ID, sales guide price, purchase tax, etc.
In this embodiment, when the third data is imported into the target template, data verification needs to be performed on the third data, and the verification mode is specifically implemented in the following manner: and carrying out format verification on the third data, carrying out duplication checking verification on the third data and carrying out validity verification on the third data.
For example: in the data item: and the manufacturer, the vehicle series, the brand, the annual payment and the vehicle type are in a five-level relation, the duplication elimination check is carried out on each level, and when the third data meet the preset conditions, the third data are imported into the target template to obtain a data report. The third data satisfies a preset condition, including: the data in the third data is valid data, and the file format of the third data is a specified format.
During the analytic verification process, the inventor finds that the third data associated with each data item may include hundreds of data, for example: the data associated with the data item of the vehicle type has a lot of data, so when the data are simultaneously imported into the target template, the analysis and verification time of the system in the import process is very long, corresponding results cannot be given immediately, even the system reports errors due to overtime, and a worker cannot confirm whether the import is successful and whether all analysis and verification of uploaded files are completed, so that the user experience is poor, and the operation pressure is high.
In order to solve the above problem, when multiple data are imported simultaneously, the multiple imported data may be cached and form a resolution check queue, for example, the multiple imported data may be stored in a temporary database, and the multiple imported data may be data corresponding to the same data item.
In the embodiment, the imported data are read one by one according to the queue sequence of the analysis and verification queue, and the corresponding analysis and verification rules are called to perform analysis and verification on the imported data. In this embodiment, the imported data may be analyzed and checked one by one, that is, after one imported data is analyzed and checked, another imported data is analyzed and checked.
Meanwhile, a timing task can be configured during analysis and verification, the timing task is configured to read one imported data at intervals of a preset period according to the queue sequence of the analysis and verification queue to perform analysis and verification, and after a plurality of imported data are received, the analysis and verification of the data in the uploaded file can be started according to the timing task at the background, so that the confusion of the analysis and verification process can be avoided.
Fig. 5 is a flowchart of a data processing method according to another embodiment of the present application. As shown in fig. 5, the method further comprises the steps of:
step S51, determining the import failure type of the third data according to the data report;
in step S52, a corresponding processing operation is performed according to the import failure type.
In this embodiment, which data in the third data fails to be imported, which data is successfully imported, and a data import failure type can be determined according to the data report. And corresponding processing operation is executed according to the import failure type, so that the time for processing the failure data by developers is saved.
As an example, when at least one data in the third data fails to be imported in the import process, a preset logistic regression model method is called to search for the data that fails to be imported, the number of pieces corresponding to the data that fails to be imported is recorded, a data report is generated by the number of pieces and the import failure type, and then a processing operation is performed according to the import failure type, where the processing operation may be: and deleting the data failed to be imported, or sending the data failed to be imported to a preset terminal, and modifying the data by a developer.
For example, when a certain field of a certain piece of data in the third data is too long, the certain field appears in the importing process, the importing is failed at this time, that is, an exception is encountered, a set of processing mechanism is introduced, it is required to determine which piece of data has a problem, and record the data, a preset bisection algorithm is called to locate the abnormal data, the data importing is continued, half of the data without the problem is submitted each time of bisection, and the half of the data with the problem is continued to be bisected, and the target data is recorded after the target data is found and removed from the data until all the data to be imported are completely imported.
It should be noted that, in this embodiment, the possible exceptions include: violation of unique constraints (some field values require no duplication with existing data in the database), violation of foreign key constraints (the value of a field must be one of a set of a field value in another table), database join failure, field over-length, mismatch between the data type of a field and the type of a data item, etc.
It can be understood that if the database is abnormal, the system can throw exception, the exception is not captured after being thrown, the operation is stopped and cannot be continued, if the exception is caught and not processed, the target template is not imported clearly and successfully, and the staff mistakenly thinks that the import is successful because the staff does not receive any exception prompt. Therefore, in this embodiment, the data that fails to be imported is found out, and the reason for the failure is recorded, which is to say, the exception is captured and then processed, so that the worker can know which data is successfully imported and which data is failed to be imported.
By the method, when the batch imported data is abnormal, the abnormal data is removed, and the normal data of the same batch of data is imported again, so that the same batch of data is prevented from rolling back, and the data import efficiency is improved.
Fig. 6 is a block diagram of a data processing apparatus provided in an embodiment of the present application, which may be implemented as part of or all of an electronic device through software, hardware, or a combination of the two. As shown in fig. 6, the data processing apparatus includes:
a first determining module 61, configured to determine a target service type;
a second determining module 62, configured to determine a target template according to the target service type;
an obtaining module 63, configured to obtain first data to be imported and data configuration information;
and the processing module 64 is configured to obtain second data corresponding to the data configuration information from the first data, and import the second data into the target template in batch to obtain a data report.
The first determining module 61 in this embodiment is specifically configured to detect a trigger operation acting on a preset control; generating a service type option according to the triggering operation; and determining the target service type according to the selected operation acting on the service type option.
The device in this embodiment further includes a third determining module, where the third determining module is specifically configured to determine a link address corresponding to the first data, and store the link address in the cloud server.
The device in the embodiment further comprises a screening module, wherein the screening module is used for acquiring data configuration information; and acquiring second data corresponding to the data configuration information from the first data.
The processing module 64 in this embodiment is specifically configured to obtain a data item in the target template; determining third data associated with the data item from the second data; and importing the third data into the target template to obtain a data report.
When the second data meet preset conditions, importing the second data into a target template to obtain a data report; the second data meets a preset condition and comprises: the data in the second data is valid data; the file format of the second data is a specified format.
Fig. 7 is a block diagram of a data processing system according to an embodiment of the present application, where the data processing system according to the embodiment includes: a data processing device 71 and a cloud server 72;
the data processing device 71 is configured to determine a target service type, determine a target template according to the target service type, acquire first data to be imported, import the first data into the target template in batch to obtain a data report, and send a link address corresponding to the first data to the cloud server;
and the cloud server 72 is used for storing the link address.
An embodiment of the present application further provides an electronic device, as shown in fig. 8, the electronic device may include: the system comprises a processor 1501, a communication interface 1502, a memory 1503 and a communication bus 1504, wherein the processor 1501, the communication interface 1502 and the memory 1503 complete communication with each other through the communication bus 1504.
A memory 1503 for storing a computer program;
the processor 1501 is configured to implement the steps of the above embodiments when executing the computer program stored in the memory 1503.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the above-described embodiments.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including AN object oriented programming language such as Java, C + +, or the like, as well as conventional procedural programming languages, such as the "C" language or similar programming languages.
It should be noted that, for the above-mentioned apparatus, electronic device and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.
It is further noted that, herein, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data processing method, comprising:
determining a target service type;
determining a target template according to the target service type;
acquiring first data to be imported and data configuration information;
and acquiring second data corresponding to the data configuration information from the first data, and importing the second data into the target template in batch to obtain a data report.
2. The method of claim 1, wherein the determining a target traffic type comprises:
detecting a trigger operation acting on a preset control;
generating a service type option according to the triggering operation;
and determining the target service type according to the selected operation acting on the service type option.
3. The method of claim 1, further comprising:
determining a link address corresponding to the first data;
and storing the link address in a cloud server.
4. The method of claim 1, wherein the obtaining data configuration information comprises:
receiving input information;
when the input information meets a preset condition, taking the input information as the data configuration information;
or, the acquiring data configuration information includes:
acquiring real-time memory information and presetting an optimal occupation ratio;
and calculating according to the real-time memory information and the optimal memory ratio according to a preset rule to obtain the data configuration information.
5. The method of claim 1, wherein the importing the second data into the target template in bulk to obtain a data report comprises:
acquiring a data item in the target template;
determining third data associated with the data item from the second data;
and importing the third data into the target template to obtain the data report.
6. The method of claim 5, wherein importing the third data into the target template to obtain the data report comprises:
when the third data meet preset conditions, importing the third data into the target template to obtain the data report;
the third data satisfies a preset condition, including:
data in the third data is valid data;
and the file format of the third data is a specified format.
7. A data processing apparatus, comprising:
the first determining module is used for determining the target service type;
the second determining module is used for determining a target template according to the target service type;
the acquisition module is used for acquiring first data to be imported and data configuration information;
and the processing module is used for acquiring second data corresponding to the data configuration information from the first data, and importing the second data into the target template in batches to obtain a data report.
8. A data processing system, comprising: a data processing device and a cloud server;
the data processing device is used for determining a target service type, determining a target template according to the target service type, acquiring first data to be imported, importing the first data into the target template in batches to obtain a data report, and sending a link address corresponding to the first data to the cloud server;
the cloud server is used for storing the link address.
9. An electronic device, comprising: the system comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor, when executing the computer program, implementing the method steps of any of claims 1-6.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method steps of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010238784.7A CN111460773A (en) | 2020-03-30 | 2020-03-30 | Data processing method, device and system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010238784.7A CN111460773A (en) | 2020-03-30 | 2020-03-30 | Data processing method, device and system, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111460773A true CN111460773A (en) | 2020-07-28 |
Family
ID=71683340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010238784.7A Pending CN111460773A (en) | 2020-03-30 | 2020-03-30 | Data processing method, device and system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111460773A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111949746A (en) * | 2020-08-18 | 2020-11-17 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and computer readable medium |
| CN112965694A (en) * | 2021-02-26 | 2021-06-15 | 上海微盟企业发展有限公司 | Data processing method, device, equipment and storage medium |
| CN113010742A (en) * | 2021-03-01 | 2021-06-22 | 歌尔微电子股份有限公司 | Data processing method, device, equipment and medium |
| CN113239019A (en) * | 2021-06-03 | 2021-08-10 | 锋泾（中国）建材集团有限公司 | Data migration method and system between ERP systems, electronic device and medium |
| CN113704269A (en) * | 2021-09-02 | 2021-11-26 | 京东城市(北京)数字科技有限公司 | Data processing method, system, storage medium and electronic equipment |
| CN113778999A (en) * | 2021-09-29 | 2021-12-10 | 平安资产管理有限责任公司 | Data modularization processing method and device, computer equipment and storage medium |
| CN114185982A (en) * | 2021-12-10 | 2022-03-15 | 建信金融科技有限责任公司 | Data export method, device, equipment and storage medium |
| CN114817267A (en) * | 2022-05-16 | 2022-07-29 | 通联支付网络服务股份有限公司 | Method, apparatus and medium for generating data report |
| WO2023246008A1 (en) * | 2022-06-24 | 2023-12-28 | 北京京东振世信息技术有限公司 | Data processing method, apparatus, and storage medium |
| CN117829117A (en) * | 2023-12-29 | 2024-04-05 | 佛山众陶联供应链服务有限公司 | Method and system for collecting pound bill information through Android pda positioning and template matching |
| CN117875881A (en) * | 2023-12-27 | 2024-04-12 | 广东艾博电力设计院(集团)有限公司 | Distribution project data generation method, system, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105447174A (en) * | 2015-12-09 | 2016-03-30 | 中国农业银行股份有限公司 | Report generation method and apparatus |
| CN105808375A (en) * | 2014-12-31 | 2016-07-27 | 航天信息软件技术有限公司 | Data import method and device |
| CN106055315A (en) * | 2016-05-19 | 2016-10-26 | 乐视控股（北京）有限公司 | Method and system for processing batched tasks |
| CN108874760A (en) * | 2018-05-23 | 2018-11-23 | 中国平安人寿保险股份有限公司 | Report file generation method, device, computer equipment and storage medium |
| CN109783426A (en) * | 2018-12-14 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Acquire method, apparatus, computer equipment and the storage medium of data |
| CN110502558A (en) * | 2019-07-23 | 2019-11-26 | 平安科技（深圳）有限公司 | Data export method, device, computer equipment and storage medium |
| CN110865883A (en) * | 2019-09-23 | 2020-03-06 | 平安科技（深圳）有限公司 | Mass data importing method and device, computer equipment and storage medium |
| CN110866383A (en) * | 2019-10-15 | 2020-03-06 | 中国直升机设计研究所 | Interactive electronic data list generation method and system |
-
2020
- 2020-03-30 CN CN202010238784.7A patent/CN111460773A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105808375A (en) * | 2014-12-31 | 2016-07-27 | 航天信息软件技术有限公司 | Data import method and device |
| CN105447174A (en) * | 2015-12-09 | 2016-03-30 | 中国农业银行股份有限公司 | Report generation method and apparatus |
| CN106055315A (en) * | 2016-05-19 | 2016-10-26 | 乐视控股（北京）有限公司 | Method and system for processing batched tasks |
| CN108874760A (en) * | 2018-05-23 | 2018-11-23 | 中国平安人寿保险股份有限公司 | Report file generation method, device, computer equipment and storage medium |
| CN109783426A (en) * | 2018-12-14 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Acquire method, apparatus, computer equipment and the storage medium of data |
| CN110502558A (en) * | 2019-07-23 | 2019-11-26 | 平安科技（深圳）有限公司 | Data export method, device, computer equipment and storage medium |
| CN110865883A (en) * | 2019-09-23 | 2020-03-06 | 平安科技（深圳）有限公司 | Mass data importing method and device, computer equipment and storage medium |
| CN110866383A (en) * | 2019-10-15 | 2020-03-06 | 中国直升机设计研究所 | Interactive electronic data list generation method and system |
Non-Patent Citations (1)
| Title |
|---|
| 叶彬: "中文Access问题与技巧精解", 清华大学出版社, pages: 198 - 199 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111949746A (en) * | 2020-08-18 | 2020-11-17 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and computer readable medium |
| CN112965694A (en) * | 2021-02-26 | 2021-06-15 | 上海微盟企业发展有限公司 | Data processing method, device, equipment and storage medium |
| CN113010742A (en) * | 2021-03-01 | 2021-06-22 | 歌尔微电子股份有限公司 | Data processing method, device, equipment and medium |
| CN113239019A (en) * | 2021-06-03 | 2021-08-10 | 锋泾（中国）建材集团有限公司 | Data migration method and system between ERP systems, electronic device and medium |
| CN113704269A (en) * | 2021-09-02 | 2021-11-26 | 京东城市(北京)数字科技有限公司 | Data processing method, system, storage medium and electronic equipment |
| CN113704269B (en) * | 2021-09-02 | 2024-04-16 | 京东城市(北京)数字科技有限公司 | Data processing method, system, storage medium and electronic equipment |
| CN113778999A (en) * | 2021-09-29 | 2021-12-10 | 平安资产管理有限责任公司 | Data modularization processing method and device, computer equipment and storage medium |
| CN114185982A (en) * | 2021-12-10 | 2022-03-15 | 建信金融科技有限责任公司 | Data export method, device, equipment and storage medium |
| CN114817267A (en) * | 2022-05-16 | 2022-07-29 | 通联支付网络服务股份有限公司 | Method, apparatus and medium for generating data report |
| WO2023246008A1 (en) * | 2022-06-24 | 2023-12-28 | 北京京东振世信息技术有限公司 | Data processing method, apparatus, and storage medium |
| CN117875881A (en) * | 2023-12-27 | 2024-04-12 | 广东艾博电力设计院(集团)有限公司 | Distribution project data generation method, system, electronic equipment and storage medium |
| CN117829117A (en) * | 2023-12-29 | 2024-04-05 | 佛山众陶联供应链服务有限公司 | Method and system for collecting pound bill information through Android pda positioning and template matching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111460773A (en) | Data processing method, device and system, electronic equipment and storage medium | |
| CN110060139B (en) | Accounting processing method and device | |
| JP6920378B2 (en) | Repair board detectors, methods and computer readable storage media | |
| CN112199277B (en) | Defect reproduction method, device, equipment and storage medium based on browser | |
| CN112215230B (en) | Information auditing method and device and electronic equipment | |
| CN110377356A (en) | Task processing method, device, system and computer readable storage medium | |
| CN112286790A (en) | Full link test method, device, equipment and storage medium | |
| CN108629699B (en) | Data uploading method, data uploading equipment, storage medium and device | |
| CN110428156B (en) | Method and device for checking service data and electronic equipment | |
| CN111045983B (en) | Nuclear power station electronic file management method, device, terminal equipment and medium | |
| CN111367703A (en) | Troubleshooting method and device | |
| CN109376285B (en) | Data sorting verification method based on json format, electronic device and medium | |
| CN116777653A (en) | Enterprise collection checking method, device, equipment and readable storage medium | |
| CN114039878B (en) | Network request processing method and device, electronic equipment and storage medium | |
| CN110796178A (en) | Decision model training method, sample feature selection method, device and electronic equipment | |
| CN113296912B (en) | Task processing method, device, system, storage medium and electronic equipment | |
| CN111198798B (en) | Service stability measuring method and device | |
| CN112488562B (en) | Service realization method and device | |
| CN111242256B (en) | Information verification method, device and terminal | |
| CN111274147B (en) | Automatic script testing method and device, electronic equipment and readable medium | |
| CN110018844B (en) | Management method and device of decision triggering scheme and electronic equipment | |
| CN111414427A (en) | Data processing method and device suitable for quasi-real-time service | |
| CN110956552A (en) | Insurance problem processing method, device, equipment and storage medium | |
| CN114063871B (en) | Data processing method and device, electronic equipment and readable medium | |
| CN113807923B (en) | Order processing method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |