CN114020760A

CN114020760A - Data processing method and related device

Info

Publication number: CN114020760A
Application number: CN202111300841.0A
Authority: CN
Inventors: 郭锦帅; 张琰; 刘宽宏; 林灿发; 谢昭杰
Original assignee: Bank of China Ltd
Current assignee: Bank of China Ltd
Priority date: 2021-11-04
Filing date: 2021-11-04
Publication date: 2022-02-08

Abstract

The embodiment of the application discloses a data processing method and a related device, which can extract user data stored in a target database into a memory, wherein the user data comprises user numbers of stored users, and then can acquire user data to be updated, and the user data to be updated is used for updating into the target database. In order to improve the data updating efficiency, the user data to be updated can be screened in the memory according to the user number corresponding to the user data to be updated and the user number of the stored user, the target user data to be updated corresponding to the stored user in the user data to be updated is determined, and then the target database is updated according to the target user data to be updated. Because the data reading speed of the memory is far higher than that of the database, the data processing efficiency can be improved and the time consumption can be reduced by screening the data through the memory.

Description

Data processing method and related device

Technical Field

The present application relates to the field of data technologies, and in particular, to a data processing method and a related apparatus.

Background

Banks are closely related to daily life of people, and for example, financial management, business handling and the like of people require the participation of the banks. When the operations such as property transaction, business transaction and the like are carried out, certain user information of the operation user is required, so that the user information can be accurately and reasonably managed, and the safety and the user experience of the user information are directly influenced.

In the related art, user data is generally stored in a database, however, since the reading speed of the database interface is slow, a lot of time and effort are required to update the user data, and the data processing efficiency is low.

Disclosure of Invention

In order to solve the technical problem, the application provides a data processing method, and a processing device can extract database data with a small data volume into a memory for analysis, so that the processing of user data to be updated is efficiently completed by utilizing the characteristic of high data reading speed in the memory, and the data processing efficiency is improved.

The embodiment of the application discloses the following technical scheme:

in a first aspect, an embodiment of the present application discloses a data processing method, where the method includes:

extracting stored user data in a target database into a memory, wherein the user data comprises user numbers of stored users;

acquiring user data to be updated;

screening the user data to be updated in the memory according to the user number corresponding to the user data to be updated and the user number of the stored user, and determining target user data to be updated corresponding to the stored user in the user data to be updated;

and updating the target database according to the target user data to be updated.

In a possible implementation manner, the extracting the stored user data in the target database to the memory includes:

dividing the stored user data into N groups of data groups according to the user number of the stored user;

and extracting the N groups of data groups into the memory in parallel.

In a possible implementation manner, the updating the target database according to the target to-be-updated user data includes:

determining a data category corresponding to the target user data to be updated;

and updating the target database according to the data type and the target to-be-updated user data.

In one possible implementation, the data category includes any one or combination of identity data, document data, household data, and institution data.

In a possible implementation manner, the updating the target database according to the data category and the target to-be-updated user data includes:

and based on the data category, the target database is updated in parallel according to the target user data to be updated.

In a second aspect, an embodiment of the present application discloses a data processing apparatus, which includes an extracting unit, an obtaining unit, a determining unit, and an updating unit:

the extracting unit is used for extracting the stored user data in the target database into the memory, wherein the user data comprises the user number of the stored user;

the acquisition unit is used for acquiring user data to be updated;

the determining unit is configured to screen the user data to be updated in the memory according to the user number corresponding to the user data to be updated and the user number of the stored user, and determine target user data to be updated corresponding to the stored user in the user data to be updated;

and the updating unit is used for updating the target database according to the target to-be-updated user data.

In a possible implementation manner, the extracting unit is specifically configured to:

and extracting the N groups of data groups into the memory in parallel.

In a possible implementation manner, the updating unit is specifically configured to:

In one possible implementation, the apparatus includes a processor and a memory:

the memory is used for storing program codes and transmitting the program codes to the processor;

the processor is configured to perform the data processing method of any one of the first aspect according to instructions in the program code.

In one possible implementation, the computer-readable storage medium is configured to store a computer program, where the computer program is configured to execute the data processing method according to any one of the first aspect.

According to the technical scheme, the data processing method can be used for extracting the user data stored in the target database into the memory, the user data comprises the user number of the stored user, then the user data to be updated can be obtained, and the user data to be updated is used for being updated into the target database. In order to improve the data updating efficiency, the user data to be updated can be screened in the memory according to the user number corresponding to the user data to be updated and the user number of the stored user, the target user data to be updated corresponding to the stored user in the user data to be updated is determined, and then the target database is updated according to the target user data to be updated. Because the data reading speed of the memory is far higher than that of the database, the data processing efficiency can be improved and the time consumption can be reduced by screening the data through the memory.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;

fig. 2 is a schematic diagram of a data processing method according to an embodiment of the present application;

fig. 3 is a schematic diagram of a data processing method according to an embodiment of the present application;

fig. 4 is a schematic diagram of a data processing method in an actual application scenario according to an embodiment of the present application;

fig. 5 is a block diagram of a data processing apparatus according to an embodiment of the present application;

fig. 6 is a block diagram of a computer device according to an embodiment of the present disclosure;

fig. 7 is a block diagram of a server according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the accompanying drawings.

In the related art, when data in a database is updated, user data to be updated needs to be read into the database one by one for comparison of user numbers, and when the database has the user number of the user data to be updated, the user data to be updated is updated into the database. Since the I/O structure of the database is slow to read, the data update takes a lot of time.

It is understood that the method may be applied to a processing device, which is a processing device capable of data processing, for example, a terminal device or a server having a data processing function. The method can be independently executed through the terminal equipment or the server, can also be applied to a network scene of communication between the terminal equipment and the server, and is executed through the cooperation of the terminal equipment and the server. The terminal device may be a computer, a mobile phone, or the like. The server may be understood as an application server or a Web server, and in actual deployment, the server may be an independent server or a cluster server.

Next, a data processing method provided by an embodiment of the present application will be described with reference to the drawings.

Referring to fig. 1, fig. 1 is a flowchart of a data processing method provided in an embodiment of the present application, where the method includes:

s101: and extracting the stored user data in the target database to the memory.

The target database is used for storing bank user data, the stored user data refers to data of a user corresponding to the bank, and the stored user data is stored in the target database. The user data includes a user number of a stored user, the stored user is a user having corresponding data in the data, the user number is used to identify the stored user, and the data of the stored user has a corresponding user number.

S102: and acquiring the user data to be updated.

The user data to be updated refers to data for updating the target database.

S103: and screening the user data to be updated in the memory according to the user number corresponding to the user data to be updated and the user number of the stored user, and determining the target user data to be updated corresponding to the stored user in the user data to be updated.

According to the user number corresponding to the user data to be updated and the user number of the stored user, the processing device may determine whether the user data to be updated is the data of the user corresponding to the target database, so that the target user data to be updated, which is used for updating the target database, may be screened out, where the target user data to be updated refers to the data of the user corresponding to the target database. When screening is performed, the memory has a faster data reading speed because the stored user data is in the memory, and thus the screening speed can be faster when screening data.

S104: and updating the target database according to the target user data to be updated.

After determining the target user data to be updated, the processing device may update the data to the target database, and complete the update of the user data to be updated.

In order to increase the data extraction speed when extracting data of the target database, in one possible implementation manner, the processing device may divide the stored user data into N groups of data sets according to the user number of the stored user, and then extract the N groups of data sets into the memory in parallel, thereby increasing the data extraction speed.

As shown in fig. 2, serial fetching of data to memory is also fast, assuming a 50-thousand data volume in the target database, but for performance, can be done in parallel per user number. The processing equipment can determine the maximum user number and the minimum user number in the database, then averagely divides the maximum user number and the minimum user number into n groups, can simply calculate the maximum value and the minimum value of the user number of each group, starts n processes, and extracts data of one number section interval to a memory hashmap according to the number section interval corresponding to the user number. For example, if the maximum subscriber number is 101 ten thousands and the minimum subscriber number is 1 ten thousands, and if the number is divided into 5 groups, the average number per group is (101-1)/5, which is 20 ten thousands, then the maximum subscriber number and the minimum subscriber number per group are approximately: 1 ten thousand to 21 ten thousand, 21 ten thousand to 41 ten thousand, 41 ten thousand to 61 ten thousand, 61 ten thousand to 81 ten thousand and 81 ten thousand to 101 ten thousand, then 5 processes are started,

the process 1 extracts the user numbers with the number segment interval of 1 ten thousand to 21 ten thousand to the memory hashmap _1,

the process 2 extracts the user numbers with the number segment interval of 21-41 ten thousand to the memory hashmap _2,

……

and the process 5 extracts the user numbers with the number segment interval of 81-101 ten thousand from the memory hashmap _ 5.

In addition, when updating the database, in order to ensure accuracy of data update, in one possible implementation manner, the processing device may determine a data category corresponding to the target user data to be updated, and then update the target database according to the data category and the target user data to be updated, so that the data of the corresponding category can be updated to a corresponding position in the target database. The data category may include any one or combination of identity data, certificate data and household data mechanism data. For example, the identity data may store the user number, name of the user; the certificate data can store the user number, the certificate type and the certificate number of the user; the household registration data can store the user number, country and user type of the user; the organization data may store a user number, organization number, etc. of the user.

Also, to further improve data update efficiency, in one possible implementation, the processing device may be based on the data category. And updating the target database in parallel according to the target user data to be updated. As shown in fig. 3, the data category may include X-type data, Y-type data, Z-type data, and P-type data, where X-type data is identity data, Y-type data is certificate data, Z-type data is family data, and P-type data is organization data. The processing device may update the data in parallel to the database.

In order to facilitate understanding of the data processing method provided in the embodiment of the present application, the data processing method provided in the embodiment of the present application will be described in conjunction with an actual application scenario.

Referring to fig. 4, fig. 4 is a schematic diagram of a data processing method in a practical application scenario provided by the embodiment of the present application, where data with X, Y, Z, P letters respectively represent various types of data. The processing device may first store the data in the memory in parallel according to the user number corresponding to the data in the database, then retrieve the user number corresponding to the data to be updated in the memory, and if the user number exists in the memory, determine the data as valid data and determine the data as the target user data to be updated; if not, the data is deleted. After determining the target user data to be updated, the processing device may update the data to the target database in parallel according to the data type, so as to complete data update.

When data in the target database is extracted, the processing equipment can extract the data in series or in parallel; the memory may be fetched into one memory, or may be fetched into multiple memories, which is not limited herein.

Based on the data processing method provided by the embodiment, the embodiment of the application further provides a data processing device. Referring to fig. 5, fig. 5 is a block diagram of a data processing apparatus 500 according to an embodiment of the present application, where the apparatus 500 includes an extracting unit 501, an obtaining unit 502, a determining unit 503, and an updating unit 504:

an extracting unit 501, configured to extract user data stored in a target database into a memory, where the user data includes a user number of a stored user;

acquiring user data to be updated;

In a possible implementation manner, the extracting unit 501 is specifically configured to:

and extracting the N groups of data groups into the memory in parallel.

In a possible implementation manner, the updating unit 504 is specifically configured to:

In one possible implementation, the data category includes any one or combination of identity data, certificate data, household data, and institution data.

The embodiment of the application also provides computer equipment which is described in the following with reference to the attached drawings. Referring to fig. 6, an embodiment of the present application provides a device, which may also be a terminal device, where the terminal device may be any intelligent terminal including a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), a Point of Sales (POS), a vehicle-mounted computer, and the terminal device is taken as the mobile phone as an example:

fig. 6 is a block diagram illustrating a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Referring to fig. 6, the handset includes: a Radio Frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a Wireless Fidelity (WiFi) module 770, a processor 780, and a power supply 790. Those skilled in the art will appreciate that the handset configuration shown in fig. 6 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile phone in detail with reference to fig. 6:

the RF circuit 710 may be used for receiving and transmitting signals during information transmission and reception or during a call, and in particular, receives downlink information of a base station and then processes the received downlink information to the processor 780; in addition, the data for designing uplink is transmitted to the base station. In general, the RF circuit 710 includes, but is not limited to, an antenna, at least one Amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuit 710 may also communicate with networks and other devices via wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), and the like.

The memory 720 may be used to store software programs and modules, and the processor 780 may execute various functional applications and data processing of the cellular phone by operating the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 720 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 730 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the cellular phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also referred to as a touch screen, can collect touch operations of a user (e.g. operations of the user on or near the touch panel 731 by using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 731 may include two portions of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts it to touch point coordinates, and sends the touch point coordinates to the processor 780, and can receive and execute commands from the processor 780. In addition, the touch panel 731 may be implemented by various types, such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 730 may include other input devices 732 in addition to the touch panel 731. In particular, other input devices 732 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 740 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The Display unit 740 may include a Display panel 741, and optionally, the Display panel 741 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 731 can cover the display panel 741, and when the touch panel 731 detects a touch operation on or near the touch panel 731, the touch operation is transmitted to the processor 780 to determine the type of the touch event, and then the processor 780 provides a corresponding visual output on the display panel 741 according to the type of the touch event. Although in fig. 6, the touch panel 731 and the display panel 741 are two independent components to implement the input and output functions of the mobile phone, in some embodiments, the touch panel 731 and the display panel 741 may be integrated to implement the input and output functions of the mobile phone.

The handset may also include at least one sensor 750, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that adjusts the brightness of the display panel 741 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 741 and/or a backlight when the mobile phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the posture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, further description is omitted here.

Audio circuitry 760, speaker 761, and microphone 762 may provide an audio interface between a user and a cell phone. The audio circuit 760 can transmit the electrical signal converted from the received audio data to the speaker 761, and the electrical signal is converted into a sound signal by the speaker 761 and output; on the other hand, the microphone 762 converts the collected sound signal into an electric signal, converts the electric signal into audio data after being received by the audio circuit 760, and then processes the audio data output processor 780, and then transmits the audio data to, for example, another cellular phone through the RF circuit 710, or outputs the audio data to the memory 720 for further processing.

WiFi belongs to short-distance wireless transmission technology, and the mobile phone can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 770, and provides wireless broadband Internet access for the user. Although fig. 6 shows the WiFi module 770, it is understood that it does not belong to the essential constitution of the handset, and can be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 780 is a control center of the mobile phone, connects various parts of the entire mobile phone by using various interfaces and lines, and performs various functions of the mobile phone and processes data by operating or executing software programs and/or modules stored in the memory 720 and calling data stored in the memory 720, thereby integrally monitoring the mobile phone. Optionally, processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 780.

The handset also includes a power supply 790 (e.g., a battery) for powering the various components, which may preferably be logically coupled to the processor 780 via a power management system, so that the power management system may be used to manage charging, discharging, and power consumption.

Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which are not described herein.

In this embodiment, the processor 780 included in the terminal device further has the following functions:

acquiring user data to be updated;

Referring to fig. 7, fig. 7 is a block diagram of a server 800 provided in this embodiment, and the server 800 may have a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 822 (e.g., one or more processors) and a memory 832, and one or more storage media 830 (e.g., one or more mass storage devices) storing an application 842 or data 844. Memory 832 and storage medium 830 may be, among other things, transient or persistent storage. The program stored in the storage medium 830 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, a central processor 822 may be provided in communication with the storage medium 830 for executing a series of instruction operations in the storage medium 830 on the server 800.

The Server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input-output interfaces 858, and/or one or more operating systems 841, such as a Windows Server^TM，Mac OS X^TM，Unix^TM,Linux^TM，FreeBSD^TMAnd so on.

The steps performed by the server in the above embodiments may be based on the server structure shown in fig. 7.

The embodiment of the present application further provides a computer-readable storage medium for storing a computer program, where the computer program is used to execute any one implementation manner of the data processing method described in the foregoing embodiments.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium may be at least one of the following media: various media that can store program codes, such as read-only memory (ROM), RAM, magnetic disk, or optical disk.

It should be noted that, in the present specification, all the embodiments are described in a progressive manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus and system embodiments, since they are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The above description is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of data processing, the method comprising:

acquiring user data to be updated;

2. The method of claim 1, wherein the retrieving the stored user data in the target database into the memory comprises:

and extracting the N groups of data groups into the memory in parallel.

3. The method of claim 1, wherein the updating the target database according to the target user data to be updated comprises:

4. The method of claim 3, wherein the data category comprises any one or more of identity data, certificate data, household data, and institution data.

5. The method of claim 3, wherein the updating the target database according to the data category and the target user data to be updated comprises:

6. A data processing apparatus characterized by comprising an extracting unit, an obtaining unit, a determining unit, and an updating unit:

the acquisition unit is used for acquiring user data to be updated;

7. The apparatus according to claim 6, wherein the extraction unit is specifically configured to:

and extracting the N groups of data groups into the memory in parallel.

8. The apparatus according to claim 6, wherein the updating unit is specifically configured to:

9. A computer device, the device comprising a processor and a memory:

the processor is configured to perform the data processing method of any one of claims 1 to 5 according to instructions in the program code.

10. A computer-readable storage medium for storing a computer program for executing the data processing method of any one of claims 1 to 5.