WO2020140645A1

WO2020140645A1 - Abnormal data provision detection method and apparatus based on data migration, and terminal device

Info

Publication number: WO2020140645A1
Application number: PCT/CN2019/120598
Authority: WO
Inventors: 周之浩
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2019-01-03
Filing date: 2019-11-25
Publication date: 2020-07-09
Also published as: CN109840160A

Abstract

Provided are an abnormal data provision detection method and apparatus based on data migration, and a terminal device, which relate to the technical field of cloud storage. The method comprises: by means of scanning source tables and a hive table corresponding to each of the source tables, determining whether there is a change table in each of the source tables; and when there is a change table, taking a target table corresponding to the change table as a target table to be synchronized, taking a data provision process associated with the target table to be synchronized as an abnormal data provision process, executing a suspend operation on the abnormal data provision process, and sending, to a target account, prompt information for instructing a target account owner to update the target table to be synchronized. By means of implementing the method, the accuracy of data provision can be ensured.

Description

Method, device and terminal equipment for detecting abnormal supply data based on data migration

This application requires the priority of the Chinese patent application submitted to the China Patent Office on January 03, 2019, with the application number 201910003213.2 and the application name as "a method, device and terminal equipment for abnormal supply detection based on data migration." The entire contents are incorporated by reference in this application.

Technical field

The present application relates to the field of cloud storage technology, and in particular to a method, device and terminal device for detecting abnormal supply based on data migration.

Background technique

At present, the process of delivering the company's business data to the third-party platform generally includes: the data management system receives the data requirements of the third-party platform, extracts the company's business data according to the data requirements, and provides the extracted business data to the third party platform Tripartite platform. In practice, the inventor realized that when some source tables are changed (such as adding and/or deleting table fields), if the relevant general business data officer is not notified in time, the data extraction will still follow the previous source table structure , Which directly affects the accuracy of business data obtained by third-party platforms.

Summary of the invention

In order to solve the technical problem of inaccurate supply in the existing data management system, the present application provides a method, device and terminal device for detecting abnormal supply based on data migration.

A method for detecting abnormal supply based on data migration, the method includes:

Scan the source table and the hive table corresponding to each of the source tables to determine whether there is a change table in the source table; wherein, the table structure of the change table and the hive table corresponding to the change table indicate The table structure is inconsistent;

When the change table exists, use the target table corresponding to the change table as the target table to be synchronized;

Use the supply process associated with the target table to be synchronized as an abnormal supply process;

Perform a suspend operation on the abnormal supply process;

Sending prompt information to the target account to instruct the target account owner to update the target table to be synchronized.

An abnormal supply detection device, including:

A scanning unit, configured to determine whether there is a change table in the source table by scanning the source table and the hive table corresponding to each of the source tables; wherein, the table structure of the change table and the change table corresponding to the change table The table structure indicated by the hive table is inconsistent;

The determining unit is configured to use the target table corresponding to the change table as the target table to be synchronized when the change table exists; and use the supply process associated with the target table to be synchronized as the abnormal supply process;

A control unit for performing a suspend operation on the abnormal supply process;

The sending unit is configured to send prompt information to the target account to instruct the target account owner to update the target table to be synchronized.

A terminal device, the terminal device includes:

processor;

A memory, on which computer readable instructions are stored. When the computer readable instructions are executed by the processor, the method as described above is implemented.

A computer-readable storage medium that stores a computer program that causes a computer to perform the above-described abnormal migration detection method based on data migration. Optionally, the computer-readable storage medium may be a computer non-volatile storage medium.

The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

This application monitors the changes of the source table by scanning the source table and the hive table corresponding to each source table, and in the case of a change in the source table, the relevant supply process is suspended in time to ensure the accuracy of the data in the supply process .

It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application.

BRIEF DESCRIPTION

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments consistent with the application, and are used together with the specification to explain the principles of the application.

Fig. 1 is a schematic diagram of an apparatus for detecting abnormal supply according to an exemplary embodiment;

Fig. 2 is a flow chart of a method for detecting abnormal supply based on data migration according to an exemplary embodiment;

Fig. 3 is a flow chart of a method for detecting abnormal supply based on data migration according to another exemplary embodiment;

Fig. 4 is a block diagram of an apparatus for detecting abnormal supply according to an exemplary embodiment;

Fig. 5 is a block diagram of an apparatus for detecting abnormal supply according to another exemplary embodiment.

detailed description

Here, a description will be given of the exemplary embodiments in detail, examples of which are shown in the drawings.

The implementation environment of the present application may be a portable terminal device, such as a smart phone, a tablet computer, or a desktop computer. Fig. 1 is a schematic diagram of an apparatus for detecting abnormal supply according to an exemplary embodiment. The abnormal supply count detection device 100 may be the above-mentioned portable terminal device. As shown in FIG. 1, the abnormal supply detection device 100 may include one or more of the following components: a processing component 102, a memory 104, a power supply component 106, a multimedia component 108, an audio component 110, a sensor component 114, and a communication component 116.

The processing component 102 generally controls the overall operation of the abnormal supply detection device 100, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 102 may include one or more processors 118 to execute instructions to complete all or part of the steps of the method described below. In addition, the processing component 102 may include one or more modules for facilitating interaction between the processing component 102 and other components. For example, the processing component 102 may include a multimedia module to facilitate interaction between the multimedia component 108 and the processing component 102.

The memory 104 is configured to store various types of data to support the operation of the abnormality detection device 100. Examples of these data include instructions for any application program or method operating on the abnormal supply detection device 100. The memory 104 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM for short), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory ( Read-Only Memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The memory 104 further stores one or more modules for the one or more modules to be configured to be executed by the one or more processors 118 to complete all or part of the steps in the method shown below.

The power supply component 106 supplies power to various components of the abnormal supply detection device 100. The power supply component 106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the abnormal supply detection device 100.

The multimedia component 108 includes a screen that provides an output interface between the abnormal supply detection device 100 and the user. In some embodiments, the screen may include a liquid crystal display (Liquid Crystal) (LCD for short) and a touch panel. If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundary of the touch or sliding action, but also detect the duration and pressure related to the touch or sliding operation. The screen may also include an organic electroluminescence display (Organic Light Emitting Display, OLED for short).

The audio component 110 is configured to output and/or input audio signals. For example, the audio component 110 includes a microphone (Microphone, MIC for short). When the abnormality detection device 100 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 104 or sent via the communication component 116. In some embodiments, the audio component 110 further includes a speaker for outputting audio signals.

The sensor assembly 114 includes one or more sensors, which are used to provide various aspects of status assessment for the abnormal supply detection device 100. For example, the sensor assembly 114 can detect the on/off state of the abnormal supply detection device 100, the relative positioning of the components, and the sensor assembly 114 can also detect the position change of the abnormal supply detection device 100 or one component of the abnormal supply detection device 100 and The temperature of the abnormal feed detection device 100 changes. In some embodiments, the sensor assembly 114 may also include a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 116 is configured to facilitate wired or wireless communication between the abnormality detection device 100 and other devices. The abnormal supply detection device 100 can access a wireless network based on a communication standard, such as WiFi (Wireless-Fidelity, wireless fidelity). In an exemplary embodiment, the communication component 116 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 116 further includes a Near Field Communication (NFC) module for facilitating short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth technology and other technologies. .

In an exemplary embodiment, the abnormal supply detection device 100 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit (ASIC for short), digital signal processor, digital signal processing equipment, programmable logic device, field Programming gate arrays, controllers, microcontrollers, microprocessors, or other electronic components are implemented to perform the following methods.

Fig. 2 is a flow chart of a method for detecting abnormal supply based on data migration according to an exemplary embodiment. As shown in Figure 2, this method includes the following steps.

Step 201, by scanning the source table and the hive table corresponding to each source table, it is determined whether there is a change table in the source table, and if so, steps 202 to 205 are executed; if not, the process is ended.

In the embodiment of the present application, the table structure of the change table and the table structure indicated by the hive table corresponding to the change table are inconsistent. The specific description is as follows: the hive table corresponding to each source table stores the table structure of the active table (table name, field information, records, indexes, etc.), and the source table can be determined by scanning the source table and the hive table corresponding to the source table Whether there is a change table. Wherein, the specific implementation of determining whether there is a change table in the source table by scanning the source table and the hive table corresponding to the source table may be when there is a source table in the source table whose table structure is inconsistent with the table structure stored in the corresponding hive table , Use the source table as a change table, that is, a source table whose table structure does not match the table structure stored in its corresponding hive table, and continue to perform steps 202 to 205; and when each source table in the source table and its corresponding hive table are When the stored table structure is consistent, it is determined that there is no change table in the source table, and this flow is ended.

Step 202: Use the target table corresponding to the change table as the target table to be synchronized.

It should be noted that the embodiments of the present application implement data extraction based on kettle. Before data extraction, it is usually necessary to create a target table corresponding to each source table, where the correspondence between the source table and the target table may be one-to-one The relationship can also be a many-to-one relationship, that is, one source table can correspond to one target table, or multiple source tables can correspond to one target table. The target table stores part or all of the field information of its corresponding source table. Kettle can realize the migration of source table data by automatically matching the field information of the source table and the target table.

Step 203: Use the supply process associated with the target table to be synchronized as the abnormal supply process.

Optionally, in an exemplary embodiment, before performing step 203, it may be determined whether a supply request sent by a third-party platform is received; when the supply request is received, the corresponding to the supply request is turned on Count process.

Among them, using the supply process associated with the target table to be synchronized as an abnormal supply process may include: searching for the supply process associated with the target table to be synchronized among the supply processes corresponding to the supply request; The supply process associated with the target table to be synchronized is regarded as an abnormal supply process. The exemplary embodiment searches the abnormal supply process based on the received supply request of the third-party platform, which can improve the search efficiency of the abnormal supply process.

Step 204, suspend the abnormal supply process.

Step 205: Send prompt information to the target account to instruct the target account owner to update the target table to be synchronized.

By executing steps 201 to 205, when the change table exists in the source table, the target table corresponding to the change table is used as the target table to be synchronized, and the supply process associated with the target table to be synchronized is suspended, and the target account is used for sending The prompt message instructs the owner of the target account to update the target table to be synchronized. By implementing this method, the accuracy of the data in the counting process can be ensured.

It should be noted that step 201 may be executed periodically or in real time, which is not limited in the embodiments of the present application. Optionally, the source table and the hive table corresponding to each source table are scanned periodically or in real time to determine whether there is a change table in the source table, and when it is determined that there is a change table in the source table, the target table corresponding to the change table is marked , And when receiving the supply request of the third-party platform, determine whether the target table corresponding to the supply request is marked, and when the target table corresponding to the supply request is marked, it will be used to indicate the The information output of the supply process corresponding to the supply request fails to be opened; and when the target table corresponding to the supply request is not marked, the information output indicating that the supply process corresponding to the supply request is successfully started is output. Based on this optional exemplary embodiment, when there is a change table in the source table, the scan result can also be obtained by comparing the change table and its corresponding hive table. Wherein, the information used to indicate that the supply process corresponding to the supply request is successfully started may include the scan result. By implementing this exemplary embodiment, it is possible to quickly determine whether to respond to the received supply request by pre-marking the target table corresponding to the change table and prevent the abnormal supply process from running.

Further optionally, in another exemplary embodiment, when a supply request sent by a third-party platform is received, the target table associated with the supply request may be used as the supply table, and the supply table The corresponding source table and hive table are used as the source table to be scanned and the hive table to be scanned, and by scanning the source table to be scanned and the hive table to be scanned, it is determined whether there is a change table in the source table to be scanned, and the source table to be scanned When there is a change table in the table, the change information for the change table is generated by comparing the source table to be scanned and the hive table to be scanned, and the change information is output so that the general business data officer can correspond to the change table in the scan source table based on the change information The target table is updated. By implementing this exemplary embodiment, the scan source table and the hive table to be scanned can be scanned in a targeted manner according to the indication of the supply request sent by the third-party platform, which can not only improve the scanning speed, but also reduce the system load.

Fig. 3 is a flow chart showing a method for detecting abnormal supply based on data migration according to another exemplary embodiment. As shown in FIG. 3, in addition to the steps shown in FIG. 2, after step 204 and before step 205, the method for detecting abnormal supply based on data migration in this embodiment further includes the following steps:

Step 305: Determine the target job number from the general business data clerk registration form according to the current time point; wherein, the general business data clerk registration form records the work period and job number of each general business data clerk.

Step 306: Obtain registration information of the general service data clerk corresponding to the target job number. The registration information includes at least the contact information of the general service data clerk corresponding to the target job number.

Step 307: Determine the target account according to the above contact information.

For the detailed description of steps 301 to 304 and step 308, please refer to the description of steps 201 to 204 and step 205 in the method for detecting abnormal supply based on data migration shown in FIG. 2. Repeat again.

In the embodiment of the present application, the above-mentioned contact method corresponding to the target job number may be one type or multiple types, which is not limited in the embodiment of the present application. If there are multiple contact methods corresponding to the target job number, and each contact method has a different priority level, optionally, in an exemplary embodiment, determining the target account according to the above contact methods may include: obtaining the above contact The priority level of each contact method in the method; the account corresponding to the contact method with the highest priority level among the above contact methods is determined as the target account. Based on this exemplary embodiment, further optionally, after sending the prompt information for instructing the target account owner to update the target table to be synchronized, it can also be determined whether the target account owner has detected The update operation of the synchronization target table, and when it is determined that the update operation of the target account owner for the target table to be synchronized is not detected, the target account is updated according to a preset rule. In the embodiment of the present application, the specific implementation method for updating the target account according to the preset rules is as follows: the contact method whose priority level is second only to the highest priority level among the above contact methods is used as the first target contact method, and the An account corresponding to a target contact method updates the target account, and when the update of the target account is completed and the update operation of the target account owner on the target table to be synchronized is not detected within the preset time period, the above contact method is prioritized The contact method with the level second only to the first target contact method is used as the second target contact method, and the target account is updated using the account corresponding to the second target contact method, and so on until all contact methods in the above contact method are traversed. By implementing this exemplary embodiment, when the update operation of the target account owner to the target table to be synchronized is not detected within a preset time period, all contacts included in the above-mentioned contact method may be based on the priority level of the above-mentioned contact method The traversal method can ensure the maximum update of the target account owner to the synchronization target table to the greatest extent.

It should be noted that the above contact method can be email, QQ, WeChat or SMS, etc. Then, the account corresponding to the above contact method can be an email account, a QQ account, a WeChat account or a mobile phone number.

Step 309: Generate table modification information for the target table to be synchronized according to the change table and the hive table corresponding to the change table.

In the embodiment of the present application, the table change information generated for the target table to be synchronized generated according to the above change table and the hive table corresponding to the above change table is used to express the change of the table structure indicated by the change table compared to its corresponding hive table . For example, if the above change table has more fields than the table structure indicated by its corresponding hive table, then the table change information may be added field information, if the above change table is compared to its corresponding hive table If the indicated table structure deletes the field, then the table information can be the deleted field information.

Step 310: Update the hive table corresponding to the change table according to the table change information.

Step 311: Send the generated change table information to the target account.

In an exemplary embodiment, after step 311 is executed, the field in the synchronization target table that matches the above-mentioned change table can also be marked, and the target can be obtained based on the change information and the field in the marked target table to be synchronized Change information. The target change information may include change field information of the target table to be synchronized. In the implementation of this exemplary embodiment, the owner of the target account described above can implement a quick table change operation on the target table to be synchronized based on the target change information, further improving the efficiency and accuracy of the supply process.

Steps 309 to 311 can be executed, the hive table corresponding to the change table can be automatically updated according to the generated change table information for the target table to be synchronized, and the generated change table information can also be sent to the target account for use by the target account. When the update operation is performed on the target table to be synchronized, the update efficiency and accuracy of the target table to be synchronized can be improved.

Optionally, in an exemplary embodiment, after step 311 is executed, it may be detected whether a termination update instruction sent by the target account owner is received, and when it is detected that the termination update instruction is received, the abnormal supply is controlled. The data process changes from the suspended state to the execution state. Further optionally, when it is detected that the termination update instruction is received, controlling the above abnormal supply process to change from the suspended state to the execution state may include: when it is detected that the termination update instruction is received, acquiring the updated target table to be synchronized Table structure; determine whether the updated table structure of the target table to be synchronized matches the table structure of the changed table; when the updated table structure of the target table to be synchronized matches the table structure of the changed table, control the above-mentioned abnormal feed process Change from the suspended state to the execution state. Specifically, the method for determining whether the updated table structure of the target table to be synchronized matches the table structure of the change table may be to determine whether the field corresponding to the change table in the updated table structure of the target table to be synchronized is the same as the change table The fields in the match, if they match, the table structure of the updated target table to be synchronized matches the table structure of the change table, if they do not match, the table structure of the updated target table to be synchronized does not match the table structure of the change table . By implementing this exemplary embodiment, the table structure of the updated target table to be synchronized and the table structure of the changed table can be compared when the termination update instruction sent by the above-mentioned target account owner is received, which can further ensure the supply of data Accuracy of data in progress.

The following is an embodiment of the abnormal supply detection device disclosed in the present application.

Fig. 4 is a block diagram of an apparatus for detecting abnormal supply according to an exemplary embodiment. As shown in FIG. 4, the abnormal supply detection device may include:

The scanning unit 401 is configured to determine whether there is a change table in the source table by scanning the source table and the hive table corresponding to each source table.

In the embodiment of the present application, the table structure of the change table and the table structure indicated by the hive table corresponding to the change table are inconsistent. The specific description is as follows: the hive table corresponding to each source table stores the table structure of the active table (table name, field information, record, index, etc.), and the scanning unit 401 can determine by scanning the source table and the hive table corresponding to the source table Whether the change table exists in the source table. When the scanning unit 401 determines that there is a source table in the source table whose table structure is inconsistent with the table structure stored in the corresponding hive table, the source table is used as a change table, that is, the table structure is inconsistent with the table structure stored in the corresponding hive table When the scanning unit 401 determines that each source table in the source table is consistent with the table structure stored in the corresponding hive table, it determines that there is no change table in the source table.

The determining unit 402 is configured to, when there is the above-mentioned change table, use the target table corresponding to the above-mentioned change table as the target table to be synchronized; and use the supply process associated with the target table to be synchronized as the abnormal supply process.

Optionally, in an exemplary embodiment, the determining unit 402 may also be used to determine whether a supply request sent by a third-party platform is received before the supply process associated with the target table to be synchronized is regarded as an abnormal supply process , And when receiving the supply request, start the supply process corresponding to the supply request.

Wherein, the determining unit 402 is configured to use the supply process associated with the target table to be synchronized as an abnormal supply process. The determination unit 402 may be used to search for the supply process corresponding to the supply request. The supply process associated with the target table to be synchronized and the found supply process associated with the target table to be synchronized are regarded as abnormal supply processes. The exemplary embodiment searches the abnormal supply process based on the received supply request of the third-party platform, which can improve the search efficiency of the abnormal supply process.

The control unit 403 is used to perform a suspend operation on the abnormal supply process.

The sending unit 404 is configured to send prompt information to the target account to instruct the target account owner to update the target table to be synchronized.

By implementing the above-mentioned abnormal feed detection device, when the source table has a change table, the target table corresponding to the change table is used as the target table to be synchronized, and the feed process associated with the target table to be synchronized is suspended and sent to the target account The prompt information instructing the owner of the target account to update the target table to be synchronized can ensure the accuracy of the data in the supply process.

It should be noted that the scanning unit 401 scans the source table and the hive table corresponding to each source table to determine whether there is a change table in the source table, which may be executed periodically or in real time, which is not limited in the embodiments of the present application.

The scanning unit 401 scans the source table and the hive table corresponding to each source table periodically or in real time to determine whether there is a change table in the source table, and when it is determined that there is a change table in the source table, mark the target table corresponding to the change table, And when receiving the supply request of the third-party platform, determine whether the target table corresponding to the supply request is marked, and when the target table corresponding to the supply request is marked, it will be used to indicate the supply The information output of the supply process corresponding to the supply request fails to be output; and when the target table corresponding to the supply request is not marked, a message output indicating that the supply process corresponding to the supply request is successfully started is output. Based on this optional exemplary embodiment, when there is a change table in the source table, the scan result can also be obtained by comparing the change table and its corresponding hive table. Wherein, the information used to indicate that the supply process corresponding to the supply request is successfully started may include the scan result. By implementing this exemplary embodiment, it is possible to quickly determine whether to respond to the received supply request by pre-marking the target table corresponding to the change table and prevent the abnormal supply process from running.

Optionally, in an exemplary embodiment, the scanning unit 401 can use the target table associated with the supply request as the supply table when receiving the supply request sent by the third-party platform, and the supply table The corresponding source table and hive table are used as the source table to be scanned and the hive table to be scanned, and by scanning the source table to be scanned and the hive table to be scanned, it is determined whether there is a change table in the source table to be scanned, and the source table to be scanned When there is a change table in the table, the change information for the change table is generated by comparing the source table to be scanned and the hive table to be scanned, and the change information is output so that the general business data officer can correspond to the change table in the scan source table based on the change information The target table is updated. By implementing this exemplary embodiment, the scan source table and the hive table to be scanned can be scanned in a targeted manner according to the indication of the supply request sent by the third-party platform, which can not only improve the scanning speed, but also reduce the system load.

Fig. 5 is a block diagram of an apparatus for detecting abnormal supply according to another exemplary embodiment. The abnormal supply detection device shown in FIG. 5 is optimized by the abnormal supply detection device shown in FIG. 4. The above-mentioned control unit 403 can also be used to perform a suspend operation on the abnormal supply process and the sending unit. 405 Before sending the prompt message for instructing the target account owner to update the target table to be synchronized to the target account, determine the target job number from the general business clerk registration form according to the current time point; The working hours and job number of each general business data officer; and obtaining the registration information of the general business data person corresponding to the target job number, which at least includes the contact information of the general business data person corresponding to the target job number; and according to the contact To determine the target account.

In the embodiment of the present application, the above-mentioned contact method corresponding to the target job number may be one type or multiple types, which is not limited in the embodiment of the present application. If there are multiple contact methods corresponding to the target job number, and each contact method has a different priority level, optionally, in an exemplary embodiment, the control unit 403 is configured to determine the target account according to the contact method Specifically, the control unit 403 may be used to obtain the priority level of each contact method in the contact method; and determine the account corresponding to the contact method with the highest priority level in the contact method as the target account.

Further optionally, after sending the prompt information for instructing the target account owner to update the target table to be synchronized to the target account, the sending unit 404 can also determine whether the target account owner has detected the target to be synchronized within a preset time period. Table update operation, and when it is determined that the target account owner's update operation for the target table to be synchronized is not detected, the target account is updated according to the preset rules. In the embodiment of the present application, the specific implementation of the sending unit 404 to update the target account according to the preset rules is as follows: The sending unit 404 uses the contact method whose priority level is second only to the highest priority level among the above contact methods as the first target Contact information, and use the account corresponding to the first target contact information to update the target account, and when the target account update is completed and the target account owner does not detect the update operation of the target table to be synchronized within the preset time, will The priority level of the above contact method is second only to the contact method of the first target contact method as the second target contact method, and the target account is updated using the account corresponding to the second target contact method, and so on until all the above contact methods are traversed Contact information. By implementing this exemplary embodiment, when the update operation of the target account owner to the target table to be synchronized is not detected within a preset time period, all contacts included in the above-mentioned contact method may be based on the priority level of the above-mentioned contact method The traversal method can ensure the maximum update of the target account owner to the synchronization target table to the greatest extent.

The abnormal supply data detection device shown in FIG. 5 may further include:

The updating unit 405 is configured to generate table change information for the target table to be synchronized based on the change table and the hive table corresponding to the change table, and update the hive table corresponding to the change table according to the table change information, and send to the target account The table information.

In the embodiment of the present application, for the description of the above table-changing information, please refer to the description below step 309 in the method for detecting abnormal supply based on data migration shown in FIG. 3, which will not be repeated in the embodiment of the present application.

The updating unit 405 can automatically update the hive table corresponding to the change table according to the generated change table information for the target table to be synchronized, and can also send the generated change table information to the target account so that the target account owner is treating the synchronization target The reference is used when the table is updated, which can improve the update efficiency and accuracy of the target table to be synchronized.

In an exemplary embodiment, the updating unit 405 may also mark the fields in the synchronization target table that match the change table, and obtain the target change information according to the change information and the fields in the marked target table to be synchronized. The target change information may include change field information of the target table to be synchronized. In the implementation of this exemplary embodiment, the above-mentioned target account owner can implement a quick table replacement operation on the target table to be synchronized based on the target change information, thereby further improving the efficiency and accuracy of the supply process.

Optionally, the control unit 403 may also be used to detect whether a termination update instruction sent by the target account owner is received, and when detecting that the termination update instruction is received, control the abnormal supply process to transition from the suspended state It is the execution status.

It should be noted that, in the embodiment of the present application, the update unit 405 may also be used to send the activation instruction to the control unit 403 after the update table information is sent to the target account to activate the control unit 403.

Further optionally, the control unit 403 is configured to control the transition of the abnormal supply process from the suspended state to the execution state when it is detected that the termination update instruction is received. The control unit 403 may be used to detect When the update instruction is terminated, obtain the updated table structure of the target table to be synchronized; determine whether the updated table structure of the target table to be synchronized matches the table structure of the changed table; when the updated table structure of the target table to be synchronized When the table structure of the change table matches, the process of controlling the above-mentioned abnormal supply is changed from the suspended state to the executed state. Specifically, the method for determining whether the updated table structure of the target table to be synchronized matches the table structure of the change table may be to determine whether the field corresponding to the change table in the updated table structure of the target table to be synchronized is the same as the change table The fields in the match, if they match, the table structure of the updated target table to be synchronized matches the table structure of the change table, if they do not match, the table structure of the updated target table to be synchronized does not match the table structure of the change table . By implementing this exemplary embodiment, the control unit 403 may compare the table structure of the updated target table to be synchronized with the table structure of the changed table when receiving the termination update instruction sent by the target account owner. To further ensure the accuracy of the data in the process of supplying data.

This application also provides a terminal device, which includes:

processor;

A memory, in which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, an abnormal feed detection method based on data migration as described above is implemented.

The terminal device may be the abnormal supply detection device 100 shown in FIG. 1.

In an exemplary embodiment, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the abnormal supply data detection based on data migration as described above is realized method.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be performed without departing from the scope thereof. The scope of this application is limited only by the appended claims.

Claims

A method for detecting abnormal supply based on data migration, characterized in that the method includes:

Scan the source table and the hive table corresponding to each of the source tables to determine whether there is a change table in the source table; wherein, the table structure of the change table and the hive table corresponding to the change table indicate The table structure is inconsistent;

When the change table exists, use the target table corresponding to the change table as the target table to be synchronized;

Use the supply process associated with the target table to be synchronized as an abnormal supply process;

Perform a suspend operation on the abnormal supply process;

Sending prompt information to the target account to instruct the target account owner to update the target table to be synchronized.
The method according to claim 1, wherein after the prompt information for instructing the target account owner to update the target table to be synchronized is sent to the target account, the method further comprises:

Generating table modification information for the target table to be synchronized according to the change table and the hive table corresponding to the change table;

Update the hive table corresponding to the change table according to the change table information;

Sending the change table information to the target account.
The method according to claim 1 or 2, wherein after the suspend operation is performed on the abnormal supply process, and the sending to the target account is used to instruct the target account owner to update the pending Before synchronizing the prompt information of the target table, the method further includes:

Determine the target job number from the general business data clerk registration form according to the current time point; wherein, the general business data clerk registration form records the work period and job number of each general business data clerk;

Obtaining registration information of the general service data clerk corresponding to the target job number, where the registration information includes at least the contact information of the general service data clerk corresponding to the target job number;

Determine the target account according to the contact information.
The method according to claim 3, wherein the determining the target account according to the contact method comprises:

Obtaining the priority level of each contact method in the contact methods;

The account corresponding to the contact method with the highest priority level among the contact methods is determined as the target account.
The method according to claim 1, wherein after the prompt information for instructing the target account owner to update the target table to be synchronized is sent to the target account, the method further comprises:

Detecting whether a termination update instruction sent by the target account owner is received;

When it is detected that the termination update instruction is received, the abnormal supply process is controlled to change from a suspended state to an execution state.
The method according to claim 5, wherein, when it is detected that the termination update instruction is received, controlling the abnormal supply process to change from a suspended state to an execution state includes:

When it is detected that the termination update instruction is received, the updated table structure of the target table to be synchronized is acquired;

Judging whether the updated table structure of the target table to be synchronized matches the table structure of the change table;

When the updated table structure of the target table to be synchronized matches the table structure of the change table, the abnormal supply process is controlled to change from a suspended state to an execution state.
The method according to claim 1, characterized in that, before using the supply process associated with the target table to be synchronized as an abnormal supply process, the method further comprises:

Determine whether a supply request sent by a third-party platform is received;

When receiving the supply request, start the supply process corresponding to the supply request;

Said using the supply process associated with the target table to be synchronized as an abnormal supply process includes:

Searching for the supply process associated with the target table to be synchronized in the supply process corresponding to the supply request;

The found supply process associated with the target table to be synchronized is regarded as an abnormal supply process.
The method according to claim 1, wherein the method further comprises:

When it is determined that there is a change table in the source table, mark the target table corresponding to the change table;

When receiving a supply request sent by a third-party platform, determine whether the target table corresponding to the supply request is marked;

When the target table corresponding to the supply request is marked, output information indicating that the supply process corresponding to the supply request fails to start;

When the target table corresponding to the supply request is not marked, a message indicating that the supply process corresponding to the supply request is successfully started is output.
An abnormal supply data detection device, characterized in that it includes:

A scanning unit, configured to determine whether there is a change table in the source table by scanning the source table and the hive table corresponding to each of the source tables; wherein, the table structure of the change table and the change table corresponding to the change table The table structure indicated by the hive table is inconsistent;

A determining unit, configured to use the target table corresponding to the change table as the target table to be synchronized when the change table exists; and use the supply process associated with the target table to be synchronized as the abnormal supply process;

A control unit for performing a suspend operation on the abnormal supply process;

The sending unit is configured to send prompt information to the target account to instruct the target account owner to update the target table to be synchronized.
The device according to claim 9, wherein the device further comprises:

An updating unit, configured to generate table change information for the target table to be synchronized according to the change table and the hive table corresponding to the change table; according to the table change information, update the table corresponding to the change table hive table

The sending unit is further configured to send the table change information to the target account.
The device according to claim 9, characterized in that

The control unit is further configured to detect whether a termination update instruction sent by the target account owner is received; when detecting that the termination update instruction is received, control the abnormal supply process to change from a suspended state to an execution state .
The device according to claim 11, characterized in that

The control unit is specifically configured to acquire the table structure of the updated target table to be synchronized when it is detected that the termination update instruction is received; to determine the table structure of the updated target table to be synchronized and the change table Whether the table structure matches; when the updated table structure of the target table to be synchronized matches the table structure of the change table, the abnormal supply process is controlled to change from the suspended state to the execution state.
The device according to claim 9, characterized in that

The determining unit is further configured to determine whether a supply request sent by a third-party platform is received before the supply process associated with the target table to be synchronized is regarded as an abnormal supply process; when the supply request is received , Start the supply process corresponding to the supply request;

When the determining unit uses the supply process associated with the target table to be synchronized as an abnormal supply process, it is specifically used to:

Searching for the supply process associated with the target table to be synchronized in the supply process corresponding to the supply request;

The found supply process associated with the target table to be synchronized is regarded as an abnormal supply process.
The device according to claim 9, characterized in that

The scanning unit is also used to mark the target table corresponding to the change table when it is determined that there is a change table in the source table; when receiving the supply request sent by the third-party platform, it is determined to correspond to the supply request Whether the target table with the mark is provided; when the target table corresponding to the feed request is marked, output information indicating that the feed process corresponding to the feed request failed to start; when When the target table corresponding to the request is not marked, a message indicating that the supply process corresponding to the supply request is successfully started is output.
A computer-readable storage medium, characterized in that it stores a computer program that causes a computer to execute the method for detecting abnormal supply based on data migration according to any one of claims 1 to 8.
A terminal device, characterized in that the terminal device includes:

processor;

A memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the following steps are implemented:

Scan the source table and the hive table corresponding to each of the source tables to determine whether there is a change table in the source table; wherein, the table structure of the change table and the hive table corresponding to the change table indicate The table structure is inconsistent;

When the change table exists, use the target table corresponding to the change table as the target table to be synchronized;

Use the supply process associated with the target table to be synchronized as an abnormal supply process;

Perform a suspend operation on the abnormal supply process;

Sending prompt information to the target account to instruct the target account owner to update the target table to be synchronized.
The terminal device according to claim 16, wherein after the processor executes the prompt information for instructing the target account owner to update the target table to be synchronized to the target account, the processor further executes the following step:

Generating table modification information for the target table to be synchronized according to the change table and the hive table corresponding to the change table;

Update the hive table corresponding to the change table according to the change table information;

Sending the change table information to the target account.
The terminal device according to claim 16, wherein after the processor executes the prompt information for instructing the target account owner to update the target table to be synchronized to the target account, the processor further executes the following step:

Detecting whether a termination update instruction sent by the target account owner is received;

When it is detected that the termination update instruction is received, the abnormal supply process is controlled to change from a suspended state to an execution state.
The terminal device according to claim 16, wherein the processor further executes the following steps before executing the process of supplying data associated with the target table to be synchronized as an abnormal supplying process:

Determine whether a supply request sent by a third-party platform is received;

When receiving the supply request, start the supply process corresponding to the supply request;

When the processor executes the process of taking the supply process associated with the target table to be synchronized as an abnormal supply process, it specifically performs the following steps:

Searching for the supply process associated with the target table to be synchronized in the supply process corresponding to the supply request;

The found supply process associated with the target table to be synchronized is regarded as an abnormal supply process.
The terminal device according to claim 16, wherein the processor further performs the following steps:

When it is determined that there is a change table in the source table, mark the target table corresponding to the change table;

When receiving a supply request sent by a third-party platform, determine whether the target table corresponding to the supply request is marked;

When the target table corresponding to the supply request is marked, output information indicating that the supply process corresponding to the supply request fails to start;

When the target table corresponding to the supply request is not marked, a message indicating that the supply process corresponding to the supply request is successfully started is output.