CN109542921B - Data checking method and device, electronic equipment and storage medium - Google Patents
Data checking method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN109542921B CN109542921B CN201811435873.XA CN201811435873A CN109542921B CN 109542921 B CN109542921 B CN 109542921B CN 201811435873 A CN201811435873 A CN 201811435873A CN 109542921 B CN109542921 B CN 109542921B
- Authority
- CN
- China
- Prior art keywords
- data
- events
- splicing
- display page
- commodity display
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0641—Shopping interfaces
Abstract
The disclosure discloses a data checking method, a data checking device, an electronic device and a storage medium. Wherein, the method comprises the following steps: acquiring splicing data of each commodity display page within a preset time period and monitoring data of all events triggered on each commodity display page; determining the monitoring number of the triggering events on each commodity display page according to the splicing data; determining the actual number of the triggering events on each commodity display page according to the monitoring data of all the events; if the monitored quantity is inconsistent with the actual quantity, performing data splicing on the monitored data of all events to obtain corrected spliced data of each commodity display page within a preset time period; and replacing the splicing data of each commodity display page in the preset time period by adopting the corrected splicing data. The embodiment of the disclosure solves the problem that no effective measure can be used for detecting and correcting the splicing data according to the actual monitoring data, and can automatically detect and correct the splicing data stored in the database according to the actual monitoring data.
Description
Technical Field
The present disclosure relates to data processing technologies, and in particular, to a data checking method and apparatus, an electronic device, and a storage medium.
Background
After different commodity display pages are released, a plurality of events of each commodity display page are monitored, and monitoring data corresponding to the events are obtained. For example, click events, purchase events, and fee deduction events for each merchandise display page are monitored. In order to facilitate analysis and statistics of data of each commodity display page, all monitoring data corresponding to each commodity display page need to be spliced to form a data table, namely, a data table corresponding to each commodity display page.
In the prior art, monitoring data of each commodity display page is usually obtained and then directly spliced, and then the spliced data is stored in a database.
The prior art has the following defects: because the data volume of the monitoring data is huge and the server is burdened, when a network problem or other system faults occur, the time for acquiring the monitoring data corresponding to an event may be delayed, so that the actual monitoring data in a certain time period does not correspond to the spliced data. For example, 4 points are acquired only when 2 points of a certain user click data, so that actual monitoring data of 2 points and spliced data do not correspond to each other. The prior art has no effective measures to detect and correct splicing data according to actual monitoring data.
Disclosure of Invention
The disclosure provides a data checking method, a data checking device, electronic equipment and a storage medium, so as to realize automatic detection and correction of splicing data stored in a database according to actual monitoring data.
In a first aspect, an embodiment of the present disclosure provides a data checking method, including:
acquiring splicing data of each commodity display page within a preset time period and monitoring data of all events triggered on each commodity display page; the splicing data is formed by splicing monitoring data of events triggered by the commodity display pages acquired within a preset time period;
determining the monitoring number of the triggering events on each commodity display page according to the splicing data;
determining the actual number of the triggering events on each commodity display page according to the monitoring data of all the events;
if the monitored quantity is inconsistent with the actual quantity, performing data splicing on the monitored data of all events to obtain corrected spliced data of each commodity display page within a preset time period;
and replacing the splicing data of each commodity display page in the preset time period by adopting the corrected splicing data.
In the above scheme, optionally, the event includes a click event, a purchase event, and a fee deduction event;
determining the monitoring number of the triggering events on each commodity display page according to the splicing data, wherein the monitoring number comprises the following steps:
respectively acquiring the number of click events, the number of purchase events and the number of fee deduction events triggered on each commodity display page according to the splicing data, and summing the numbers;
and taking the summation result as the monitoring number of the trigger events on each commodity display page.
In the foregoing scheme, optionally, the method further includes:
if the monitoring number is not consistent with the actual number, determining problem events according to the splicing data and the monitoring data of all events;
and generating problem feedback information according to the monitoring data of the problem event.
In the foregoing scheme, optionally, the data splicing is performed on the monitoring data of all events to obtain the corrected splicing data of each commodity display page within the preset time period, and the method includes:
the method comprises the steps of fragmenting monitoring data of all events to obtain a preset number of fragmented data packets;
sending the fragment data packets to corresponding data splicing servers so that each data splicing server respectively performs data splicing on monitoring data in the fragment data packets;
and acquiring the corrected splicing data of each commodity display page in the preset time period generated by each data splicing server.
In the foregoing scheme, optionally, the fragmentation is performed on the monitoring data of all events to obtain fragmentation data packets of a preset number, including:
determining the number of events triggered on each commodity display page;
aiming at each fragment data packet, distributing the monitoring data of all events triggered on the same commodity display page into the same fragment data packet;
summarizing the fragmented data packets to obtain a preset number of fragmented data packets; and the number of events corresponding to the monitoring data in the fragment data packet is equal to the number of events triggered on at least one commodity display page.
In the foregoing scheme, optionally, the preset number is equal to the number of the data splicing servers.
In a second aspect, an embodiment of the present disclosure further provides a data checking apparatus, including:
the data acquisition module is used for acquiring the splicing data of each commodity display page within a preset time period and the monitoring data of all events triggered on each commodity display page; the splicing data is formed by splicing monitoring data of events triggered by the commodity display pages acquired within a preset time period;
the first quantity determining module is used for determining the monitoring quantity of the trigger events on each commodity display page according to the splicing data;
the second quantity determining module is used for determining the actual quantity of the trigger events on each commodity display page according to the monitoring data of all the events;
the data splicing module is used for splicing the monitoring data of all events to obtain the corrected splicing data of each commodity display page within a preset time period if the monitored number is judged to be inconsistent with the actual number;
and the data replacement module is used for replacing the splicing data of each commodity display page in the preset time period by adopting the corrected splicing data.
In the above scheme, optionally, the event includes a click event, a purchase event, and a fee deduction event;
the first quantity determination module includes:
the quantity summing unit is used for respectively acquiring the number of click events, the number of purchase events and the number of fee deduction events triggered on each commodity display page according to the splicing data and summing the numbers;
and the quantity determining unit is used for taking the summation result as the monitoring quantity of the trigger events on each commodity display page.
In the foregoing scheme, optionally, the method further includes:
the time determining module is used for determining the problem events according to the splicing data and the monitoring data of all events if the monitoring number is inconsistent with the actual number;
and the information generation module is used for generating the problem feedback information according to the monitoring data of the problem event.
In the foregoing scheme, optionally, the data splicing module includes:
the data fragmentation unit is used for fragmenting the monitoring data of all events to obtain a preset number of fragmentation data packets;
the data packet sending unit is used for sending the fragment data packets to the corresponding data splicing servers so that the data splicing servers respectively perform data splicing on the monitoring data in the fragment data packets;
and the data acquisition unit is used for acquiring the corrected splicing data of each commodity display page in the preset time period generated by each data splicing server.
In the foregoing scheme, optionally, the data slicing unit includes:
the quantity determining subunit is used for determining the quantity of the events triggered on each commodity display page;
the data distribution subunit is used for distributing the monitoring data of all events triggered on the same commodity display page to the same fragmented data packet aiming at each fragmented data packet;
the collecting subunit is used for collecting the fragmented data packets to obtain a preset number of fragmented data packets; and the number of events corresponding to the monitoring data in the fragment data packet is equal to the number of events triggered on at least one commodity display page.
In the foregoing scheme, optionally, the preset number is equal to the number of the data splicing servers.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the data checking method according to the embodiment of the present disclosure.
In a fourth aspect, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the data checking method according to the disclosed embodiments.
The embodiment of the disclosure obtains the splicing data of each commodity display page within a preset time period and the monitoring data of all events triggered on each commodity display page, wherein the splicing data is formed by splicing the monitoring data of the events triggered on each commodity display page acquired within the preset time period, then the monitoring number of the events triggered on each commodity display page is determined according to the splicing data, the actual number of the events triggered on each commodity display page is determined according to the monitoring data of all the events, when the monitoring number is judged to be inconsistent with the actual number, the monitoring data of all the events are subjected to data splicing to obtain the corrected splicing data of each commodity display page within the preset time period, and the corrected splicing data is adopted to replace the splicing data of each commodity display page within the preset time period, so as to solve the problem that the splicing data can be detected and corrected according to the actual monitoring data without effective measures in the prior art, the splicing data stored in the database can be automatically detected and corrected according to the actual monitoring data.
Drawings
Fig. 1 is a flowchart of a data checking method provided by an embodiment of the present disclosure;
fig. 2 is a flowchart of a data checking method provided by an embodiment of the present disclosure;
FIG. 3 is a flow chart of a data checking method provided by an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a data checking apparatus according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the disclosure and are not limiting of the disclosure. It should be further noted that, for the convenience of description, only some of the structures relevant to the present disclosure are shown in the drawings, not all of them.
Fig. 1 is a flowchart of a data checking method provided in an embodiment of the present disclosure, where the present embodiment is applicable to a case of checking spliced data, the method may be executed by a data checking apparatus, the apparatus may be implemented in a software and/or hardware manner, and the apparatus may be configured in an electronic device, for example, a terminal device or a server. As shown in fig. 1, the method may include the steps of:
Wherein the monitoring data is data related to the event. The splicing data is formed by splicing monitoring data of events triggered by the commodity display pages acquired within a preset time period. The event can be a click event, a purchase event or a fee deduction event triggered on each commodity display page. The click event is an event related to a click operation of the user on each commodity display page. The monitoring data corresponding to the click event may include a click position and a click time at which the user performs a click operation on the goods presentation page. The purchase event is an event related to a purchase operation of the user on each merchandise display page. The monitoring data corresponding to the purchase event may include a payment time and a payment amount for the user to perform the purchase operation on the merchandise display page. The fee deduction event is an event for deducting fee based on the operation of the user on each commodity display page. The monitoring data corresponding to the purchase event may include a debit time and a debit amount.
Specifically, events triggered on each commodity display page are monitored, and monitoring data corresponding to each event are generated. The method comprises the steps of obtaining monitoring data of events triggered on each commodity display page obtained within a preset time period, and fragmenting the obtained monitoring data to obtain fragmented data packets with preset number. And sending the fragment data packets to corresponding data splicing servers so that each data splicing server respectively performs data splicing on the monitoring data in the fragment data packets to obtain data splicing results. Data splicing is an operation of integrating several data stored in different cells (tables) into one cell (table) in a vertical direction. After each data splicing server acquires the fragment data packets, data splicing is carried out on the acquired monitoring data of the fragment data packets, and data splicing results are obtained. The data splicing server is used for splicing the monitoring data in the fragment data packet. And the data splicing result is the splicing data of each commodity display page in a preset time period. And acquiring the splicing data of each commodity display page in a preset time period generated by each data splicing server, and storing the splicing data into a database, so that the data of each commodity display page can be conveniently analyzed and counted subsequently.
The method comprises the steps of obtaining splicing data of each commodity display page within a preset time period and monitoring data of all events triggered by each commodity display page within the preset time period. The monitoring data of all events triggered on each commodity display page within the preset time period is the monitoring data of the events triggered on each commodity display page within the preset time period acquired at the current moment, and may include the monitoring data of a certain event acquired after data splicing due to network problems or other system faults, and is the actual monitoring data of the events triggered on each commodity display page.
The preset time period is a time interval set according to business requirements and can be adjusted according to requirements. Optionally, 24 preset time periods are set according to 24 hours of a day: 0 to 1, 1 to 2, 2 to 3, 3 to 4, 4 to 5, 5 to 6, 6 to 7, 7 to 8, 8 to 9, 9 to 10, 10 to 11, 11 to 12, 12 to 13, 13 to 14, 14 to 15, 15 to 16, 16 to 17, 17 to 18, 18 to 19, 19 to 20, 20 to 21, 21 to 22, 22 to 23, 23 to 24. The splicing data is formed by splicing the monitoring data of the events triggered by the commodity display pages acquired in each preset time period. For example, the splicing data of each commodity display page in each preset time period within the previous 24 hours and the monitoring data of all events triggered on each commodity display page may be sequentially acquired at the 0 point of each day, and the splicing data of each commodity display page within the previous 24 hours is checked.
And 102, determining the monitoring number of the trigger events on each commodity display page according to the splicing data.
The monitoring number is the number of events triggered on each commodity display page determined according to the splicing data of each commodity display page. After the splicing data of the commodity display pages in the preset time period are obtained, the splicing data of the commodity display pages are distinguished according to the identification information of the commodity display pages, and then the monitoring number of the trigger events on the commodity display pages is respectively determined.
For example, events include click events, purchase events, and debit events. According to the splicing data of a certain commodity display page, the number of the click events triggered on the commodity display page is determined to be 20, the number of the triggered purchase events is determined to be 15, and the number of the triggered fee deduction events is determined to be 5. The monitoring number of the trigger events on a certain commodity display page is the sum of the number of the events. I.e. the monitored number of trigger events on the merchandise display page is 40.
And 103, determining the actual number of the trigger events on each commodity display page according to the monitoring data of all the events.
The actual number is the number of events triggered on each commodity display page determined according to the currently acquired monitoring data of all events triggered on each commodity display page, and is the accurate number of the current events triggered on each commodity display page. After the monitoring data of all events triggered on each commodity display page within a preset time period are obtained, the monitoring data of all events triggered on each commodity display page are distinguished according to the identification information of each commodity display page, and then the actual number of the events triggered on each commodity display page is respectively determined.
For example, events include click events, purchase events, and debit events. According to the monitoring data of all events triggered on a certain commodity display page, the number of click events triggered on the commodity display page is determined to be 20, the number of triggered purchase events is determined to be 15, and the number of triggered fee deduction events is determined to be 5. The monitoring number of the triggering events on each commodity display page is the sum of the number of the events. I.e. the actual number of trigger events on the merchandise display page is 40.
And step 104, if the monitored quantity is judged to be inconsistent with the actual quantity, performing data splicing on the monitored data of all events to obtain corrected splicing data of each commodity display page within a preset time period.
And judging whether the monitored quantity of the trigger events on each commodity display page is consistent with the actual quantity.
And if the monitored quantity is judged to be inconsistent with the actual quantity, the splicing data of each commodity display page in the preset time period is shown to be not corresponding to the currently acquired monitored data, and data splicing is carried out again according to the acquired monitored data of all events triggered by each commodity display page in the preset time period to obtain the corrected splicing data of each commodity display page in the preset time period.
For example, due to a network problem or other system failure, the currently acquired monitoring data may include monitoring data of some event that is acquired after data splicing, so that the monitoring number is smaller than the actual number, i.e., the monitoring number is inconsistent with the actual number.
If the monitored quantity is consistent with the actual quantity, the splicing data of the commodity display pages in the preset time period is corresponding to the currently acquired monitored data, and subsequent operation on the splicing data of the commodity display pages in the preset time period is not needed.
And 105, replacing the splicing data of each commodity display page in the preset time period with the corrected splicing data.
The splicing data of each commodity display page in the preset time period is replaced by the corrected splicing data of each commodity display page in the preset time period, so that the splicing data of each commodity display page in the preset time period corresponds to the currently acquired monitoring data, and the correction of the splicing data of each commodity display page in the preset time period is completed.
The technical scheme of this embodiment includes acquiring the splicing data of each commodity display page within a preset time period and the monitoring data of all events triggered on each commodity display page, where the splicing data is formed by splicing the monitoring data of the events triggered on each commodity display page acquired within the preset time period, determining the monitoring number of the events triggered on each commodity display page according to the splicing data, determining the actual number of the events triggered on each commodity display page according to the monitoring data of all the events, and when it is determined that the monitoring number is inconsistent with the actual number, performing data splicing on the monitoring data of all the events to obtain the corrected splicing data of each commodity display page within the preset time period, and replacing the splicing data of each commodity display page within the preset time period with the corrected splicing data, so as to solve the problem that the splicing data can be detected and corrected according to the actual monitoring data without effective measures in the prior art, the splicing data stored in the database can be automatically detected and corrected according to the actual monitoring data.
Fig. 2 is a flowchart of a data checking method provided in an embodiment of the present disclosure, which may be combined with various alternatives in one or more embodiments described above, where in the embodiment, the event may include a click event, a purchase event, and a fee deduction event.
And determining the monitoring number of the triggering events on each commodity display page according to the splicing data, wherein the monitoring number comprises the following steps: respectively acquiring the number of click events, the number of purchase events and the number of fee deduction events triggered on each commodity display page according to the splicing data, and summing the numbers; and taking the summation result as the monitoring number of the trigger events on each commodity display page.
And, may further include: if the monitoring number is not consistent with the actual number, determining problem events according to the splicing data and the monitoring data of all events; and generating problem feedback information according to the monitoring data of the problem event.
As shown in fig. 2, the method may include the steps of:
In a specific example, it is determined that the number of click events triggered on a certain merchandise display page is 20, the number of triggered purchase events is 15, and the number of triggered deduction events is 5 according to the splicing data on the merchandise display page. The above numbers of events are summed and the result is 40.
And step 203, taking the summation result as the monitoring quantity of the trigger events on each commodity display page.
And step 204, determining the actual number of the trigger events on each commodity display page according to the monitoring data of all the events.
In one embodiment, the number of click events, the number of purchase events and the number of fee deduction events triggered on each merchandise display page are determined to be 21, 15 and 5 respectively according to the monitoring data of the click events, the purchase events and the fee deduction events triggered on the merchandise display page. The above numbers of events are summed and the result is 41.
And step 205, if the monitored quantity is not consistent with the actual quantity, performing data splicing on the monitored data of all events to obtain corrected spliced data of each commodity display page within a preset time period.
And step 206, replacing the splicing data of each commodity display page in the preset time period with the corrected splicing data.
And step 207, determining problem events according to the splicing data and the monitoring data of all events.
Wherein the problem event is an event that causes the monitored quantity to be inconsistent with the actual quantity. And if the monitored quantity is inconsistent with the actual quantity, determining the problem event according to the splicing data of each commodity display page in the preset time period and the monitoring data of the click event, the purchase event and the fee deduction event triggered on each commodity display page.
In a specific example, it is determined that the number of click events triggered on a certain merchandise display page is 20, the number of triggered purchase events is 15, and the number of triggered deduction events is 5 according to the splicing data on the merchandise display page. The above numbers of events are summed and the result is 40. And determining that the number of the click events triggered on each commodity display page is 21, the number of the triggered purchase events is 15 and the number of the triggered fee deduction events is 5 according to the monitoring data of the click events, the purchase events and the fee deduction events triggered on each commodity display page. The above numbers of events are summed and the result is 41. The monitored quantity is less than the actual quantity. And determining that the problem event is a certain click event according to the splicing data of the commodity display pages in the preset time period and the monitoring data of the click event, the purchase event and the fee deduction event triggered on each commodity display page.
And step 208, generating problem feedback information according to the monitoring data of the problem event.
After the problem event is determined according to the splicing data and the monitoring data of all the events, problem feedback information is generated according to the monitoring data of the problem event. The issue feedback information may include relevant details of the monitored data for the issue event. Such as a data source for the monitoring data for the problem event and a storage location for the monitoring data for the problem event.
Optionally, the problem feedback information is sent to a mailbox of the relevant responsible person to assist the relevant responsible person in locating the problem in the system.
According to the technical scheme of the embodiment, the splicing data of each commodity display page in a preset time period and the monitoring data of the click event, the purchase event and the fee deduction event triggered by each commodity display page are obtained; the method comprises the steps that splicing data are formed by splicing click events, purchase events and monitoring data of fee deduction events which are triggered at each commodity display page and acquired within a preset time period, then the number of the click events, the number of the purchase events and the number of the fee deduction events which are triggered at each commodity display page are acquired according to the splicing data respectively, the number of the fee deduction events and the number of the purchase events are summed up, the sum is used as the monitoring number of the event triggering at each commodity display page, when the monitoring number is judged to be inconsistent with the actual number, problem events are determined according to the splicing data and the monitoring data of all events, problem feedback information is generated, splicing data which are formed by splicing the click events, the purchase events and the monitoring data of the fee deduction events which are acquired within the preset time period and triggered at each commodity display page can be automatically detected and corrected according to the actual monitoring data, and problems can be generated according to the monitoring data of the problem events which cause the And the feedback information assists the relevant responsible person to locate the problems existing in the system.
Fig. 3 is a flowchart of a data checking method provided in an embodiment of the present disclosure, where this embodiment may be combined with each optional solution in one or more embodiments, in this embodiment, data splicing is performed on monitoring data of all events to obtain corrected spliced data of each commodity display page in a preset time period, where the data splicing method includes: the method comprises the steps of fragmenting monitoring data of all events to obtain a preset number of fragmented data packets; sending the fragment data packets to corresponding data splicing servers so that each data splicing server respectively performs data splicing on monitoring data in the fragment data packets; and acquiring the corrected splicing data of each commodity display page in the preset time period generated by each data splicing server.
As shown in fig. 3, the method may include the steps of:
301, acquiring splicing data of each commodity display page within a preset time period and monitoring data of all events triggered on each commodity display page; the splicing data is formed by splicing monitoring data of events triggered by the commodity display pages acquired within a preset time period.
And step 302, determining the monitoring number of the trigger events on each commodity display page according to the splicing data.
And step 304, if the monitoring number is judged to be inconsistent with the actual number, the monitoring data of all events are fragmented to obtain a preset number of fragmented data packets.
If the monitoring number is judged to be inconsistent with the actual number, the monitoring data of all events triggered on each commodity display page in the preset time period are fragmented to obtain fragmented data packets with the preset number.
Alternatively, the preset number may be equal to the number of data splicing servers. The data splicing server is used for splicing the monitoring data in the fragment data packet. That is, the number of the fragmented data packets is consistent with the number of the data splicing servers. Each data splicing server can acquire one fragment data packet for data splicing.
In a specific example, the number of events triggered on each commodity display page is determined according to the monitoring data of all events triggered on each commodity display page within a preset time period. And then, fragmenting the monitoring data based on the number of events triggered according to each commodity display page through a greedy algorithm: and distributing the monitoring data of all events triggered on the same commodity display page into the same fragment data packet, and enabling the difference value between the number of the events corresponding to the monitoring data in each fragment data packet to be within a preset difference value range. And summarizing the fragmented data packets to obtain a preset number of fragmented data packets.
The basic idea of the greedy algorithm is to proceed step by step from a certain initial solution of the problem, and according to a certain optimization measure, each step is required to ensure that a local optimal solution can be obtained. Only one data is considered in each step, and the selection of the data should meet the condition of local optimization. If the next data and partial optimal solution are no longer feasible solutions to join, the data is not added to the partial solution until all the data is enumerated, or the algorithm can no longer be added. According to the method and the device, the optimal scheme for fragmenting the monitoring data by utilizing the greedy algorithm iterative computation is utilized, the monitoring data corresponding to the same commodity display page are distributed into the same fragmentation data packet, and the number of events corresponding to the monitoring data in each fragmentation data packet is basically consistent.
Optionally, the fragmenting the monitoring data of all events to obtain a preset number of fragmented data packets may include: determining the number of events triggered on each commodity display page; aiming at each fragment data packet, distributing the monitoring data of all events triggered on the same commodity display page into the same fragment data packet; summarizing the fragmented data packets to obtain a preset number of fragmented data packets; and the number of events corresponding to the monitoring data in the fragment data packet is equal to the number of events triggered on at least one commodity display page.
The monitoring data of all events triggered on each commodity display page in a preset time period are distinguished according to the identification information of each commodity display page, and then the number of the events triggered on each commodity display page is respectively determined. For example, events include click events, purchase events, and debit events. According to the monitoring data of the click events, the purchase events and the fee deduction events triggered on a certain commodity display page, the number of the click events triggered on the commodity display page is 20, the number of the triggered purchase events is 15, and the number of the triggered fee deduction events is 5. The number of events triggered on the commodity display page is the sum of the number of the events. I.e. the number of events triggered on the merchandise display page is 40.
And (3) fragmenting the monitoring data based on the number of events triggered by each commodity display page: and distributing the monitoring data of all events triggered on the same commodity display page into the same fragment data packet, and enabling the difference value between the number of the events corresponding to the monitoring data in each fragment data packet to be within a preset difference value range. That is, it is ensured that the monitoring data corresponding to the same commodity display page is distributed into the same fragmented data packet, and the number of events corresponding to the monitoring data in each fragmented data packet is substantially consistent.
And after the monitoring data of all events triggered on each commodity display page within a preset time period are distributed to each fragment data packet, summarizing each fragment data packet to obtain a preset number of fragment data packets. The preset number is equal to the number of data splicing servers. The monitoring data of all events triggered on the same commodity display page are distributed into the same fragment data packet, so that the number of events corresponding to the monitoring data in the fragment data packet is equal to the number of events triggered on at least one commodity display page.
And 305, sending the fragmented data packets to corresponding data splicing servers, so that each data splicing server performs data splicing on the monitoring data in the fragmented data packets respectively.
And sending the fragment data packet to a corresponding data splicing server. For example, the preset number is equal to the number of data splicing servers. The number of data splicing servers is 5. And sending the obtained 5 fragmented data packets to corresponding data splicing servers, so that each data splicing server obtains one fragmented data packet. After each data splicing server acquires the fragment data packets, data splicing is carried out on the acquired monitoring data of the fragment data packets, and data splicing results are obtained. And the data splicing result is the corrected splicing data of each commodity display page in the preset time period.
In a specific example, the fragmented data packet sent to the data splicing server only includes monitoring data of all events triggered on one commodity display page. And the data splicing server for acquiring the fragment data packet performs data splicing on the acquired monitoring data of the fragment data packet to obtain a data splicing result of the commodity display page. Namely, all monitoring data corresponding to a commodity display page are spliced to obtain a data table corresponding to the commodity display page.
In another embodiment, the fragmented data packet sent to the data splicing server includes monitoring data of all events triggered on the multiple merchandise display pages. After the data splicing server of the fragment data packet acquires the monitoring data of the acquired fragment data packet, the monitoring data of all events triggered by a plurality of commodity display pages are distinguished according to the identification information of each commodity display page, and data splicing is respectively carried out to obtain the data splicing result corresponding to each commodity display page. That is, all the monitoring data of the plurality of commodity display pages included in the fragmented data packet are spliced to obtain a data table corresponding to each commodity display page.
Therefore, each data splicing server only needs to be responsible for the corresponding data splicing task, and the problems that when the data volume of the monitored data is huge, the server is overloaded, the server is blocked, and the data splicing and storing process is affected are solved.
And step 306, acquiring the corrected splicing data of each commodity display page in the preset time period generated by each data splicing server.
And 307, replacing the splicing data of each commodity display page in the preset time period with the corrected splicing data.
According to the technical scheme, the monitoring data of all events are fragmented to obtain the fragmented data packets with the preset number, and then the fragmented data packets are sent to the corresponding data splicing servers, so that the data splicing servers respectively perform data splicing on the monitoring data in the fragmented data packets, the correction splicing data of each commodity display page in the preset time period generated by each data splicing server is obtained, the monitoring data can be fragmented according to the events triggered by each commodity display page, and therefore the data splicing tasks are distributed.
Fig. 4 is a schematic structural diagram of a data inspection apparatus according to an embodiment of the present disclosure, which is applicable to a situation of inspecting spliced data. The apparatus can be implemented in software and/or hardware, and the apparatus can be configured in an electronic device. As shown in fig. 4, the apparatus may include: a data acquisition module 401, a first quantity determination module 402, a second quantity determination module 403, a data splicing module 404, and a data replacement module 405.
The data acquisition module 401 is configured to acquire splicing data of each commodity display page within a preset time period and monitoring data of all events triggered on each commodity display page; the splicing data is formed by splicing monitoring data of events triggered by the commodity display pages acquired within a preset time period; a first quantity determining module 402, configured to determine, according to the splicing data, the monitoring quantity of the trigger events on each commodity display page; a second quantity determining module 403, configured to determine, according to the monitoring data of all events, an actual quantity of trigger events on each commodity display page; the data splicing module 404 is configured to perform data splicing on the monitoring data of all events to obtain corrected splicing data of each commodity display page within a preset time period if it is determined that the monitoring number is inconsistent with the actual number; and the data replacement module 405 is configured to replace the mosaic data of each commodity display page within the preset time period with the corrected mosaic data.
The technical scheme of this embodiment includes acquiring the splicing data of each commodity display page within a preset time period and the monitoring data of all events triggered on each commodity display page, where the splicing data is formed by splicing the monitoring data of the events triggered on each commodity display page acquired within the preset time period, determining the monitoring number of the events triggered on each commodity display page according to the splicing data, determining the actual number of the events triggered on each commodity display page according to the monitoring data of all the events, and when it is determined that the monitoring number is inconsistent with the actual number, performing data splicing on the monitoring data of all the events to obtain the corrected splicing data of each commodity display page within the preset time period, and replacing the splicing data of each commodity display page within the preset time period with the corrected splicing data, so as to solve the problem that the splicing data can be detected and corrected according to the actual monitoring data without effective measures in the prior art, the splicing data stored in the database can be automatically detected and corrected according to the actual monitoring data.
Optionally, on the basis of the above technical solution, the event includes a click event, a purchase event, and a fee deduction event; the first quantity determination module 402 may include: the quantity summing unit is used for respectively acquiring the number of click events, the number of purchase events and the number of fee deduction events triggered on each commodity display page according to the splicing data and summing the numbers; and the quantity determining unit is used for taking the summation result as the monitoring quantity of the trigger events on each commodity display page.
Optionally, on the basis of the above technical solution, the method may further include: the time determining module is used for determining the problem events according to the splicing data and the monitoring data of all events if the monitoring number is inconsistent with the actual number; and the information generation module is used for generating the problem feedback information according to the monitoring data of the problem event.
Optionally, on the basis of the foregoing technical solution, the data splicing module 404 may include: the data fragmentation unit is used for fragmenting the monitoring data of all events to obtain a preset number of fragmentation data packets; the data packet sending unit is used for sending the fragment data packets to the corresponding data splicing servers so that the data splicing servers respectively perform data splicing on the monitoring data in the fragment data packets; and the data acquisition unit is used for acquiring the corrected splicing data of each commodity display page in the preset time period generated by each data splicing server.
Optionally, on the basis of the above technical solution, the data slicing unit may include: the quantity determining subunit is used for determining the quantity of the events triggered on each commodity display page; the data distribution subunit is used for distributing the monitoring data of all events triggered on the same commodity display page to the same fragmented data packet aiming at each fragmented data packet; the collecting subunit is used for collecting the fragmented data packets to obtain a preset number of fragmented data packets; and the number of events corresponding to the monitoring data in the fragment data packet is equal to the number of events triggered on at least one commodity display page.
Optionally, on the basis of the above technical solution, the preset number is equal to the number of the data splicing servers.
The data inspection device provided by the embodiment of the disclosure can execute the data inspection method provided by the embodiment of the disclosure, and has corresponding functional modules and beneficial effects of the execution method.
Referring now to fig. 5, a schematic diagram of an electronic device (e.g., a terminal device or a server) 500 suitable for use in implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure when executed by the processing device 501.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring splicing data of each commodity display page within a preset time period and monitoring data of all events triggered on each commodity display page; the splicing data is formed by splicing monitoring data of events triggered by the commodity display pages acquired within a preset time period; determining the monitoring number of the triggering events on each commodity display page according to the splicing data; determining the actual number of the triggering events on each commodity display page according to the monitoring data of all the events; if the monitored quantity is inconsistent with the actual quantity, performing data splicing on the monitored data of all events to obtain corrected spliced data of each commodity display page within a preset time period; and replacing the splicing data of each commodity display page in the preset time period by adopting the corrected splicing data.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules and units described in the embodiments of the present disclosure may be implemented by software or hardware. For example, the data replacement module may be further described as a "module for replacing the splicing data of each commodity display page within a preset time period with the corrected splicing data", and the quantity summing unit may be further described as a "unit for respectively obtaining and summing the number of click events, the number of purchase events, and the number of fee deduction events triggered on each commodity display page according to the splicing data".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Claims (10)
1. A method of data inspection, comprising:
acquiring splicing data of each commodity display page within a preset time period and monitoring data of all events triggered at each commodity display page within the preset time period; the splicing data is formed by splicing monitoring data of events triggered at each commodity display page acquired within the preset time period, and the monitoring data of all the events triggered at each commodity display page within the preset time period is actual monitoring data of the events triggered at each commodity display page;
determining the monitoring number of the triggering events on each commodity display page according to the splicing data;
determining the actual number of the triggering events on each commodity display page according to the monitoring data of all the events;
if the monitored quantity is not consistent with the actual quantity, performing data splicing on the monitored data of all events to obtain corrected splicing data of each commodity display page in the preset time period;
and replacing the splicing data of each commodity display page in the preset time period by adopting the corrected splicing data.
2. The method of claim 1, wherein the events include click events, purchase events, and debit events;
the determining the monitoring number of the triggering events on each commodity display page according to the splicing data comprises the following steps:
respectively acquiring the number of click events, the number of purchase events and the number of fee deduction events triggered on each commodity display page according to the splicing data, and summing the numbers;
and taking the summation result as the monitoring number of the trigger events on each commodity display page.
3. The method of claim 1, further comprising:
if the monitoring number is not consistent with the actual number, determining problem events according to the splicing data and the monitoring data of all events;
and generating problem feedback information according to the monitoring data of the problem event.
4. The method according to claim 1, wherein the performing data splicing on the monitoring data of all the events to obtain the corrected spliced data of each commodity display page in the preset time period comprises:
fragmenting the monitoring data of all events to obtain a preset number of fragmented data packets;
sending the fragment data packets to corresponding data splicing servers so that each data splicing server respectively performs data splicing on monitoring data in the fragment data packets;
and acquiring the corrected splicing data of each commodity display page in the preset time period, which is generated by each data splicing server.
5. The method according to claim 4, wherein the fragmenting the monitoring data of all the events to obtain a preset number of fragmented data packets comprises:
determining the number of events triggered on each commodity display page;
aiming at each fragment data packet, distributing the monitoring data of all events triggered on the same commodity display page into the same fragment data packet;
summarizing the fragment data packets to obtain a preset number of fragment data packets; and the number of events corresponding to the monitoring data in the fragment data packet is equal to the number of events triggered on at least one commodity display page.
6. The method of claim 4, wherein the preset number is equal to the number of data splicing servers.
7. A data inspection apparatus, comprising:
the data acquisition module is used for acquiring the splicing data of each commodity display page in a preset time period and the monitoring data of all events triggered by each commodity display page in the preset time period; the splicing data is formed by splicing monitoring data of events triggered at each commodity display page acquired within the preset time period, and the monitoring data of all the events triggered at each commodity display page within the preset time period is actual monitoring data of the events triggered at each commodity display page;
the first quantity determining module is used for determining the monitoring quantity of the triggering events on each commodity display page according to the splicing data;
the second quantity determining module is used for determining the actual quantity of the trigger events on each commodity display page according to the monitoring data of all the events;
the data splicing module is used for splicing the monitoring data of all the events to obtain corrected splicing data of each commodity display page within the preset time period if the monitored number is judged to be inconsistent with the actual number;
and the data replacement module is used for replacing the splicing data of each commodity display page in the preset time period by adopting the corrected splicing data.
8. The apparatus of claim 7, wherein the events include click events, purchase events, and debit events;
the first quantity determination module includes:
the quantity summing unit is used for respectively acquiring the number of click events, the number of purchase events and the number of fee deduction events triggered on each commodity display page according to the splicing data and summing the numbers;
and the quantity determining unit is used for taking the summation result as the monitoring quantity of the trigger events on each commodity display page.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a data inspection method as claimed in any one of claims 1 to 6.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data checking method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811435873.XA CN109542921B (en) | 2018-11-28 | 2018-11-28 | Data checking method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811435873.XA CN109542921B (en) | 2018-11-28 | 2018-11-28 | Data checking method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109542921A CN109542921A (en) | 2019-03-29 |
| CN109542921B true CN109542921B (en) | 2021-03-02 |
Family
ID=65852266
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811435873.XA Active CN109542921B (en) | 2018-11-28 | 2018-11-28 | Data checking method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109542921B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105404469A (en) * | 2015-10-22 | 2016-03-16 | 浙江宇视科技有限公司 | Video data storage method and system |
| CN106507406A (en) * | 2016-11-30 | 2017-03-15 | 四川九洲电器集团有限责任公司 | A kind of equipment of wireless network accesses the Forecasting Methodology of number and equipment |
| CN106776150A (en) * | 2017-01-04 | 2017-05-31 | 上海上讯信息技术股份有限公司 | A kind of method and apparatus for obtaining transaction journal dump file information |
| CN107918523A (en) * | 2016-10-11 | 2018-04-17 | 慧荣科技股份有限公司 | Data storage device and data writing method thereof |
| CN108255672A (en) * | 2017-12-29 | 2018-07-06 | 东软集团股份有限公司 | The method, apparatus and storage medium and electronic equipment of data check |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9742818B2 (en) * | 2014-12-10 | 2017-08-22 | Oracle International Corporation | Pushing events to web pages used for interaction with applications |
-
2018
- 2018-11-28 CN CN201811435873.XA patent/CN109542921B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105404469A (en) * | 2015-10-22 | 2016-03-16 | 浙江宇视科技有限公司 | Video data storage method and system |
| CN107918523A (en) * | 2016-10-11 | 2018-04-17 | 慧荣科技股份有限公司 | Data storage device and data writing method thereof |
| CN106507406A (en) * | 2016-11-30 | 2017-03-15 | 四川九洲电器集团有限责任公司 | A kind of equipment of wireless network accesses the Forecasting Methodology of number and equipment |
| CN106776150A (en) * | 2017-01-04 | 2017-05-31 | 上海上讯信息技术股份有限公司 | A kind of method and apparatus for obtaining transaction journal dump file information |
| CN108255672A (en) * | 2017-12-29 | 2018-07-06 | 东软集团股份有限公司 | The method, apparatus and storage medium and electronic equipment of data check |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109542921A (en) | 2019-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111679990B (en) | Test data generation method and device, readable medium and electronic equipment | |
| CN113806212A (en) | Application program exception positioning method and device and electronic equipment | |
| CN115660589A (en) | Business auditing method, device, equipment, computer readable medium and program product | |
| US20220269504A1 (en) | Client-side enrichment and transformation via dynamic logic for analytics | |
| CN109902726B (en) | Resume information processing method and device | |
| CN113495820A (en) | Method and device for collecting and processing abnormal information and abnormal monitoring system | |
| CN112954056B (en) | Method and device for processing monitoring data, electronic equipment and storage medium | |
| CN112884376A (en) | Work order processing method and device, electronic equipment and computer readable storage medium | |
| CN109542743B (en) | Log checking method and device, electronic equipment and computer readable storage medium | |
| CN109542921B (en) | Data checking method and device, electronic equipment and storage medium | |
| CN109947659B (en) | System, method and apparatus for testing applications | |
| CN109522133B (en) | Data splicing method and device, electronic equipment and storage medium | |
| CN114913005A (en) | Mobile risk data detection method, mobile risk data detection system, electronic device and storage medium | |
| CN111741046B (en) | Data reporting method, data acquisition method, device, equipment and medium | |
| CN115203178A (en) | Data quality inspection method and device, electronic equipment and storage medium | |
| CN114116480A (en) | Method, device, medium and equipment for determining application program test coverage rate | |
| CN109960659B (en) | Method and device for detecting application program | |
| CN113760768A (en) | Test method, monitoring platform, electronic equipment and storage medium | |
| CN112416989A (en) | Management method and device of Internet performance broker platform and electronic equipment | |
| CN112445697A (en) | Method and apparatus for testing applications | |
| CN112184406A (en) | Data processing method, system, electronic device and computer readable storage medium | |
| CN110875856A (en) | Method and apparatus for activation data anomaly detection and analysis | |
| CN110471841B (en) | Method, device, medium and electronic equipment for comparing drawing information | |
| CN110489341B (en) | Test method and device, storage medium and electronic equipment | |
| CN117824974B (en) | Switch drop test method, device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |