CN108093275B - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN108093275B CN108093275B CN201611050440.3A CN201611050440A CN108093275B CN 108093275 B CN108093275 B CN 108093275B CN 201611050440 A CN201611050440 A CN 201611050440A CN 108093275 B CN108093275 B CN 108093275B
- Authority
- CN
- China
- Prior art keywords
- data
- audience rating
- data processing
- rating data
- processing type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25866—Management of end-user data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/4508—Management of client data or end-user data
Abstract
The invention discloses a data processing method and a data processing device, relates to the technical field of internet, and mainly aims to improve the processing efficiency of audience rating data. The method comprises the following steps: acquiring audience rating data; configuring a priority rule for the audience rating data according to the indication information; determining a data processing type according to the data characteristics corresponding to the audience rating data; and processing the audience rating data according to the determined data processing type and the priority rule. The method is mainly used for processing the audience rating data.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a data processing method and device.
Background
The audience rating is the percentage of the number of people watching a certain television program in a certain period of time to the total number of people watching the television, and the audience rating analysis is the basis of the television audience rating market, and is the main index for evaluating the television program.
The audience rating data processing is the basis of audience rating analysis, the audience rating analysis system can normally process the audience rating data under the condition that a client provides the audience rating data, and once the client cannot provide the audience rating data in time or meets an abnormal condition during the audience rating data processing, a worker needs to manually operate to reprocess the audience rating data, so that the audience rating data processing mode is not flexible enough, and the audience rating data processing efficiency is not high.
Disclosure of Invention
In view of the above problems, the present invention has been made to provide a data processing method and apparatus capable of improving the processing efficiency of ratings data, which overcomes or at least partially solves the above problems.
In one aspect, the present invention provides a data processing method, including:
acquiring audience rating data;
configuring a priority rule for the audience rating data according to the indication information;
determining a data processing type according to the data characteristics corresponding to the audience rating data;
and processing the audience rating data according to the determined data processing type and the priority rule.
Further, when the indication information indicates that the data is sorted according to date, the configuring a priority rule for the audience rating data according to the indication information includes:
sequencing the audience rating data in a positive sequence according to dates to configure a priority rule for the audience rating data; or
Sorting the audience rating data in a reverse order according to dates to configure a priority rule for the audience rating data; or
Circularly sequencing the audience rating data according to dates to configure priority rules for the audience rating data;
when the indication information indicates that the data processing is performed according to the frequency of data processing failure, configuring a priority rule for the audience rating data according to the indication information includes:
and sequencing the audience rating data according to the number of times of data processing failure to configure a priority rule for the audience rating data.
Further, the determining the data processing type according to the data characteristics corresponding to the audience rating data includes:
acquiring a data processing log corresponding to the audience rating data;
extracting data characteristics of the data processing log;
and determining the data processing type corresponding to the audience rating data according to the data characteristics.
Further, the data processing type comprises a normal data processing type, an updated data processing type, an incremental data processing type and an abnormal data processing type;
the determining the data processing type according to the data characteristics corresponding to the audience rating data comprises:
if the audience rating data does not have a corresponding historical processing record, determining that the data processing type corresponding to the audience rating data is a normal data processing type;
if the audience rating data is the update of part of original data, determining the data processing type corresponding to the audience rating data as an update data processing type;
if the audience rating data is the update of the incremental data, determining that the data processing type corresponding to the audience rating data is the incremental data processing type;
and if the audience rating data is abnormal data which fails to be processed, determining that the data processing type corresponding to the audience rating data is an abnormal data processing type.
Further, the method further comprises:
and carrying out statistics on the processed audience rating data to obtain an audience rating statistical result.
In another aspect, the present invention provides a data processing apparatus comprising:
an acquisition unit for acquiring audience rating data;
the configuration unit is used for configuring a priority rule for the audience rating data according to the indication information;
the determining unit is used for determining a data processing type according to the data characteristics corresponding to the audience rating data;
and the processing unit is used for processing the audience rating data according to the determined data processing type and the priority rule.
Further, when the indication information indicates that the audience rating data is sorted according to dates, the configuration unit is specifically configured to forward sort the audience rating data according to dates to configure a priority rule for the audience rating data; or
The configuration unit is specifically configured to sort the audience rating data in reverse order according to dates to configure priority rules for the audience rating data; or
The configuration unit is specifically configured to perform circular sequencing on the audience rating data according to dates to configure priority rules for the audience rating data;
when the indication information indicates that the data processing fails to be performed according to the number of times of data processing, the configuration unit is specifically further configured to sequence the audience rating data according to the number of times of data processing failure to configure the priority rule for the audience rating data.
Further, the determining unit includes:
the acquisition module is used for acquiring a data processing log corresponding to the audience rating data;
the extraction module is used for extracting the data characteristics of the data processing log;
and the determining module is used for determining the data processing type corresponding to the audience rating data according to the data characteristics.
Further, the data processing type comprises a normal data processing type, an updated data processing type, an incremental data processing type and an abnormal data processing type;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is a normal data processing type if the corresponding history processing record does not exist in the audience rating data;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is an updated data processing type if the audience rating data is an update of part of original data;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is an incremental data processing type if the audience rating data is updated by incremental data;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is an abnormal data processing type if the audience rating data is abnormal data that has failed to be processed.
Further, the apparatus further comprises:
and the statistical unit is used for carrying out statistics on the processed audience rating data to obtain an audience rating statistical result.
By means of the technical scheme, the audience rating data are obtained firstly, then the priority rules are configured for the audience rating data according to the indication information, the priority rules can be different priority rules preset according to different user requirements, the data processing types are further determined according to data characteristics corresponding to the audience rating data, different data processing types are adopted for the audience rating data of different types in a more targeted mode, and finally the audience rating data are processed according to the determined data processing types and the priority rules, so that the high-efficiency processing efficiency is obtained. Compared with the prior art that the audience rating data is processed through manual operation under the condition of abnormal data, the method has the advantages that the priority rule is arranged on the audience rating data in advance, different processing types are adopted according to different data types, the stability and the safety of the data in the data processing process are guaranteed, manual operation is not needed, the audience rating data with different data characteristics is further processed according to the priority rule, the audience rating data can be processed rapidly in batches, and the audience rating data processing efficiency is improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a schematic flow chart illustrating a data processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart of another data processing method provided by the embodiment of the invention;
FIG. 3 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
An embodiment of the present invention provides a data processing method, as shown in fig. 1, the method is mainly used for processing audience rating data, and includes the following specific steps:
101. audience rating data is obtained.
The audience rating is the percentage of the target audience who watches the television program in a certain time slot to the total target population, and is expressed by percentage. Generally, the audience rating of family members can be recorded in detail by installing a special measuring instrument in a user segment, then, data collected by each measuring instrument is counted, and audience rating data is further obtained. It should be noted that, in the embodiment of the present invention, the method for acquiring the audience rating data is not limited, and may be specifically selected according to actual situations.
The embodiment of the invention can be realized by a digital set-top box, a local audience rating data statistical function of the digital set-top box is needed, and the function can be realized by set-top box software because the digital set-top box is provided with an independent CPU system. The data to be collected can include viewing and operating data such as user startup, channel change, access to set top box menu, and the like, and the viewing rate data can be obtained without intervention of other measuring instruments.
102. And configuring a priority rule for the audience rating data according to the indication information.
Generally, audience rating data is provided by days, so that the audience rating data is processed by days, however, when the provided audience rating data is data for multiple days, because the data volume is large, some data needing emergency processing cannot be effectively processed according to the original rule, so that a priority level needs to be arranged for processing the audience rating data, and the audience rating data can be timely processed.
The priority rule may be configured according to actual needs of a user, for example, when the indication information is sorted according to dates, the priority rule may be sorted according to a forward order of dates, sorted according to a reverse order of dates, or sorted according to a date cycle manner, when the indication information is sorted according to the number of data processing failures, the priority rule may be sorted according to the number of data processing failures, and the data with the larger number of processing failures has the higher priority.
103. And determining the data processing type according to the data characteristics corresponding to the audience rating data.
The data characteristics here mainly refer to original data information corresponding to the audience rating data, and if the data is a history record in a database, whether the audience rating data is updated by the original data, whether the data is data that has failed to be processed abnormally, or the like, the data processing type is further determined according to the data characteristics corresponding to the audience rating data.
For example, if the audience rating data of the day has no history processing record in the database, the audience rating data of the day is represented as new data, and the data processing type is determined to be a normal data processing type; if the audience rating data of the day is the incremental data of the historical data, the audience rating data of the day needs to be added to the historical data, the data processing type is determined to be the incremental data processing type, if the audience rating data of the day is the incremental data of the historical data, the data is represented to be the error data, and the data processing type is determined to be the abnormal data processing type.
104. And processing the audience rating data according to the determined data processing type and the priority rule.
The embodiment of the invention further processes the audience rating data with the priority rule by adopting the determined data processing type, obtains different data processing modes for the audience rating data with different data characteristics, and ensures the accuracy of the data in the data processing process.
It can be seen from the foregoing implementation manner that, in the data processing method provided in the embodiment of the present invention, the audience rating data is obtained first, then the priority rule is configured for the audience rating data according to the indication information, where the priority rule may be different priority rules preset according to different user requirements, a data processing type is further determined according to data characteristics corresponding to the audience rating data, different data processing types are adopted for the audience rating data of different types with more pertinence, and finally, the audience rating data is processed according to the determined data processing type and the priority rule, so as to obtain higher processing efficiency. Compared with the prior art that the audience rating data is processed through manual operation under the condition of abnormal data, the method has the advantages that the priority rule is arranged on the audience rating data in advance, different processing types are adopted according to different data types, the stability and the safety of the data in the data processing process are guaranteed, manual operation is not needed, the audience rating data with different data characteristics is further processed according to the priority rule, the audience rating data can be processed rapidly in batches, and the audience rating data processing efficiency is improved.
In order to describe a data processing method proposed by the present invention in more detail, especially for a step of extracting data features corresponding to audience rating data when performing data processing, another data processing method is further provided in an embodiment of the present invention, as shown in fig. 2, and the specific steps of the method include:
201. audience rating data is obtained.
The audience rating data is the data volume which is acquired by digital set-top box software and can reflect that a user watches television programs in a certain time period, wherein whether the audience rating data is real or effective is closely related to the setting of data acquisition in the set-top box software, and the acquired data is required to be accurate and effective as much as possible.
Furthermore, the data size of the audience rating data is huge and the audience rating data of different television stations and different television programs are diverse, for example, the audience rating data of a hot-broadcast television play in a golden time period is much larger than that of other television plays, so that the audience rating data needs to be processed and analyzed, and the audience rating data is effectively utilized.
202. And configuring a priority rule for the audience rating data according to the date sequence of the audience rating data.
Since the statistics of the ratings data are typically obtained on a daily basis, and of course, on a monthly or preset time interval basis, therefore, when the audience rating analyzing system receives the audience rating data, firstly, the audience rating data is sorted according to the date according to the recording date of the audience rating data to obtain the audience rating data of different dates, for example, 3 shares of rating data are received in total on 9/14 th day in 2015, the rating data on the day of 9/14 th day, the rating data on 9/10 th day and the rating data on 9/7/9/8 th day, for embodiments of the present invention the dates may be ordered in the order of the rating data dates from early to late, of course, the date sorting may be performed in the order of the rating data date from late to early, or the date sorting may be performed in a forward or reverse order according to the rating data date.
It should be noted that the priority rule may be configured by a common date preference, specifically, different priority rules may be configured in the date preference according to actual needs of a user, and in the embodiment of the present invention, the priority rule may be configured for the audience rating data according to a data date or the number of times of data processing failure.
203. Acquiring a data processing log corresponding to the audience rating data;
the data processing log is mainly used for recording original data information, and further can judge whether the data is incremental data, whether the data is updating data, whether the data is full data, whether the data is abnormal data or not through the original data information, and further extracts data characteristics of audience rating data. The incremental data refers to that the data is new data, the original data of the new data is recorded in the original data, the updated data refers to that the data needs to be corrected due to errors, the data of the updated record is marked in the original data, the full data refers to that the data is complete data, no record of the original data exists, the abnormal data refers to that the data is left due to processing failure, and the data of the abnormal record is marked in the original data.
204. And extracting the data characteristics of the data processing log.
Because different audience rating data have different data characteristics, the embodiment of the invention needs to adopt different data processing types for the audience rating data with different data characteristics, thereby further ensuring the accuracy of data processing. Therefore, before the audience rating data is processed, the data characteristics of the data processing log need to be extracted so as to obtain the data characteristics of the audience rating data.
205. And determining the data processing type corresponding to the audience rating data according to the data characteristics.
For the embodiment of the present invention, after the data features of the audience rating data are extracted in step 204, different data processing types are adopted according to different data features of the audience rating data, when the data is incremental data, the data is newly added audience rating data, an incremental data processing type is adopted, when the data is updated data, the data is audience rating data to be corrected, an updated data processing type is adopted, when the data is full data, the data is complete audience rating data, a normal data processing type is adopted, and when the data is abnormal data, the data is data with processing failure, an abnormal data processing type is adopted.
In addition, the embodiment of the present invention may further select a data processing type according to a log date corresponding to the audience rating data, for example, if the log date of the audience rating data appears for the first time, the audience rating data is normal data, a normal processing type is selected for the data, and if the log date of the audience rating data appears for multiple times, the data is not normal data, and operations such as modification or addition need to be performed on the data, so as to further select the data processing type.
206. And processing the audience rating data according to the determined data processing type and the priority rule.
For the embodiment of the invention, different data processing types are corresponded to different audience rating data, so different data processing flows are started, and some common processes exist in the different data processing flows, so that a plurality of data processing modules can be arranged in the common processing flow in the development design of products, and further different modules are selected for different audience rating data to complete the whole data processing work.
The data processing module can comprise an incremental original data automatic docking module, a file information monitoring scanning module, a daily data processing module, an incremental data processing module, an error data deleting module, a processing component configuration module and an ETL processing module.
It should be noted that, due to the abnormal situation, the processing of the audience rating data may fail, when the abnormal situation occurs, it is determined to which step the audience rating data is processed, then the audience rating data processed in the step is stored, and after the system is repaired, the stored audience rating data is automatically started, so that the audience rating data before the abnormal situation is continuously processed.
207. And carrying out statistics on the processed audience rating data to obtain an audience rating statistical result.
The processed audience rating data records data of different television programs, so that the audience rating data needs to be counted according to the different television programs in order to obtain the audience ratings of the different programs, and the audience rating data of the different television programs is obtained.
The specific steps of the embodiment of the present invention may include, but are not limited to, the following implementation manners: firstly, acquiring three shares of audience rating data by installing set top box software, further sequencing the audience rating data according to dates according to user requirements, further configuring priority rules for the audience rating data according to the sequence of the dates, sequencing the three shares of audience rating data from early to late according to the sequence of the dates, then respectively extracting data characteristics of the three shares of audience rating data according to original data information recorded in a data processing log, wherein the first share of audience rating data is full data, the second share of audience rating data is incremental data, the third share of audience rating data is abnormal data, different data processing types are selected according to the data characteristics corresponding to the audience rating data, the first share of audience rating data does not have any record in the original data, a normal data processing type is selected, the second share of audience rating data records partial original data in the original data, the incremental data processing type is selected, and the third share of audience rating data records data which fails to be processed in the original data, selecting an abnormal data processing type, processing the audience rating data according to the selected data processing type and the arranged priority rule, and further, in order to ensure that the audience ratings of different programs are obtained, counting the processed audience rating data according to different television programs to obtain the audience rating counting results of the different television programs so as to analyze the audience ratings of the different television programs.
According to the other data processing method provided by the embodiment of the invention, the requirement of a user on the timeliness of data is met by configuring the priority rule for the audience rating data, the matched data processing mode can be found at the fastest speed by configuring different data processing methods for the audience rating data with different data characteristics, the automatic processing mode can be completely realized aiming at various abnormal conditions in the data processing, the integrity and the accuracy of the data in the data processing process are ensured, manual intervention is not needed in the whole data processing process, and the processing efficiency of the audience rating data is improved.
Further, as a specific implementation of the method shown in fig. 1, an embodiment of the present invention provides a data processing apparatus, where the apparatus embodiment corresponds to the foregoing method embodiment, and for convenience of reading, details in the foregoing method embodiment are not described in detail by the apparatus, but it should be clear that the apparatus in the present embodiment can correspondingly implement all the contents in the foregoing method embodiment, as shown in fig. 3, the apparatus includes:
the acquiring unit 31 may be configured to acquire audience rating data, where the acquiring unit 31 is a main function module in the apparatus for acquiring audience rating data, and specifically may collect audience rating data through digital set-top box software;
the configuration unit 32 may be configured to configure a priority rule for the audience rating data according to the indication information, where the configuration unit 32 is a main function module in the device configured with the priority rule, and may specifically perform priority sorting according to dates and may also perform priority sorting according to data failure times;
the determining unit 33 may be configured to determine a data processing type according to the data feature corresponding to the audience rating data, where the determining unit 33 is a main function module in the apparatus that determines the data processing type of the audience rating data, and specifically may determine the data processing type according to the data feature corresponding to the audience rating data acquired by the acquiring unit 31;
the processing unit 34 may be configured to process the audience rating data according to the determined data processing type and the priority rule, where the processing unit is a main functional module in the present apparatus for processing the audience rating data, and specifically configured to process the audience rating data acquired by the acquiring unit 31 according to the data processing type determined by the determining unit 33 and the priority rule.
The data processing device provided by the embodiment of the invention firstly obtains the audience rating data, then configures the priority rules for the audience rating data according to the indication information, wherein the priority rules can be different priority rules preset according to different user requirements, further determines the data processing types according to the data characteristics corresponding to the audience rating data, adopts different data processing types for the audience rating data of different types in a more targeted manner, and finally processes the audience rating data according to the priority rules according to the determined data processing types, so as to obtain more efficient processing efficiency. Compared with the prior art that the audience rating data is processed through manual operation under the condition of abnormal data, the method has the advantages that the priority rule is arranged on the audience rating data in advance, different processing types are adopted according to different data types, the stability and the safety of the data in the data processing process are guaranteed, manual operation is not needed, the audience rating data with different data characteristics is further processed according to the priority rule, the audience rating data can be processed rapidly in batches, and the audience rating data processing efficiency is improved.
Further, as a specific implementation of the method shown in fig. 2, an embodiment of the present invention provides another statement detection apparatus, where an embodiment of the apparatus corresponds to the foregoing method embodiment, and for convenience of reading, details in the foregoing method embodiment are not described in detail by the apparatus one by one, but it should be clear that the apparatus in this embodiment can correspondingly implement all the contents in the foregoing method embodiment, as shown in fig. 4, the apparatus includes:
the acquisition unit 41 may be configured to acquire audience rating data, where the acquisition unit 41 is a main function module in the apparatus for acquiring audience rating data, and specifically may collect audience rating data through digital set-top box software;
the configuration unit 42 may be configured to configure a priority rule for the audience rating data according to the indication information, where the configuration unit 42 is a main function module in the device configured with the priority rule, and may specifically perform priority sorting according to dates and may also perform priority sorting according to data failure times;
the determining unit 43 may be configured to determine a data processing type according to the data feature corresponding to the audience rating data, where the determining unit 43 is a main function module in the present apparatus that determines the data processing type of the audience rating data, and specifically may determine the data processing type according to the data feature corresponding to the audience rating data acquired by the acquiring unit 41;
the processing unit 44 may be configured to process the audience rating data according to the determined data processing type and the priority rule, where the processing unit 44 is a main function module in the present apparatus for processing the audience rating data, and specifically configured to process the audience rating data acquired by the acquiring unit 41 according to the data processing type determined by the determining unit 43 and the priority rule;
the statistical unit 45 may be configured to perform statistics on the processed audience rating data to obtain an audience rating statistical result, where the statistical unit 45 is a main function module of the apparatus for statistically collecting audience rating data, and is specifically configured to perform statistics on the audience rating data obtained by the processing unit 44 to obtain an audience rating statistical result.
Further, when the indication information indicates that the sorting is performed according to the date,
the configuration unit 42 is specifically configured to perform positive ordering on the audience rating data according to dates according to the indication information to obtain ordered audience rating data; or
The configuration unit 42 is further specifically configured to sort the audience rating data in reverse order according to dates according to the indication information, so as to obtain sorted audience rating data; or
The configuration unit 42 is further specifically configured to perform circular sorting on the audience rating data according to dates according to the indication information, so as to obtain sorted audience rating data.
Further, when the indication information indicates that the data processing fails to be sorted according to the number of times of the data processing,
the configuration unit 42 is further specifically configured to sort the audience rating data according to the number of times of data processing failure according to the indication information, so as to obtain sorted audience rating data, where the greater the number of times of failure, the higher the priority of sorting.
Further, the determination unit 43 includes:
an obtaining module 431, configured to obtain a data processing log corresponding to the audience rating data;
an extracting module 432, configured to extract data features of the data processing log;
a determining module 433, configured to determine a data processing type corresponding to the audience rating data according to the data feature.
Further, the data processing type comprises a normal data processing type, an updated data processing type, an incremental data processing type and an abnormal data processing type;
the determining unit 43 is specifically configured to determine that the data processing type corresponding to the audience rating data is a normal data processing type if the corresponding history processing record does not exist in the audience rating data;
the determining unit 43 is further specifically configured to determine that a data processing type corresponding to the audience rating data is an updated data processing type if the audience rating data is an update of a part of original data;
the determining unit 43 is further specifically configured to determine that the data processing type corresponding to the audience rating data is an incremental data processing type if the audience rating data is updated by incremental data;
the determining unit 43 is specifically further configured to determine that the data processing type corresponding to the audience rating data is an abnormal data processing type if the audience rating data is abnormal data that has failed to be processed
According to the other data processing device provided by the embodiment of the invention, the requirement of a user on data timeliness is met by configuring the priority rule for the audience rating data, the matched data processing mode can be found at the fastest speed by configuring different data processing methods for the audience rating data with different data characteristics, the automatic processing mode can be completely realized for various abnormal conditions in data processing, the integrity and the accuracy of the data in the data processing process are ensured, manual intervention is not needed in the whole data processing process, and the processing efficiency of the audience rating data is improved.
The statement detection device comprises a processor and a memory, wherein the acquisition unit 31, the configuration unit 32, the determination unit 33, the processing unit 34 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, labor is saved by adjusting kernel parameters, and the processing efficiency of audience rating data can be improved.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The present application further provides a computer program product adapted to perform program code for initializing the following method steps when executed on a data processing device: acquiring audience rating data; configuring a priority rule for the audience rating data according to the indication information; determining a data processing type according to the data characteristics corresponding to the audience rating data; and processing the audience rating data according to the determined data processing type and the priority rule.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A data processing method, comprising:
acquiring audience rating data, wherein the audience rating data are used for representing the data volume of watching television programs in a preset time period;
configuring a priority rule for the audience rating data according to indication information, wherein the indication information is sorted according to dates or sorted according to the frequency of data processing failure;
determining a data processing type according to the data characteristics corresponding to the audience rating data, and starting different data processing flows aiming at different audience rating data; the data characteristics are determined by original data information corresponding to the audience rating data;
processing the audience rating data according to the determined data processing type and the priority rule;
the determining the data processing type according to the data characteristics corresponding to the audience rating data comprises:
acquiring a data processing log corresponding to the audience rating data;
extracting data characteristics of the data processing log;
determining a data processing type corresponding to the audience rating data according to the data characteristics, wherein the data processing type comprises at least one of the following types: normal data processing type, updated data processing type, incremental data processing type and abnormal data processing type;
aiming at different audience rating data, different data processing types are corresponded, and then different data processing flows are started, including:
for audience rating data of different data processing types, different data processing modules are selected from a plurality of preset data processing modules to start different data processing flows, wherein the different data processing modules correspond to different data processing types, and each data processing module can start the data processing flow corresponding to the data processing type.
2. The method of claim 1,
when the indication information is that the indications are sorted according to dates, the configuring a priority rule for the audience rating data according to the indication information includes:
sequencing the audience rating data in a positive sequence according to dates to configure a priority rule for the audience rating data; or
Sorting the audience rating data in a reverse order according to dates to configure a priority rule for the audience rating data; or
Circularly sequencing the audience rating data according to dates to configure priority rules for the audience rating data;
when the indication information indicates that the data processing is performed according to the frequency of data processing failure, configuring a priority rule for the audience rating data according to the indication information includes:
and sequencing the audience rating data according to the number of times of data processing failure to configure a priority rule for the audience rating data.
3. The method of any of claims 1-2, wherein the data processing types include a normal data processing type, an update data processing type, an incremental data processing type, and an exception data processing type;
the determining the data processing type according to the data characteristics corresponding to the audience rating data comprises:
if the audience rating data does not have a corresponding historical processing record, determining that the data processing type corresponding to the audience rating data is a normal data processing type;
if the audience rating data is the update of part of original data, determining the data processing type corresponding to the audience rating data as an update data processing type;
if the audience rating data is the update of the incremental data, determining that the data processing type corresponding to the audience rating data is the incremental data processing type;
and if the audience rating data is abnormal data which fails to be processed, determining that the data processing type corresponding to the audience rating data is an abnormal data processing type.
4. The method of claim 3, further comprising:
and carrying out statistics on the processed audience rating data to obtain an audience rating statistical result.
5. A data processing apparatus, comprising:
the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring audience rating data, and the audience rating data is used for representing the data volume of watching television programs in a preset time period;
the configuration unit is used for configuring a priority rule for the audience rating data according to indication information, and the indication information is sorted according to dates or sorted according to the times of data processing failure;
the determining unit is used for determining a data processing type according to the data characteristics corresponding to the audience rating data, corresponding to different data processing types aiming at different audience rating data, and further starting different data processing flows; the data characteristics are determined by the characteristic information of the original data corresponding to the audience rating data;
the processing unit is used for processing the audience rating data according to the determined data processing type and the priority rule;
the determination unit includes:
the acquisition module is used for acquiring a data processing log corresponding to the audience rating data;
the extraction module is used for extracting the data characteristics of the data processing log;
the determining module is used for determining a data processing type corresponding to the audience rating data according to the data characteristics;
the determining unit is specifically configured to select different data processing modules from a plurality of preset data processing modules for the audience rating data of different data processing types to start different data processing flows, where the different data processing modules correspond to different data processing types, and each data processing module can start a data processing flow corresponding to the data processing type.
6. The apparatus of claim 5,
when the indication information indicates that the sorting is performed according to the date,
the configuration unit is specifically configured to forward sort the audience rating data according to dates to configure priority rules for the audience rating data; or
The configuration unit is specifically configured to sort the audience rating data in reverse order according to dates to configure priority rules for the audience rating data; or
The configuration unit is specifically configured to perform circular sequencing on the audience rating data according to dates to configure priority rules for the audience rating data;
when the indication information indicates that the ordering is performed according to the number of times of data processing failure,
the configuration unit is specifically configured to rank the audience rating data according to the number of times of data processing failure to configure a priority rule for the audience rating data.
7. The apparatus of any of claims 5-6, wherein the data processing types include a normal data processing type, an update data processing type, an incremental data processing type, and an exception data processing type;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is a normal data processing type if the corresponding history processing record does not exist in the audience rating data;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is an updated data processing type if the audience rating data is an update of part of original data;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is an incremental data processing type if the audience rating data is updated by incremental data;
the determining unit is specifically configured to determine that a data processing type corresponding to the audience rating data is an abnormal data processing type if the audience rating data is abnormal data that has failed to be processed.
8. The apparatus of claim 7, further comprising:
and the statistical unit is used for carrying out statistics on the processed audience rating data to obtain an audience rating statistical result.
9. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program runs, a device in which the storage medium is located is controlled to execute the data processing method of any one of claims 1 to 4.
10. A processor for running a program, wherein the program is to execute the data processing method of any one of claims 1 to 4 when the program is run.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611050440.3A CN108093275B (en) | 2016-11-22 | 2016-11-22 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611050440.3A CN108093275B (en) | 2016-11-22 | 2016-11-22 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108093275A CN108093275A (en) | 2018-05-29 |
| CN108093275B true CN108093275B (en) | 2020-09-25 |
Family
ID=62170299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611050440.3A Active CN108093275B (en) | 2016-11-22 | 2016-11-22 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108093275B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109785077A (en) * | 2019-01-30 | 2019-05-21 | 北京互金新融科技有限公司 | The treating method and apparatus of order |
| CN111985944A (en) * | 2019-05-21 | 2020-11-24 | 北京沃东天骏信息技术有限公司 | Method, device and equipment for processing material data and storage medium |
| CN112579658B (en) * | 2019-09-27 | 2023-04-18 | 深圳市赛格车圣智联科技有限公司 | Method for analyzing daytime and nighttime of vehicle in multi-process manner |
| US11030646B1 (en) * | 2020-09-21 | 2021-06-08 | Alphonso Inc. | Computer program product that implements a machine learning process using a random forest model for predicting advertisement spending |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010171606A (en) * | 2009-01-21 | 2010-08-05 | Video Research:Kk | Viewing and listening data processing system, viewing and listening data processing apparatus, method and program |
| JP2012175142A (en) * | 2011-02-17 | 2012-09-10 | Mic Ware:Kk | Program viewing-information processing device and method, and program |
| CN102833624A (en) * | 2011-06-14 | 2012-12-19 | 联想(北京)有限公司 | Handling method for digital television and electronic device |
| CN105049887A (en) * | 2015-08-13 | 2015-11-11 | 四川长虹电器股份有限公司 | Method for presenting real-time audience rating |
| CN105407364A (en) * | 2015-10-27 | 2016-03-16 | 四川长虹电器股份有限公司 | Channel comprehensive competitiveness realization method based on intelligent television rating system |
-
2016
- 2016-11-22 CN CN201611050440.3A patent/CN108093275B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010171606A (en) * | 2009-01-21 | 2010-08-05 | Video Research:Kk | Viewing and listening data processing system, viewing and listening data processing apparatus, method and program |
| JP2012175142A (en) * | 2011-02-17 | 2012-09-10 | Mic Ware:Kk | Program viewing-information processing device and method, and program |
| CN102833624A (en) * | 2011-06-14 | 2012-12-19 | 联想(北京)有限公司 | Handling method for digital television and electronic device |
| CN105049887A (en) * | 2015-08-13 | 2015-11-11 | 四川长虹电器股份有限公司 | Method for presenting real-time audience rating |
| CN105407364A (en) * | 2015-10-27 | 2016-03-16 | 四川长虹电器股份有限公司 | Channel comprehensive competitiveness realization method based on intelligent television rating system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108093275A (en) | 2018-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108093275B (en) | Data processing method and device | |
| EP3165984B1 (en) | An event analysis apparatus, an event analysis method, and an event analysis program | |
| US10515083B2 (en) | Event analysis apparatus, an event analysis system, an event analysis method, and an event analysis program | |
| CN105446706B (en) | Method and device for evaluating form page use effect and providing original data | |
| US10860454B2 (en) | Analyzing large-scale data processing jobs | |
| CN109934268B (en) | Abnormal transaction detection method and system | |
| JP2021532459A (en) | Target cell labeling methods, devices, storage media and terminal devices | |
| CN109145981B (en) | Deep learning automatic model training method and equipment | |
| CN104199754A (en) | Production failure analysis system | |
| CN111984442A (en) | Method and device for detecting abnormality of computer cluster system, and storage medium | |
| US10467206B2 (en) | Data sampling in a storage system | |
| CN108427675B (en) | Method and equipment for constructing index | |
| US20170083531A1 (en) | Selecting an incremental backup approach | |
| CN114325232B (en) | Fault positioning method and device | |
| JP2016143107A (en) | Source code evaluation system and method | |
| CN115238779A (en) | Anomaly detection method, device, equipment and medium for cloud disk | |
| CN107846612B (en) | Audience rating analysis method and device | |
| JP2009193306A (en) | Environmental data management apparatus and method | |
| CN109729393B (en) | Data processing method and device | |
| CN110489967B (en) | Method and device for analyzing program running risk | |
| CN113672426A (en) | Storage device abnormality determination method and apparatus, storage medium, and electronic apparatus | |
| CN115237917A (en) | Data computing method, device and equipment for data center station and readable storage medium | |
| TW201721319A (en) | Method, system and computer program product for automated monitoring | |
| CN112817948A (en) | Data detection method and device, readable storage medium and electronic equipment | |
| CN112882854A (en) | Request exception handling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: Beijing Guoshuang Technology Co.,Ltd. Address before: 100086 Cuigong Hotel, 76 Zhichun Road, Shuangyushu District, Haidian District, Beijing Applicant before: Beijing Guoshuang Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |