CN115391612A - Data processing method, device, equipment and readable storage medium - Google Patents
Data processing method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN115391612A CN115391612A CN202211067904.7A CN202211067904A CN115391612A CN 115391612 A CN115391612 A CN 115391612A CN 202211067904 A CN202211067904 A CN 202211067904A CN 115391612 A CN115391612 A CN 115391612A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- segment
- segments
- applicable
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar Systems Or Details Thereof (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a data processing method, a data processing device, data processing equipment and a readable storage medium. The method comprises the following steps: acquiring first data collected by a target object at a target time point within preset time; the method comprises the steps that a preset time corresponds to a plurality of data segments collected by a target object, the data segments comprise target data segments, and the target data segments comprise first data; if it is determined that a strong correlation event exists in a time period corresponding to the target data segment, determining that the first data is not applicable; strongly correlated events are used to characterize factors that cause the first data to be inapplicable; in this way, based on the strong correlation event, it is determined that the data collected by the target object of the device at the target time point is not applicable, thereby shielding the influence of the inapplicable data on the device.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium.
Background
At present, various hardware modules, such as a position hardware module, a gyroscope sensor, an acceleration sensor, a temperature sensor, a humidity sensor, and the like, are integrated in large vehicles such as airplanes and trains, and small electronic devices such as mobile terminals. The data collected by the hardware modules such as the sensor is not suitable for the problem of poor use effect of the equipment and the like, and even a serious accident is triggered. The data provided by the hardware module is not suitable for a plurality of reasons, which may be that the hardware module is damaged or interfered, so that the raw data acquired by the hardware module is not accurate, or the raw data acquired by the hardware module is error-free but is not suitable for some software on the device in some scenes. Therefore, how to determine whether the data provided by the hardware module in a certain period of time is suitable is a problem to be solved.
Disclosure of Invention
The present application provides a data processing method, an apparatus, a device, a computer-readable storage medium, and a computer program product, which are used to solve the problem of how to determine whether data provided by a hardware module at a certain time point is suitable.
In a first aspect, the present application provides a data processing method, including:
acquiring first data collected by a target object at a target time point within preset time; the method comprises the steps that a preset time corresponds to a plurality of data segments collected by a target object, the data segments comprise target data segments, and the target data segments comprise first data;
if it is determined that a strong correlation event exists in a time period corresponding to the target data segment, determining that the first data is not applicable; the strongly correlated events are used to characterize the factors that cause the first data to be inapplicable.
In one embodiment, if it is determined that a strong correlation event does not exist in a time period corresponding to a target data segment, determining a first numerical value based on data in a preset time; based on the first data, the first value, and the first threshold, it is determined that the first data is not applicable due to anomalous data.
In one embodiment, determining that the first data is not applicable due to anomalous data based on the first data, the first value, and the first threshold comprises:
if the absolute value of the difference between the value of the first data and the first numerical value is greater than or equal to a first threshold value, determining that the first data is not applicable due to abnormal data; the proportion of abnormal data in the target data segment is less than or equal to a second threshold value.
In one embodiment, if the first data is not applicable due to abnormal data, the target data segment is a monotone data segment, and at least one weak association event exists in the time segment corresponding to the target data segment, determining that the first data is not applicable; at least one weakly associated event is used to characterize factors that may render the first data inapplicable.
In one embodiment, if the first data is not applicable due to abnormal data, the target data segment is not a monotonic data segment, at least one weak correlation event exists in the time segment corresponding to the target data segment, and the time segment corresponding to the target data segment is behind the time segments corresponding to at least two monotonic data segments, determining that the first data is not applicable; the plurality of data segments includes at least two monotonic data segments.
In one embodiment, the at least two monotonic data segments are N consecutive monotonic data segments or at least M discontinuous monotonic data segments, N being a positive integer greater than or equal to 2 and M being a positive integer greater than or equal to N +1.
In a second aspect, the present application provides a data processing apparatus comprising:
the first processing module is used for acquiring first data acquired by a target object at a target time point within preset time; the preset time corresponds to a plurality of data segments collected by a target object, the plurality of data segments comprise target data segments, and the target data segments comprise the first data;
the second processing module is used for determining that the first data are not applicable if the strong correlation event exists in the time period corresponding to the target data segment; the strongly correlated events are used to characterize the factors that cause the first data to be inapplicable.
In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and a bus;
a bus for connecting the processor and the memory;
a memory for storing operating instructions;
and the processor is used for executing the data processing method of the first aspect of the application by calling the operation instruction.
In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program for executing the data processing method of the first aspect of the present application.
In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the data processing method of the first aspect of the present application.
The technical scheme provided by the embodiment of the application at least has the following beneficial effects: acquiring first data acquired by a target object at a target time point within preset time; the method comprises the steps that a preset time corresponds to a plurality of data segments collected by a target object, the data segments comprise target data segments, and the target data segments comprise first data; if it is determined that a strong correlation event exists in the time period corresponding to the target data segment, determining that the first data is not applicable; strongly correlated events are used to characterize factors that cause the first data to be inapplicable; in this way, based on the strongly correlated event, it is determined that the data provided by the target object of the device at the target point in time is not applicable, thereby shielding the influence of the inapplicable data on the device.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of another data processing method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the embodiments set forth below in connection with the drawings are exemplary descriptions for explaining technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates at least one of the items defined by the term, e.g., "a and/or B" indicates either an implementation as "a", or an implementation as "B", or an implementation as "a and B".
It is understood that in the specific implementation of the present application, data related to data processing is referred to, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of the related data need to comply with relevant laws and regulations and standards of relevant countries and regions.
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The embodiment of the application provides a data processing method. For better understanding and description of the embodiments of the present application, some technical terms used in the embodiments of the present application will be briefly described below.
The sensor: the sensor is a detection device which can sense the information to be measured and convert the sensed information into an electric signal or other information in a required form according to a certain rule to be output so as to meet the requirements of information transmission, processing, storage, display, recording, control and the like.
And (3) strongly correlated events: there are factors that with a high probability can cause data provided by a certain target object, e.g. a hardware module, to be inapplicable. The existence of a high probability means that in practice, from the viewpoint of guaranteeing the use effect of the user, avoiding safety accidents and the like, data collected by the target object at certain target time points should be considered to be inapplicable when a strong correlation event occurs. Therefore in the present embodiment a strongly correlated event is used to characterize the factors that cause the first data to be inapplicable.
Weakly associated events: there are factors that do not belong to a strongly associated event, but still have a certain probability that can cause data provided by a certain target object to be inapplicable. In practice, the mere occurrence of weakly correlated events does not identify the data collected by the target object at certain target time points as not applicable. Therefore, in the embodiment of the present application, the weak association event is used for characterizing factors that may cause the first data to be inapplicable.
The strong and weak correlation events are determined by those skilled in the art according to the actual situation of the device and the target object, such as a hardware module. In principle the criteria for determining strongly correlated events should be strict and the criteria for determining weakly correlated events may be suitably relaxed. A certain hardware module in a device may not have a strongly associated event on the device, but a suitable relaxation criterion must find a weakly associated event, which may be at least one.
The data provided by the target object, such as a hardware module, is not suitable for a large number of reasons, which may be that the hardware module is damaged or disturbed, so that the raw data collected by the hardware module is not accurate, or the raw data collected by the hardware module is error-free but is not suitable for some software on the device in some scenarios. The above reasons are considered when determining both strongly and weakly correlated events, and are described below by way of example.
The first example is as follows: sensors on many electronic devices are sensitive to humidity, and if the humidity is too high, the normal operation of the sensors is affected, so that the original data acquired by the sensors are inaccurate. For the situation, the strong correlation event can be that the humidity value provided by software such as weather forecast exceeds a set threshold, and the equipment is judged to be outdoors based on the position information; the weak association event can be that the humidity value provided by software such as weather forecast exceeds a set threshold value, and whether the equipment is located outdoors cannot be accurately judged. Determining the main reason of the strong correlation event and the weak correlation event, namely if the humidity value exceeds a set threshold value and the equipment is located outdoors, and the sensor is directly exposed to a high-humidity environment, the probability of inaccuracy of the raw data acquired by the sensor is high; the indoor environment is relatively stable, the change range of indexes such as humidity and the like is not large compared with the outdoor environment, the accuracy of the original data acquired by the sensor can be ensured, if the equipment is located outdoors or indoors, the possibility that the original data acquired by the sensor is inaccurate can only be considered based on the fact that the humidity value exceeds a set threshold value, and the probability that the original data are inaccurate is not enough to be considered.
Example two: at present, devices such as a mobile terminal and the like are generally provided with position hardware modules, such as a GPS, a Beidou and the like. Users of the mobile terminals often install similar software such as an ordering APP (Application), and the ordering APP can directly obtain the location data collected by the location hardware module or indirectly obtain the location data collected by the location hardware module from other software. The probability of damage or interference to the hardware module at the position is low, and the collected original data is accurate. The meal ordering APP or similar software provides services such as restaurant recommendation and the like to the user through the position data. Based on the method, under the conditions that the position of the user is changed rapidly in the traffic process and the like, the position data collected by the position hardware module is not suitable for meal ordering APP or similar software. For example, the order APP determines restaurant a closest to location X based on location X and recommends restaurant a to the user, where the user has moved to location Y, restaurant a is not the closest restaurant to location Y, and restaurant a recommended by the order APP to the user is inaccurate. For this case, the strongly associated event may be — the user has started the navigation software; the weak association event may be that a base station accessed by the mobile terminal changes or a WIFI network accessed by the mobile terminal changes. Determining the main reason of the strong correlation event and the weak correlation event, namely considering that the probability of rapid position change of the user in the traffic process is higher according to the function of navigation software if the user starts the navigation software; the position information is also associated with mobile communication data, WIFI (Wireless Fidelity, wireless local area network based on IEEE 802.11b standard) data and the like to a certain extent, if a base station accessed by the mobile terminal changes or a WIFI network accessed by the mobile terminal changes, the position of the mobile terminal may also change, but there are many reasons that may cause the change of the base station or the change of the WIFI network, and only the change of the base station or the change of the WIFI network occurs, it can be considered that there is a possibility of a rapid change of the user position, but it is not enough to determine that the probability of the rapid change of the user position is large.
After determining all strong correlation events and weak correlation events of each target object, for example, each hardware module, those skilled in the art may establish a relationship table of target object-strong correlation event-weak correlation event-inapplicable data influence range for each target object. Where inapplicable data scope of influence refers to the set of hardware and software on the device that uses the data provided by the target object. For example, in the first example, the inapplicable data influence range includes all software and hardware of the device which uses the raw data collected by the sensor; in the second example, the inapplicable data influence range includes all software on the device that obtains the result by analyzing the position data collected by the position hardware module.
The technical solution of the present application will be described in detail below with specific examples. These several specific embodiments may be combined with each other below, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 shows a schematic flowchart of a data processing method provided in an embodiment of the present application, where the method may be executed by any electronic device, such as a terminal, a server, and the like; as an alternative embodiment, the method may be performed by the terminal. For convenience of description, in the following description of some alternative embodiments, a terminal will be taken as an example of the method execution subject. As shown in fig. 1, a data processing method provided in an embodiment of the present application includes the following steps.
S201, acquiring first data acquired by a target object at a target time point within preset time; the preset time corresponds to a plurality of data segments collected by the target object, the plurality of data segments comprise target data segments, and the target data segments comprise first data.
Illustratively, the target object may be a hardware module of the device, the hardware module of the device may be a GPS, a compass, a sensor, and the like, and the device may be a mobile terminal. The preset time can be 24 hours, the target time point can be 23 hours 00 minutes 00 seconds, and the first data can be data acquired by the hardware module in 23 hours 00 minutes 00 seconds; dividing 24 hours into 48 time periods, namely each time period is half an hour, and each time period corresponds to one data segment acquired by the hardware module, namely 24 hours correspond to 48 data segments acquired by the hardware module; the target data segment is the 47 th data segment in the 48 data segments, the time segment corresponding to the 47 th data segment is from 23 hours 00 minutes 00 seconds to 23 hours 30 minutes 00 seconds, the time segment includes the starting point 23 hours 00 minutes 00 seconds and does not include the ending point 23 hours 30 minutes 00 seconds, and the 47 th data segment includes the first data, namely the data collected by the hardware module at 23 hours 00 minutes 00 seconds.
The target time point in the target data segment may be set to one or more, and the corresponding first data may be set to one or more. In practice, in order to improve accuracy, a plurality of target time points are usually set in the target data segment, for example, the time points at which the hardware modules themselves collect the raw data may be set as the target time points. If the hardware module starts to collect one original data per second from a time 00 min 00 s, there are 60 × 30=1800 target time points corresponding to 1800 first data in a time period from 23 min 00 s to 23 min 30 min 00 s. If the raw data collected by the hardware module with time as an argument can be regarded as a function of time, each raw data can be regarded as a data point, and the time interval of adjacent data points in each data segment is the same. If the original data collected by a certain hardware module is qualitative data, rules can be set to convert the qualitative data into quantitative data, and the quantitative analysis can still be carried out on the data.
S202, if it is determined that a strong correlation event exists in a time period corresponding to the target data segment, determining that the first data is not applicable; the strongly correlated events are used to characterize the factors that cause the first data to be inapplicable.
Illustratively, the target data segment is 47 th data segment in 48 data segments, and if it is determined that a strong association event exists in a time segment (from 23 hours 00 minutes 00 seconds to 23 hours 30 minutes 00 seconds) corresponding to the 47 th data segment, and the existence time point of the strong association event is also the target time point, it is determined that the first data corresponding to the target time point is not applicable. If the target data segment includes a plurality of first data, it may be determined that all of the first data is not applicable.
Illustratively, as in the foregoing example scenario two, the hardware module of the mobile terminal is a GPS. In a time period corresponding to 47 data segments, namely from 23 hours 00 minutes 00 seconds to 23 hours 30 minutes 00 seconds, the time period includes the starting point 23 hours 00 minutes 00 seconds and does not include the ending point 23 hours 30 minutes 00 seconds. At any point in the time period, the user of the discovery device has started the navigation software, and may determine that the first data in the time period is not applicable, for example, the 1800 first data shown above may be determined to be not applicable. The impact of strongly correlated events is typically persistent, so the scope of determining the first data as unsuitable can be expanded appropriately.
In the embodiment of the application, first data collected by a target object at a target time point within a preset time are obtained; the method comprises the steps that a preset time corresponds to a plurality of data segments collected by a target object, wherein the data segments comprise target data segments, and the target data segments comprise first data; if it is determined that a strong correlation event exists in a time period corresponding to the target data segment, determining that the first data is not applicable; strongly correlated events are used to characterize factors that cause the first data to be inapplicable; in this way, based on the strongly correlated event, it is determined that the data provided by the target object of the device at the target point in time is not applicable, thereby shielding the influence of the inapplicable data on the device.
In one embodiment, if it is determined that a strong correlation event does not exist in a time period corresponding to a target data segment, determining a first numerical value based on data in a preset time; based on the first data, the first value, and the first threshold, it is determined that the first data is not applicable due to anomalous data.
Specifically, the outlier data may be an outlier data point caused by a sporadic deviation. The abnormal data points resulting from sporadic deviations are typically characterized by either too large or too small a data value for the abnormal data points. If a certain data segment satisfies the following conditions, the data segment can be considered to have abnormal data.
(A) The proportion of abnormal data points relative to all data points in the data segment does not exceed a set threshold, i.e. a second threshold, for example, the second threshold is 3% -5%, and the second threshold can be set according to the actual situation of the data. All data points in the data segment refer to all raw data collected by a target object, such as a hardware module, in the data segment, and if the raw data is qualitative data, the raw data is converted into quantitative data.
(B) Any non-abnormal data point satisfies formula (1), and any abnormal data point satisfies formula (2); the formula (1) and the formula (2) are respectively as follows:
|U normal formula (1) of-a | ≦ K
|U abnormal -a | ≧ bK equation (2)
Wherein, U normal Data values representing non-abnormal data points, U abnormal A data value representing an abnormal data point; the value of b can be set according to the actual condition of data, but is not suitable to be less than 1, for example, b can be a value between 1.5 and 3; a is a first value and bK is a first threshold.
When the step (B) is implemented, firstly, a second threshold value and a value of B need to be set, after the second threshold value and the value of B are determined, all non-abnormal data points and all abnormal data points in all data points in the data segment are determined according to the second threshold value, and then whether a and K simultaneously satisfy the formula (1) and the formula (2) is judged according to the data values of all non-abnormal data points and all abnormal data points, wherein a and K are specific values of the same physical unit as the data value of the data point.
In one embodiment, determining that the first data is not applicable due to anomalous data based on the first data, the first value, and the first threshold comprises:
if the absolute value of the difference between the value of the first data and the first numerical value is greater than or equal to a first threshold value, determining that the first data is not applicable due to abnormal data; the proportion of abnormal data in the target data segment is less than or equal to a second threshold value. Specifically, according to equation (2), if the absolute value of the difference between the value of the first data and the first numerical value (a) is greater than or equal toEqual to the first threshold (bK), it is determined that the first data is not applicable due to abnormal data, i.e. the value of the first data is in U abnormal Within the range, the first data is determined to be abnormal data.
In one embodiment, if data (first data) at a certain point in time (target point in time) is determined to be an abnormal data point on a data segment containing the data (target data segment) based on its data value, the first data is not applicable. If the data (data point) at the time point is not determined as an abnormal data point on the data segment containing the data, the data point needs to be compared with all non-abnormal data points of other data segments. The comparison method comprises the following steps: in all the data segments within a period of time (preset time), after abnormal data points of the data segments are removed from each data segment, non-abnormal data points of all the data segments form a large data set, and then the judgment is carried out through the definitions (A) and (B), and if the data points belong to abnormal data points in the large data set, the data points are still judged to be inapplicable.
Generally, the first data is not applicable to the abnormal data means that the first data itself is determined to be an abnormal data point according to the method provided by the embodiment of the application. In practice, however, if a certain data segment cannot be regarded as having abnormal data according to the method provided by the embodiment of the present application, but one or more first data included in the data segment has a larger or smaller data value relative to other data in the data segment, and experience or other evidence suggests that the first data may have a problem, the first data may still be regarded as being inapplicable due to the abnormal data.
In one embodiment, if the first data is not applicable due to abnormal data, the target data segment is a monotone data segment, and at least one weak association event exists in the time segment corresponding to the target data segment, determining that the first data is not applicable; at least one weakly correlated event is used to characterize factors that may render the first data inapplicable.
Specifically, if a data segment (target data segment) including data (first data) of the time point (target time point) is a monotonic data segment, and at least one weakly associated event exists in a time range (time segment) corresponding to the data segment including the time point data, it is determined that the first data is not applicable. The monotonic data segments can be: after the abnormal data points are eliminated from the data segment, the rest data points which exceed the set proportion in the data segment present a monotonous relation by taking time as an independent variable and satisfy a formula (3); equation (3) is as follows:
wherein, U max And U min Respectively, a maximum data value and a minimum data value in data points exhibiting a monotonic relationship, and c is a threshold value set according to the actual situation of the data, for example, c is a value between 0.3 and 0.4. The set ratio is, for example, 85%, and the set ratio can be set according to the actual condition of the data.
It should be noted that, in consideration of factors such as data errors, even after all data points in a certain data segment exclude abnormal data points, it is unlikely that all remaining data points actually exhibit a monotonic relationship, as long as most remaining data points exhibit a monotonic relationship, and therefore a set ratio, which is a variable that allows setting according to the actual situation of data, is defined. The monotone relationship is consistent with the monotone relationship defined in general mathematics, and comprises two monotone relationships of monotone increase and monotone decrease.
In one embodiment, if the first data is not applicable due to abnormal data, the target data segment is not a monotonic data segment, at least one weak correlation event exists in the time segment corresponding to the target data segment, and the time segment corresponding to the target data segment is behind the time segments corresponding to at least two monotonic data segments, determining that the first data is not applicable; the plurality of data segments includes at least two monotonic data segments.
Specifically, if a data segment (target data segment) including data (first data) of the time point (target time point) is not a monotone data segment, but the data segment belongs to a subsequent data segment of at least two monotone data segments (the time segment corresponding to the target data segment is after the time segment corresponding to the at least two monotone data segments), and at least one weakly correlated event exists in a time range (time segment) corresponding to the data segment including the time point data, it is determined that the first data is not applicable. The at least two monotonic data segments can be: at least N continuous monotone data sections or at least M discontinuous monotone data sections exist in the continuous data sections before the data section including the time point data. Usually the consecutive data segment should be adjacent to the data segment comprising the time point data, i.e. the end of the consecutive data segment corresponds to the start of the data segment comprising the time point data. The contiguous data segment may be defined as: the multiple data segments can be combined into a longer data segment according to the time sequence, and the longer data segment has no gap; the gapless data refers to that any two adjacent data points are taken from the longer data segment, the time intervals of the two adjacent data points are the same, and the situation that the time interval of some two adjacent data points is larger than that of other two adjacent data points does not exist. Accordingly, the continuous monotonic segments can be defined as: the plurality of monotonous data segments can be combined into a longer data segment according to the time sequence, and the longer data segment has no gap, and the significance of no gap is the same as the above. Obviously, the continuous data segment is composed of a plurality of data segments, and the number of the data segments composing the continuous data segment can be set according to the actual situation of the data, but is not suitable to be too large, for example, the number of the data segments composing the continuous data segment does not exceed 20% of the total number of the data segments in the preset time; n and M are positive integers, N is more than or equal to 2, and M is more than or equal to N +1.
In one embodiment, the at least two monotonic data segments are N consecutive monotonic data segments or at least M discontinuous monotonic data segments, N being a positive integer greater than or equal to 2 and M being a positive integer greater than or equal to N +1.
Specifically, the at least two monotonic data segments can be: at least N continuous monotone data sections or at least M discontinuous monotone data sections exist in continuous data sections before a data section including the point-in-time data. The definition of the continuous data segment is the same as above. The number of the data segments forming the continuous data segment can be set according to the actual situation of the data, but is not too large, for example, the number of the data segments forming the continuous data segment does not exceed 20% of the total number of the data segments in the preset time; n and M are positive integers, N is more than or equal to 2, and M is more than or equal to N +1.
In one embodiment, on the premise that only one hardware module of the device is provided, if the data collected by the hardware module is not applicable, the operating system of the device generally cannot obtain more applicable data from other ways, but may take certain temporary measures. For example, if a user of the device is using or is preparing to use hardware or software that is not within the data impact range, then a striking risk prompt is presented to the user; for another example, the authority level of the hardware or software in the data influence range is lowered, and the hardware or software is not allowed to execute the operation of higher authority level, such as short-time control of the device. Because the data applicability analysis is usually continuously performed, if the subsequent analysis conclusion shows that the data collected by the hardware module is not applicable, the operating system of the device can remove the temporary measures.
The embodiment of the application has at least the following beneficial effects: determining that data provided by a hardware module of the mobile terminal is not applicable based on the strong association event; determining that the data provided by the hardware module is abnormal data caused by accidental deviation, wherein the abnormal data is inapplicable; the target data section is a monotone data section, and the data provided by the hardware module is determined to be inapplicable based on the monotone data section and the weak correlation event; the target data segment is not a monotone data segment, and the data provided by the hardware module is determined to be inapplicable based on other monotone data segments (at least two monotone data segments) and weak association events; the data provided by the hardware module is determined to be unsuitable through the various modes, so that the influence of the unsuitable data on the equipment is shielded.
In order to better understand the method provided by the embodiment of the present application, the following further describes the scheme of the embodiment of the present application with reference to an example of a specific application scenario.
In a specific application scenario embodiment, for example, a data processing scenario, referring to fig. 2, a processing flow of a data processing method is shown, and as shown in fig. 2, the processing flow of the data processing method provided in the embodiment of the present application includes the following steps:
s301, the mobile terminal obtains first data collected by the hardware module at a target time point within a preset time.
Specifically, the hardware module is a hardware module of the mobile terminal, and the preset time corresponds to a plurality of data segments collected by the hardware module, wherein the plurality of data segments include a target data segment, and the target data segment includes first data.
S302, the mobile terminal judges whether a strong correlation event exists in the time period corresponding to the target data segment, and if the mobile terminal determines that the strong correlation event exists in the time period corresponding to the target data segment, the step S307 is carried out; if the mobile terminal determines that the strong association event does not exist in the time period corresponding to the target data segment, the process goes to step S303.
S303, the mobile terminal judges that the first data is not applicable due to abnormal data, and if the mobile terminal determines that the first data is not applicable due to abnormal data, the step S307 is carried out; if the mobile terminal determines that the first data is not applicable due to the abnormal data, the process goes to step S304.
S304, the mobile terminal judges whether the target data segment is a monotone data segment, and if the mobile terminal determines that the target data segment is the monotone data segment, the step S305 is carried out; if the mobile terminal determines that the target data segment is not a monotone data segment, the process goes to step S306.
S305, if the mobile terminal determines that at least one weakly associated event exists in the time period corresponding to the target data segment, go to step S307.
S306, if the mobile terminal determines that at least one weakly associated event exists in the time period corresponding to the target data segment and the time period corresponding to the target data segment is after the time periods corresponding to at least two monotone data segments, the processing goes to the step S307.
S307, the mobile terminal determines that the first data collected by the hardware module is not applicable.
The embodiment of the application has at least the following beneficial effects: determining that data provided by a hardware module of the mobile terminal is not applicable based on the strong association event; determining that the data provided by the hardware module is abnormal data caused by accidental deviation, wherein the abnormal data is inapplicable; the target data segment is a monotone data segment, and the data provided by the hardware module is determined to be inapplicable based on the monotone data segment and the weak correlation event; the target data segment is not a monotone data segment, and the data provided by the hardware module is determined to be inapplicable based on other monotone data segments (at least two monotone data segments) and weak association events; the data provided by the hardware module is determined to be unsuitable through the various modes, so that the influence of the unsuitable data on the equipment is shielded.
The embodiment of the present application further provides a data processing apparatus, a schematic structural diagram of the data processing apparatus is shown in fig. 3, and the data processing apparatus 40 includes a first processing module 401 and a second processing module 402.
The first processing module 401 is configured to acquire first data acquired by a target object at a target time point within a preset time; a plurality of data segments which are acquired corresponding to a target object at preset time, wherein the data segments comprise target data segments, and the target data segments comprise the first data;
a second processing module 402, configured to determine that the first data is not applicable if it is determined that a strong association event exists in a time period corresponding to the target data segment; strongly correlated events are used to characterize the factors that cause the first data to be inapplicable.
In one embodiment, the second processing module 402 is further configured to:
if it is determined that no strong correlation event exists in the time period corresponding to the target data segment, determining a first numerical value based on data in preset time;
based on the first data, the first value, and the first threshold, it is determined that the first data is not applicable due to anomalous data.
In an embodiment, the second processing module 402 is specifically configured to:
if the absolute value of the difference between the value of the first data and the first numerical value is greater than or equal to a first threshold value, determining that the first data is not applicable due to abnormal data; the proportion of abnormal data in the target data segment is less than or equal to a second threshold value.
In one embodiment, the second processing module 402 is further configured to:
if the first data is not applicable due to abnormal data, the target data segment is a monotone data segment, and at least one weak association event exists in the time segment corresponding to the target data segment, determining that the first data is not applicable; at least one weakly associated event is used to characterize factors that may render the first data inapplicable.
In one embodiment, the second processing module 402 is further configured to:
if the first data is not applicable due to abnormal data, the target data segment is not a monotone data segment, at least one weak correlation event exists in the time period corresponding to the target data segment, and the time period corresponding to the target data segment is behind the time periods corresponding to at least two monotone data segments, determining that the first data is not applicable; the plurality of data segments includes at least two monotonic data segments.
In one embodiment, the at least two monotonic data segments are N consecutive monotonic data segments or at least M discontinuous monotonic data segments, N being a positive integer greater than or equal to 2 and M being a positive integer greater than or equal to N +1.
The application of the embodiment of the application has at least the following beneficial effects:
acquiring first data collected by a target object at a target time point within preset time; the method comprises the steps that a preset time corresponds to a plurality of data segments collected by a target object, the data segments comprise target data segments, and the target data segments comprise first data; if it is determined that a strong correlation event exists in the time period corresponding to the target data segment, determining that the first data is not applicable; strongly correlated events are used to characterize factors that cause the first data to be inapplicable; in this way, based on the strongly correlated event, it is determined that the data provided by the target object of the device at the target point in time is not applicable, thereby shielding the influence of the inapplicable data on the device.
An embodiment of the present application further provides an electronic device, a schematic structural diagram of the electronic device is shown in fig. 4, and an electronic device 4000 shown in fig. 4 includes: a processor 4001 and a memory 4003. Processor 4001 is coupled to memory 4003, such as via bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, and the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data. In addition, the transceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.
The Processor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein. The processor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.
The Memory 4003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer, and is not limited herein.
The memory 4003 is used for storing computer programs for executing the embodiments of the present application, and execution is controlled by the processor 4001. The processor 4001 is used to execute computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.
Among them, electronic devices include but are not limited to: terminals, servers, etc.
The application of the embodiment of the application has at least the following beneficial effects:
acquiring first data collected by a target object at a target time point within preset time; the method comprises the steps that a preset time corresponds to a plurality of data segments collected by a target object, the data segments comprise target data segments, and the target data segments comprise first data; if it is determined that a strong correlation event exists in the time period corresponding to the target data segment, determining that the first data is not applicable; strongly correlated events are used to characterize the factors that cause the first data to be inapplicable; in this way, based on the strongly correlated event, it is determined that the data provided by the target object of the device at the target point in time is not applicable, thereby shielding the influence of the inapplicable data on the device.
Embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when being executed by a processor, the computer program may implement the steps and corresponding contents of the foregoing method embodiments.
Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.
Based on the same principle as the method provided by the embodiment of the present application, the embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in any of the alternative embodiments of the present application.
It should be understood that, although each operation step is indicated by an arrow in the flowchart of the embodiment of the present application, the implementation order of the steps is not limited to the order indicated by the arrow. In some implementation scenarios of the embodiments of the present application, the implementation steps in the flowcharts may be performed in other sequences as desired, unless explicitly stated otherwise herein. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on an actual implementation scenario. Some or all of these sub-steps or stages may be performed at the same time, or each of these sub-steps or stages may be performed at different times, respectively. In a scenario where execution times are different, an execution sequence of the sub-steps or the phases may be flexibly configured according to requirements, which is not limited in the embodiment of the present application.
The foregoing is only an optional implementation manner of a part of implementation scenarios in this application, and it should be noted that, for those skilled in the art, other similar implementation means based on the technical idea of this application are also within the protection scope of the embodiments of this application without departing from the technical idea of this application.
Claims (10)
1. A data processing method, comprising:
acquiring first data collected by a target object at a target time point within preset time; the preset time corresponds to a plurality of data segments collected by the target object, the data segments comprise target data segments, and the target data segments comprise the first data;
if it is determined that a strong correlation event exists in the time period corresponding to the target data segment, determining that the first data is not applicable; the strong association event is used to characterize factors that cause the first data to be inapplicable.
2. The method of claim 1, further comprising:
if it is determined that no strong correlation event exists in the time period corresponding to the target data segment, determining a first numerical value based on the data in the preset time;
determining that the first data is not applicable due to anomalous data based on the first data, the first value, and a first threshold.
3. The method of claim 2, wherein determining that the first data is inappropriate due to anomalous data based on the first data, the first numerical value, and a first threshold comprises:
if the absolute value of the difference between the value of the first data and the first numerical value is greater than or equal to a first threshold, determining that the first data is not applicable due to abnormal data; and the abnormal data proportion in the target data segment is less than or equal to a second threshold value.
4. The method of claim 2, further comprising:
if the first data is not applicable due to abnormal data, the target data segment is a monotone data segment, and at least one weak correlation event exists in the time segment corresponding to the target data segment, determining that the first data is not applicable; the at least one weakly correlated event is used to characterize factors that may render the first data inapplicable.
5. The method of claim 2, further comprising:
if the first data is not applicable due to abnormal data, the target data segment is not a monotone data segment, at least one weak correlation event exists in the time segment corresponding to the target data segment, and the time segment corresponding to the target data segment is behind the time segments corresponding to at least two monotone data segments, determining that the first data is not applicable; the plurality of data segments includes the at least two monotonic data segments.
6. The method according to claim 5, wherein the at least two monotonic data segments are N consecutive monotonic data segments or at least M discontinuous monotonic data segments, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N +1.
7. A data processing apparatus, characterized by comprising:
the first processing module is used for acquiring first data acquired by a target object at a target time point within preset time; the preset time corresponds to a plurality of data segments collected by the target object, the data segments comprise target data segments, and the target data segments comprise the first data;
a second processing module, configured to determine that the first data is not applicable if it is determined that a strong association event exists in a time period corresponding to the target data segment; the strong association event is used to characterize factors that cause the first data to be inapplicable.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the method of any of claims 1-6.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1-6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211067904.7A CN115391612B (en) | 2022-09-01 | 2022-09-01 | Data processing method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211067904.7A CN115391612B (en) | 2022-09-01 | 2022-09-01 | Data processing method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115391612A true CN115391612A (en) | 2022-11-25 |
| CN115391612B CN115391612B (en) | 2023-06-06 |
Family
ID=84124872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211067904.7A Active CN115391612B (en) | 2022-09-01 | 2022-09-01 | Data processing method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115391612B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103746871A (en) * | 2013-12-26 | 2014-04-23 | 力合科技（湖南）股份有限公司 | Data validity detection device and detection method |
| CN111641723A (en) * | 2020-06-03 | 2020-09-08 | 中国科学院自动化研究所 | Equipment state tracking method and device |
| CN111855019A (en) * | 2020-07-27 | 2020-10-30 | 歌尔科技有限公司 | Temperature measuring method and device, wrist-worn equipment and storage medium |
| US20210191958A1 (en) * | 2020-06-30 | 2021-06-24 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method and apparatus of user clustering, computer device and medium |
| CN114064427A (en) * | 2021-10-15 | 2022-02-18 | 深圳力维智联技术有限公司 | Method, device and equipment for processing operation and maintenance monitoring data of machine room |
-
2022
- 2022-09-01 CN CN202211067904.7A patent/CN115391612B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103746871A (en) * | 2013-12-26 | 2014-04-23 | 力合科技（湖南）股份有限公司 | Data validity detection device and detection method |
| CN111641723A (en) * | 2020-06-03 | 2020-09-08 | 中国科学院自动化研究所 | Equipment state tracking method and device |
| US20210191958A1 (en) * | 2020-06-30 | 2021-06-24 | Beijing Baidu Netcom Science Technology Co., Ltd. | Method and apparatus of user clustering, computer device and medium |
| CN111855019A (en) * | 2020-07-27 | 2020-10-30 | 歌尔科技有限公司 | Temperature measuring method and device, wrist-worn equipment and storage medium |
| CN114064427A (en) * | 2021-10-15 | 2022-02-18 | 深圳力维智联技术有限公司 | Method, device and equipment for processing operation and maintenance monitoring data of machine room |
Non-Patent Citations (1)
| Title |
|---|
| 陈作聪;: "基于核独立成分分析和聚类中值的水下传感器故障诊断", no. 02, pages 1 - 2 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115391612B (en) | 2023-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107798412B (en) | Route recommendation method and device | |
| CN107819631B (en) | Equipment anomaly detection method, device and equipment | |
| CN111238492B (en) | Fixed point drift correction method and device, terminal equipment and storage medium | |
| CN109005514B (en) | Backfill method and device of user position information, terminal equipment and storage medium | |
| US20170032266A1 (en) | Anomaly detection apparatus, method, and computer program using a probabilistic latent semantic analysis | |
| EP3089086A1 (en) | Work management system | |
| CN110149654B (en) | Method and device for determining faults of base station antenna feeder system | |
| JP2009200926A (en) | Position detection system, position detection server, and terminal | |
| CN111915913A (en) | Severe weather prompting method, prompting system, storage medium and electronic device | |
| CN115391612A (en) | Data processing method, device, equipment and readable storage medium | |
| CN111818551B (en) | VOLTE call drop reason determination method and device | |
| CN111428197A (en) | Data processing method, device and equipment | |
| CN113453261B (en) | Abnormal cell identification method and device and electronic equipment | |
| US8612660B2 (en) | Interrupt latency measurement | |
| CN114332766A (en) | Dangerous behavior early warning method and device based on action rule and application thereof | |
| CN110740190B (en) | Event reporting method, power terminal and computer readable storage medium | |
| CN112990745A (en) | Security detection method, device, equipment and computer storage medium | |
| CN108934036B (en) | Method and device for acquiring state information of base station | |
| CN111369787A (en) | Vehicle track prediction method and device and electronic equipment | |
| CN114125846A (en) | Integrity protection method and system | |
| CN116828509B (en) | Network blind area detection method and system | |
| CN112508466A (en) | Position identification method and device, computer readable storage medium and electronic equipment | |
| CN104714879A (en) | Picture testing method and device | |
| CN114040462B (en) | Mobile phone signaling-based travel chain identification method and related device | |
| CN110809232A (en) | Positioning method, positioning device, positioning equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |