CN113139817A - Data classification method, data classification device, medium, and electronic apparatus - Google Patents

Data classification method, data classification device, medium, and electronic apparatus Download PDF

Info

Publication number
CN113139817A
CN113139817A CN202110468156.2A CN202110468156A CN113139817A CN 113139817 A CN113139817 A CN 113139817A CN 202110468156 A CN202110468156 A CN 202110468156A CN 113139817 A CN113139817 A CN 113139817A
Authority
CN
China
Prior art keywords
internet
things
data
abnormal
normal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110468156.2A
Other languages
Chinese (zh)
Inventor
吴昊
陈鹏
徐峰
崔海明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202110468156.2A priority Critical patent/CN113139817A/en
Publication of CN113139817A publication Critical patent/CN113139817A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/018Certifying business or products
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2474Sequence data queries, e.g. querying versioned data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Accounting & Taxation (AREA)
  • Marketing (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Biophysics (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Finance (AREA)
  • Biomedical Technology (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Fuzzy Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)

Abstract

The present disclosure relates to the field of computers, and in particular, to a data classification method, a data classification device, a computer-readable storage medium, and an electronic device, including: acquiring data of the Internet of things, wherein the data of the Internet of things comprises normal data of the Internet of things and abnormal data of the Internet of things; inputting the data of the Internet of things into a data classification model, and detecting abnormal data of the Internet of things in the data of the Internet of things; and correcting the abnormal data of the Internet of things into normal data of the Internet of things, and classifying the normal data of the Internet of things according to the data classification model. Through the technical scheme of the embodiment of the disclosure, the problem of data classification errors of the Internet of things can be solved.

Description

Data classification method, data classification device, medium, and electronic apparatus
Technical Field
The present disclosure relates to the field of computers, and in particular, to a data classification method, a data classification apparatus, a computer-readable storage medium, and an electronic device.
Background
With the high-speed development of informatization and the improvement of living standard of people, people pay more and more attention to the problem of commodity traceability, more hopefully can know the production information and the logistics information of commodities, and can trace the commodity source through two-dimensional codes, bar codes, RFID and the like. For example, one can scan the two-dimensional code to view static information of a web page of the merchandise.
In the related technology, when the dynamic information of the commodity needs to be checked, the data collected by the internet of things equipment can be classified and stored in the database, so that the dynamic information of the commodity is stored, and a user can check conveniently.
However, with the increasing of nodes of the internet of things, the structure between the nodes of the internet of things is more and more complex, so that the data volume of the internet of things is increased continuously, the data of the internet of things collected by the nodes has the characteristics of high dimensionality, complexity, instantaneity and the like, the problem of data abnormity is easy to occur, and data classification errors are caused.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The purpose of the present disclosure is to provide a data classification method, a data classification device, a computer-readable storage medium, and an electronic device, which can solve the problem of data classification errors of the internet of things.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to a first aspect of the present disclosure, there is provided a data classification method, including: the method comprises the steps of obtaining data of the Internet of things, wherein the data of the Internet of things comprises normal data of the Internet of things and abnormal data of the Internet of things; inputting the data of the Internet of things into a data classification model, and detecting abnormal data of the Internet of things in the data of the Internet of things; and correcting the abnormal data of the Internet of things into normal data of the Internet of things, and classifying the normal data of the Internet of things according to the data classification model.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the detecting internet of things abnormal data in the internet of things data includes: acquiring first Internet of things normal data corresponding to the Internet of things data; the first Internet of things normal data is Internet of things normal data of a previous time unit of the Internet of things data; and detecting the abnormal data of the Internet of things in the data of the Internet of things according to the normal data of the first Internet of things.
In an exemplary embodiment of the disclosure, based on the foregoing scheme, the detecting, according to the first internet of things normal data, internet of things abnormal data in the internet of things data includes: acquiring an abnormal threshold of the data of the Internet of things; and detecting the abnormal data of the Internet of things in the data of the Internet of things according to the normal data of the first Internet of things and the abnormal threshold value of the data of the Internet of things.
In an exemplary embodiment of the disclosure, based on the foregoing scheme, the detecting, according to the first internet of things normal data and the internet of things data abnormal threshold, internet of things abnormal data in the internet of things data includes: calculating a difference value between the first Internet of things normal data and the Internet of things data; and determining whether the data of the Internet of things is abnormal data of the Internet of things according to the size relationship between the difference value and the abnormal threshold value of the data of the Internet of things.
In an exemplary embodiment of the disclosure, based on the foregoing scheme, the correcting the abnormal data of the internet of things into the normal data of the internet of things includes: acquiring one or more second internet of things normal data corresponding to the internet of things abnormal data, and acquiring abnormal data correction factors; the second networking normal data are the networking normal data in the unit interval of the adjacent time of the networking abnormal data; and correcting the abnormal data of the Internet of things into normal data of the Internet of things according to the normal data of the second Internet of things and the abnormal data correction factor.
In an exemplary embodiment of the disclosure, based on the foregoing solution, the correcting the abnormal data correction factor includes a first correlation correction factor and a second correlation correction factor, and the correcting the abnormal data of the internet of things into the normal data of the internet of things according to the second networking normal data and the abnormal data correction factor includes: adjusting the normal data of the second networking according to the first correlation correction factor and the second correlation correction factor to obtain second networking prediction data; the second networking normal data comprises networking normal data in a front adjacent time unit interval and networking normal data in a rear adjacent time unit interval; and correcting the abnormal data of the Internet of things into normal data of the Internet of things according to the second Internet of things prediction data.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the data classification model includes an exception handling long and short memory network model.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the exception handling long and short memory network model includes a forgetting gate and an input gate, and the method further includes: forming an updating door according to the forgetting door and the input door; the forgetting gate and the updating gate respectively control the temporary memory unit and the first memory unit to determine a second memory unit; the first memory unit is generated by the exception handling long and short memory network model in the previous time unit; the temporary memory unit is a memory unit generated by the exception handling long and short memory network model in the current time unit; the second memory unit is generated by the exception handling long and short memory network model in the current time unit.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, first implicit state information of the exception handling long and short memory network model is obtained; determining second hidden state information according to the first hidden state information and the second memory unit; the first hidden state information is hidden state information generated by the exception handling long and short memory network model in the previous time unit; the second hidden state information is hidden state information generated by the exception handling long and short memory network model in the current time unit.
According to a second aspect of the present disclosure, there is provided a data sorting apparatus, the apparatus comprising: the Internet of things data acquisition module is used for acquiring Internet of things data; the data of the Internet of things comprises normal data of the Internet of things and abnormal data of the Internet of things; the abnormal data monitoring module is used for inputting the Internet of things data into a data classification model and detecting the Internet of things abnormal data in the Internet of things data; and the Internet of things data classification module is used for correcting the abnormal data of the Internet of things into normal data of the Internet of things and classifying the normal data of the Internet of things according to the data classification model.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data classification method as described in the first aspect of the embodiments above.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising:
a processor; and
memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the data classification method as described in the first aspect of the embodiments above.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
in the data classification method provided by the embodiment of the disclosure, after the data of the internet of things is acquired, the data of the internet of things can be input into a data classification model, the abnormal data of the internet of things in the data of the internet of things is detected, the abnormal data of the internet of things is corrected into the normal data of the internet of things, and the normal data of the internet of things is classified according to the data classification model
According to the embodiment of the disclosure, abnormal data in the data of the internet of things can be corrected into normal data, and the normal data of the internet of things can be classified. On one hand, abnormal data can be processed, the accuracy of data classification is improved, and the problem that the user experience is reduced as the abnormal data is inquired by the user and is directly classified and stored is avoided; on the other hand, abnormal data are corrected instead of being directly eliminated, and the continuity of the data of the Internet of things can be guaranteed, so that the product trust of the user is improved, and the purchasing confidence of the user is further enhanced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
FIG. 1 schematically illustrates a schematic diagram of an exemplary system architecture for a data classification method in an exemplary embodiment of the disclosure;
FIG. 2 schematically illustrates a flow chart of a data classification method in an exemplary embodiment of the disclosure;
fig. 3 schematically illustrates a flowchart of detecting internet of things abnormal data in internet of things data according to first internet of things normal data in an exemplary embodiment of the disclosure;
fig. 4 schematically illustrates a flowchart of detecting abnormal data of the internet of things in the data of the internet of things according to the first normal data of the internet of things, the data of the internet of things and the abnormal threshold value of the data of the internet of things in the exemplary embodiment of the disclosure;
fig. 5 schematically shows a flowchart for determining whether the data of the internet of things is abnormal data of the internet of things according to a magnitude relationship between the difference value and the abnormal threshold of the data of the internet of things in the exemplary embodiment of the disclosure;
fig. 6 schematically illustrates a flowchart of correcting the internet of things abnormal data into the internet of things normal data according to the second internet of things normal data and the abnormal data correction factor in an exemplary embodiment of the present disclosure;
fig. 7 schematically illustrates a flowchart of correcting the abnormal data of the internet of things to the normal data of the internet of things according to the second networking prediction data in the exemplary embodiment of the present disclosure;
fig. 8 schematically illustrates a flowchart of classifying normal data of the internet of things according to a determined forgetting gate in an exemplary embodiment of the present disclosure;
fig. 9 schematically illustrates a flowchart of a forgetting gate and an updating gate respectively controlling a temporary storage unit and a first storage unit to determine a second storage unit in an exemplary embodiment of the present disclosure;
FIG. 10 schematically illustrates a flow chart for determining second implicit state information based on the first implicit state information and the second cell in an exemplary embodiment of the present disclosure;
FIG. 11 is a diagram schematically illustrating the structure of an exception handling long and short memory network model in an exemplary embodiment of the present disclosure;
fig. 12 schematically illustrates a schematic diagram of an internet of things normal data search system in an exemplary embodiment of the present disclosure;
fig. 13 schematically illustrates a composition diagram of a data sorting apparatus in an exemplary embodiment of the present disclosure;
fig. 14 schematically shows a schematic structural diagram of a computer system of an electronic device suitable for implementing an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in the form of software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.
Fig. 1 shows a schematic diagram of an exemplary system architecture to which the data classification method of the embodiments of the present disclosure may be applied.
As shown in fig. 1, the system architecture 1000 may include one or more of terminal devices 1001, 1002, 1003, a network 1004, and a server 1005. The network 1004 is used to provide a medium for communication links between the terminal devices 1001, 1002, 1003 and the server 1005. Network 1004 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. For example, the server 1005 may be a server cluster composed of a plurality of servers.
A user may use the terminal devices 1001, 1002, 1003 to interact with a server 1005 via a network 1004 to receive or transmit messages or the like. The terminal devices 1001, 1002, 1003 may be various electronic devices having a display screen, including but not limited to smart phones, tablet computers, portable computers, desktop computers, and the like. In addition, the server 1005 may be a server that provides various services.
In an embodiment, an executing subject of the data classification method disclosed by the present disclosure may be a server 1005, and the server 1005 may acquire the data of the internet of things sent by the terminal devices 1001, 1002, and 1003, input the data of the internet of things into a data classification model, detect abnormal data of the internet of things in the data of the internet of things, correct the abnormal data of the internet of things into normal data of the internet of things, and classify the normal data of the internet of things according to the data classification model to complete a data classification process. In addition, the data classification method disclosed by the disclosure can be executed through the terminal devices 1001, 1002, 1003 and the like, so as to realize a process of classifying the data of the unique internet of things according to the data classification model.
In addition, the implementation process of the data classification method of the present disclosure may also be implemented by the terminal devices 1001, 1002, 1003 and the server 1005 together. For example, the terminal devices 1001, 1002, and 1003 may acquire the data of the internet of things, input the acquired data of the internet of things into the data classification model, detect abnormal data of the internet of things in the data of the internet of things, and send the acquired abnormal data of the internet of things and normal data of the internet of things to the server 1005, so that the server 1005 may correct the abnormal data of the internet of things into normal data of the internet of things, and classify the normal data of the internet of things according to the data classification model.
With the continuous improvement of living standard of people, people pay more and more attention to the problem of commodity traceability. For some special commodities, such as food, etc., people want to know the production information of the commodities and the logistics transportation information. In the related art, a two-dimensional code may be passed. And tracing the source of the commodity in the modes of bar codes, RFID and the like. For example, a two-dimensional code attached to a commodity may be scanned by using a mobile phone, and some static information (place of origin, etc.) of the commodity is stored in the two-dimensional code. However, a series of real-time dynamic information of the growth, production, transportation, and the like of the commodity cannot be checked, and therefore, the scheme in the related art cannot meet the commodity tracing requirement of the current user.
When dynamic information of commodities needs to be checked in real time, a plurality of internet of things devices (such as sensors) can be arranged in the manufacturing (generating) environment and logistics process of the commodities, the internet technology is utilized to transmit the acquired internet of things data, and the acquired data are classified so that a user can check the data conveniently. However, as nodes of the internet of things are continuously increased, the network structure of the internet of things is more and more complex, the amount of transmitted data of the internet of things is more and more, the data has the characteristics of high dimension, complexity, instantaneity and the like, and due to the factors such as instability of network transmission and the like, the problem of data abnormality is easily caused, and if the abnormal data is processed according to normal data, the use experience of a user is greatly influenced.
According to the data classification method provided in the exemplary embodiment, after the data of the internet of things is acquired, the data of the internet of things can be input into the data classification model, abnormal data of the internet of things in the data of the internet of things is detected, the abnormal data of the internet of things is corrected into normal data of the internet of things, and the normal data of the internet of things is classified according to the data classification model. As shown in fig. 2, the data classification method may include the steps of:
step S210, Internet of things data is obtained, wherein the Internet of things data comprises Internet of things normal data and Internet of things abnormal data;
step S220, inputting the data of the Internet of things into a data classification model, and detecting abnormal data of the Internet of things in the data of the Internet of things;
and step S230, correcting the abnormal data of the Internet of things into normal data of the Internet of things, and classifying the normal data of the Internet of things according to the data classification model.
In the data classification method provided by the exemplary embodiment, after the data of the internet of things is acquired, the data of the internet of things can be input into the data classification model, the abnormal data of the internet of things in the data of the internet of things is detected, the abnormal data of the internet of things is corrected into the normal data of the internet of things, and the normal data of the internet of things is classified according to the data classification model.
According to the embodiment of the disclosure, abnormal data in the data of the internet of things can be corrected into normal data, and the normal data of the internet of things can be classified. On one hand, abnormal data can be processed, the accuracy of data classification is improved, and the problem that the user experience is reduced as the abnormal data is inquired by the user and is directly classified and stored is avoided; on the other hand, abnormal data are corrected instead of being directly eliminated, and the continuity of the data of the Internet of things can be guaranteed, so that the product trust of the user is improved, and the purchasing confidence of the user is further enhanced.
Hereinafter, the steps S210 to S230 of data classification in the present exemplary embodiment will be described in more detail with reference to the drawings and the embodiments.
Step S210, Internet of things data is obtained, wherein the Internet of things data comprises Internet of things normal data and Internet of things abnormal data;
in an example embodiment of the present disclosure, the internet of things is an extended and expanded network based on the internet, and a huge network is formed by combining various information sensing devices and the network, and the core and the foundation of the internet of things are still the internet, and the internet of things is an extended and expanded network based on the internet, and a user end of the internet of things is extended and expanded to any article to perform information exchange and communication. The Internet of things can connect any article with the Internet according to an agreed protocol through information sensing equipment such as radio frequency identification, infrared sensors, a global positioning system and a laser scanner, so as to exchange and communicate information, and realize a network for intelligently identifying, positioning, tracking, monitoring and managing the article.
In an example embodiment of the present disclosure, internet of things data may be obtained. Specifically, the internet of things data may include status data, which may represent real-time dynamic data of the suppliers and consumers about the internet of things. For example, the state data may indicate whether the compressor in the refrigerator is operating normally; the data of the internet of things can also comprise behavior-providing reference data, the behavior-providing reference data can represent state data with subsequent plans, the automatic technology capable of changing the real-time state of the system is relied on, and persuasion capable of enabling people to change behavior habits or make long-line investment is achieved. For example, the available behavioral reference data may include water waste data; the internet of things data may also include feedback data, the internet of things creating a feedback loop from the consumer to the producer, where the product producer can verify the actual performance of the product by a moderate level of privacy, security, and anonymity, and encourage continued product improvement and innovation; the data of the internet of things can also comprise positioning data, and the position of the current article can be determined through the positioning data; the internet of things data can also comprise personalized data, and the personalized data refers to personal preference data. It should be noted that the specific type of the data of the internet of things is not particularly limited in the present disclosure.
In an example embodiment of the present disclosure, the internet of things data may be collected by a variety of sensors. For example, for a milk product, a user pays more attention to real-time growth environment information of a cow, such as soil information, temperature information, humidity information, and the like, and data of interest to the user can be collected through a soil sensor, a temperature sensor, and a humidity sensor. It should be noted that the present disclosure is not limited to the manner of collecting the data of the internet of things.
Further, after the data of the internet of things is acquired through the sensor, the data of the internet of things can be processed. Specifically, since the internet of things data is usually from different internet of things devices and therefore may have different formats, the internet of things data may be standardized or converted into a uniform format before being used or processed, so as to ensure that the standardized or converted format is compatible with an application program for analyzing or processing the internet of things data.
In an example embodiment of the present disclosure, data of the internet of things may be obtained, where the data of the internet of things includes normal data of the internet of things and abnormal data of the internet of things. For example, because the environment of the internet of things device is complex, transient faults are likely to occur, and some abnormal data can be collected at this time, or because the network structure of the internet of things is complex and the network transmission is unstable, the problem of data abnormality can also occur in the data transmission process. At this moment, the obtained data of the internet of things comprises normal data of the internet of things and abnormal data of the internet of things, and the abnormal data of the internet of things can influence the accuracy rate of data classification and greatly influence the user experience.
Step S220, inputting the data of the Internet of things into a data classification model, and detecting abnormal data of the Internet of things in the data of the Internet of things;
in an example embodiment of the present disclosure, the internet of things data may be input into a data classification model. Specifically, the data classification model can be used for detecting whether the input data of the internet of things is normal data or abnormal data of the internet of things, correcting the abnormal data of the internet of things, and classifying the corrected normal data of the internet of things and the original normal data of the internet of things. It should be noted that the present disclosure is not limited to the specific form of the data classification model.
In an example embodiment of the present disclosure, the abnormal data of the internet of things in the data of the internet of things may be detected through a data classification model. Specifically, the data of the internet of things obviously not conforming to the current scene can be determined as the abnormal data of the internet of things, and the rule or method for detecting the abnormal data of the internet of things is integrated in the data classification model. For example, for growth environment information of cows, 14:00, 14:10, 14:20, 14: the temperature information obtained by the 30 is respectively 20 ℃, 21 ℃, 35 ℃ and 19 ℃, at this time, the 35 ℃ corresponding to 14:20 obviously does not accord with the current temperature scene, at this time, the 35 ℃ corresponding to 14:20 can be determined as the abnormal data of the internet of things, 14:00, 14:10, 14:20 and 14: the humidity information acquired by the 30 is 45%, 50%, 20% and 50% respectively, at this time, 20% corresponding to 14:20 obviously does not accord with the current humidity scene, and at this time, 20% corresponding to 14:20 can be determined as the abnormal data of the internet of things. And a normal data range can be set, and when certain data of the internet of things exceeds the normal data range, the data of the internet of things is determined as abnormal data of the internet of things. It should be noted that, the method for detecting the abnormal data of the internet of things is not particularly limited in the present disclosure.
In an example embodiment of the disclosure, first internet-of-things normal data corresponding to internet-of-things data may be acquired; the first internet of things normal data are internet of things normal data of a previous time unit of the internet of things data, and the internet of things abnormal data in the internet of things data can be detected according to the first internet of things normal data. Referring to fig. 3, detecting the abnormal data of the internet of things in the data of the internet of things according to the normal data of the first internet of things may include the following steps S310 to S320:
step S310, acquiring first Internet of things normal data corresponding to the Internet of things data; the first Internet of things normal data is Internet of things normal data of a previous time unit of the Internet of things data;
in an example embodiment of the disclosure, first internet-of-things normal data corresponding to internet-of-things data may be acquired. Specifically, when the internet of things data is acquired, the internet of things data can be acquired according to time units. For example, the data of the internet of things may be acquired every 10 minutes, or the data acquired every 10 minutes may be acquired after being processed. After the current internet of things data is acquired, the internet of things normal data of the previous time unit can be acquired, and the internet of things normal data of the previous time unit is the first internet of things normal data. It should be noted that, the time unit for acquiring the data of the internet of things is not particularly limited in the present disclosure.
Step S320, detecting abnormal data of the Internet of things in the data of the Internet of things according to the normal data of the first Internet of things.
In an example embodiment of the disclosure, after the first internet of things normal data is acquired, the internet of things abnormal data in the internet of things data may be detected according to the first internet of things normal data. Specifically, because the change range of the data of the internet of things is small along with the change of time, the current data of the internet of things can be compared with the normal data of the first internet of things, and when the difference value between the normal data of the first internet of things and the current data of the internet of things is large, the current data of the internet of things is determined as the abnormal data of the internet of things. Further, the difference value between every two pieces of normal data of the internet of things in the data of the internet of things can be calculated, the average value of the difference values is calculated, and the average value of the difference values is used as a standard for evaluating whether the difference value between the first normal data of the internet of things and the current data of the internet of things is larger. It should be noted that, the method for determining the abnormal data of the internet of things according to the normal data of the first internet of things is not particularly limited in the present disclosure.
Through the steps S310 to S320, the first normal data of the Internet of things corresponding to the data of the Internet of things can be obtained, and the abnormal data of the Internet of things in the data of the Internet of things can be detected according to the first normal data of the Internet of things.
In an example embodiment of the disclosure, an internet of things data anomaly threshold value may be obtained, and internet of things anomaly data in the internet of things data is detected according to the first internet of things normal data, the internet of things data and the internet of things data anomaly threshold value. Referring to fig. 4, detecting the abnormal data of the internet of things in the data of the internet of things according to the first normal data of the internet of things, the data of the internet of things and the abnormal threshold of the data of the internet of things may include the following steps S410 to S420:
step S410, acquiring an Internet of things data exception threshold;
in an example embodiment of the present disclosure, an internet of things data exception threshold may be set, where the internet of things data exception threshold may indicate that when the obtained data exceeds the internet of things data exception threshold, the current internet of things data is determined as the internet of things exception data. Specifically, when the internet of things data abnormal threshold is set, a proper internet of things data abnormal threshold can be set according to a specific service scene, and the internet of things data abnormal threshold can also be determined in a mathematical operation mode, for example, 5 maximum internet of things normal data in a certain period of time can be obtained, an average number of the 5 internet of things normal data is calculated, and the average number is used as the internet of things data abnormal threshold. Specifically, the internet of things data exception threshold may be stored in a memory of the terminal device or during a service, and when the internet of things data exception threshold needs to be used, the internet of things data exception threshold may be called from the server or the memory. It should be noted that, the setting manner of the data anomaly threshold of the internet of things is not particularly limited in the present disclosure.
Step S420, detecting abnormal data of the Internet of things in the Internet of things data according to the first normal data of the Internet of things, the Internet of things data and the abnormal threshold of the Internet of things data.
In an example embodiment of the disclosure, after the internet of things data abnormal threshold value is obtained through the above steps, the internet of things abnormal data in the internet of things data may be detected according to the first internet of things normal data, the internet of things data and the internet of things data abnormal threshold value. Specifically, when the acquired internet of things data exceeds an interval defined by an abnormal threshold of the internet of things data, the internet of things data is determined as the abnormal internet of things data. For example, for the temperature information of the cow, the internet of things data abnormal threshold may be set in units of days, for example, the internet of things data abnormal threshold set on a certain day in summer is 20 ℃ and 43 ℃, when the obtained temperature is 10 ℃, 10 ℃ is less than the internet of things data abnormal threshold 20 ℃, and the 10 ℃ may be determined as the internet of things abnormal data.
Through the steps S410-S420, the abnormal data threshold of the Internet of things can be obtained, and the abnormal data of the Internet of things in the data of the Internet of things can be detected according to the first normal data of the Internet of things, the data of the Internet of things and the abnormal data threshold of the data of the Internet of things.
In an example embodiment of the disclosure, a difference value between the first internet-of-things normal data and the internet-of-things data may be calculated, and whether the internet-of-things data is the internet-of-things abnormal data or not may be determined according to a magnitude relation between the difference value and the internet-of-things data abnormal threshold. Referring to fig. 5, determining whether the data of the internet of things is the data of the internet of things anomaly according to the magnitude relationship between the difference and the data of the internet of things anomaly threshold may include the following steps S510 to S520:
step S510, calculating a difference value between the first Internet of things normal data and the Internet of things data;
in an example embodiment of the present disclosure, after the first internet of things normal data is acquired, a difference value between the first internet of things normal data and the internet of things data may be calculated. Specifically, the data of the internet of things can be obtained in real time, and the data of the internet of things and the corresponding normal data of the first internet of things are used as difference values.
Step S520, whether the data of the Internet of things is abnormal data of the Internet of things is determined according to the size relation between the difference value and the abnormal threshold value of the data of the Internet of things.
In an example embodiment of the disclosure, after the difference value between the first internet of things normal data and the internet of things data is obtained, whether the internet of things data is the internet of things abnormal data or not may be determined according to a magnitude relation between the difference value and the internet of things data abnormal threshold. Specifically, the data anomaly threshold of the internet of things can represent the deviation degree of the data of the internet of things, the data anomaly threshold of the internet of things can be set in advance, the data anomaly threshold of the internet of things is stored in a memory of the server or the terminal device, and the data anomaly threshold of the internet of things can be called in the memory of the server or the terminal device when the data anomaly threshold of the internet of things needs to be used. When the abnormal threshold value of the data of the internet of things is obtained, the abnormal threshold value of the data of the internet of things and the difference value can be compared, and when the difference value is larger than the abnormal threshold value of the data of the internet of things, the data of the internet of things corresponding to the difference value is determined as the abnormal data of the internet of things.
In an example embodiment of the present disclosure, internet of things data x may also be settNormal data x of the first internet of thingst-1When the square of the difference value is larger than the abnormal threshold value m of the data of the internet of things, determining the data of the internet of things as the abnormal data of the internet of things, wherein a specific expression is as follows:
|xt-xt-1|22
|xt-xt-1|2>m
through the steps of S510 to S520, the difference between the first internet of things normal data and the internet of things data can be calculated, and whether the internet of things data is the internet of things abnormal data or not is determined according to the size relationship between the difference and the internet of things data abnormal threshold.
And step S230, correcting the abnormal data of the Internet of things into normal data of the Internet of things, and classifying the normal data of the Internet of things according to the data classification model.
In an example embodiment of the present disclosure, after the abnormal data of the internet of things is obtained, the data of the internet of things may be corrected into normal data of the internet of things. Specifically, when the acquired data of the internet of things is determined to be abnormal data of the internet of things, an error occurs in the data acquisition end (such as a sensor) of the internet of things or the data of the internet of things in the transmission process, and the abnormal data of the internet of things needs to be corrected to normal data of the internet of things at the moment so as to ensure the accuracy and the continuity of the data of the internet of things. For example, the abnormal data of the internet of things can be corrected into the data of the internet of things of the previous time unit corresponding to the data of the internet of things, or the normal data of the internet of things corresponding to the abnormal data of the internet of things in a certain period of time can be obtained, and the average value of the normal data of the internet of things is used as the corrected normal data of the internet of things, or the normal data of the internet of things corresponding to the time unit of the previous day can be used as the corrected normal data of the internet of things. It should be noted that, the manner of correcting the abnormal data of the internet of things to the normal data is not particularly limited in the present disclosure.
In an example embodiment of the disclosure, one or more second networking normal data corresponding to the internet of things abnormal data may be acquired, an abnormal data correction factor is acquired, and the internet of things abnormal data is corrected into the internet of things normal data according to the second networking normal data and the abnormal data correction factor. Referring to fig. 6, the correcting the abnormal data of the internet of things to the normal data of the internet of things according to the normal data of the second internet of things and the abnormal data correction factor may include the following steps S610 to S620:
step S610, acquiring one or more second internet of things normal data corresponding to the internet of things abnormal data, and acquiring abnormal data correction factors; the second internet of things normal data are internet of things normal data in a unit interval of adjacent time of the internet of things abnormal data;
in an example embodiment of the disclosure, one or more second internet of things normal data corresponding to the internet of things abnormal data may be acquired. Specifically, the second internet of things normal data may include internet of things normal data in an adjacent time unit interval of the internet of things abnormal data. For example, for humidity information, the acquired internet of things anomaly data is 48 ℃, and the corresponding time is 10: 20, the normal data of the internet of things in the adjacent time unit interval of the abnormal data of the internet of things can be acquired at the moment, if the temperature information is acquired every 10 minutes, it can be determined that one adjacent time unit interval is 30 minutes at the moment, and the normal data of the internet of things are acquired for 6 times in total in the adjacent time unit interval (three times are acquired for the first three time units of the abnormal data of the internet of things, and three times are acquired for the last three time units of the abnormal data of the internet of things). The abnormal data correction factor can be obtained, specifically, the abnormal data correction factor can be used for correcting the abnormal data of the internet of things, and the specific numerical value of the abnormal data correction factor can be adjusted according to the service scene. It should be noted that the specific form and specific numerical value of the abnormal data correction factor in the present disclosure are not particularly limited.
And S620, correcting the abnormal data of the Internet of things into normal data of the Internet of things according to the normal data of the second Internet of things and the abnormal data correction factor.
In an example embodiment of the disclosure, after the second networking normal data and the abnormal data correction factor are acquired, the internet of things abnormal data may be corrected into the internet of things normal data according to the second networking normal data and the abnormal data correction factor. Specifically, the plurality of second internet-of-things normal data may be multiplied by the abnormal data correction factor, and then the multiplied results may be adaptively adjusted, and the adjusted internet-of-things data may be used as the corrected internet-of-things normal data.
Through the steps S610 to S620, one or more second internet of things normal data corresponding to the internet of things abnormal data can be obtained, the abnormal data correction factor is obtained, and the internet of things abnormal data is corrected into the internet of things normal data according to the second internet of things normal data and the abnormal data correction factor.
In an example embodiment of the disclosure, the second internet of things normal data may be adjusted according to the first correlation correction factor and the second correlation correction factor to obtain second internet of things predicted data, and the internet of things abnormal data is corrected into the internet of things normal data according to the second internet of things predicted data. Referring to fig. 7, the step of correcting the abnormal data of the internet of things into the normal data of the internet of things according to the second networking prediction data may include the following steps S710 to S720:
step S710, adjusting the normal data of the second networking according to the first correlation correction factor and the second correlation correction factor to obtain second networking prediction data; the second networking normal data comprise networking normal data in a front adjacent time unit interval and networking normal data in a rear adjacent time unit interval;
in an example embodiment of the disclosure, the plurality of second networking normal data are respectively adjusted according to the first correlation correction factor and the second correlation correction factor, and the adjusted results are added to obtain second networking prediction data. Specifically, the normal data of the internet of things in the time unit interval adjacent to the front of the abnormal data of the internet of things can be adjusted according to the first relevant correction factor, the normal data of the internet of things in the time unit interval adjacent to the rear of the abnormal data of the internet of things can be adjusted according to the second relevant correction factor, and the adjusted data are added to obtain the second internet of things prediction data.
For example, the time corresponding to the abnormal data of the internet of things is 11: 00, acquiring data of the internet of things once every 10 minutes, and assuming that an adjacent time unit interval is 30 minutes, the normal data of the second internet of things in the previous adjacent time unit interval of the abnormal data of the internet of things are respectively 10: 30 normal data of the internet of things, 10: 40, normal data of the internet of things, 10: 40, may be calculated from the first correlation correction factor pair 10: 30 normal data of the internet of things, 10: 40, normal data of the internet of things, 10: 40, adjusting the data of the normal data of the internet of things, wherein the normal data of the second internet of things in the adjacent time unit intervals after the abnormal data of the internet of things are respectively 11: 10, normal data of the internet of things, 11: 20, internet of things normal data, 11: 30, may correct the data according to the second correlation correction factor pair 11: 10, normal data of the internet of things, 11: 20, internet of things normal data, 11: and 30, adjusting the normal data of the internet of things, and adding the adjusted data to obtain second internet of things predicted data.
And S720, correcting the abnormal data of the Internet of things into normal data of the Internet of things according to the second Internet of things prediction data.
In an example embodiment of the present disclosure, after the second networking prediction data is obtained through the above steps, the internet of things abnormal data may be corrected to the internet of things normal data according to the second networking prediction data. Specifically, the second internet of things predicted data may be averaged, and the averaged data may be used as corrected normal data of the internet of things.
Further, when averaging the second networking prediction data, the influence of the first correlation correction factor and the second correlation correction factor needs to be considered. For example, the first correlation correction factor β has a value range of (0, 0.5), the second correlation correction factor γ is an infinite constant close to 1, and the normal data of the second network in the previous adjacent time unit intervals are x respectivelyt-1And xt-2The normal data of the second network of things in the later adjacent time unit interval are x respectivelyt+1And xt+2The obtained abnormal data of the Internet of things is xtAt this time, the corrected normal data x of the internet of thingsinThe expression of (a) is as follows:
Figure BDA0003044167620000161
through the steps S710 to S720, the second internet of things normal data can be adjusted according to the first correlation correction factor and the second correlation correction factor to obtain second internet of things predicted data, and the internet of things abnormal data is corrected into internet of things normal data according to the second internet of things predicted data.
In an example embodiment of the present disclosure, the data classification model may include an exception handling long and short memory network model, and the exception handling long and short memory network model may detect whether the input data of the internet of things is abnormal data of the internet of things, and correct the abnormal data of the internet of things into normal data of the internet of things if the input data of the internet of things is abnormal data of the internet of things. The exception handling Long-Short Memory network model may include a Long Short-Term Memory network (LSTM) that includes a forgetting gate that determines what information should be forgotten, an input gate that indicates what information is stored, and an output gate that determines which portion needs to be output.
In an example embodiment of the present disclosure, a forgetting gate, which is used to decide how much information to discard from a memory of a previous time unit, may be mapped to a value between 0 and 1. Where 0 indicates complete discard and 1 indicates complete retention. f. oftThe representation of the forgetting gate is based on the input data x of the Internet of thingstAnd implicit state information h of the previous time unitt-1To determine the retention state of the memory cell in the previous time unit, where σ represents the sigmod function, W(f)Weight matrix, U, representing a forgetting gate(f)Express weight matrix that forgets the door of the preceding time unit of forgetting the door, in the scheme of this disclosure, thing networking normal data includes corrects thing networking abnormal data into thing networking normal data xtnAnd original normal data x of the Internet of thingstAt this time, the expression of the forgetting gate is as follows:
Figure BDA0003044167620000162
in an example embodiment of the disclosure, after the internet of things data is acquired, whether the square of the absolute value of the difference value between the internet of things data and the first internet of things normal data is less than or equal to an internet of things data abnormity threshold value or not is judged, when the square of the absolute value of the difference value between the data of the internet of things and the normal data of the first internet of things is larger than the abnormal threshold value of the data of the internet of things, the data of the internet of things is represented as abnormal data of the internet of things, the abnormal data of the internet of things can be corrected into normal data of the internet of things, the forgetting gate is determined according to the corrected normal data of the internet of things, when the square of the absolute value of the difference value between the data of the Internet of things and the normal data of the first Internet of things is less than or equal to the abnormal threshold value of the data of the Internet of things, the data of the Internet of things is represented as normal data of the Internet of things, the forgetting gate can be directly determined according to the normal data of the Internet of things, and finally the normal data of the Internet of things is classified according to the determined forgetting gate. As shown in fig. 8, classifying the normal data of the internet of things according to the determined forgetting gate may include the following steps S810 to S860: step S810, acquiring Internet of things data; step S820, judging whether the data of the Internet of things is normal data of the Internet of things; step S830, when the data of the Internet of things is abnormal data of the Internet of things, the abnormal data of the Internet of things is corrected into normal data of the Internet of things; step 840, determining a forgetting gate according to the corrected normal data of the internet of things; step S850, when the data of the Internet of things is normal data of the Internet of things, determining a forgetting gate according to the normal data of the Internet of things; and step S860, classifying the normal data of the Internet of things according to the determined forgetting gate.
In an exemplary embodiment of the disclosure, the refresh gate may be composed of a forgetting gate and an input gate, and the forgetting gate and the refresh gate respectively control the temporary storage unit and the first storage unit to determine the second storage unit. As shown in fig. 9, the forgetting gate and the refresh gate respectively control the temporary storage unit and the first storage unit to determine the second storage unit, which may include the following steps S910 to S920:
step S910, forming an updating door according to the forgetting door and the input door;
in step S920, the forgetting gate and the refresh gate respectively control the temporary memory unit and the first memory unit to determine the second memory unit.
In an example embodiment of the present disclosure, the first memory unit is a memory unit generated by the previous time unit exception handling long and short memory network model, the temporary memory unit is a memory unit generated by the current time unit exception handling long and short memory network model, and the second memory unit is a memory unit generated by the current time unit exception handling long and short memory network model. Specifically, the forgetting gate and the input gate can form an updating gate, the temporary memory unit is controlled by the updating gate, and the first memory unit is controlled according to the forgetting gate. For example, in the exception handling long and short memory network model, the forgetting gate is ftThe temporary memory cell is C'tThe first memory cell is Ct-1The second memory cell CtThe expression of (a) is:
Ct=(1-ft)·C′t+ft·Ct-1
through the steps S910 to S920, the refresh gate can be formed by the forgetting gate and the input gate, and the forgetting gate and the refresh gate respectively control the temporary storage unit and the first storage unit to determine the second storage unit.
In an example embodiment of the present disclosure, first hidden state information of an exception handling long-short memory network model may be obtained, and second hidden state information may be determined according to the first hidden state information and a second memory unit. Referring to fig. 10, determining the second hidden state information according to the first hidden state information and the second memory cell may include the following steps S1010 to S1020:
step S1010, acquiring first implicit state information of the exception handling long and short memory network model;
in step S1020, second hidden state information is determined according to the first hidden state information and the second memory unit.
In an example embodiment of the present disclosure, first implicit state information of an exception handling long and short memory network model may be obtained. The first hidden state information is generated by the previous time unit exception handling long and short memory network model, and the second hidden state information is generated by the current time unit exception handling long and short memory network model. Specifically, the second hidden state information can be determined according to the first hidden state information and the second memory unit, and because the difference between the internet of things data of the previous time unit and the internet of things data of the next time unit is small in the internet of things environment, the effect of the first hidden state information can be enhanced, and the accuracy of the abnormal processing long and short memory network model on data classification is higher. For example, the first implicit status information is ht-1The second memory cell is ctThe candidate memory cell is otTanh represents an activation function, which is a normal-curvature-binary-cut function to be used when candidate memory cells need to be generated, and second hidden-state information htThe expression of (a) is:
ht=ottanh(ct)+xttanh(ht-1)
through the steps S1010 to S1020, the first hidden state information of the exception handling long-short memory network model can be obtained, and the second hidden state information is determined according to the first hidden state information and the second memory unit.
In an example embodiment of the present disclosure, it may be determined whether the input data of the internet of things is abnormal data of the internet of things, if so, the abnormal data of the internet of things is corrected to normal data of the internet of things, all normal data of the internet of things are input, an update gate is set in the abnormality processing length and length memory network model, and the role of the first hidden state information is increased when the second hidden state information is determined. Specifically, the structure of the exception handling long and short memory network model is shown in fig. 11.
In an example embodiment of the present disclosure, the normal data of the internet of things can be divided into different categories and stored in the database. Specifically, the normal data of the internet of things may include multiple categories, such as temperature, humidity, location, weight, soil information, and other categories. The normal data of the internet of things can be stored in corresponding databases according to categories, an internet of things normal data search system can be provided for a user, and the user can conveniently inquire the normal data of the internet of things, as shown in fig. 12, the internet of things normal data search system is a schematic diagram of the internet of things normal data search system, a plurality of sensors can be installed in a physical world, the internet of things data collected by the sensors are stored in edge nodes, then an abnormal processing long and short memory network model is input for classification, the normal data of the internet of things are input into the databases of the corresponding categories according to different categories, the user can acquire the internet of things data collected by the internet of things sensors through context awareness, analyze and count the data, establish an inquiry matching function, link inquiry matching with a search engine, and perform fusion analysis on the normal data of the internet of things stored in the databases, the knowledge base is constructed on the basis of data mining, so that personalized recommendation and common function recommendation can be performed on a user, an index is established to accelerate the searching speed, the index is linked with a search engine, and the user can monitor and check normal data of the Internet of things through mobile phone software or the search engine at the front end of a webpage.
Further, since the internet of things data generally corresponds to time, the bifluxdb database with a time stamp may be used to store the internet of things normal data. In particular, InfluxDB is an open source time series database, which is intended to handle high write and query loads and provides an SQL-like query language called InfluxQL for interacting with data. The reason why infiluxdb supports the networked system is that millions of reads can be made per second, which can meet the requirements of maximum monitoring and IoT deployment. The InfluxDB database is adopted to store networking normal data, and the data of the Internet of things can be processed (such as minimum, average, summation and the like) by directly utilizing a function of the database which is carried by the database and is related to time.
In the data classification method provided by the exemplary embodiment, after the data of the internet of things is acquired, the data of the internet of things can be input into the data classification model, the abnormal data of the internet of things in the data of the internet of things is detected, the abnormal data of the internet of things is corrected into the normal data of the internet of things, and the normal data of the internet of things is classified according to the data classification model.
According to the embodiment of the disclosure, abnormal data in the data of the internet of things can be corrected into normal data, and the normal data of the internet of things can be classified. On one hand, abnormal data can be processed, the accuracy of data classification is improved, and the problem that the user experience is reduced as the abnormal data is inquired by the user and is directly classified and stored is avoided; on the other hand, abnormal data are corrected instead of being directly eliminated, and the continuity of the data of the Internet of things can be guaranteed, so that the product trust of the user is improved, and the purchasing confidence of the user is further enhanced.
It is noted that the above-mentioned figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
In addition, in an exemplary embodiment of the present disclosure, a data classification apparatus is also provided. Referring to fig. 13, a data sorting apparatus 1300 includes: the system comprises an internet of things data acquisition module 1310, an abnormal data monitoring module 1320 and an internet of things data classification module 1330.
The system comprises an Internet of things data acquisition module, a data acquisition module and a data processing module, wherein the Internet of things data acquisition module is used for acquiring Internet of things data; the data of the Internet of things comprises normal data of the Internet of things and abnormal data of the Internet of things; the abnormal data monitoring module is used for inputting the data of the Internet of things into the data classification model and detecting the abnormal data of the Internet of things in the data of the Internet of things; and the Internet of things data classification module is used for correcting the abnormal data of the Internet of things into normal data of the Internet of things and classifying the normal data of the Internet of things according to the data classification model.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the device for detecting abnormal data of the internet of things in the data of the internet of things includes: the first internet of things normal data acquisition unit is used for acquiring first internet of things normal data corresponding to the internet of things data; the first Internet of things normal data is Internet of things normal data of a previous time unit of the Internet of things data; and the abnormal data detection unit is used for detecting the abnormal data of the Internet of things in the data of the Internet of things according to the normal data of the first Internet of things.
In an exemplary embodiment of the disclosure, based on the foregoing scheme, the device detects the abnormal data of the internet of things in the data of the internet of things according to the normal data of the first internet of things, and includes: the Internet of things data anomaly threshold value acquisition unit is used for acquiring an Internet of things data anomaly threshold value; and the abnormal threshold detection unit is used for detecting the abnormal data of the internet of things in the data of the internet of things according to the normal data of the first internet of things and the abnormal threshold of the data of the internet of things.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the device detects the abnormal data of the internet of things in the data of the internet of things according to the first normal data of the internet of things and the abnormal threshold of the data of the internet of things, and includes: the difference value calculating unit is used for calculating the difference value between the first Internet of things normal data and the Internet of things data; and the abnormal data determining unit is used for determining whether the data of the internet of things is abnormal data of the internet of things according to the size relation between the difference value and the abnormal threshold value of the data of the internet of things.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the device corrects the abnormal data of the internet of things into the normal data of the internet of things, and includes: the second internet of things normal data acquisition unit is used for acquiring one or more second internet of things normal data corresponding to the internet of things abnormal data and acquiring an abnormal data correction factor; the second internet of things normal data are internet of things normal data in a unit interval of adjacent time of the internet of things abnormal data; and the first abnormal data correction unit is used for correcting the abnormal data of the Internet of things into the normal data of the Internet of things according to the normal data of the second Internet of things and the abnormal data correction factor.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the abnormal data correction factor includes a first correlation correction factor and a second correlation correction factor, and the abnormal data of the internet of things is corrected into the normal data of the internet of things according to the normal data of the second internet of things and the abnormal data correction factor, and the apparatus includes: the second networking prediction data acquisition unit is used for adjusting the second networking normal data according to the first correlation correction factor and the second correlation correction factor to obtain second networking prediction data; the second networking normal data comprise networking normal data in a front adjacent time unit interval and networking normal data in a rear adjacent time unit interval; and the second abnormal data correcting unit is used for correcting the abnormal data of the internet of things into normal data of the internet of things according to the second networking prediction data.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the data classification model includes an exception handling long and short memory network model.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the exception handling long and short memory network model includes a forgetting gate and an input gate, and the apparatus further includes: the updating door determining unit is used for forming an updating door according to the forgetting door and the input door; the second memory unit determining unit is used for respectively controlling the temporary memory unit and the first memory unit by the forgetting gate and the updating gate so as to determine the second memory unit; the first memory unit is generated by processing the exception of the previous time unit by the long and short memory network model; the temporary memory unit is generated by processing the exception of the current time unit by the long and short memory network model; the second memory unit is generated by the memory network model for processing the exception of the current time unit.
In an exemplary embodiment of the present disclosure, based on the foregoing scheme, the apparatus further includes: the first hidden state information acquisition unit is used for acquiring the first hidden state information of the long and short memory network model for exception handling; the second implicit state information determining unit is used for determining second implicit state information according to the first implicit state information and the second memory unit; the first hidden state information is generated by a previous time unit exception handling long and short memory network model; the second hidden state information is generated by the current time unit exception handling long and short memory network model.
For details that are not disclosed in the embodiments of the apparatus of the present disclosure, please refer to the embodiments of the data classification method of the present disclosure for the details that are not disclosed in the embodiments of the apparatus of the present disclosure.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the data classification method is also provided.
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 1400 according to such an embodiment of the present disclosure is described below with reference to fig. 14. The electronic device 1400 shown in fig. 14 is only an example and should not bring any limitations to the functionality and scope of use of the embodiments of the present disclosure.
As shown in fig. 14, the electronic device 1400 is embodied in the form of a general purpose computing device. The components of the electronic device 1400 may include, but are not limited to: the at least one processing unit 1410, the at least one memory unit 1420, the bus 1430 that connects the various system components (including the memory unit 1420 and the processing unit 1410), and the display unit 1440.
Where the storage unit stores program code, the program code may be executed by processing unit 1410 such that processing unit 1410 performs steps according to various exemplary embodiments of the present disclosure described in the "exemplary methods" section above in this specification. For example, the processing unit 1410 may execute step S210 shown in fig. 2 to obtain data of the internet of things, where the data of the internet of things includes normal data of the internet of things and abnormal data of the internet of things; step S220, inputting the data of the Internet of things into a data classification model, and detecting abnormal data of the Internet of things in the data of the Internet of things; and step S230, correcting the abnormal data of the Internet of things into normal data of the Internet of things, and classifying the normal data of the Internet of things according to the data classification model.
As another example, the electronic device may implement the various steps shown in FIG. 2.
The storage unit 1420 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)1421 and/or a cache memory unit 1422, and may further include a read only memory unit (ROM) 1423.
Storage unit 1420 may also include a program/utility 1424 having a set (at least one) of program modules 1425, such program modules 1425 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Bus 1430 may be any type of bus structure including a memory cell bus or memory cell controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 1400 may also communicate with one or more external devices 1470 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 1400, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 1400 to communicate with one or more other computing devices. Such communication can occur via an input/output (I/O) interface 1450. Also, the electronic device 1400 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 1460. As shown, the network adapter 1460 communicates with the other modules of the electronic device 1400 via the bus 1430. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 1400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the disclosure described in the above-mentioned "exemplary methods" section of this specification, when the program product is run on the terminal device.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes included in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims (12)

1. A method of data classification, comprising:
the method comprises the steps of obtaining data of the Internet of things, wherein the data of the Internet of things comprises normal data of the Internet of things and abnormal data of the Internet of things;
inputting the data of the Internet of things into a data classification model, and detecting abnormal data of the Internet of things in the data of the Internet of things;
and correcting the abnormal data of the Internet of things into normal data of the Internet of things, and classifying the normal data of the Internet of things according to the data classification model.
2. The method of claim 1, wherein the detecting the abnormal data of the internet of things in the data of the internet of things comprises:
acquiring first Internet of things normal data corresponding to the Internet of things data; the first Internet of things normal data is Internet of things normal data of a previous time unit of the Internet of things data;
and detecting the abnormal data of the Internet of things in the data of the Internet of things according to the normal data of the first Internet of things.
3. The method according to claim 2, wherein the detecting of the internet of things abnormal data in the internet of things data according to the first internet of things normal data comprises:
acquiring an abnormal threshold of the data of the Internet of things;
and detecting the abnormal data of the Internet of things in the data of the Internet of things according to the normal data of the first Internet of things and the abnormal threshold value of the data of the Internet of things.
4. The method of claim 3, wherein the detecting the abnormal data of the internet of things according to the first normal data of the internet of things and the abnormal threshold value of the internet of things data comprises:
calculating a difference value between the first Internet of things normal data and the Internet of things data;
and determining whether the data of the Internet of things is abnormal data of the Internet of things according to the size relationship between the difference value and the abnormal threshold value of the data of the Internet of things.
5. The method of claim 1, wherein the correcting the abnormal data of the internet of things into normal data of the internet of things comprises:
acquiring one or more second internet of things normal data corresponding to the internet of things abnormal data, and acquiring abnormal data correction factors; the second networking normal data are the networking normal data in the unit interval of the adjacent time of the networking abnormal data;
and correcting the abnormal data of the Internet of things into normal data of the Internet of things according to the normal data of the second Internet of things and the abnormal data correction factor.
6. The method as claimed in claim 5, wherein the abnormal data correction factor comprises a first correlation correction factor and a second correlation correction factor, and the correcting the abnormal data of the internet of things into the normal data of the internet of things according to the second internet of things normal data and the abnormal data correction factor comprises:
adjusting the normal data of the second networking according to the first correlation correction factor and the second correlation correction factor to obtain second networking prediction data; the second networking normal data comprises networking normal data in a front adjacent time unit interval and networking normal data in a rear adjacent time unit interval;
and correcting the abnormal data of the Internet of things into normal data of the Internet of things according to the second Internet of things prediction data.
7. The method of claim 1, wherein the data classification model comprises an exception handling long and short memory network model.
8. The method of claim 7, wherein the exception handling long and short memory network model comprises a forgetting gate and an input gate, and wherein the method further comprises:
forming an updating door according to the forgetting door and the input door;
the forgetting gate and the updating gate respectively control the temporary memory unit and the first memory unit to determine a second memory unit;
the first memory unit is generated by the exception handling long and short memory network model in the previous time unit; the temporary memory unit is a memory unit generated by the exception handling long and short memory network model in the current time unit; the second memory unit is generated by the exception handling long and short memory network model in the current time unit.
9. The method of claim 8, further comprising:
acquiring first hidden state information of the exception handling long and short memory network model;
determining second hidden state information according to the first hidden state information and the second memory unit;
the first hidden state information is hidden state information generated by the exception handling long and short memory network model in the previous time unit; the second hidden state information is hidden state information generated by the exception handling long and short memory network model in the current time unit.
10. An apparatus for classifying data, the apparatus comprising:
the Internet of things data acquisition module is used for acquiring Internet of things data; the data of the Internet of things comprises normal data of the Internet of things and abnormal data of the Internet of things;
the abnormal data monitoring module is used for inputting the Internet of things data into a data classification model and detecting the Internet of things abnormal data in the Internet of things data;
and the Internet of things data classification module is used for correcting the abnormal data of the Internet of things into normal data of the Internet of things and classifying the normal data of the Internet of things according to the data classification model.
11. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 9.
12. An electronic device, comprising:
a processor; and
memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9.
CN202110468156.2A 2021-04-28 2021-04-28 Data classification method, data classification device, medium, and electronic apparatus Pending CN113139817A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110468156.2A CN113139817A (en) 2021-04-28 2021-04-28 Data classification method, data classification device, medium, and electronic apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110468156.2A CN113139817A (en) 2021-04-28 2021-04-28 Data classification method, data classification device, medium, and electronic apparatus

Publications (1)

Publication Number Publication Date
CN113139817A true CN113139817A (en) 2021-07-20

Family

ID=76816304

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110468156.2A Pending CN113139817A (en) 2021-04-28 2021-04-28 Data classification method, data classification device, medium, and electronic apparatus

Country Status (1)

Country Link
CN (1) CN113139817A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118075782A (en) * 2024-04-17 2024-05-24 沈阳松陵三航机械制造有限公司 AI-driven 5G (Internet of things) based data processing system

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109379228A (en) * 2018-11-02 2019-02-22 平安科技(深圳)有限公司 Accidentally warning information recognition methods and device, storage medium, electric terminal
CN109753049A (en) * 2018-12-21 2019-05-14 国网江苏省电力有限公司南京供电分公司 The exceptional instructions detection method of one provenance net load interaction industrial control system
CN109947079A (en) * 2019-03-20 2019-06-28 阿里巴巴集团控股有限公司 Region method for detecting abnormality and edge calculations equipment based on edge calculations
CN110375787A (en) * 2019-07-25 2019-10-25 重庆市计量质量检测研究院 A kind of measuring instrument operating status on-line evaluation method
CN110377447A (en) * 2019-07-17 2019-10-25 腾讯科技(深圳)有限公司 A kind of abnormal deviation data examination method, device and server
CN110457550A (en) * 2019-07-05 2019-11-15 中国地质大学(武汉) The bearing calibration of misoperation data in a kind of sintering process
CN110505196A (en) * 2019-07-02 2019-11-26 中国联合网络通信集团有限公司 Internet of Things network interface card method for detecting abnormality and device
CN111143102A (en) * 2019-12-13 2020-05-12 东软集团股份有限公司 Abnormal data detection method and device, storage medium and electronic equipment
CN111476212A (en) * 2020-05-18 2020-07-31 哈尔滨理工大学 Motor fault detection system based on long-time and short-time memory method
CN111767930A (en) * 2019-04-01 2020-10-13 北京百度网讯科技有限公司 Method for detecting abnormal time series data of Internet of things and related equipment thereof
CN112085384A (en) * 2020-09-08 2020-12-15 华侨大学 Mailing risk evaluation method and system based on combination of fuzzy reasoning and LSTM
CN112202736A (en) * 2020-09-15 2021-01-08 浙江大学 Industrial control system communication network abnormity classification method based on statistical learning and deep learning
CN112270434A (en) * 2020-10-22 2021-01-26 国网山西省电力公司电力科学研究院 Wind-solar integrated power prediction method
CN112373253A (en) * 2020-04-26 2021-02-19 青岛慧拓智能机器有限公司 Automatic detection system and method for tire burst and undervoltage of unmanned vehicle
CN112396225A (en) * 2020-11-13 2021-02-23 浙江电力交易中心有限公司 Short-term electricity price prediction method based on long-term and short-term memory network

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109379228A (en) * 2018-11-02 2019-02-22 平安科技(深圳)有限公司 Accidentally warning information recognition methods and device, storage medium, electric terminal
CN109753049A (en) * 2018-12-21 2019-05-14 国网江苏省电力有限公司南京供电分公司 The exceptional instructions detection method of one provenance net load interaction industrial control system
CN109947079A (en) * 2019-03-20 2019-06-28 阿里巴巴集团控股有限公司 Region method for detecting abnormality and edge calculations equipment based on edge calculations
CN111767930A (en) * 2019-04-01 2020-10-13 北京百度网讯科技有限公司 Method for detecting abnormal time series data of Internet of things and related equipment thereof
CN110505196A (en) * 2019-07-02 2019-11-26 中国联合网络通信集团有限公司 Internet of Things network interface card method for detecting abnormality and device
CN110457550A (en) * 2019-07-05 2019-11-15 中国地质大学(武汉) The bearing calibration of misoperation data in a kind of sintering process
CN110377447A (en) * 2019-07-17 2019-10-25 腾讯科技(深圳)有限公司 A kind of abnormal deviation data examination method, device and server
CN110375787A (en) * 2019-07-25 2019-10-25 重庆市计量质量检测研究院 A kind of measuring instrument operating status on-line evaluation method
CN111143102A (en) * 2019-12-13 2020-05-12 东软集团股份有限公司 Abnormal data detection method and device, storage medium and electronic equipment
CN112373253A (en) * 2020-04-26 2021-02-19 青岛慧拓智能机器有限公司 Automatic detection system and method for tire burst and undervoltage of unmanned vehicle
CN111476212A (en) * 2020-05-18 2020-07-31 哈尔滨理工大学 Motor fault detection system based on long-time and short-time memory method
CN112085384A (en) * 2020-09-08 2020-12-15 华侨大学 Mailing risk evaluation method and system based on combination of fuzzy reasoning and LSTM
CN112202736A (en) * 2020-09-15 2021-01-08 浙江大学 Industrial control system communication network abnormity classification method based on statistical learning and deep learning
CN112270434A (en) * 2020-10-22 2021-01-26 国网山西省电力公司电力科学研究院 Wind-solar integrated power prediction method
CN112396225A (en) * 2020-11-13 2021-02-23 浙江电力交易中心有限公司 Short-term electricity price prediction method based on long-term and short-term memory network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
周建华;吕鹏远;王春;: "一种适用于海上风机监测数据实时处理的方法研究", 中国水利水电科学研究院学报, no. 04 *
张晶晶;黄浩;胡英;吾守尔・斯拉木;: "循环神经网络中基于特征融合的口语理解", 现代电子技术, no. 20 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118075782A (en) * 2024-04-17 2024-05-24 沈阳松陵三航机械制造有限公司 AI-driven 5G (Internet of things) based data processing system

Similar Documents

Publication Publication Date Title
US11449712B2 (en) Anomaly detection and reporting for machine learning models
US10915602B2 (en) Automatic detection of outliers in multivariate data
US11625602B2 (en) Detection of machine learning model degradation
US20200380309A1 (en) Method and System of Correcting Data Imbalance in a Dataset Used in Machine-Learning
US20160132904A1 (en) Influence score of a brand
JP2019519027A (en) Learning from historical logs and recommending database operations on data assets in ETL tools
US20200026759A1 (en) Artificial intelligence engine for generating semantic directions for websites for automated entity targeting to mapped identities
JP2023514465A (en) machine learning platform
US20110153542A1 (en) Opinion aggregation system
US11621892B2 (en) Temporal-based network embedding and prediction
US11734310B2 (en) Insights for multi-dimensional planning data
US20220222688A1 (en) Methodology of analyzing consumer intent from user interaction with digital environments
CN113139817A (en) Data classification method, data classification device, medium, and electronic apparatus
US10503575B2 (en) Computer systems monitoring using beat frequency analysis
US20230259569A1 (en) Systems and methods for automatic and adaptive browser bookmarks
US20190065987A1 (en) Capturing knowledge coverage of machine learning models
KR20190061317A (en) Method of providing stock information
CN113094584A (en) Method and device for determining recommended learning resources
CN112148976A (en) Data processing method and device, electronic equipment and storage medium
CN111767290A (en) Method and apparatus for updating a user representation
US11669424B2 (en) System and apparatus for automated evaluation of compatibility of data structures and user devices based on explicit user feedback
US11907508B1 (en) Content analytics as part of content creation
US11830081B2 (en) Automated return evaluation with anomoly detection
US11973657B2 (en) Enterprise management system using artificial intelligence and machine learning for technology analysis and integration
US11836656B2 (en) Cognitive enabled blockchain based resource prediction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination