AU2017386444A1 - Reliability management-based uncertainty elimination context awareness system and working method thereof - Google Patents

Reliability management-based uncertainty elimination context awareness system and working method thereof Download PDF

Info

Publication number
AU2017386444A1
AU2017386444A1 AU2017386444A AU2017386444A AU2017386444A1 AU 2017386444 A1 AU2017386444 A1 AU 2017386444A1 AU 2017386444 A AU2017386444 A AU 2017386444A AU 2017386444 A AU2017386444 A AU 2017386444A AU 2017386444 A1 AU2017386444 A1 AU 2017386444A1
Authority
AU
Australia
Prior art keywords
context information
context
information
module
inconsistency
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
AU2017386444A
Other versions
AU2017386444B2 (en
Inventor
Min Chen
Baozhen DU
Zhengfeng Du
Mingyang JI
Wei Ji
Lingling PAN
Hongji XU
Hui Yuan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Publication of AU2017386444A1 publication Critical patent/AU2017386444A1/en
Application granted granted Critical
Publication of AU2017386444B2 publication Critical patent/AU2017386444B2/en
Ceased legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/02Knowledge representation; Symbolic representation
    • G06N5/022Knowledge engineering; Knowledge acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/02Knowledge representation; Symbolic representation
    • G06N5/022Knowledge engineering; Knowledge acquisition
    • G06N5/025Extracting rules from data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/04Inference or reasoning models
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02Services making use of location information
    • H04W4/025Services making use of location information using location based information parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/80Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

A reliability management-based uncertainty elimination context awareness system and a working method thereof. The system comprises a context information acquisition module, a context information processing module, a knowledge base module, a context response module, a context information application module, a context information retrieval/subscription module, a context information correction module, and a user feedback module. The context processing module comprises a multi-algorithm incompleteness elimination unit, an algorithm result inconsistency elimination unit, a credibility management unit, a reliability management unit, an information source inconsistency elimination unit, a context information fusion and reasoning unit, and a self-adaptive management unit. By means of the present invention, more reliable context information can be obtained, and the accuracy, reliability and self-adaptability of the context awareness system are significantly improved.

Description

CONTEXT-AWARE SYSTEM AND METHOD FOR UNCERTAINTY ELIMINATION BASED
ON RELIABILITY MANAGEMENT
CROSS-REFERENCE TO RELATED PATENT APPLICATIONS [0001] This application is based upon and claims priority to Chinese Patent Application No. 201611251838.3, filed on Dec. 30, 2016, the entire contents of which are incorporated herein by reference.
FIELD OF THE INVENTION [0002] The invention generally relates to the field of context awareness. More particularly, the invention relates to a context-aware system and method for uncertainty elimination based on reliability management.
BACKGROUND OF THE INVENTION [0003] With the rapid improvement of wireless sensing, human-computer interaction and intelligent computing technologies, context-aware technology has been developed rapidly, and context-aware systems are gradually integrated into people's daily lives. The context-aware system is a human-centered computing system in which sensing devices can automatically perceive context information and its changes, and provide users with services related to the current context information. [0004] The ideal context information should be accurate, complete and consistent, but in actual context-aware applications, context information is collected by different/heterogeneous sources in various ways. Due to the jitter of acquisition intervals, unreliable conditions, packet loss and delay of network transmission and so on, the collected context information could be inaccurate (i.e., there is a big difference between the collected context information and real context information), incomplete (i.e., the collected context information is missing at certain time) and inconsistent (i.e., the collected context information is conflicting).
[0005] Original context information usually needs to be fused and reasoned into high-level context information before it is utilized by applications and devices, and the quality of the original context information plays a vital role in the result of fusion reasoning of context information. Therefore, the basis of effectively using the original context information is to eliminate the uncertainty of the original context information, and then improve the accuracy and reliability of fusion reasoning of context information.
[0006] The existing context-aware systems usually eliminate one aspect of the uncertainty problems, and do not consider comprehensive uncertainty elimination. In terms of context incompleteness, single elimination algorithms are usually adopted, which could result in low accuracy; in terms of context inconsistency, the credibility evaluation mechanism based on single source is usually adopted, which could result in low reliability. Therefore, how to improve the functions of context-aware system and increase the accuracy of uncertainty elimination has become the challenge of context-aware technology.
SUMMARY [0007] In view of the above problems, the present invention proposes a context-aware system and method for uncertainty elimination based on reliability management.
[0008] In the present invention, context inaccuracy problem is seen as context incompleteness problem, and for context incompleteness problem, multiple algorithms such as neural network, Dempster-Shafer (D-S) evidence theory, expectation-maximization (EM) algorithm, fuzzy set theory, etc., are initially adopted to solve incompleteness problem from horizontal and vertical directions, where the horizontal direction refers to different times of the same sensor and the vertical direction refers to the same time of different sensors, then the voting algorithm is used to do simple inconsistency elimination for the results of multiple algorithms, finally the complete context information with higher accuracy can be obtained.
[0009] In the present invention, for context inconsistency problem, D-S evidence theory based on reliability management is adopted to eliminate inconsistent contexts, and then highly reliable context information can be obtained, where the reliability is calculated by combining sensor precision and credibility parameter.
[0010] The system proposed by the present invention can realize effective elimination of uncertainty problems such as inaccuracy, incompleteness and inconsistency.
[0011] In order to achieve these objectives, the present invention adopts the following technical solutions.
[0012] The present invention provides a context-aware system for uncertainty elimination based on reliability management, and it comprises a context information acquisition module, a context information processing module, a knowledge base module, a context information response module, a context information application module, a context information retrieval or subscription module, a context information correction module and a user feedback module.
[0013] Wherein the context information acquisition module, the context information processing module and the knowledge base module are connected in turn; the knowledge base module, the context information application module, the context information retrieval or subscription module and the context information correction module are connected in turn from beginning to end; the knowledge base module, the context information response module and the context information application module are connected in turn; the user feedback module is connected with the knowledge base module and the context information application module, respectively.
[0014] The context information acquisition module periodically collects original context information by multiple physical sensors, virtual sensors and logical sensors, and then sends the collected original context information to the context information processing module, where the original context information is multi-source context information, i.e., the context information collected by multiple sensors. For example, the user’s location information can be collected by Bluetooth, WiFi, infrared sensor, Zigbee, etc.
[0015] The context information processing module processes the original context information from the context information acquisition module.
[0016] The knowledge base module stores user feedback information, context fusion reasoning information, context retrieval or subscription information and context application information, where context retrieval or subscription information is stored into the knowledge base module going through the context correction module, and context application information is stored into the knowledge base module going through the context retrieval or subscription module and the context correction module.
Meanwhile, it provides various context application information for the context information response module and the required context information for the context information processing module.
[0017] The context information application module displays the output comprehensive information from the context information response module and the user feedback module on the interface of the context-aware system, and sends the comprehensive context information to the context information retrieval or subscription module.
[0018] The context information retrieval or subscription module searches corresponding context information in the knowledge base module according to the retrieval requirements of the context information application module. On the other hand, it sends the relevant subscription information to the context information correction module according to the subscription requirements of the context information application module.
[0019] The context information response module searches relevant context information in the knowledge base module according to the requirements from the context information application module, and then sends the required context information to the context information application module. [0020] The context information correction module corrects the context information sent by the context information retrieval or subscription module, and then sends the corrected context information to the knowledge base module.
[0021] The user feedback module stores the context information from users in certain environments into the knowledge base module, and provides the context information application module with the required application information.
[0021] According to the preferred embodiment of the present invention, the context information processing module comprises a context information modeling unit, a multi-algorithm incompleteness elimination unit, an inconsistency elimination of multi-algorithm results unit, a multi-source inconsistency elimination unit, a credibility management unit, a reliability management unit, a context information fusion reasoning unit and an adaptive management unit.
[0022] Wherein the context information acquisition module, the context information modeling unit, the multi-algorithm incompleteness elimination unit, the inconsistency elimination of multi-algorithm results unit, the credibility management unit, the reliability management unit, the multi-source inconsistency elimination unit and the context information fusion reasoning unit are connected in turn; the adaptive management unit is connected with the multi-algorithm incompleteness elimination unit; the credibility management unit, the multi-source inconsistency elimination unit and the context information fusion reasoning unit respectively; the multi-source inconsistency elimination unit is connected with the credibility management unit; the context information fusion reasoning unit is connected with the knowledge base module; the knowledge base module is connected with the adaptive management unit.
[0023] The context information modeling unit models the multi-source context information collected by the context information acquisition module according to the context information modeling method in the knowledge base module. In the present invention, the modeling mode is “type of context awareness + context-aware information + context-aware accuracy”, where the type of context awareness refers to the type of context-aware information, the context-aware information refers to the original context information collected by sensors, and the context-aware accuracy refers to the precision of the sensor that collects the current context-aware information. For example, for the context-aware information “bedroom” collected by the sensor of 90% precision, its model is “type of context awareness—user location” + “context-aware information—bedroom” + “context-aware accuracy — 90%”. After that, the context information modeling unit sends the modeled context information to the multi-algorithm incompleteness elimination unit.
[0024] The multi-algorithm incompleteness elimination unit simultaneously adopts multiple algorithms to eliminate incompleteness, and then sends the elimination results of these algorithms to the inconsistency elimination of multi-algorithm results unit, where the incompleteness refers to a lack of original context information collected by certain sensor and the multiple algorithms include neural network, D-S evidence theory, EM, voting algorithm and fuzzy set theory, etc.
[0025] The inconsistency elimination of multi-algorithm results unit firstly adopts the voting algorithm to solve inconsistency problem among the algorithm results from the multi-algorithm incompleteness elimination unit, gets complete context information with higher accuracy, and then sends the processed complete context information to the credibility management unit, where the inconsistency is that there exists inconsistency among the results of eliminating the incompleteness of context information by multiple algorithms.
[0026] The credibility management unit judges the accuracy of the processed complete context information according to the received user feedback information, where the user feedback information refers to the context information that users actively feed back based on their environments, and then calculates the credibility of the corresponding sources according to the following equation and stores them:
N
Credibility(s) = —— (1)
Joint where .v denotes the current context information source, Nmr denotes the number of correct context information, and Nlolal denotes the total number of context information collected by the current context information source.
[0027] The reliability management unit manages the context information sources based on their reliability, where the reliability is related to the sensor precision and the credibility of the source. The reliability can be calculated by „ , , , „ „ Precisions Credibility s)
Reliability^) = 2x---------------------(J)
Precision + Credibility(s) where Precision denotes the precision of the current used sensor and is static measurement value. Credibility (s) is dynamic evaluation value. Then the reliability values of these sources are sent to the multi-source inconsistency elimination unit.
[0028] The multi-source inconsistency elimination unit adopts D-S evidence theory based on reliability management to eliminate the inconsistency among multiple sources, where the inconsistency refers to the inconsistency among the context information collected by different sensors, for example, infrared sensor detects that the user is in the bedroom, but Zigbee detects that the user is in the living room at this time, it is obvious that the user is impossible to appear in two places at the same time, thus there exists inconsistency. Then the inconsistency elimination result is sent to the credibility management unit and the context information fusion reasoning unit, respectively.
[0029] The context information fusion reasoning unit uses ontology reasoning, rule-based reasoning, D-S evidence theory or Bayesian network to deduce high-level context information according to the results of inconsistency elimination and the historical information in the adaptive management unit, and the high-level context information is stored into the knowledge base module, where the historical context information refers to the rule engine and rule set based on rule system in the adaptive management unit, and the high-level context information refers to the context information which can be used directly by users or various devices after fusion and reasoning.
[0030] The adaptive management unit provides user feedback information and correct information for the multi-algorithm incompleteness elimination unit, provides user feedback information for the credibility management unit, and provides the rule engine and rule set based on rule system for the context information fusion reasoning unit. On the other hand, it can make some proper adjustments on various parameters according to current context information, so that the context-aware system is of better adaptability.
[0031] According to the preferred embodiment of the present invention, the context information acquisition module comprises multiple physical sensors, virtual sensors and logical sensors.
[0032] The present invention further provides a method for uncertainty elimination based on reliability management using the above system, which comprises the following steps:
[0033] SOI: Collect context information and modeling [0034] Collect context information by multiple physical sensors, virtual sensors and logical sensors, and model the collected context information according to the modeling mode stored in the knowledge base, where the modeling mode is “type of context awareness + context-aware information + context-aware accuracy”.
[0035] S02: Detect whether the context information is complete or not [0036] Detect whether the modeled context information is complete or not, i.e., whether there exists missing context information, if not, then execute step S02’, otherwise, execute step S03.
[0037] S02’: Store relevant context information [0038] Store context information for the usage of the subsequent step S06 and step S09.
[0039] S03: Compute incompleteness rate [0040] Compute the incompleteness rate of the modeled context information by the following computation equation:
N .
Incompleteness rate = —(3) ^ιοιαί where Nmis denotes the number of missing context information.
[0041] S04: Judge whether the incompleteness is within a controllable range [0042] Judge whether the incompleteness is within a controllable range, if it is, then execute step SOS’, otherwise, execute step SOS. For example, if the controllable range of the incompleteness is set to be 15%, when the incompleteness rate is 10%, which is less than the controllable range, i.e., within the controllable range.
[0043] SOS: Delete the context information out of the controllable range [0044] When the incompleteness of the context information is out of the controllable range, delete the context information directly.
[0045] SOS’: Judge whether there exists user feedback [0046] Detect whether there exists user feedback at this time, if it exists, then execute step S06’, otherwise, execute step S06.
[0047] S06: Eliminate the incompleteness using multiple algorithms [0048] Adopt simultaneously multiple algorithms to eliminate the incompleteness of the context information from the horizontal and vertical directions, where the horizontal direction refers to different times of the same sensor, the vertical direction refers to the same time of different sensors, and the multiple algorithms include neural network, EM, voting algorithm, D-S evidence theory and fuzzy set theory algorithm, etc.
[0049] S06’: Fill up the incomplete context information directly [0050] Fill up the incomplete context information according to the user feedback information, then get the complete context information, and go back to step S02’.
[0051] S07: Judge whether there exists inconsistency among the results of algorithms [0052] Judge whether there exists inconsistency among the results of incompleteness elimination using multiple algorithms, if there exists, then execute step S08, otherwise, go back to step S02’.
[0053] S08: Eliminate the inconsistency of the algorithm results [0054] Adopt the voting algorithm to eliminate the inconsistency of the algorithm results, and get higher quality of complete context information.
[0055] S09: Judge whether the context information from multiple sensors is inconsistent [0056] Judge whether the context information collected from multiple sensors is inconsistent, if it is, then execute step S10’, otherwise, execute step S10;
[0057] S10: Compute the credibility of the source [0058] Compute the credibility of the source according to the equation (1), and then execute step S12, where the accuracy of context information is judged based on the user feedback information, and the credibility of the source is 1 before the context inconsistency occurs.
[0059] S10’: Judge whether there exists user feedback [0060] Judge whether there exists user feedback at this time, if there exists, then go back to step S10, otherwise, execute step Sil.
[0061] Sil: Eliminate inconsistency of multi-source context information [0062] Adopt D-S evidence theory to eliminate context inconsistency.
[0063] S12: Compute the reliability of the source [0064] Compute reliability on the basis of credibility parameter and sensor precision according to equation (2), and then apply the reliability of the source to context inconsistency elimination of step Sil, which can make the judgment result more true and reliable.
[0065] Suppose that Θ is a frame of discernment. If for any subset A that belongs to Θ , it satisfies m(0)=O and Σ m(A) = 1, then m is called the basic belief assignment (BPA) on the Θ, m(A) Adj is the basic belief value of A , and it reflects the reliability of A .
[0066] Assume that ml and m2 are two BPAs based on the same Θ , whose focal elements are respectively Al,A2,...,Ak and Bl,B2,...,Bk, and then the Dempster's Combination Rule (DCR) is as follows:
Σ MxJM·5/) m{(-} -------------- (4) V 7 l-K where m(C) reflects the joint support degree of two evidences that ml and m2 corresponds.
~ A Σ jnfA/n/Bj) js t|le confpct[ng degree of evidence, when K= 0, it is no conflict at all; when 0 < K < 1, it is incomplete conflict; when K = 1, it is complete conflict. The DCR satisfies the exchange law and the combination law, and the combination of multiple evidences can be obtained by repeatedly using the equation (4).
[0067] The present invention has the following outstanding advantages:
[0068] (1) The system proposed by the present invention has more perfect functions, which can effectively eliminate the uncertainty problems of original context information such as inaccuracy, incompleteness and inconsistency, and has strong practicability.
[0069] (2) High accuracy: the method proposed by the present invention firstly obtains preliminary complete context information using multiple algorithms from the horizontal and vertical directions, then eliminates the inconsistency of the algorithm results, and finally gets complete context information with higher accuracy.
[0070] (3) High reliability: the reliability management method combining the credibility of the source and the precision of the sensor is used to eliminate context inconsistency and more reliable context information can be obtained. At the same time, the users can actively feed back the information according to the changes of their own needs and the state of the environment, which can significantly improve the accuracy, reliability and adaptability of the context-aware system.
BRIEF DESCRIPTION OF THE DRAWINGS [0071] FIG. 1 is block diagram of a context-aware system for uncertainty elimination based on reliability management.
[0072] FIG. 2 is flow diagram of a method for uncertainty elimination based on reliability management. [0073] FIG. 3 is simulation diagram of the performance of the existing context-aware systems.
[0074] FIG. 4 is simulation diagram of an embodiment on the overall performance of the system proposed by the present invention.
DETAILED DESCRIPTION OF THE EMBODIMENTS [0075] The present invention is further described by incorporating the drawings and the specific embodiments hereinafter. Obviously, the described embodiments are only a part of the present invention, not all embodiments. Furthermore, it is worth noting that the embodiments are merely used for illustrating the present invention, rather than limiting the present invention.
Embodiment 1 [0076]As shown in FIG. 1, a context-aware system for uncertainty elimination based on reliability management comprises a context information acquisition module, a context information processing module, a knowledge base module, a context information response module, a context information application module, a context information retrieval or subscription module, a context information correction module and a user feedback module.
[0077] Wherein the context information acquisition module, the context information processing module and the knowledge base module are connected in turn; the knowledge base module, the context information application module, the context information retrieval or subscription module and the context information correction module are connected in turn from beginning to end; the knowledge base module, the context information response module and the context information application module are connected in turn; the user feedback module is connected with the knowledge base module and the context information application module, respectively.
[0078] The context information acquisition module periodically collects original context information by multiple physical sensors, virtual sensors and logical sensors, and then sends the collected original context information to the context information processing module, where the original context information is multi-source context information, i.e., the context information collected by multiple sensors. For example, the user’s location information can be collected by Bluetooth, WiFi, infrared sensor, Zigbee, etc.
[0079] The context information processing module processes the original context information from the context information acquisition module.
[0080] The knowledge base module stores user feedback information, context fusion reasoning information, context retrieval or subscription information and context application information, where context retrieval or subscription information is stored into the knowledge base module going through the context correction module, and context application information is stored into the knowledge base module going through the context retrieval or subscription module and the context correction module. Meanwhile, it provides various context application information for the context information response module and the required context information for the context information processing module.
[0081] The context information application module displays the output comprehensive information from the context information response module and the user feedback module on the interface of the context-aware system, and sends the comprehensive context information to the context information retrieval or subscription module.
[0082] The context information retrieval or subscription module searches corresponding context information in the knowledge base module according to the retrieval requirements of the context information application module. On the other hand, it sends the relevant subscription information to the context information correction module according to the subscription requirements of the context information application module.
[0083] The context information response module searches relevant context information in the knowledge base module according to the requirements from the context information application module, and then sends the required context information to the context information application module. [0084] The context information correction module corrects the context information sent by the context information retrieval or subscription module, and then sends the corrected context information to the knowledge base module.
[0085] The user feedback module stores the context information from users in certain environments into the knowledge base module, and provides the context information application module with the required application information.
[0086] The context information processing module comprises a context information modeling unit, a multi-algorithm incompleteness elimination unit, an inconsistency elimination of multi-algorithm results unit, a multi-source inconsistency elimination unit, a credibility management unit, a reliability management unit, a context information fusion reasoning unit and an adaptive management unit.
[0087] Wherein the context information acquisition module, the context information modeling unit, the multi-algorithm incompleteness elimination unit, the inconsistency elimination of multi-algorithm results unit, the credibility management unit, the reliability management unit, the multi-source inconsistency elimination unit and the context information fusion reasoning unit are connected in turn; the adaptive management unit is connected with the multi-algorithm incompleteness elimination unit; the credibility management unit, the multi-source inconsistency elimination unit and the context information fusion reasoning unit respectively; the multi-source inconsistency elimination unit is connected with the credibility management unit; the context information fusion reasoning unit is connected with the knowledge base module; the knowledge base module is connected with the adaptive management unit.
[0088] The context information modeling unit models the multi-source context information collected by the context information acquisition module according to the context information modeling method in the knowledge base module. In the present invention, the modeling mode is “type of context awareness + context-aware information + context-aware accuracy”, where the type of context awareness refers to the type of context-aware information, the context-aware information refers to the original context information collected by sensors, and the context-aware accuracy refers to the precision of the sensor that collects the current context-aware information. For example, for the context-aware information “bedroom” collected by the sensor of 90% precision, its model is “type of context awareness—user location” + “context-aware information—bedroom” + “context-aware accuracy — 90%”. After that, the context information modeling unit sends the modeled context information to the multi-algorithm incompleteness elimination unit.
[0089] The multi-algorithm incompleteness elimination unit simultaneously adopts multiple algorithms to eliminate incompleteness, and then sends the elimination results of these algorithms to the inconsistency elimination of multi-algorithm results unit, where the incompleteness refers to a lack of original context information collected by certain sensor and the multiple algorithms include neural network, D-S evidence theory, EM, voting algorithm and fuzzy set theory, etc.
[0090] The inconsistency elimination of multi-algorithm results unit firstly adopts the voting algorithm to solve inconsistency problem among the algorithm results from the multi-algorithm incompleteness elimination unit, gets complete context information with higher accuracy, and then sends the processed complete context information to the credibility management unit, where the inconsistency is that there exists inconsistency among the results of eliminating the incompleteness of context information by multiple algorithms.
[0091] The credibility management unit judges the accuracy of the processed complete context information according to the received user feedback information, where the user feedback information refers to the context information that users actively feed back based on their environments, and then calculates the credibility of the corresponding sources according to the following equation and stores them:
N
Credibility^) = —— (1)
Nolal where .v denotes the current context information source, Nmr denotes the number of correct context information, and Nlolal denotes the total number of context information collected by the current context information source.
[0092] The reliability management unit manages the context information sources based on their reliability, where the reliability is related to the sensor precision and the credibility of the source. The reliability can be calculated by „ , , , , „ Precisions Credibility!s)
Reliability^) = 2s------------------Precision + Credibility^) where Precision denotes the precision of the current used sensor and is static measurement value. Credibility (s) is dynamic evaluation value. Then the reliability values of these sources are sent to the multi-source inconsistency elimination unit.
[0093] The multi-source inconsistency elimination unit adopts D-S evidence theory based on reliability management to eliminate the inconsistency among multiple sources, where the inconsistency refers to the inconsistency among the context information collected by different sensors, for example, infrared sensor detects that the user is in the bedroom, but Zigbee detects that the user is in the living room at this time, it is obvious that the user is impossible to appear in two places at the same time, thus there exists inconsistency. Then the inconsistency elimination result is sent to the credibility management unit and the context information fusion reasoning unit, respectively.
[0094] The context information fusion reasoning unit uses ontology reasoning, rule-based reasoning, D-S evidence theory or Bayesian network to deduce high-level context information according to the results of inconsistency elimination and the historical information in the adaptive management unit, and the high-level context information is stored into the knowledge base module, where the historical context information refers to the rule engine and rule set based on rule system in the adaptive management unit, and the high-level context information refers to the context information which can be used directly by users or various devices after fusion and reasoning.
[0095] The adaptive management unit provides user feedback information and correct information for the multi-algorithm incompleteness elimination unit, provides user feedback information for the credibility management unit, and provides the rule engine and rule set based on rule system for the context information fusion reasoning unit. On the other hand, it can make some proper adjustments on various parameters according to current context information, so that the context-aware system is of better adaptability.
[0096] The context information acquisition module comprises multiple physical sensors, virtual sensors and logical sensors.
Embodiment 2 [0097] The working method of the above system described in the embodiment 1 is illustrated in FfG 2. Take a typical scenario of context-aware computing—smart home as an example, in the smart home scenario, the location information of the user can be collected by WiFi, Bluetooth, infrared sensor and Zigbee, which can be expressed respectively by I„Fl , IBhuXMh, Ilnfrared and IZighee the method ?
specifically comprises the following steps:
[0098] SOI: Collect context information and modeling [0099] Collect context information by multiple physical sensors, virtual sensors and logical sensors, and model the collected context information according to the modeling mode stored in the knowledge base, where the modeling mode is “type of context awareness + context-aware information + context-aware accuracy”.
[0100] in the embodiment, the modeled context information is as follows:
[0101] When it is at t{ time,
IWIFI = type of context awareness-user location + context- aware information- living room + context - aware accuracy - 90%,
IBluaMh = type of context awareness- user location + context - aware information- living room + context - aware accuracy - 92%,
Izighee = type of context awareness-user location + context -aware information- NaN(missing) + context - aware accuracy - 94%,
I,„fmred = type of context awareness-user location + context-aware information-bedroom + 'context - aware accuracy - 96% ; [0102] When it
ImF1 = type °f context awareness- user location context - aware accuracy - 90%, leiueiooih = type of context awareness- user location context - aware accuracy - 92%,
Is hee = type of context awareness-user location context - aware accuracy - 94%,
Ii„frared = type of context awareness- user location context - aware accuracy - 96% ; [0103] When it
I wifi = type of context awareness- user location context - aware accuracy - 90%, leiueiooih = type of context awareness- user location context - aware accuracy - 92%,
Is hee = type of context awareness-user location context - aware accuracy - 94%,
Ii„frared = type of context awareness- user location context - aware accuracy - 96% ; [0104] When it
Imfi = type of context awareness- user location context - aware accuracy - 90%, leiueiooih = type of context awareness- user location context - aware accuracy - 92%,
Is hee = type of context awareness-user location context - aware accuracy - 94%,
Ii„frared = type of context awareness- user location is at t2 time, + context - aware information- bedroom+ + context - aware information- bedroom+ + context - aware information- bedroom+ + context - aware information- bedroom+ is attime, + context - aware information- living room+ + context - aware information- outdoor+ + context - aware information- bedroom+ + context - aware information- NaN+ is at t4time, + context - aware information- bedroom+ + context - aware information- living room+ + context - aware information- bedroom+ + context - aware information- bedroom+ context - aware accuracy - 96%.
[0105] The time interval of each sensing device is fixed and appropriate, and the timeliness is high.
[0106] S02: Detect whether the context information is complete or not [0107] The location information at / time is “outdoor”, which belongs to inaccurate information, it can be seen as incomplete information and solved by using incompleteness elimination algorithms. Besides, the context information collected by WIFI is complete, so execute step S02’, but the context information collected by other sensors except WIFI is incomplete, so execute step S03.
[0108] S02’: Store relevant context information [0109] Store context information for the usage of the subsequent step S06 and step S09.
[0110] S03: Compute incompleteness rate [0111] Compute the incompleteness rate of the modeled context information by the following computation equation:
N .
Incompleteness rate = —(3) ^lolai where Nmis denotes the number of missing context information.
[0112] In the embodiment, the incompleteness rate of the four sensors can be computed by the equation (3) and they are ImF, = 10%, IBtumolh = 8%, IInfmred = 6% and IZighee = 4% , respectively.
[0113] S04: Judge whether the incompleteness is within a controllable range [0114] The incompleteness rate values of the four sensors are IWIFI =10% , IBiueirMh = 8% ,
I infrared = and Izigbee = , and the controllable range the system presets is 15%, so they are all within the controllable range, then execute step SOS’.
[0115] SOS’: Judge whether there exists user feedback [0116] Detect whether there exists user feedback at this time, if there exists, then execute step S06’, otherwise, execute step S06.
[0117] According to the current location information users actively feed back, the system adjusts the processing method of uncertain context information, which can make the system more stable and reliable.
[0118] S06: Eliminate the incompleteness using multiple algorithms [0119] The current user dose not feed back location information at this time, so multiple algorithms such as EM, voting algorithm, D-S evidence theory and fuzzy set theory are simultaneously adopted to eliminate the incompleteness of the context information from the horizontal and vertical directions.
[0120] S06’: Fill up the incomplete context information directly [0121] If there exists the user feedback information at this time, for example, the location information that the user feeds back at this time is bedroom, then “bedroom” is used to fill up the missing information and the context information will become complete. After that, it goes back to step S02’.
[0122] S07: Judge whether there exists inconsistency among the results of algorithms [0123] Judge whether there exists inconsistency among the results of incompleteness elimination of location information using multiple algorithms, the results are all “bedroom”, i.e., there exists no inconsistency, so go back to step S02’.
[0124] S08: Eliminate the inconsistency of the algorithm results [0125] If there exist inconsistency among the results of incompleteness elimination of location information using multiple algorithms, i.e., the result of neural network algorithm is “bedroom”, the result of EM algorithm is “living room”, the result of D-S evidence theory is “bedroom”, the result of fuzzy set theory is “living room” and the result of voting algorithm is “bedroom”, then the voting algorithm is adopted to eliminate the inconsistency of the algorithms results. After that, higher quality of complete context information can be obtained and the result after inconsistency elimination is used to fill up the positions of missing information, i.e., 1 Zigbee = living room at t{ time,
Uiaeiooib = living room and = bedroom at ( time.
[0126] S09: Judge whether the context information from multiple sensors is inconsistent [0127] In the embodiment, the context information collected at t3, t3 and t4 times is inconsistent, so execute step S10’.
[0128] S10: Compute the credibility of the source [0129] At t2 time, the context information is consistent, compute the credibility, then use the user feedback information to evaluate the credibility of corresponding source according to the equation (1). Note that, the accuracy of the location information is judged on the basis of the user feedback information.
[0130] S10’: Judge whether there exists user feedback [0131] Judge whether the user actively feeds back location information at this time, if there exists the user feedback information, then go back to step S10, otherwise, execute step SH.
[0132] SH: Eliminate inconsistency of context information [0133] In the embodiment, the context information collected at t{, t3 and t4 times is inconsistent, D-S evidence theory is adopted to eliminate context inconsistency.
[0134] S12: Compute reliability of the source [0135] According to the equation (1), when the credibility of Ii„fmred is 92%, the inherent precision of the sensor is 96%, so the reliability of Ii„fmred is
0.92x 0.96
Reliability = 2x-----------=93.96% . Then the reliability of the source is applied to context
0.92 + 0.96 inconsistency elimination of step SH so as to make the judge results more true and reliable.
[0136] FIG. 3 is simulation diagram of the performance of the existing context-aware systems; FIG. 4 is simulation diagram of an embodiment on the overall performance of the system proposed by the present invention. Comparing FIG. 3 and FIG 4, it can be seen that the system proposed by the present invention is more accurate and reliable than the existing context-aware system. When the user feedback rate is 0.1 and the number of context information N is 3000, the accuracy of the existing context-aware system is 97.25%, while that of the system proposed by the present invention is 99.34%.
[0137] The contents which are not illustrated in detail in the present invention belong to the well known technologies in the art.
[0138] Of course, the present invention can also provide a variety of embodiments. According to the disclosure of the present invention, those skilled in the art can make various corresponding changes and transformations without departing from the spirit and essence of the present invention. However, these corresponding changes and transformations shall all fall within the protection scope of the claims of the present invention.

Claims (4)

What is claimed is:
1. a context-aware system for uncertainty elimination based on reliability management, which comprises a context information acquisition module, a context information processing module, a knowledge base module, a context information response module, a context information application module, a context information retrieval or subscription module, a context information correction module and a user feedback module.
Wherein the context information acquisition module, the context information processing module and the knowledge base module are connected in turn; the knowledge base module, the context information application module, the context information retrieval or subscription module and the context information correction module are connected in turn from beginning to end; the knowledge base module, the context information response module and the context information application module are connected in turn; the user feedback module is connected with the knowledge base module and the context information application module, respectively.
The context information acquisition module periodically collects original context information by multiple physical sensors, virtual sensors and logical sensors, and then sends the collected original context information to the context information processing module, where the original context information is multi-source context information, i.e., the context information collected by multiple sensors. For example, the user’s location information can be collected by Bluetooth, WiFi, infrared sensor, Zigbee, etc.
The context information processing module processes the original context information from the context information acquisition module.
The knowledge base module stores user feedback information, context fusion reasoning information, context retrieval or subscription information and context application information, where context retrieval or subscription information is stored into the knowledge base module going through the context correction module, and context application information is stored into the knowledge base module going through the context retrieval or subscription module and the context correction module. Meanwhile, it provides various context application information for the context information response module and the required context information for the context information processing module.
The context information application module displays the output comprehensive information from the context information response module and the user feedback module on the interface of the context-aware system, and sends the comprehensive context information to the context information retrieval or subscription module.
The context information retrieval or subscription module searches corresponding context information in the knowledge base module according to the retrieval requirements of the context information application module. On the other hand, it sends the relevant subscription information to the context information correction module according to the subscription requirements of the context information application module.
The context information response module searches relevant context information in the knowledge base module according to the requirements from the context information application module, and then sends the required context information to the context information application module.
The context information correction module corrects the context information sent by the context information retrieval or subscription module, and then sends the corrected context information to the knowledge base module.
The user feedback module stores the context information from users in certain environments into the knowledge base module, and provides the context information application module with the required application information.
2. The context-aware system for uncertainty elimination based on reliability management according to claim 1, wherein the context information processing module comprises a context information modeling unit, a multi-algorithm incompleteness elimination unit, an inconsistency elimination of multi-algorithm results unit, a multi-source inconsistency elimination unit, a credibility management unit, a reliability management unit, a context information fusion reasoning unit and an adaptive management unit;
Wherein the context information acquisition module, the context information modeling unit, the multi-algorithm incompleteness elimination unit, the inconsistency elimination of multi-algorithm results unit, the credibility management unit, the reliability management unit, the multi-source inconsistency elimination unit and the context information fusion reasoning unit are connected in turn; the adaptive management unit is connected with the multi-algorithm incompleteness elimination unit; the credibility management unit, the multi-source inconsistency elimination unit and the context information fusion reasoning unit respectively; the multi-source inconsistency elimination unit is connected with the credibility management unit; the context information fusion reasoning unit is connected with the knowledge base module; the knowledge base module is connected with the adaptive management unit.
The context information modeling unit models the multi-source context information collected by the context information acquisition module according to the context information modeling method in the knowledge base module. In the present invention, the modeling mode is “type of context awareness + context-aware information + context-aware accuracy”, where the type of context awareness refers to the type of context-aware information, the context-aware information refers to the original context information collected by sensors, and the context-aware accuracy refers to the precision of the sensor that collects the current context-aware information. For example, for the context-aware information “bedroom” collected by the sensor of 90% precision, its model is “type of context awareness—user location” + “context-aware information—bedroom” + “context-aware accuracy — 90%”. After that, the context information modeling unit sends the modeled context information to the multi-algorithm incompleteness elimination unit.
The multi-algorithm incompleteness elimination unit simultaneously adopts multiple algorithms to eliminate incompleteness, and then sends the elimination results of these algorithms to the inconsistency elimination of multi-algorithm results unit, where the incompleteness refers to a lack of original context information collected by certain sensor and the multiple algorithms include neural network, D-S evidence theory, EM, voting algorithm and fuzzy set theory, etc.
The inconsistency elimination of multi-algorithm results unit firstly adopts the voting algorithm to solve inconsistency problem among the algorithm results from the multi-algorithm incompleteness elimination unit, gets complete context information with higher accuracy, and then sends the processed complete context information to the credibility management unit, where the inconsistency is that there exists inconsistency among the results of eliminating the incompleteness of context information by multiple algorithms.
The credibility management unit judges the accuracy of the processed complete context information according to the received user feedback information, where the user feedback information refers to the context information that users actively feed back based on their environments, and then calculates the credibility of the corresponding sources according to the following equation and stores them:
Credibility(s) = —— (1)
Nolal where s denotes the current context information source, Nmr denotes the number of correct context information, and Nlolal denotes the total number of context information collected by the current context information source.
The reliability management unit manages the context information sources based on their reliability, where the reliability is related to the sensor precision and the credibility of the source. The reliability can be calculated by „ , , , , „ Precisions Credibility(s)
Reliability^) = 2x-------------------C)
Precision + Credibility's) where Precision denotes the precision of the current used sensor and is static measurement value. Credibility (s) is dynamic evaluation value. Then the reliability values of these sources are sent to the multi-source inconsistency elimination unit.
The multi-source inconsistency elimination unit adopts D-S evidence theory based on reliability management to eliminate the inconsistency among multiple sources, where the inconsistency refers to the inconsistency among the context information collected by different sensors, for example, infrared sensor detects that the user is in the bedroom, but Zigbee detects that the user is in the living room at this time, it is obvious that the user is impossible to appear in two places at the same time, thus there exists inconsistency. Then the inconsistency elimination result is sent to the credibility management unit and the context information fusion reasoning unit, respectively.
The context information fusion reasoning unit uses ontology reasoning, rule-based reasoning, D-S evidence theory or Bayesian network to deduce high-level context information according to the results of inconsistency elimination and the historical information in the adaptive management unit, and the high-level context information is stored into the knowledge base module, where the historical context information refers to the rule engine and rule set based on rule system in the adaptive management unit, and the high-level context information refers to the context information which can be used directly by users or various devices after fusion and reasoning.
The adaptive management unit provides user feedback information and correct information for the multi-algorithm incompleteness elimination unit, provides user feedback information for the credibility management unit, and provides the rule engine and rule set based on rule system for the context information fusion reasoning unit. On the other hand, it can make some proper adjustments on various parameters according to current context information, so that the context-aware system is of better adaptability.
3. The context-aware system for uncertainty elimination based on reliability management according to claim 1, wherein the context information acquisition module comprises multiple physical sensors, virtual sensors and logical sensors.
4. The method for for uncertainty elimination based on reliability management according to claim 2, which comprises the following steps:
SOI: Collect context information and modeling
Collect context information by multiple physical sensors, virtual sensors and logical sensors, and model the collected context information according to the modeling mode stored in the knowledge base, where the modeling mode is “type of context awareness + context-aware information + context-aware accuracy”.
S02: Detect whether the context information is complete or not
Detect whether the modeled context information is complete or not, i.e., whether there exists missing context information, if not, then execute step S02’, otherwise, execute step S03.
S02’: Store relevant context information
Store context information for the usage of the subsequent step S06 and step S09.
S03: Compute incompleteness rate
Compute the incompleteness rate of the modeled context information by the following computation equation:
N .
Incompleteness rate = —(3) trivial where Nmis denotes the number of missing context information.
S04: Judge whether the incompleteness is within a controllable range
Judge whether the incompleteness is within a controllable range, if it is, then execute step SOS’, otherwise, execute step SOS. For example, if the controllable range of the incompleteness is set to be 15%, when the incompleteness rate is 10%, which is less than the controllable range, i.e., within the controllable range.
SOS: Delete the context information out of the controllable range
When the incompleteness of the context information is out of the controllable range, delete the context information directly.
SOS’: Judge whether there exists user feedback
Detect whether there exists user feedback at this time, if it exists, then execute step S06’, otherwise, execute step S06.
S06: Eliminate the incompleteness using multiple algorithms
Adopt simultaneously multiple algorithms to eliminate the incompleteness of the context information from the horizontal and vertical directions, where the horizontal direction refers to different times of the same sensor, the vertical direction refers to the same time of different sensors, and the multiple algorithms include neural network, EM, voting algorithm, D-S evidence theory and fuzzy set theory algorithm, etc.
S06’: Fill up the incomplete context information directly
Fill up the incomplete context information according to the user feedback information, then get the complete context information, and go back to step S02’.
S07: Judge whether there exists inconsistency among the results of algorithms
Judge whether there exists inconsistency among the results of incompleteness elimination using multiple algorithms, if there exists, then execute step S08, otherwise, go back to step S02’.
S08: Eliminate the inconsistency of the algorithm results
Adopt the voting algorithm to eliminate the inconsistency of the algorithm results, and get higher quality of complete context information.
S09: Judge whether the context information from multiple sensors is inconsistent
Judge whether the context information collected from multiple sensors is inconsistent, if it is, then execute step S10’, otherwise, execute step S10;
S10: Compute the credibility of the source
Compute the credibility of the source according to the equation (1), and then execute step S12, where the accuracy of context information is judged based on the user feedback information, and the credibility of the source is 1 before the context inconsistency occurs.
S10’: Judge whether there exists user feedback
Judge whether there exists user feedback at this time, if there exists, then go back to step S10, otherwise, execute step SH.
SH: Eliminate inconsistency of multi-source context information
Adopt D-S evidence theory to eliminate context inconsistency.
S12: Compute the reliability of the source
Compute reliability on the basis of credibility parameter and sensor precision according to equation (2), and then apply the reliability of the source to context inconsistency elimination of step SH, which can make the judgment result more true and reliable.
Suppose that Θ is a frame of discernment. If for any subset A that belongs to Θ , it satisfies ηι(φ)=0 and Σ m(A) = 1, then m is called the basic belief assignment (BPA) on the Θ, m(A) is the basic belief value of A , and it reflects the reliability of A .
Assume that if and if> are two BPAs based on the same Θ , whose focal elements are respectively Al,A2,...,Ak and Bl,B2,...,Bk . and then the Dempster's Combination Rule (DCR) is as follows:
Σ MxJM·5/) m(C\ = ^L--------- (4) V ’ \-K where m(C) reflects the joint support degree of two evidences that if and m2 corresponds.
B ~ is the conflicting degree of evidence, when K = 0, it is no conflict at all;
when 0 < K < 1, it is incomplete conflict; when K = 1, it is complete conflict. The DCR satisfies the exchange law and the combination law, and the combination of multiple evidences can be obtained by repeatedly using the equation (4).
AU2017386444A 2016-12-30 2017-09-30 Reliability management-based uncertainty elimination context awareness system and working method thereof Ceased AU2017386444B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201611251838.3A CN106650941B (en) 2016-12-30 2016-12-30 A kind of uncertain elimination context aware system and its working method based on reliability management
CN201611251838.3 2016-12-30
PCT/CN2017/104792 WO2018120962A1 (en) 2016-12-30 2017-09-30 Reliability management-based uncertainty elimination context awareness system and working method thereof

Publications (2)

Publication Number Publication Date
AU2017386444A1 true AU2017386444A1 (en) 2019-06-27
AU2017386444B2 AU2017386444B2 (en) 2021-03-18

Family

ID=58836846

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2017386444A Ceased AU2017386444B2 (en) 2016-12-30 2017-09-30 Reliability management-based uncertainty elimination context awareness system and working method thereof

Country Status (3)

Country Link
CN (1) CN106650941B (en)
AU (1) AU2017386444B2 (en)
WO (1) WO2018120962A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106650941B (en) * 2016-12-30 2019-01-25 山东大学 A kind of uncertain elimination context aware system and its working method based on reliability management
CN107705037B (en) * 2017-10-26 2021-03-19 山东大学 QoX quality system facing context information processing and working method thereof
CN108846481B (en) * 2018-06-25 2021-08-27 山东大学 Situation information uncertainty elimination system based on QoX self-adaptive management and working method thereof
CN109982287B (en) * 2019-03-13 2021-10-15 北京工业大学 Forestry multi-sensor fire alarm system based on zigbee wireless sensor network
CN111062427B (en) * 2019-12-11 2023-04-18 山东大学 Multi-criterion decision-making multi-mode scene information uncertainty processing method and system
CN112672299B (en) * 2020-12-09 2022-05-03 电子科技大学 Sensor data reliability evaluation method based on multi-source heterogeneous information fusion
CN113533962B (en) * 2021-07-29 2022-08-12 上海交通大学 Induction motor health diagnosis system based on decision fusion of multiple physical signal sensors

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FI20095570L (en) * 2009-05-22 2009-09-11 Valtion Teknillinen Context recognition in mobile devices
CN103246819B (en) * 2013-05-20 2016-10-05 山东大学 A kind of discordance context towards general fit calculation eliminates system and method
CN104346451A (en) * 2014-10-29 2015-02-11 山东大学 Situation awareness system based on user feedback, as well as operating method and application thereof
CN105389405B (en) * 2015-12-30 2018-11-30 山东大学 A kind of more inference engine integrating context sensory perceptual systems and its working method
CN105654134B (en) * 2015-12-30 2018-11-23 山东大学 It is a kind of based on the context aware system and its working method and application that have supervision self feed back
CN106650941B (en) * 2016-12-30 2019-01-25 山东大学 A kind of uncertain elimination context aware system and its working method based on reliability management

Also Published As

Publication number Publication date
CN106650941B (en) 2019-01-25
WO2018120962A1 (en) 2018-07-05
AU2017386444B2 (en) 2021-03-18
CN106650941A (en) 2017-05-10

Similar Documents

Publication Publication Date Title
AU2017386444A1 (en) Reliability management-based uncertainty elimination context awareness system and working method thereof
US10445527B2 (en) Differential privacy and outlier detection within a non-interactive model
US10958749B2 (en) Method and device for pushing application message
US10527309B2 (en) Building air conditioning control system and control method thereof
US9432332B2 (en) Proxy device for a network of devices
US20210365811A1 (en) Context information uncertainty elimination system and working method based on qox adaptive management
US9294509B2 (en) Hierarchical management of devices
US10652209B2 (en) Router address type identification method and apparatus
CN107486863B (en) Robot active interaction method based on perception
US20160380856A1 (en) Systems and methods for tracking devices status and malfunctions in machine-to-machine networks
EP4114044A1 (en) Electronic device positioning method and apparatus, and server, system and storage medium
WO2021035645A1 (en) Sensing system, sensing device and control method therefor, movable platform, and storage medium
CN108234493A (en) The space-time crowdsourcing statistical data dissemination method of secret protection under insincere server
CN104504868A (en) Device and method for monitoring bad habits of terminal device user
WO2021205234A1 (en) System for calculating highly accurate gps location for mobile devices
CN105654134B (en) It is a kind of based on the context aware system and its working method and application that have supervision self feed back
EP3244586B1 (en) Response message processing method, device and system
CA3187865A1 (en) Localization system and method
US20200018506A1 (en) System and method for dual occupancy detection
CN111461198A (en) Action determination method, system and device
Sun et al. Edge computing in Internet of things: a novel sensing-data reconstruction algorithm under intelligent-migratoin stragegy
WO2020001095A1 (en) Context uncertainty elimination system based on hierarchical comprehensive quality index qox, and working method therefor
EP4035045A1 (en) Maintaining privacy during attribution of a condition
Bruza et al. Modelling the information seeking user by the decision they make
KR20140103658A (en) A method and apparatus of transmitting and receiving notification applying filter criteria of subscriber in M2M environment

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)
MK14 Patent ceased section 143(a) (annual fees not paid) or expired