EP2997526A2 - Système d'aide à la décision multicritère avec génération automatique d'explications, et procédé correspondant - Google Patents
Système d'aide à la décision multicritère avec génération automatique d'explications, et procédé correspondantInfo
- Publication number
- EP2997526A2 EP2997526A2 EP14726924.5A EP14726924A EP2997526A2 EP 2997526 A2 EP2997526 A2 EP 2997526A2 EP 14726924 A EP14726924 A EP 14726924A EP 2997526 A2 EP2997526 A2 EP 2997526A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- criteria
- alternative
- score
- given
- contributions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 238000000034 method Methods 0.000 title claims description 33
- 230000003993 interaction Effects 0.000 claims abstract description 93
- 238000011156 evaluation Methods 0.000 claims abstract description 35
- 239000000654 additive Substances 0.000 claims description 7
- 230000000295 complement effect Effects 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 description 49
- 230000002776 aggregation Effects 0.000 description 48
- 230000006870 function Effects 0.000 description 10
- 238000002360 preparation method Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000000996 additive effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000010791 quenching Methods 0.000 description 1
- 239000010981 turquoise Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/045—Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
Definitions
- the present invention relates to a multicriterion decision support system, and a corresponding method. It finds an application in particular to the evaluation of alternatives relating to a decision to be made in a multicriteria-type context, for example when it comes to choosing the location of a plant, an investment strategy, to evaluate projects, etc.
- Multi-criteria decision support systems and processes are based on a multi-criteria decision-making model that is developed from expert knowledge of the relevant application domain.
- multi-criteria decision support systems make it possible to evaluate the alternatives available to a decision-maker, and thus to make a decision quickly which can be complex, in particular because it involves many criteria, depending on the calculated scores for different alternatives.
- a decision support system that only evaluates alternatives is not enough to meet such a need, because it does not explain the origin of a score of an alternative.
- This process generates explanations based on the criteria or aggregations of a given level in the model, using a method using four anchors for the generation of explanations. These anchors are not related to the contribution of the different elements in the calculation of the difference between the scores of two compared alternatives, or in the calculation of the evaluation of an alternative.
- One of the aims of the invention is therefore to solve the aforementioned problems, in particular to propose a system and a method for generating explanations on the score of an alternative, or on the comparison of an alternative with respect to another , based on the contributions of the different elements influencing this score, regardless of the level of these elements in the multi-criteria and multi-level decision-making model.
- the subject of the invention is a multicriterion decision support system comprising a knowledge base, an alternative evaluation module, an explanations generating module, and a module determination of contributions.
- the knowledge base forms a multi-criteria decision model.
- criteria relating to a given decision-making, possible values associated with all or part of the criteria, criteria weights associated with all or part of the criteria, and interaction weights associated with the interactions between criteria, a given combination of possible values of at least a part of the criteria forming an alternative.
- the alternative evaluation module is able to determine a score for a given alternative, depending on the criteria weights, and / or interaction weights, and / or values.
- the explanation generation module is able to construct explanations on the score of a given alternative in an absolute manner or in relation to the score of one or more other alternatives.
- Criteria are stored in the knowledge base in a multi-level hierarchical manner, with groups of terminal criteria grouped at the terminal level under higher level aggregated criteria, the aggregated criteria being themselves grouped at the top level under other criteria. aggregate criteria of even higher level.
- the contribution determination module is able to determine, for criteria belonging to several levels of the multicriterion decision model, the contributions of the criteria weights, and / or the interaction weights and / or values, in a score evaluated by the alternative evaluation module for a given alternative.
- the explanation generation module is capable of constructing explanations on a score of a given alternative, in an absolute manner or in relation to the score of one or more other alternatives, from contributions determined by the module of determination of contributions.
- system further comprises one or more of the following characteristics, taken individually or in any technically possible combination:
- the alternative evaluation module uses an integral type cost function of Choquet 2-additive or weighted sum type to determine a score for a given alternative;
- the contribution determination module is able to determine the contributions of the weights and / or interaction weights associated with the higher level aggregated criteria in the multicriteria decision model;
- the contribution determination module is able to determine the contributions of the weights and / or the interaction weights associated with the end criteria in the multicriteria decision model;
- the contribution determination module is able to determine the contributions of the values associated with the criteria
- the contribution determination module is able to carry out a normalization of the determined contributions, preferably between the values 0 and 1;
- the contribution determination module is able to determine the positive contributions of the criteria weights, and / or interaction weights and / or values, in a score evaluated by the alternative evaluation module for a given alternative ;
- the contribution determination module is capable of determining the negative contributions, below a given threshold, of the criteria weights, and / or the interaction weights and / or values, in a score evaluated by the evaluation module alternatives for a given alternative;
- the explanation generation module is able to generate an explanation of the score of the alternative given with respect to the score of another alternative, the explanation presenting one or more contributions that contribute to the superiority of the score of the alternative; one of the two alternatives on the score of the other;
- the explanation generation module is capable of generating an explanation of the score of the alternative given with respect to the score of another alternative, the explanation presenting one or more contributions which contribute to the inferiority of the score of one of two alternatives on the score of the other;
- the possible scores of alternatives, the possible values of criteria, the possible values of weight and the possible values of interaction weight are grouped by interval, a level value being associated with each interval, and the generation module of explanation is able to generate an explanation presenting one or more contributions of a given criterion weight, or a given interaction weight, or a given criterion value, the explanation using the level value associated with the criterion interval to which the given weight of criteria belongs, or the given interaction weight, or the given criterion value;
- the system comprises a base of patterns in which at least one pattern containing one or more variables corresponding to an alternative name, and / or a value of criterion weight level and / or interaction weight between criterion, and / or an alternative value, and the explanation generation module is able to generate an explanation using the pattern;
- the explanation generation module is capable of generating an explanation by replacing the variables contained in the pattern by the values corresponding to the alternative name, and / or the value of the weight level and / or the interaction weight. , and / or the value of alternative, relating to a given alternative;
- the explanation generation module is suitable, when an explanation concerns the contribution of an interaction weight associated with two criteria presenting two values whose difference is smaller than a given threshold, to generate a complementary explanation on the discontinuity the explanation according to the variations of the values;
- the explanation generation module is capable of constructing explanations of the score of the alternative given with respect to a higher score corresponding to a fictitious alternative and / or compared to a lower score corresponding to a fictitious alternative.
- the subject of the invention is also, according to a second aspect, a decision support method comprising a step of constructing a multicriterion decision model, a step of evaluating an alternative by means of a module of evaluating alternatives, a step of generating explanations by means of an explanation generating module, and a step of determining contributions by means of a contribution determining module.
- the multicriteria decision-making model includes criteria concerning a given decision-making, possible values associated with all or some of the criteria, criteria weights associated with all or some of the criteria, and interaction weights associated with the interactions between criteria, a combination data of possible values of at least part of the criteria forming an alternative, the multi-criteria decision model being stored in a knowledge base.
- the step of evaluating an alternative determines a score for a given alternative, based on criteria weights, and / or interaction weights, and / or values.
- the explanation generation step constructs explanations of the score of the given alternative, either absolutely or in relation to the score of one or more other alternatives.
- the criteria are organized in the knowledge base in a hierarchical manner on a terminal level and at least a higher level, with groups of terminal criteria grouped at the terminal level under higher level aggregated criteria, the aggregated criteria being able to be themselves grouped at the top level under other aggregate criteria of even higher level.
- the contribution determination step determines, for criteria belonging to several levels of the multicriterion decision model, the contributions of the criteria weights, and / or the interaction weights and / or values, in the score evaluated during the evaluation. step of evaluating alternatives for the given alternative.
- the explanation generating step constructs the explanations of the score of the given alternative, in an absolute manner or with respect to the score of one or more other alternatives, from the contributions determined during the step of determination of contributions.
- the method further comprises one or more of the following characteristics, taken in isolation or in any technically possible combination:
- the evaluation step uses an integral type cost function of Choquet 2- additive or weighted sum type; the contribution determination step determines the contributions of the weights and / or interaction weights associated with the higher level aggregated criteria in the multicriteria decision model;
- the contribution determination step determines the contributions of the weights and / or interaction weights associated with the end criteria in the multicriteria decision model
- the contribution determination stage determines the contributions of the values associated with the criteria
- the step of determining the contributions performs a normalization of the determined contributions, preferably between the values 0 and 1;
- the contribution determination step determines the positive contributions of the criteria weights, and / or the interaction weights and / or values, in the evaluated score during the alternatives evaluation step for the alternative given;
- the contribution determining step determines the negative contributions, below a given threshold, of the criteria weights, and / or interaction weights and / or values, in the score evaluated during the evaluation step alternatives for the given alternative;
- the explanation generation step generates an explanation of the score of the alternative given with respect to the score of another alternative, the explanation presenting one or more contributions that contribute to the superiority of the score of one of the two alternatives on the score of the other;
- the explanation concerns the contribution of a weight that is not a minimum weight, or an interaction weight that is not a minimum interaction weight, or a value that is not a minimum value, associated with a criterion;
- the explanation generation step generates an explanation of the score of the alternative given with respect to the score of another alternative, the explanation presenting one or more contributions that contribute to the inferiority of the score of one two alternatives on the score of the other;
- the explanation presents the contribution of a weight that is a minimum weight, or an interaction weight that is a minimum interaction weight, or a value that is a minimum value, associated with a criterion;
- the possible scores of alternatives, the possible values of criteria, the possible values of weight and the possible values of interaction weight are grouped by interval, a level value is associated with each interval, and the generation step explanation generates an explanation presenting one or more contributions of a given criterion weight, or a given interaction weight, or a given criterion value, the explanation using the level value associated with the interval to which belongs the given weight of criteria, or the given interaction weight, or the given criterion value;
- the explanation generating step generates an explanation by using at least one pattern containing one or more variables corresponding to an alternative name, and / or a criterion weight level value and / or a weight value of interaction between criterion, and / or an alternative value, the patterns and variables being stored in a pattern database;
- the explanation generation step generates an explanation by replacing the variables contained in the pattern by the values corresponding to the alternative name, and / or the value of the weight level and / or the interaction weight, and / or the value of alternative, relating to a given alternative;
- the explanation generating step when an explanation concerns the contribution of an interaction weight associated with two criteria having two values whose difference is smaller than a given threshold, the explanation generating step generates a complementary explanation on the discontinuity of the explanation according to the variations of the values;
- the explanation generation step constructs explanations on the score of the given alternative with respect to a higher score corresponding to a fictitious alternative and / or compared to a lower score corresponding to a fictitious alternative.
- the system and the method of the invention make it possible to identify the strongest contributions and, secondly, to exploit them to generate explanations, the all without the need for language knowledge specific to one particular area or another.
- FIG. 1 schematic representation of a first example of multi-criteria multi-criteria decision model
- FIG. 2 schematic representation of a second example of multi-criteria multi-criteria decision model
- FIG. 3 schematic representation of a third example of multi-criteria multi-criteria decision model
- FIG. 4 schematic representation of an exemplary system according to the invention.
- Figures 1 to 3 show three different examples of applications in which a multilevel multi-criteria decision model can be constructed.
- the schematic representation takes the form of a tree structure in which are organized the various criteria making it possible to model the making of a decision.
- criteria C are defined at the terminal level. Such criteria C have no children in the tree. Symbolically, we attach to these terminal criteria C universes U, which may or may not have the same name as the criteria C.
- These criteria C are grouped by aggregation under higher level criteria, which are called aggregations A. These aggregations A can also be grouped by aggregation under even higher level criteria A, and so on.
- Figure 1 is for a site protection assessment model
- Figure 2 is for a home purchase model
- Figure 3 is for a model for the assessment of a site protection. 'a remote meeting.
- the model is stored in a knowledge base 1.
- This knowledge base 1 is broken down into a base 1b in which the elements defining the model strictly speaking are stored: the criteria and aggregations A, C (with their hierarchical organization), the values v of these criteria A, C, the weights a k of these criteria A, C as well as the weights b jk relating to the interactions between criteria A, C.
- the other part 1 of the knowledge base 1 includes the description of the alternatives E, W which correspond to different possible decisions.
- a combination of value v of end criteria C determines a possible alternative.
- Values v can be text values.
- This model distinguishes at the first level (just below the main aggregation "Remote Meeting Evaluation") the functional aspects, which are related to management during the meeting and after the meeting, of non-functional aspects, which are related to the security and the type of terminals that can be used.
- An apprenticeship was used to inform the different weights (including the interaction weights) of the model, which are not shown in Figure 3.
- an alternative on a given aggregation A corresponds to the result given by a cost function, such as a Choquet integral or a weighted sum, on the criteria A or C aggregated under this aggregation A.
- a cost function such as a Choquet integral or a weighted sum
- a 2-additive Choquet integral is used, which makes it possible to take into account the possible interactions between criteria.
- H (x 1; ..., x n ) ⁇ v k X x k " ⁇ ⁇ ], k ⁇ £ M lj, k X
- - Xi, x n are the n aggregated A or C criteria or sub-aggregations under an aggregation A on which the alternative E is evaluated;
- - M represents all the criteria at the level considered.
- the local coefficient in favor of an alternative corresponds to all the weights that highlight a force of the alternative.
- the local coefficient to the detriment of an alternative corresponds, for its part, to the set of "min", ie the set of minimum weights that highlight a weakness of the alternative rather than a force.
- the Choquet integral is applied at each level to obtain local satisfactions or scores on each aggregation.
- the scores of the alternatives thus evaluated are stored in a storage area 7, and a first type of explanations EXP can be generated automatically by an explanation generation module 3, for example:
- a contribution determination module 4 also makes it possible to determine the contributions Cb of the criteria weights a k , and / or the interaction weights b j! K and / or the values v, in a score evaluated by the module of evaluation of alternatives 2 for a given alternative E, W.
- weights and interactions are on the same scale, while the values on the criteria and aggregations are comparable.
- weights and interactions are generally higher for higher-level A aggregations than for C-criteria, especially for models with more than three levels.
- the first calculation corresponds to a global perspective. It focuses only on the higher level, and allows to estimate the contribution of the weights and interactions of this level.
- the second calculation corresponds to a "model" perspective and makes it possible to evaluate the contribution of each criterion and aggregation of the model.
- the third calculation corresponds to a detailed perspective. It focuses on the weights associated with the criteria: interactions, minimum weight and local coefficients.
- the contribution determination module 4 makes it possible to identify the elements having the greatest contribution, in the case of the comparison between two alternatives E, W, and in the case of the evaluation of an alternative.
- a 1 corresponds to the set of parent aggregations of the criterion i, ⁇ ⁇ , ⁇ ⁇ 0 ( ⁇ ) (respectively ⁇ ⁇ , ⁇ ⁇ 0 ( ⁇ )) is the local coefficient of the criterion (or aggregation) i 'in an integral of Choquet (or in a weighted sum) in favor of the alternative X
- the weight contribution of a criterion, or terminal weight is therefore a function of the weights on the parent aggregations.
- a is the minimum weight on criterion i, ci, j is a substitutability (or negative synergy) between criteria i and j
- H (X) - H (Y) ⁇ k e N, k ⁇ i ( ⁇ k) e A k k (x) xc k (X) ⁇ Where k - ⁇ k .e A k k ( Y) ⁇ c k (Y) ⁇ k )
- contribution calculation Cb the result is normalized between 0 and 1 to obtain a normalized contribution Cn.
- the normalization can constitute to carry out the following operation:
- i is a criterion or a combination of elements
- N is the set of criteria or combinations of elements.
- the standardized contribution thus obtained is expressed as a percentage, which is simpler to understand, and which retains the sign of the corresponding gross contribution.
- the filtering and this grouping are performed by a filtering module 6.
- weight is considered very important. The value of "E” here is very good while the value of "W” is bad here.
- Hole sentences or patterns, stored as sentences and variables in a pattern database 5 are used.
- a complementary explanation can be added to the explanation associated with these criteria or aggregations. This is an explanation that indicates to the user that the contribution of the element in question may change if the values close to the alternative on one criterion and on another criterion are reversed because of their interaction.
- the representation of the system architecture as given in the example of FIG. 4 is a functional representation, which does not necessarily prejudge the exact organization of the different functional elements (separate, shared, remote, etc.). .) included in the system.
- the calculations for determining the contributions Cb given in the present description are examples of nonlimiting calculations of the invention, which notably make it possible to take into account the multi-criteria and multi-level nature of a decision model. Other types of calculations can be used, which allow to take into account this character.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Evolutionary Computation (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Economics (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Tourism & Hospitality (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1301148A FR3005769A1 (fr) | 2013-05-17 | 2013-05-17 | Systeme d'aide a la decision multicritere avec generation automatique d'explications et procede correspondant |
PCT/EP2014/060170 WO2014184381A2 (fr) | 2013-05-17 | 2014-05-16 | Système d'aide à la décision multicritère avec génération automatique d'explications, et procédé correspondant |
Publications (1)
Publication Number | Publication Date |
---|---|
EP2997526A2 true EP2997526A2 (fr) | 2016-03-23 |
Family
ID=49474456
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14726924.5A Ceased EP2997526A2 (fr) | 2013-05-17 | 2014-05-16 | Système d'aide à la décision multicritère avec génération automatique d'explications, et procédé correspondant |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP2997526A2 (fr) |
CA (1) | CA2912751A1 (fr) |
FR (1) | FR3005769A1 (fr) |
WO (1) | WO2014184381A2 (fr) |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10127240B2 (en) | 2014-10-17 | 2018-11-13 | Zestfinance, Inc. | API for implementing scoring functions |
US20180322406A1 (en) * | 2017-05-04 | 2018-11-08 | Zestfinance, Inc. | Systems and methods for providing machine learning model explainability information |
US11941650B2 (en) | 2017-08-02 | 2024-03-26 | Zestfinance, Inc. | Explainable machine learning financial credit approval model for protected classes of borrowers |
US11960981B2 (en) | 2018-03-09 | 2024-04-16 | Zestfinance, Inc. | Systems and methods for providing machine learning model evaluation by using decomposition |
WO2019212857A1 (fr) | 2018-05-04 | 2019-11-07 | Zestfinance, Inc. | Systèmes et procédés pour enrichir des outils de modélisation et une infrastructure de modélisation en éléments sémantiques |
US11816541B2 (en) | 2019-02-15 | 2023-11-14 | Zestfinance, Inc. | Systems and methods for decomposition of differentiable and non-differentiable models |
US10977729B2 (en) | 2019-03-18 | 2021-04-13 | Zestfinance, Inc. | Systems and methods for model fairness |
US11321638B2 (en) | 2020-03-16 | 2022-05-03 | Kyndryl, Inc. | Interoperable smart AI enabled evaluation of models |
US11720962B2 (en) | 2020-11-24 | 2023-08-08 | Zestfinance, Inc. | Systems and methods for generating gradient-boosted models with improved fairness |
-
2013
- 2013-05-17 FR FR1301148A patent/FR3005769A1/fr active Pending
-
2014
- 2014-05-16 EP EP14726924.5A patent/EP2997526A2/fr not_active Ceased
- 2014-05-16 CA CA2912751A patent/CA2912751A1/fr active Pending
- 2014-05-16 WO PCT/EP2014/060170 patent/WO2014184381A2/fr active Application Filing
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
WO2014184381A2 (fr) | 2014-11-20 |
CA2912751A1 (fr) | 2014-11-20 |
WO2014184381A3 (fr) | 2015-02-19 |
FR3005769A1 (fr) | 2014-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2014184381A2 (fr) | Système d'aide à la décision multicritère avec génération automatique d'explications, et procédé correspondant | |
Akamani et al. | Toward the adaptive governance of transboundary water resources | |
Chrun et al. | Corporate environmentalism: Motivations and mechanisms | |
Ackers | Corporate social responsibility assurance: how do South African publicly listed companies compare? | |
Nutley et al. | Policy transfer and convergence within the UK: the case of local government performance improvement regimes | |
GB2418499A (en) | Information analysis arrangement | |
Turcan et al. | Empirical studies on legitimation strategies: A case for international business research extension | |
Vieira et al. | From carbon dependence to renewables: The European oil majors' strategies to face climate change | |
Tse et al. | Fast‐expanding markets: The revolution of the microeconomy | |
Pascoe et al. | Does membership matter? Individual influences in natural resource management decision making | |
Tan-Mullins | Dancing to China's tune: Understanding the impacts of a rising China through the political-ecology framework | |
Zumofen | Public accountability: A summary analysis | |
Khodadadi | The impact of knowledge management on social responsibility at social security organization in East Azarbaijan province | |
Engel et al. | Accommodating multiple interests in local forest management: a focus on facilitation, actors and practices | |
Brown | Investigating the impact of the external environment on strategic information systems planning: a qualitative inquiry | |
Park | The future journey of international journal of multinational corporation strategy | |
Hansen‐Magnusson et al. | Studying contemporary constitutionalism: Memory, myth and horizon | |
Ogiri et al. | Motivations of legitimacy theory for CSR reporting in the Niger Delta region of Nigeria: a theoretical framework | |
Brooks et al. | Walking a fine line: examining the position of the construction quality auditor through the lens of financial audit literature | |
Henike | Imitate or Deviate? How Cognitive Safety Impacts Entrepreneurial Business Modelling | |
Burke | Getting your point across: The WWF communication style | |
CN109800289B (zh) | 鉴定网络用户的方法及系统、网络信息的屏蔽方法及系统 | |
Åkvik et al. | A case study on Business model innovation to empower sustainable development in the oil and gas supplier industry. | |
Keenan et al. | Tokyo's booms and busts: Placing Japan in the global financial network | |
Laemmle | Monumentally Inadequate: Conservation at Any Cost Under the Antiquities Act |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20151117 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20161019 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
APBK | Appeal reference recorded |
Free format text: ORIGINAL CODE: EPIDOSNREFNE |
|
APBN | Date of receipt of notice of appeal recorded |
Free format text: ORIGINAL CODE: EPIDOSNNOA2E |
|
APBR | Date of receipt of statement of grounds of appeal recorded |
Free format text: ORIGINAL CODE: EPIDOSNNOA3E |
|
APAF | Appeal reference modified |
Free format text: ORIGINAL CODE: EPIDOSCREFNE |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R003 |
|
APBT | Appeal procedure closed |
Free format text: ORIGINAL CODE: EPIDOSNNOA9E |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20240212 |