WO2001027764A1 - Tolerance aux failles de logiciel de programmes concurrents par reexecution commandee - Google Patents
Tolerance aux failles de logiciel de programmes concurrents par reexecution commandee Download PDFInfo
- Publication number
- WO2001027764A1 WO2001027764A1 PCT/US2000/028568 US0028568W WO0127764A1 WO 2001027764 A1 WO2001027764 A1 WO 2001027764A1 US 0028568 W US0028568 W US 0028568W WO 0127764 A1 WO0127764 A1 WO 0127764A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- execution
- concurrent
- programs
- critical
- concurrent programs
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/366—Software debugging using diagnostics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3632—Software debugging of specific synchronisation aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3636—Software debugging by tracing the execution of the program
Definitions
- the present invention is generally directed to methods for correcting synchronization faults in concurrently executed computer programs and, more particularly, to methods and systems for fault tolerance of concurrently executed software programs using controlled re-execution of the programs.
- Concurrent programs include all parallel programming paradigms such as multi-threaded programs, shared-memory parallel programs, message-passing distributed programs, distributed shared-memory programs, etc.
- a parallel entity may be referred to as a process, although in practice it may also be a thread.
- transient software failures motivated a new approach to software fault tolerance based on rolling back the processes to a previous state and then restarting them (possibly with message reordering) , in the hope that the transient failure will not recur in the new execution.
- Methods based on this approach have mostly relied on chance in order to recover from a transient software failure. In the special case of synchronization faults, however, it is desirable to do better.
- the present invention controls the re-execution of concurrent programs in order to avoid a recurrence of the synchronization failure.
- the invention provides a method of (i) tracing an execution, (ii) detecting a synchronization failure, (iii) determining a control strategy, and (iv) re-executing under control.
- Control is achieved by tracing information during an execution and using this information to add synchronizations during the re- execution.
- a method of providing fault tolerance in concurrently executing computer programs by controlling the re-execution of concurrent programs in order to avoid a recurrence of synchronization failures comprising:
- application of a control strategy includes causing a re-execution of said concurrent programs under a control derived from tracing information during an execution, and wherein said control includes using said information to add synchronizations to said concurrent programs during re-execution.
- Figure 1 is an illustration of a tracing and controlling during rollback recovery
- Figure 2 is an illustration of variants of Off-line exclusions
- FIG 3 is an illustration of the algorithm used for Off-line independent read-write mutual exclusion
- Figure 4 is an flow diagram showing the main steps of the invention.
- Figure 5 is an illustration of tolerating races in a concurrent execution.
- the present invention controls the re-execution of concurrent programs, based on information traced during a failed execution.
- the following description will describe the management of races, tracing, failure detection, and re-execution under control in great depth.
- FIG. 1 illustrated is an example of tracing and controlling during rollback recovery.
- CS1-CS4 are the critical sections of the execution.
- the synchronizations between processes are shown as arrows from one process execution to another.
- a synchronization ensures that the execution after the head of the arrow can proceed only after the execution before the tail has completed.
- a race occurs when two critical sections execute simultaneously.
- CS1 and CS2 may have a race, since the synchronizations do not prevent them from executing simultaneously.
- a control strategy is a set of added synchroniza tions that would ensure that a race does not occur. The race can be avoided by adding synchronizations, shown as broken arrows in Figure 1(b).
- the present invention allows for the development of a control strategy by determining which synchronizations to add to an execution trace in order to tolerate a synchronization fault during the re-execution of a computer program, or programs, encountering fault.
- This proves to be an important problem in its own right and can be applied in areas other than software fault tolerance, such as concurrent debugging.
- the problem is generalized using a framework known as the off-line predica te con trol problem.
- the problem, as it applied to debugging was introduced in a paper entitled "Predicate control for active debugging of distributed programs," by A. Tarafdar and V.K. Garg. IEEE Proceedings of the 9 th symposium on Parallel and Distributed Processing, Orlando, USA, April 1998.
- off-line predicate control specifies that, given a computation and a property on the computation, one must determine a controlled computation (one with more synchronizations) that maintains the property (used is the term computa tion for a formal model of an execution) .
- Previous attempts solved the predicate control problem for a class of properties called disj unctive predicates .
- Applying the results of the Tarafdar et al. study to software fault tolerance would mean avoiding synchronization failures of the form: l 1 ⁇ l 2 "l3, where li is a local property specified on process P ⁇ . For example, if li specifies that a server is unavailable, the synchronization failure is that all servers are unavailable at the same time.
- the invention addresses a class of off-line predicate control problems, characterized by the mutual exclusion property that is especially useful in tolerating races.
- Four classes of mutual exclusion properties are considered: off-line mutual exclusion , off-line readers writers, off-line independent mutual exclusion, and off-line independent read-write mutual exclusion .
- off-line mutual exclusion a class of mutual exclusion property that is especially useful in tolerating races.
- four classes of mutual exclusion properties are considered: off-line mutual exclusion , off-line readers writers, off-line independent mutual exclusion, and off-line independent read-write mutual exclusion .
- necessary and sufficient conditions under which the problem may be solved are determined.
- an efficient algorithm that solves the most general of the problems, off-line independent read-write mutual exclusion, is presented, which also solves each of the other three problems.
- the algorithm takes O (np) time, where n is the number of concurrent processes and p is the number of critical sections.
- off-line problems to distinguish them from their more popular on-line variants (i.e. the usual mutual exclusion problems).
- the difference between the online and off-line problems is that in the on-line case, the computation is provided on-line, whereas in the off-line case, the computation is known a priori . Ignorance of the future makes online mutual exclusion a harder problem to solve.
- the model that is now presented is of a single execution of the concurrent program.
- the model is not at the programming language level, but at a lower level, at which the execution consists of a sequence of states for each process and the communications that occurred among them.
- S be a finite set of elementary entities known as sta tes .
- S is partitioned into subsets Sj, S 2 , . . . , S n , where n > 1. These partitions correspond to n processes in the system.
- a subset G of S is called a global s ta te if Vi:
- ⁇ 1.
- G denote the unique element in GnS x .
- a global predica te is a function that maps a global state onto a boolean value.
- a computa tion is a partial order -> on 5 such that Vi: ⁇ i is a total order on S , where — > represents - restricted to the set S 2 .
- V ⁇ z S : ⁇ -consistent (K) 3 global state Gi K : -»-consistent (G)
- extensibility allows for the extension of a consistent set of states to a consistent global state. Any computation in S can be made extensible by adding "dummy" states to
- ⁇ be a relation on global states defined as: G ⁇ H ⁇ Vi : ⁇ G ⁇ ⁇ i Hi) v ( G ⁇ H ) . It is a well- known fact that the set of ->-consistent global states is a lattice with respect to the ⁇ relation.
- a computation -> c is called a controlling computation of B in — > if (1) —>•£—» c , and
- FIG. 2 an illustration of variants of off- line mutual exclusion is provided.
- the simplest specification for mutual exclusion is: no two critical sections execute at the same time. This corresponds to the semantics of a single exclusive lock for all the critical sections.
- the corresponding problem can be referred to as an Off-line Mutual Exclusion Problem.
- This can be generalized to the Off-line Readers Wri ters Problem by specifying that only critical sections that "write” must be exclusive, while critical sections that "read” need not be exclusive. This corresponds to the semantics of read-exclusive-write locks.
- Another way to generalize the Off-line Mutual Exclusion Problem is to allow the semantics of independent locks. In this Off-line Independent Mutual Exclusion Problem, no two critical sections of the same lock can execute simultaneously. Finally, critical sections may be associated with the semantics of independent read- exclusive-write locks. This is the Off-line Independent Read-Wri te Mutual Exclusion Problem .
- Figure 2 illustrates the relative generality of the four problems.
- Off-line Mutual Exclusion is a specialization of Off-line Predicate Control to the following class of global predicates:
- cri tical is a function that maps a state onto a boolean value.
- B mutex specifies that at most one process may be critical in a global state.
- CS. firs t and CS.last be the minimum and maximum states respectively in CS (w.r.t. — > x ) .
- —> be a relation on critical sections defined as: CS ⁇ CS' ⁇ CS. first - CS' . last A CS ⁇ CS' Thus, -» orders a critical section before another if some state in the first happened before some state in the second. Note that —> may have cycles. All computations will have the same total order — 1 for each
- Theorem 1 (Necessary Condition) For a computa tion -+ of S, and a global predica te B mut e ⁇ ,
- Case 1 [Si e CS ! v s 2 e CS 2 ] In this case ->B mutex ( G ⁇ ) .
- Case 2 [si € CSi ⁇ S 2 € CS 2 ] Since sx CS lr there are two ways to position s ⁇ : (a) s x ⁇ - c CSi. first or (b) CSi.last —> Sj . In sub-case (a), since CS 2 - last > C CS 1 . first, either CS 2 .last
- ⁇ k can be rewritten as: ( ⁇ * ⁇ 2 ⁇ CS k -x.last, CS k . first) ⁇ ) + .
- -» ⁇ * is irreflexive and transitive (which together imply asymmetry) .
- Irreflexivity Let s ⁇ k t. There are two possibilities: either s ⁇ k ⁇ t or s ⁇ k'1 Cs k - ! .last, ⁇ Cs k . first ⁇ * "1 t. In the first case, the inductive hypothesis indicates that -»* ⁇ 2 is irreflexive and so s ⁇ t.
- part (i) of the inductive hypothesis indicates that ⁇ k' ⁇ is transitive
- part (2) of the inductive hypothesis indicates that CS k . first > k'1 CS k - i. last and so s ⁇ t.
- - c is a controlling computation of B rw in ⁇ .
- G is a global state such that - ⁇ B r w(G) . Therefore, states s and t can be found such that write critical (s) and critical (t) .
- CS be a write critical section that contains s and let CS' be a critical section that contains t.
- SCC and SCC 3 be the strongly connected components that contain CS and CS' respectively.
- SCCi is distinct from SCCj since, otherwise, there would be a cycle in— > that contains a write critical section. Without loss of generality, let i ⁇ j .
- —» c is a controlling computation of B rw in — > .
- Exclusion Problem is a specialization of the Off-line Predicate Control Problem to the following class of global predicates:
- Bi nd (G) V distinct s, t Ve G : i :-t (critical ⁇ (s) ⁇ critical (t) )
- an i-cri tical section is defined in an analogous fashion to the critical sections that were defined before. Note that the definition allows independent critical sections on the same process to overlap. In particular the same set of states may correspond to two different critical sections (corresponding to a critical section with multiple locks) . Let ⁇ be a relation on all critical sections defined as before.
- Theorem 5 (Necessary Condition)
- Theorem 6 (Sufficient Condition) For a computa tion —> of S, and a global predicate Bi nd ,
- Off-line Independent Read-Write Mutual Exclusion Problem is a specialization of the Off-line Predicate Control Problem to the following class of global predicates:
- Bind-rw (G) Vdistinct s, t e G : Vi :-, (write- critical (s) ⁇
- Theorem 7 (Necessary and Sufficient Condition)
- Figure 3 shows the algorithm to find a controlling computation of Bi nd - w in -> . Since the other forms of mutual exclusion are special cases of Bi nd - rw , this algorithm can be applied to any of them.
- the input to the algorithm is the computation, represented by n lists of critical sections C ⁇ , ... , C n .
- Critical sections are totally ordered on each process. Each critical section is represented as its process id, its first and last states, a type identifier cs_id that specifies the cri tical cs i d function, and a flag indicating if it is a write or read critical section.
- the partial order is implicitly maintained by vector clocks associated with the first and last states of each critical section.
- the algorithm outputs the —» n relation specified as a list of ordered pairs of states.
- the goal of each iteration of the first loop is to add an sec, which is minimal w.r.t. -», to ordered (where —> is the relation on scc's defined in the proof of Theorem 4).
- This sec it first computes the set of sec' s among the leading critical sections in C 2 ,...C n . Since no sec can contain two critical sections from the same process, it is sufficient to consider only the leading critical sections. From the set of scc's, it determines the set of minimal scc's, crossable . In then randomly selects one of the minimal scc's.
- the main while loop of the algorithm executes p times in the worst case, where p is the number of critical sections in the computation.
- p is the number of critical sections in the computation.
- Each iteration takes 0 (n 2 ) since it must compute the scc's.
- a simple implementation of the algorithm will have a time complexity of 0 (n 2 p) .
- a better implementation of the algorithm would amortize the cost of computing scc's over multiple iterations of the loop.
- Each iteration would compare each of the critical sections that have newly reached the heads of the lists with the existing scc's, thus forming new scc's.
- Each of the p critical section therefore, reaches the head of the list just once, when it is compared with n - 1 critical sections to determine the new scc's.
- the time complexity of the algorithm with this improved implementation is, therefore, O(np) . Note that a naive algorithm based directly on the constructive proof of the sufficient condition in Theorem 7 would take 0 (p 2 ) . The complexity has been significantly reduced by using the fact that the critical sections in a process are totally ordered.
- the algorithm has implicitly assumed a total ordering of critical sections in each process. As noted before, however, independent critical sections on the same process may overlap, and may even coincide exactly (a critical section with multiple locks is treated as multiple critical sections that completely overlap) .
- the algorithm can be extended to handle such cases by first determining the scc's within a process. These scc's correspond to maximal sets of overlapping critical sections. The input to the algorithm would consist of n lists of such process-local scc's. The remainder of the algorithm remains unchanged.
- the present invention in its application to software fault tolerance, consists of four parts: (A) tracing an execution, (B) detecting a synchronization failure, (C) determining a control strategy, and (D) re-executing under control.
- the algorithm used in the present invention determines which synchronizations to add in order to avoid very general forms of mutual exclusion violation.
- the other three parts of our scheme have been addressed as independent problems. All of the pieces are at this point put together for a comprehensive look at how race failures (mutual exclusion violations) can be tolerated.
- Figure 5 illustrates an example of tolerating races in a concurrent execution.
- the problem of determining a control strategy was placed in a very general model of concurrent execution. Tracing, detection, and controlled re-execution, however, depend greatly on the particular concurrent paradigm. A simple example is best chosen to demonstrate the key issues that will arise in most concurrent paradigms .
- a distributed system of processes that write to a single shared file. The file system itself does not synchronize accesses and so the processes are responsible for synchronizing their accesses to the file. If they do not do so, the writes may interleave and the data may be corrupted. Since the file data is very crucial, it must be ensured that races can be tolerated.
- Synchronization occurs through the use of explicit message passing between the processes.
- the first part of our mechanism involves tracing the execution.
- the concern during tracing is to reduce the space and time overhead, so that tolerating a possible fault does not come at too great a cost.
- a vector clock mechanism is used, updating the vector clock at each send and receive point.
- This vector clock needs to be logged for each of the writes to the file (for the algorithm of the present invention) .
- the vector clock values must also be logged for each receive point (for replay purposes) .
- the writes are typically very long and therefore are performed asynchronously. Thus, execution continues while the write is in progress.
- the process may receive a message from another process during its write to the file. Inserting some computation at the send, receive, write initiation, and write completion points can be achieved either by code instrumentation, or by modifying the run-time environment (message- passing interface and the file system interface) .
- the second part of our mechanism is detecting when a race occurs. Since message passing is used as the synchronization mechanism, the methods described in a Ph.D. thesis by R.H.B. Netzer, entitled “Race condition detection for debugging shared- memory parallel programs," University of Wisconsin-Madison, 1991, are particularly applicable. Once a race has been detected, all processes are rolled back to a consistent global state prior to the race.
- the file is also rolled back to a version consistent with the rolled-back state of the processes. (A versioned file system with the ability to roll back must be assumed. )
- the section of the traced vector clock values that occur after the rolled-back state is then noted.
- the section indicates the critical section entry and exit points required by the algorithm.
- the algorithm would take O(np) time, where n is the number of processes and p is the number of critical sections that have been rolled back.
- the output of the algorithm is the set of added synchronizations specified as pairs of critical section boundary points.
- Figure 5 demonstrates a possible scenario.
- the semantics of mutual exclusion correspond to a single exclusive lock. Therefore, the necessary and sufficient condition is that there are no cycles in the critical section graph shown in Figure 4 (b) .
- Applying the algorithm would add synchronizations to give the controlling computation shown in Figure 5(c).
- the next step is to replay the processes using the logged vector clock values of the receive points.
- Each receive point must be blocked until the same message arrives as in the previous execution. This is a standard replay mechanism.
- additional synchronizations must be imposed. For example, suppose [ s, t) in one of the synchronizations output by our algorithm. The state s is a critical section exit point while t is a critical section entry point. Each of these additional synchronizations is implemented by a control message sent from s and received before t. Thus, at each critical section exit point, the added synchronizations must be checked to decide if a control message must be sent.
- the added synchronizations must be checked to decide if the process must block waiting for a control message.
- the points at which computation must be added are the write initiation and completion points, and the send and receive points. Again, this can be accomplished by code instrumentation or run-time environment modification.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU80251/00A AU8025100A (en) | 1999-10-13 | 2000-10-13 | Software fault tolerance of concurrent programs using controlled re-execution |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15925399P | 1999-10-13 | 1999-10-13 | |
US60/159,253 | 1999-10-13 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2001027764A1 true WO2001027764A1 (fr) | 2001-04-19 |
Family
ID=22571749
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2000/028568 WO2001027764A1 (fr) | 1999-10-13 | 2000-10-13 | Tolerance aux failles de logiciel de programmes concurrents par reexecution commandee |
Country Status (2)
Country | Link |
---|---|
AU (1) | AU8025100A (fr) |
WO (1) | WO2001027764A1 (fr) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2266026A1 (fr) * | 2008-03-11 | 2010-12-29 | University of Washington | Multitraitement déterministe efficace |
US8453120B2 (en) | 2010-05-11 | 2013-05-28 | F5 Networks, Inc. | Enhanced reliability using deterministic multiprocessing-based synchronized replication |
US8694997B2 (en) | 2007-12-12 | 2014-04-08 | University Of Washington | Deterministic serialization in a transactional memory system based on thread creation order |
-
2000
- 2000-10-13 WO PCT/US2000/028568 patent/WO2001027764A1/fr active Application Filing
- 2000-10-13 AU AU80251/00A patent/AU8025100A/en not_active Abandoned
Non-Patent Citations (1)
Title |
---|
A. TARAFDAR AND V. GARG: "Software Fault Tolerance of Concurrent Programs Using Controlled Re-execution", PROCEEDINGS OF THE 13TH INTERNATIONAL SYMPOSIUM ON DISTRIBUTED COMPUTING (DISC), 27 September 1999 (1999-09-27) - 29 September 1999 (1999-09-29), Bratislava, Slovak Republic, pages 210 - 224, XP002156645, Retrieved from the Internet <URL:http://www.cs.utexas.edu/users/ashis/papers.html> [retrieved on 20010105] * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8694997B2 (en) | 2007-12-12 | 2014-04-08 | University Of Washington | Deterministic serialization in a transactional memory system based on thread creation order |
EP2266026A1 (fr) * | 2008-03-11 | 2010-12-29 | University of Washington | Multitraitement déterministe efficace |
EP2266026A4 (fr) * | 2008-03-11 | 2012-01-11 | Univ Washington | Multitraitement déterministe efficace |
US8739163B2 (en) | 2008-03-11 | 2014-05-27 | University Of Washington | Critical path deterministic execution of multithreaded applications in a transactional memory system |
US8453120B2 (en) | 2010-05-11 | 2013-05-28 | F5 Networks, Inc. | Enhanced reliability using deterministic multiprocessing-based synchronized replication |
Also Published As
Publication number | Publication date |
---|---|
AU8025100A (en) | 2001-04-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7516361B2 (en) | Method for automatic checkpoint of system and application software | |
Bernick et al. | NonStop/spl reg/advanced architecture | |
US10474471B2 (en) | Methods and systems for performing a replay execution | |
US6058491A (en) | Method and system for fault-handling to improve reliability of a data-processing system | |
CN109891393B (zh) | 使用检查器处理器的主处理器错误检测 | |
US7987385B2 (en) | Method for high integrity and high availability computer processing | |
Berryhill et al. | Robust shared objects for non-volatile main memory | |
US6772367B1 (en) | Software fault tolerance of concurrent programs using controlled re-execution | |
US20060190702A1 (en) | Device and method for correcting errors in a processor having two execution units | |
KR100304319B1 (ko) | 시간 지연 이중화 기술을 구현하는 장치 및 방법 | |
US9032190B2 (en) | Recovering from an error in a fault tolerant computer system | |
Nicola | Checkpointing and the modeling of program execution time | |
JP2008518308A (ja) | マルチプロセッサシステム内のデータを分配するための方法およびデータ分配ユニット | |
CN116610752A (zh) | 事务性分布式数据同步方法、装置、系统及存储介质 | |
CA2694198C (fr) | Module de traitement d'ordinateur de haute integrite et haute disponibilite | |
Schuette et al. | Exploiting instruction-level parallelism for integrated control-flow monitoring | |
Long et al. | Implementing forward recovery using checkpoints in distributed systems | |
Tarafdar et al. | Software fault tolerance of concurrent programs using controlled re-execution | |
US20070150709A1 (en) | Method and system for state tracking and recovery in multiprocessing computing systems | |
Smith et al. | Surviving peripheral failures in embedded systems | |
WO2001027764A1 (fr) | Tolerance aux failles de logiciel de programmes concurrents par reexecution commandee | |
US8745440B1 (en) | Computer-implemented system and method for providing software fault tolerance | |
Montezanti et al. | Soft errors detection and automatic recovery based on replication combined with different levels of checkpointing | |
Rebaudengo et al. | Software-level soft-error mitigation techniques | |
Chen et al. | MSRP-FT: Reliable resource sharing on multiprocessor mixed-criticality systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
122 | Ep: pct application non-entry in european phase | ||
NENP | Non-entry into the national phase |
Ref country code: JP |