KR102385811B1 - Observation evaluation support system, observation evaluation support method, and program stored in recording medium - Google Patents

Abstract

The present invention relates to an observation evaluation support system for supporting observation evaluation with respect to emergency preparedness training of a government agency. The system includes: an evaluation criteria information setting unit setting evaluation criteria information including evaluation indicators for emergency preparedness training of government agencies and evaluation criteria and evaluation points for each evaluation indicator; an evaluation criteria information providing unit providing the evaluation criteria information set in the evaluation criteria information setting unit to an observation evaluator connected to the observation evaluation support system; an evaluation result input unit receiving an evaluation result input from the observation evaluator with respect to evaluation criteria of an evaluation target agency; and an evaluation result calculation unit using the evaluation result input by the observation evaluator to determine an evaluation score and an evaluation grade for each evaluation indicator based on the evaluation point set in the evaluation criteria information setting unit with respect to the evaluation target agency targeted by the observation evaluator.

Description

OBSERVATION EVALUATION SUPPORT SYSTEM, OBSERVATION EVALUATION SUPPORT METHOD, AND PROGRAM STORED IN RECORDING MEDIUM

The present invention relates to the evaluation of emergency preparedness training, and more particularly, to an observational evaluation support system for supporting observational evaluation of emergency preparedness drills of government agencies, an observational evaluation support method, and a program stored in a recording medium.

The emergency preparedness government training is a comprehensive training conducted to master the implementation procedures of national crisis management and wartime preparedness plans, and to check and supplement effectiveness. The emergency preparedness government training is conducted as a nationwide drill with the participation of all government agencies, including all central administrative agencies and local governments, in connection with the military exercises every year.

One of the emergency preparedness government training exercises, the Eulji exercise, in preparation for wartime, catastrophe or national emergency, reviews, supplements, and develops the effectiveness of the wartime preparedness plan in cooperation with administrative agencies. It is carried out with the purpose of making you familiar with it. Through the Eulji exercise, it is possible to efficiently control and operate national resources in preparation for wartime, catastrophe, or national emergency to support military operations, and to maintain the ability to sustain war while maintaining the functions of the government in wartime and taking measures to protect the people and live safety. there is.

Emergency preparedness drills are conducted in the form of map exercises, practical exercises, and discussion exercises. The exercise on the road refers to a situational action exercise that allows the exercise practitioner to take various measures to resolve the situation according to the event message that reflects the various crises and war realities expected in an emergency. Practical training refers to training that uses manpower, materials, equipment, etc. to master the implementation procedure of the wartime preparation plan through actual actions. Discussion-type exercise is a practice in which relevant organizations and related personnel attend the same place to present and discuss selected pending wartime tasks in order to solve problems that need to be supplemented in the wartime preparation plan and problems raised during emergency preparedness drills. say

The evaluation of the exercise is conducted to achieve the exercise goal, diagnose the possibility of performing wartime missions, supplement the wartime preparation plan, and calculate the exercise requirements for the next year. Directly visit the evaluation target institution for the entire process up to and including observation and evaluation step by step.

The conventional observational evaluator visited the evaluation target institution during the emergency preparedness training period and wrote the descriptive evaluation form by hand according to the evaluation criteria. In the descriptive evaluation form, the evaluation criteria for evaluating the evaluation index of the observation evaluator by actually observing the training situation of the evaluation target institution are subjective, and therefore, it is difficult to objectively evaluate, and it is difficult to systematically manage and quantitatively evaluate the evaluation result. , the reliability of the results could not be sufficiently secured.

Since emergency preparedness training is usually conducted within a short period of about one week, it took a lot of time for the central evaluation team to analyze and manage the handwritten evaluation documents for all evaluation target organizations. In addition, the evaluation target institution information and detailed evaluation criteria and evaluation items for each evaluation target institution may be set differently for each training year. It was costly and time consuming.

The problem to be solved by the present invention is to solve the problems of the prior art described above, and objective evaluation is possible, systematic and quantitative evaluation is possible, analysis and management of evaluation result is easy, and emergency preparedness training is possible. It is to provide an observational assessment support system, an observational assessment support method, and a program that can reduce the cost and time for assessment.

As a means for solving the above problems, the observation and evaluation support system according to an embodiment of the present invention is an observation and evaluation support system for supporting the observation and evaluation of emergency preparedness training of government agencies, the emergency of a plurality of government agencies an evaluation criterion information setting unit for setting evaluation criterion information including a plurality of evaluation indicators for preparation training, evaluation criteria for each evaluation indicator, and evaluation points; an evaluation criterion information providing unit providing the evaluation criterion information set in the evaluation criterion information setting unit to an observation evaluator connected to the observation evaluation support system; an evaluation result input unit receiving evaluation results for a plurality of evaluation criteria of an evaluation target institution from the observation evaluator; and calculating an evaluation result for determining an evaluation score and an evaluation grade for each evaluation index based on the evaluation result input by the observation evaluator based on the evaluation score set in the evaluation criterion information setting unit for the evaluation target institution of the observation evaluator may include wealth.

In an embodiment, the observation evaluation support system generates and provides an evaluation report including the evaluation score and evaluation grade calculated by the evaluation result calculation unit, and the evaluation result of the observation evaluator inputted to the evaluation result input unit It may further include a report providing unit.

In an embodiment, the evaluation criterion information further includes information about an evaluation field to which each evaluation index belongs, an evaluation area for each evaluation field, an achievement goal for each evaluation index, and an evaluation period, wherein the evaluation criterion includes one or more evaluation items may include

In one embodiment, the evaluation item of the evaluation criterion information is composed of content that can be determined by distinguishing as 'yes' or 'no' whether one or more evaluation items corresponding to each evaluation criterion are satisfied, and The evaluation score may define an evaluation score according to the number of satisfied evaluation items corresponding to each evaluation criterion.

In an embodiment, the evaluation reference information may be set differently depending on the year of implementation of the emergency preparedness training subject to observation evaluation and the government agency conducting the emergency preparedness training.

In an embodiment, one evaluation grade among a plurality of evaluation grades may be assigned to an evaluation score for each evaluation index, and different evaluation grades may be displayed in different colors.

In an embodiment, the evaluation result input unit may additionally request an opinion on the lowest evaluation reason or the highest evaluation reason with respect to the evaluation result of the observation evaluator to which the lowest evaluation score or the highest evaluation score is given.

In an embodiment, the evaluation report providing unit may generate and provide an evaluation result comparison table for each year in which the evaluation result of the current year of the observed evaluator is compared with the evaluation result of the previous year.

In one embodiment, the evaluation report providing unit analyzes the evaluation results and the lowest or highest evaluation reason for all government agencies that conducted emergency preparedness training for the year, and generates a full evaluation report of the emergency preparedness training for the year. can provide

As another means for solving the above-mentioned problem, the observation evaluation support method according to an embodiment of the present invention supports the observation evaluation performed by the observation evaluation support system for supporting the observation evaluation for emergency preparedness training of government agencies. A method comprising: an evaluation criterion information setting step of setting evaluation criterion information including a plurality of evaluation indicators for emergency preparedness training of each government agency, evaluation criteria for each evaluation indicator, and evaluation points; an evaluation criterion information providing step of providing the evaluation criterion information set in the evaluation criterion information setting step to an observation evaluator connected to the observation evaluation support system; an evaluation result input step of receiving evaluation results according to a plurality of evaluation criteria of an evaluation target institution from the observation evaluator; and calculating an evaluation result for determining an evaluation score and an evaluation grade for each evaluation index based on the evaluation result input by the observation evaluator based on the evaluation score set in the evaluation criterion information setting step for the evaluation target institution of the observation evaluator may include steps.

As another means for solving the above problems, the program according to an embodiment of the present invention may be a program stored in a recording medium to perform the observation evaluation support method by a computer.

According to an embodiment of the present invention, a plurality of evaluation indicators for emergency preparedness training of a plurality of government agencies, evaluation criteria information including evaluation criteria and evaluation points for each evaluation indicator are set, and the evaluation criterion information is evaluated by the observation Provided to the observation evaluator connected to the support system, and the evaluation result input by the observation evaluator, the evaluation score for each evaluation index based on the evaluation score set in the evaluation criterion information setting unit for the evaluation target institution of the observation evaluator And by determining the evaluation grade, objective evaluation is possible, systematic and quantitative evaluation of evaluation results is possible, analysis and management of evaluation results are facilitated, and cost and time for evaluation of emergency preparedness training can be reduced. Observational assessment support systems, methods and programs for observational assessment support may be provided.

1 is a diagram illustrating a connection relationship of an observation evaluation support system according to an embodiment of the present invention.
2 is a block diagram showing the configuration of an observation evaluation support system according to an embodiment of the present invention.
Figure 3 shows an institution and practice division selection screen of the observation evaluation support system according to an embodiment of the present invention.
4 shows an observation evaluation screen of the observation evaluation support system according to an embodiment of the present invention.
5 shows a screen in which an evaluation result is input with respect to an observation evaluation item of the observation evaluation support system according to an embodiment of the present invention.
6 is a view showing a determination result screen for the observation evaluation index of the observation evaluation support system according to an embodiment of the present invention.
7 shows a flowchart of an observational evaluation support method performed by an observational evaluation support system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a diagram illustrating a connection relationship of an observation evaluation support system 100 according to an embodiment of the present invention.

Referring to FIG. 1 , the observation evaluation support system 100 may be connected to a central evaluation group terminal 200 and a plurality of observation rater terminals 300 through a communication network N. The communication network N may include both a wired communication network and a wireless communication network.

The observational evaluator can conduct observational evaluation after directly observing the emergency preparedness training of the evaluation target institution. After performing the observation evaluation, the observation evaluator may access the observation evaluation support system 100 through the observation evaluator terminal 300 and input the observation evaluation result of the emergency preparedness training of the evaluation target institution. The observation evaluation support system 100 may determine the evaluation score and evaluation grade of the evaluation target institution based on the observation evaluation result input by the observation evaluator, and may generate an evaluation report from the determined evaluation score and evaluation grade.

The central evaluation group may access the observation evaluation support system 100 through the central evaluation group terminal 200 to inquire the observation evaluation result and evaluation report of emergency preparedness training of a plurality of evaluation target organizations. The central evaluation group may access the observation evaluation support system 100 to inquire the observation evaluation result and evaluation report of the emergency preparedness training of the previous year as well as the current year. In addition, the central evaluation group can inquire not only the evaluation results and evaluation reports of all evaluation target institutions in the current year, but also the overall evaluation reports of all evaluation target institutions in the current year.

Observation evaluation support system 100 sets evaluation criteria information including a plurality of evaluation indicators for emergency preparedness training of a plurality of government agencies, evaluation criteria for each evaluation indicator, and evaluation points, and observes the set evaluation criteria information can be provided to The observation evaluator may input an evaluation result for an objective evaluation item that can be answered with 'yes' or 'no' based on the evaluation criteria information provided by the observation evaluation support system 100 .

The observation evaluation support system 100 may determine the evaluation score and the evaluation grade based on the evaluation score of the evaluation criterion information based on the evaluation result input by the observation evaluator. The central evaluation team may access the observation evaluation support system 100 to inquire evaluation scores, evaluation grades, and evaluation reports for each evaluation target institution.

2 is a block diagram showing the configuration of the observation evaluation support system 100 according to an embodiment of the present invention.

Referring to FIG. 2 , the observation evaluation support system 100 includes an evaluation criterion information setting unit 10 , an evaluation criterion information providing unit 20 , an evaluation result input unit 30 , an evaluation result calculation unit 40 , and an evaluation report system. It may include a study 50 and a database unit 60 .

The evaluation criterion information setting unit 10 may set evaluation criterion information including a plurality of evaluation indicators for emergency preparedness training of a plurality of government agencies, evaluation criteria for each evaluation indicator, and evaluation points. The evaluation criteria information setting unit 10 may set a plurality of evaluation indicators, evaluation criteria for each evaluation indicator, and evaluation points from evaluation criteria information input by the central evaluation group before the implementation of emergency preparedness training for the current year.

Evaluation criteria information may be set differently for each government institution. Also, the evaluation reference information may be set differently for each year. The evaluation criteria for each evaluation index may include one or more evaluation items.

The evaluation criterion information set by the evaluation criterion information setting unit 10 may further include information about an evaluation field to which each evaluation index belongs, an evaluation area for each evaluation field, an achievement goal for each evaluation index, and an evaluation time. For example, the evaluation field may include a preparation phase, image practice, actual training, discussion type practice, and the like, and the evaluation area may include practice preparation, initial response, and the like.

The evaluation items of the evaluation criteria information may be composed of contents that can be distinguished by 'yes' or 'no' to determine whether one or more evaluation items corresponding to each evaluation criterion are satisfied. In one embodiment, when the evaluation index is "whether to prepare a practice preparation plan/implementation plan specification or not", the evaluation criterion is to be set as (1) the adequacy of the preparation plan, (2) whether an execution plan in line with the practice goal is prepared. can

For example, the evaluation items for the evaluation criteria of the adequacy of the preparation plan are (1-a) “Is an appropriate level of practice planning team organized?” and (1-b) “Is the training planning team jointly participated in preparing for training?” , (1-c) “Is it included in the preparation plan of the task suitable for the preparation of the government institution’s practice?” may include three evaluation items. For the evaluation items of (1-a) to (1-c), the observation evaluator examines the training preparation plan establishment status of the evaluation target institution, and determines whether each evaluation item is met by clearly distinguishing it as 'yes' or 'no' It consists of possible contents.

The evaluation items for the evaluation criteria of whether the implementation plan in line with the practice goal is prepared are (2-a) "Is the training contents of government agencies comprehensively established?", (2-b) "The training plan of each institution Has the plan been established through the evaluation process (risk assessment) for natural and human disasters occurring in the jurisdictional area?”, (2-c) “Create and update resource lists in the field of resource management, resource gathering points, resource movement plans, It can include three evaluation items, “Has the plan been established by reflecting the mutual aid agreements with each institution?” The evaluation items of (2-a) to (2-c) are also observable. The evaluator examines the training preparation plan establishment status of the evaluation target institution, and can determine whether each evaluation item is satisfied by 'yes' or 'no'. It consists of content.

The evaluation criterion information providing unit 20 may provide the evaluation criterion information set in the evaluation criterion information setting unit 10 to the observation evaluator connected to the observation evaluation support system 100 . When the observation evaluator accesses the observation evaluation support system 100 and selects an evaluation target institution and an evaluation category, an evaluation result input screen corresponding to the selected evaluation target institution and evaluation category may be displayed. The evaluation criterion information providing unit 20 may provide evaluation criteria information such as evaluation area, evaluation index, achievement goal, evaluation time, score, evaluation criteria, and evaluation items to the observation evaluator.

From the screen on which the evaluation criterion information provided by the evaluation criterion information providing unit 20 is displayed, the observing evaluator determines what achievement goal the evaluation index has, when evaluation should be performed, what the specific evaluation criteria are, and each evaluation criterion. It is possible to understand at a glance what the detailed evaluation items are and how the points vary depending on the number of satisfied evaluation items.

The evaluation result input unit 30 may receive evaluation results for a plurality of evaluation criteria of the evaluation target institution from the observation evaluator. When the observation evaluator checks the evaluation criterion information provided by the evaluation criterion information providing unit 20 and responds by using a check box to determine whether the evaluation items for each evaluation index are satisfied, the result of the observation evaluator's response is returned to the evaluation result input unit (30) can be entered.

The evaluation result calculation unit 40 calculates the evaluation result input by the observation evaluator based on the evaluation score set in the evaluation criterion information setting unit 10 for the evaluation target institution of the observing evaluator, the evaluation score and evaluation grade for each evaluation index can be decided

For example, in the above embodiment, among evaluation items (1-a) to (1-c), 1.6 points when one item is satisfied, 1.8 points when two items are satisfied, and all three items are It is set that 2 points are given when satisfied, and as an evaluation result, when the observation evaluator checks items (1-a) and (1-b) and does not check item (1-c), the evaluation set 1.8 points may be awarded for the corresponding evaluation index according to the distribution points.

In addition, it is set that a grade of 'Number' for an evaluation score of 2 points, a grade of 'Right' for an evaluation score of 1.8, and a 'Mis' grade for an evaluation score of 1.6 are given, and the corresponding evaluation index by the observation evaluator When the evaluation score of 1.8 is 1.8, a 'good' grade may be given to the corresponding evaluation index.

The evaluation report providing unit 50 may generate and provide an evaluation report including the evaluation score and evaluation grade calculated by the evaluation result calculation unit 40, and the evaluation result of the observed evaluator input to the evaluation result input unit 30 there is. If the observation evaluator performs evaluation on all evaluation indicators for the evaluation target institution and then inputs the evaluation result, the evaluation result calculation unit 40 automatically determines the evaluation score and evaluation grade, and the evaluation report providing unit 50 may generate an evaluation report including an evaluation result of an observation evaluator, an evaluation score, and an evaluation grade. The generated evaluation report may be inquired by the central evaluation team by accessing the observation evaluation support system 100 .

The evaluation report may further include an evaluation result comparison table for each year in which the evaluation result of the current year of the observation evaluator is compared with the evaluation result of the previous year. In addition, the evaluation report providing unit 50 analyzes the evaluation results and the lowest or highest evaluation reasons for all government agencies that conducted emergency preparedness training for the year using text mining techniques, A full evaluation report can be generated and provided.

In the text mining technique, which is one of the big data analysis methods for generating meaningful information by analyzing text data, it is assumed that a plurality of documents are each composed of a set of arbitrary topics, and the topics and words used in the plurality of documents are assumed. There is topic modeling that presents the importance of as a probability.

Similar to how humans can derive important topics by looking at the entire document, topic modeling can play a role in deriving topics from big data similar to humans. There is a limit to summarizing the subject by looking at all the vast amounts of text, and by using pick modeling, a comprehensive evaluation of the overall evaluation result of the organization subject to the evaluation of emergency preparedness training can be made quickly. In an embodiment of the present invention, information on an index evaluated as excellent or an index evaluated as insufficient in the emergency preparedness training of the year by using topic modeling may be included in the overall evaluation report and provided.

The evaluation report providing unit 50 may perform topic classification through topic modeling on text data of the lowest evaluation reason or the highest evaluation reason written by the observation evaluator. For topic modeling, the evaluation report providing unit 50 may select a key word and create a word/document matrix. When selecting key words, the noun extraction process and word frequency are used, along with the Korean natural language difference package, KoNLP (available from https://cran.r-project.org/src/contrib/Archive/KoNLP/ ). Stop-word dictionaries and morpheme analysis may be used.

The evaluation report providing unit 50 extracts a noun composed of 2 to 10 syllables and then applies a weight to the TF-IDF along with the word frequency to calculate the frequency of word use in the document. TF-IDF is a statistical number indicating how important a specific word is in a document, and it can be obtained by multiplying the frequency of occurrence of a specific word by TF (Term Frequency) and IDF (Inverse Document Frequency), which is the reciprocal of document frequency. After ranking for each word according to the frequency of use of the word in the identified document, the core word may be selected by calculating the cumulative share of the usage ratio for each word.

The evaluation report providing unit 50 applies a latent dirichlet allocation (LDA) model to the frequency of the selected key words to select a major topic related to the lowest evaluation reason or the highest evaluation reason for the evaluation target organization of emergency preparedness training. there is. When selecting major topics, the number of topics is not arbitrarily determined, and objective criteria can be applied.

The LDA (Latent Dirichlet Allocation) model is applied to the frequency of key words to select a major topic related to the lowest or highest evaluation reason, and when selecting a major topic, the cumulative share of (TF-IDF) in Equation 1 below You can use words that account for 30% and group words that are related to each other.

[Equation 1]

The database unit 60 includes the identification information of the observation evaluator, information on a plurality of government agencies of the evaluation target, a plurality of evaluation indicators for emergency preparedness training of a plurality of government agencies, evaluation criteria and evaluation points for each evaluation indicator Data used in the observation and evaluation support system 100 , such as evaluation criteria information, an evaluation result of an observation evaluator, an evaluation score and evaluation grade determined based on evaluation points, and an evaluation report, may be stored, corrected, and updated.

In an embodiment, the identification information of the observation evaluator may include personal identification information, account information, contact information, and the like of the observation evaluator who performs observation evaluation for each evaluation target institution. Government agency information may include agency identification numbers, account information, contact information, personnel information, and the like, of government agencies to which emergency preparedness drills are conducted and observational assessments are conducted by observation assessors.

3 shows an institution and practice division selection screen of the observation evaluation support system 100 according to an embodiment of the present invention. Figure 3 (a) is a case in which the evaluation target institution is the Ministry of Veterans Affairs, which is a government department, and the national crisis response exercise is selected as the exercise category, (b) in Figure 3 (b) is the evaluation target institution is Seoul, a local government, and practice The display screen when the national crisis response exercise is selected as the division is shown.

4 shows an observation evaluation screen of the observation evaluation support system 100 according to an embodiment of the present invention.

Referring to FIG. 4 , when an observation evaluator accesses the observation evaluation support system 100 and selects an average target institution and practice category, an evaluation field, an evaluation area, an evaluation index, and a score may be displayed. When the observing evaluator scrolls down the screen, he can also see the remaining evaluation areas, evaluation areas, evaluation indicators and points. When the observation evaluator selects (clicks) a specific evaluation index, a screen for inputting an evaluation result for the selected evaluation index may be displayed.

5 shows a screen in which an evaluation result is input for an observation evaluation item of the observation evaluation support system 100 according to an embodiment of the present invention.

Referring to FIG. 5 , the observation evaluator may check an evaluation area, an evaluation index, an achievement goal, an evaluation time, a score, an evaluation standard, and an evaluation item through the screen. When it is determined that the corresponding evaluation item is satisfied, the observation evaluator may input an evaluation result by checking a check box of the corresponding evaluation item.

When an evaluation result is entered by means of a check box, based on the evaluation score of the evaluation criteria, 1.6 points when one item is satisfied, 1.8 points when two items are satisfied, and 2 points when three items are satisfied can be given The evaluation score may be automatically calculated based on the evaluation score of the evaluation criterion information by automatically checking the number of items that are satisfied when the observation evaluator inputs the evaluation result through the check box.

In addition, when one item is satisfied (lowest score) and when three items are satisfied (highest score) for a specific evaluation criterion, a notification message may be displayed to request the observation evaluator to input a specific reason. In this case, the observing evaluator may input a specific reason for the lowest or highest score.

6 shows a screen of a determination result for an observation evaluation index of the observation evaluation support system 100 according to an embodiment of the present invention.

Referring to FIG. 6 , the observation evaluation support system 100 may determine the evaluation grades as grades such as number, goodness, beauty, quantity, etc. based on the result of evaluation by the observational evaluator for each evaluation index. Such an evaluation level may be determined based on the evaluation level set by the evaluation criterion information of the evaluation criterion information setting unit 10 .

For example, the evaluation grade is "G(GREEN) normal/smooth" when the score is 80 or more based on a 100-point scale, "Y (YELLOW) caution/partial restriction", 60 when the score is 70 or more and less than 80 A score of more than 70 points may be given as “R(RED) Concern/Limited”, and a score less than 60 points may be assigned as “B(BLACK) Severe/Paralysis”.

In the case of FIG. 5, 4 points are scored with respect to the 4-point distribution, and it corresponds to a 'G (GREEN)' grade and is indicated by a green "G" mark. The evaluation grade according to the evaluation scoring section may be set to an evaluation scoring section different from the above embodiment, and the mark and color of the grade may be set in various embodiments.

7 shows a flowchart of an observational assessment support method performed by the observational assessment support system 100 according to an embodiment of the present invention.

Referring to FIG. 7 , in the evaluation criterion information setting step (S10), the evaluation criterion information setting unit 10 sets a plurality of evaluation indicators for emergency preparedness training of each government agency, evaluation criteria and evaluation points for each evaluation indicator It is possible to set the evaluation criteria information included.

In the evaluation target institution and evaluation category selection step 20 , the observation evaluator accesses the observation evaluation support system 100 to select the evaluation target institution and the evaluation category that have performed observational evaluation.

In the evaluation criterion information providing step (S30), the evaluation criterion information providing unit 20 may provide the evaluation criterion information set in the evaluation criterion information setting step (S10) to the observation evaluator connected to the observation evaluation support system.

In the evaluation result input step S40 , the evaluation result input unit 30 may receive evaluation results according to a plurality of evaluation criteria of the evaluation target institution from the observation evaluator.

In the evaluation result calculation step ( S40 ), the evaluation result calculation unit 40 calculates the evaluation result input by the observation evaluator based on the evaluation score set in the evaluation criterion information setting step 10 for the evaluation target institution of the observation evaluator It is possible to determine the evaluation score and evaluation grade for each evaluation index.

After the evaluation result calculation step (S40), it may be determined whether the evaluation of all evaluation indices for the evaluation target institution by the observation evaluator is completed (S50). When the evaluation of all evaluation indicators has not been completed (No in S50), the evaluation result input step (S40) may be performed.

When the evaluation of all evaluation indicators is completed ('Yes' in S50), the evaluation report providing unit 50 of the observation evaluation support system 100 may generate an evaluation report for the evaluation target institution (S60) . The evaluation report may include the evaluation result of the observation evaluator, the evaluation score and evaluation grade calculated in the evaluation result calculation step ( S40 ).

The steps or processes described above may be executed by a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, a step or process described in the embodiments may be, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a PLU It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit, microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and used by those skilled in the art. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (11)

An observational assessment support system to support observational assessment of emergency preparedness drills by government agencies, comprising:
an evaluation criterion information setting unit configured to set evaluation criterion information including a plurality of evaluation indicators for emergency preparedness training of a plurality of government agencies, evaluation criteria for each evaluation indicator, and evaluation points;
an evaluation criterion information providing unit providing the evaluation criterion information set in the evaluation criterion information setting unit to an observation evaluator connected to the observation evaluation support system;
an evaluation result input unit receiving evaluation results for a plurality of evaluation criteria of an evaluation target institution from the observation evaluator;
an evaluation result calculation unit for determining an evaluation score and an evaluation grade for each evaluation index of the evaluation result input by the observation evaluator based on the evaluation score set in the evaluation criterion information setting unit with respect to the evaluation target institution of the observation evaluator; and
Evaluation report providing unit for generating and providing an evaluation report including the evaluation score and evaluation grade calculated by the evaluation result calculating unit, and the evaluation result of the observed evaluator input to the evaluation result input unit
including,
The evaluation report providing unit analyzes the evaluation results and the lowest or highest evaluation reasons for all government agencies that conducted emergency preparedness training for the current year, and generates and provides an overall evaluation report of the emergency preparedness training for the current year;
The evaluation report providing unit extracts a noun composed of 2 to 10 syllables from the text data of the lowest evaluation reason or the highest evaluation reason, and then applies a weight to the TF-IDF along with the word frequency to calculate the frequency of word use in the document, Key words are selected based on the frequency of use, and the latent dirichlet allocation (LDA) model is applied to the frequency of the selected key words. [Equation 1] of [Equation 1] to select an observation evaluation support system.
[Equation 1]

(here,

is the document

words appearing in

represents the frequency of appearance of

is the word

represents the number of documents including , where N is the total number of documents,

is the word,

indicates individual documents) delete According to claim 1,
The evaluation criterion information further includes information about an evaluation field to which each evaluation index belongs, an evaluation area for each evaluation field, an achievement goal for each evaluation index, and an evaluation period,
The evaluation criteria is an observation evaluation support system including one or more evaluation items. 4. The method of claim 3,
The evaluation item of the evaluation criterion information is composed of contents that can be distinguished by 'yes' or 'no' to determine whether one or more evaluation items corresponding to each evaluation criterion are satisfied,
The evaluation score of the evaluation criterion information is an observation evaluation support system that defines an evaluation score according to the number of satisfaction of evaluation items corresponding to each evaluation criterion. According to claim 1,
The evaluation standard information is an observation evaluation support system that is set differently depending on the year of implementation of the emergency preparedness training subject to the observation evaluation and the government agency that conducts the emergency preparedness training. According to claim 1,
An observation evaluation support system in which one evaluation grade among a plurality of evaluation grades is given for the evaluation score for each evaluation index, and the different evaluation grades are displayed in different colors. The method of claim 1,
The evaluation result input unit, observation evaluation support system for additionally requesting an opinion on the lowest evaluation reason or the highest evaluation reason with respect to the evaluation result of the observation evaluator given the lowest evaluation score or the highest evaluation score. The method of claim 1,
The evaluation report providing unit observation evaluation support system for generating and providing an evaluation result comparison table for each year in comparison with the evaluation result of the previous year with the evaluation result of the current year of the observation evaluator. delete An observational assessment support method performed by an observational assessment support system to support observational assessment of emergency preparedness drills in government agencies, the method comprising:
an evaluation criterion information setting step of setting evaluation criterion information including a plurality of evaluation indicators for emergency preparedness training of each government agency, evaluation criteria for each evaluation indicator, and evaluation points;
an evaluation criterion information providing step of providing the evaluation criterion information set in the evaluation criterion information setting step to an observation evaluator connected to the observation evaluation support system;
an evaluation result input step of receiving evaluation results according to a plurality of evaluation criteria of an evaluation target institution from the observation evaluator; and
An evaluation result calculation step of determining an evaluation score and an evaluation grade for each evaluation index based on the evaluation result input by the observation evaluator based on the evaluation score set in the evaluation criterion information setting step for the evaluation target institution of the observation evaluator ; and
An evaluation report providing step of generating and providing an evaluation report including the evaluation score and evaluation grade calculated by the evaluation result calculating step, and the evaluation result of the observation evaluator input in the evaluation result input step
including,
In the step of providing the evaluation report, the evaluation result and the lowest or highest evaluation reason for all government agencies that conducted emergency preparedness training for the year are analyzed, and a full evaluation report of the emergency preparedness training for the current year is generated and provided,
In the step of providing the evaluation report, a noun composed of 2 to 10 syllables is extracted from the text data of the lowest evaluation reason or the highest evaluation reason, and the word frequency in the document is calculated by applying a weight to the TF-IDF along with the word frequency. to select a key word based on the frequency of use, and apply the latent dirichlet allocation (LDA) model to the frequency of the selected key word to determine the lowest or highest reason for the evaluation of the organization subject to the evaluation of emergency preparedness training An observational evaluation support method that selects using [Equation 1] below.
[Equation 1]

(here,

is the document

words appearing in

represents the frequency of appearance of

is the word

represents the number of documents including , where N is the total number of documents,

is the word,

indicates individual documents) A program stored in a recording medium for performing the observation evaluation support method according to claim 10 by a computer.

Images