KR102276761B1 - How to automatically extract information on the cause of disaster - Google Patents

Abstract

The present invention relates to a method for automatically extracting only sentences directly related to a disaster from a plurality of disaster records provided to a computer and, more specifically, to an algorithm building and using a database of a disaster cause information dictionary, which is formed by analyzing and constructing existing disaster records, to search for disaster cause information sentences from disaster records, wherein a disaster cause information automatic extraction technology is a natural language processing technology searching and extracting texts which can become disaster cause information from disaster records. According to the present invention, the method comprises: a disaster cause keyword dictionary construction process using existing disaster records containing disaster keywords to produce a disaster cause keyword dictionary, wherein an original sentence for each disaster keyword is included; a disaster cause keyword dictionary pre-processing process of constructing a database for each disaster keyword by extracting only morphemes from the original sentences for each disaster keyword of the disaster cause keyword dictionary; a disaster record pre-processing process of separating the disaster records to be investigated into sentences and extracting only morphemes from each sentence to construct creates a database for each sentence; and a disaster cause information extraction process of comparing the morphemes of each sentence extracted in the disaster record pre-processing process with the morphemes in all disaster keyword sentences extracted in the disaster cause keyword dictionary pre-processing process to determine the sentence as a sentence related to a disaster when a coincidence rate is higher than a predetermined value.

Description

{How to automatically extract information on the cause of disaster}

The present invention relates to a method for automatically extracting only sentences directly related to a disaster cause from among a plurality of disaster records,

Disaster cause information automatic extraction technology is a natural language processing technology that searches and extracts sentences that can become disaster cause information from disaster records. It is about an algorithm that searches for disaster cause information sentences.

To explain in more detail, first, a dictionary of disaster cause keywords is constructed using the original sentences in each disaster record, and only morphemes are extracted from the original sentences for each disaster keyword in the disaster cause keyword dictionary to build a database for each disaster keyword. ,

By comparing the morphemes of each sentence extracted in the disaster record pre-processing process with the morphemes in all disaster keyword sentences extracted in the pre-processing process of the disaster cause keyword dictionary, if the coincidence rate is above a certain value, the sentence is determined to be related to a disaster It relates to the method of extracting disaster cause information.

Disasters or disasters are mainly caused by abnormal natural phenomena or man-made accidents, causing great social and economic damage. Recently, abnormal climate phenomena are occurring all over the world. ·There are heavy rain, heavy snow, floods, tsunamis, and earthquakes, and other man-made disasters include fire, explosion, release of radioactive materials, automobile accidents, sinking of ships, and aircraft distress.

The 2017 Jecheon Sports Fire and the 2018 Miryang Nursing Hospital Fire occurred due to similar causes such as illegal extension of the Incheon Hope House fire accident that occurred 20 years ago, absence of an evacuation route, no sprinkler installation, and lack of safety awareness. has occurred.

Disregarding the saying that rivers and mountains change after 10 years, accidents similar to those of the past are repeatedly occurring in disasters. Social disasters often occur or expand due to man-made negligence, surrounding risk factors, complex social and building structures, etc. In particular, in the case of similar disasters, repetitive causes exist as risk factors. In order to manage and understand the possibility of disasters, the research institute collects records of past disasters as prior research, and establishes disaster archiving and disaster profiling technology while conducting research to identify the causes of similar disasters by identifying the causes from the records. did. Similar disaster analysis technology starts with cause analysis of past disasters, and in order to analyze the causes, the collection of records of past disasters should be carried out first.

Therefore, it is necessary to continuously collect qualitative and quantitative reliable records, and to improve the usability of the collected records through standardization of the causes identified in the records.

In this respect, it is necessary to informatize and standardize the cause. If the causal terms of the collected records are standardized, the causal terms mainly used in the records can be systematically managed and the scope of records and records review that needs to be collected in the future can be processed. In addition, it is possible to systematically manage records by reclassifying the types of collected records.

To meet the necessity of collecting records of various disasters and categories in order to increase the usability of records collection and techniques to identify the causes of disasters, we analyze the causes and causes, and the correlations and associations between causes and effects by subdividing the causes of disasters. Then, it is possible to predict the pattern of disaster occurrence due to successive causes, and it is possible to consider the linkage with other disasters without being limited to one type of disaster.

In addition, while reviewing the records, it is possible to provide the disaster information along with the cause as a basis for the development of technology for extracting disaster cause information by arranging the year of the disaster, the area where it occurred, and the casualties and damage to property.

In other words, it is necessary to collect and analyze a large amount of disaster records for a more accurate disaster analysis. Since there are only a few sentences directly related to the cause of the disaster among the contents contained in the disaster records, it takes a very long time to extract and analyze them. there was

In particular, in the case of the prior art, since the analysis of vast disaster records relied on the manual work of daily checking by researchers, it is very difficult and time-consuming and expensive to extract only the sentences directly related to the disaster from among the disaster records. There was a problem in that the work for disaster analysis and prevention was delayed or suffered considerable difficulties.

Patent Document 1: Korean Patent Publication No. 2013-0010780 Patent Document 2: Korean Patent Publication No. 2013-00434224

Therefore, the present invention was made in view of the problems in the prior art, and the main object of the present invention is to automatically extract sentences directly related to disasters from a computer when collecting and analyzing a large amount of disaster records. The purpose of this is to enable rapid and accurate analysis of texts related to the causes of disasters from vast disaster records.

In addition, there is another purpose of this, in that the cost and time required for the analysis of disaster records are reduced, and secondarily, the work for disaster analysis and prevention is not delayed or suffers difficulties.

In the method for automatically extracting disaster cause information of the present invention for achieving the above object;

In the method for automatically extracting only the sentences directly related to the cause of the disaster from among the disaster records,

A disaster cause keyword dictionary construction process that uses existing disaster records to produce a disaster cause keyword dictionary, but includes the original sentence for each disaster keyword;

A pre-processing process of the disaster cause keyword dictionary that extracts only morphemes from the original sentences for each disaster keyword in the disaster cause keyword dictionary and builds a database for each disaster keyword;

A disaster record pre-processing process that separates the disaster records to be investigated into sentences, extracts only morphemes from each sentence, and creates a database for each sentence;

The morphemes of each sentence extracted in the disaster record pre-processing process are compared with morphemes in all disaster keyword sentences extracted in the pre-processing process of the disaster cause keyword dictionary. It is characterized in that it includes the process of extracting the disaster cause information to be determined.

In the process of constructing the disaster cause keyword dictionary, the dictionary may include index information, disaster cause keyword, and cause sentence information.

In the pre-processing of the disaster cause keyword dictionary, in addition to extracting morphemes from the original sentences for each disaster keyword (original sentence)), only morphemes related to the disaster keywords were extracted from the original sentences for each disaster keyword (cause nouns (original text) )) and extracting only the morphemes related to the disaster keyword for the cause from the original sentence for each disaster keyword (causal nouns (organization)) can be used together to build a database.

In the disaster record preprocessing process, in addition to the morpheme content extracted from each sentence, a database is created for each sentence so that the contents of the appearance position of the morpheme matching the morpheme of the disaster cause keyword dictionary are also recorded.

In the disaster cause information extraction process, morphemes of each sentence extracted in the disaster record pre-processing process and morphemes in all disaster keyword sentences extracted in the pre-processing process of the disaster cause keyword dictionary are compared, and two or more morphemes in each sentence When the coincidence rate (the number of morphemes that match the morphemes in the disaster cause keyword dictionary in each sentence / the number of all morphemes in the corresponding sentence in the disaster cause keyword dictionary) exceeds 0.75, the sentence is determined to be related to the disaster. .

Meanwhile, in the disaster cause information extraction process, the morphemes of each sentence extracted in the disaster record preprocessing process are compared with the morphemes of the disaster cause keyword dictionary,

As morphemes of the disaster cause keyword dictionary, in addition to extracting morphemes from the original sentence for each disaster keyword (original sentence), only morphemes related to the disaster keyword are extracted from the original sentence for each disaster keyword (causal noun (original text)) and, from the original sentence for each disaster keyword, extracting only the morphemes related to the disaster keyword for the cause (causal noun (organization)) are compared together,

In any case, if the coincidence rate is above a certain value, the sentence is determined to be related to a disaster.

If the coincidence rate (the number of morphemes matching the morphemes in the disaster cause keyword dictionary in each sentence / the number of all morphemes in the corresponding sentence in the disaster cause keyword dictionary) exceeds 0.75 and two or more morphemes match in each sentence, the sentence determined to be related to the disaster.

In the method for automatically extracting information on disaster causes according to the present invention, when a large amount of disaster records are to be collected and analyzed, sentences directly related to disasters can be automatically extracted from the computer, It is possible to quickly and accurately analyze related sentences.

In addition, this has the effect of reducing the cost and time required for the analysis of disaster records, and secondarily, preventing delays or difficulties in the work for disaster analysis and prevention.

1 is a flowchart for explaining a method for automatically extracting information on a disaster cause according to the present invention;
Figure 2 is a view schematically showing the process of Figure 1,
3 is a view showing the pre-processing process of the disaster cause keyword dictionary in the present invention,
4 is a view showing the pre-processing of disaster records in the present invention,
5 is a diagram illustrating a process of extracting disaster cause information in the present invention.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. However, it goes without saying that the scope of the present invention is not limited thereto.

In the present specification, the present embodiment is provided so that the disclosure of the present invention is complete, and is provided to completely inform those of ordinary skill in the art to which the present invention belongs, the scope of the invention, the scope of the present invention is only It is only defined by the claims. Thus, in some embodiments, well-known components, well-known operations, and well-known techniques have not been specifically described in order to avoid obscuring the present invention.

The terms used herein are for the purpose of describing the embodiments and are in no way intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. In addition, elements and operations referred to as 'include (or include)' do not exclude the presence or addition of one or more other elements and operations.

Disaster cause information automatic extraction technology is a natural language processing technology that searches for and extracts sentences that can become disaster cause information from disaster records. In the present invention, an algorithm for searching the disaster cause information sentences from the disaster records by using the database of the disaster cause keyword dictionary analyzed and constructed from the existing disaster records was researched and developed.

The present invention is a method for automatically extracting only a sentence directly related to a disaster from among a plurality of disaster records provided to a computer,

As shown in FIGS. 1 and 2, a keyword dictionary construction process for producing a disaster keyword dictionary using existing disaster records including disaster keywords, but including the original sentences for each disaster keyword;

A pre-processing process of a keyword dictionary for constructing a database for each disaster keyword by extracting only morphemes from the original sentences for each disaster keyword in the disaster cause keyword dictionary;

A disaster record pre-processing process that separates the disaster records to be investigated into sentences, extracts only morphemes from each sentence, and creates a database for each sentence;

The morphemes of each sentence extracted in the disaster record pre-processing process are compared with morphemes in all disaster keyword sentences extracted in the pre-processing process of the disaster cause keyword dictionary. It consists of the process of extracting the disaster cause information to be determined.

For the present invention, it is necessary to first build a keyword dictionary database for disaster cause keyword dictionary.

The established disaster cause keyword dictionary, that is, the keyword dictionary, consists of index information such as accident number and record number, MEPS code, disaster cause keyword, cause noun (original text and arrangement), result, and cause sentence information as shown in the table below. .

Among them, a database is constructed using information on the cause of disaster keyword, cause noun, and cause sentence, which is judged to be of great value for disaster cause information.

In FIG. 1, the "original sentence" of the pre-processed disaster keyword dictionary is the same as the cause sentence and refers to the original sentence, and the "cause noun (original text)" is extracted from the original sentence only the content related to the disaster keyword, and the "cause noun" (Summary)" is extracted from the original sentence, only the content related to the disaster keyword for the cause,

For example, if "the original sentence" is "despite the change in the nature of infectious disease risk, our quarantine system has been leaning towards the traditional index-based quarantine system", then "disaster keyword" becomes "infectious disease awareness insufficient" and "cause "Noun (original text)" becomes "adherence to the traditional quarantine system in response to changes in infectious disease risk", "causal noun (reorganization)" becomes "adherence to the traditional quarantine system",

If the “original sentence” is “a lack of basic competencies of the public health system (infectious disease management governance, hospital infection control),” “disaster keyword” becomes “capacity shortage”, and “causal noun (original text)” is “basic public health system basics” “Lack of competency”, and “causal noun (organization)” becomes “lack of basic competency”.

Since the original sentences for each disaster keyword created in the process of constructing the disaster cause keyword dictionary are in a natural language state that is difficult to apply directly to the algorithm, the dictionary database is analyzed and pre-processed prior to analyzing the disaster records.

For this purpose, pre-processing is not required since most of the disaster cause keywords among each item in the dictionary database, namely, disaster cause keyword, cause noun, and cause sentence information are nouns, but morphological analysis is performed on cause nouns and cause sentences to determine the information value. Create and save a list of the highest nouns. (See Fig. 3)

That is, in the pre-processing process of the disaster cause keyword dictionary, in addition to extracting morphemes from the original sentence for each disaster keyword (the morpheme extracted original sentence), only the morphemes related to the disaster keyword from the original sentence for each disaster keyword were extracted (morphemes) The database should be built using at least one of the extracted causal noun (original text)) and extracts of only the morphemes related to the disaster keyword for the cause from the original sentence for each disaster keyword (the morpheme-extracted causal noun (reorganization)). do.

1, in the pre-processing of the disaster cause keyword dictionary, if the "original sentence" of the dictionary is "despite the change in the nature of the infectious disease risk, our quarantine system has been inclined to the traditional index-based quarantine system" "Original sentence", that is, the original sentence extracted from the morphemes becomes like "Infectious disease / risk / personality change / quarantine system / traditional / indicator",

If the "causal noun (original text)" is "adhering to the traditional quarantine system against changes in the risk of infectious diseases," the preprocessed "causative noun (original text)", that is, the morphologically extracted causative noun (original text) is "infectious disease / risk / change / preparation / traditional / It becomes like "the quarantine system / master",

If the "causal noun (reorganization)" is "adhering to the traditional quarantine system", the preprocessed "causal noun (reorganization)", that is, the morphologically extracted causative noun (reorganization) becomes the same as "traditional / quarantine system / adherence", and "infectious disease is insufficient." A database of disaster keywords will be built.

By repeating this process, information on massive disaster keywords can be pre-processed and built into a disaster cause keyword dictionary.

On the other hand, since an object of the present invention is to develop and apply an algorithm for searching for a sentence that can be a cause of a disaster, it should be possible to separate the inputted disaster record into sentences.

In the present invention, a rule-based sentence separation algorithm was designed and developed based on the result of morpheme analysis and used for pre-processing of records.

In the disaster record pre-processing process, the disaster records to be investigated are separated into sentences, and only morphemes are extracted from each sentence to create a database for each sentence. The separated sentences are the original sentence, the result of morpheme analysis, and the location of appearance in the records. , whether the cause sentence and other related information are stored in the structure. (See Fig. 4)

In the disaster record preprocessing process, in addition to the morpheme content extracted from each sentence, a database is created for each sentence so that the contents of the appearance position of the morpheme matching the morpheme of the disaster cause keyword dictionary are also recorded.

For example, as in FIG. 1 , the content of the disaster record for analysis is “Internet of Things,” as the meaning of the word, 'things' are 'interconnected (Internet)' or 'things'. Unlike the existing Internet, in which computers and mobile phones capable of wireless Internet are connected to each other, the Internet of Things is connected to all objects in the world, such as desks, cars, bags, trees, and pets. It is a structured Internet, so tangible or intangible objects existing in the world are connected to each other in various ways, so that it is possible to provide new services that individual objects could not provide. In addition, various sensors can be used to detect disaster occurrence, prevent the spread of response, etc. Technology can be provided."

In the process of pre-processing disaster records, first, the sentence “Internet of Things” means “things” as the meaning of the word, “Internet” or “Internet composed of things.” , ".Unlike the existing Internet, in which computers and mobile phones capable of wireless Internet are connected to each other, the Internet of Things is an Internet in which all objects in the world are connected, such as desks, cars, bags, trees, and pets. "So, tangible or intangible objects existing in the world are connected to each other in various ways, making it possible to provide new services that individual objects could not provide." and "Other various sensors? It is possible to provide technology that can be used in various ways, such as prevention of spread.”

For each sentence, "Things / Internet / Internet / of / Things / words / meaning / as it is...", "Existing / Internet / Computer / Wireless / Internet / Each other / Connection..." and "World / Existence..." It creates a database for each sentence by extracting only the morphemes of / tangible / intangible / object ..." and "sensor / disaster / occurrence / detection / response / spread / prevention ...".

Finally, in the disaster cause information extraction process, a list of nouns stored in each sentence object, that is, a database for each sentence, and a list of tokens for each item stored in the dictionary object for extracting disaster cause information That is, the occurrence of disaster keywords is investigated by collating the database for each disaster keyword constructed in advance.

The order of appearance does not matter, and if more than a certain percentage of the dictionary items appear as a result of the comparison, it is recorded as true in the cause sentence or not.

That is, in the disaster cause information extraction process, morphemes of each sentence extracted in the disaster record preprocessing process and morphemes in all disaster keyword sentences extracted in the preprocessing process of the disaster cause keyword dictionary are compared, and two or more in each sentence If the morphemes match and the coincidence rate (the number of morphemes that match the morphemes in the disaster cause keyword dictionary in each sentence / the number of all morphemes in the corresponding sentence in the disaster cause keyword dictionary) exceeds a certain value, the sentence is determined to be related to a disaster will do

For example, in FIG. 1, the morpheme of each sentence extracted in the pre-processing of the disaster record is "things / Internet / Internet / of / Things / words / meaning / as it is" in the case of the first sentence, and "existing" in the case of the second sentence. / Internet / computer / wireless / Internet / each other / connection", in the third sentence "world / existence / tangible / intangible / object", and in the fourth sentence "sensor / disaster / occurrence / detection / response / spread" / prevention",

The matching part is compared with the morphemes in all disaster keyword sentences extracted in the pre-processing of the disaster cause keyword dictionary,

In the case of the first sentence, as " things / Internet / Internet / of / Things / words / meaning / as it is", 4 out of 8 morphemes match, so the coincidence rate is 0.5,

In the case of the second sentence, as "existing / Internet / computer / wireless / Internet / each other / connection ", 5 of the total 7 morphemes match, so the concordance rate is 0.71,

In the case of the third sentence, as "world / existence / type / intangible / object ", one of the five morphemes matches, so the concordance rate is 0.2,

In the case of the 4th sentence, as " sensor / disaster / occurrence / detection / response / spread / prevention ", 6 out of a total of 7 morphemes match and the agreement rate is 0.86,

When two or more morphemes match in each sentence, the coincidence rate (the number of morphemes that match the morphemes in the disaster cause keyword dictionary in each sentence / the number of all morphemes in the corresponding sentence in the disaster cause keyword dictionary) exceeds a certain value, for example, 0.75 If it is determined that the sentence is related to the disaster. Of the above four sentences, only the second and fourth sentences can be extracted as sentences directly related to the disaster.

Meanwhile, in the disaster cause information extraction process, the morphemes of each sentence extracted in the disaster record preprocessing process are compared with the morphemes of the disaster cause keyword dictionary,

As morphemes of the disaster cause keyword dictionary, in addition to extracting morphemes from the original sentence for each disaster keyword (the morpheme-extracted original sentence), only the morphemes related to the disaster keyword from the original sentence for each disaster keyword were extracted (the morpheme-extracted cause noun) (original text)) and extracting only the morphemes related to the disaster keyword for the cause from the original sentence for each disaster keyword (the morpheme-extracted causal noun (summary)),

In either case, if the coincidence rate is above a certain value, the sentence may be determined as related to a disaster.

The morpheme of the disaster cause keyword dictionary is a morpheme extracted from the original sentence for each disaster keyword (original sentence from which the morpheme is extracted), for example, "infectious disease / risk / personality / change / quarantine system / orthodox / indicator",

Extracting only the morphemes related to the disaster keyword from the original sentence for each disaster keyword (the morpheme-extracted causal noun (original text)), for example, "infectious disease / risk / change / preparedness / traditional / quarantine system",

Extracting only the morphemes related to the disaster keyword for the cause from the original sentence for each disaster keyword (the morpheme-extracted causal noun (organization)), for example, “traditional / quarantine system / adherence”,

By comparing the morphemes of each sentence extracted in the disaster record preprocessing process and each of them. For example, in the case of the first sentence, by comparing morphemes in the disaster cause keyword dictionary, the coincidence rate with "the morpheme-extracted cause noun (original text)" (only the morphemes related to the disaster keyword are extracted from the original sentence for each disaster keyword) is 1 Exceeding the politics, the coincidence rate with "the morpheme-extracted causal noun (reorganization)" (only the morphemes related to the disaster keyword for the cause are extracted from the original sentence for each disaster keyword) is less than a certain value, and the "Original sentence with morpheme extraction" If the coincidence rate with (the morpheme extracted from the original sentence for each disaster keyword) exceeds a certain value, it can be expressed as “first sentence (○,×,○)”, and “the original sentence with morpheme extracted” and “ Since the coincidence rate in "the morpheme-extracted causal noun (original text)" is above a certain value, it will be possible to determine the first sentence as related to disaster.

In addition, in the case of the second sentence, by comparing the morphemes of the disaster cause keyword dictionary, "the morpheme-extracted cause noun (original text)" (only morphemes related to the disaster keyword are extracted from the original sentence for each disaster keyword) and "the cause of the morpheme extraction" Noun (organization)" (extracting only morphemes related to the disaster keyword for the cause from the original sentence for each disaster keyword) and "Original sentence extracted from morphemes" (extracting morphemes from the original sentence for each disaster keyword) and If the concordance rate of all of the above is less than a certain value, it can be expressed as “second sentence (×, ×, ×)”, and in any case, since the coincidence rate is less than a certain value, the second sentence can be determined as not related to a disaster. will be able

Preferably, when the coincidence rate (the number of morphemes matching the morpheme of the disaster cause keyword dictionary in each sentence / the number of all morphemes in the corresponding sentence in the disaster cause keyword dictionary) exceeds 0.75, two or more morphemes in each sentence match, The sentence can be identified as related to the disaster.

As described above, in the method for automatically extracting information on the cause of a disaster according to the present invention, when a large amount of disaster records are to be collected and analyzed, sentences directly related to the disaster can be automatically extracted from the computer. It is possible to quickly and accurately analyze sentences related to disaster causes from records.

In addition, this has the effect of reducing the cost and time required for the analysis of disaster records, and secondarily, preventing delays or difficulties in the work for disaster analysis and prevention.

Although the technical idea of the present invention has been specifically described in the preferred embodiment, it should be noted that the above-described embodiment is for the description and not the limitation. It is obvious to those skilled in the art that various changes and modifications can be made within the scope of the technical spirit of the present invention, and therefore, it is natural that such variations and modifications fall within the scope of the appended claims.

Claims (7)

In a method for automatically extracting only a sentence directly related to a disaster from among a plurality of disaster records provided to a computer,
Prepare a dictionary of disaster cause keywords using existing disaster records containing disaster keywords, but the "original text" for each disaster keyword, and "cause nouns (original text)" which extracts only the content related to the disaster keywords from the original text, A disaster cause keyword dictionary construction process to include a "cause noun (organization)" that extracts only the content related to the disaster keyword for the cause from the original sentence;
Pre-processing of the disaster cause keyword dictionary that constructs a database for each disaster keyword by extracting only morphemes from each of the "original sentence", "causal noun (original text)", and "causal noun (organization)" for each disaster keyword in the disaster cause keyword dictionary process and
A disaster record pre-processing process that separates the disaster records to be investigated into sentences, extracts only morphemes from each sentence, and creates a database for the sentences along with the appearance position of each sentence;
The morpheme of each sentence extracted in the pre-processing of the disaster record, and the “original sentence”, “noun of cause (original text)”, and “noun of cause (organization)” for the disaster keyword in the pre-processing process of the disaster cause keyword dictionary are extracted, respectively. By comparing the morphemes, the matching rate (number of matching morphemes / total number of morphemes) with each sentence morpheme extracted in the pre-processing of the disaster record is determined by the "original sentence", "noun of cause (original text)", "noun of cause (reorganization)")", a method of automatically extracting disaster cause information, characterized in that it includes a disaster cause information extraction process that determines that the sentence is related to a disaster if it exceeds a certain value in any one of the morphemes extracted from the morphemes.
The method of claim 1,
In the process of constructing the disaster cause keyword dictionary, the keyword dictionary automatically extracts disaster cause information, characterized in that it includes index information, disaster cause keyword and cause sentence information.
delete delete delete delete The method of claim 1,
In the disaster cause information extraction process, when two or more morphemes in each sentence match while the coincidence rate exceeds 0.75, the sentence is determined to be related to a disaster. Way.

Images