KR101637504B1 - System and method for processing informal data - Google Patents

Abstract

Disclosed are a system and a method for processing unstructured data. The system for processing unstructured data according to an exemplary embodiment of the present invention comprises: a pattern providing portion which provides a pattern of the unstructured data based on a format of the unstructured data; a regulation providing portion which makes at least one item match the pattern, and provides regulation including a relationship between the item and the pattern; and a regulation execution engine which generates characteristics by adapting the regulation to the unstructured data.

Description

[0001] SYSTEM AND METHOD FOR PROCESSING INFORMAL DATA [0002]

The technical idea of the present invention relates to a system and method for processing unstructured data, and more particularly to a system and method for extracting characteristics from unstructured data.

The present invention is derived from research conducted and conducted by Saltlux Co., Ltd. as part of the SW Technology Computing Industry Source Technology Development Project (SW) of the future creation science department. [Research period: 2014.05.01 ~ 2015.02.28, Research institute: Information and Communication Technology Promotion Center, Research title: WiseKB: Big data understanding based self-learning knowledge base and reasoning technology development, task number: 10044494]

Knowledge base construction can be performed by classifying the collected data (data) into a lexical system and storing it in a database. Data collected for building a knowledge base can have a wide variety of sources. For example, the data collected for building a knowledge base may be data collected via the Internet, such as news, academic information, dictionaries, etc., from other established knowledge bases (e.g., expert knowledge base) Data that is collected offline, or data that is directly input by the user. Also, the data collected can be in a wide variety of formats. For example, data collected for knowledge base construction may be text-based data, image-based data, or voice and video-based data. Thus, extracting necessary information from various types of data and managing the extracted information may be very important in building a knowledge base.

The technical idea of the present invention provides an unstructured data processing system and method for effectively extracting characteristics from unstructured data.

According to an aspect of the present invention, there is provided an unstructured data processing system including a data interface unit for receiving unstructured data from outside, a characteristic extracting unit for extracting characteristics of the unstructured data, And a characteristic information generating unit that generates characteristic information by setting relation information on the characteristic of the atypical data based on the format of the atypical data, A rule providing unit for associating at least one item with the pattern and providing a rule including a correspondence relationship of the item and a pattern, and a rule execution engine for generating the characteristic by applying the rule to the atypical data .

According to an exemplary embodiment of the present invention, the atypical data processing system includes a data storage unit including a pattern storage unit for storing a plurality of patterns and a rule storage unit for storing a plurality of rules, The pattern providing unit may generate a pattern based on the input signal and store the pattern in the pattern storing unit. The rule providing unit may generate a pattern based on the input signal, And store it in the rule storage unit.

According to an exemplary embodiment of the present invention, the pattern providing unit may include a pattern recommendation unit that provides at least one recommended pattern among a plurality of patterns stored in the pattern storage unit based on the format of the atypical data, And a pattern execution unit that extracts data included in the information area from the irregular data based on the pattern defined by the pattern definition unit, . ≪ / RTI >

According to an exemplary embodiment of the present invention, the pattern recommendation unit can select the recommendation pattern based on the type or source of the knowledge data.

According to an exemplary embodiment of the present invention, the pattern definition unit may identify at least one information area by analyzing the format of the atypical data, and may determine the information area as a pattern based on the input signal and / Or may be excluded from the pattern.

According to an exemplary embodiment of the present invention, the pattern definition unit may group a plurality of information areas having the same format.

According to an exemplary embodiment of the present invention, the rule providing unit provides at least one recommendation rule selected from a plurality of rules stored in the rule storage unit based on data extracted from the information area of the irregular data according to the pattern And a rule defining unit that defines rules corresponding to the irregular data based on the input signal and / or the recommendation rule.

According to an exemplary embodiment of the present invention, the rule recommendation section may select the recommendation rule based on the type or source of the knowledge data.

According to an exemplary embodiment of the present invention, the rule defining unit can identify an item corresponding to the information area by analyzing the extracted data, and associate the information area with the item.

According to an exemplary embodiment of the present invention, the rule defining section may store a plurality of candidate items, may update the candidate items based on the input signal, and may map the information region to one of the candidate items .

According to an exemplary embodiment of the present invention, the data storage unit may further include a knowledge data storage unit for storing knowledge data, and the unstructured data processing system may include external knowledge data received from the interface, And a knowledge data management unit for converting the characteristic information into knowledge data based on the stored knowledge data and verifying the converted knowledge data.

According to an exemplary embodiment of the present invention, the characteristic information generation unit may further include a characteristic extraction management unit that classifies the atypical data according to the data type and generates a control signal to change the extraction method according to the corresponding data type , The pattern providing unit may analyze the format of the atypical data based on the control signal.

According to the unstructured data processing system and method according to the technical idea of the present invention, information included in the unstructured data can be effectively extracted by using patterns and rules.

Further, according to the unstructured data processing system and method according to the technical idea of the present invention, a plurality of patterns and rules are provided, and valid information is automatically extracted from unstructured data by recommending rules and patterns suitable for received unstructured data .

1 is a block diagram illustrating an unstructured data processing system in accordance with an exemplary embodiment of the present invention.
FIG. 2 is a block diagram showing an embodiment of the characteristic extracting unit of FIG. 1 according to an exemplary embodiment of the present invention.
3 to 5 are diagrams for explaining the operation of the characteristic extracting unit of FIG.
Figure 6 is a block diagram illustrating an embodiment of the pattern provider of Figure 2, in accordance with an exemplary embodiment of the present invention.
Figure 7 is a block diagram illustrating an implementation of the rule provider of Figure 2, in accordance with an exemplary embodiment of the present invention.
8 is a flowchart schematically showing a method for processing unstructured data according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. The present invention is capable of various modifications and various forms, and specific embodiments are illustrated and described in detail in the drawings. It should be understood, however, that the invention is not intended to be limited to the particular forms disclosed, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for similar elements in describing each drawing. In the accompanying drawings, the dimensions of the structures are enlarged or reduced from the actual dimensions for the sake of clarity of the present invention.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a part or a combination thereof is described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be construed to have meanings consistent with the contextual meanings of the related art and are not to be construed as ideal or overly formal meanings as are expressly defined in the present application .

Figure 1 is a block diagram illustrating an unstructured data processing system 10 in accordance with an exemplary embodiment of the present invention. 1, the atypical data processing system 10 may include a data interface unit 100, a characteristic information generation unit 200, a user interface 300, and a data storage unit 400. The atypical data shown in Fig. The processing system 10 may also be referred to as a knowledge base building system. In the following, each component may be a hardware block or a software block. For example, each of the components may be an independent hardware block that sends and receives signals to each other, or may be a software block that executes on one processor.

The data interface unit 100 may receive data from a data pool external to the atypical data processing system 10. [ Data pools can indicate that data can be created, maintained, and distributed, such as the Internet, databases, cloud sourcing, or social networks. The data pool may also include data provided directly to the unstructured data processing system 10 by the public or individuals.

The data interface unit 100 may receive informal data or unstructured data or knowledge data from the data pool. Unstructured data is data that is not implemented in a fixed form and is compared with formal data (structured data) containing contents corresponding to a corresponding field. For example, a database or a spreadsheet may be regular data, and a text document, voice data, image data, and the like may be unstructured data. Although XML or HTML may not be stored in a fixed field but may include metadata or schema, XML or HTML may be classified as semi-structured data, but the present invention is not limited to semi-structured data, It can be assumed that it is assumed to be a work type. Unstructured data can be created, retained and distributed through cloud sourcing or social networks, among other examples of data pools described above.

The above-described fixed data or unstructured data is referred to as data before processing, and the secondary data obtained by processing the data may be referred to as information. The knowledge data received by the data interface unit 100 may be meta information on how to use the information. For example, the wind speed, wind direction and humidity obtained in observing the climate may correspond to data, and the predicted weather may be information. At this time, it can be concluded that knowledge, such as trial and error and analysis of cumulative information, for example, leads to a higher incidence of accidents in case of snow, which may correspond to knowledge data. Hereinafter, in order to distinguish knowledge data inputted from the outside and knowledge data generated and managed by the unstructured data processing system 10, the former is divided into external knowledge data and the latter is divided into internal knowledge data. The data interface unit 100 may receive external knowledge data from a wiki, DBpedia, FreeBase, or the like.

In this manner, the data interface unit 100 can automatically receive irregular data or external knowledge data from the outside through a search engine or the like. In response to a request made by the characteristic information generation unit 200 or the knowledge data management unit 500 or a request made by another function block of the unstructured data processing system 10, unstructured data or external knowledge data is received from the data pool .

The user interface unit 300 can exchange signals with external users of the unstructured data processing system 10. [ For example, the user can input an input signal for setting a method of analyzing unstructured data through the user interface unit 300. [ In addition, the user interface unit 300 may provide the user with an output signal indicating the result of analyzing the unstructured data.

Although the data interface unit 100 and the user interface unit 300 are shown as independent components in the example shown in FIG. 1, it is to be understood that this is merely an example, and the technical idea of the present invention is not limited thereto . For example, when the atypical data processing system 10 receives data through the Internet and transmits and receives signals to the user via the Internet, the atypical data processing system 10 accesses the atypical data processing system 10 via one interface, The information can be exchanged with the outside.

The characteristic information generation unit 200 extracts the characteristics of the inputted irregular data and sets the relationship information about the characteristic to generate characteristic information about the unstructured data. The atypical data characteristic information generation unit 200 may include a characteristic extraction management unit 220, a characteristic extraction unit 240, and a characteristic relation setting unit 260.

The characteristic extraction management unit 220 may classify the irregular data according to the data type and generate the control signal so that the method of extracting the characteristic according to the corresponding data type is different. For example, when the unstructured data is text-based data, the characteristic extraction management unit 220 may generate the control signal so that the characteristics are extracted based on the frequency of the words included in the unstructured data. Alternatively, when the data type of the irregular data is audio or video, the characteristic extraction management unit 220 can generate the control signal so that the characteristic is extracted based on the frequency spectrum of the irregular data.

The characteristic extracting unit 240 may extract the characteristic from the unstructured data in response to the control signal. For example, the characteristic extraction unit 240 can extract words having a high frequency of occurrence as characteristics. Or the characteristic extracting unit 240 may define an object in the image according to the analysis result of the frequency spectrum. In this case, the characteristic extracting unit 240 can extract objects such as eyes, nose, and mouth with respect to the face image. The property extractor 240 may include a module (not shown) for converting the format from the frequency spectrum to the object.

The characteristic relation setting unit 260 may set the relationship information on the characteristic by giving semantic information to the characteristic extracted from the characteristic extracting unit 240. [ For example, the characteristic relation setting unit 260 may assign semantic information to a word having a high frequency of occurrence by tagging an object name using a vocabulary dictionary. Further, the characteristic-relation setting unit 260 may analyze the association between at least two pieces of semantic information in the characteristic, and assign the newly set or generated semantic information to the characteristic. For example, when the words included in the text are mobile phones and household appliances, the characteristic relation setting unit 260 may assign semantic information such as an electronic product to these characteristics. At this time, the characteristic relation setting unit 260 can perform the above analysis using the internal knowledge data stored in the knowledge data storage unit 420 of the data storage unit 400. [

The characteristic information thus generated is transmitted to the knowledge data management unit 500. The knowledge data management unit 500 converts the characteristic information received from the characteristic information generation unit 200 into internal knowledge data based on the knowledge data received from the data interface unit 100, The verification method is used. To this end, the knowledge data management unit 500 may include a knowledge data conversion unit (not shown) and a conversion verification unit (not shown).

The knowledge data conversion unit may convert the characteristic information into the formal data using a semantic technology. Semantic technology is an intelligent technology that enables communication between computers by defining the language and rules that a computer understands, such as a person reading the screen and understanding the meaning. Semantic technology aims to express the relationship - semanteme between the objects belonging to the environment in the form of ontology that can be processed by a machine, that can be processed by a computer, and to be processed by an automated machine. An ontology is a model that abstracts and shares what people think about things, and is a formalized and explicitly defined type of concept or constraint of use. In computer science, an ontology is a data model that represents a particular domain and is defined as a formal data describing the relationship between concepts and concepts belonging to a particular domain. Ontology is a tool to implement semantic technology. It is used as a tool to connect data semantically. It can process the concept of a human being in a computer in the form of a database.

In the field of semantic technology, the expression form of a triple is used as means for expressing a relationship. A triple is a representation of a concept in the form of a subject, a predicate, or an object. Each subject, predicate, and object can be represented by a URI (Uniform Resource Identifier) of XML. Currently, RDF, OWL proposed by W3C and TopicMaps proposed by ISO are standard languages that describe semantic web ontology at present.

The knowledge data conversion unit may use the external knowledge data in converting the characteristic information into the internal knowledge data in the triple form. For example, the knowledge data conversion unit may use the external knowledge data in forming the internal knowledge data or connecting the additional objects, etc. by forming the relationship between subject, predicate, and object included in the characteristic information. For example, the knowledge data conversion unit can convert characteristics information of person A, person B, and marriage to "A married to B", using knowledge data about weddings of Wikipedia A and B, And B were married at a Hyatt hotel on August 10, 2013 ". Since the above example is for illustrative purposes, it may be independent of the knowledge data classified according to the data and knowledge described above.

In converting the characteristic information into the triple-type internal knowledge data, the knowledge data conversion unit may assign weight to the characteristic information (or characteristic) or the external knowledge data. For example, for property information that gives semantic information of home about the characteristics of furniture and household appliances included in arbitrary text, we consider the weight of household appliances to be higher than that of furniture And can be generated as internal knowledge data related to the newlyweds. Alternatively, the knowledge data conversion section may further assign a weight to characteristic information of the person A, the person B, and the marriage, rather than the external knowledge data that the person A, the person C, the love and the A are unmarried, Based on the characteristic information, A can generate internal knowledge data of married. At this time, the knowledge data conversion unit A generates the pending internal knowledge data such as unconfirmed whether married or not, and then, based on the accumulated characteristic information or the external knowledge data, A calculates the final internal knowledge data on married status Can be generated.

The conversion verification unit can verify the internal knowledge data (temporary internal knowledge data) generated from the knowledge data conversion unit using a different verification method and process it as the verified internal knowledge data. The internal knowledge data verified by the conversion verification unit is stored in the knowledge data storage unit 420 of the data storage unit 400.

2 is a block diagram showing an embodiment of the characteristic extracting unit 240 of FIG. 1 according to an exemplary embodiment of the present invention. 3 to 5 are views for explaining the operation of the characteristic extracting unit 240 of FIG. Specifically, FIG. 3 is a diagram showing an example of a pattern, FIG. 4 is a diagram showing an example of a rule, and FIG. 5 is a diagram showing characteristics generated from unstructured data by executing a rule.

As described above, the characteristic extracting unit 240 can extract the characteristics of the irregular data. 2, the characteristic extracting unit 240 may include a pattern providing unit 242, a rule providing unit 244, and a rule execution engine 246. [ 2, the data storage unit 400 may further include a pattern storage unit 440 and a rule storage unit 460 as well as a knowledge data storage unit 420. Hereinafter, an unstructured data processing system according to an exemplary embodiment of the present invention, as described above, may be applied to various types of atypical data processing systems, It will be understood that the present invention can be applied to data.

According to an exemplary embodiment of the present invention, pattern providing 242 may provide a pattern corresponding to unstructured data based on the format of the unstructured data. The pattern may function as a reference used to extract the information area included in the unstructured data, and the information area may refer to an area containing useful information in the unstructured data. For example, in an HTML document which is a kind of unstructured data, the information area may be a text area. The information region can be extracted from the unstructured data according to the pattern in the unstructured data. The pattern used to extract the information area from the irregular data may be defined by the user through the user interface unit 300 or may be selected from a plurality of patterns stored in the pattern storage unit 440 of the data storage unit 400 It is possible.

Referring to FIG. 3, as a kind of unstructured data, a pattern in an HTML document of a social network can be used to extract a text region. As shown in the left side of FIG. 3, the HTML document of the social network may include a plurality of text regions separated from each other, and the pattern may classify a total of seven text regions as keywords included in a class . As shown on the right side of FIG. 3, the information areas, i.e., text areas, extracted from the HTML document of the social network using the pattern may each contain text. As described above, the pattern providing unit 242 can extract the information area from the unstructured data by providing a pattern corresponding to the unstructured data based on the format of the unstructured data. A detailed description of the operation of the pattern providing unit 242 will be described later with reference to FIG.

Meanwhile, according to an exemplary embodiment of the present invention, when the unstructured data is audio or video based data, data associated with a specific reference value can be extracted as an information region from unstructured data, and the pattern can determine the reference value. For example, in the case of speech-based data, the pattern may be used to extract sound above a certain db or sound containing a certain frequency.

According to an exemplary embodiment of the present invention, the rule providing unit 244 may correspond to at least one item in the pattern provided by the pattern providing unit 242, and may provide a rule including a correspondence relationship of items and patterns have. That is, the rule may include at least one item, and the item may correspond to the pattern provided by the pattern providing unit 242. Also, the rule can determine a method of extracting only necessary data from the information area extracted by the pattern for each information area. The rules may be defined by the user through the user interface unit 300 or may be selected from a plurality of rules stored in the rule storage unit 460 of the data storage unit 400. [

In the example shown in Fig. 4, the rule may map a plurality of items to the pattern of Fig. That is, as shown in the left column of FIG. 4, the rule includes seven items such as 'fullname', 'username', 'time', 'tweet-text', 'reply', 'retweet' , And each item may correspond to the information area (or text area) of Fig. 3 including the item as a keyword. In addition, the rule can determine the method of processing data of the information area, as shown in the right column of Fig. 4, so that only necessary data can be extracted from the data included in the information area. For example, the rule may determine that the data corresponding to the 'fullname' item is to extract the entire text contained in the text area, while the data corresponding to the 'retweet' item may be determined to extract only the numbers.

In accordance with an exemplary embodiment of the present invention, the rule enforcement engine 246 may generate properties of unstructured data by applying rules to unstructured data. That is, as shown in FIG. 5, the rule execution engine 246 generates the characteristic by associating each item with a value (i.e., a result of processing the data included in the information area) by executing the rule of FIG. . In the example shown in Fig. 5, the characteristics of the HTML document of the social network may include seven items and values corresponding to the items. 1, the characteristics of the irregular data generated by the rule execution engine 246 may be used by the property relationship setting unit 260 to generate property information, and the property information may be used by the knowledge data management unit 500 Knowledge data.

As described above, the atypical data processing system 10 according to the exemplary embodiment of the present invention processes the data of the pattern and information area defining the useful information area in the unstructured data based on the format of the unstructured data, It is possible to extract the characteristic from the unstructured data using the rule that associates the information area with the item based on the data of the unregulated data. Thus, the atypical data can be analyzed effectively, and the characteristics of the atypical data for generating the knowledge data can be extracted effectively.

Figure 6 is a block diagram illustrating an embodiment of pattern remover 242 of Figure 2, in accordance with an exemplary embodiment of the present invention. 6, the pattern providing unit 242 may include a pattern recommending unit 242_2, a pattern defining unit 242_4, and a pattern execution engine 242_6.

The pattern recommendation unit 242_2 can provide a recommendation pattern judged to be suitable for the irregular data. The pattern recommendation unit 242_2 can receive the atypical data from the data interface unit 100 and access the pattern storage unit 420. [ The pattern recommendation unit 242_2 may select at least one of a plurality of patterns stored in the pattern storage unit 440 based on the format of the unstructured data received from the data interface unit 100, It can be provided to the pattern defining section 242_4 as a pattern.

According to the exemplary embodiment of the present invention, the pattern recommending section 242_2 can select a recommendation pattern based on the type and / or source of the unstructured data. For example, the pattern recommendation unit 242_2 can analyze the source of the HTML document, for example, domain information, because the unstructured data received from the data interface unit 100 is an HTML document. When the domain information corresponds to the service providing the social network, the pattern shown in FIG. 3 among the plurality of patterns stored in the pattern storage unit 420 can be selected, and the selected pattern is used as the recommendation pattern in the pattern definition unit 242_4. As shown in FIG.

The pattern definition unit 242_4 can determine a pattern to be applied to the irregular data. That is, the pattern corresponding to the atypical data can be determined based on the input signal received from the user through the user interface unit 300 and / or the recommendation pattern received from the pattern recommendation unit 242_2. For example, the pattern defining unit 242_4 can identify at least one information region included in the atypical data by analyzing the format of the atypical data. For example, the pattern definition unit 242_4 can identify a plurality of text areas in an HTML document. The pattern defining section 242_4 may exclude some of the plurality of information areas according to the recommendation pattern received from the pattern recommending section 242_2 based on the input signal or may set an additional information area in the recommendation pattern. For example, the pattern defining section 242_4 can define a pattern such that an unnecessary information region included in the atypical data, for example, an information region including advertisement information, is not extracted based on the input signal of the user. Accordingly, a new pattern can be defined, and the pattern definition unit 242_4 can store a new pattern in the pattern storage unit 420. [

The pattern definition unit 242_4 may group a plurality of information areas having the same format. For example, a plurality of search results derived by a search engine or users' replies in a social network may exist as a plurality in one unstructured data, and may have the same format as each other. The pattern definition unit 242_4 can group or layer the information areas having this same format.

The pattern execution engine 242_6 can generate the result of applying the pattern to the irregular data. That is, the pattern execution engine 242_6 can extract the data of the information area from the irregular data based on the pattern defined by the pattern defining part 242_4. The pattern execution engine 242_6 can provide the extracted data to the user through the user interface unit 300. The input signal fed back by the user through the user interface unit 300 to the provided data is supplied to the pattern definition unit 242_4 ) May be reflected in the pattern defined by the user. Accordingly, the user can set the pattern while confirming the result of applying the pattern to the atypical data. In addition, data extracted from the pattern execution engine 242_6 may be provided to the rule providing unit 244. [

FIG. 7 is a block diagram illustrating an embodiment of the rule providing unit 244 of FIG. 2, according to an exemplary embodiment of the present invention. As shown in FIG. 7, the rule providing unit 244 may include a rule recommending unit 244_2 and a rule defining unit 244_4.

The rule recommendation unit 244_2 may provide a recommendation rule judged to be suitable for the unstructured data. The rule recommendation unit 244_2 can receive the pattern and extracted data from the pattern execution engine 242_6 of the pattern providing unit 242 and can access the rule storage unit 460. [ The rule recommendation unit 244_2 may select at least one of a plurality of rules stored in the rule storage unit 460 based on the pattern and the extracted data, and the at least one rule selected as the recommendation rule is defined as the rule definition unit 244_4. As shown in FIG. For example, as shown in FIGS. 3 and 4, the rule recommendation unit 244_2 may classify the rules stored in the rule storage unit 244_2, based on the characteristics of each information region or a combination of a plurality of keywords, At least one of the plurality of rules stored in the memory 460 may be selected. That is, the rule recommending unit 244_2 can determine that the unstructured data is the HTML document of the social network based on the seven keyword combinations shown in FIG. 3, and accordingly, the plurality of rules stored in the rule storage unit 460 Can be selected. According to the exemplary embodiment of the present invention, the pattern recommendation section 242_2 may select a recommendation pattern based on the type and / or source of the unstructured data.

The rule definition unit 244_4 can determine a rule to be applied to the irregular data. That is, the rule corresponding to the unstructured data can be determined based on the input signal received from the user through the user interface unit 300 and / or the recommendation rule received from the rule recommendation unit 244_2. For example, the rule defining unit 244_4 may analyze the information contained in the information area extracted from the unstructured data (for example, by analyzing the keyword included in the class in FIG. 3, or determining whether the format of the text indicates the date, It is possible to identify the item corresponding to the information area, and to associate the information area with the item).

According to an exemplary embodiment of the present invention, the rule defining unit 244_4 may store a plurality of candidate items, update the candidate items based on the input signal received from the user through the user interface unit 300 , The information area can be mapped to one of the candidate items. For example, in the example of FIG. 4, "October 31" corresponds to the "time" item, but if there is "date" among the candidate items stored in the rule definition unit 244_4, In the data, a text area containing "time" in the class can be mapped to the "date" item.

As described above, the atypical data processing system 10 may store a plurality of patterns and a plurality of rules, and may provide a recommendation pattern and a recommendation rule determined to be suitable for the unstructured data. In addition, by providing an interface for defining patterns and rules from the user, it is possible to define patterns and rules suitable for the unstructured data, and as a result, the characteristics can be effectively extracted from the unstructured data.

8 is a flowchart schematically showing a method 20 for processing unstructured data according to an exemplary embodiment of the present invention. As shown in FIG. 8, the atypical data processing method 20 according to the exemplary embodiment of the present invention may include receiving (S10) unstructured data. Referring to FIG. 1, the data interface unit 100 may receive unstructured data from a data pool.

The atypical data processing method 20 may include defining a pattern based on the input signal and / or a recommendation pattern (S20). Referring to FIG. 6, the pattern definition unit 242_4 may generate a pattern corresponding to irregular data based on an input signal received from a user through the user interface unit 300 and / or a recommendation pattern received from the pattern recommendation unit 242_2. Can be defined. Next, the atypical data processing method 20 may include storing and executing the defined pattern (S30). Referring to FIG. 6, the pattern definition unit 242_4 may store the defined pattern in the pattern storage unit 440, and the pattern execution engine 242_6 may extract the information region by executing the defined pattern.

The atypical data processing method 20 may include defining a rule based on the input signal and / or the recommendation rule (S40). Referring to FIG. 7, the rule defining unit 244_4 may generate a rule corresponding to the unstructured data based on the input signal received from the user through the user interface unit 300 and / or the recommendation rule received from the rule recommending unit 244_2 Can be defined. Next, the atypical data processing method 20 may include storing and executing the defined pattern (S50). Referring to FIG. 7, the rule defining unit 244_4 may store the defined rule in the rule storing unit 460, and the rule executing engine 246 may extract the characteristic of the unstructured data by executing the defined rule .

As described above, exemplary embodiments have been disclosed in the drawings and specification. While the embodiments have been described herein with reference to specific terms, it should be understood that they have been used only for purposes of describing the technical idea of the invention and not for limiting the scope of the invention as defined in the claims . Therefore, those skilled in the art will appreciate that various modifications and equivalent embodiments are possible without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

10: atypical data processing system 20: atypical data processing method
100: data interface unit 200: characteristic information generating unit
220: characteristic extraction management section 240: characteristic extraction section
242: pattern providing part 242_2: pattern recommending part
242_4: Pattern definition section 242_6: Pattern execution engine
244: Rule providing unit 244_2: Rule recommending unit
244_4: Rule definition section 246: Rule execution engine
260: characteristic relation setting unit 300: user interface unit
400: data storage unit 420: knowledge data storage unit
440: pattern storage unit 460: rule storage unit
500: Knowledge data management unit

Claims (12)

A data interface unit for receiving atypical data from outside; And
A characteristic extracting unit for extracting the characteristics of the atypical data and a characteristic information generating unit for generating characteristic information by assigning semantic information to the characteristic and setting relation information about the characteristic and the characteristic, ,
Wherein the characteristic extracting unit comprises:
A pattern providing unit for providing a pattern of the atypical data based on the format of the atypical data;
A rule providing unit for associating at least one item with the pattern and providing a rule including a correspondence relationship of the item and the pattern; And
And a rule execution engine that generates the property by applying the rule to the atypical data. The method according to claim 1,
Wherein the atypical data processing system comprises:
A data storage unit including a pattern storage unit for storing a plurality of patterns and a rule storage unit for storing a plurality of rules; And
Further comprising a user interface unit for receiving an input signal from a user and providing an output signal to the user,
Wherein the pattern providing unit generates a pattern based on the input signal and stores the pattern in the pattern storing unit,
Wherein the rule providing unit generates a rule based on the input signal and stores the generated rule in the rule storing unit. 3. The method of claim 2,
Wherein the pattern providing unit comprises:
A pattern recommending unit for providing at least one recommended pattern among a plurality of patterns stored in the pattern storage unit based on the format of the atypical data;
A pattern defining unit that determines a pattern corresponding to the irregular data based on the input signal and / or the recommendation pattern; And
And a pattern execution engine for extracting data included in the information area from the irregular data based on the pattern defined by the pattern defining section. The method of claim 3,
Wherein the pattern recommendation unit selects the recommendation pattern based on the kind or origin of the atypical data. The method of claim 3,
Wherein the pattern definition section identifies at least one information area by analyzing the format of the atypical data and sets or excludes the information area from the pattern based on the input signal and / An unstructured data processing system. 6. The method of claim 5,
Wherein the pattern definition unit groups the plurality of information areas having the same format. 3. The method of claim 2,
The rule-
A rule recommendation unit for providing at least one recommendation rule selected from a plurality of rules stored in the rule storage unit based on data extracted from the information area of the irregular data according to the pattern; And
And a rule defining unit that defines rules corresponding to the irregular data based on the input signal and / or the recommendation rule. 8. The method of claim 7,
Wherein the rule recommendation unit selects the recommendation rule based further on the type or source of the unstructured data. 8. The method of claim 7,
Wherein the rule defining unit identifies an item corresponding to the information area by analyzing the extracted data, and associates the information area with the item. 8. The method of claim 7,
Wherein the rule defining unit stores a plurality of candidate items, updates the candidate items based on the input signal, and maps the information region to one of the candidate items. 3. The method of claim 2,
Wherein the data storage unit further comprises a knowledge data storage unit for storing knowledge data,
The atypical data processing system further includes a knowledge data management unit that converts the characteristic information into knowledge data based on external knowledge data received from the interface and knowledge data stored in the knowledge data storage unit and verifies the converted knowledge data And an unstructured data processing system. The method according to claim 1,
Wherein the characteristic information generation unit further includes a characteristic extraction management unit for classifying the atypical data according to a data type and generating a control signal to change an extraction method according to a corresponding data type,
Wherein the pattern providing unit analyzes the format of the atypical data based on the control signal.

Images