EP3172701A2

EP3172701A2 - A general formal concept analysis (fca) framework for classification

Info

Publication number: EP3172701A2
Application number: EP15841555.4A
Authority: EP
Inventors: Michael J. O'brien; James BENVENUTO; Rajan Bhattacharyya
Original assignee: HRL Laboratories LLC
Current assignee: HRL Laboratories LLC
Priority date: 2014-07-23
Filing date: 2015-07-23
Publication date: 2017-05-31
Also published as: EP3172701A4; CN106575380A; WO2016043846A4; CN106575380B; WO2016043846A2; WO2016043846A3

Abstract

Described is a system for data classification using formal concept analysis (FCA). In a training phase, the system generates a FCA classification lattice, having a structure, using a set of training data. The set of training data comprises training presentations and classifications corresponding to the training presentations, in a classification phase, a set of test data having classes that are hierarchical in nature is classified using the structure of the FCA classification lattice.

Description

[000 J] A GENERAL FORMAL CONCEP T ANALYSIS (FCA) FRAMEWORK FOR

CLASSIFICATION

[0002] GOVERNMENT LICENSE RIGHTS

[0003] This invention was made with government support under U.S. Government Contract Number FA8650-I3-C-7356. The government has certain rights in the invention.

[0004] CROSS-REFERENCE TO RELATED APPLICATIONS

[0005] This is a Cominuation~m~Part Application of U.S. Non-Provisional Application No.

14/489,313, filed in the United States on September 17, 2014, entitled, "y p in Across Domains to Extract Conceptual Knowledge Representation from Neural Systems," which is incorporated herein by reference in its entirely.

[0006] This is ALSO a Non-Provisional Application of U.S. Provisional Applicatio No.

62/028,171, filed in the United States o July 23, 2014, entitled, " General Formal Concept Analysis (FCA) Framework for Classification," which is incorporated herein by reference in its entirety.

[0007] BACKGROUND OF INVENTION

[0008] (I) Field of invention

[0009] The present invention relates to a system for data classification and, more particularly, to a system for data classification using formal concept analysis (FCA).

[00010] (2) Description of Related Art

[000.1 1] Classification through machine learning is a very important field of study, as it allows for systems to evolve the ability to solve difficult problems, such as face recognition, anomaly detection, and failure prediction. Classification may be described as the problem of identifying which of a set of categories applies to a new observation, based on a training set of data. [00012] Formal concept analysis (FCA) is a principled way of deriving a partial order on a set of objects each defined by a set of attributes. It is a technique in data and knowledge processing that has applications in data visualization, data mining, information retrieval, and knowledge management (see the List of Incorporated Literature References.

Literature Reference No. 3). The principal with which it organizes data is a partial order induced by a inclusion relation between object's attributes. In addition, FCA admits rule mining from structured data. It is widely applied for data analysis, especially in Germany and France.

[00013] Literature Reference No, ? provides a survey of FCA-type classification tools and concludes that none of them work well, generally demonstrating high error rates. Details of the algorithms are sparse and development and analysis is often done in French or German.

[00014] Further, Literature Reference No. 5 proposes a specific instantiation of an FCA

classifier for classification of simple symbol recognition. The work relies on finding a single node within the lattice upon which to do classification, making it apparent that in a more complex, noisy setting, the classification is likely to fail. This type of data certainly comes up in the biological realm, such as electroencephalography (EEG), functional magnetic resonance imaging blood-oxygen-level dependent (f RI BOLD), functional near infrared spectroscopy (fNlRS), and magnetoencephalography (MEG), where the noise to signal ratio is very large.

[00015] Literature Reference No. 6 proposes an iterative version of FCA classification, which yields good results in their specific test problem (again, simple symbol classification) but suffers from a potentially large number of expensive iterations, thus requiring substantial computational time.

[00016] Thus, a continuing need exists for an efficient system for classifying classes that are hierarchical in nature using formal concept analysis such thai the hierarchical struct ures of the data are revealed and exploited. [00017] SUMMARY OF THE INVENTION

[00018] The present invention relates to a system for data classification and, more

particularly, to a system for data classification using formal concept analysis (FCA), The system comprises one or more processors and a .memory having instructions such thai when the instructions are executed, the one or more processors perform multiple operations. The system generates with die one or more processors, in a training phase, a formal concept analysis (FCA) classification lattice using a set of training data and a plurality of classifications corresponding to the set of training data. Using the structure of the FCA classification lattice, a classification of a set of input data is generated during a classification phase.

[00019] in another aspect, in the training phase, a context table is generated from the set of training data, the context table having rows of object labels and columns of attribute labels. For each training presentation, in the training phase, at least one class column for a classification corresponding to the training presentation is appended to the context table. The FCA classification lattice is generated from the context table.

[00020] In another aspect, during generation of the FCA classification lattice, the at least one class column is treated as a normal attribute, wherein a sub-structure comprising a plurality of nodes within the FCA classification, lattice that is spanned by a given class- attribute is associated with the corresponding classification.

[000213 In another aspect, the system generates, in the classification phase, a presentation context vector, trip, from the set of test data, wherein ip is a set of attributes associated w ith a presentatio p in the set of test data . In the classification phase, a set of votin nodes in the FCA classification, lattice is selected and used to. vote for a classification value for the presentation/?. [00022] In another aspect, the set of voting nodes is selected according to a selection function operating on at least and the FCA classification lattice.

[00023] In another aspect, a classification value, c, is voted on according to a voting function operating on at least the output of the selection function, the FCA classification lattice, and ΤϊΙρ .

[00024] I another aspec t, the voting function returns a sum of an associated class value of each of the set of voting nodes.

[00025] In another aspect, each associated class value is weighted by a number of attributes that it shares with the presentation p,

[00026] In another aspect, each voting node has an extent comprising a set of objects, wherein the voting function returns a sum of an associated class value of each voting node, the sum is normalized by a number of objects within the voting node's extent, and the normalized sums across all voting nodes are then summed.

[00027] In another aspect, each voting node has an intent comprising a set of attributes, and the associated class value for each voting node is weighied by a number of attributes in its intent.

[00028] in another aspect, the set of training dat includes objects having attributes, and the FC A classification lattice is generated by treating the pluralit of classifications as attributes of objects in the training data.

[00029] In another aspect, the set of input data is acquired using at least one of an fMRl sensor, an image sensor, and a sound sensor, and wherein the classification is performed for purposes of at least one of object recognition, image recognition, and sound recognition. [00030] In another aspect, the present invention also comprises a metho for earning a processor to perform the operations described herein and performing the listed operations. [0003.1] Finally, in yet another aspect the present invention also comprises a computer

program product comprising computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform the operations described herein. [00032] BRIEF DESCRIPTION OF THE DRAWINGS

[00033] The file of this patent or patent application publication contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. [00034] The objects, features and advantages of the present invention wil 1 be apparent from the following detailed descriptions of the various aspects of the invention in conjunction with, reference to the following drawings, where:

[00035] FIG. 1 is a block diagram depicting the components of a system for data classificatio according to various embodiments;

[00036] FIG. 2 is an illustration of a computer program product according to various

embodiments; [00037] FIG. 3 is an illustration of a first context table according to various embodiments;

[00038] FIG. 4A is an illustration of a second context table according to various

embodiments; [00039] FIG. 4B is an illustration of a lattice resulting front the data in the second context table according to various embodiments; [00040] FIG. 5 is an illustration of formal concept analysis (FCA) lattice classification according to various embodiments; [0004.1] FIG. 6A is an illustration of a context table appended with class columns according to various embodiments; and

[00042] FIG. 6B is an illustration of a lattice resulting from the data in the context table

appended with class columns according to various embodiments.

[00043] DETAILED DESCRIPTION

[00044] The present invention relates to a system for data classification and, more

particularly, to a system for data classification using formal concept analysis (FCA), The following description is presented to enable one of ordinary skill in the art to make and use the invention and to incorporate it in the context of particular applications. Various modifications, as well as a variety of uses in different applications will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to a wide range of aspects. Thus, the present invention is not intended to be limited to the aspects presented, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[00045] In the following detailed description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without necessarily being limited to these specific details. I other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.

[00046] The reader's attention is directed to ail papers and documents which are filed

concurrently with this specification and which are open to public inspection with this specification, and the contents of all such papers aod documents are incorporated herein by reference- AH the features disclosed in this specification, (including any accompanying claims, abstract, and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

[00047] Furthermore, any element in a claim that does not explicitly state "means for"

performing a specified function, or "step for" performing a specific function, is not to he interpreted as a "means" or "step'¹ clause as specified in 35 U.S. Section 1 12,

Paragraph 6, in particular, the use of "step of or "act of in the claims herei is not intended to invoke the provisions of 35 U.S.C. 1 12, Paragraph 6.

[00048] Please note, if used, the labels left, right, front, back, top, bottom, forward, reverse, clockwise and counter-clockwise have been used for convenience purposes only and are not intended to imply any particular fixed direction, instead, they are used to reflect relative locations and/or directions between various portions of an object. As such, as the present invention is changed, the above labels may change their orientation.

[00049] Before describing the invention in detail, first a list of incorporated literature

references as used in the description is provided. Next, a description of various principal aspects of the present invention is provided. Following that is an introduction that provides an overview of the present invention. Finally, specific details of the present invention are provided to give an understanding of the specific aspects. [00050] ( 1 ) List of incorporated Liter ture References

[00051] The following references are incorporated and cited throughout this application. For clarity and convenience, the references are listed herein as a central resource for the reader. The following references are hereby incorporated b reference as though fully included herein. The references are cited in the application by referring to the corresponding literature reference number, as follows^'. 1 . V. Arulmozhi. Classification task by using Matiab Neural Network Tool Box - A beginners, international Journal of Wisdom Based Computing, 201.1.

2. K. Bache and M Lichman. UCi machine learning repository. University of California. Irvine. School of Information and Computer Sciences, 2013.

available at h t p : / / archiveics.uci.edu/ml/datasets/Iris taken on. May 5, 2015.

3. G. Romano C. Carpioeto, Concept Data Analysis: Theory and Applications.

Wiley, 2004.

4. B. Ganter and . Wille. Formal Concept Analysis: Mathematical Foundations.

Springer- Verlag, Chapters 0-2, pages 1 -94, 1 98,

5. Stephanie Guil as, Kareli Bertet, and Jean-Marc Ogier. A generic description of the concept lattices' classifier: application to symbol recognition. In

GREC: IAPR International Workshop on Graphics Recognition, 2005.

6. Stephanie GuilSas, arell Bertet; and Jean-Marc Ogier. Concept lattice

classifier: a first step towards an iterative process of recognition of noised graphic objects, in CLA; Concept Lattices and Their Applications, number section 2, pages 257-263, 2006.

7. Olga Prokasheva, A!ioa Onishchenfco, and Sergey Gorov. Classification Methods Based on Formal Concept Analysis. In PDAIR: Formal Concept Analysis Meets Information Retrieval, pages 95-104, 2013,

8. M, Swain, S. L Dash, S. Dash, and A. Mohapatra. An approach for RIS

plant classification using neural network. International Journal of Soft Computing, 20.12.

] (2) Principal Aspects

] The present invention Iras three "principal" aspects. The first is a system: for data classification using formal concept analysis (FCA). The system is typically in the form of a computer system operating software or in the form of a "hard-coded" instruction set. This system may be incorporated into a wide variety of devices that provide different functionalities. The second principal aspect is a method, typically i the form of software, operated using a data processing system (computer). The third principal aspect is a computer program product. The computer program product generally represents computer-readable instructions stored on a non-transitory computer-readable medium such as an optical storage device, e.g.. a compact disc (CD) or digital versatile disc (DVD), or a magnetic storage device such as a floppy disk or magnetic tape. Oilier, non- limiting examples of computer-readable media include har d disks, read-only memory (ROM), and flash-type memories. These aspects will be described in more detail below.

[00054] A block diagram depicting an example of a system (i.e., computer system 100) of the present invention is provided in FIG. I . The computer system 100 is configured to perform calculations, processes, operations, and/or functions associated with a program or aigorithra. in one aspect, certain processes and steps discussed herein are realized as a series of instructions (e.g., software program) that reside within computer readable memory units and are exec uted by one or more processors of the computer system 1 0, When executed, the instructions cause the computer system 100 to perform specific actions and exhibit specific behavior, such as described herein. The one or more processors may have an associated memory with executable instructions encoded thereon such that when executed, the one or more processors perform multiple operations. The associated memory is, for example, a non-transitory computer readable medium. [00055] The computer system 100 ma include an address data bus 102 that is configured to communicate information. Additionally, one or more data processing units, such as a processor 104 (or processors), are coupled with the address/data bus 102, The processor 1.04 Is configured to process information and instructions. In an aspect, the processor 104 is a microprocessor. Alternatively, the processor 104 may he a different type of processor such as a parallel processor, or a field programmable gate array.

[00056] The computer system 1 0 is configured to utilize one or more data storage units. The computer system 1.00 may include a volatile memory unit 106 (e.g., random access memory ("RAM"), static RAM. dynamic RAM, etc.) coupled with the address/data bus 102, wherein a volatile memory unit 106 is configured to store information and instructions for the processor 1 4. The computer system 100 further may include a non- volatile memory unit 108 (e.g., read-only memory ("ROM"), programmable ROM

("PROM"), erasable programmable ROM ("EPROM"), electrically erasable

programmable ROM "EEPRO "), flash memory, etc. ) coupled with the address/data bus 102. wherein the non-volatile memory unit 108 is configured to store static information and instructions for the processor 104. Alternatively, the computer system 00 may execute instructions retrieved from an online data storage unit such as in "Cloud"

computing. In an aspect, the computer system 100 also ma include one or more interfaces, such as an interface 3 10, coupled with the address/data bus 102. The one or more interfaces are configured to enable the computer system 100 to interface with othe electronic devices and computer systems. The communication interfaces implemented by the one or more interfaces ma include wireline (e.g., serial cables, modems, network adaptors, etc) and/or wireless (e.g., wireless modems, wireless network adaptors, etc.) communication technology . In one aspect, the computer system 1 0 may include an input device 1 12 coupled with the address/dat bus 102, wherein the input device 1 12 is configured to

communicate information and command selections to the processor 100, In accordance with one aspect, the input device 1 12 is an alphanumeric input device, such as a

keyboard, thai may include alphanumeric and or function keys. Alternatively, the input device 1 12 may be an input device other than an alphanumeric input device, in an aspect, the computer system .100 may include a cursor control device 1 .14 coupled with the address/data bus 102, wherein the cursor control device 1 14 is configured to

communicate user input information and/or command selections to the processor 100. in an aspect, the cursor control device 1 14 is implemented using a device such as a mouse, a track-ball, a track-pad, an optical tracking device, or a touch screen. The foregoing ftotwifhstanding, in an aspect, the cursor control device 1 14 is directed and/or activated via input from the input device 1 12, such as in response to the use of special keys and key sequence commands associated with the input device 112. In an alternative aspect, the cursor control device 1 14 is configured to be directed or guided by voice commands. [00058] In an aspect, the computer system 100 further may include one or more optional computer usable data storage devices, such as a storage device 1 16, coupled with the address data bus 102. The storage device 1 16 is configured to store information and/or computer executable instructions. In one aspect, the storage device 1 16 is a storage device such as a magnetic or optical disk drive (e.g., hard disk drive ("HDD"), floppy diskette, compact disk read only memory ("CD-ROM"), digital versatile disk ("D VD")). Pursuant to one aspect, a display device 1 18 is coupled with the address/data bus 102, wherein the display de vice 1 I S is configured to display video and/or graphics. In an aspect, the display device 1 18 may include a cathode ray tube ("CRT"), liquid crystal display ("LCD"), field emission display ("FED"), plasma display, or any other display device suitable for displaying video and/or graphic images and alphanumeric characters recognizable to a user.

[00059] The computer system 100 presented herein is an example computing environment i accordance with an aspect. However, the non-limiting example of the computer system 100 is not strictly limited to being a computer system. For example, an aspect provides that the computer system 100 represents a type of data processing analysis that may be used in accordance with various aspects described herein. Moreover, other computing sysiems may a!so be implemented. Indeed, the spirit and scope of the present technology is not limited to any single data processing environment. Thus, in an aspect, one or more operations of various aspects of the present technology are controlled or implemented using computer-executable instructions, such as program modules, being executed by a computer, in one implementation, such program modules include routines, programs, objects, components and/or data structures that are configured to perform particular tasks or implement particular abstract data types, in addition, an aspect provides that one or more aspects of the present technology are i mplemented by utilizing one or more distributed computing environments, such as where tasks are performed by remote processing devices that are linked through a communications network, or such as where various program modules are located i both local and remote computer-storage media including memory-storage de vices, [00060] An illustrative diagram of a computer program product (i.e., storage device) embodying an aspect of the present in vention is depicted in FIG. 2. The computer program product is depicted as floppy disk 200 or an. optical disk 202 such as a CD or DVD. However, as mentioned previously, the computer program product generally represents computer-readable instructions stored on any compatible no« ransitory computer-readable medium. The term "instructions" as used with respect to this invention generally indicates a set of operations to be performed on a computer, and may represent pieces of a whole program or individual, separable, software modules. Non- limiting examples of "instruction" include computer program code (source or object code) and "hard-coded" electronics (i.e. computer operations coded into a computer chip). The "instruction" is stored on any non-transitory computer-readable medium, such as .in the memory of a computer or on a floppy disk, a CD-ROM, and a flash drive, in either event, the instructions are encoded on a non-transitory computer-readable medium.

[000613 (3) Introduction

[00062] Formal concept analysis (FCA) is a principled way of deriving a concept hierarchy or formal ontology from a c llection of objects and their properties or attributes, it is a creation of a partial order of the objects based on an ordering rela tion defined by set inclusion of attributes. Formally, a context ~^: (G, M, ./) consists of two sets G and M and a relation /, caiied the incidence relation, between them. The elements of G are called the objects, and t!ie elements of are called the attributes (see Literature Reference No. 4). if an object g £ G lias the attribute m 6 , then write glm or (g, ni)€ /. A context can be represented by a cross table, or context table, which is a rectangular table where the rows are headed by objects and the columns are headed by attributes, an example of which is illustrated m FIG. 3. An "X" in the intersection of row g and column m means that object ^'has attribute m. For a set A cz G of objects, one can define A'—

[m€ M I glm V g e A . In words, for some subset of objects A, A' represents the set of attributes common to all the objects in A. Correspondingly, one can define B'™

{g e G\ glm V m€ M}> In words, for some subset of attributes ΙΪ, B' represents the set of objects which have all the attributes in B. [00063] A formal concept can now be defined. A formal concept of the context (G, M, 1) is a pair (A, B) with A c G, B <=. Μ, Α' ~ B, and B' ~ ,4. A is called the extent, and B is called the intent of the concept (A, B). {(/, , /) denotes the set of all concepts of the context (G, M, ./). A concept is represented within a context table by a maximal

contiguous bl ock of "X'"s after arbitrary rearrangement of rows and columns, as shown in FIG. 3. Algorithms for determining concept lattices are described in Literature Reference Nos. 3 and 4. Mathematically, the key aspect of concept lattices is that a concept lattice M, I) is a complete lattice in which the intltmun and supremum are, respectively, given by:

[00064] Referring to FIG. 3, an object (e.g., lion) has the attributes from the columns

corresponding to the "X"'s (e.g., preying, mammal). The contiguous block; of grey 300 is maximal, under any rearrangements of rows and columns, and forms a formal concept. The supremum is called the join and is written z V y or sometimes VS (the join of the set S). The intlmum is called the meet and is written z A y or sometimes Λ S (the meet of the set 8). An extensive description of formal concept analysis is given in Literature

Reference No. 4.

[00065] (3.1) Example of a Context and Concept Lattice

[00066] A concept lattice is a mathematical object represented by (G, , I) as described

above, A concept, lattice can be visualized by a Basse diagram, a directed acyclic graph where the nodes represent concepts and lines represent the inclusion relationship between the nodes. In the case of formal concept analysis, the Hasse diagram has a single top node representing all objects (given by G), and a single bottom node representing all attributes (given by M). Ail the nodes in between represent the various concepts

comprised of some subset of objects and attributes. A line between two nodes represents the order information. The node above is considered greater than the node below, in a Hasse diagram, a node n with attribute set m and object set g has the following properties: • m— g', is the set of all attributes shared by every object in g.

• g = m', is the set of all objects that have all attributes in m.

• Even^; child node of n has ail of m in its intent.

• Every parent node of n has all of g in its extent.

[00067] Thus, the ordering of the nodes within the lattice n > k implies that the extent of n is contained in the extent of k and, equivalentiy, the intent of n is contained in the intent of L The upset of a node n consists of all of its ancestor nodes within the lattice. The downset of n consists of all its children nodes within the lattice.

[00068] FIGs. 4A and 48 illustrate a context table and the corresponding Hasse diagram of the concept lattice induced by the formal content, respectively. The objects are nine planets, and the attributes are properties, such as size, distance to the sun, and presence or absence of moons. Each node (represented by circles, such as elements 400 and 402) corresponds to a concept, wit its objects consisting of the union of all objects from nodes connecting from above, and attributes consisting of the intersection of ail attribotes of all the nodes connecting from below. Ultimately, the top most node 404 contains all the objects, (7, and no attiibutes. Correspondingly, the botiom most node 406 contains all the attiibutes, . , and no objects.

[00069] (4) Specific Details of the Invention

[00070] (4, 1 ) Classification with FCA

[00071] In order to leverage FCA in classification, the system according to various

embodiments appends a set of class columns to a context table, where each training presentation lias at least one of the columns marked for the corresponding class. In the lattice construction process, these classes are treated as normal attributes. I the completed lattice, the sub-structure spanned by a given class-attribute is then associated with the corresponding class. This can be leveraged for classification through a node- voting scheme. Each of these aspec ts will be described in further detail below. 00072] FCA classification according to various embodiments proceeds as follows.

A. Training phase

1. Build context table from training data.

2. Append class columns for each class type, filling in the corresponding context values for each training presentation.

3. Build the FCA classification lattice from the context table, call it CLAT. B. Classification phase

1. Build context vector, ϊϊΐρ, from the data for presentation p (ϊϊΐρ represents the attributes of/?),

2. Select a set of voting .nodes N = SeiectNodes(m_p, CLAT),

3. Use the nodes to vote for the classification value to be returned,

c ~ Vote(rn.p, N, CLAT).

00073] FIG. 5 illustrates a flowchart of the FCA lattice classification according to various embodiments, in a first operation 500, a context table is built (generated), in a second operation 502, columns of class contexts for each class type are appended to the context table. In a third operation 504, an FCA classification lattice, comprising a plurality of nodes, is generated from the data in the context table, wherein the FCA classification lattice is denoted CLAT 506. As described above, in a fourth operation 508, a presentation context vector is generated. A set of voting nodes in CLA T is selected in a fifth operation 510. Finally, the selected nodes from the fifth operation 510 is used to vote for a classification value to be returned in a sixth operation 51.2, the classification result being denoted c 51 ,

[00074] For illustrative purposes, HGs. 6A and 6B illustrate a non-limiting example of a context table of data from the Iris data set having appended class columns (FIG. 6A) and a FCA classification lattice generated from the data (FIG. B). The Iris data set is available in the University of California Irvine (UCL) machine learning repository (see Literature Reference No. 2 for the Iris data set). In this data set, the goal is to classify the Iris type based on measurements, such as "sepal length", "sepal width", "and petal length". The table in FIG. 6A depicts a non-limiting example of a context table which has been appended with a set of class columns. In this example, the objects are the measufements (i.e.. petal length, petal width, sepal length) and the classes are the iris types: Setosa, Versicolor, and Virginica.

[00075] The FCA classification lattice that is generated according to various embodiments as shown in FIG. 6B depicts the sub-structures which span a given class-attribute as described above. For instance, the FCA classification lattice is highlighted to

demonstrate the sub-structures with red representing Iris Setosa, green representing iris Versicolor, and blue representing Iris Virgimea.

[00076] (4.2) Example Atomic Functions

[00077] The function SelectNod s and Vote can be varied and still provide success within this framework. The underlying classification problem should be considered in pickin these functions. Below, several examples are given. The following notation is used: define the sets ?n_n and g_n to be the intent and extent, respectively, of the formal concept represented by node n€ CLAT. Likewise, Ktlp is taken to be the set of attributes associated with presentation p. Further, e._m i- entry (m) is the entry node of attribute m in CLA J denoting the node within the lattice that itself has m in its intent, but has no ancestors with m in their intent. Graphically, e_m is the highest node in the lattice that has m in its intent. With this notation, define £ = e_m | ?n. £ titpj to be the set of all entry nodes for a set of attributes. With this notation, the following are some possible algorithms that can be used for SeieetNodes and Vote:

SeleciNodes function. ^:::

• MeetSele t: The following is the simplest example of SeleciNodes and returns the meet node of the presentation attributes, given by :

MeetSelect(m_pt CLAT) =Λ 8. [00078] This node selection (SeieciNo es) algorithm performs poorly on difficult problems with noisy data (such as iMRI BOLD data). This is due to the fact that while a meet is guaranteed to exist because of the completeness of the lattice, the meet is often the bottom of the lattice, which contains all attributes and no objects. This does not help in classification.

» AitributeSpanSelect This is the next simplest example. This function just returns the set of ail nodes that share attributes with p:

AttributeSpanSelectlpip, CLAY) ~ {

* UpseiFUierSeleci: This algorithm finds what is termed the horizon of the attribute span. In words, it finds the nodes that are deepest in the lattice while still sharing attributes with the presentation p. The idea behind this is that nodes that are deeper in the lattice contain more attributes (in particular, more attributes in common with p\ and, thus, are more specific in their classification abilities. To these ends, the horizon can be computed by using a novel upset filtering technique. The upsei(«) is a function that returns the set of ancestors of the node n, which is the set C m_n}. Thus, any node k in the upse will share ail of its attributes with w, so n should be a better classifier of //, with one caveat described below. To proceed, let \k \_p— |m_p Π m_k \ be the rank of the node k. With this notation, the algorithm begins by selecting an initial set of nodes:

N_Mt ~ AttributeSpanSeiect( „, CLv4T).

[00079] Then, the list of nodes is sorted in descending order by rank, N∑= Sort(N _njt).

Define orz ~ { }. The Up&etFilter algorithm continues as:

[00080] Require: N sorted, Horz

1 , while N≠ ø do 2. n_o := /V [0]

3. rig '= true

4. U Upset(%)

5. for all k e U u N do

6. f ife jp < j^'itjj jp the

?. N *- N\{k}

8. else

9. rig = false

10. end if

1 1. end for

12. if Tig then

13. Horz <— Horz u {n₀}

14. end if

1 5. iV ~ N\{n_Q]

16. end while

17. return Horz In line 1.3. notice that ^'?% (the candidate node is added to Harz only if tig is true. 7tq is the good-node flag, indicating whether the current node should be kept for voting which is the ease only if it has a strictly higher rank than its ancestors. To begin the iteration, it is assume that the node is good, but if an ancestor k that has the same rank as the candidate node is found, that means that the candidate node ove -classifies. The two nodes will have the same rank if and only if they share the exact same attributes with the presentation p. Since TIQ is a child of k, this implies that MQ has strictly more attributes than k, and the increase in attributes does not help to classify p. Thus, k will be a better classifier (and is kept in the list) than TIQ , SO ?ly is thrown out and the process moves on. TIQ can still he leveraged to filter the node set wi thout loss of potential horizon nodes. [00082] Vote function =

• Cl ssVoie: This returns the sum of the associated class value of each of the voting nodes, which are generated via the lattice construction process from the class columns within the context table.

» AtfribtiteWeightOassFote: This returns the same values as Class Vote, except that each vote is weighted by the number of attributes (in its intent) that it shares with the presentation p.

• Object Vote: Each voting node votes based on the objects within its extent. For each object in the nodes extent, the associated class value is used. These votes are summed and normalized by the number of objects within the node's extent, so nodes higher in the lattice do not have stronger voting power. The votes across all voting nodes are then summed.

• Attribute WeightedObject Vote: This returns the same as ObjectVote except the node votes are weighted by the number of attributes in their respective intent.

[00083] The above are a few of the possibilities for the respective functions. In practice, the choice largely depends on the underlying problem.

[00084] (4.3) Experimental Studies

[00085] (4.3 J ) Iris

[00086] The system according to various embodiments has been successfully applied to the Iris data set available in the University of California Irvine (UCI) machine learning repository (see Literature Reference No. 2 for the Iris data set). In this data set, the goal is to classify the iris type based on "sepal length", "sepal width⁵', "petal length", and "petal width". Using the present invention, it is possible to classify the data set with

1 0% accuracy with a certain choice of attributes. Though 1 0% is a great classification result, the corresponding attribute choice resulted in a relatively large lattice of over 500 nodes. However, because the system accordi ng to vari ous embodiments makes use of the data's underlvina structure, a hi ah result of 97% can be achieved with a much smaller lattice, of about 50 nodes, through the flexibility of feature choice and the underlying atomic functions. This is a significantly greater result than thai received by the prior art, such as published SO A (service-oriented architecture) classification techniques as used on this data set (which are described in Literature Reference Nos, 1 and 8).

[00087] (4.3.2) fMRI BOLD Responses

[00088] fMRI BOLD responses are used to represent a level of neural activity within the brain in a non-invasive way. Various stimuli (e.g., spoken words, written words, images) are presented, representing semantic or conceptual input. During this presentation, the brain's responses are recorded. A baseline of nidi activity is subtracted out. and the difference between this neutral brain state and the brain's state in response to the stimuli is extracted. The set of stimuli (whether individual words of sentences, spoken words, images, etc.) represent the objects of FCA, and the extracted fMRI BOLD responses for the voxels within the brain represent the attributes of the objects. FCA classification can then be applied to the fMRI BOLD responses in an effort to classify the thought process of a human. The training data consists of a set of object presentations and the resulting voxel recordings. After some pre-processing, the data is compiled into a context: table, from which the FCA lattice is built. This concludes the training phase, and the testing phase consists of new object presentations, but without a known classification of the object. The FCA classification algorithm is then used to extract predictions f om the voxel data as to what the object presentation is.

[00089] This process can be bootstrapped by compiling a semantic lattice for the same

objects, using a known expert deri ved ontology, such as WordNet. The two lattices can then be combined together into a single lattice encompassing both neural data and semantic data. This bootstrapped system would sen'e as a more thorough classification system, while simultaneously revealing similarities between neural and semantic architectures. This process of combining lattices into a. single lattice encompassing neural data and semantic data was described in U.S. on-Provisional Application No. 14/489,313, which is hereby incorporated by reference as though fully set forth herein. [00090] As machine learning is an important field of study with applications to error detection, prediction, pattern classification, as well as others, the system according to various embodiments is likewise widely applicable, excelling within a hierarchical setting. As a non-limiting example, FCA classification is instrumental to the

classification offM I BOLD responses to presented stimuli. Given a set of trial data, the present invention builds a structured system in which neural responses correspond to different classes of objects, providing an efficient analysis tool that can reliably classify new object presentation into their respective classes.

[00091] Furthermore, the invention described herein allows for knowledge discovery, yielding such concepts as CAT and DOG are MAMMALS based on the hierarchical structure thai underlies the classification process.

[00092] In addition, the present invention can be utilized to classify inefficiencies within a production line or a circuit design, since many inefficiencies are dependency based, resulting from the hidden structures within the production process,

[00093] In various embodiments, the systems and methods described herein may be applied to applications for classification problems. For example, various embodiments may be used as part of data mining procedures, image recognition, medical imaging and analysis, optical character recognition, video tracking, drug discovery and development, speech recognition, handwriting recognition, biomeiric identification, biological classification, natural language processing, document classification, credit scoring, and/or pattern recognition.

Claims

CLAIMS hat is claimed is:

1. A system for data classification using formal concept analysis (FCA), the system

comprising:

one or more processors having associated memory with executable instructions encoded thereon such that when executed, the one or more processors perform operations of:

generating with the one or more processors, in a training phase, a formal concept analysis (FCA) classification lattice using a set of training data and a plurality of classifications corresponding to the set of training data; and using the structure of the FCA classification lattice, generating a classification of a set of input data during a classification phase.

2. The system as set forth in Claim i , wherein the one or more processors further perform operations of:

generating, in the training phase, a context table from the set of training data, the context table having rows of object labels and columns of attribute labels;

for each training presentation, appending, in the training phase, at least one class column for a classification corresponding to the training presentation to the context table; and

generating the FCA classification lattice from the context table.

3. The system as set forth in Claim 2, wherein during generation of the FCA classification lattice the at least one class column is treated as a normal attribute, and wherein a substructure comprisin a plurality of nodes within the FCA classification lattice that is spanned by a given class-attribute is associated with the corresponding classification.

4. The system as set forth in Claim I , wherein the one or more processors further perform operations of: generating, in the classification phase, a presentation context vector, trip, from the set. of test data, wherein trip is a set of attributes associated with a presentation p in the set of test data;

selecting, in the classification phase, a set of voting nodes in the FCA classification lattice; and

using, in the classification phase, the set of voting nodes to vote for a classification value for the presentation/.'.

5. The system as set forth in Claim , wherein the set of voting nodes is selected according to a selection function operating on at least m_paftd the FCA classification lattice,

6. The system as set forth in Claim 5, wherein a classification value, t\ is voted, on

according to a voting function operating on at least the output of the selection function, the FCA classification lattice, and m^ .

7. The system as set forth in Claim 6, wherein the voting func tion returns a sum of an. associated class value of each of the set of voting nodes.

8. The system as set forth in Claim 7, wherein each associated class value is weighted by a number of attributes that it shares with the presentation p.

9. The system as set forth in Claim 6, wherem each voting node has an extent comprising a set of objects, and wherein the voting function returns a sum of an associated class value of each voting node, wherein the sum is normal ized by a number of objects within the voting node's extent, wherein the normalized sums across all voting nodes are then summed.

1.0. The system as set forth in Claim 9, wherein each voting node has an intent comprising a set of attributes, and wherein the associated class value for each voting node is weighted by a number of attributes in its intent.

11. The system as set Forth in Claim I , wherein the set of training data includes objects having attributes, and the FCA classification lattice is generated by treating the plurality of classifications as attributes of objects in the training data.

12. The system as set forth in Claim I, wherein the set of input data is acquired using at least one of an fMR! sensor, an image sensor, and a sound sensor, and wherein the

classification is performed for purposes of at least one of object recognition, image recognition, and sound recognition.

13. A computer-implemented method for data classification using formal concept analysis (FCA), the computer-implemented method using one or more processors to perform operations of:

generating with the one or more processors, in a training phase, a formal concept analysis (FCA) classification lattice using a set of training data and plurality of classifications corresponding to the set of training data; and

using the structure of the FCA classification lattice, generating classification of a set of input data during a classifi cation phase.

1 . The method as set forth in Claim 13, wherein the one or more processors further perform operations of:

generating the FCA classification lattice from the context table.

The method as set forth in Claim 14, wherein during generation of the FCA classification lattice the at least one class column is treated as a normal attribute, and wherein a sub- structure comprising a plurality of nodes within the FCA classi fication, lattice that is spanned by a given class-attribute is associated with the corresponding classification.

16. The method as set forth in Claim 13, wherein the one or more processors further perform 5 operations of:

generating, in the classification phase, a presentation context vector, ip, from the set of test data, wherein m_p is a set of attributes associated with a presentation p in the set of test data;

selecting, in the classification phase, a set of voting nodes in the FCA

10 classification lattice; and

using, in the classification phase, the set of voting nodes to vote for a

classification value for the presentation p.

17. A computer program product for data classification using formal concept analysis (FCA), i 5 the computer program product comprising:

computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having: one or more processors for causing the one or more processors to perform opera ions of:

generating with the on or more processors, in a training phase, a formal 0 concept analysis (FCA) classification lattice using a set of training data and a plurality of classifications corresponding to the set of training data; and

using the structure of the FCA classification lattice, generating a classification of a set of input data during a classification phase, 5

18. The computer program product as set forth in Claim 17, further comprising, instructions for causing the one or more processors to perform operations of:

generating, in the training phase, a context table .from the set of training data, the context table having rows of object, labels and columns of attribute labels:

for each training presentation, appending, in the training phase, at least one class 0 column for a classification corresponding to the training presentation to the context table; and generating the FC A classification lattice from the context table.

19. The compiiter program product as set forth in Claim 18, whereto during generation of the FCA classification lattice the at least one class column is treated as a normal attribute, and wherein a sub-structure comprising a plurality of nodes within the FCA classification lattice that is spanned by a given class-attribute is associated with the corresponding classification.

20. The computer program product as set. forth in Claim 17, farther comprising instructions for causing the one or more processors to perform operations of:

generating, in the classification phase, a presentation context vector., Trip, from the set of test data, wherein m_p is a set of attributes associated with a presentation p in the set of test data;

selecting, in the classification phase, a set of voting nodes in the FCA classificatio lattice; and

using, in the classification phase, the set of voting nodes to vote for a classification value for the presentation p.