JP7350549B2 - Determination device, determination method and determination program - Google Patents

Description

The present invention relates to a determining device, a determining method, and a determining program.

BACKGROUND ART Conventionally, a DNN (Deep Neural Network) technique is known in which input information is classified using a model in which layers each having a plurality of nodes are connected in multiple stages. For example, when predetermined input information is input to a model, the model is trained so that the output of the model approaches the output information corresponding to the input information, thereby classifying the input information according to desired characteristics. Techniques for learning models are known.

JP2016-006617A

However, the above-mentioned techniques do not take into account flexibility in the model, making it difficult to perform flexible inferences and predictions.

The present application was made in view of the above, and aims to provide a decision device, a decision method, and a decision program that can perform optimization according to the situation by constructing a flexible model. shall be.

The determining device according to the present application includes a first generating section, a second generating section, and a determining section. The first generation unit has a plurality of layers, and generates a mapping space that satisfies cohomology by regarding the connection relationship of a model in which each layer is connected to a node as an event. The second generation unit generates a category in category theory based on the mapping space generated by the first generation unit. The determining unit determines connection parameters between the layers based on the category equivalence relationship in the category theory generated by the second generating unit.

According to one aspect of the embodiment, by constructing a flexible model, optimization can be performed depending on the situation.

FIG. 1 is a diagram illustrating an example of a process executed by a determination device according to an embodiment. FIG. 2 is a block diagram of the determination device according to the embodiment. FIG. 3 is a conceptual diagram of cohomology. FIG. 4 is a flowchart showing the processing procedure executed by the determination device according to the embodiment. FIG. 5 is a hardware configuration diagram showing an example of a computer that implements the functions of the providing device.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Below, a mode for implementing a determining device, a determining method, and a determining program (hereinafter referred to as "embodiments") according to the present application will be described in detail with reference to the drawings. Note that the determination device, determination method, and determination program according to the present application are not limited by this embodiment. Further, in each of the embodiments below, the same parts are given the same reference numerals, and redundant explanations will be omitted.

[1. Information processing〕
First, an example of the process executed by the determination device will be described using FIG. 1. FIG. 1 is a diagram illustrating an example of a process executed by a determination device according to an embodiment. In FIG. 1, the determining device 10 is an information processing device that executes a determining process described below, and is realized by, for example, a server device, a cloud system, or the like.

More specifically, the determining device 10 is capable of communicating with any device via a predetermined network such as the Internet.

In recent years, DNN technology has become known as a model, in which layers containing multiple nodes are set up in multiple stages, nodes between each layer are connected via connection paths, and various connection coefficients are set for the connection paths. . In addition to such DNNs, layers containing nodes such as RNN (Recurrent Neural Network), LSTM (Long short-term memory), CNN (Convolutional Neural Network), and DSSM (Deep Structured Semantic Models) are configured in multiple stages. Various models are known.

These various models have layers called an input layer, a hidden layer (hidden layer), and an output layer, and the information input from the input layer (i.e., input information) is output while propagating through each connection path. Transmit to layers. At this time, the model generates output information corresponding to the input information by performing arithmetic processing based on the connection coefficients set for each connection path. Furthermore, each layer is composed of a plurality of neurons, and each neuron is a function of a matrix or the like.

By the way, in the conventional technology, giving flexibility to the model has not been considered, and it has been difficult to perform flexible inference and prediction.

Therefore, the decision device 10 abstracts the model M using the concept of category theory, and adds flexibility to the structure of the model M through flexible fitting using derived categories (derived equivalence). I decided to keep it. In other words, the determining device 10 calculates the connection relationship (i.e., cohomology) between each layer when adjacent layers are smoothly shifted, using the derived category, and considers those with the same cohomology connection structure to be equivalent. and determine connection parameters between layers. For example, the determining device 10 determines, as a connection parameter, a shift amount D indicating a connection relationship between a certain layer and an adjacent layer when a certain layer is used as a reference, and considers those having the same structure of the shift relationship to be equivalent.

In other words, the determining device 10 determines the equivalent structure based on the connection relationship between each layer by shifting the structure between the layers by the shift amount D, and even if the individual connection parameters are different, the connection structure is the same. If there is, it is assumed that they are the same value and learning is performed. This makes it possible to grasp the connection relationship between each layer in a more abstract concept, and to perform optimization with flexibility.

Specifically, first, the determination device 10 generates a mapping space that satisfies cohomology based on the model M (step S1). For example, the determination device 10 considers the complexes X and Y, which indicate the connection relationship between neurons in the layers constituting the model M, as "objects" in the structure of category theory, and determines the connection relationship between the "objects" as homotopic. By converting to equivalence, a mapping space that satisfies cohomology is generated.

ここで、（式１）および（式２）において、Ｘ^－１，Ｘ^０，Ｘ^１、Ｘ^２・・・およびＹ^－１，Ｙ^０，Ｙ^１、Ｙ^２・・・は、それぞれニューロ間の接続関係を示す関数であり、ｄ^－１、ｄ^０、ｄ^１・・・およびｄ´^－１、ｄ´^０、ｄ´^１・・・は、それぞれ対応する斜を示す。 When the complexes X and Y are regarded as "objects", each object is defined as shown in (Formula 1) and (Formula 2) below. It is assumed that complex X indicates the connection relationship of each neuron in layer L1, and complex Y indicates the connection relationship of each neuron in layer L2.

Here, in (Formula 1) and (Formula 2), X ^-1 , X ⁰ , X ¹ , X ² ... and Y ^-1 , Y ⁰ , Y ¹ , Y ² ... are respectively d ^-1 , d ⁰ , d ^{1 .} . . and d ^{' -1} , d' ⁰ , d' ¹ . . . each indicate the corresponding slope.

（式４）は、コホモロジーを満たす写像空間を示す。 At this time, the category formed by the complex X and the complex Y can be expressed by the following (Formula 3), and the following (Formula 4) is obtained by summarizing the category equivalence of (Formula 3) using homotopy.

(Formula 4) indicates a mapping space that satisfies cohomology.

Then, the determination device 10 generates a derived category based on the above (Equation 4) (step S2). The derived category is expressed by the following (Formula 5), and a correspondence relationship is established for all "n" in the following (Formula 6).

Further, the derived category can also be expressed by the following (Equation 7).

“Qis” shown in (Formula 7) corresponds to an image in cohomology, and “Hom _k(R) (·)” corresponds to a kernel in cohomology. That is, (Equation 7) indicates cohomology between complex X and complex Y.

Here, the derived category is the property of the triangular category disclosed in Non-Patent Document 1 (http://annals.math.princeton.edu/wp-content/uploads/annals-v166-n2-p01.pdf). have Therefore, by using the derived category, it is possible to calculate the connection relationship between the complexes X and Y, that is, the cohomology when the shift amount D is changed smoothly.

In this way, by generating the derived category, the determination device 10 can continuously grasp the cohomology between each layer and obtain equivalence relations based on the structure, so it can be used with flexibility. It becomes possible to determine connection parameters.

Since the determining device 10 can optimize the connection relationship based on the connection parameters, it becomes possible to theoretically optimize the model M. Furthermore, by providing flexibility to the model M, flexible inference and prediction become possible.

Therefore, according to the determination device 10 according to the embodiment, when creating new ideas, new sentences, images, etc. using DNN, it is possible to expand the range of data that are considered to be the same. As a result, it is possible to obtain results that are closer to those created using human-like soft thinking.

[2. Configuration of decision device]
Next, a configuration example of the determination device 10 according to the embodiment will be described using FIG. 2. FIG. 2 is a block diagram of the determination device 10 according to the embodiment. As shown in FIG. 2, the determination device 10 includes a communication section 20, a storage section 30, and a control section 40. The communication unit 20 is realized by, for example, a NIC (Network Interface Card). The communication unit 20 is connected to a predetermined network by wire or wirelessly, and transmits and receives information to and from an external device (not shown).

The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 30 also stores a model database 31.

The model database 31 is a database that stores models M. Note that the model database 31 may store a plurality of models M that are different from each other, or may store a single model M. Note that the model M may be acquired from an external device or may be learned by the determining device 10.

The control unit 40 is a controller, and for example, various programs stored in a storage device inside the determination device 10 are stored in a RAM or the like by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). This is achieved by executing it as a work area. Further, the control unit 40 is a controller, and may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

Further, as shown in FIG. 2, the control unit 40 includes a first generation unit 41, a second generation unit 42, and a determination unit 43. The first generation unit 41 generates a mapping space that satisfies cohomology by regarding the connection relationship of a model in which each layer is connected to a node as an event.

FIG. 3 is a conceptual diagram of cohomology. As shown in FIG. 3, cohomology indicates the connection relationship between the front and back in a network, and can be expressed as "kernel (Ker g)/image (Im f)".

The first generation unit 41 regards the complexes X and Y that indicate the connection relationships within each layer of the model M as objects in the structure of category theory, and calculates the category equivalence by performing the calculations up to (Equation 4) above. generates a mapping space organized by homotopy.

By the way, as described above, each component of the complexes X and Y is a function indicating the connection relationship between each neuron in each layer. That is, from another perspective, the complexes X and Y can be regarded as a set of points of contact between the functions indicated by the neurons in the corresponding complexes, and can be regarded as the differential space of the neurons. can.

In this way, by setting the object of the mapping space to be a differential space, it becomes possible to grasp the neuron with an abstract concept.

Note that the first generation unit 41 is disclosed in Non-Patent Document 2 (http://www2.math.kyushu-u.ac.jp/~m-kudo/kudo-combination-2015-2.pdf), for example. The concept of Gröbner bases may be used to generate a mapping space that satisfies cohomology.

Returning to the explanation of FIG. 2, the second generation unit 42 will be explained. The second generation unit 42 generates a category in category theory based on the mapping space generated by the first generation unit 41. Specifically, the second generation unit 42 generates the derived category shown by the above (Equation 7).

The determining unit 43 determines connection parameters between layers based on the category equivalence relationship in category theory generated by the second generating unit 42 . In other words, the determining unit 43 performs learning by assuming that even if the individual connection parameters are different due to the equivalence relation (derived equivalence) based on the derived category, if the connection structure is the same, the values are the same.

Thereby, the model M can be optimized with flexibility.

Here, a method for theoretically determining connection parameters will be described from another perspective. When the relationship between the complexes X and Y and the diagonal f is expressed as (Equation 8), the expression that represents the convolution of the two complexes X and Y can be expressed by the following (Equation 9).

Z shown in (Formula 9) corresponds to Cone (f), and is also called the "cone" of the complexes X and Y. Here, as shown in (Formula 9), the slope δ ⁿ between the union of complexes X and Y can be expressed as (Formula 10).

Note that in (Equation 10), d ⁿ represents the slope of the complex X, d ^′n+1 represents the slope of the complex Y, and f ⁿ⁺¹ represents the slope of the slope (d ⁿ , d ^′n+1 ). In other words, f ⁿ⁺¹ is a diagonal between diagonals (d ⁿ , d ^′n+1 ), and indicates a shift between the complexes X and Y.

Therefore, in (Equation 10), by determining "n" so that the slope δ ⁿ is minimized, the deviation in the connection mode between the complexes X and Y can be reduced. In other words, optimization of the model M can be theoretically realized by determining "n" so that the slope δ ⁿ is minimized.

Here, the oblique δ ⁿ is a discrete expression, but by deriving the derived category of the oblique δ ⁿ , it has the properties of a triangular category, and it is possible to capture the oblique δ ⁿ continuously. Become. In other words, (Formula 10) expressed in a derived category becomes the above (Formula 7).

In this way, by setting the connection parameters using the derived category, the determining unit 43 can determine the optimal connection parameters based on the slope δn when each layer is smoothly shifted. Become.

[3. Information processing flow]
Next, the processing procedure executed by the determination device 10 according to the embodiment will be described using FIG. 4. FIG. 4 is a flowchart showing the processing procedure executed by the determining device 10 according to the embodiment.

As shown in FIG. 4, the determination device 10 according to the embodiment first considers the connectivity of the model M as an event, generates a mapping space that satisfies cohomology (step S101), and based on the mapping space, A derived category is generated (step S102).

Next, the determination device 10 determines connection parameters between layers based on the equivalence relations in the derived category (step S103), optimizes the structure of the model M based on the connection parameters (step S104), and performs processing. end.

[4. Hardware configuration]
The determination device 10 according to the embodiments described above is realized, for example, by a computer 1000 having a configuration as shown in FIG. FIG. 4 is a hardware configuration diagram showing an example of a computer 1000 that implements the functions of the determining device 10. Computer 1000 has CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

CPU 1100 operates based on a program stored in ROM 1300 or HDD 1400, and controls each part. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, programs depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. Communication interface 1500 receives data from other devices via communication network 500 and sends it to CPU 1100, and sends data generated by CPU 1100 to the other devices via communication network 500.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and mouse via an input/output interface 1600. CPU 1100 obtains data from an input device via input/output interface 1600. Further, CPU 1100 outputs the generated data to an output device via input/output interface 1600.

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200. CPU 1100 loads this program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the determination device 10 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 40 by executing a program loaded onto the RAM 1200. Furthermore, data in the storage unit 30 is stored in the HDD 1400. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from another device via the communication network 500.

[5. effect〕
As described above, the determining device 10 according to the embodiment includes the first generating section 41, the second generating section 42, and the determining section 43. The first generation unit 41 considers the connection relationship of the model M, which has a plurality of layers and each layer is connected to a node, as an event, and generates a mapping space that satisfies cohomology. The second generation unit 42 generates a category in category theory based on the mapping space generated by the first generation unit 41. The determining unit 43 determines connection parameters between layers based on the category equivalence relationship in category theory generated by the second generating unit 42 .

Therefore, according to the decision device 10 according to the embodiment, by constructing a flexible model, it is possible to perform optimization according to the situation.

Furthermore, in the determination device 10 according to the embodiment, the first generation unit 41 determines connection parameters between layers using an equivalence relation of categories in category theory as an equivalence relation in a derived category.

Therefore, according to the determination device 10 according to the embodiment, it is possible to optimize the connection parameters between each layer with flexibility.

Furthermore, in the determination device 10 according to the embodiment, the determination unit 43 determines connection parameters between layers by using the equivalence relationship of categories in category theory as the equivalence relationship in the derived category.

Therefore, according to the determination device 10 according to the embodiment, the structure of the model M can be made flexible.

Furthermore, in the determination device 10 according to the embodiment, the first generation unit 41 generates a mapping space based on the Gröbner basis.

Therefore, according to the determination device 10 according to the embodiment, it is possible to appropriately generate a mapping space that satisfies cohomology.

[6. others〕
Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of the process can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

Furthermore, the processes described in the embodiments described above can be combined as appropriate within a range that does not conflict with the process contents.

Furthermore, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the first generation unit 41 can be replaced with a first generation means or a first generation circuit.

10 Determination device 20 Communication unit 30 Storage unit 31 Model database 40 Control unit 41 First generation unit 42 Second generation unit 43 Determination unit

Claims (6)

a first generation unit that generates a mapping space that satisfies cohomology by regarding the connection relationship of a model having a plurality of layers and each layer being connected to a node as an event;
a second generation unit that generates a category in category theory based on the mapping space generated by the first generation unit;
and a determining unit that determines connection parameters between the layers based on the category equivalence relationship in the category theory generated by the second generating unit. The determining unit is
The determination device according to claim 1, wherein the connection parameter between the layers is determined using an equivalence relationship in a derived category to an equivalence relationship of categories in the category theory. The first generation unit includes:
The method according to claim 1 or 2, characterized in that a complex indicating a connection relationship within the layer is regarded as an object in a structure of category theory, and the mapping space is generated in which equivalence classes between the objects are summarized by homotopy. Determination device as described. The first generation unit includes:
The determination device according to claim 1 or 2, wherein the mapping space is generated based on a Gröbner basis. A computer-implemented decision method, comprising:
a first generation step of generating a mapping space that satisfies cohomology by regarding the connection relationships of a model having a plurality of layers and each layer being connected to nodes as an event;
a second generation step of generating a category in category theory based on the mapping space generated by the first generation step;
and a determining step of determining a connection parameter between the layers based on the category equivalence relationship in the category theory generated by the second generating step. a first generation step of generating a mapping space that satisfies cohomology by regarding the connection relationship of a model having a plurality of layers and each layer being connected to a node as an event;
a second generation procedure that generates a category in category theory based on the mapping space generated by the first generation procedure;
A determination program that causes a computer to execute a determination procedure for determining a connection parameter between the layers based on an equivalence relationship of categories in the category theory generated by the second generation procedure.

Images