JP2017224264A - Data processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processing system capable of flexible data processing and improving a processing speed as compared to stream reasoning and the like.SOLUTION: A data processing system comprises a sensor actuator control module 14 and a service module 18. The sensor actuator control module 14 assigns unique key information to a data stream from a sensor 10, and stores the data stream in a Redis data store 16. The service module 18 infers key information corresponding to a query using ontology data stored in an RDF store 22, and selects a data stream having the corresponding key information as a first step. The service module performs data processing using the selected data as a second step.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a data processing system.

  In recent years, various data processing methods such as stream data middleware, composite event processing (CEP), and stream reasoning have been developed and proposed. In stream data middleware and complex event processing, real-time processing is performed by flowing data into a predefined analysis scenario in real time, but one context is assigned to one data, that is, a direct processing template for RAW data Since the process is applied, there is a disadvantage that there is no flexibility. In Stream Reasoning, processing is performed while inferring data based on RDF (Resource Description Framework) and Ontology, that is, RAW data is processed into data that can be inferred and processed into one data. Multiple ontologies can be assigned and data processing is flexible. However, if backward and forward inference is performed on RAW data, the amount of data processing increases and real-time performance may be reduced.

  In US Pat. No. 6,057,059, a user application and various content databases to be searched to simplify the creation of a user application for a mobile device that uses a location-based service that uses an ontology-based search system. A middleware system located between the two is described. The middleware system presents one or more services to the user application. For example, the service presents a service that allows a user to annotate and / or tag a known semantic location. The proposed semantic POI is selected based on the user's location and context-dependent information.

Special table 2013-507695 gazette

  An object of the present invention is to provide a data processing system that enables flexible data processing and has an improved processing speed compared to conventional techniques such as stream reasoning.

  According to the first aspect of the present invention, a storage means for storing ontology data and a data stream including key information uniquely assigned to the sensor data are used for a query using the ontology data stored in the storage means. A data processing system including a selection unit that selects a data stream having corresponding key information, and a processing unit that processes the selected data stream.

  According to the second aspect of the present invention, the selection unit is inferred from a receiving unit that receives a query, an acquisition unit that acquires ontology data from a storage unit, and an inference unit that infers key information using the query and ontology data. The data processing system according to claim 1, further comprising: a selection unit that selects a data stream using the key information.

  According to a third aspect of the present invention, the processing means includes an abstraction section that abstracts the selected data stream, a structuring section that structures the abstracted data stream, and ontology data stored in the storage means. The data processing system according to claim 1, further comprising: a processing unit that processes a structured data stream using the data.

  According to a fourth aspect of the present invention, the storage means stores a data processing routine template associated with ontology data, and the processing means processes the selected data stream with the data processing routine template. A data processing system according to any one of the above.

  The invention according to claim 5 is the data processing system according to any one of claims 1 to 4, wherein the ontology data is ontology data of a work procedure manual.

  In the invention described in claim 6, the ontology data is ontology data related to product assembly, the sensor data is sensor data related to product assembly, and the inference unit extracts an assembly operation element of product assembly using a query. The data processing system according to claim 2, wherein key information is inferred from the extracted assembly operation element.

  In the invention according to claim 7, the key information includes an ID that is arranged in the assembly process of the product assembly and uniquely identifies the sensor hub that receives the sensor data, and the inference unit identifies the ID from the extracted assembly operation element. The data processing system according to claim 1, wherein key information is inferred by inferring.

  The invention according to claim 8 is the processing means according to claim 1, wherein the processing means performs processing using data obtained by cutting the sensor data with a window having a predetermined time width and a learning result obtained by machine learning. A data processing system.

  According to a ninth aspect of the present invention, the ontology data is ontology data relating to product assembly, the sensor data is sensor data relating to product assembly, and the processing means includes product assembly sequence information obtained by processing the sensor data. 9. The data processing system according to claim 8, wherein an abnormality in product assembly is detected using product assembly sequence information obtained from ontology data.

  According to the tenth aspect of the present invention, the time information is included in the data obtained by cutting out the sensor data with a window having a predetermined time width, and the processing means includes sequence information obtained by processing the sensor data, and ontology data. The data processing system according to claim 9, wherein abnormality of product assembly time is detected using the obtained sequence information.

  According to the first aspect of the present invention, flexible data processing can be performed, and the processing speed can be improved as compared with conventional techniques such as stream reasoning.

  According to the second aspect of the present invention, the corresponding key information can be selected flexibly.

  According to the third aspect of the present invention, it is possible to further process data while inferring.

  According to the fourth aspect of the present invention, it is possible to quickly process data using a template.

According to the fifth aspect of the present invention, the data processing of the work procedure can be further performed.
According to the sixth to eighth aspects of the present invention, sensor data relating to product assembly can be further processed.

  According to the ninth aspect of the present invention, the sensor data can be further processed to detect abnormality in product assembly.

  According to the tenth aspect of the present invention, the sensor data can be further processed to detect an abnormality in the product assembly time.

It is a system configuration figure of an embodiment. It is a system configuration figure (2) of an embodiment. It is processing explanatory drawing of the real-time data acquisition part of embodiment. It is a functional block diagram of the real-time data acquisition part of an embodiment. It is a functional block diagram of the real time data processing part of an embodiment. It is a conceptual explanatory view of ontology data. It is a conceptual diagram of the data processing system using ontology data. It is ontology explanatory drawing of an assembly operation | movement element. It is arrangement | positioning explanatory drawing of a sensor and an actuator hub. FIG. 11 is an explanatory diagram for giving a key value. It is a figure which shows the relationship between an assembly process and an assembly manual. It is explanatory drawing which shows the inference of a key value. It is division | segmentation explanatory drawing of sensor data. It is learning explanatory drawing of sensor data. It is detailed learning explanatory drawing of sensor data. It is comparison explanatory drawing of sensor data and learning data. It is sequence information comparison explanatory drawing (the 1) of RAW data and ontology data. It is sequence information comparison explanatory drawing (the 2) of RAW data and ontology data. It is a figure which shows the specific example of an assembly procedure manual. It is ontology data explanatory drawing of an assembly procedure manual. It is ontology data explanatory drawing (the 1) of a work procedure. It is ontology data explanatory drawing (the 2) of a work procedure.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, an ontology that is a prerequisite for this embodiment will be described.

  Ontology is a structural framework for organizing information, and means representing them along a certain classification system, along with properties and relationships about entities, ideas, and events.

  FIG. 6 conceptually shows the ontology. Standard data 50, data 52 created for each area, and individual information 54 are collected, and relationships are described at the concept level, domain level, and individual information level, and structured at each level. By using structured data, it is possible for a computer to make judgments and process data as a human thinks. To build an ontology, a technique for structuring unstructured data such as dictionaries, standard data, and documents is required, and natural language processing and machine learning are used.

  FIG. 7 shows a conceptual configuration of a general processing system using an ontology.

  The dictionary, standard data, work procedure manual 60, domain knowledge, and the like are converted into knowledge information structured data, that is, ontology data 68 by natural language processing and machine learning.

  On the other hand, unstructured data such as the document 62, structured data such as the database data 64, sensor data from the sensor 66, and the like are structured by a document data structuring module, a database data conversion module, and a sensor data structuring module, respectively. , Converted into structured data (RDF) 67. Here, RDF (Resource Description Framework) is a framework for a method of expressing metadata.

  The structured data 67 and ontology data 68 are supplied to the inference engine, and the structured data 67 and the ontology data 68 are matched, thereby obtaining structured data 70 to which knowledge information is added. The structured data 70 to which the knowledge information is given is stored in the database 72 and used for various processes. Various types of processing can be performed by adding knowledge information. For example, a knowledge information visualization tool 74 is generated by extracting context (various information necessary for program execution), and data integration is performed according to the situation. For example, secondary data 76 is generated, semantic retrieval 78 is performed by semantic inference, and customer database data 80 is reconstructed by attribute integration.

  In such a processing system, as a reasoning using the structured data 67 and the ontology data 68, stream reasoning processing is known in which RAW data is processed into data that can be inferred and processed while inferencing. As described above, there is a problem that the amount of data processing increases and the real-time property decreases.

  Therefore, in the present embodiment, a method is used in which inference processing and RAW data processing are separated, inference processing is executed in the first step, necessary RAW data is selected, and RAW data selected in the second step is processed.

  Next, the data processing system of the present embodiment will be specifically described by taking sensor data from various sensors as an example. In addition, although this data processing system is applicable to what is called IoT (Internet of Things), it is not necessarily limited to this.

  In the present embodiment, “module” or “engine” means components such as software and hardware that can be logically separated. Therefore, the module in this embodiment means not only a module in a computer program but also a module in a hardware configuration. The module may correspond to the function 1: 1, but one module may be configured by one program, or a plurality of modules may be configured by one program. The plurality of modules may be executed by a single processor or computer, or may be executed by a plurality of processors or computers in a distributed or parallel environment. In the processing by each module, target information is read from the storage device, processed by a processor such as a CPU, and the processing result is output to the storage device and written. The storage device includes an HDD, a RAM, a register in the CPU, and the like.

  Also, “inference” in the present embodiment refers to giving knowledge information, that is, meaning to structured data by matching with ontology data.

<System configuration>
FIG. 1 shows a configuration diagram of a data processing system in the present embodiment. This system receives and processes signals from sensors and actuators. The data processing system includes a sensor / actuator control module 14, a Redis data store 16, a service module 18, a metadata extraction module 20, and an RDF store 22. The sensor / actuator control module 14, the service module 18, and the metadata extraction module 20 are realized as a computer program of a single computer, for example. Or you may implement | achieve as a computer program of a respectively different computer. The Redis data store 16 and the RDF store 22 are realized as databases accessible by a computer.

  The sensor / actuator control module 14 is connected to the sensor 10 and the actuator 12 so as to be able to transmit and receive data by wire or wirelessly. The sensor / actuator control module 14 includes a control library for controlling the sensor 10 and the actuator 12, and a Redis API that converts received data into a key / Value format in real time and stores it in the Redis data store 16. Here, Redis (Remote dictionary server) is software that can build a key-value store (KVS) on a memory, and KVS has a unique corresponding to any data (value) to be stored. This is a database in which signs are set and these are stored in pairs. Redis constructs KVS on the main memory of a computer, and enables data to be saved and read from an external program. Specifically, the Redis API generates a key information bit string using a predetermined algorithm based on the time and the device ID, and assigns the data to the data, thereby converting the data into a key / value format and using the Redis data store. 16. The Redis API stores a correspondence relationship between the generated key information and the device ID as a table. In addition, RedisAPI may generate a key information bit string using an encryption protocol such as DEC, MD5 hash, and SHA-1 hash, and add the key information bit string to the data. It may be generated and attached to the data.

  The Redis data store 16 sequentially stores data (RAW data) converted from the sensor / actuator control module 14 into the key / value format.

  The metadata extraction module 20 extracts metadata from key / value RAW data and structures it. The metadata extraction module 20 structures the RAW data into subject information, subject attribute information, target information, target attribute information, and environment information. The subject information is key information uniquely assigned. The structured data is stored in the RDF store 22.

  The RDF store 22 functions as storage means for storing ontology data, and stores ontology data separately constructed. Ontology data is created by structuring a dictionary or work procedure manual by natural language processing and machine learning. Natural language processing includes morphological analysis and dependency analysis. The ontology data is roughly divided into two areas of data. The first is common ontology data that can be used in common for individual applications such as basic human behavior information or sensor attribute libraries, and the second is individual information (created from work procedure manuals, etc.) Local ontology data composed of The RDF store 22 acquires a table that defines the correspondence between the key information and the device ID held by the sensor / actuator control module 14 and associates the key information with ontology data.

  The service module 18 receives the query and processes the RAW data in two steps according to ontology data stored in the RDF store 22. The service module 18 includes a real-time data acquisition unit 181 that executes a first step of selecting a data stream, and a real-time data processing unit 182 that executes a second step of processing the selected data stream.

  The real-time data acquisition unit 181 cuts out and acquires the stream data channel from the RAW data stored in the Redis data store 16 at an arbitrary timing. The real-time data acquisition unit 181 includes a query reception unit, a query rewrite unit, a Redis API, an inference engine, and an RDF API.

  The query receiving unit receives a query.

  The RDF API accesses the RDF store 22 and acquires ontology data.

  The inference engine infers key information from the query and ontology data. That is, a query desired to be processed in real time, for example, a motion analysis of a specific model and basic motion ontology data are matched, and key information matching the basic motion ontology is extracted. The query rewriting unit removes unnecessary information from the query based on the inferred key information and rewrites (rewrites) the query. Query rewrite is well known and is a technique for expanding a query into a primitive query.

  The Redis API selects a stream data channel having corresponding key information from the Redis data store 16 using a rewrite query.

  The real-time data processing unit 182 performs data processing on the selected RAW data.

  In the system of this embodiment, data processing is performed by the sensor / actuator control module 14 and the service module 18. However, a plurality of these modules may exist, and a plurality of sensor / actuator control modules 14 and a plurality of service modules 18 exist. In this case, these modules may operate in parallel or asynchronously.

  FIG. 2 shows a system configuration example in the case where a plurality of sensor / actuator control modules 14 and a plurality of service modules 18 exist.

  As the sensor / actuator control module 14, a sensor / actuator control module 14-1 (sensor / actuator control module 1) and a sensor / actuator control module 14-2 (sensor / actuator control module 2) are provided. Data from the sensor 10-1 (sensor 1) is supplied to the sensor / actuator control module 14-1, and data from the sensor 10-2 (sensor 2) is supplied to the sensor / actuator control module 14-2 (sensor / actuator control module). 2).

  A plurality of Redis data stores 16 are also provided in a master-slave connection format. Redis data store 16-1 (master Redis), Redis data store 16-2 (slave Redis1), Redis data store 16-3 (slave Redis2), Redis. A data store 16-4 (slave Redis3) and a Redis data store 16-5 (slave Redis4) are provided. The Redis data store 16-2 to Redis data store 16-5 are all connected to the Redis data store 16-1. The sensor / actuator control module 14-1 registers RAW data, for example, in the Redis data store 16-2, and the sensor / actuator control module 14-2 registers, for example, the RAW data in the Redis data store 16-3.

  On the other hand, as the service module 18, a service module 18-1 (service module 1) and a service module 18-2 (service module 2) are provided. The service module 18-1 accesses the Redis data store 16-4 and processes data in parallel and asynchronously with the other modules. The service module 18-2 also accesses the Redis data store 16-5 to access the other modules. And parallel / asynchronous data processing.

  Next, the first step executed by the real-time data acquisition unit 181 and the second step executed by the real-time data processing unit 182 will be described in more detail.

<First step>
FIG. 3 shows the flow of processing in the real-time data acquisition unit 181 of the service module 18. This is a process of inferring key information corresponding to a query and selecting a data stream corresponding to the inferred key information.

The Redis data store 16 stores key / value Raw data. In the figure, as an example of stored RAW data,
key1 / data stream 1
key2 / data stream 2
key3 / data stream 3
key4 / data stream 4
Indicates. The key 1 is key information uniquely assigned to the data stream 1, and is created based on, for example, time and device ID. The same applies to key2, key3, and key4.

The real-time data acquisition unit 181 cuts out the data stream having the corresponding key information according to the accepted query. For example, when the query is a motion analysis of a specific model, key information that matches the basic motion ontology data included in this context is inferred, and a data stream having the key information obtained by the inference is selected. Of the above RAW data, if key1 and key4 are the corresponding key information, the real-time data acquisition unit 181 reads from the Redis data store 16,
key1 / data stream 1
key4 / data stream 4
Select. FIG. 3 shows the selected RAW data as data 17.

  FIG. 4 shows a functional block diagram of the real-time data acquisition unit 181.

  The desired query received by the query receiving unit is supplied to the inference engine. Ontology data acquired from the RDF store 22 using the RDF API is supplied to the inference engine. The inference engine infers key information that matches ontology data based on the query and ontology data. The query rewriting unit removes noise from the inferred key information and rewrites the query. The Redis API uses a rewrite query to acquire a data stream having corresponding key information from the Redis data store 16 as a KVDB (key-value database).

<Second step>
FIG. 5 shows a functional block diagram of the real-time data processing unit 182 of the service module 18. In this step, the RAW data 17 selected in the first step is processed.

  The real-time data processing unit 182 temporarily converts the RAW data 17 into data that can be inferred and processes the data. Therefore, the real-time data processing unit 182 includes a CEP engine, and the CEP engine further includes a data window cutout module, a data abstraction module, a data pattern matching module, and a data labeling / time stamp module.

  The CEP engine is supplied with key1 / data stream 1 and key4 / data stream 4 selected by the real-time data acquisition unit 181.

  The data window cut-out module sets the windows (windows) while sequentially moving on the time axis, and sequentially supplies the data in the windows to the data abstraction module.

  The data abstraction module includes a frequency component extraction module, a data averaging module, and a machine learning module, and performs conversion processing such as frequency component conversion (Fourier transform) and average conversion on the data in the window to obtain the abstract data. Generate. A plurality of abstract data may be generated from one RAW data. The machine learning module is structured into four-factor fact data based on the extracted data. The quaternary fact data is specifically time information / subject information / attribute information / sensor data. As time information, time stamp information is assigned. As the main body information, information (ID or the like) of the sensor mounting main body is assigned. As the attribute information, attribute information defined in advance for the sensor or an attribute attached to the data conversion process is assigned. The sensor data is the abstracted data itself.

  The data pattern matching module infers by performing matching between structured abstract data and ontology data acquired from the RDF store 22.

  The generation of ontology data for inference is as follows, for example, using a work procedure manual as an example.

  First, the work procedure manual is digitized by OCR processing or the like. Next, the work procedure manual is subjected to natural language processing and converted into four-way rule data. That is, morphological decomposition of the work procedure manual is performed, and it is decomposed into nouns, verbs, etc., and in consideration of dependency, it is assigned and converted into the order ID / subject / predicate / object form in the order of appearance in the work procedure manual . At this time, the appearance frequency of the rule data may be evaluated and given as the fifth element data.

  Next, a series of operations are connected by attributes in the order of the work procedure, and a plurality of procedures are collected and labeled with a superordinate concept to create work procedure ontology data.

  The data labeling / time stamp module adds a label and a time stamp to the data matched by the data pattern matching module and registers the data in the database 70. Data registered in the database 70, that is, structured data to which knowledge information is given by ontology data is subjected to various processes.

  As described above, the real-time data processing unit 182 converts the RAW data into data that can be inferred and then processes the data. Instead, the data processing routine template that processes the RAW data is associated with the ontology data. In the first step, the RAW data is selected in the first step and at the same time the data processing routine template is selected and acquired from the RDF store 22, and in the second step, the RAW data is processed by the selected data processing routine template. Also good. Or you may perform combining both suitably.

The data processing method in the second step is summarized as follows.
(A) RAW data is converted into data that can be inferred, and data processing is performed while advancing the inference (B) RAW data is processed with the selected data processing routine template (C) (A) and (B ) Combinations

  In this embodiment, as a first step, a data stream to be processed is selected using key information (real time data acquisition), and as a second step, data processing is performed on the selected data stream (real time data processing). The process is performed in one step, and the data stream to be processed in real time is narrowed down to efficiently process the data. Therefore, if the ontology data in the work procedure manual is used as the ontology data, the production site data is analyzed in real time. Abnormalities can be detected in real time, which makes it possible to respond quickly. In addition, the cause of the abnormality can be analyzed by tracing back the data up to the occurrence of the abnormality. Also, it is possible to extract work differences between people and improve work. Furthermore, the work can be continuously improved by feeding back the work state to the worker.

  Next, the process of this embodiment will be described more specifically.

FIG. 8 shows ontology data of an assembling operation element of an image forming apparatus having functions of a certain product, for example, a copy, a printer, and a fax machine, as an example of ontology data.
The assembly operation element is composed of what, where, and how. How, what is the part name, where is where, and how is movement. Correspond. There may be sensor data for each of these.

  On the other hand, as ontology data, an image forming apparatus component part ontology 30 describing information on parts of the image forming apparatus, a standard book ontology 32 describing a general configuration of the image forming apparatus, and arrangement information of parts of the image forming apparatus are described. The image forming apparatus arrangement configuration ontology 34, the 3D CAD data 36 of the image forming apparatus, and the basic operation ontology 38 describing the basic operation of the assembly operator when assembling the image forming apparatus are prepared. The part name of the assembly operation element is inferred using the standard book ontology 32, and corresponding sensor data is extracted. Further, the position of the assembly operation element is inferred using the arrangement configuration ontology 34 in the image forming apparatus and the 3D CAD data 36, and the corresponding sensor data is extracted. Furthermore, the motion of the assembly operation element is inferred using the basic operation ontology 38, and sensor data corresponding to this is extracted.

  FIG. 9 shows a physical arrangement of a sensor / actuator hub as an example of the sensor / actuator control module 14.

  The assembly line of the image forming apparatus is composed of a plurality of assembly processes, and only the assembly process 1, the assembly process 2, and the assembly process 3 are illustrated in the drawing. A sensor 10 is installed in each of the assembly process 1, the assembly process 2, and the assembly process 3. On the other hand, an assembly worker exists in each assembly process, and a sensor 10 for detecting the assembly worker is also installed. Data detected by these sensors is supplied to a sensor / actuator hub 141 installed for each of a plurality of processes. The sensor / actuator hub 141 may be installed for each process. A unique address is given to the sensor / actuator hub 141, and it is possible to specify which sensor / actuator hub 141 is present when there are a plurality of sensor / actuator hubs 141.

  Upon receiving data from each sensor 10, the sensor / actuator hub 141 converts the received data into a key / value format in real time and stores it in the KVS.

  FIG. 10 schematically shows processing of the sensor / actuator hub 141. The RAW data from the sensor 10 is transmitted to the sensor / actuator hub 141. The sensor / actuator hub 141 gives a timestamp (timestamp) and ID (address of the sensor / actuator hub 141) as a key value, and as a value value. The RAW data is converted into a key-value format and stored in the KVS 142. As described above, the KVS 142 is a database that stores a pair of unique markers (keys) corresponding to arbitrary data to be stored, and is configured on a computer memory by a Redis API.

  FIG. 11 schematically shows the relationship between the assembly process and the assembly instructions. As shown in FIG. 9, the line is composed of a plurality of assembly steps 1, assembly step 2, assembly step 3,..., For example, assembly step 1 includes an assembly instruction to be executed in this assembly step. Correspond. There may be a plurality of assembly instructions as shown in the figure. In the assembly manual, a plurality of assembly operation elements are described in time series.

Assembly operation element 1 → assembly operation element 2 → assembly operation element 3
Etc. As shown in FIG. 8, each assembly operation element defines what (Where), where (Where), and how (How), and these correspond to the part name, position, and operation, respectively. .

  This means that if an assembly operation element is obtained, an assembly instruction corresponding to the assembly operation element is obtained, and if an assembly instruction is obtained, an assembly process corresponding to the assembly instruction is obtained. . In the assembly process, the sensor / actuator hub 141 is installed one-to-one or many-to-one (see FIG. 9), and if the corresponding assembly process is obtained, the sensor / actuator hub 141 corresponding to the assembly process is obtained. The actuator hub 141 is obtained.

  FIG. 12 schematically shows the inference of the Key value executed by the inference engine of the real-time data acquisition unit 181.

  Assembling operation elements are detected and extracted from the required technical standards, parts, and regions and operations in 3D CAD using SPARQL (S1). Here, SPARQL is a query language for searching / manipulating data described in RDF (Resource Description Framework).

  Next, an assembly manual corresponding to the retrieved assembly operation element is extracted (S2).

  Next, the assembly process corresponding to this is extracted from the extracted assembly manual (S3).

  Next, the address of the sensor / actuator hub 141 corresponding to the extracted assembly process is specified (S4).

  Next, a key value is extracted from the specified address (S5). As described above, since the key value is composed of a time stamp and an ID (address), the key value corresponding to the key value can be uniquely obtained by designating the address.

  When the key value is inferred as described above, the real-time data acquisition unit 181 cuts out the data stream having the corresponding key value from the Redis data store 16 and supplies the data stream to the real-time data processing unit 182.

13 and 14 schematically show processing in the real-time data processing unit 182. The assembly instructions include a plurality of assembly operation elements Assembly operation element 1 → assembly operation element 2 → assembly operation element 3
Is specified. On the other hand, when there is information such as video corresponding to each assembly operation element, the sensor data obtained from the sensor 10 is divided by using this information, and each assembly operation element is associated with the sensor data. For example, the sensor data is divided into sensor data 1, sensor data 2, and sensor data 3 using video,
Assembly operation element 1 ... Sensor data 1
Assembly operation element 2 ... sensor data 2
Assembly operation element 3 ... sensor data 3
Associate with.

Next, as shown in FIG. 14, learning data 192 classified into a plurality of categories is obtained by machine learning using the divided sensor data (sensor data 1, sensor data 2, sensor data 3) as teacher data 190. obtain. Specifically, the similarity with each classification result is calculated as vector data. For example, class 1, class 2, and class 3 are classified.
Classification 1 = [: Assembly operation element 1 [0.1, 0.2],: Assembly operation element 2 [0.3, 0.2],: Assembly operation element [0.8, 0.7]]
Etc. Here, the degree of similarity with the assembly operation element 1 is shown as a vector [0.1, 0.2]. The same applies to the degree of similarity with the assembly operation elements 2 and 3. Learning data 192 obtained by machine learning is a comparison target of actual sensor data.

  FIG. 15 shows the machine learning method in more detail. The teacher data 190 including the sensor data 1, the sensor data 2, the sensor data 3,... Is cut into sensor data having a predetermined time width by sliding a preset window W on the time axis.

Next, the sensor data cut out in a predetermined time width is supplied to the preprocessing unit 191-1 and subjected to various preprocessing such as filter processing (high frequency filter, low frequency filter, etc.), differentiation / integration processing, edge detection, and the like. It is executed and supplied to the feature extraction unit 191-2. The feature extraction unit 191-2 extracts features of the sensor data waveform by FFT (Fast Fourier Transform), peak detection, zero cross count, and the like. The extracted features are further supplied to the learning algorithm unit 191-3. In the learning algorithm unit 191-3, a known machine learning algorithm, for example, a neural network, a deep neural network, a recurrent neural network, or the like can be used. As a classification method,
Adaptive Naive Bayes Classifier
・ K-Nearest Neighbor
・ AdaBoost
・ Decision Trees
・ Dynamic Time Wrapping
・ Support Vector Machine
・ Hidden Markov Model
Gaussian Mixture Model
Etc. can be used. Learning data 192 is obtained by machine learning in the learning algorithm unit 191-3.

  FIG. 16 schematically shows a comparison process between the sensor data sequentially obtained from the sensor 10 and the learning data 192. Similar to the teacher data 190, the sensor data is cut into sensor data having a predetermined time width by sliding a preset window W on the time axis. Then, the cut sensor data is compared with the learning data 192 to generate vector data.

For example, the cut sensor data is compared with the learning data 192, and the ratio of each classification is [: Category 1 0.3: Category 2 0.1: Category 3 0.6]
The sensor data clipped using the similarity to the assembly motion element in each category is
[: Assembly operation element 1 [0.3, 0.2]: assembly operation element 2 [0.1, 0.2]: assembly operation element 3 [0.6, 0.7]]
And so on.

By generating vector data in this way and generating vector data sequentially for the cut sensor data, a matrix as a set of these is generated.
For example, from three of the cut sensor data,
[: Assembly motion element 1 [0.3,0.2]: Assembly motion element 2 [0.1 0.2]: Assembly motion element 3 [0.6, 0.7]]
[: Assembly motion element 1 [0.3,0.2]: Assembly motion element 2 [0.1 0.2]: Assembly motion element 3 [0.6, 0.7]]
[: Assembly motion element 1 [0.3,0.2]: Assembly motion element 2 [0.1 0.2]: Assembly motion element 3 [0.6, 0.7]]
A matrix is generated. In these matrices, the assembly operation element having the highest similarity is extracted.

[: Assembly motion element 1 [0.3, 0.2]: Assembly motion element 2 [0.1 0.2]: Assembly motion element 3 [0.6, 0.7]] The highest similarity is the assembly motion element 3 [0.6, 0.7], the assembly operation element 3 is extracted from each vector data. Then, the extracted assembly operation element is reconstructed as sequence information,
Assembly operation element 3 → assembly operation element 3 → assembly operation element 3
Is finally obtained.

  As described above, when the assembly operation element as the sequence information is extracted from the sensor data, the assembly operation along the ontology data is compared with the sequence information obtained from the ontology data of the image forming apparatus assembly. It is possible to detect whether or not, that is, an abnormality during product assembly.

FIG. 17 schematically shows matching between sensor data and assembly information. The sequence information 200 reconstructed from the data obtained by cutting the sensor data (RAW) data from the sensor 10 in the window W is compared with the sequence information 202 reconstructed from ontology data of the image forming apparatus assembly, and the ontology is compared. The closest one is extracted from the sequence information 202 reconstructed from the data. In the case of FIG. 17, the sequence information 200 obtained by reconstructing from the RAW data is
Assembly operation element 3 → assembly operation element 3 → assembly operation element 3
The sequence information 202 that is closest to this is the assembly operation element 3. Therefore, the RAW data can be determined to mean the assembly operation element 3 in the ontology data. In the figure, the correspondence between the sequence information 200 and the sequence information 202 is indicated by a broken line.

On the other hand, FIG. 18 shows another example of sequence information 200 reconstructed from RAW data. In this example,
Assembly operation element 3 → assembly operation element 1 → assembly operation element 3
The assembly operation element 1 is included between the two assembly operation elements 3. Originally, as a result of comparison with the sequence information 202 obtained by reconstructing from ontology data, the assembly operation element 3 should be included. If an assembly operation element 1 different from this is included, the RAW data is The meaning assembly operation element 1 can estimate that some abnormality has occurred and can output the abnormality.

  In addition, when cutting sensor data in the window W, a time stamp may be given for each window, and time information may be given to each of the cut sensor data, thereby detecting an abnormality in the product assembly time. it can. Referring to FIG. 17, time information is given to each of the assembly operation elements 3 constituting the sequence information 200 reconstructed from the RAW data.

Then, time information is given to the sequence information 200,
Assembly operation element 3 → assembly operation element 3 → assembly operation element 3
The time required for a series of sequences is obtained.

  On the other hand, if time information is similarly given to the sequence information 202 reconstructed from ontology data, the time required for both is compared, and the time required for the ontology data is compared with the time required for the ontology data, or the RAW data. The degree of variation in the required time can be evaluated. When the required time of RAW data is excessively long, there is an abnormality in the product assembly time, and when the variation in the required time of RAW data itself is excessively large, it can be determined that the product assembly time is abnormal. About the variation in required time, the skill level for every assembly worker can also be evaluated.

  As described above, by comparing the assembly operation element obtained from the sensor data with the assembly operation element obtained from the ontology data, it is evaluated whether the assembly operator is assembling the product in the correct procedure or the desired required time. The result can be output. By feeding back such output to the assembly operator, more efficient and reliable product assembly can be achieved.

FIG. 19 shows a specific example of an assembly procedure manual for the image forming apparatus (product A). A schematic diagram and a parts table are described, and a work procedure is further described. The work procedure is
1. Attach the part-1 to the 1-1 position on the right side of the frame as shown in the figure. 2. Temporarily fix the part-1 with the screw 2. 3. At the 1-2 position on the right side of the frame, as shown in the figure. Install part-2. 4. Fix part-2 temporarily with screw 2. 5. Adjust part position 6. Fix part-1 with screw 1. 7. Fix part-2 with screw.

FIG. 20 shows ontology data of the assembly procedure manual shown in FIG.
Page 1 (page 1) of the assembly procedure manual includes a schematic diagram, a parts table, and a work procedure. The schematic diagram includes original 3D CAD data, and the 3D CAD data includes a frame and components.

  The parts table includes a part 1-1, a part 1-2, a screw 1, and a screw 2.

  The work procedure includes a specific work procedure.

  FIG. 21 shows ontology data of a work procedure.

The work procedure includes a procedure text and a procedure. The procedure includes procedures 1 to 7, and each procedure is in accordance with the work procedure. That is,
Procedure 1: Attach part-1 to position 1-1 on the right side of the frame shown in the figure Procedure 2: Temporarily fix part-1 with screw 2 Procedure 3: 1-2 on the right side of the frame shown in the figure The part-2 is attached to the position Step 4: The part-2 is temporarily fixed with the screw 2. The part position adjustment procedure 6: The part-1 is fixed with the screw 1. The part-2 is fixed with the screw.

  FIG. 22 is still another example of work procedure ontology data.

Step 1: Install part-1 at position 1-1 on the right side of the frame shown in the figure, and perform natural language processing, that is, morphological analysis and dependency analysis.
Frame / right side / 1-1 position / part-1 / attachment analysis.

  When “frame” and “component-1” are included as the component ontology data of the image forming apparatus (product A), the “frame” in the ontology data is matched with the “frame” obtained by natural language processing. And “part-1” in the ontology data is matched with “part-1” obtained by natural language processing.

  Similarly, when “installation” and “screw tightening” are included as the operation ontology data of the image forming apparatus (product A), “attachment” in this ontology data and “attachment” obtained by natural language processing To match.

  If the position ontology data of the image forming apparatus (product A) includes “frame”, “right side”, and “1-1 position”, natural language processing is performed on the “right side” in the ontology data. The “right side surface” obtained in the above is matched, and “in the 1-1 position” in the ontology data is matched with “1-1 position” obtained by natural language processing. In the figure, the dashed-dotted arrows schematically show these matching processes.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, A various deformation | transformation is possible.

  For example, in the present embodiment, the sensor / actuator control module 14 assigns unique key information to the sensor data from the sensor 10 to generate key / value RAW data. However, the sensor 10 assigns unique key information. The sensor / actuator control module 14 may be supplied by wire or wirelessly. The sensor / actuator control module 14 stores the received key / value RAW data in the Redis data store 16.

  In this embodiment, one key information is assigned to one data stream, but one key information may be assigned to a plurality of data streams.

  In this embodiment, in the second step, the CEP engine processes the data stream to which the key information is assigned. However, in addition to the data stream to which the key information is assigned, the data stream to which no key information is assigned. May also be supplied to the CEP engine and processed together in the CEP engine.

  Furthermore, in this embodiment, the service module 18 infers key information and selects a data stream having key information corresponding to the inferred key information. However, the service module 18 supplies the accepted query to the RDF store 22. Then, the key information may be inferred in the RDF store 22 and returned to the service module 18, and the service module 18 may select a data stream having the key information corresponding to the acquired key information. In this case, it can be said that the first step is executed in cooperation with the service module 18 and the RDF store 22.

10 sensors, 12 actuators, 14 sensor / actuator control modules, 16 Redis data stores, 18 service modules, 20 metadata extraction modules, 22 RDF stores, 181 real-time data acquisition units, 182 real-time data processing units.

Claims (10)

Storage means for storing ontology data;
Selecting means for selecting a data stream having key information corresponding to a query using ontology data stored in the storage means for a data stream including key information uniquely assigned to the sensor data;
Processing means for processing the selected data stream;
A data processing system comprising: The selection means is
A reception unit for receiving a query;
An acquisition unit for acquiring ontology data from the storage means;
An inference unit that infers key information using a query and ontology data;
A selector that selects a data stream using the inferred key information;
A data processing system according to claim 1. The processing means is
An abstraction section that abstracts the selected data stream;
A structuring unit for structuring the abstracted data stream;
A processing unit for processing a structured data stream using ontology data stored in the storage means;
A data processing system according to any one of claims 1 and 2. The storage means stores a data processing routine template associated with ontology data,
The processing means processes the selected data stream with a data processing routine template.
The data processing system according to claim 1. Ontology data is the ontology data of the work procedure manual.
The data processing system in any one of Claims 1-4. Ontology data is ontology data related to product assembly,
Sensor data is sensor data related to product assembly,
The inference unit extracts the assembly operation element of the product assembly using the query,
The data processing system according to claim 2, wherein key information is inferred from the extracted assembly operation element. The key information includes an ID that uniquely identifies a sensor hub that is arranged in the assembly process of product assembly and receives sensor data.
The data processing system according to claim 1, wherein the inference unit infers key information by inferring an ID from the extracted assembly operation element. The data processing system according to claim 1, wherein the processing means performs processing using data obtained by cutting the sensor data with a window having a predetermined time width and a learning result obtained by machine learning. Ontology data is ontology data related to product assembly,
Sensor data is sensor data related to product assembly,
The data processing system according to claim 8, wherein the processing means detects product assembly abnormality using product assembly sequence information obtained by processing sensor data and product assembly sequence information obtained from ontology data. Time data is included in the data obtained by cutting out the sensor data in a window with a predetermined time width.
The data processing system according to claim 9, wherein the processing means detects abnormality in the product assembly time using sequence information obtained by processing sensor data and sequence information obtained from ontology data.