JP7268121B2 - Information processing device, method and program - Google Patents

Description

The present embodiment relates to an information processing device, method, and program.

In general, at a manufacturing site, training, education, and instruction for workers to acquire necessary skills are essential to ensure productivity. In addition, due to the rapid turnover of personnel, such as the expansion of overseas factories and the employment of foreigners due to the recent fluidity and globalization of the labor market, it has become important how quickly workers with little practical experience can be trained and sent to the site. is coming. Therefore, there is a demand for a technology that can efficiently promote the training of workers.

Japanese Patent Application Laid-Open No. 2001-166681 JP 2017-64095 A

A problem to be solved by the embodiments of the present invention is to provide an information processing apparatus, method, and program capable of promoting training of workers.

An information processing device according to an embodiment communicates with multiple types of sensor devices. The information processing apparatus includes first acquisition means, second acquisition means, third acquisition means, estimation means, determination means, presentation means, selection means, and second storage means. The first acquisition means acquires, from each of the sensor devices, first motion data indicating a motion when the first worker performs a predetermined work, and also acquires the type of the sensor device, the measurement site, and the predetermined work. Accompanying data indicating at least the contents of the work are respectively obtained. The second acquisition means acquires, from each of the sensor devices, second motion data representing motions of a second worker who is more proficient than the first worker when performing the predetermined work, and obtains second motion data from each of the sensor devices. Get the data respectively. The third acquisition means acquires proficiency level data indicating a proficiency level of the first worker for the predetermined work. The estimating means analyzes each of the first motion data and each of the second motion data, and determines one or more types of motions by the first worker and the second worker indicated by the motion data. Estimation is performed, and estimation result data capable of identifying the type of the estimated motion is added to each motion data. The determination means compares each of the first motion data and each of the second motion data to determine similarity. The presenting means presents to the first worker instruction data indicating points to be improved regarding actions in performing a predetermined work according to the result of determination by the determining means. The selection means selects instruction data to be presented to the first worker from a first storage means for storing a plurality of instruction data according to a determination result by the determination means. The second storage means associates and stores the accompanying data, the second action data, and the proficiency level data. The determination means determines the second action data associated with the acquired proficiency level data and the accompanying data acquired together with the first action data based on the accompanying data and the proficiency level data. Acquired from the second storage means as a comparison target of the first motion data, and extracting from each of the first motion data and the second motion data, portions added with estimation result data indicating the same type of motion , comparing the motion data, detecting a difference between the first motion data and the second motion data, and a matching portion of the first motion data and the second motion data to determine the similarity of the motion data; The result of the determination is used as determination result data.

FIG. 1 is a diagram showing a schematic configuration example of a motion improvement promotion system including an information processing device according to the first embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus according to the embodiment; FIG. 3 is a block diagram showing an example of the functional configuration of an operation improvement promotion program executed by the information processing apparatus according to the embodiment; FIG. 4 is a diagram showing an example of subject motion data according to the embodiment. FIG. 5 is a diagram showing an example of expert motion data according to the embodiment. FIG. 6 is a diagram exemplifying a case where subject motion data and expert motion data are superimposed and compared according to the embodiment. FIG. 7 is a diagram showing an example of the data structure of performance improvement instruction data according to the embodiment. FIG. 8 is a flowchart showing an example of an operation improvement promotion process according to the embodiment; FIG. 9 is a block diagram showing an example of a functional configuration of an operation improvement promotion program different from that shown in FIG. FIG. 10 is a diagram showing an example of the data structure of motion data according to the embodiment. FIG. 11 is a diagram showing an example of a data structure of motion data different from that in FIG. FIG. 12 is a block diagram showing an example of a functional configuration of an operation improvement promotion program different from that shown in FIG. FIG. 13 is a block diagram showing an example of the functional configuration of an operation improvement promotion program according to the second embodiment. FIG. 14 is a diagram showing an example of subject motion data according to the embodiment. FIG. 15 is a diagram exemplifying a case where subject motion data and expert motion data are superimposed and compared according to the embodiment. FIG. 16 is a diagram showing an example of a functional configuration of an operation improvement promoting program different from that shown in FIG. FIG. 17 is a diagram showing an example of the data structure of adverb expression data according to the embodiment. FIG. 18 is a diagram for explaining generation of an operation improvement instruction according to the embodiment;

Embodiments will be described below with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration example of a motion improvement promotion system including an information processing device according to the first embodiment. The performance improvement promotion system shown in FIG. 1 includes a plurality of sensor devices 11 to 14, a data transmitter/receiver 15, and an information processing device 16.

The plurality of sensor devices 11 to 14 are sensors capable of measuring motion data related to the motion of a person (user) when performing a predetermined task. Examples include the wearable sensors 11 to 13, the pressure sensor 14, and the like. These sensor devices 11 to 14 are attached to a person who performs a predetermined work or to a tool or the like used by the person to perform the predetermined work. Various motion data measured by each of the sensor devices 11 to 14 are data together with accompanying data indicating the type of sensor that measured the motion data (sensor type) and the site where the sensor was attached (mounted site, measurement site). It is transmitted to the information processing device 16 via the transceiver 15 . Incidentally, the accompanying data is set by, for example, the wearer (user) when the sensor devices 11 to 14 are worn.

In addition, the sensor device is not limited to the wearable sensors 11 to 13 and the pressure sensor 14 described above, and any various sensors can be used as the sensor device as long as they can measure motion data related to the motion of a person performing a predetermined work. May be used. For example, an imaging device (camera) capable of continuously photographing the transition of the motion of a person performing a predetermined task may be used as an additional sensor device or as a sensor device in place of the sensor devices 11-14.

In this embodiment, as shown in FIG. 1, each of the sensor devices 11 to 14 is composed of a person UL who has a low proficiency level for a predetermined task (hereinafter referred to as a "subject") and a person _UL who is proficient in the predetermined task. Right hand (right arm), left hand (left arm) _{, abdomen, test subject UL and} expert _UH with a highly skilled person (hereinafter referred to as "expert") UH assume. Further, in this embodiment, it is assumed that the above-described predetermined work is "screw tightening". However, the attachment sites of the sensor devices 11 to 14 and the contents of the predetermined work are not limited to those described above.

The information processing device 16 transmits, via the data transmitter/receiver 15, motion data of the subject measured by the sensor devices 11 to 14 (hereinafter referred to as "subject motion data") and expert motion data (hereinafter referred to as "expert operation data") is acquired, based on the subject operation data and the expert operation data, items to be determined and selected in order to increase the proficiency of the predetermined work, and the results are used in the operation. A motion improvement prompting process to be presented to the subject _UL as an improvement instruction is executed. The motion data of the subject and the motion data of the expert may be raw motion data measured by the sensor devices 11 to 14, or may be motion data obtained by processing the raw motion data. Also good.

FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing device 16. As shown in FIG.

As shown in FIG. 2, the information processing device 16 includes a CPU 21, a system controller 22, a main memory 23, a graphics controller 24, a BIOS-ROM 25, a nonvolatile memory 26, a wireless communication device 27, an embedded controller (EC) 28, and the like. , for example, a personal computer (PC). In this embodiment, the information processing device 16 is assumed to be a PC, but is not limited to this, and the information processing device 16 may be a tablet computer, a smart phone, various wearable devices, or the like. .

The CPU 21 is a processor that controls operations of various modules in the information processing device 16 . The CPU 21 executes various software loaded from the nonvolatile memory 26 to the main memory 23 . These software include an operating system (OS) and various application programs. The application programs include an operation improvement promotion program, and the operation improvement promotion process is realized by the CPU 21 executing the operation improvement promotion program.

CPU 21 also executes a basic input/output system (BIOS) stored in BIOS-ROM 25 . BIOS is a program for hardware control.

The system controller 22 is a device that connects between the local bus of the CPU 21 and various components. The system controller 22 also incorporates a memory controller for controlling access to the main memory 23 . The system controller 22 also has a function of communicating with the graphics controller 24 via a PCI EXPRESS standard serial bus or the like.

The graphics controller 24 is a display controller that controls a display monitor (LCD) provided in the information processing device 16 and a display monitor connected to the information processing device 16 .

The wireless communication device 27 is a device configured to perform wireless communication such as wireless LAN or 3G/4G mobile communication. The EC 28 is a one-chip microcomputer including an embedded controller for power management, and has a function of powering on/powering off the information processing device 16 according to user's operation.

FIG. 3 is a block diagram showing an example of the functional configuration of the performance improvement promotion program.

The motion improvement promotion program includes, as functional units, a subject data acquisition unit 101, an expert data acquisition unit 102, a similarity determination unit 103, a motion improvement instruction selection unit 104, a presentation unit 105, and the like. The functional units 101 to 105 will be described in detail below, and the operation improvement instruction database 201 will also be described in detail. The operation improvement instruction database 201 is configured by a storage device within the information processing apparatus 16, a storage device externally attached to the information processing apparatus 16, or a cloud server apparatus that provides storage services or the like.

The subject data acquisition unit 101 acquires subject movement data and associated data associated with the subject movement data from the sensor devices 11a to 14a attached to the subject _UL . More specifically, the subject data acquisition unit 101 acquires subject movement data attached with identification data indicating that the sensor devices 11a to 14a are attached to the subject _UL , and accompanying data attached to the subject movement data. get. For example, _when a myoelectric sensor is worn on the right hand of the subject _UL , the subject data acquisition unit 101 acquires the muscle strength of the right hand as shown in FIG. Time-series data of the amount of activity (relative myoelectric output) is acquired as subject motion data.

It should _be noted that, here, a case where a myoelectric sensor is worn on the right hand of the subject _UL , and the subject movement data and accompanying data are acquired from the myoelectric sensor, is described. (For example, myoelectric sensors attached to the left hand and abdomen, pressure sensors attached to tools used by the subject _UL , etc.) similarly acquire subject movement data and associated data. At this time, it is preferable that the subject data acquiring unit 101 acquires the subject motion data from each of the sensor devices 11a to 14a after synchronizing the timing of acquiring the subject motion data from each of the sensor devices 11a to 14a.

Similarly to the subject data acquisition unit 101, the expert data acquisition unit 102 acquires expert operation data (in other words, expert operation data to which the above identification data is not attached) from the sensor devices 11b to 14b attached to the expert _UH . motion data) and its accompanying data. It is preferable that the expert data acquisition unit 102 also acquires the expert action data from each of the sensor devices 11b to 14b after synchronizing the timing of acquiring the expert action data from each of the sensor devices 11b to 14b. FIG. 5 shows an example of time-series data of the amount of muscle activity of the right hand acquired from the myoelectric sensor worn on the right hand of the expert _UH .

The motion data acquired by the subject data acquisition unit 101 and the expert data acquisition unit 102 and the accompanying data associated with the motion data are output to the similarity determination unit 103 .

When the similarity determination unit 103 receives the input of the subject motion data output from the subject data acquisition unit 101 and the expert motion data output from the expert data acquisition unit 102, the similarity determination unit 103 determines the accompanying data input together with the motion data. The motion data of the subject and the motion data of the expert having the same sensor type and wearing site indicated by the data are compared to determine the similarity of the motion data.

More specifically, when the motion data is time-series data as shown in FIGS. 4 and 5, the similarity determination unit 103 superimposes the motion data of the subject and the motion data of the expert as shown in FIG. Then, focusing on various items such as amplitude, frequency, phase, and displacement in order, the similarity of motion data is determined. For example, when focusing on the amplitude, the similarity determination unit 103 determines whether or not a difference equal to or larger than a predetermined value is detected between the amplitude of the maximum peak of the motion data of the subject and the amplitude of the maximum peak of the motion data of the expert. . Note that the predetermined value is set in advance for each item, and the similarity determination unit 103 performs similar processing for each item.

In the similarity determination unit 103, in order to accurately superimpose the motion data of the subject and the motion data of the expert, it is necessary to synchronize the motion data of the subject and the motion data of the expert. As a synchronization method, there is a method of detecting an abrupt rise in data common to two motion data and aligning the positions of the rise times. For example, if one of the sensor devices is a 3-axis acceleration sensor that can acquire three types of sensor signals (time-series signals in the x-axis, y-axis, and z-axis directions), the square of the signal per unit time can be used for calculation. The time when the value obtained by adding the power of the three sensor signals increases by a certain threshold value or more is regarded as the rise time, and the two operation data are synchronized. According to this, the motion data of the subject and the motion data of the expert measured at different times can be accurately superimposed.

As a result of the above determination, if there is an item for which a difference equal to or greater than a predetermined value is detected, the similarity determination unit 103 determines that subject _UL has a motion that can be improved, and detects a difference equal to or greater than the predetermined value. and determination result data indicating what kind of difference the difference is when the difference is based on the expert _UH (for example, [item] amplitude is [what kind of difference] is small, [item] The timing (phase) [what kind of difference] is early, etc.) is output to the operation improvement instruction selection unit 104 . Note that, in addition to the determination result data, the similarity determination unit 103 also uses accompanying data (in other words, accompanying data acquired together with the action data to be compared) that accompanies the action data used for the decision to improve the action. Output to instruction selection unit 104 .

Note that, as a result of the determination described above, when a difference equal to or greater than a predetermined value is detected for a plurality of items, the similarity determination unit 103 only determines the determination result data regarding the item with the largest difference, or the determination result data regarding the item with the largest difference. A plurality of determination result data may be output to the operation improvement instruction selection unit 104 in order from the result data. According to this, it becomes possible to present a motion improvement instruction to the subject _UL targeting a motion that should be improved immediately. On the other hand, similarity determination section 103 outputs only the determination result data regarding the item with the smallest difference, or a plurality of determination result data regarding the item with the smallest difference to operation improvement instruction selection section 104. Also good. According to this, it becomes possible to present a motion improvement instruction to the subject _UL targeting motions that are easy to improve.

Further, here, the description has been made on the assumption that the motion data is time-series data. It is determined whether or not the subject _UL has a motion that can be improved.

Furthermore, in the present embodiment, the similarity determination unit 103 detects a difference equal to or larger than a predetermined value between the motion data of the subject and the motion data of the expert in order to present motions that can be improved to the subject _UL . Not limited to this, for example, in order to present the subject _UL with a motion that does not need to be improved, that is, a motion that should be continued, even if a matching portion is further detected between the subject motion data and the expert motion data. good.

According to this, in addition to motions that can be improved, motions that should be continued can also be presented to the subject _UL , so that the subject _UL can know not only the bad parts of himself but also the good parts. Therefore, it is possible to expect the effect of maintaining or improving the motivation of the subject _UL to improve the movement.

Here, before explaining the action improvement instruction selection unit 104, the action improvement instruction database 201 will be explained.

7, the action improvement instruction database 201 associates the character string data output from the action improvement instruction selection unit 104 with the accompanying data and the determination result data input to the action improvement instruction selection unit 104. It stores instruction data.

For example, according to the motion improvement instruction data a1 in FIG. 7, the sensor type indicated by the accompanying data input to the action improvement instruction selection unit 104 is the "myoelectric sensor", and the attachment site indicated by the accompanying data is " right hand” and the determination result data input to the motion improvement instruction selection unit 104 indicates that the “amplitude is small”, the character string data of the expression “please apply force to the right hand” is the motion improvement instruction. It is indicated that it is suitable as character string data output from the selection unit 104 .

Here, only the operation improvement instruction data a1 in FIG. 7 has been described as an example, but since the other operation improvement instruction data a2 to a4 shown in FIG. 7 are the same, detailed explanation thereof will be omitted here.

Further, in this embodiment, the case where character string data is included in the operation improvement instruction data as an output from the operation improvement instruction selection unit 104 has been exemplified, but the present invention is not limited to this. For example, audio data, image (video) data, or the like may be included in the operation improvement instruction data as an output from the operation improvement instruction selection unit 104 . Furthermore, an action signal for tactile feedback using electricity, sound waves, or mechanical force may be included in the action improvement instruction data as an output from the action improvement instruction selection unit 104 .

Upon receiving the input of the determination result data and the accompanying data output from the similarity determining unit 103, the action improvement instruction selection unit 104 selects from the action improvement instruction database 201 based on the inputted judgment result data and accompanying data. Suitable operation improvement instruction data is selected, and character string data included in the selected operation improvement instruction data is obtained. The acquired character string data is output to presentation unit 105 .

When the character string data output from the operation improvement instruction selection unit 104 is input, the presentation unit 105 displays the character string indicated by the input character string data on a display monitor provided in the information processing device 16 or an information processing device. The character string is displayed on a display monitor connected to the device 16, and the subject _UL is presented with the movement improvement instruction.

The presenting unit 105 presents the motion improvement instruction to the subject _UL each time the character string data is input from the motion improvement instruction selection unit 104, that is, presents the motion improvement instruction to the subject _UL in real time. Alternatively, the input character string data is stored in a temporary memory (not shown), and at a predetermined timing, the subject _UL is collectively instructed to improve movement based on the character string data stored in the temporary memory. You may present it.

Next, an example of the motion improvement promoting process realized by the motion improvement promoting program having the functional configuration as described above will be described with reference to the flowchart of FIG.

First, the subject data acquisition unit 101 acquires subject motion data and accompanying data from the sensor devices 11a to 14a attached to the subject _UL (step S1). In parallel with the processing of step S1, the expert data acquisition unit 102 acquires the expert operation data and accompanying data from each of the sensor devices 11b to 14b attached to the expert _UH (step S2). ).

Note that both the acquired motion data and accompanying data are output to the similarity determination unit 103 .

Subsequently, the similarity determination unit 103 receives input of the motion data and accompanying data output from the subject data acquisition unit 101 and the expert data acquisition unit 102 . Then, the similarity determination unit 103 sequentially compares the motion data of the subject and the motion data of the expert whose sensor type and wearing site indicated by the input accompanying data are the same, and determines that the subject _UL has a motion that can be improved. It is determined whether or not (step S3).

As a result of the determination in step S3, it is determined that there is no difference equal to or greater than a predetermined value between any of the motion data of the subject and the motion data of the expert, and that the subject _UL has no motion that can be improved (step S3). NO), the process ends here.

On the other hand, as a result of the determination in step S3, if there is a difference of a predetermined value or more between the motion data of the subject and the motion data of the expert, and it is determined that the subject _UL has a motion that can be improved (YES in step S3). ), the similarity determination unit 103 generates determination result data indicating items in which a difference equal to or greater than a predetermined value is detected and what kind of difference the difference is based on the expert _UH , and the motion of the subject. The data and accompanying data attached to the expert action data are output to the action improvement instruction selection unit 104 (step S4).

Next, the operation improvement instruction selection unit 104 receives input of the determination result data and the accompanying data output from the similarity determination unit 103 . Then, the action improvement instruction selection unit 104 selects/acquires the action improvement instruction data including the contents of the input accompanying data and the determination result data from the action improvement instruction database 201 as suitable action improvement instruction data, and selects/acquires the action improvement instruction data. Character string data included in the acquired action improvement instruction data is output to presentation unit 105 (step S5).

After that, when the presentation unit 105 receives the input of the character string data output from the operation improvement instruction selection unit 104, the presentation unit 105 displays and outputs the character string data on the display monitor (step S6), and the processing here ends. .

In this _{embodiment ,} the subject _UL and the expert _UH performed the same predetermined work at the same time. It is not always possible to perform the prescribed work. Even if the subject _UL performs a predetermined task, if there is no expert _UH who performs the same predetermined task at the same time, there will be no input to the expert data acquisition unit 102. An object to be compared cannot be obtained, and an inconvenience arises in that a movement improvement instruction cannot be presented to the subject _UL .

For this reason, as shown in FIG. 9, instead of the expert data acquisition unit 102, a motion database 202 that stores expert motion data in advance may be separately provided.

As shown in FIG. 10, the action database 202 stores action data in which expert action data and accompanying data are associated with each other. Note that the accompanying data in this case indicates the content of a predetermined work (work content) in addition to the sensor type and mounting site described above. For example, the motion data b1 in FIG. 10 is "expert motion data 1" which is the expert motion data when the expert _UH attaches the "myoelectric sensor" to the "right hand" and performs "screw tightening". including.

Here, only the motion data b1 in FIG. 10 has been described as an example, but the other motion data b2 to b4 shown in FIG. 10 are also the same, so detailed description thereof is omitted here.

As shown in FIG. 9 , when a motion database 202 is separately provided instead of the expert data acquisition unit 102 , the expert motion data is acquired from the motion database 202 by the similarity determination unit 103 . The function of the similarity determination unit 103 when the action database 202 is provided will be described below.

The similarity determination unit 103 receives the input of subject motion data and its associated data output from the subject data acquisition unit 101, and acquires expert motion data from the motion database 202 based on the input associated data. Specifically, the similarity determination unit 103 acquires from the motion database 202 the expert motion data associated with the sensor type, attachment site, and work content indicated by the input accompanying data.

As described above, the motion database 202 is provided separately instead of the expert data acquisition unit 102, and the similarity determination unit 103 is configured to be able to acquire the expert motion data from the motion database ₂₀₂ . Even if the expert _UH is not present when performing a predetermined task, it is possible to present a suitable motion improvement instruction to the subject _UL .

On the other hand, when the motion database 202 is separately provided as described above, and in this motion database 202, one expert motion data to be taken as a model is stored in association with the sensor type, the mounting site, and the work content. Inconveniences such as

For example, if the subject _UL is a beginner whose proficiency level for a given task is very low, there is a large discrepancy in the proficiency level in the expert motion data stored in the motion database 202 described above, and a motion that can be improved. are likely to be detected. In this case, a wide variety of motion improvement instructions are presented to the subject _UL , which may confuse the subject _UL .

In order to solve this problem, the motion data stored in the motion database 202 has a data structure as shown in FIG. may be added.

FIG. 11 is a diagram showing an example of the data structure of motion data stored in the motion database 202. As shown in FIG. The motion data shown in FIG. 11 is different from the motion data shown in FIG. 10 in that proficiency level data is further associated in addition to the expert motion data and accompanying data. The proficiency level data indicates the proficiency level of the subject _UL for the work content indicated by the associated accompanying data. Here, the proficiency level is an N-level evaluation from 1 to N, and the higher the value, the higher the proficiency level.

For example, the operation data c1 in FIG. 11 is the expert operation data when the expert _UH attaches the "myoelectric sensor" to the "right hand" and performs "screw tightening", and the proficiency level is "3". This is the "expert motion data 1a" for the test subject _UL . On the other hand, the motion data c2 in FIG. 10 is similar to the motion data c1 when the expert _UH attaches the "myoelectric sensor" to the "right hand" and performs "screw tightening". However, it is "expert motion data 1b" for subject _UL whose proficiency level is "2".

Although only the motion data c1 and c2 shown in FIG. 11 have been described here as an example, the other motion data c3 and c4 shown in FIG.

The proficiency level acquisition unit 106 has a function of acquiring proficiency level data indicating the proficiency level of the subject _UL and outputting it to the similarity determination unit 103 . The proficiency level data may be acquired (input) from an external device possessed by a supervisor at the site where the subject _UL works, or the subject movement data acquired by the subject data acquisition unit 101 may be analyzed, It may be generated as the analysis result.

In this case, when the similarity determination unit 103 receives input of the subject motion data and its associated data output from the subject data acquisition unit 101 and the proficiency level data output from the proficiency level acquisition unit 106, the input Suitable expert motion data is obtained from the motion database 202 based on the accompanying data and the proficiency data. Specifically, the similarity determination unit 103 extracts the expert action data in the action data including the sensor type, attachment part, and work content indicated by the accompanying data, and the proficiency level indicated by the proficiency level data, into the action database 202 . Get from

Here, it is assumed that the similarity determination unit 103 acquires the expert motion data in the motion data including the proficiency level indicated by the acquired proficiency level data. may acquire the expert action data in the action data including the proficiency level one level higher than the proficiency level indicated by the acquired proficiency level data.

In addition to the character string data from the action improvement instruction selection unit 104, the presentation unit 105 executes the action improvement instruction indicated by the character string data, thereby generating a character string stating that the proficiency level is improved by one level. Data may be presented, or character string data may be presented to the effect that "working time of a predetermined work will be shortened by M seconds" by executing the operation improvement instruction indicated by the character string data. It should be noted that the value of M described above is obtained by analyzing the expert action data of the expert whose proficiency level is one level higher than the proficiency level indicated by the proficiency level data input to the similarity determination unit 103 _. corresponds to the difference between the time required for the predetermined task and the time required for the subject _UL to perform the predetermined task obtained by analyzing the subject motion data input to the similarity determination unit 103 .

As described above, the motion database 202 is provided to store expert motion data classified according _to proficiency level even for the same work content. By adopting a configuration capable of acquiring motion data of an expert suitable for the level of proficiency, it is possible to prevent the motion data of an expert whose proficiency level is greatly deviated from being compared with the motion data of the _subject . It is possible to improve proficiency.

Here, the case where the expert motion data suitable for the subject _UL is selected according to the proficiency level is exemplified, but the present invention is not limited to this. Appropriate expert motion data may be selected according to.

According to the first embodiment described above, the information processing device 16 determines the similarity between the subject _UL and the expert _UH regarding the motions during a predetermined work, and gives the subject _UL a suitable Since it has a configuration capable of presenting motion improvement instructions, it can be expected to improve the efficiency of motion training and shorten the required skill acquisition. <Second embodiment>
Next, a second embodiment will be described. In this embodiment, a case will be described in which the motion improvement promoting program shown in FIG. 3 is further provided with a motion estimation unit 107 shown in FIG. Note that, in the following description, focusing mainly on the differences from the first embodiment, the same reference numerals are given to the same configurations, and the description may be omitted. Also, in this embodiment, the accompanying data indicates the type of sensor, the site where the sensor is attached, and the content of work.

FIG. 13 is a block diagram showing an example of the functional configuration of an operation improvement promotion program according to the second embodiment.

The motion estimating unit 107 receives the input of the subject motion data and the accompanying data output from the subject data acquisition unit 101, extracts feature amounts from the subject motion data, and performs pre-learning based on the feature amounts. The motion of the subject _UL is estimated by machine learning and discriminant analysis such as a known neural network, SVM (Support Vector Machine), random forest, etc., using the obtained learning results.

For example, when the sensor type indicated by the input accompanying data is "myoelectric sensor", the wearing part is "right hand", and the work content is "screw tightening", motion estimation unit 107 A feature amount is extracted from the motion data of the subject, and based on this feature amount, a learning dictionary that has been learned in advance is used to determine whether the motion of the subject _UL is "grip", "release", or "twist". The motion of the subject _UL is estimated by determining whether it is applicable.

The motion estimating unit 107 adds (labels) the input subject motion data with estimation result data indicating the motion type of the subject _UL , and outputs the data to the similarity determining unit 103 . For example, when _the motion estimating unit 107 estimates that the motion of the subject _UL in the section A of the subject motion data shown in FIG. The motion data of the subject is output to the similarity determination unit 103 after being labeled so that the data can be understood.

The motion estimating unit 107 also performs similar processing on the expert motion data output from the expert data acquisition unit 102, and receives estimation result data indicating the type of motion of the expert _UH . After labeling the expert action data, the label is output to the similarity determination unit 103 .

The following advantages can be obtained by labeling the motion data of the subject and the motion data of the expert by the motion estimating unit 107 to indicate the type of motion of the subject _UL and the expert _UH .

When the motion data of the subject and the motion data of the expert are time-series data, the similarity determination unit 103 superimposes and compares the motion data to determine the similarity of the motion data. However, since there may be a time lag between the motion data of the subject and the motion data of the expert, the motion data cannot be directly superimposed for comparison. For this reason, the timing at which the subject data acquisition unit 101 acquires the subject motion data and the timing at which the expert data acquisition unit 102 acquires the expert motion data may be synchronized in advance, or the acquired subject motion data and the skill level may be adjusted. It becomes necessary to perform processing such as aligning the heads of the user motion data.

On the other hand, the motion estimating unit 107 labels the motion data of the subject and the motion data of the expert to indicate the motion types of the subject _UL and the expert _UH . Even if there is a deviation in the occurrence time of , it is possible to easily compare by extracting only the section to be compared.

In the above-described first embodiment, the movement improvement instruction selection unit 104 selects and acquires suitable movement improvement instruction data from the movement improvement instruction database 201, and uniquely determines the movement improvement instruction to the subject _UL . Although the case has been exemplified, by using the estimation result data from the motion estimation unit 107, it is also possible to present a more detailed motion improvement instruction to the subject _UL .

In this case, as shown in FIG. 16, an adverbial expression database 203 is provided instead of the action improvement instruction database 201 shown in FIG. provided as part of

As shown in FIG. 17, the adverbial expression database 203 stores adverbial expression data in which (part of) accompanying data, determination result data, estimation result data, and adverbial expressions are associated with each other. For example, according to the adverb expression data d1 in FIG. 17, the sensor type indicated by the accompanying data is "myoelectric sensor", the mounting site indicated by the accompanying data is "right hand", and the determination result data is "amplitude When the action type indicated by the estimation result data is "grasp", the adverb expression is "strongly". Further, according to the adverb expression data d2 of FIG. 17, the sensor type indicated by the accompanying data is "myoelectric sensor", the mounting site indicated by the accompanying data is "right hand", and the determination result data is "timing sensor". is fast", and the adverb expression is "slowly" when the action type indicated by the estimation result data is "twist".

The motion improvement instruction generation unit 108 shown in FIG. 16 includes the determination result data and accompanying data output from the similarity determination unit 103, and the estimation result data ( In other words, upon receiving the input of the estimation result data added to the motion data to be compared, the motion improvement instruction is generated based on each of the input data and the adverb expression database 203 .

More specifically, as shown in FIG. 18(a), the action improvement instruction generation unit 108 recognizes the attachment site indicated by the input accompanying data as the site to be improved. Further, as shown in FIG. 18B, the motion improvement instruction generation unit 108 recognizes the motion type indicated by the input estimation result data as a motion to be improved. Furthermore, as shown in FIG. 18C, the action improvement instruction generating unit 108 selects and selects suitable adverbial expression data from the adverbial expression database 203 based on the input accompanying data, determination result data, and estimation result data. An adverbial expression to be acquired and added to the motion improvement instruction to be presented to the subject _UL is determined.

The action improvement instruction generation unit 108 generates an action improvement instruction by combining the recognized/determined site to be improved, the action to be improved, and the adverbial expression according to a preset logic. More specifically, the action improvement instruction generating unit 108 generates the action improvement instruction according to the logic "Please do [adverb expression] [movement] on [site]."

For example, the sensor type indicated by the input accompanying data is a "myoelectric sensor", the attachment site indicated by the accompanying data is "right hand", and the input determination result data is "small amplitude". , and if the action type indicated by the estimation result data is "grasp" and the suitable adverb expression in this case is "strongly", the action improvement instruction generation unit 108 generates "[right hand] to (to) [ It generates an action improvement instruction of "please hold tightly". The motion improvement instruction data indicating the generated motion improvement instruction is output to the presentation unit 105 and presented to the subject _UL by the presentation unit 105 .

Note that step-by-step adverb expressions may be used for the action improvement instruction generated by the action improvement instruction generation unit 108 . Specifically, in addition to the adverbial expression "strongly", graded adverbial expressions such as "very strongly" and "somewhat strongly" may be used.

In this case, the similarity determination unit 103 does not merely determine whether or not there is a difference equal to or greater than a predetermined value between the motion data of the subject and the motion data of the expert, but rather based on thresholds set stepwise. make a judgment. For example, when a first threshold value and a second threshold value (where the first threshold value is less than the second threshold value) are set as stepwise threshold values, the similarity determination unit 103 determines the difference between the motion data of the subject and the motion data of the expert. It is determined whether or not there is a difference greater than or equal to the first threshold and less than the second threshold, or whether or not there is a difference greater than or equal to the second threshold. For example, when the motion data of the subject has a smaller amplitude than the motion data of the expert, and the difference is greater than or equal to the first threshold and less than the second threshold, the similarity determination unit 103 determines that the amplitude is small. When the data is output and the difference is equal to or greater than the second threshold, the similarity determination unit 103 outputs determination result data indicating that "amplitude is very small".

Since the judgment result data becomes finer in this way, the adverb expression data stored in the adverb expression database 203 can also be subdivided (for example, the judgment result data "small amplitude" has "strong"). , and the determination result data “very small amplitude” is associated with the adverb “very strong”, etc.). to generate performance improvement instructions using

According to the second embodiment described above, the information processing device 16 estimates the motion types of the subject _UL and the expert _UH , and transmits estimation result data indicating the estimated motion types to the subject motion data and the expert. Since the motion estimator 107 capable of labeling the motion data is further provided, the motion data of the subject and the motion data of the expert can be easily compared, and the subject _UL can be given more detailed motion improvement instructions. can be presented.

According to at least one embodiment described above, it is possible to provide an information processing apparatus, method, and program capable of promoting training of workers (subjects U _L ).

It should be noted that although several embodiments of the invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

Reference numerals 11 to 14: sensor device, 15: data transceiver, 16: information processing device, 101: subject data acquisition unit, 102: expert data acquisition unit, 103: similarity determination unit, 104: action improvement instruction selection unit, 105 Presentation unit 106 Proficiency level acquisition unit 107 Action estimation unit 108 Action improvement instruction generation unit 201 Action improvement instruction database 202 Action database 203 Adverb expression database U _L Test subject _UH …expert.

Claims (8)

An information processing device that communicates with multiple types of sensor devices,
Acquiring first motion data indicating motions of the first worker when performing a predetermined work from each of the sensor devices, and indicating at least the type of the sensor device, the measurement site, and the content of the predetermined work. a first acquisition means for acquiring each associated data;
A second operation data indicating an operation when a second operator having higher proficiency than the first operator performs the predetermined work is acquired from each of the sensor devices, and a second operation data is acquired from each of the accompanying data. acquisition means;
a third acquisition means for acquiring proficiency level data indicating the proficiency level of the first worker for the predetermined work;
analyzing each of the first motion data and each of the second motion data, and estimating one or more motion types by the first worker and the second worker indicated by the motion data; an estimating means for adding estimation result data capable of identifying the type of motion to each motion data;
determining means for comparing each of the first motion data and each of the second motion data to determine similarity;
presenting means for presenting to the first worker instruction data indicating points to be improved regarding actions in performing a predetermined work according to a determination result by the determining means;
selection means for selecting instruction data to be presented to the first worker from a first storage means for storing a plurality of instruction data according to a determination result by the determination means;
a second storage means for storing the associated data, the second action data, and the proficiency level data in association with each other;
and
The determination means is
Accompanying data acquired together with the first action data, and second action data associated with the acquired proficiency level data, based on the accompanying data and the proficiency level data, the first action data Parts to which estimation result data indicating the same type of motion are added are extracted from the first motion data and the second motion data obtained from the second storage means as objects for comparison, and these motion data are extracted from the first motion data and the second motion data. comparing and detecting a difference between the first motion data and the second motion data and a matching portion between the first motion data and the second motion data to determine the similarity of the motion data; as determination result data. 2. The information processing apparatus according to claim 1, further comprising said first storage means. The first storage means is
storing instruction data in which at least the accompanying data, the determination result data, and an operation improvement instruction to the first worker are associated;
The selection means is
when the detected difference is equal to or greater than a predetermined value, instruction data including determination result data regarding the difference and accompanying data obtained together with the first motion data to be compared when the difference is detected; The information processing apparatus according to claim 1, wherein the instruction data is selected as the instruction data to be presented to the first operator. generating means for generating a motion improvement instruction for the first worker according to the result of determination by the determining means and using this as instruction data;
a third storage means for storing a plurality of adverbial expression data in which the adverbial expression included in the generated action improvement instruction, the determination result data, the accompanying data, and the estimation result data are associated;
The information processing apparatus according to claim 1, further comprising: The generating means is
determination result data relating to the detected difference; accompanying data acquired together with the first motion data to be compared when detecting the difference; and estimation added to the first motion data to be compared Get the adverb expression data containing the result data and
5. The action improvement instruction is generated by combining the wearing part indicated by the accompanying data, the type of action indicated by the estimation result data, and the adverbial expression indicated by the acquired adverbial expression data. Information processing equipment. 6. The information processing apparatus according to any one of claims 1 to 5, further comprising a display for displaying said instruction data. A method executed by an information processing device that communicates with multiple types of sensor devices,
Acquiring first motion data indicating motions of the first worker when performing a predetermined work from each of the sensor devices, and indicating at least the type of the sensor device, the measurement site, and the content of the predetermined work. respectively obtaining ancillary data;
Acquiring from each of the sensor devices second motion data indicating motions of a second worker who is more proficient than the first worker when performing the predetermined work, and acquiring each of the accompanying data. ,
Acquiring proficiency level data indicating the proficiency level of the first worker for the predetermined work;
analyzing each of the first motion data and each of the second motion data, and estimating one or more motion types by the first worker and the second worker indicated by the motion data; adding estimation result data capable of identifying the type of motion to each motion data;
comparing each of the first motion data and each of the second motion data to determine similarity;
presenting to the first worker, according to the result of the determination, instruction data indicating points to be improved regarding actions when performing a predetermined work;
Selecting instruction data to be presented to the first worker from a first storage means for storing a plurality of instruction data according to the result of the determination;
associating the accompanying data, the second action data, and the proficiency level data and storing them in a second storage means;
and
Determining the similarity includes:
Accompanying data acquired together with the first action data, and second action data associated with the acquired proficiency level data, based on the accompanying data and the proficiency level data, the first action data Parts to which estimation result data indicating the same type of motion are added are extracted from the first motion data and the second motion data obtained from the second storage means as objects for comparison, and these motion data are extracted from the first motion data and the second motion data. comparing and detecting a difference between the first motion data and the second motion data and a matching portion between the first motion data and the second motion data to determine the similarity of the motion data; as determination result data. A program executed by a computer of an information processing device that communicates with multiple types of sensor devices, the program causing the computer to:
Acquiring first motion data indicating motions of the first worker when performing a predetermined work from each of the sensor devices, and indicating at least the type of the sensor device, the measurement site, and the content of the predetermined work. a first acquisition means for acquiring each associated data;
A second operation data indicating an operation when a second operator having higher proficiency than the first operator performs the predetermined work is acquired from each of the sensor devices, and a second operation data is acquired from each of the accompanying data. acquisition means;
a third acquisition means for acquiring proficiency level data indicating the proficiency level of the first worker for the predetermined work;
analyzing each of the first motion data and each of the second motion data, and estimating one or more motion types by the first worker and the second worker indicated by the motion data; an estimating means for adding estimation result data capable of identifying the type of motion to each motion data;
determining means for comparing each of the first motion data and each of the second motion data to determine similarity;
presenting means for presenting to the first worker, according to the result of the determination by the determining means, instruction data indicating points to be improved regarding actions in performing a predetermined work;
selection means for selecting instruction data to be presented to the first worker from a first storage means for storing a plurality of instruction data according to a determination result by the determination means;
Operate as storage means for associating the accompanying data, the second action data, and the proficiency level data and storing them in a second storage means;
The determination means is
Accompanying data acquired together with the first action data, and second action data associated with the acquired proficiency level data, based on the accompanying data and the proficiency level data, the first action data Parts to which estimation result data indicating the same type of motion are added are extracted from the first motion data and the second motion data obtained from the second storage means as objects for comparison, and these motion data are extracted from the first motion data and the second motion data. comparing and detecting a difference between the first motion data and the second motion data and a matching portion between the first motion data and the second motion data to determine the similarity of the motion data; A program that uses the result of as judgment result data.

Images