JP6803431B2 - Programs, information processing devices and information processing methods - Google Patents

Description

The present invention relates to a program for analyzing moving images and the like.

Conventionally, there is known a technique of capturing a moving image of a work performed by an object such as a person or a machine and analyzing an operating element obtained by dividing the captured moving image by time (for example, Patent Document 1).

Japanese Patent No. 4677526

However, the technique disclosed in Patent Document 1 has a problem that it is difficult to accurately analyze a moving image such as calculation of a rating rate when a user has little experience in using software.

One aspect is to provide a program or the like that can easily analyze a moving image.

One aspect of the program is a program used for an information processing device, in which a plurality of images extracted from a moving image are displayed on the information processing device, and a work type indicating a type of work performed by an object in the displayed image. Whether or not the object is operating, including an invalid time indicating the time when the object has performed an invalid operation for the work and a quasi-operating time indicating the time when the machine has operated for the work. The operation information to be shown is received in association with the image, and based on the received operation information, the operation time related to the work and the time when the object in each image has performed an effective operation for the work are shown. The time is calculated, a graph showing the ratio of the operating time, the invalid time, and the semi-operating time to the calculated working time is generated and displayed , and the divided moving image obtained by dividing the moving image into each work is displayed. , Multiple types of the divided video are accepted, a work table corresponding to the work time indicating the length of each divided video and the received multiple types is generated, and the plurality of work tables generated for each of the divided videos are switched. To execute the display process.

On one side, video analysis can be facilitated.

It is explanatory drawing which shows the outline of an information processing system. It is a block diagram which shows the hardware group of an information processing apparatus. It is a figure which shows an example of the data stored in the element information DB. It is a figure which shows an example of the data stored in the aggregated information DB. It is a figure which shows an example of the data stored in the type information DB. It is a figure which shows an example of the data stored in a comment DB. It is explanatory drawing which shows an example of the analysis screen for performing the analysis of the operation element. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is explanatory drawing which shows the outline of an information processing system. It is explanatory drawing which shows an example of the analysis screen. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. This is an element information DB that stores data related to worker A. This is an element information DB that stores data related to worker B. It is explanatory drawing which shows an example of the viewpoint analysis screen which selected View1. It is explanatory drawing which shows an example of the viewpoint analysis screen which selected View2. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a figure which shows an example of the data stored in a non-stationary information DB. It is explanatory drawing which shows an example of a cycle screen. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a figure which shows an example of the data stored in a sampling table and a corresponding table. It is a figure which shows an example of the data stored in the correspondence table which associates the work process and the image ID which were put together for each of a plurality of work types. It is a figure which shows an example of the data stored in a sampling summary table. It is explanatory drawing which shows an example of the work sampling screen for executing a work sampling method. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a figure which shows an example of the data stored in a template DB. It is explanatory drawing which shows an example of the image registration screen. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a figure which shows an example of the data stored in a sampling table. It is a figure which shows an example of the data stored in the element information DB. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is explanatory drawing which shows an example of the comparison screen. It is explanatory drawing which shows an example of the detailed comparison screen. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is explanatory drawing which shows an example of the display window of the analysis screen. It is a flowchart which showed the processing procedure of the information processing system in this embodiment. It is a flowchart which showed the processing procedure of the information processing system in this embodiment.

Embodiment 1
Hereinafter, the present embodiment will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing an outline of an information processing system. The information processing system in this embodiment includes an information processing device 1, an image pickup device 2A, 2B, 2C, and the like. The image pickup devices 2A, 2B, 2C ... Are represented by the image pickup device 2 in the following cases. The information processing device 1 and the image pickup device 2 are connected to each other by a communication network N such as the Internet, a LAN (Local Area Network), or a mobile phone network.

The outline of the information processing system will be described below. The image pickup device 2 is an image pickup device for capturing a work performed by an object such as a person or a machine, and is, for example, a CCD (Charge-coupled device) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, an infrared camera, or the like. The image pickup devices 2A, 2B, and 2C capture the work performed by the object with a plurality of moving images based on a plurality of positions and angles, and output the captured plurality of moving images to the information processing device 1.

The information processing device 1 is, for example, a computer or a smartphone. The information processing device 1 acquires a plurality of output moving images. The information processing device 1 receives an input from a user by an input unit 14 described later, and divides a plurality of moving images acquired based on the received input into a plurality of operation elements (divided moving images) divided for each work. The information processing device 1 performs a process of calculating a standard time indicating a standard working time performed by an object by displaying an operating element or information related to the operating element. Standard time is the time that a person with standard skills works. However, it is difficult to identify a person with standard skills, and the skills of a person often change from work to work. For this reason, in general, the standard time is the operation time indicating the time when the object has performed an effective operation for the work, and the rating rate (evaluation value) for the object displayed in each divided video is integrated. calculate. For example, the information processing device 1 calculates the standard time by the following method. The information processing device 1 calculates the operating time by deleting the wasted time in the operating element. By watching the divided video, the user determines whether the worker in the divided video is a veteran or is accustomed to the work, and determines the rating rate. For example, when it is determined that the worker is a veteran and is accustomed to the work, the information processing apparatus 1 sets the rating rate to a value lower than 1. The information processing device 1 calculates a standard time earlier than the working time in the divided moving image by integrating the rating rate of a value lower than 1 with the calculated operating time.

The details of the information processing system will be described below. FIG. 2 is a block diagram showing a hardware group of the information processing device 1. The information processing device 1 shown in FIG. 2 includes a CPU 11 which is one or more arithmetic control devices, a storage unit 12 which is a storage device such as a hard disk and a semiconductor memory disk, a SRAM (Static RAM), a DRAM (Dynamic RAM), or a flash memory. RAM 13, input unit 14, which is an input device such as a mouse, keyboard, touch panel, pen tablet, microphone, etc., display unit 15, display, projector, head mount display, etc., communication unit 16 that communicates with the communication network N. including.

The storage unit 12 stores various data, element information DB 121, total information DB 122, type information DB 123, comment DB 124, and program 12P required when the CPU 11 performs processing.

The communication unit 16 acquires a moving image captured by the image pickup device 2 via the communication network N. In the present embodiment, the moving image is acquired from the imaging device 2, but the present invention is not limited to this, and the moving image may be acquired from a USB (Universal Serial Bus) memory, an optical disk, a storage unit 12, or the like.

FIG. 3 is a diagram showing an example of data stored in the element information DB 121. The element information DB 121 is a database that stores information about operating elements. The element information DB 121 includes an operation element column, a type column, a start time column, an end time column, an element time column, an operation time column, an invalid time column, a comparison value column, a difference column, a non-stationary column, a division column, a rating rate column, and a standard. It has a time column, a moving image data column, a manual column, and the like. The name of the operation element is stored in the operation element string. Multiple types indicating the types of operating elements are stored in the type column. The type is, for example, each work process such as "manual work" or "first process". The types of the type column are divided by the type classification for classifying the types. The type classification is, for example, type 1, type 2, type 3, and the like. For example, it is stored in the type 1 column of the type column that the type of "welding" is "manual work" or the type of "fixing the panel" is "tool work". Further, in the type 2 column of the type column, it is stored that the type of "welding" is "first process" or the type of "bolt tightening" is "second process". Further, in the type 3 column of the type column, it is stored that the type of "welding" is "manual work" or the type of "bolt tightening" is "mechanical work".

In the start time column, the start time at which the motion element is started in the moving image is stored in association with the motion element. In the end time column, the end time at which the operation element is terminated in the moving image is stored in association with the operation element. In the element time column, the element time (working time) indicating the length of time of the operation element is stored in association with the operation element. The operating time is stored in the operating time sequence in association with the operating element. In the invalid time column, the invalid time indicating the time when the object performed an invalid operation for the work is stored in association with the operation element. In the comparison value string, the comparison value for comparison with the operation time is stored in association with the operation element. For example, the comparison value sequence stores an estimated standard time indicating the standard time expected by the user. In the difference column, the difference between the comparison value and the operation time is stored in association with the operation element. The unit of the start time column, end time column, element time column, operation time column, invalid time column, comparison value column, and difference column is, for example, seconds. In the non-stationary sequence, non-stationary information (periodic information) indicating whether or not the object is performing a periodic motion is stored in association with the motion element. The periodic operation is, for example, an operation in which the worker A sets the metal fittings a plurality of times. Further, the non-periodic operation is, for example, an operation in which the worker A sets the metal fitting only once. ○ indicates that the object is operating without periodicity. X indicates that the object has a periodic motion. In the classification column, operation information of operation, semi-operation, and non-operation is stored in association with the operation element. In the present specification, operation is a state in which a person is working, semi-operation is a state in which a machine is operating, and non-operation is a state in which neither a person nor a machine is operating, such as a waiting state. Say.

The rating rate is stored in the rating rate column in association with the operating element. The standard time is stored in the standard time sequence in association with the operating element. In the moving image data string, moving image data obtained by dividing the captured moving image into each operating element is stored. For example, in the moving image data string, moving image data obtained by dividing the moving image A into each operating element is stored. The manuals corresponding to the operating elements are stored in the manual column.

The CPU 11 receives the input of the operating element, the type of the operating element, the start time, the end time, the element time, the operating time, the invalid time, the comparison value, the non-stationary information, the classification or the rating rate on the analysis screen 5 described later in the input unit 14. .. The CPU 11 stores the received operation element, operation element type, start time, end time, element time, operation time, invalid time, comparison value, non-stationary information, classification or rating rate in the element information DB 121. The CPU 11 stores the difference obtained by subtracting the comparison value from the operating time in the difference sequence. The CPU 11 stores the standard time obtained by integrating the rating rate with the operating time in the standard time sequence. It is assumed that the user has registered the moving image data column and the manual column of this embodiment in advance.

FIG. 4 is a diagram showing an example of data stored in the aggregated information DB 122. The aggregated information DB 122 is a database in which aggregated information that aggregates information related to operating elements is stored. The aggregated information DB 122 includes an element total time column, an operation total time column, an operation total time column, an invalid total time column, a total difference column, an overall rating rate, a quasi-operation total time column, a standard total time column, and the like. The element total time (total time), which is the sum of the element times of a plurality of operating elements, is stored in the element total time column. The total operation time, which is the sum of the operation times of a plurality of operation elements, is stored in the total operation time column. In the total operation time column, the total operation time, which is the sum of the operation times of a plurality of operation elements in which the object performs the operation operation, is stored. The invalid total time column stores the invalid total time, which is the sum of the invalid times of a plurality of operating elements. In the total difference column, the total time of the difference between the standard time and the comparison value is stored. The overall rating rate, which indicates the average value of a plurality of rating rates, is stored in the overall rating rate. The total quasi-operation time (total extraction time), which is the total of the quasi-operation time, is stored in the quasi-operation total time column. In the standard total time column, the standard total time obtained by adding the overall rating rate to the total operation time is stored. The storage method of the aggregated information DB 122 will be described later.

FIG. 5 is a diagram showing an example of data stored in the type information DB 123. The type information DB 123 manages the classification of work or operation. As shown in FIG. 5A, for example, when one moving image is to be divided into work and process, the work type may be input to type 1 and the process type may be input to type 2. Further, in the type 3, the type can be used to input the work depending on the person and the work depending on the machine. Further, by setting the color for each work type as shown in FIG. 5B, it becomes easy to compare the work time when displaying the time of the work type in a bar graph.

FIG. 6 is a diagram showing an example of data stored in the comment DB 124. FIG. 7 is an explanatory diagram showing an example of an analysis screen 5 for analyzing an operating element. The comment DB 124 includes an operation element string, an individual comment string, and a general comment string. Action elements are stored in the action element string. The individual comment columns include one window column, two window columns, three window columns, and four window columns. In the individual comment column, an individual comment indicating a comment for the moving image data of each operating element displayed in the display window 51 shown in FIG. 7 is stored. Each of the window 1 column, the window 2 column, the window 3 column, and the window 4 column corresponds to the display windows 51A, 51B, 51C, and 51D described later. The operation element of the operation element string corresponds to the operation element sequence of the element information DB 121. The CPU 11 receives an input of an individual comment for the operation element selected in the operation element table (work table) 52 in the individual comment field 56 described later by the input unit 14. The CPU 11 stores the received individual comment in the individual comment string of the comment DB 124. In the summary comment column, a summary comment indicating a comment for the entire acquired video is stored. The CPU 11 receives the input of the general comment in the general comment field 57 by the input unit 14. The CPU 11 stores the received summary comment in the aggregated information DB 122.

The analysis screen 5 has display windows 51A, 51B, 51C and 51D divided in the upper part, an operation element table 52 provided in the lower part of the display windows 51A, 51B, 51C and 51D, and a type graph 53A provided in the lower left part. And 53B, the tabulation table 54 provided on the right side of the display window 51D, the switching button 510 provided on the right side of the tabulation table 54, the operation graph 55 provided on the lower part of the tabulation table 54, and the operation elements. It includes an individual comment column 56 provided on the right side of Table 52 and a general comment column 57 provided at the bottom of the individual comment column 56. A total column 523 is provided at the bottom of the operating element table 52. The rating column 521 and the unsteady column 522 are provided in the operating element table 52 and the total column 523. A rating rate column 511 is provided at the left end of the display window 51A. A determination rating rate column 512 is provided in the upper left portion of the display window 51A. The display windows 51A, 51B, 51C, 51D and the like may be represented by the display window 51 in some cases. The type graphs 53A, 53B, etc. are represented by the type graph 53 in some cases. The operation graph 55 is composed of a non-operating part 55A, a semi-operating part 55B, and a moving part 55C. A comment tab 561 is provided at the top of the individual comment field 56. The moving images displayed in the display windows 51A, 51B, and 51C are moving images taken or reproduced on the same time axis. Here, it is assumed that the moving image of the worker A is displayed in the display window 51A and the moving image of the worker B is displayed in the display window 51B.

The display window 51 is a window for displaying information related to operating elements such as moving image data or a manual. The rating rate column 511 is a column for inputting the rating rate. The determination rating rate column 512 is a column for displaying the rating rate of the operating element. The operating element table 52 is a table that displays information about the operating elements. The rating column 521 is a column provided with a check box for determining whether or not to perform rating. The non-stationary column 522 is a column provided with a check box for determining whether or not the information is non-stationary information. The total column 523 is a table displaying total related information obtained by totaling information related to a plurality of operating elements. The type graph 53 is a bar graph displaying the ratio of the element time corresponding to the type for each type classification. The summary table 54 is a table that displays the summary information. The operation graph 55 is a pie chart displaying the ratios of the total operation time, the quasi-total operation time, and the invalid total time. The individual comment column 56 is a column for displaying individual comments. The comment tab 561 is a tab for switching individual comments. There are a plurality of comment tabs 561, and they correspond to the display windows 51A, 51B, 51C, and 51D from the left end, respectively. The summary comment column 57 is a column for displaying a summary comment for the entire acquired video. The switching button 510 is a button for switching the operation element table 52 corresponding to the moving image data of each display window 51.

The operation of the analysis screen 5 is as follows. In this embodiment, it is assumed that each moving image captured in advance is divided into operating elements. The CPU 11 refers to the element information DB 121, and displays the operation element, type, start time, element time, comparison value, difference, classification, unsteady information, rating rate, etc. in the operation element table 52. Further, the operation element table 52 can be set in the display windows 51A, 51B and 51C. When the user selects the switching button 510 of FIG. 7 in the input unit 14, the CPU 11 can switch the operation element table 52 for the display windows 51A, 51B or 51C and display it on the analysis screen 5. In addition, non-stationary information can be input by the check box in the non-stationary column 522. When each column of the operation element table 52 is selected by the input unit 14, the CPU 11 accepts the selection of the operation element by the input unit 14. The CPU 11 displays a plurality of moving image data corresponding to the selected operating element in the display window 51. For example, when the CPU 11 selects the operation element "welding" of the worker A displayed on the display window 51A, a moving image of another worker on the coaxial time is displayed in each of the display windows 51B and 51C. The CPU 11 displays the manual A corresponding to the operation element "welding" in the display window 51D.

By viewing the moving image data displayed in the display window 51, the user determines whether or not the worker is performing an invalid operation for the work in the moving image data. When the user determines that the worker is performing an invalid operation for the work, the user selects the display window 51 in the input unit 14 to calculate the invalid time. Specifically, the CPU 11 calculates the time from selecting the display window 51 by the input unit 14 to selecting the display window 51 again by the input unit 14 while playing back the moving image data as the invalid time. The CPU 11 calculates the operating time by subtracting the invalid time from the element time. The CPU 11 may further receive the input of the rating rate in the input unit 14, and calculate the operation time by integrating the received rating rate into a value obtained by subtracting the invalid time from the element time. The CPU 11 displays the calculated invalid time and operating time in the operating element table 52. In the operating element table 52, colors are associated with each type. The CPU 11 refers to the type information DB 123, and displays the colors associated with each type in the types of the operation element table 52.

The CPU 11 displays the element time for each type on the type graph 53 in chronological order. For example, the CPU 11 sets the colors corresponding to the work unit types "manual work", "tool work", "machine work", and "manual work" on the type graph 53A to "manual work", "tool work", and "machine". Display in chronological order of "work" and "manual work". Further, the CPU 11 displays the colors corresponding to the types "first process" and "second process" of the process unit on the type graph 53B in chronological order of "first process" and "second process". The ratio of each color of the type graphs 53A and 53B corresponds to the ratio of each type of element time. The type graph 53 of the present embodiment displays the element times for each type in chronological order, but the present invention is not limited to this. Further, in the present embodiment, there are three types, but more or less types can be set. For example, the type graph 53 may display the operating time, invalid time, or standard time for each type in chronological order. Further, the type graph 53 can also display the element times for each type of workers displayed in different windows in chronological order. For example, by displaying the type of the process unit of the worker A displayed on the display window 51A and the worker B displayed on the display window 51B, the work of the worker A and the work of the worker B can be compared.

An example of the calculation method of the rating rate and the standard time is as follows. The user views the moving image data on the same time axis in the display windows 51A, 51B and 51C. The user confirms the manual A of the display window 51D, and determines that the worker A complies with the manual A. The user can display the type of the work unit of the worker A displayed in the display window 51A on the type graph 53 and the type of the work unit of the worker B displayed in the display window 51B to display the type of the work unit of the worker A. It is possible for the worker B to easily confirm how much the same work is delayed. Here, when it is considered that the operation time of the operation element of the worker A is slower than the normal work, the user selects 1.1 times in the rating rate column 511. The CPU 11 displays 1.1 times in the determination rating rate column 512. For example, when rating the "welding" of the "operating element" as described above, the CPU 11 sets the rating rate "1.1" to the operating time "3.50" of the operating element in the same row as the rating column 521. The standard time "3.85" is calculated by integrating. The CPU 11 displays the rating rate “1.1” and the standard time “3.85” in the operating element table 52, and stores them in the element information DB 121. Further, when the operation time of the operation element "set the metal fitting A" is considered to be slower than the normal operation, the user selects 0.8 times in the rating rate column 511. The CPU 11 displays 0.8 times in the determination rating rate column 512. The CPU 11 calculates the standard time "6.40" by adding the rating rate "0.8" to the operating time "8.00" of the operating elements in the same column as the rating column 521. The CPU 11 displays the rating rate “0.8” and the standard time “6.40” in the operating element table 52, and stores them in the element information DB 121. Hereinafter, the same applies, and the description is omitted for the sake of brevity.

If the user wants to watch the moving element video data in standard time, the moving element video data can be played back in standard time by the following procedure. The CPU 11 receives an input in the check box in the rating field 521 by the input unit 14. For example, when the user inputs the check box of "1.1" in the first line in the rating field 521 in the input unit 14, the playback time of the moving image 101 of the display window 51A is multiplied by 1.1, and the operation time "3". The standard time is set to "3.85", which is a value obtained by multiplying "5.5" by the rating rate "1.1". The CPU 11 slow-plays the moving image 101 so that the playback time becomes the standard time "3.85". Further, when the user selects the switching button 510 in the input unit 14, the operation element table 52 of the worker B is displayed on the analysis screen 5. Hereinafter, the moving image of the display window 51B is played back slowly or at a high playback speed so that the playback time becomes the standard time by the same operation. The same applies to the display window 51C, and the description thereof is omitted for the sake of brevity.

The CPU 11 calculates total element time, total operation time, total invalid time, total comparison value, total difference, overall rating rate, total semi-operation time, total operation time, standard total time, and the like. For example, the CPU 11 calculates the total element time "27.00", which is the sum of all the element times in the operating element table 52. The CPU 11 calculates the total operation time "22.00", which is the sum of all the operation times in the operation element table 52. The CPU 11 calculates the total invalid time "5.00", which is the sum of all the invalid times in the operating element table 52.

Since the first, second, and fourth columns of the operation element table 52 of the CPU 11 are "quasi-operation", the operation times "3.50", "2.50", and "8.00" are totaled. The calculated quasi-operating total time "14.00" is calculated, and a value is input to the quasi-operating total time shown in FIG. The CPU 11 calculates the overall rating rate "1.07" by calculating the average value of all rating rates. The CPU 11 calculates the standard total time "23.54" by adding the rating rate "1.07" to the total operation time. The CPU 11 calculates a total comparison value "23.5" which is the sum of all the comparison values in the operating element table 52. The CPU 11 calculates the total difference "0.04" by subtracting the standard total time "23.54" from the total comparison value "23.5". The CPU 11 calculates the total operating time "8.00" by subtracting the quasi-total operating time "14.00" from the total operating time "22.00". In the present embodiment, the total element time, the total operation time, the total invalid time, and the total semi-operation time are the total values of all the operation elements of the acquired moving image, but are not limited to this. For example, the total element time, total operation time, total invalid time, total quasi-operation time, and total operation time are the total element time, total operation time, total invalid time, and quasi of the one or more selected operation elements in the operation element table 52. It may be the total operating time and the total operating time.

The CPU 11 displays the calculated total element time, total operation time, total invalid time, total difference, overall rating rate, total semi-operation time, and standard total time in the total column 523. The CPU 11 stores the calculated total element time, total operation time, total invalid time, total difference, overall rating rate, total semi-operation time, and standard total time in the aggregated information DB 122.

The CPU 11 displays the calculated total element time "27.00", total operation time "22.00", and total invalid time "5.00" in the summary table 54. The CPU 11 may display the quasi-operating total time or the operating total time in the summary table 54.

The process by which the CPU 11 displays the operation graph 55 is as follows. The CPU 11 generates the non-operating portion 55A at a ratio of the invalid total time "5.00" to the total element time "27.00". The CPU 11 generates the quasi-operating portion 55B at a ratio of the quasi-operating total time "14.00" to the element total time "27.00". The CPU 11 generates the operating portion 55C at a ratio of the total operating time "8.00" to the total element time "27.00". The CPU 11 generates and displays an operation graph 55 based on the generated non-operating portion 55A, semi-operating portion 55B, and operating portion 55C.

The user selects the tab corresponding to the display window 51A from the comment tabs 561 in the input unit 14. The CPU 11 receives the input of the individual comment corresponding to the moving image 201 of the operation element selected in the operation element table 52 in the individual comment field 56 by the input unit 14. The user inputs "working hard" in the individual comment field 56 in the input unit 14. The CPU 11 stores the received individual comment "working hard" in the comment DB 124. The user selects the tab corresponding to the display window 51B from the comment tabs 561 in the input unit 14. The CPU 11 receives the input of the individual comment corresponding to the moving image 201 of the operation element selected in the operation element table 52 in the individual comment field 56 by the input unit 14. The user inputs "one person is taking a break" in the individual comment field 56 in the input unit 14. The CPU 11 stores the received individual comment "one person is taking a break" in the comment DB 124. Hereinafter, the same applies, and the description is omitted for the sake of brevity. The CPU 11 receives the input of the general comment in the general comment field 57 by the input unit 14. The user confirms the operation graph 55, and inputs "overall semi-operation total time is long" in the summary comment field 57 in the input unit 14. The CPU 11 stores the received general comment "overall, the total quasi-operating time is long" in the comment DB 124.

8 to 9 are flowcharts showing the processing procedure of the information processing system according to the present embodiment. The CPU 11 reads the operating element, type, start time, end time, element time, operating time, invalid time, comparison value, difference, classification, unsteady information, rating rate, standard time, etc. from the storage unit 12 or uses the input unit 14. Accept the input (step S11). The CPU 11 has an operating element table 52 and a type graph 53 based on an operating element, type, start time, end time, element time, operating time, invalid time, comparison value, difference, classification, unsteady information, rating rate, standard time, and the like. Is displayed (step S12).

The CPU 11 accepts the selection of the rating rate in the display window 51 in the input unit 14 (step S13). The CPU 11 calculates the standard time by accumulating the rating rate received in the operating time (step S14). The CPU 11 has an operating element, an operating element type, a start time, an element time, an operating time, an invalid time, a comparison value, a difference, a classification, unsteady information, a rating rate, a standard time, etc. The invalid total time, the total comparison value, the total difference, the overall rating rate, the semi-operating total time, the operating total time, the standard total time, and the like are calculated (step S15).

The CPU 11 displays the total column 523 based on the total element time, total operation time, total invalid time, total difference, overall rating rate, total semi-operation time, standard total time, etc., and total element time, total operation time, and total invalid time. The summary table 54 is displayed based on the time and the like (step S16). The CPU 11 generates and displays an operation graph 55 based on the total element time, the total operation time, the quasi-total operation time, and the invalid total time (step S17). The total element time can be calculated by summing the total invalid time, the total semi-operating time, and the total operating time. Therefore, the operation graph 55 can be generated and displayed based on the total invalid time, the quasi-operation total time, and the operation total time. The CPU 11 receives the selection of the operating element at the input unit 14 (step S18). The CPU 11 displays the moving image data of the operating elements at the same time in the display window 51 among the plurality of moving image data corresponding to the received operating elements (step S19). The CPU 11 displays the manual corresponding to the received operation element in the display window 51 (step S20).

The CPU 11 receives the input of the individual comment and the general comment of the operation element received in the individual comment field 56 and the general comment field 57 in the input unit 14 (step S21). The CPU 11 displays the received individual comment and the general comment in the individual comment column 56 and the general comment column 57 (step S22), shifts the process to S11, and repeats the process.

The display window 51A in the present embodiment may be capable of displaying an angle between any two lines. Further, in the present embodiment, the operation graph 55 is generated and displayed based on the total element time, the total operation time, the quasi-total operation time, and the invalid total time, but the operation graph 55 is generated based on the total operation time and the invalid total time. And may be displayed.

In one aspect, it is possible to easily analyze a plurality of objects.

In one aspect, it is possible to easily analyze a plurality of objects on the same time axis.

On one aspect, since the type graph 53 can display the element time for each type on the same time axis, it becomes easy to compare each element, and it becomes easy to find a process that can improve efficiency.

On one side, displaying individual comments for each video data can be useful for understanding each video data.

For example, the user can immediately confirm in the operation graph 55 that the ratio of the invalid total time or the semi-operation total time is large. Therefore, the user can perform flexible rating work such as rating from the work with a large amount of wasted time by checking the operation graph 55.

On one side, the analysis can be performed in comparison with the work of other workers, allowing more efficient standard time setting.

Embodiment 2
The second embodiment relates to an embodiment that acquires biological information measured from an object or environmental information that measures the environment around the object and displays it in a display window 51. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the first embodiment, and the description thereof will be omitted for the sake of brevity. The information processing device 1 in the present embodiment includes a time measuring unit (not shown). The timekeeping unit measures the time after capturing the moving image and outputs the time measured to the CPU 11. The time after the moving image is captured is the playback time of the moving image. FIG. 10 is an explanatory diagram showing an outline of the information processing system. The information processing system further includes biosensors 61A and 61B for measuring biometric information. The biosensors 61A, 61B and the like may be represented by the biosensor 610. Each of the biosensors 61A and 61B is, for example, a pressure sensor, and is attached to each of the workers A and B. The pressure sensor is an acceleration sensor such as a capacitance type acceleration sensor or a piezoresistive type acceleration sensor. The biosensor 610 measures the pulse rate of the waist or arm of the workers A and B, and outputs the measured pulse rate to the information processing device 1. The CPU 11 acquires the pulse rate from the biosensor 610. The CPU 11 displays the acquired pulse rate in the display window 51E as a time-series biological graph.

FIG. 11 is an explanatory diagram showing an example of the analysis screen 5. In the display window 51E, a biograph of the pulse rate acquired from the biosensors attached to the workers A and B is displayed. The vertical axis is the pulse rate, and the unit is bpm. The horizontal axis is time and the unit is seconds. The user can know the muscle load of the workers A and B from the biological graph displayed on the display window 51E. Therefore, it is possible to objectively judge the slowness of the worker's work speed, instead of subjectively judging the slowness of the worker's work speed. Specifically, even if the worker feels that the work speed is slow, if the muscle load during that time period is large, it can be judged that the worker's work speed is not slow and is appropriate. ..

FIG. 12 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. Since the processing of steps S11 to S22 is the same as the information processing system according to the first embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S19, the CPU 11 acquires biometric information from the biosensor 610 by the communication unit 16 (step S31). The CPU 11 displays the biological graph in the display window 51 based on the acquired biological information (step S32), and shifts the process to S20.

The CPU 11 in the present embodiment acquires the biological information of the object and displays the biological graph based on the acquired biological information, but the present invention is not limited to this. The CPU 11 may acquire environmental information such as temperature or humidity measured by an environmental sensor such as a hygrometer or a thermometer, and display an environmental graph displaying the environmental information in chronological order based on the acquired environmental information. Specifically, in a workplace where the humidity or temperature changes depending on the season, it is possible to set the standard time according to the season while checking the humidity or temperature change.

By confirming the biological information or the environmental information, the user can objectively determine the rating rate, which has been subjectively performed, and can calculate the standard time with higher accuracy.

Embodiment 3
The third embodiment relates to an embodiment in which the standard total time is calculated based on the classification. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the first embodiment, and the description thereof will be omitted for the sake of brevity. The CPU 11 refers to FIG. 3 and extracts the operating elements “welding”, “panel fixing”, and “metal fitting A set” whose divisions are semi-operational. The CPU 11 has a standard total time "16" which is the sum of the standard times "3.85", "3.00" and "9.60" of the extracted operating elements "welding", "fixing the panel" and "setting the metal fitting A". .45 ”is calculated, and the calculated standard total time“ 16.45 ”is displayed in the total column 523 and the summary table 54.

FIG. 13 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. Since the processing of steps S11 to S22 is the same as the information processing system according to the first embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S14, the CPU 11 extracts an operating element whose division is semi-operation (step S41). The CPU 11 calculates the standard total time of the extracted operating elements (step S42), and shifts the process to S15.

Since the CPU 11 can calculate the standard total time based on the classification, it is possible to calculate the standard total time excluding the operation elements mainly performed by the machine. As a result, operating elements that do not need to be analyzed can be omitted, so that the time can be shortened.

Embodiment 4
The fourth embodiment relates to an embodiment that displays a viewpoint analysis screen 6 for analyzing one moving image from a plurality of viewpoints. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the first embodiment, and the description thereof will be omitted for the sake of brevity.

FIG. 14 is an element information DB 12A for storing data relating to the worker A. FIG. 15 is an element information DB 12B for storing data related to the worker B. The element information DB 121 in the present embodiment includes a plurality of databases such as the element information DB 12A and the element information DB 12B for each viewpoint of analyzing the operating element. The viewpoint in the present embodiment is for observing each object, but is not limited to this. The viewing element information DB 121 includes a measurement number sequence. The number of measurements obtained by measuring the invalid time is stored in the number of measurements column. The CPU 11 increments the number of measurements each time the invalid time is measured in the operating element. The CPU 11 stores the incremented measurement count in the measurement count sequence corresponding to the operating element.

FIG. 16 is an explanatory diagram showing an example of the viewpoint analysis screen 6 in which View 1 is selected. FIG. 17 is an explanatory diagram showing an example of the viewpoint analysis screen 6 in which View 2 is selected. On the viewpoint analysis screen 6, for example, information on the operating elements divided according to the viewpoints of the worker A, the worker B, and the like is displayed. The details of the viewpoint analysis screen 6 will be described below. The viewpoint analysis screen 6 has a display window 61 divided into the upper left part, type tables 68A and 68B provided in the upper right part, an operation element table 62 provided in the lower part of the type tables 68A and 68B, and an operation element table 62. The tabulation table 64 provided at the bottom of the tabulation table 64, the type graphs 63A and 63B provided at the bottom of the tabulation table 64, the operation graph 65 provided at the bottom of the type graphs 63A and 63B, and the operation graph 65 provided at the bottom of the display window 61. The invalid column 66 provided and the individual comment column 67 provided on the right side of the operation graph 65 are provided. A rating column 621 and a non-stationary column 623 are provided at the right end of the operating element table 62. A tab 622 is provided at the top of the operating element table 62. A rating rate column 611 is provided at the left end of the display window 61. A determination rating rate column 612 is provided in the upper left portion of the display window 61. The type graphs 63A, 63B, etc. are represented by the type graph 63 depending on the case. The type graphs 63A, 63B, etc. are displayed by aligning the time axis for each type, such as type 1 and type 2 or type 2 and type 3. The operation graph 65 is composed of a non-operating part 65A, a semi-operating part 65B, and a moving part 65C. The invalid column 66 includes invalid ratio graphs 66A, 66B, 66C and 66D, a measurement number column 661, and an overall measurement column 662. The invalidity ratio graphs 66A, 66B, 66C and 66D are optionally represented by the invalidity ratio graph 660. The type tables 68A and 68B are represented by the type table 68 as the case may be. The display window 61, the type graph 63, the summary table 64, the operation graph 65, and the individual comment column 67 have the same configuration and operation as the display window 51, the type graph 53, the summary table 54, the operation graph 55, and the individual comment column 56. The description is omitted for the sake of brevity.

The type table 68 is a table showing the types of operating elements. The tab 622 is a tab for switching the operation element table 62 for each viewpoint. The invalid column 66 is a column for displaying information regarding the invalid time. The invalidity ratio graph 660 is a bar graph showing the ratio of the invalidity time to the element time. The measurement number column 661 is a column for displaying the measurement number. The total measurement column 662 is a column for displaying the total number of measurements indicating the number of times the invalid time is measured in the acquired moving image.

The operation of the viewpoint analysis screen 6 is as follows. As shown in FIG. 16, when the user selects “View 1” on the tab 622 in the input unit 14, the CPU 11 refers to the element information DB 12A and displays the information on the operating elements related to the worker A in the display window 61, the operating element table 62, The type graph 63, the summary table 64, and the operation graph 65 are displayed. The CPU 11 displays the type of the operation element of the worker A in the type table 68A, and displays the type of the operation element of the work type in the type table 68B. In the invalid column 66, the CPU 11 displays the ratio of the invalid time to the element time at the lower side of the invalid ratio graph 660, and displays the ratio of the operation time to the element time at the upper side of the invalid ratio graph 660. For example, when the CPU 11 selects the operation element in the first row of the operation element table 62, the CPU 11 selects the leftmost invalid ratio graph 66A corresponding to the row number. The CPU 11 refers to the element information DB 12A and displays the number of measurements corresponding to the selected operating element in the number of times of measurement column 661. The CPU 11 calculates the total number of measurements by summing the number of measurements in the number of measurements columns of a plurality of databases such as the element information DBs 12A and 12B. The CPU 11 displays the calculated total number of measurements in the total measurement column 662.

As shown in FIG. 17, when the user selects “View 2” on the tab 221 in the input unit 14, the CPU 11 refers to the element information DB 12B and displays the information on the operating elements related to the worker B in the display window 61, the operating element table 62, The type graph 63, the summary table 64, the operation graph 65, the type table 68, and the invalid column 66 are displayed.

By switching the tab 622, the user can compare the information of the worker A and the worker B and use it for setting the standard time of each of the worker A and the worker B. For example, when the invalid time of the operation element "welding on the right side" of the worker A is "0.50" and the invalid time of the operation element "weld on the left side" of the worker B is "1.00". The user can see from the invalidation ratio graph 660 that the ratio of the invalid time of the worker A to the worker B is small. The user knows that the number of measurements corresponding to the worker A is "3" and the number of measurements corresponding to the worker B is "4" in the measurement number column 661. The user knows that the total number of measurements corresponding to the worker A is "12" and the total number of measurements corresponding to the worker B is "16" in the total measurement column 662.

From the above, the user can understand that the worker A is a skilled worker. As a result, the user inputs a value obtained by increasing the rating rate of the operating element corresponding to the worker A in the input unit 14. The CPU 11 calculates an appropriate standard time based on the input rating rate and operating time. The user selects "View 1" on the tab 221 in the input unit 14, and the CPU 11 accepts the input in the individual comment field 67 in the input unit 14. The user inputs "worker A is skilled" in the individual comment field 67. The CPU 11 displays "Worker A is skilled" in the individual comment column 67. The user selects "View 2" on the tab 622 in the input unit 14, and the CPU 11 accepts the input in the individual comment field 67 in the input unit 14. The user inputs "worker B is a newcomer" in the individual comment field 67. The CPU 11 displays "Worker B is a newcomer" in the individual comment column 67.

18 to 19 are flowcharts showing the processing procedure of the information processing system according to the present embodiment. The CPU 11 accepts the selection of the viewpoint on the tab 221 at the input unit 14 (step S51). The CPU 11 reads the element information DB 121 based on the selected viewpoint (step S52). The CPU 11 reads the operating element, non-stationary information, type, start time, element time, operating time, invalid time, comparison value, difference, classification, rating rate, standard time, etc. from the storage unit 12 or accepts input from the input unit 14. (Step S53), the process is moved to S12. Since the processes of steps S12 to S18 are the same as the information processing system according to the first embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of S18, the CPU 11 displays one moving image data in the display window 61 (step S54). The CPU 11 displays the invalid ratio graph 660 based on the element time, the operating time, and the invalid time (step S55). The CPU 11 displays the number of measurements and the total number of measurements in the measurement number column 661 and the total measurement column 662, respectively (step S56). The CPU 11 displays an individual comment in the individual comment field 67 (step S57), and the input unit 14 accepts the selection of the viewpoint on the tab 221 (step S58). The CPU 11 determines whether or not a different viewpoint is selected (step S59). When the CPU 11 determines that a different viewpoint has not been selected (step S59: NO), the CPU 11 waits until a different viewpoint is selected. When the CPU 11 determines that a different viewpoint has been selected (step S59: YES), the CPU 11 switches to a screen for displaying the operation element table 62 and the like having different contents (step S60), shifts the process to step S52, and repeats the process.

On one aspect, by switching the display of the type graph 63 for each viewpoint by tab 221, it becomes easy to compare the type or operation time for each viewpoint.

On one side, by switching the tab 622, it is possible to compare each object, so that the superiority or inferiority of the work between the workers can be immediately understood. This can be useful in calculating the rating rate.

On one aspect, by switching the display of the operation graph 65 for each viewpoint by tab 622, it becomes easy to compare the invalid total time or the quasi-operation total time for each viewpoint. This can be useful in calculating the rating rate.

On one side, displaying the invalidity ratio graph 660 makes it easy to understand the ratio of invalid time for each type. This can be useful in calculating the rating rate.

On one side, by displaying non-stationary information for each operating element, the user can understand whether or not the work is periodic. Therefore, in the case of periodic work, it is possible to use it for calculating the rating rate, such as setting the rating rate low because it is accustomed to it.

Embodiment 5
The fifth embodiment relates to an embodiment that displays a cycle screen 7 for analyzing information regarding non-stationary information. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the first embodiment, and the description thereof will be omitted for the sake of brevity. The storage unit 12 further includes a non-stationary information DB 127.

FIG. 20 is a diagram showing an example of data stored in the non-stationary information DB 127. The non-stationary information DB 127 is a database that stores information related to non-stationary information. The non-stationary information DB 127 includes an operating element sequence, a non-stationary sequence, a type 1 row, a 1-cycle row, a 2-cycle row, a 3-cycle row and the like. Hereinafter, in some cases, the 1-cycle row, the 2-cycle row, the 3-cycle row, and the like are represented by the cycle row. The cycle column indicates the number of cycles indicating the number of times the work in the operating element is repeated, and includes a start time column and an element time column. The operation element sequence, non-stationary column, type 1 column, start time column, and element time column of the non-stationary information DB 127 have the same configuration as the element information DB 121, and the description thereof will be omitted for the sake of brevity. The CPU 11 refers to the element information DB 121, and stores the operating element, the unsteady information, and the type in the unsteady information DB 127 in the operating element sequence, the unsteady column, and the worker A column in the cycle table 72. The CPU 11 receives the input of the start time and the element time of each cycle column on the cycle screen 7 in the input unit 14, and stores the accepted start time and the element time in the start time column and the element time string of each cycle column.

FIG. 21 is an explanatory diagram showing an example of the cycle screen 7. The cycle screen 7 has a display window 71 divided in the upper left part, a type table 78 provided in the upper right part, a cycle table 72 provided in the lower part of the type table 78, and a cycle comment provided in the lower part of the display window 71. A column 77 is provided. A rating rate column 711 is provided at the left end of the display window 71. A determination rating rate column 712 is provided in the upper left portion of the display window 71. The cycle table 72 has a total column 721 at the bottom. The cycle table 72 is a table for displaying information regarding non-stationary information. The display window 71, the cycle comment column 77, and the type table 78 have the same configuration and operation as the display window 61, the individual comment column 67, and the type table 68, and the description thereof is omitted for the sake of brevity.

The operation of the cycle screen 7 is as follows. The CPU 11 accepts the display selection of the cycle screen 7 at the input unit 14. The user selects the display of the cycle screen 7, and the CPU 11 displays the cycle screen 7. The CPU 11 refers to the element information DB 121, and displays the operation element, the unsteady information, and the type in the operation element column and the worker A column in the cycle table 72. The CPU 11 determines whether or not the non-stationary information of the operating element indicates a periodic operation.

The CPU 11 determines that the unsteady information sequence of the operating elements "welding", "fixing the panel", and "tightening the bolt" is x. For example, when the worker A performs the work of the operating elements "welding", "fixing the panel" and "tightening the bolt" three times, the CPU 11 performs the operation elements "welding", "fixing the panel" and "bolting". Inputs are received by the input unit 14 in the 1-cycle row, the 2-cycle row, the 3-cycle row, and the like of "tightening". The user inputs the start time and the element time of the 1-cycle column, the 2-cycle column, and the 3-cycle column in the input unit 14. When the CPU 11 determines that the non-stationary information of the operating element indicates a non-periodic operation, the CPU 11 refers to the element information DB 121, and sets the start time of the element information DB 121 and the element information DB 121 in one cycle sequence corresponding to the operating element "set metal fitting A". Display element time. The CPU 11 calculates the total element time of the 1-cycle column, the 2-cycle column, and the 3-cycle column "27.00", "15.00", and "18.00", and the total calculated element time is "27.00". , "15.00" and "18.00" are displayed in the total column 721 at the bottom of each cycle column.

FIG. 22 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. The CPU 11 accepts the display selection of the cycle screen 7 at the input unit 14. Since the processing of steps S11 to S22 is the same as the information processing system according to the first embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S22, the CPU 11 accepts the display selection of the cycle screen 7 at the input unit 14 (step S61). The CPU 11 displays the cycle table 72 based on the element information DB 121 (step S62). Specifically, the CPU 11 refers to the element information DB 121, and displays the operation element, the unsteady information, and the type in the operation element column and the type 1 column of the cycle table 72.

The CPU 11 determines whether or not the non-stationary information of the operating element is a periodic operation (step S63). When the CPU 11 determines that the non-stationary information of the operating element indicates a periodic operation (step S63: YES), the input unit inputs the start time and the element time to each cycle column corresponding to the operating element in the cycle table 72. Accepted at 14 (step S64). The CPU 11 displays the received start time and element time in the start time column and element time column of each cycle column (step S65), and shifts the process to S67. When the CPU 11 determines that the non-stationary information of the operating element indicates a non-periodic operation (step S63: NO), the CPU 11 refers to the element information DB 121, and sets the start time of the element information DB 121 and the element information DB 121 in one cycle sequence corresponding to the operating element. The element time is displayed (step S66). The CPU 11 determines whether or not all the operating elements have been determined (step S67). When the CPU 11 determines that all the operating elements have not been determined (step S67: NO), the process shifts to step S63. When it is determined that the CPU 11 has determined all the operating elements (step S67: YES), the total of the element times is calculated and displayed in the total column 721 (step S68), and the process is moved to S11.

According to one aspect, periodic movements can be analyzed in detail.

Embodiment 6
The sixth embodiment relates to an embodiment in which information about operating elements is created by using a work sampling method. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those of the fourth and fifth embodiments, and the description thereof will be omitted for the sake of brevity. Work sampling is to instantly measure each work performed by a worker and statistically aggregate and classify it. Since the observation is performed instantaneously, the operator can perform the work without being aware of the shooting action, and therefore, it is generally used as a method of observing in a state relatively close to the actual measurement. The specific work sampling method is as follows. Images of the object are extracted a predetermined number of times from the moving image at regular intervals or random intervals, and the composition ratio of the number of times related to the work type performed by the object is calculated by analyzing the extracted images of the object. The user is a method of creating information on operating elements such as standard time, operating time, and invalid time on the analysis screen 5, the viewpoint analysis screen 6, the cycle screen 7, and the like based on the calculated composition ratio. Hereinafter, the details of the information processing system according to the present embodiment will be described.

The storage unit 12 according to the present embodiment further includes a sampling table 125, a corresponding table 126, and a sampling aggregation table 127. FIG. 23 is a diagram showing an example of data stored in the sampling table 125 and the corresponding table 126. The sampling table 125 is a database that stores sampling data used in the work sampling method. The sampling table 125 in this embodiment includes a plurality of databases such as a sampling table 12A and a sampling table 12B for each number of samples (objects). The number of samples is the number of workers shown in one image. FIG. 23A is a diagram showing an example of data stored in the sampling table 12A of the worker A. FIG. 23B is a diagram showing an example of data stored in the sampling table 12B of the worker B.

The sampling table 125 includes an image ID column, an image string, a work type column, an operation information column, and the like. An image ID for identifying an image is stored in the image ID column. The extracted image is stored in the image string in association with the image ID. In the work type column, the work type indicating the type of work performed by the object corresponding to the displayed image is stored. The work type is, for example, "welding on the left side" or "assembly by a robot". For example, it is stored in the work type column that the work type of the worker A is "welding on the left side" or the work type of the worker A is "assembly by a robot". In the operation information column, operation information indicating whether or not the object is operating is stored in association with the image ID. For example, operation, non-operation, or semi-operation is stored in the operation information column. Operation indicates that the object is operating. Non-operation indicates that the object is non-operation. Semi-operation indicates that the object is in semi-operation.

The CPU 11 receives input of the work type, operation information, and image ID on the work sampling screen 8 described later by the input unit 14, and stores the received information in the corresponding table 126.

FIG. 24 is a diagram showing an example of data stored in a correspondence table 126 for associating a work process collected for each of a plurality of work types with an image ID and the like. The correspondence table 126 includes a work process column and an image ID column. The work process is stored in the work process column. The image ID is stored in the image ID column in association with the work process. The CPU 11 accepts the selection of the work process and the image ID, and stores the received work process and the image ID in the corresponding table 126 in association with each other.

FIG. 25 is a diagram showing an example of data stored in the sampling aggregation table 127. The sampling aggregation table 127 is a database that stores sampling aggregation data that aggregates sampling data. The sampling aggregation table 127 includes a generated moving image sequence, a work process column, a sampling number sequence, a working sequence, a semi-operating sequence, and a non-operating sequence. The name of the generated moving image generated from the image is stored in the generated moving image column. The generated moving image is generated by extracting images from the acquired moving image and arranging the extracted images in chronological order. The work process is stored in the work process column. The sampling count is stored in the sampling count sequence. The number of samplings is the number of acquisitions of images. In the operation number sequence, the number of operations obtained by counting the number of images whose operation information is "operation" is stored. The quasi-operation number sequence stores the quasi-operation number that counts the number of images whose operation information is "quasi-operation". The non-operating number sequence stores the number of non-operating numbers that counts the number of images whose operation information is "non-operating". The total of the sampling count, the working number, the semi-working number, and the non-working number is stored in the last row of the sampling number column, the working number column, the semi-working number column, and the non-working number column, respectively. The CPU 11 automatically generates the name of the moving image. The CPU 11 stores the name of the generated moving image in the sampling aggregation table 127. The CPU 11 calculates the number of samplings, the number of operations, the number of semi-operations, and the number of non-operations by the calculation method described later, and stores the calculated number of samplings, the number of operations, the number of semi-operations, and the number of non-operations in the sampling aggregation table 127.

FIG. 26 is an explanatory diagram showing an example of a work sampling screen 8 for executing the work sampling method. The work sampling screen 8 includes an observation mode selection field 82 provided in the upper left, a measurement selection field 83 provided at the bottom of the observation mode selection field 82, and a start button 84 provided at the bottom of the measurement selection field 83. An image field 81 provided on the right side of the observation mode selection field 82, a measurement selection field 83, and a start button 84, a work type table 85 provided at the bottom of the image field 81, and a work type table 85 provided at the bottom of the work type table 85. The sampling table 86, the calculation column 87 provided in the lower left part of the sampling table 86, the operation graph 88 provided in the right part of the calculation column 87, the image columns 81A and 81B provided in the right part, and the like. It includes buttons 89A, 89B, 89C, 89D and the like provided on the upper right, a registration button 80, and a button 810.

Hereinafter, the image fields 81A, 81B, etc. are represented by the image field 811 in some cases. The operation graph 88 is composed of a non-operating part 88A, a semi-operating part 88B, and a moving part 88C. Hereinafter, the buttons 89A, 89B, 89C, 89D and the like are represented by the button 89 in some cases. The image field 81 is a field for displaying the selected image. The image field 811 is a field for displaying the acquired image. The observation mode selection field 82 is a field for selecting an observation mode. The measurement selection field 83 is a field for selecting a measurement method. The start button 84 is a button for executing work sampling. The work type table 85 is a table for displaying the work types. The sampling table 86 is a table displaying the data used in the work sampling method. The calculation column 87 is a column for inputting into the formula for calculating the required number of samplings in order to obtain sufficient certainty. The operation graph 88 is a pie chart showing the ratio of the number of operations, the number of semi-operations, and the number of non-operations. Each of the buttons 89A, 89B, 89C and 89D is a button for displaying each of the analysis screen 5, the viewpoint analysis screen 6, the cycle screen 7 and the work sampling screen 8. The registration button 80 is a button for registering a work type. Since the button 810 is used in the embodiment described later, the description thereof will be omitted in the present embodiment.

The operation of the work sampling screen 8 is as follows. The CPU 11 accepts the selection of the registration button 80 by the input unit 14 for the registration button 80 described later, and when the user selects the registration button 80 by the input unit 14, the CPU 11 inputs the work type to the work type table 85 by the input unit 14. Accept. The CPU 11 displays the received work type in the work type table 85 and stores it in the sampling table 125. In the observation mode selection field 82, an observation mode for each time zone and a multi-person observation mode can be selected. When the user selects the time zone observation mode in the input unit 14, the work sampling is executed by paying attention to one object. When the user selects the multi-person observation mode in the input unit 14, the work sampling is executed by paying attention to a plurality of objects.

In the measurement selection field 83, random timetable measurement and scheduled measurement can be selected. When the user selects random timetable measurement in the input unit 14, the CPU 11 extracts an image from the acquired moving image at a random time. When the user selects scheduled measurement in the input unit 14, the CPU 11 extracts images from the acquired moving image at predetermined time intervals. The predetermined time interval is, for example, 0.1 seconds.

For example, when the user selects the observation mode for each time zone in the input unit 14, selects the random timetable measurement, and selects the start button 84, the image of the worker A is randomly extracted from the acquired video and extracted. The image of the worker A is displayed in the image column 811. Further, for example, when the user selects the multi-person observation mode in the input unit 14, selects the scheduled measurement, and selects the start button 84, the images of the workers A and B are extracted from the acquired moving images at predetermined time intervals. Then, the extracted images of the workers A and B are displayed in the image column 811.

When the user selects, for example, the image A of the image field 81A, the CPU 11 displays the image A in the image field 81. The CPU 11 accepts the selection of the work type in the work type table 85 by the input unit 14. The user selects "welding on the left side" from the work type table 85 on the input unit 14. The CPU 11 associates the image ID "001" of the image A with the work type "welding on the left side" and stores it in the sampling table 125. The CPU 11 receives the input of operation information at the input unit 14. The user inputs the operation information "quasi-operation". The CPU 11 stores the received operation information "quasi-operation" in the sampling table 125 in association with the image ID "001" of the image A.

After the user finishes associating the image ID, work type, and operation information, the CPU 11 refers to the corresponding table 126 and the sampling table 125, and determines the number of samplings, the number of operations, the number of semi-operations, and the number of non-operations corresponding to the work process. calculate. Specifically, since the image ID corresponding to welding in the CPU 11 is the image ID "001-004", as shown in FIGS. 23A and 23B, the number of samplings is "7", the number of operations is "2", and the number of semi-operations is " 3 ”and the number of non-operating numbers“ 2 ”are calculated. The same applies hereinafter, and the description is omitted for the sake of brevity. The CPU 11 calculates the total of the calculated number of samplings, the number of operations, the number of quasi-operations and the number of non-operations, divides the total of the calculated number of operations, the number of quasi-operations and the number of non-operations by the number of samplings, and integrates 100. Calculate the ratio. The composition ratio is the ratio of the number of operating, the number of semi-operating and the number of non-operating. The CPU 11 displays the name of the moving image, the work process, the number of samplings, the number of operations, the number of semi-operations, and the number of non-operations in the sampling table 86. The CPU 11 stores the name of the moving image, the work process, the number of samplings, the number of operations, the number of semi-operations, and the number of non-operations in the sampling aggregation table 127.

The CPU 11 receives a numerical value in each column of the calculation column 87 by the input unit 14. The user inputs a numerical value in each field. The CPU 11 substitutes the input numerical value into the equation and calculates the required number of samplings. The formula displayed in the calculation column 87 is, for example, as follows.
N = t × t × (1-P) ÷ (S × S × P).
N: Required number of samplings, t: Reliability, S: Accuracy, P: Composition ratio The CPU 11 determines whether or not the number of samplings is equal to or less than the required number of samplings. When the CPU 11 determines that the number of samplings is less than or equal to the required number of samplings, the CPU 11 displays a predetermined message in the calculation column 87. The predetermined message is, for example, "insufficient sampling count".

The CPU 11 refers to the sampling table 125 and generates an operation graph 88 based on the operation information and the working time. The process by which the CPU 11 generates the operation graph 88 is as follows. The CPU 11 generates the non-operating portion 88A at the ratio of the number of non-operating numbers to the number of samplings. The CPU 11 generates the quasi-operating portion 88B at the ratio of the quasi-operating number to the number of samplings. The CPU 11 generates the operating portion 88C at the ratio of the operating number to the sampling number. The CPU 11 generates and displays an operation graph 88 based on the generated non-operating portion 88A, semi-operating portion 88B, and operating portion 88C. That is, the CPU 11 displays the ratio at which the object in each image is operating. For example, the user can immediately confirm in the operation graph 88 that the ratio of the number of non-operations, the number of operations, or the number of semi-operations is large. Therefore, the user can determine the approximate rating rate by checking the operation graph 88.

When the user selects each of the buttons 89A, 89B and 89C in the input unit 14, the CPU 11 displays the analysis screen 5, the viewpoint analysis screen 6 and the cycle screen 7, respectively. Buttons 89A, 89B and 89C are also provided on the analysis screen 5, the viewpoint analysis screen 6 and the cycle screen 7, and perform the same operation. Further, when the user selects the button 89D on the analysis screen 5, the viewpoint analysis screen 6, and the cycle screen 7, the work sampling screen 8 is displayed.

27 to 29 are flowcharts showing the processing procedure of the information processing system according to the present embodiment. Since the processes of steps S11 to S22, S51 to S58 and S61 to 68 are the same as those of the information processing system according to the above-described embodiments 4 and 5, the description thereof will be omitted for the sake of brevity. The CPU 11 receives the observation mode input from the input unit 14 at the input unit 14 (step S71). The CPU 11 receives the input of the measurement method from the input unit 14 at the input unit 14 (step S72). The CPU 11 determines whether or not the start button 84 has been selected by the input unit 14 (step S73). When the CPU 11 determines that the start button 84 has not been selected by the input unit 14 (step S73: NO), the processing shifts to S71. When the CPU 11 determines that the start button 84 has been selected by the input unit 14 (step S73: YES), the CPU 11 executes image extraction by the work sampling method based on the observation mode and the measurement method (step S74).

The CPU 11 receives an image selection at the input unit 14 at the input unit 14 (step S75). The CPU 11 receives the input of the work type and the operation information in the input unit 14 (step S76). The CPU 11 associates the work type and the operation information with the image ID of the received image (step S77). The CPU 11 refers to the corresponding table 126 and the sampling table 125, and calculates the number of samplings, the number of operations, the number of semi-operations, and the number of non-operations corresponding to the work process (step S78). The CPU 11 calculates the total of the calculated number of samplings, the number of operations, the number of quasi-operations and the number of non-operations, divides the total of the calculated number of operations, the number of quasi-operations and the number of non-operations by the number of samplings, and integrates 100. The ratio is calculated (step S79). The CPU 11 displays the name of the moving image, the work process, the number of samplings, the number of operations, the number of semi-operations, and the number of non-operations in the sampling table 86 (step S80). The CPU 11 receives an input in each field of the calculation field 87 by the input unit 14 (step S81). The CPU 11 calculates the required number of samplings based on the values input in each column of the calculation column 87 (step S82). The CPU 11 determines whether or not the number of samplings is equal to or less than the required number of samplings (step S83).

When the CPU 11 determines that the number of samplings is not less than or equal to the required number of samplings (step S83: NO), that is, when it determines that the number of samplings exceeds the required number of samplings, the processing shifts to S85. When the CPU 11 determines that the number of samplings is equal to or less than the required number of samplings (step S83: YES), the CPU 11 displays a predetermined message in the calculation column 87 (step S84). The CPU 11 displays the operation graph 55 based on the total of the number of operations, the number of semi-operations, the number of non-operations, and the number of samplings (step S85).

The CPU 11 determines whether or not the display of the analysis screen 5 is selected (step S86). That is, the CPU 11 determines whether or not the button 89A is selected. When the CPU 11 determines that the display of the analysis screen 5 is selected (step S86: YES), the process shifts to step S11 and the display process of the analysis screen 5 is performed. When the CPU 11 determines that the display of the analysis screen 5 has not been selected (step S86: NO), it determines whether or not the display of the viewpoint analysis screen 6 has been selected (step S87). That is, the CPU 11 determines whether or not the button 89B is selected. When the CPU 11 determines that the display of the viewpoint analysis screen 6 is selected (step S87: YES), the process shifts to S51 and the display process of the viewpoint analysis screen 6 is performed.

When the CPU 11 determines that the display of the viewpoint analysis screen 6 has not been selected (step S87: NO), it determines whether or not the display of the cycle screen 7 has been selected (step S88). That is, the CPU 11 determines whether or not the button 89C is selected. When the CPU 11 determines that the display of the cycle screen 7 is selected (step S88: YES), the process shifts to S61. When the CPU 11 determines that the display of the cycle screen 7 is not selected (step S88: NO), the process shifts to S71.

According to one aspect, the moving images using the work sampling method are the moving images observed in a state relatively close to the actual measurement, and by analyzing these moving images or performing viewpoint analysis, more accurate analysis / analysis results can be obtained. You can get it.

In addition, according to one aspect, the composition ratio of the operating status can be displayed immediately after using the work sampling method, and the analysis time can be increased because the composition ratio can be immediately moved to the analysis screen of the moving image. It can be shortened.

Embodiment 7
The seventh embodiment relates to an embodiment in which work types, operation information, and images are associated with each other by using image processing such as template matching or pattern matching. Template matching is used for image processing in this embodiment. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the fifth embodiment, and the description thereof will be omitted for the sake of brevity.

The storage unit 12 further includes a template DB 128. FIG. 30 is a diagram showing an example of data stored in the template DB 128. The template DB 128 is a database that stores a template image (work image) used for image processing in association with a work type. The template DB 128 includes a template image string, a work type column, an operation information string, and the like. The template image is stored in the template image column. A template image is, for example, an image in which an object is performing work of a work type. The work type is stored in the work type column. Operation information is stored in the operation information column.

The method of storing the template image and the work type in the template DB 128 is as follows. When the user inputs the registration button 80 in the present embodiment by the input unit 14, the CPU 11 displays an image registration screen 9 for registering the template image shown in FIG. FIG. 31 is an explanatory diagram showing an example of the image registration screen 9. The image registration screen 9 includes operation information registration fields 91A and 91B provided in the upper left portion and the lower left portion, and an image registration column 92 provided in the right portion. Hereinafter, in some cases, the operation information registration columns 91A and 91B are represented by the operation information registration column 91. The operation information registration field 91 includes a work type field 911 provided at the top, an operation information field 912 provided at the right side of the work type field 911, and an image field 913 and a template image field provided at the left and right parts. It is equipped with 914. The image registration field 92 includes a template image field 921 provided at the upper part, a range designation field 922 provided at the lower left of the image field 921, a threshold value field 923 provided at the right side of the range designation field 922, and a range designation. It includes a column 922 and a comment column 924 provided at the bottom of the threshold column 923.

The operation information registration column 91 is a column for registering operation information. The image registration field 92 is a field for registering a template image. The work type column 911 is a field for inputting a work type. The operation information column 912 is a check box for selecting operation information. The image column 913 is a column for displaying the extracted image. The template image field 914 is a field for displaying the range of the template image. The template image field 921 is a field for displaying the range of the template image. The range designation field 922 is a field for designating the range of the template image. The threshold value column 923 is a column for setting a threshold value of the degree of agreement with the template image. The comment field 924 is a field for displaying a comment on the template image.

The operation of the image registration screen 9 is as follows. The CPU 11 displays the selected image A in the operation information registration field 91A in the image field 913 and the template image field 921. The CPU 11 accepts the selection of the range designation method in the range designation field 922 in the input unit 14. The range designation method is, for example, a rectangle, an ellipse, a lasso, or the like. For example, when the user selects a lasso in the input unit 14, the CPU 11 can display a ring having an Nth-order curve in the template image field 921. The CPU 11 receives the input of the threshold value in the threshold value field 923 by the input unit 14. When the user inputs an arbitrary threshold value in the input unit 14, the input unit 14 accepts the input of the range to be the template image in the template image field 921. The user surrounds the worker A and the machine by the input unit 14 with a ring having an Nth-order curve. The CPU 11 extracts an image surrounded by a ring, and stores the extracted image in the template DB 128 as a template image A corresponding to the work type “welding on the left side”.

The CPU 11 displays the same image as the image in the template image field 921 in the template image field 914. The CPU 11 receives the input of the work type in the work type field 911 in the input unit 14. When the user inputs the work types "welding on the left side" and "welding on the right side" in the work type column 911, the template image A is associated with the work types "welding on the left side" and "welding on the right side" in the template DB 128. Remember. The CPU 11 receives the selection of the operation information in the operation information field 912 by the input unit 14. When the user selects the operation information "quasi-operation" in the operation information field 912, the template image A and the operation information "quasi-operation" are associated with each other and stored in the template DB 128. The CPU 11 receives a comment in the comment field 924 by the input unit 14. The user inputs "a human being is set as a template image" in the input unit 14. Hereinafter, since the image registration screen 9 performs the same operation for the operation information registration field 91B, the description thereof will be omitted.

When the user inputs the button 810 with the input unit 14, the image is extracted by the work sampling method based on the observation mode and the measurement method, the template image is read from the template DB 128, and the extracted image A is subjected to template matching. To do. The CPU 11 calculates the degree of agreement between the extracted image A and the template image A. Specifically, the CPU 11 calculates the correlation between the image A and the template image A by taking a correlation function between the extracted image A and the template image A. That is, the degree of agreement is a correlation value between the extracted image and the template image. When the degree of coincidence exceeds the threshold value input in the threshold value field 923, the CPU 11 refers to the template DB 128 and displays the work types "welding on the left side" and "welding on the right side" and the operation information "quasi-operation" corresponding to the template image A. Identify. The CPU 11 associates the specified work types "welding on the left side" and "welding on the right side" and the operation information "quasi-operation" with the image A, and stores them in the sampling table 125. Hereinafter, the CPU 11 performs the same processing on the extracted image.

32 to 33 are flowcharts showing the processing procedure of the information processing system according to the present embodiment. Since the processes of steps S11 to S89 are the same as those of the information processing system according to the sixth embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S74, the CPU 11 determines whether or not the display of the image registration screen 9 is selected (step S91). That is, the CPU 11 determines whether or not the registration button 80 has been input by the input unit 14. When the CPU 11 determines that the display of the image registration screen 9 has not been selected (step S91: NO), the process proceeds to step S95. When the CPU 11 determines that the display of the image registration screen 9 has been selected (step S91: YES), the CPU 11 displays the image registration screen 9 (step S92). The CPU 11 receives the template image, the work type, and the operation information on the image registration screen 9 in the input unit 14 (step S93). The CPU 11 stores the received template image, work type, and operation information in the template DB 128 (step S94).

The CPU 11 determines whether or not template matching has been selected (step S95). That is, the CPU 11 determines whether or not the button 810 has been input by the input unit 14. When the CPU 11 determines that the template matching has not been selected (step S95: NO), the process shifts to S75. When the CPU 11 determines that the template matching has been selected (step S95: YES), the CPU 11 reads the template image from the template DB 128 and executes image processing on the extracted image (step S96). The CPU 11 calculates the degree of agreement between the displayed image and the template image (step S97). The CPU 11 determines whether or not the calculated degree of agreement exceeds the threshold value (step S98). When the CPU 11 determines that the calculated degree of agreement does not exceed the threshold value (step S98: NO), the processing shifts to step S101. When the CPU 11 determines that the calculated degree of agreement exceeds the threshold value (step S98: YES), the CPU 11 refers to the template DB 128 and specifies the work type and operation information corresponding to the template image (step S99). The CPU 11 associates the specified work type and operation information with the image (step S100).

The CPU 11 determines whether or not image processing has been executed for all the images (step S101). When the CPU 11 determines that the image processing has not been executed for all the images (step S101: NO), the processing is moved to step S96. When the CPU 11 determines that the image processing has been executed for all the images (step S101: YES), the processing is transferred to S75.

According to one aspect, the work type can be associated with the image by image processing. As a result, the processing time of the work sampling method can be shortened.

Embodiment 8
In the eighth embodiment, the motion element is selected from the image by associating the image ID of the image with the playback time of the moving image and the moving image ID, and associating the playback time of the moving image with the time of the motion element in the first embodiment. With respect to embodiments. Template matching is used for image processing in this embodiment. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the seventh embodiment, and the description thereof will be omitted for the sake of brevity.

FIG. 34 is a diagram showing an example of data stored in the sampling table 125. The sampling table 125 further includes a reproduction time sequence, a moving image ID column, and an operation element sequence. The playback time of the moving image is stored in the playback time column. A moving image ID for identifying a moving image is stored in the moving image ID column. The operation element associated with the moving image ID is stored in the operation element string. When the image is extracted, the CPU 11 specifies the reproduction time and the moving image ID of the moving image when the image is extracted, and stores the specified playing time and the moving image ID in the sampling table 125. The CPU 11 receives the input of the operating element and stores the accepted operating element in the sampling table 125.

FIG. 35 is a diagram showing an example of data stored in the element information DB 121. A plurality of element information DB 121 shown in FIG. 35 is provided for each object. Each element information DB 121 is associated with a moving image ID. For example, the element information DB 121 shown in FIG. 35 is associated with the moving image ID “11”. Further, the moving image ID "12" is associated with another element information DB 121 (not shown). The CPU 11 specifies the reproduction time and the moving image ID of the moving image from which the image is extracted, and stores the specified playing time and the moving image ID in the sampling table 125 in association with the image.

The CPU 11 can rearrange the images using the playback time associated with the images by the following method. When extracting an image, the CPU 11 specifies the playback time and the moving image ID of the moving image from which the image is extracted. The CPU 11 arranges the image A of the image ID "001" associated with the reproduction time "2.0" in the image field 81A among the moving images of the moving image ID "11", and associates the image A with the reproduction time "4.0". The image C of the obtained image ID "003" is arranged in the image field 81B. The CPU 11 arranges the image B of the image ID "002" associated with the reproduction time "3.0" in the moving image of the moving image ID "12" on the left side of the image field 81A, and has a reproduction time of "5.0". The image D of the image ID “004” associated with is arranged on the left side of the image field 81B. That is, the CPU 11 rearranges the image fields 811 in order of playback time for each moving image ID.

The method of selecting the action element from the image is as follows. The user selects the image A of the image field 811 in the input unit 14. The CPU 11 accepts the selection of the image A via the input unit 14. The CPU 11 displays the received image A in the image field 81. The CPU 11 refers to the sampling table 125, extracts the operation element "welding" associated with the image ID "001" of the image A, and extracts the moving image ID "11" associated with the operation element "welding". When the user selects each of the buttons 89A, 89B and 89C in the input unit 14, the CPU 11 displays each of the analysis screen 5, the viewpoint analysis screen 6 and the cycle screen 7 with the operation element "welding" selected. For example, when the user selects the button 89A, the CPU 11 refers to the element information DB 121 corresponding to the extracted moving image ID "11", displays the analysis screen 5 shown in FIG. 7, and displays the operation element "welding" in the display window 51A. The moving image 101 corresponding to the above is displayed, and the operation element table 52 corresponding to the element information DB 121 of the extracted moving image ID “11” is displayed. Further, for example, when the user selects the button 89B, the CPU 11 refers to the element information DB 121 of the extracted moving image ID "11" and corresponds to the operation element "welding" in the display window 61 of the viewpoint analysis screen 6 shown in FIG. The moving image 101 is displayed, and the state in which the invalid ratio graph 66A corresponding to the operation element "welding" of the element information DB 121 of the extracted moving image ID "11" is selected is displayed. For example, when the user selects the button 89C, the CPU 11 refers to the element information DB 121 of the extracted moving image ID "11", and the moving image 101 corresponding to the operation element "welding" is displayed in the display window 71 of the cycle screen 7 shown in FIG. Is displayed.

FIG. 36 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. Since the processing of steps S11 to S89 is the same as the information processing system according to the seventh embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S74, the CPU 11 specifies the playback time of the moving image and the moving image ID when the image is extracted (step S101). The CPU 11 rearranges the images in order of playback time for each moving image ID (step S102), and shifts the processing to S75. After completing the process of S75, the CPU 11 refers to the sampling table 125, extracts the motion element and the moving image ID associated with the motion element based on the image ID of the received image (step S103), and performs the process in S76. Move.

According to one aspect, since the motion elements can be automatically extracted, the correspondence between the image and the motion elements can be immediately understood.

Embodiment 9
The ninth embodiment relates to an embodiment in which an operation graph calculated by the work sampling method and an operation graph calculated from information on operation elements are compared. Template matching is used for image processing in this embodiment. Hereinafter, configurations and actions other than the configurations and actions particularly described are the same as those in the fifth embodiment, and the description thereof will be omitted for the sake of brevity.

When the user selects the operation graph 88 in the input unit 14, the CPU 11 accepts the input of the moving image ID. The CPU 11 refers to the element information DB 121 corresponding to the received moving image ID, and displays a comparison screen 20 for comparing the operation graph calculated by the work sampling method with the operation graph calculated from the information of the operation elements. FIG. 37 is an explanatory diagram showing an example of the comparison screen 20. The comparison screen 20 is provided in the display window 201 provided in the upper left part, the operation graph 202 provided in the left center part, the comment column 203 provided in the lower part of the operation graph 202, and the upper right part of the operation graph 202. The display window 204, the operation graph (ratio graph) 205 provided at the lower right of the display window 204, and the comment column 206 provided at the lower part of the operation graph 205 are provided. The semi-operating part of the operation graph 202 is composed of the semi-operating parts 207, 208 and 209. The display windows 201 and 204 are windows for displaying moving image data. The operation graph 202 is a pie chart displaying the ratios of the total operation time (operating time), the semi-operating total time (quasi-operating time), and the invalid total time (non-operating time) calculated from the information of the operating elements. The operation graph 202 is generated by the method of the first embodiment. The operation graph 205 is a pie chart displaying the ratios of the number of operations, the number of semi-operations, and the number of non-operations calculated by the work sampling method. The operation graph 205 is generated by the method of the sixth embodiment. The comment fields 203 and 206 are fields for displaying comments of operating elements. The semi-operating parts 207, 208 and 209 are the semi-operating parts further divided by the operating elements. Similarly, the moving portion, the semi-operating portion, and the non-operating portion of the operation graph 205 are further divided by the work process, and the semi-operating portions 210 and 211 further divide the semi-operating portion by the operating elements. The method of dividing the operation graph 202 and the operation graph 205 is an example, and is not limited to this method.

The operation of the comparison screen 20 is as follows. The moving part, the semi-working part, and the non-working part of the working graph 202 are further divided by the working elements. For example, when the user selects the semi-moving portion 207 in the input unit 14, the element information DB 121 and the comment DB 124 are referred to, and the “moving image 101” associated with the moving element “welding” of the semi-moving portion 207 is displayed in the display window 201. And display "working hard" in the comment field 203. Further, when the user selects the semi-operating portion 208 in the input unit 14, the element information DB 121 and the comment DB 124 are referred to, and the "video 102" associated with the operating element "fixing the panel" of the semi-operating portion 207 is displayed in the display window. It is displayed in 201, and "working hard" is displayed in the comment column 203.

The moving part, the semi-working part, and the non-working part of the operation graph 202 are further divided for each work process. When the user selects the semi-operating portion 209 of the operation graph 202 in the input unit 14, a moving image of the semi-operating work process “welding” is displayed in the display window 201. The CPU 11 receives an input in the comment field 203 by the input unit 14. The user inputs "working hard" in the comment field 203 in the input unit 14. The CPU 11 displays "working hard" in the comment field 203.

When the user selects the operation graph 202 or 205 in the input unit 14, the detailed comparison screen 21 for comparing the details of the operation elements and the moving images of the work process is displayed. FIG. 38 is an explanatory diagram showing an example of the detailed comparison screen 21. The detailed comparison screen 21 includes a selection operation element 211 provided at the upper part, display windows 212 and 218 provided at the lower part of the selection operation element 211, and a standard time column 213 provided at the lower part of the display windows 212 and 218. The rating column 214 provided at the bottom of the standard time column 213, the comment columns 216 and 224 provided at the bottom of the rating rate column 214, and the operation provided at the left part of the standard time column 213 and the rating rate column 214. The graph 215 and the operation element table 217 provided at the lower part of the comment column 216 are provided. The semi-operating part of the operation graph 215 is composed of the semi-operating parts 227, 228 and 229.

The display windows 212 and 218, the comment columns 216 and 224, the operation graphs 215 and 222 are the same as the display windows 201 and 204, the comment columns 203 and 206, the operation graphs 202 and 205, and the operation element table 52. Omit. The selected action element 211 is a table that displays information about the selected action element. The standard time column 213 is a column for displaying the standard time of the operating element. The rating rate column 214 is a column for displaying the rating rate of the operating element.

The operation of the detailed comparison screen 21 is as follows. The CPU 11 refers to the element information DB 121 corresponding to the moving image ID and displays the operation element table at the bottom. When the user selects the semi-operating portion 227 in the input unit 14, the CPU 11 selects the operating element "welding" of the semi-operating portion 227 and displays the information about the operating element "welding" on the selected operating element 211. Further, the CPU 11 displays the standard time "3.85" of the operating element "welding" in the standard time column 213, displays the rating rate "1.10" of the operating element "welding" in the rating rate column 214, and displays the operating element "1.10". "Video 101" associated with "welding" is displayed in the display window 212, and "workers are working hard" is displayed in the comment column 216. Further, the reproduction time of the moving image data of the display window 212 can be changed by operating the rating rate column 214 as in the first embodiment.

For example, when the user selects the semi-operating portion 229 of the operation graph (ratio graph) 215 in the input unit 14, the generated moving image associated with the semi-operating in the work process “welding” is displayed in the display window 212. The CPU 11 receives an input in the comment field 224 by the input unit 14. The user inputs "skilled" in the comment field 224 in the input unit 14. The CPU 11 displays "skilled" in the comment field 224. As a result, the operation graph calculated from the work sampling method and the information of the operating elements can be displayed on the same screen. Further, on the same screen, it is possible to play the divided moving image corresponding to the type and the moving image corresponding to the work type from the operation graph. This makes it easy to compare the divided moving image of the analysis screen 5 with the generated moving image generated by the work sampling method. The lower operation element table 217 and the selection operation element 211 are not displayed even if the operation graph 222 is selected.

FIG. 39 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. Since the processes of steps S11 to S103 are the same as those of the information processing system according to the eighth embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S103, the CPU 11 accepts the input of the moving image ID (step S110). The CPU 11 displays the operation graphs 202 and 205 and 215 and 222 corresponding to the moving image ID on the same screen (step S111). The CPU 11 divides each part of the operation graphs 202 and 215 in association with each operation element (step S112). The CPU 11 divides each part of the operation graphs 205 and 222 in association with each work process (step S113). The CPU 11 accepts the selection of each divided portion in the operation graphs 202, 215, 205 and 222 at the input unit 14 (step S114). The selected work process or operation element may be displayed on the same screen.

The CPU 11 determines whether or not the received portion is associated with the work process (step S115). When the CPU 11 determines that the received portion is associated with the work process (step S115: YES), the CPU 11 shifts the process to step S116. The CPU 11 refers to the sampling tabulation table 127, displays the generated moving image corresponding to the work process on the same screen as the other moving images (step S116), and shifts the processing to S76. That is, the moving image corresponding to the work process is displayed on the same screen as the moving image of the moving image ID corresponding to the operation element. When the CPU 11 determines that the received portion is not associated with the work process (step S115: NO), that is, when it is determined that the received portion is associated with the operating element, the CPU 11 shifts the process to step S117. The CPU 11 refers to the element information DB 121 and displays the moving image corresponding to the operating element on the same screen as the other moving images (step S117). The CPU 11 displays the selection operation element 211 (step S118), and shifts the process to S76. That is, the moving image corresponding to the operation element is displayed on the same screen as the moving image of the moving image ID corresponding to the work process. In the present embodiment, the moving image calculated by the work sampling method described in Example 7 and the moving image described in Example 1 are compared, but the comparison between the moving image calculated by the work sampling method and the image after the work improvement is compared. You may go. Further, the CPU 11 may display a divided moving image that performs the same work as the work performed by the object displayed in the generated moving image on the same screen as the operation graphs 202 and 205 and 215 and 222. The improvement effect becomes clearer by comparing the video calculated by the work sampling method, which is a standard work video, with the video after work improvement. Further, the moving image calculated by the work sampling method may be compared with the moving image of a veteran worker. By comparing the video calculated by the work sampling method, which is a standard work video, with the video of a veteran worker, hints for improving work efficiency may be found. Also, when compared with the video of a beginner worker, the problems of the beginner worker become clear.

According to one aspect, it is possible to use it for calculating the rating rate by comparing the operation graphs calculated from the work sampling method and the information of the operating elements.

Embodiment 10
The tenth embodiment relates to an embodiment in which a selection figure for selecting a type is displayed. FIG. 40 is an explanatory diagram showing an example of the display window 51A of the analysis screen 5. The display window 51A further includes a selection figure 58. The selection figure 58 is composed of selection portions 58A, 58B and the like. The selected portion 58A or 58B or the like is represented by the selected portion 58 in some cases below. The selection figure 58 is a gesture map for supporting the input by the input unit 14. The selected figure 58 is, for example, a circle. The selected figure 58 may be any figure such as a rectangle or a triangle.

The operation of the selected figure 58 is as follows. When the user performs an arbitrary input operation on the display window 51 in the input unit 14, the user refers to the type information DB 123 and displays the type in the selection portion 581. Specifically, the CPU 11 displays type 1, type 2, type 3 and type 4 in each of the selected portions 581 (FIG. 40A). For example, when the user selects the selection portion 58A in the input unit 14, the CPU 11 corresponds to the type 1 in the "first step", "second step", "third step", "fourth step", and "fifth step". Is displayed on the selected figure 58 (FIG. 40B). When the user selects, for example, the selected portion 58B in the input unit 14, the other selected portion 581 is erased and only the selected portion 58B is displayed (FIG. 40C). The selection method of the selection portion 581 by the input unit 14 is, for example, moving the mouse cursor to the selection portion and releasing the left or right button. Alternatively, the method of passing through the selected portion with the mouse cursor may be used. The CPU 11 associates the operation element of the moving image data displayed in the display window 51A with the type "first process", and stores the type "first process" in the element information DB 121. The radius of the circle of the selected figure 58 in the present embodiment increases as the number of types increases. As a result, the lower the number of types, the greater the number of types. Therefore, when the number of types increases, it is possible to deal with it by increasing the radius of the circle.

FIG. 41 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. Since the processing of steps S11 to S22 is the same as the information processing system according to the first embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S19, the CPU 11 receives an input in the display window 51 by the input unit 14 (step S121). The CPU 11 refers to the type information DB 123 and displays the selected figure 58 by displaying the type in the selection portion 581 (step S122). The CPU 11 receives the selection in the selection unit 581 by the input unit 14 (step S123). The CPU 11 associates the operation element of the moving image data displayed in the display window 51A with the type displayed in the selected selection portion 581 (step 124), and shifts the process to S20.

According to one aspect, the type can be selected on the display window 51. As a result, it is possible to display a circle table for each work type and facilitate input of the image type.

Embodiment 11
The eleventh embodiment relates to an embodiment in which the distance between two points in the moving image of the display window 51 is calculated by image processing.

For example, the CPU 11 causes the display window 51A to display the moving image 101. The CPU 11 calculates the distance between two points on the display window 51A by performing image processing based on the angle and the resolution.

FIG. 42 is a flowchart showing a processing procedure of the information processing system according to the present embodiment. Since the processing of steps S11 to S22 is the same as the information processing system according to the first embodiment described above, the description thereof will be omitted for the sake of brevity. After completing the process of step S19, the CPU 11 calculates the distance by performing image processing on the display window 51 (step S131). The CPU 11 displays the calculated distance in the display window 51 (step S132), and shifts the process to S20.

According to one aspect, displaying the distance can be useful for understanding the moving image data.

The embodiments disclosed this time should be considered to be exemplary in all respects and not restrictive. The scope of the present invention is indicated by the scope of claims, not the above-mentioned meaning, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. In addition, each embodiment can be appropriately combined as long as the processing contents do not contradict each other.

1 Information processing device 12P program 52 Operation element table (work table)
202 Operation graph (1st graph)
205 Operation graph (second graph)
215 Operation graph (1st graph)
222 operation graph (second graph)

Claims (5)

In the program used for information processing equipment
In the information processing device
Display multiple images extracted from the video
The work type indicating the type of work performed by the object in the displayed image, the invalid time indicating the time during which the object performed an invalid operation for the work, and the time during which the machine operated for the work. The operation information indicating whether or not the object is operating, including the quasi-operation time shown, is received in association with the image.
Based on the received operation information, the work time related to the work and the operation time indicating the time when the object in each image has performed an effective operation for the work are calculated.
A graph showing the ratio of the operating time, the invalid time, and the semi-operating time to the calculated working time is generated and displayed .
A divided video obtained by dividing the video for each work is displayed.
Accepting multiple types for the split video,
Generate a work table that associates the work time showing the length of each divided video with multiple types received,
A program that executes a process of switching and displaying a plurality of work tables generated for each divided moving image . Read the template image of the moving object for each work type,
Calculate the degree of matching between the displayed image and the template image,
The program according to claim 1, wherein when the calculated degree of agreement exceeds the threshold value, the displayed image, the work type, and the operation information are stored in association with each other. The previous Symbol videos to display a plurality of divided movie is divided into each work,
Accepts the input of the evaluation value for the object displayed in each divided video,
Claim 1 or claim 1 in which the standard time indicating the standard of the work is calculated by adding the evaluation value to the operation time indicating the time when the object displayed in each of the divided moving images has performed an effective operation for the work. The program according to claim 2. An image display unit that displays multiple images extracted from one or more videos, and
The work type indicating the type of work performed by the object in the displayed image, the invalid time indicating the time during which the object performed an invalid operation for the work, and the time during which the machine operated for the work are shown. A reception unit that receives operation information indicating whether or not the object is operating, including the quasi-operating time shown, in association with the image.
Based on the received operation information, a calculation unit that calculates the work time related to the work and the operation time indicating the time when the object in each image has performed an effective operation for the work.
A ratio display unit that generates and displays a graph showing the ratio of the operating time, the invalid time, and the semi-operating time to the calculated working time .
A split video display unit that displays a split video that is divided for each work
A type reception unit that accepts multiple types for the split video,
A generation unit that generates a work table that associates the work time indicating the length of each divided video with multiple types received, and
An information processing device including a switching display unit that switches and displays a plurality of work tables generated for each of the divided moving images . In the information processing method in the information processing device
Display multiple images extracted from one or more videos,
The work type indicating the type of work performed by the object in the displayed image, the invalid time indicating the time during which the object performed an invalid operation for the work, and the time during which the machine operated for the work are shown. The operation information indicating whether or not the object is operating, including the quasi-operation time shown, is received in association with the image.
Based on the received operation information, the work time related to the work and the operation time indicating the time when the object in each image has performed an effective operation for the work are calculated.
A graph showing the ratio of the operating time, the invalid time, and the semi-operating time to the calculated working time is generated and displayed .
A divided video obtained by dividing the video for each work is displayed.
Accepting multiple types for the split video,
Generate a work table that associates the work time showing the length of each divided video with multiple types received,
An information processing method for switching and displaying a plurality of work tables generated for each divided moving image .

Images