JP2020008932A - Image processing device, image processing method, and program - Google Patents

Abstract

To improve collation accuracy in determining whether an operation is correctly conducted or not.SOLUTION: An image processing device comprises: generation means which generates detection label sequences on the basis of operation detection results in an attention area of a video obtained by capturing the operation; dividing means which divides the detection label sequences into a plurality of partial detection label sequences on the basis of information on an operating environment; selection means which selects a method for collating each of the plurality of detection label sequences with a reference label sequence; and collation means which collates the plurality of partial detection label sequences and the reference label sequence by the collation method selected by the selection means.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an image processing device, an image processing method, and a program.

  Conventionally, as a method of analyzing whether the assembly work of a factory is performed correctly, a method of analyzing a movement of a worker in a video taken of a scene of the assembly work and comparing it with the contents of a work procedure manual is known. I have. For example, there is a method in which a rectangle is set in a place where an assembling operation occurs in a video, and whether the assembling operation is performed is determined based on whether or not the rectangle is manipulated (see Patent Document 1).

JP 2001-209694 A

  The order of work performed in the assembly work may be described in the work procedure manual, and it may be necessary to consider the order of work in order to judge the quality of the assembly work performed by the worker . The analysis of the assembly work taking the order of such work into consideration cannot be realized merely by paying attention to the presence or absence of the individual work as in Patent Document 1, and it is necessary to analyze the order of the occurrence. is there. As such a method, there is known a sequence matching represented by DP matching, which allows ambiguity in the arrangement of the sequence. However, there are some types of work such as fine adjustment of parts that require high accuracy, and the work content and work time vary greatly from one individual to another regardless of the skill level. When sequence matching is performed on a sequence including such an operation, the overall matching accuracy may be reduced.

  The image processing apparatus according to the present invention includes: a generation unit configured to generate a detection label sequence based on a result of motion detection of a region of interest of a video image of a work; and a plurality of detection partial label sequences based on work environment information. Dividing means, selecting means for selecting a matching method with a reference label string for each of the plurality of detected partial label strings, and each of the plurality of detected partial label strings with the matching method selected by the selecting means. Collating means for collating with the reference label sequence.

  ADVANTAGE OF THE INVENTION According to this invention, the collation precision which determines whether the operation | work was performed correctly can be improved.

FIG. 2 is a diagram illustrating an example of a hardware configuration of a work evaluation control device. It is a figure showing an example of functional composition of a work evaluation control device. It is a figure which shows the example of an assembly operation, etc. It is a figure showing an example of a work procedure manual. It is a figure showing an example of a table. 4 is a flowchart illustrating an example of information processing according to the first embodiment. FIG. 9 is a diagram illustrating an example of sequence matching. FIG. 9 is a diagram illustrating an example of application of DP matching. 13 is a flowchart illustrating an example of information processing according to a second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First embodiment>
FIG. 1 is a diagram illustrating an example of a hardware configuration of the work evaluation control device 100. The work evaluation control device 100 is an example of an image processing device.
The control unit 101 is configured by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and the like, and performs an arithmetic operation and a logical determination for information processing. The control unit 101 controls each component connected to the system bus 106 via the system bus 106.
The display unit 102 includes a controller that controls display of image information and a liquid crystal panel, or a projector, or an output device such as a warning.
The communication unit 104 is a network controller or the like, and is an external communication unit that controls connection with another device. More specifically, the communication unit 104 controls connection with other devices by a notification method such as LAN, 3G, 4G, Bluetooth (registered trademark), and RFID (Radio Frequency Identification). However, the communication unit 104 may employ another communication method that can achieve the same purpose. The work evaluation control device 100 receives, via the communication unit 104, a video from the video acquisition device 309 that captures a production line and a person (including a person's behavior).

The measurement unit 105 is a GPS (Global Positioning System) sensor, a gyro sensor, a human body motion sensor, an electronic compass, or the like. The measurement unit 105 measures the position of the person, the posture of the person (direction, rotation / angle / angular velocity of each joint), the operation status of tools on the production line, noise, and the like.
A RAM (Random Access Memory) 107 is used for temporarily storing various data from each component. For example, the RAM 107 temporarily stores various measurement data, image data, and the like.
By using a physical medium such as an external storage unit 108, a flash memory, an HDD, or an optical disk, in addition to the program executed in the present embodiment, measurement data and image data such as various data described in FIG. 4 are stored.
The work evaluation control device 100 composed of the above-described components responds to various images supplied from the image recording unit 103, various inputs via the network supplied from the communication unit 104, or measurement values input from the measurement unit 105. Work. That is, when an input from any of the video recording unit 103, the communication unit 104, and the measurement unit 105 is supplied, an interrupt signal is sent to the control unit 101. Then, the control unit 101 reads out various programs stored in the external storage unit 108 and executes various controls according to the programs. The control unit 101 executes the processing based on the program, thereby realizing the functional configuration of the work evaluation control device 100 in FIG. 2 described later and the processing in the flowcharts in FIGS. 6 and 9 described later.

FIG. 2 is a diagram illustrating an example of a functional configuration of the work evaluation control device 100.
The attention area control unit 201 manages, from each frame of the moving image obtained by the video recording unit 103, an attention area where the movement of the subject, the presence / absence change, and the like may occur, using a coordinate group and a label corresponding to the attention area. The object detection unit 202 actually detects a movement or a presence / absence change in each attention area. The detection label sequence control unit 203 performs a process of storing the detection result as a label sequence.

  First, an example of a factory assembling operation, the purpose, a technical problem, and a solution of the operation evaluation control device 100 will be outlined. FIG. 3A shows a workplace where a worker 308 performs an assembly work on a workbench 301. The workbench 301 includes a workbench 302, an object to be assembled 303, a seal case 304, a screw case 305, a trash can 306, a driver receiver 307, and an image acquisition device 309. In this work place, an assembly target such as a developing unit and a fixing unit to be incorporated in the copying machine is placed on a work bench to assemble the assembly target. An operator performs an assembly operation according to an assembly operation procedure manual. Therefore, the purpose of the work evaluation control device 100 is to determine whether the worker has performed the assembly work according to the assembly work procedure manual.

FIG. 3B illustrates the entire components when actually performing work detection on the workbench 301. In the example of the present embodiment, the work evaluation control device 100 sets one or more regions of interest in advance at places where operations related to work may occur in each frame of the video acquisition result obtained from the video acquisition device 309. For example, when a seal is attached, the object to be assembled 303 and the seal case 304 are used. When a screwdriver is used, the screw case 305 and the object to be assembled 303 are used. Can be expected. The work evaluation control device 100 gives an attention area to such a place. This attention area is specified by an arbitrary rectangular range, and the work evaluation control device 100 assigns a mutually distinguishable label such as a number to each. In a series of recognition processes described later, the work evaluation control device 100 performs detection of passage of a hand and detection of the presence or absence of an object in the attention area. In the following description, each movement required for assembly is referred to as an “operation”, and each “work” is composed of a plurality of “operations”. In FIG. 3B, a dotted rectangle is a region of interest. The underlined number is the number of the attention area.
FIG. 3C shows a specific example of the assembly object 303 created in the workbench 302. It is an example of a product assuming that a thin sheet-shaped object passes. A metal fitting 311 is attached to the main body 310 with a screw 312. The clearance play measuring device 313 is for confirming whether the clearance between the sheets is appropriate.

4A and 4B are examples of a work procedure manual. The work procedure manual contains a detailed explanation of the work procedure. The assembling work in the example of the present embodiment mainly includes fitting of a metal fitting (work 1), screw tightening (work 2), fine adjustment of metal fitting position (work 3), sticking of a seal (work 4), and the like.
Work 1: Mounting of metal fittings (metal fitting 311)
Operation 1: Mounting the metal fitting 311 on the main body 310 Work 2: Screw fastening (screw 312)
Operation 1: driver acquisition, operation 2: screw acquisition, operation 3: screw tightening, operation 4: driver return work 3: fine adjustment of metal fitting position (metal fitting 311)
Operation 1: Gap measurement with the clearance play measuring device 313, fine adjustment of the metal fitting 311 Work: sticker operation 1: Seal acquisition, operation 2: sticker right end sticker, operation 3: sticker left end sticker, operation 4: discard seal backing paper ( (Omitted after work 5)
Further, a specific assembling work of each work will be described. In the work 2, a driver is taken with a right hand, a screw is taken out of a screw box with a left hand, and a screw is screwed into a screw hole of a black circle at the center of an object to be assembled. It is the flow of returning. In operation 3, the play is confirmed using the clearance play measuring tool 313, the screw 312 is appropriately loosened, the position of the metal fitting 311 is adjusted, and then the screw 312 is tightened again. In the operation 4, a seal is taken, the sticker is attached to the left half of the object to be assembled, and the back paper is discarded in a trash can. These are described in the work procedure manual, and the worker must proceed with the assembly work in this order.

  The region definition table 501 in FIG. 5 stores the definition of the region of interest shown in FIG. The detection label string storage table 502 stores the history of the work recognized by the apparatus of the present embodiment in association with the attention area. The table illustrated in FIG. 5 is stored in the external storage unit 108.

FIG. 6 is a flowchart illustrating an example of information processing of the work evaluation control device 100 according to the first embodiment. The entire flow chart has a structure in which a loop is performed for each frame acquired by the video acquisition device 309. The work evaluation control device 100 detects a change in the image in the above-described region of interest in each frame of the captured image, and if there is an expected correspondence between the flow of the assembling work and the change in the region of interest, Judge that the assembly work was performed correctly. What accumulates the labels detected from the beginning to the end of the assembling work is called a detection label sequence. A column in which labels are arranged based on the order of operations described in the work procedure manual is called a reference label column. The work evaluation control device 100 verifies whether the detected label sequence is more appropriate than the reference label sequence. The work evaluation control device 100 determines that the assembling work has been correctly performed if the arrangement is appropriate, and otherwise determines that the assembling work has failed.
The assembly operation is photographed by the video acquisition device 309. The image acquisition device 309 captures an image from a camera installed at an upper part of a work place or the like. In step S601, the control unit 101 receives a video frame from the video acquisition device 309.
In step S602, the control unit 101 obtains an image of a region of interest in a video frame. Subsequently, a loop is executed from S603 to S606 for each attention area acquired here.

In step S603, the control unit 101 detects a motion in the attention area. The motion detection is performed using an inter-frame difference in image luminance, an inter-frame difference in a feature amount such as SIFT, a comparison with a reference luminance value, the passage of an object within a camera view angle, a change in object shape, and the like.
In step S604, the control unit 101 determines whether motion detection has been performed. The control unit 101 shifts to S605 if a motion is detected, and shifts to S606 if no motion is detected.
In step S605, the control unit 101 reflects the detection result on the detection label sequence. More specifically, the control unit 101 treats the label assigned to the detected area of interest as a detection label, and generates and accumulates the label in chronological order in association with the occurrence time. FIGS. 7A and 7B show an example of a detection label string actually detected. The operator strikes the metal fitting 311 on the main body 310 with the screw 312 in the screw fastening operation. Subsequently, the gap is measured using the gap play measuring device 313 in order to check whether the gap between the sheets is appropriate. At this time, if the gap is not formed according to the standard, the gap is finely adjusted by loosening the screw 312 or pressing a metal fitting. Since this fine adjustment work occurs irregularly for each individual to be produced, as shown in the example shown in FIGS. 7A and 7B, the length of the label row, the head of the label, The occurrence time difference from the end and the type of label included tend to vary. In such a situation, the control unit 101 determines that the collation accuracy between the reference label string and the detection label string is reduced. And it is effective to adaptively change the matching method. This is because, in the case where the whole is detected by the collation method using one type, irregularities in the label length and the number of label types due to the fine adjustment work hinder the desirable association with the labels of other works.
Therefore, in step S606, the control unit 101 updates the object presence / absence state in each attention area in order to efficiently acquire a change in the work environment. The presence / absence determination target includes a worker's hand, a worker's arm, parts used for the work, equipment used for the work, and the like.
In step S607, the control unit 101 divides the detection label string and divides the detection label string into one or more detection partial label strings. Then, the control unit 101 loops S608 to S609 for each partial detection label string.

In step S608, the control unit 101 selects a partial sequence matching unit based on a condition determined in advance.
Here, the details of the partial sequence matching means will be described. In the example of this embodiment, sequence matching is basically used. Sequence matching means that even if two time-series data do not exactly match, two time-series data are regarded as the same if the generation order of labels constituting the time-series data is similar, which is ambiguous. Is a matching method that allows for The reason for introducing sequence matching is that the hand may enter the attention area in addition to the operation, such as during the transition from one operation to the next operation, so that even if the assembly work is performed correctly, the detection label string and the reference label string are accurate Because it does not match. In the present embodiment, sequence matching called DP matching is used. DP matching is a method of measuring the similarity between two time-series data (reference and observation), and links the label of the observation time-series data to the label of the reference time-series data so as to minimize the cost of matching. To go. In the process, DP matching can be used as sequence matching because it performs processing to minimize the influence of noise. FIG. 8A shows a collation method using more specific cases. 1, 2, 3, 4 are used as reference time series data, and 1, 2, 4, 3, 4 are used as observation time series data. The first four of the two fours in the observation time series data are noise. In FIG. 8 (A), assuming that labels are linked from the lower left to the upper right, each cell has a minimum value from the lower left until the label of each time series of reference and observation is shared from the lower left. Write down the cost. In calculating the lowest cost of each cell, the lowest cost is calculated from the three cells, that is, the lower left, the lower left, and the left, and the lower left to the upper right are linked by a simple operation of repeating this. Derive the lowest cost of In addition, since which cell of the three cells is adopted is recorded, in the final derivation of the lowest cost on the upper right, it is possible to derive what route has been taken. Although the detailed calculation is omitted, the result of the DP matching shown in FIG. 8A is as shown in FIG. 8B, and the path following the dark cell is the path that realizes the lowest cost. FIG. 8C specifically shows a result of the linking, in which the noise “4” is linked not as “4” of the reference time-series data but as noise accompanying “2”.

On the other hand, in an operation such as a play adjustment operation in which there is a large variation between individuals, there is a problem that the operation matching accuracy is reduced in DP matching assuming that the reference label string is fixed as shown in FIG. Therefore, in the present embodiment, in order to maintain the detection accuracy even when the type of work changes, the control unit 101 adaptively changes the detection method using the motion detection and the object presence / absence state of each attention area collected in S603 and S606. select. FIG. 7D shows a specific example. The difference from FIG. 7C is that the control unit 101 is modified to measure the number of label types and their time occupancy in the fine adjustment work. Further, as shown in FIG. 7 (E), the control unit 101 uses, as a trigger for the switching, the attention area (ID = 7) of the predetermined place of the specific object such as the gap play measuring tool 313 which is a tool used. The presence / absence of the gap play measuring instrument 313 is detected. The presence or absence of the specific object can be used as a change in the work environment related to the work in progress. Therefore, the control unit 101 divides the label sequence into partial sequences based on this information, and selects a partial sequence matching unit that is not limited to DP matching for each partial sequence. More specifically, the control unit 101 switches the detection method according to the characteristics of the work target (FIG. 7F). For example, in addition to the example described in the present embodiment, the control unit 101 may determine that the time required for the work mainly pressing down, the average luminance value in the attention area for the painting work, etc. Is obvious, an appropriate one can be selected, such as excluding it from the collation target. By doing so, it is possible to perform redundant matching for fine adjustment work in which the breakdown of the work content is more important than the work order. In addition, redundant work collation can be performed even when the work order is changed, such as performing fine adjustment work again after the sticking of the seal. Here, the control unit 101 divides the detection label string into a plurality of detection partial label strings based on the position of the specific object, the direction of the specific object, and the type of the measured specific object, not only the presence or absence of the specific object. You may. As a matching method (partial sequence matching means) other than DP matching, matching based on the length of the division time, the length of the detection label string, the number of types of the detection label string, the ratio of each label, the average luminance of the video, and the like. There is a way.
In step S609, the control unit 101 performs collation of the partial sequences using the selected partial sequence collation unit.

In step S610, the control unit 101 integrates the partial sequence collation results and determines whether the work has been correctly performed in the entire work procedure manual. The control unit 101 may intentionally select a plurality of detection methods for each partial sequence and run them in parallel, and then integrate the results. For example, the control unit 101 performs a majority decision on the results of a plurality of methods when detecting the presence or absence of a work omission as an integration method. This makes it possible to realize redundant detection even when the result of detection by each method becomes unstable, such as in a situation where data collection is not sufficient.
In S611, the control unit 101 outputs a detection result.
In addition, the conditions for selecting the partial sequence matching unit in S608 described in the example of the present embodiment are determined in advance before the implementation, and the supervisor gives feedback in the process of accumulating the work history. For example, it may be determined dynamically. By doing so, it is possible to apply even when it is difficult to determine which sequence matching means is appropriate and there are few precedents. The division criterion of the detection label string division processing in S607 is not limited to the presence or absence of one attention area. For example, by using a combination of presence or absence in a plurality of regions of interest, more complex and redundant division conditions can be expected.

  As described above, in the example of the present embodiment, the detection method is changed after dividing the detection label sequence into a plurality of detection partial label sequences based on the change in the work environment related to the operation configuring the work. This has the effect of preventing the effects of work in which the work content and work time vary from affecting the entire work analysis target, and as a result, the collation accuracy for judging whether or not the work has been correctly performed is improved, The supervisor of the assembling work will be able to give instructions such as re-assembly work accurately.

<Embodiment 2>
Next, as a second embodiment, an example will be described in which a sound acquisition result such as a device operation sound read from a sensor or the like arranged in a line in operation, a vibration state measurement result, and an operation state change measurement result are used. According to this example, more various work collations can be performed.
This embodiment has the same configuration as the first embodiment. FIG. 9 is a flowchart illustrating an example of information processing of the work evaluation control device 100 according to the second embodiment. In the present embodiment, in order to further increase the matching accuracy, the processing of S907 to S908 is added before the detection label string division processing. The details of the detection label string dividing process and the difference between the subsequent processes caused by the addition will be described below.

In step S907, the control unit 101 collects an audio signal such as an operation sound of the operation device, a vibration state, and an operation state from sensors and the like arranged in the line. More specifically, the sensors include a microphone attached to the image acquisition device 309, a power ON / OFF sensor for a drill, a power ON / OFF sensor for a vacuum cleaner, a vibration sensor attached to the worktable 301, and the like. This is because in production work, tools that emit loud operating sounds or vibrations may be used, and therefore, it is expected that the collation means can be efficiently switched based on information from various sensors.
In step S908, the control unit 101 analyzes the collected sensor measurement values. By extracting the amount of noise and the intensity of a specific frequency from the voice as a feature value and determining the reference value (output 50%, etc.), the operating state (ON / OFF, etc.) of the drill or the vacuum cleaner can be obtained finely. it can. In addition, the control unit 101 can estimate the work content at the time of utterance by voice-recognizing the content of the conversation of the worker.

  In step S910, the control unit 101 selects a partial sequence matching unit based on the information. For example, a description will be given of an example of a work 7, a screw part dust removing work. This is an operation for sucking out fine dust generated when the screw is repeatedly attached and detached when the metal fitting 311 is mounted or finely adjusted, in order to prevent defective products from occurring. With detection using only the attention area, it is possible to track that the tip of the vacuum cleaner has passed nearby, but it is difficult to determine whether or not the cleaner is actually sucking. In addition, as in the fine adjustment operation, the extraction timing and the number of times are irregularly generated. Therefore, the control unit 101 switches the collation method according to the movement of the power ON / OFF sensor of the vacuum cleaner or the presence or absence of the detection of the operating sound of the vacuum cleaner recorded from the microphone. More specifically, the control unit 101 estimates that the corresponding detection label string is to be extracted during the extraction, and then changes to a collation method of measuring the number of label types and their time occupancy. I do. By doing so, it is difficult to determine the validity by detecting the motion of only the attention area, and it is possible to accurately detect the work in which there is a variation for each individual. Note that if the relationship between the specific adjustment work and the specific work is clear in advance, such as when there is a strong association between the fine adjustment work and the dust removal work, the control unit 101 determines the number of label types detected in each work. The validity of the work may be determined based on the degree of coincidence with the time occupancy. By doing so, it is possible to efficiently extract an operation that is considered to have a high quality risk, such as when an individual is created that performs a small amount of dust removal operation despite a large number of fine adjustment operations. As a matching method (partial sequence matching means) other than the DP matching of the present embodiment, there is a matching method based on the signal amount of voice, the presence or absence of a specific frequency, and the like.

  In the foregoing, an example has been described in which more various work collations can be performed by using a voice signal and a vibration value read from a sensor or the like arranged in a line. As a result, it is difficult to determine the validity by detecting the motion of only the attention area, and the effect of accurately detecting the work that varies from one individual to another can be obtained.

(Modification)
In the first and second embodiments, examples of the cell production operation performed based on the work table are shown, but the present invention is not limited to these use cases. The present invention is widely applicable to use cases in which the operation history can be tracked based on video. For example, the present invention can be applied to packing and packing work in a distribution warehouse. In this case, the control unit 101 can increase the matching accuracy of the entire work by separately cutting out the work that varies for each individual depending on the remaining amount of the gum tape or the like from the other work and performing the sequence matching.
Further, in the second embodiment, an example is shown in which the detection label sequence is divided based on the results obtained from the sensors and the like arranged in the line in operation, but the control unit 101 is not limited to this. A label sequence may be generated based on the values. This makes it possible to obtain a label sequence with higher accuracy than a label sequence relying only on video. For example, when the tip of the hand passes through a certain area of interest, the control unit 101 treats whether the operation state of the vacuum cleaner is ON or OFF as another label.
In the above-described embodiment, a method using DP matching and a degree of coincidence between the number of label types and the time occupancy are disclosed as a method of sequence matching, but the present invention is not limited thereto. For example, it is also effective when a method using a model such as an automaton or an HMM is used. The pre-design and application of the model can be performed in exactly the same way as in DP matching. These models are effective, for example, when it is known in advance that the movement of the hand draws a figure of eight in a series of operations, or when the transition of the label sequence is characteristic.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium. The present invention can also be realized by processing in which one or more processors in a computer of the system or the apparatus read and execute a program. Further, it can be realized by a circuit (for example, an ASIC) that realizes one or more functions.

As described above, an example of the embodiment of the present invention has been described in detail, but the present invention is not limited to the specific embodiment.
For example, the hardware configuration of the work evaluation control device 100 does not need to include the control unit 101, the RAM 107, the external storage unit 108, and the like. A plurality of control units may execute processing based on a program while using data stored in a plurality of RAMs, an external storage unit, and the like.

  As described above, according to each of the above-described embodiments, the detection label sequence is divided into a plurality of detection partial label sequences based on a change in the work environment related to the operation configuring the work, and then the detection method is changed. Prevent the effects of work with exceptions and time variations from affecting the entire analysis target. As a result, the collation accuracy for judging whether or not the work has been correctly performed is improved, and the supervisor of the assembling work can accurately give an instruction to start the assembling work again.

100 work evaluation control device 101 control unit

Claims (13)

Generating means for generating a detection label sequence based on a result of the motion detection of the attention area of the video of the work,
Division means for dividing the detection label string into a plurality of detection partial label strings based on information on a work environment,
Selection means for selecting a matching method with a reference label string for each of the plurality of detection partial label strings,
Collating means for collating each of the plurality of detected partial label strings with the reference label string by a collation method selected by the selecting means,
An image processing apparatus having:   The image processing apparatus according to claim 1, further comprising an output unit that outputs a result of the comparison by the comparison unit.   The image processing apparatus according to claim 1, wherein the division unit divides the detection label string into the plurality of detection partial label strings based on the presence or absence of a specific object in the attention area as information on the work environment.   The image processing apparatus according to claim 1, wherein the division unit divides the detection label string into the plurality of detection partial label strings based on a position of a specific object in the attention area as information on the work environment.   The image processing apparatus according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on a direction of a specific object in the attention area as information on the work environment.   3. The image processing apparatus according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on a type of a tool used by an operator in the attention area as information on the work environment. 4.   3. The image processing apparatus according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on a noise amount included in a sound during work as information on the work environment. 4.   3. The image processing apparatus according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on the content of a conversation included in a working voice as information on the work environment. 4.   3. The image processing apparatus according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on an operation sound of a working device in operation as the information on the work environment. 4.   3. The image processing apparatus according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on a state of vibration generated in association with a work operation as information on the work environment. 4. .   3. The image according to claim 1, wherein the division unit divides the detection label sequence into the plurality of detection partial label sequences based on an operation state of an operation device generated in association with a work operation as information on the work environment. 4. Processing equipment. An image processing method executed by the image processing device,
A generation step of generating a detection label sequence based on the result of the motion detection of the attention area of the video of the work,
A dividing step of dividing the detection label string into a plurality of detection partial label strings based on information on a work environment;
A selecting step of selecting a matching method with a reference label string for each of the plurality of detected partial label strings,
A matching step of matching the reference label string with each of the plurality of detected partial label strings by the matching method selected by the selecting step,
An image processing method including:   A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.

Images