TWI823327B - Monitoring area setting device, monitoring area setting method and monitoring area setting program - Google Patents

Abstract

The setting method includes the following steps: acquiring a setting moving image from a camera that reflects the work site and an operator working at the work site; and detecting each frame of the setting moving image according to the same command as the analysis program. The position of the operator; providing the first screen for playing the dynamic image for setting; and setting the monitoring area in response to the operation on the first screen. The first screen includes a mark indicating the detected position of the operator. The mark is displayed superimposed on the dynamic image for setting. Thereby, the setting method can support appropriate and simple setting of the monitoring area corresponding to the steps.

Description

Monitoring area setting device, monitoring area setting method and and monitoring area setting program

This disclosure relates to a setting device, a setting method and a setting program.

Conventionally, attempts have been made to improve work procedures included in the work site using moving images captured by cameras installed at work sites such as factories. For example, Japanese Patent Laid-Open No. 2020-204819 (Patent Document 1) discloses an information processing device that performs image analysis on moving images obtained by shooting with a ceiling camera. The information processing device performs image analysis on the dynamic image to determine whether there is an operator present in the monitoring area associated with each step, and generates data indicating the period during which it is determined that the operator is present in the monitoring area. Based on the above information, it is easy to grasp the operation period in each step.

[Prior art documents]

[Patent Document]

Patent Document 1: Japanese Patent Application Publication No. 2020-204819

In the technology disclosed in Patent Document 1, monitoring areas are preset for each step. area. If the monitoring area set for a certain step deviates from the position of the operator working in the step, the accuracy of the analysis results will decrease. Therefore, it is ideal to appropriately and easily set the monitoring area corresponding to the procedure.

This disclosure was made in view of the above problems, and its purpose is to provide a setting device, a setting method, and a setting program that support appropriate and simple setting of the monitoring area corresponding to the steps.

According to an example of the present disclosure, the setting device sets a monitoring area for the analysis program. The analysis program includes: a first command to detect an operator's position in each frame of a dynamic image for analysis obtained from a camera that shoots a job site, the job site including a plurality of steps; and a second command based on The multiple steps respectively correspond to the monitoring area and the detected position of the operator to determine whether there is an operator in the monitoring area. The setting device includes an acquisition part, a detection part, a provision part and a setting part. The acquisition unit acquires a setting moving image from the camera that maps the work site and the operator working at the work site. The detection unit detects the position of the operator for each frame of the setting dynamic image based on the same command as the first command. The providing unit provides the first screen of the dynamic image for playback setting. The setting unit sets the monitoring area in response to the operation on the first screen. The first screen includes a mark indicating the position detected by the detection unit. The mark is displayed superimposed on the dynamic image for setting.

According to this disclosure, the user can set the monitoring area of each step while observing the operator reflected in the setting dynamic image played on the first screen. Indicates the operator detected based on the same command as the first command of the analysis program. The position mark is displayed superimposed on the setting dynamic image. Therefore, the user only needs to adjust the size and position of the monitoring area to include the mark. Thereby, the user can easily set a monitoring area for each step that is suitable for the position of the operator who performs the operation of the step. In this way, the setting device can support appropriate and simple setting of the monitoring area corresponding to the steps.

In the above disclosure, the providing unit further provides a second screen showing the analysis results obtained by using the setting dynamic image and the analysis program of the monitoring area set by the setting unit.

According to this disclosure, the user can verify whether the monitoring area has been appropriately set by confirming the second screen.

In the disclosure described above, the second screen includes a diagram showing, for each of the plurality of steps, a time period during which an operator exists in the corresponding monitoring area. Alternatively, the second screen includes a box chart showing a deviation in the time when the operator is present in the corresponding monitoring area for each of the plurality of steps.

According to this disclosure, the user can easily determine whether the monitoring area is appropriate by confirming a graph or a box plot indicating the time period when an operator is present in the monitoring area.

In the disclosure, the second command includes: a command to extract a plurality of consecutive frames including the object frame from the dynamic image for analysis; and to calculate the number of frames contained in the object monitoring area among the plurality of frames by the first command. Execution of a command to detect a proportion of a frame of an operator's position, the object monitoring area corresponding to an object step among the plurality of steps; and corresponding to a situation where the proportion exceeds a threshold value, it is determined to be in the object The operator is required to be present in the object monitoring area at the time of shooting of the frame. Provide departmental advancement A third screen for setting threshold values is provided. The setting unit further sets the threshold value in response to the operation on the third screen.

According to this revelation, the analysis result of the analysis program changes corresponding to the threshold value. The user can set an appropriate threshold value while confirming the second screen.

In the disclosure, the first command includes: a command to detect a rectangular area where the operator is reflected in each frame of the dynamic image for analysis; and a command to determine the center point of the rectangular area as the position of the operator. The detection unit detects a rectangular area for each frame of the setting dynamic image based on the same command as the first command. The first screen further includes a first frame line representing the rectangular area detected by the detection unit. The first frame line is overlapped and displayed on the setting dynamic image.

According to this disclosure, the user can easily grasp the area where the operator is reflected in the setting dynamic image.

In the above disclosure, the dynamic image for setting is displayed in the first screen according to the method selected from play, pause, frame-by-frame forward, frame-by-frame rewind, and time specification.

According to this disclosure, the user can set the monitoring area while confirming the setting dynamic image displayed in a desired manner.

In the disclosure, the first picture includes more than one second frame lines respectively representing more than one monitoring area. The setting unit determines a monitoring area represented by a designated second frame line among the one or more second frame lines as a monitoring area to be edited, and determines the monitoring area as an editing target in response to an operation on the designated second frame line. The monitoring area of the object can be edited.

According to this disclosure, the user can easily move throughout the monitoring area by adjusting the size and position of the second frame line.

According to an example of the present disclosure, the setting method sets a monitoring area for the analysis program. The analysis program includes: a first command to detect an operator's position in each frame of a dynamic image for analysis obtained from a camera that shoots a job site, the job site including a plurality of steps; and a second command based on The multiple steps respectively correspond to the monitoring area and the detected position of the operator to determine whether there is an operator in the monitoring area. The setting method includes the first step to the fourth step. The first step is a step of acquiring a setting moving image from a camera, showing a work site and an operator working at the work site. The second step is a step of setting each frame of the moving image to detect the position of the operator based on the same command as the first command. The third step is a step of providing a first screen for playing back the setting dynamic image. The fourth step is a step of setting the monitoring area corresponding to the operation on the first screen. The first screen includes a mark indicating the detected position of the operator. The mark is displayed superimposed on the dynamic image for setting.

According to an example of the present disclosure, the setting program causes the computer to execute the setting method.

Through these disclosures, appropriate and simple settings of the monitoring area corresponding to the steps can also be supported.

According to the present disclosure, appropriate and simple setting of the monitoring area corresponding to the steps can be supported.

1: System

2:Working site

3. 3_1~3_n, S1~S4: steps

5. 5a~5c: Operator

10:Memory Department

11: Acquisition Department

12, 16: Inspection Department

13:Providing Department

14: Setting Department

15:Analysis Department

17: Judgment Department

18:Generation Department

20:Static image

30: Setting screen

31, 31a~31c, 71a, 71b: mark

32, 32a, 32b, 41, 41a~41c: frame line

33:Dynamic image playback area

34: List display column

35:Delete button

36:Add button

37: Analysis processing start button

38:Save button

39:Dynamic image play button group

40: Progress bar

50, 60: Screen

51:Gantt chart

52: Box plot

61:Input field

72a~72c: Monitoring area

100: Set device

101: Processor

102:Memory

103:Storage

104:Camera interface

105:Display controller

106:Input interface

107: Communication interface

110: Setting program

120: Motion image for setting

130: Dynamic images for analysis

200:Camera

300:Display device

350:Input device

400:Analysis program

401, 402: Command

410: Setting information

420:Time period information

FIG. 1 is a diagram showing an example of a system to which the setting device according to the embodiment is applied.

FIG. 2 is a diagram showing an example of a still image acquired for setting a monitoring area in the system of the reference form.

FIG. 3 is a diagram showing an example of the relationship between a monitoring area set according to a conventional setting method and an operator's working position.

FIG. 4 is a schematic diagram showing an example of the hardware structure of the setting device according to the embodiment.

FIG. 5 is a diagram showing an example of the functional structure of the setting device according to the embodiment.

FIG. 6 is a diagram showing an example of a setting screen.

FIG. 7 is a diagram showing an example of a screen provided in response to an operation of clicking an analysis start button.

FIG. 8 is a diagram showing an example of a screen for setting a threshold value.

Embodiments of the present invention will be described in detail with reference to the drawings. In addition, the same or equivalent parts in the figures are marked with the same symbols and their descriptions will not be repeated. Each of the modifications described below may be appropriately and selectively combined.

§1 Application examples

An outline of a system to which the setting device of this embodiment is applied will be described. FIG. 1 is a diagram showing an example of a system to which the setting device according to the embodiment is applied. As shown in FIG. 1 , the system 1 includes a setting device 100 , a camera 200 , a display device 300 , and an input device. 350 and analysis program 400. The system 1 is installed at a work site 2 which is a production site such as a factory.

The job site 2 includes multiple steps 3 (step 3_1 ~ step 3_n). One or more workers 5 perform work in each step 3 while moving between a plurality of steps 3 at the work site 2 .

The camera 200 is installed at a position overlooking the work site 2 (for example, on the ceiling) to photograph the work site 2 . The camera 200 is, for example, a super wide-angle camera or a wide-angle camera. The camera 200 outputs moving image data obtained by shooting (hereinafter referred to as "moving image").

The analysis program 400 is a program for performing image analysis on the moving image for analysis acquired from the camera 200 . The analysis program 400 can also be installed on a computer device (not shown) and executed by the processor of the computer device. Alternatively, the analysis program 400 can also be installed in the setting device 100 and executed by the processor of the setting device 100 .

The analysis program 400 includes: a command 401 to detect the position of the operator for each frame of the dynamic image for analysis; and a command 402 based on the monitoring areas corresponding to the plurality of steps 3 and the detected positions of the operators, To determine whether there are operators in the monitoring area. The monitoring area is an area on the dynamic image, such as a rectangle. However, the monitoring area is not limited to a rectangular shape and can also be in other shapes (such as circles, ellipses, etc.). Hereinafter, the case where the monitoring area is a rectangle will be explained.

The setting device 100 is, for example, a general-purpose computer, and is connected to the display device 300 and the input device 350 . The display device 300 is, for example, a liquid crystal display. Input device 350 For example, it includes keyboard, mouse, touch panel, etc.

The setting device 100 supports the setting of monitoring areas required for execution of the analysis program 400 . Specifically, the setting device 100 executes steps S1 to S4 shown in FIG. 1 . Steps S1 to S4 are executed by the processor of the setting device 100 .

In step S1 , the setting device 100 acquires a setting moving image showing the work site 2 and the operator 5 working at the work site 2 from the camera 200 .

In step S2, the setting device 100 detects the position of the operator 5 for each frame of the setting dynamic image based on the same command as the command 401 of the analysis program 400.

In step S3, the setting device 100 provides the setting screen 30 for playing back the setting moving image. Specifically, the setting device 100 causes the display device 300 to display the setting screen 30 .

In step S4, the setting device 100 sets the monitoring area in response to the operation on the setting screen 30. Specifically, on the setting screen 30 , a rectangular frame line 32 corresponding to the monitoring area is displayed in response to the input to the input device 350 . The user operates the input device 350 to adjust the position and size of the frame line 32 on the setting screen 30 . The setting device 100 determines the area surrounded by the frame line 32 as the monitoring area, and generates setting information indicating the coordinates of the four vertices of the monitoring area. The analysis program 400 determines the monitoring area based on the generated setting information, and uses the determined monitoring area to perform image analysis.

The setting screen 30 includes a display indicating the operator 5 detected in step S2 The location of the marker 31. The mark 31 is displayed superimposed on the setting moving image on the setting screen 30 .

According to this embodiment, the user can set the monitoring area of each step 3 while observing the operator 5 reflected in the setting moving image played on the setting screen 30 . The mark 31 indicating the position of the operator 5 detected based on the same command as the command 401 of the analysis program 400 is superimposed on the setting moving image. Therefore, the user can adjust the position and size of the frame line 32 so as to include the mark 31 when the operator 5 performs the work of each step 3 while observing the appearance of the operator 5 reflected in the setting dynamic image. Thereby, the user can easily set a monitoring area for each step 3 that is suitable for the position of the operator 5 who performs the operation of step 3. In this way, the setting device 100 of this embodiment can support appropriate and simple setting of the monitoring area corresponding to step 3.

§2 Specific examples

<Problems with conventional monitoring area setting methods>

Before describing a specific example of the setting device 100 of this embodiment, problems in the conventional monitoring area setting method will be described with reference to FIGS. 2 and 3 . Conventionally, the monitoring area is set using still images captured by the camera 200 during a period when the operator 5 is not performing work.

FIG. 2 is a diagram showing an example of a still image acquired for setting a monitoring area in the system of the reference form. The still image 20 is obtained by photographing a period when the operator 5 is not performing work, so only the work site 2 is reflected. As shown in FIG. 2 , generally speaking, a large number of equipment and various devices are installed at the work site 2 . the These equipment and various devices are configured according to the sequence of multiple steps included in the job site 2. Therefore, the user can determine the location of each step by confirming the equipment and various devices reflected in the still image 20 . The user imagines the working position of the operator 5 for the determined location of each step, and sets the frame line 32 of a size and position suitable for the imagined working position. Thereby, the area surrounded by the frame line 32 is set as the monitoring area. In FIG. 2 , ten frame lines 32 are respectively set corresponding to the ten steps.

If a still image in which the operator 5 is reflected is used, part of the equipment and various devices will be obscured by the operator 5 , and it is considered that the location of each step cannot be determined with high accuracy. Therefore, conventionally, the monitoring area is set using the still image 20 obtained by photographing a period when the operator 5 is not performing work.

However, since the static image 20 in which the operator 5 is not reflected is used to set the monitoring area, the set monitoring area may deviate from the actual working position. As a result, there arises a problem that the time period during which the work is performed in each step cannot be measured with high accuracy.

FIG. 3 is a diagram showing an example of the relationship between a monitoring area set according to a conventional setting method and an operator's working position. In FIG. 3 , marks 71 a and 71 b respectively indicate the positions of the operator 5 a and the operator 5 b detected based on the command 401 of the analysis program 400 . As shown in Figure 3, three monitoring areas 72a~72c are respectively set for three consecutive steps. The monitoring area 72a is close to the monitoring area 72b. The monitoring area 72b is close to the monitoring area 72c.

Marker 71b is located approximately in the center of monitoring area 72c. Therefore, it is recognized that the operator 5b is performing work in the step corresponding to the monitoring area 72c.

On the other hand, the mark 71a is located near the boundary between the monitoring area 72a and the monitoring area 72b. Therefore, due to slight movement of the operator 5a, the determination result of the area where the operator 5a is present may change between the monitoring area 72a and the monitoring area 72b. Therefore, although the operator 5a is actually working in the step corresponding to the monitoring area 72b, it may be mistakenly recognized that the operator 5a is working in the step corresponding to the monitoring area 72a.

In this way, in the conventional setting method, the set monitoring area may deviate from the actual working position. Especially when an ultra-wide-angle camera or a wide-angle camera is used as the camera 200 , the deviation is likely to occur. Therefore, in order to eliminate such deviation, it is necessary to repeatedly verify the analysis results of the analysis program 400 using the set monitoring area and reset the monitoring area. Therefore, there arises a problem that it is time-consuming and labor-intensive to appropriately set the monitoring area corresponding to each step. The setting device 100 of this embodiment solves this problem.

<Set the hardware structure of the device>

The hardware structure of the setting device 100 will be described with reference to FIG. 4 . In the following, description will be given taking the setting device 100 installed with the analysis program 400 as an example. That is, the setting device 100 also operates as an analysis device that executes the analysis program 400 .

FIG. 4 is a schematic diagram showing an example of the hardware structure of the setting device according to the embodiment. As shown in FIG. 4 , the setting device 100 typically has a structure that follows a general computer architecture.

Specifically, the setting device 100 includes a central processing unit (Central Processing Unit, CPU) or a micro-processing unit (Micro-Processing Unit, MPU) and other processors 101, memory 102, storage 103, camera interface 104, display controller 105, input interface 106 and communication interface 107. These various units can be connected to each other for data communication via the bus.

The processor 101 implements various processes according to this embodiment by expanding various programs stored in the storage 103 into the memory 102 for execution.

The memory 102 is typically a volatile memory device such as a Dynamic Random Access Memory (DRAM), and stores the program read from the memory 103, dynamic images received from the camera 200, processing results, and Artifact information, etc.

The storage 103 is typically a non-volatile magnetic memory device such as a hard disk drive. The memory 103 stores the setting program 110 and the analysis program 400 executed by the processor 101, and the setting information 410 used for executing the analysis program 400. Furthermore, the memory 103 stores the setting moving image 120 and the analysis moving image 130 received from the camera 200 . Furthermore, the number of each of the setting dynamic image 120 and the analysis dynamic image 130 stored in the memory 103 is not limited to one, but may also be multiple. The setting program 110 and the analysis program 400 installed in the memory 103 are distributed in a state of being stored in a memory card or the like.

The setting information 410 is generated by the processor 101 executing the setting program 110 and is saved to the memory 103 .

The camera interface 104 mediates data transfer between the processor 101 and the camera 200 . More specifically, the processor 101 outputs the setting moving image 120 or the analysis moving image 130 to the camera 200 via the camera interface 104 shooting instructions. The camera interface 104 outputs the moving image received from the camera 200 according to the shooting instruction of the setting moving image 120 to the processor 101 as the setting moving image 120 . The camera interface 104 outputs the dynamic image received from the camera 200 according to the shooting instruction of the dynamic image for analysis 130 as the dynamic image for analysis 130 to the processor 101 . The processor 101 saves the setting dynamic image 120 and the analysis dynamic image 130 to the memory 103 .

The display controller 105 is connected to the display device 300 and outputs signals for displaying various information to the display device 300 according to internal instructions from the processor 101 .

The input interface 106 mediates data transmission between the processor 101 and input devices 350 such as keyboard, mouse, and touch panel. That is, the input interface 106 accepts operation instructions given by the user by operating the input device 350 .

The communication interface 107 mediates data transmission between the processor 101 and external devices. The communication interface 107 typically includes Ethernet (registered trademark) or Universal Serial Bus (Universal Serial Bus, USB), etc. Furthermore, the configuration program 110 and the analysis program 400 can also be downloaded from a distribution server or the like through the communication interface 107 .

When using a computer having a structure conforming to a general computer architecture as described above, in addition to applications for providing the functions of this embodiment, an operating system (OS) for providing basic functions of the computer may also be installed. In this case, the program of this embodiment may be a program that calls required modules among the program modules provided as part of the OS in a predetermined order and timing to execute processing. That is, the program itself in this embodiment may not include the above-mentioned module. Instead, processing is performed in cooperation with the OS.

Furthermore, alternatively, part or all of the functions provided by the execution of the setting program 110 may be implemented as dedicated hardware circuits.

<Set the functional structure of the device>

FIG. 5 is a diagram showing an example of the functional structure of the setting device according to the embodiment. As shown in FIG. 5 , the setting device 100 includes a storage unit 10 , an acquisition unit 11 , a detection unit 12 , a provision unit 13 , a setting unit 14 and an analysis unit 15 . The memory unit 10 is implemented by the memory 102 and the storage 103 shown in FIG. 4 . The acquisition part 11 is implemented by the camera interface 104 and the processor 101 executing the setting program 110 or the analysis program 400. The detection unit 12 is implemented by the processor 101 executing the setting program 110 . The providing part 13 is realized by the display controller 105 and the processor 101 executing the setting program 110. The setting part 14 is realized by the input interface 106 and the processor 101 executing the setting program 110. The analysis unit 15 is implemented by the processor 101 executing the analysis program 400 .

The acquisition unit 11 acquires the setting moving image 120 and the analysis moving image 130 from the camera 200 . The acquisition unit 11 stores the acquired setting moving image 120 and analysis moving image 130 in the storage unit 10 .

For example, when the operator 5 is working at the work site 2 , the acquisition unit 11 acquires the setting moving image 120 of a specified duration in response to an input to the input device 350 . Furthermore, the acquisition unit 11 periodically acquires the analysis moving image 130 of a predetermined duration during the period from the start of the work to the end of the work at the work site 2 . Furthermore, the acquisition unit 11 may acquire a part of the plurality of analysis moving images 130 as the setting moving image 120 .

The detection unit 12 detects the position of the operator 5 for each frame of the setting dynamic image 120 based on the same command as the command 401 of the analysis program 400 (see FIG. 1 ). That is, the setting program 110 includes the same command as the command 401. The detection unit 12 is implemented by the processor 101 executing the command.

Specifically, the detection unit 12 uses well-known object recognition technology to perform recognition processing of the human body (that is, the operator 5 ) on each frame of the setting dynamic image 120 . The detection unit 12 determines, for each identified operator 5 , a rectangular area in which the operator 5 is mapped. The detection unit 12 detects the center of the determined rectangular area as the position of the operator 5 . The detection unit 12 calculates the coordinates of the four vertices of the rectangular area in which the operator 5 is mapped and the coordinates of the position of the operator 5 .

The providing unit 13 provides a setting screen for setting the monitoring area corresponding to each step 3. Play the setting dynamic image on the setting screen. The providing unit 13 causes the display device 300 to display a setting screen.

The setting unit 14 receives an operation on the setting screen from the input device 350 and sets the monitoring area corresponding to each step 3 based on the operation. The setting unit 14 generates setting information 410 indicating the monitoring area set for each step 3, and saves the setting information 410 in the storage unit 10. For each step 3, the setting information 410 associates a step ID identifying the step 3 with area information indicating the size and position of the monitoring area set for the step 3. The area information represents, for example, the coordinates of the four vertices of a rectangular monitoring area.

The analysis unit 15 performs image analysis on the analysis moving image 130 . As shown in FIG. 5 , the analysis unit 15 includes a detection unit 16 , a determination unit 17 , and a generation unit 18 .

The detection unit 16 detects the position of the operator 5 for each frame of the analysis moving image 130 according to the command 401 (see FIG. 1 ). The command 401 includes a command to detect a rectangular area in which the operator 5 is mapped for each frame of the analysis dynamic image 130 and a command to determine the center point of the rectangular area as the position of the operator. The detection unit 16 is implemented by executing these commands. The detection method performed by the detection unit 16 is the same as the detection method performed by the detection unit 12 . Therefore, detailed explanation is omitted.

The determination unit 17 determines based on the command 402 (see FIG. 1 ) whether, for each of the plurality of steps 3, at the shooting time of each frame of the analysis moving image 130, the monitoring area set for the step 3 is Operator 5 exists. The determination unit 17 refers to the setting information 410 to determine the monitoring area in each step 3.

When an operator 5 is working in a certain step 3, the operator 5 will continuously stay in the monitoring area corresponding to step 3. In other words, if the operator 5 simply passes through a certain monitoring area, the operator 5 does not perform the operation in step 3 corresponding to the monitoring area. Furthermore, there may be a case in which the detection unit 12 fails to recognize the human body even though the operator 5 is reflected in the frame of the analysis dynamic image 130 . Therefore, the determination unit 17 may also determine based on the following commands (1) to (3) included in the command 402: at the shooting time of the target frame included in the analysis dynamic image 130, at the same time as the target Whether there is operator 5 in the object monitoring area corresponding to step 3.

Command (1): Extract a plurality of consecutive frames including the target frame from the dynamic image 130 for analysis. For example, the object frame and five frames before and after the object frame are extracted, totaling eleven frames.

Command (2): Calculate the proportion of frames in which the position of the operator 5 detected by the detection unit 12 is included in the target monitoring area among the plurality of frames obtained by executing the command (1).

Command (3): In response to the situation where the ratio calculated by execution of command (2) exceeds the threshold value, it is determined that the operator 5 is present in the target monitoring area at the time when the target frame is photographed.

Furthermore, for each step 3, the determination unit 17 generates time period information 420 for each time period for one or more time periods in which the operator 5 is present in the monitoring area set for the step 3. The determination unit 17 saves the generated period information 420 to the storage unit 10 . The time period information 420 represents step ID, start time, and end time.

For each step 3, the determination unit 17 identifies a plurality of consecutive frames in which it is determined that the operator 5 exists in the monitoring area corresponding to the step 3, from the dynamic image 130 for analysis. The determination unit 17 generates time period information 420 indicating the step ID for identifying the step 3 for the determined plurality of frames. The time period information 420 represents the shooting time of the first frame among the determined frames as the start time, and represents the shooting time of the last frame among the determined frames as the end time.

The generation unit 18 generates various graphs based on the time period information 420 . For example, the generation unit 18 may also generate a Gantt chart for each of the plurality of steps 3 indicating the time period in which the operator 5 exists in the corresponding monitoring area (ie, the time period indicated by the time period information 420 ). ). Alternatively, the generation unit 18 may generate, for each of the plurality of steps 3, a box shape indicating the deviation of the time when the operator 5 exists in the corresponding monitoring area (that is, the length of the period indicated by the period information 420). Figure. Depend on The graph generated by the generation unit 18 is provided in response to the user's request.

<Example of setting screen>

FIG. 6 is a diagram showing an example of a setting screen. The setting screen 30 illustrated in FIG. 6 is provided by the providing unit 13 and displayed on the display device 300 . As shown in FIG. 6 , the setting screen 30 includes a moving image playback area 33, a list display field 34, a delete button 35, an add button 36, an analysis process start button 37, a save button 38, a moving image playback button group 39, and a progress bar. (seek bar)40.

In the moving image playback area 33, the setting moving image 120 is displayed. The setting moving image 120 is displayed in response to the operation of the moving image play button group 39 or the progress bar 40 . The moving image playback button group 39 includes buttons for selecting any one of playback, pause, frame forward, frame rewind, and time specification. The setting moving image 120 is displayed in the moving image playback area 33 according to the selected mode of play, pause, frame forward, frame backward, and time specification. When "play" is selected, the setting dynamic image 120 is played. When "Pause" is selected, maintain the frame that was displayed when "Pause" was selected. When "frame forward" is selected, the setting dynamic image 120 is played back at a speed that is slow enough to confirm each frame. When "frame rewind" is selected, the setting dynamic image 120 is played back at a speed slow enough to confirm each frame. When "time designation" is selected, a frame at the designated time in the setting dynamic image 120 is displayed.

In the moving image playback area 33 , a frame 41 indicating a rectangular area in which the operator 5 is reflected, and a mark 31 are displayed overlapping the setting moving image 120 . Location. As mentioned above, mapping has The rectangular area of the operator 5 and the position of the operator 5 are detected by the detection unit 12 . In the example shown in FIG. 6 , frame lines 41 a to 41 c and marks 31 a to 31 c are respectively displayed for the operators 5 a to 5 c. Thereby, the user can grasp the position of each operator 5 detected when executing the analysis program 400 .

In response to the click operation of the add button 36, the providing unit 13 displays the frame 32 for setting a new monitoring area in the moving image playback area 33, and adds a name for identifying the new monitoring area to the list. Column 34 is displayed.

When the add button 36 is clicked, the providing unit 13 assigns an arbitrary name different from the name already displayed in the list display column 34 to the new monitoring area. The providing unit 13 accepts editing of the name displayed in the list display column 34 . In this way, the user can give a desired name to the monitoring area.

The providing unit 13 deletes the selected frame line 32 from the moving image playback area 33 in response to an operation of clicking the delete button 35 while one frame line 32 displayed in the moving image playback area 33 is selected. Furthermore, the providing unit 13 deletes the name corresponding to the deleted frame line 32 from the list display column 34 .

Alternatively, in response to an operation of clicking the delete button 35 while a name displayed in the list display column 34 is selected, the selected name is deleted from the list display column 34 . Furthermore, the providing unit 13 deletes the frame 32 corresponding to the deleted name from the moving image playback area 33 .

In the example shown in FIG. 6 , two frame lines 32 a and 32 b for respectively setting two monitoring areas are displayed in the moving image playback area 33 . The setting unit 14 edits the size and position of the frame 32 displayed in the moving image playback area 33 Acceptance. Specifically, the setting unit 14 determines the monitoring area indicated by a specified one of the one or more frame lines 32 as the monitoring area to be edited, corresponding to the moving image playback area 33 of the setting screen 30 to edit the monitoring area that is the editing object. That is, the user only needs to adjust the size and position of the frame line 32 so as to include the work being performed in a certain step while confirming the setting dynamic image 120 and the mark 31 indicating the detected position of the operator 5 or 5 corresponding to the mark 31. Thereby, the user can easily set a monitoring area suitable for the position of the operator 5 who performs the operation of step 3.

The setting unit 14 generates area information indicating the coordinates of the four vertices of the monitoring area surrounded by the frame lines 32 displayed in the moving image playback area 33 in response to the click operation of the save button 38 . The setting unit 14 generates setting information 410 in which each area information is associated with the step ID input for the area information. For example, the setting unit 14 may use the name displayed in the list display column 34 as the step ID. Alternatively, the setting unit 14 may display the input screen of the step ID corresponding to each monitoring area on the display device 300, and set the step ID for each monitoring area in response to the input to the input screen.

The providing unit 13 provides a screen showing the analysis results obtained using the analysis program 400 using the setting dynamic image 120 and the set monitoring area in response to the click operation of the analysis start button 37 .

Specifically, the providing unit 13 instructs the analysis unit 15 to process using the setting moving image 120 and the set monitoring area. The analysis unit 15 takes the setting dynamic image 120 as an analysis object and performs image analysis using the monitoring area. As a result, Period information 420 is generated (see FIG. 5 ). Furthermore, a Gantt chart or a box chart is generated based on the time period information 420. The Gantt chart represents the time period in each step 3 when it is determined that the operator 5 exists in the monitoring area. The box chart represents each time period. In Step 3, the deviation in the length of the period during which the operator 5 is present in the monitoring area is determined.

FIG. 7 is a diagram showing an example of a screen provided in response to an operation of clicking an analysis start button. As shown in FIG. 7 , the screen 50 includes a Gantt chart 51 for each of the plurality of steps 3 indicating the time period during which the operator 5 exists in the corresponding monitoring area. Furthermore, the screen 50 includes a box chart 52 for each of the plurality of steps 3 that shows a variation in the time when the operator 5 is present in the corresponding monitoring area. Furthermore, the screen 50 may also include only one of the Gantt chart 51 and the box chart 52 .

If the monitoring area is appropriately set according to the position of the operator 5 who performs work at a certain step, the deviation in the time when it is determined that the operator 5 is present in the monitoring area will be small.

On the other hand, if the monitoring area is not set appropriately corresponding to the position of the operator 5 who performs work at a certain step (for example, if the monitoring area 72a in FIG. 3 is not set), it is determined that the operator 5 is present in the monitoring area. The existing time deviation will become larger. Furthermore, in the Gantt chart 51, the time period for performing work between two adjacent steps changes slightly.

Therefore, the user can verify whether the monitoring area is appropriately set by confirming the screen shown in FIG. 7 .

In this way, the user can easily set and perform each step 3 while observing the setting screen 30 shown in FIG. 6 and the screen 50 shown in FIG. 7 The position of operator 5 of operation 3 is suitable for the monitoring area.

<Modification>

As described above, the determination unit 17 of the analysis unit 15 may also determine based on the commands (1) to (3) included in the command 402: at the shooting time of the object frame, the object corresponding to step 3 as the object is Monitor whether there is an operator 5 in the area. At this time, the threshold value used in the execution of command (3) will affect the analysis of the period when the operator 5 is present in the monitoring area. That is, the Gantt chart 51 and the box chart 52 shown in FIG. 7 change according to the threshold value.

Therefore, the providing unit 13 may further provide a screen for setting the threshold value used in execution of command (3).

FIG. 8 is a diagram showing an example of a screen for setting a threshold value. As shown in FIG. 8 , the screen 60 includes an input field 61 for inputting a threshold value.

The setting unit 14 sets a threshold value in response to the operation on the screen 60 and includes the set threshold value in the setting information 410 . Command (3) of the analysis program 400 is executed using the threshold value included in the setting information 410 .

Thereby, the user can set an appropriate threshold value while confirming the Gantt chart 51 and the box chart 52 included in the screen 50 shown in FIG. 7 .

In the example shown in FIGS. 4 and 5 , the analysis program 400 is installed in the setting device 100 . However, as mentioned above, the analysis program 400 can also be installed in a device different from the setting device 100 . At this time, the setting information 410 generated by the setting device 100 is transmitted to the device where the analysis program 400 is installed. Furthermore, the providing unit 13 of the setting device 100 may also instruct the device in which the analysis program 400 is installed to use the setting action. The state image 120 and the set monitoring area are processed. Furthermore, the providing unit 13 only needs to acquire the time period information 420 (see FIG. 5 ) as the analysis result from the device in which the analysis program 400 is installed, and generate the screen 50 shown in FIG. 7 based on the acquired time period information 420 .

§3 Notes

As described above, this embodiment includes the following disclosure.

(Structure 1)

A setting device 100 for setting a monitoring area for an analysis program 400 including a first command 401 and a second command 402 for an analysis dynamic image 130 acquired from a camera 200 of a shooting work site 2 Each frame is used to detect the position of the operator 5. The job site 2 includes a plurality of steps 3. The second command 402 is based on the monitoring area corresponding to the plurality of steps 3 and the detected The position of the operator 5 is used to determine whether there is the operator 5 in the monitoring area. The setting device 100 includes: acquisition units 11, 101, 104, which acquire the image of the operation site 2 from the camera 200. The setting moving image 120 of the operator 5 who is working at the work site 2; the detection unit 12, 101, based on the same command as the first command 401, detects the setting moving image 120. each frame to detect the position of the operator 5; the providing units 13, 101, and 104 provide the first screen 30 for playing the setting dynamic image 120; and the setting units 14, 101, and 106 correspond to the settings for The operation of the first screen 30 To set the monitoring area, the first screen 30 includes a mark 31 indicating the position detected by the detection units 12 and 101 , and the mark 31 is superimposed and displayed on the setting dynamic image 120 .

(Structure 2)

The setting device 100 according to the structure 1, wherein the providing units 13, 101, and 104 further provide a second screen 50. The second screen 50 represents the use of the setting dynamic image 120 and the setting unit 14. , 101, 106, the analysis results obtained by the analysis program 400 of the monitoring area set.

(Structure 3)

The setting device 100 according to the structure 2, wherein the second screen 50 includes a diagram 51 for each step of the plurality of steps 3 indicating a time period during which the operator 5 exists in the corresponding monitoring area.

(Structure 4)

The setting device 100 according to the structure 2, wherein the second screen 50 includes a box shape indicating a deviation in the time when the operator 5 exists in the corresponding monitoring area for each step of the plurality of steps 3. Figure 52.

(Structure 5)

The setting device 100 according to any one of structures 2 to 4, wherein the second command 402 includes: A command that extracts a plurality of consecutive frames including an object frame from the dynamic image for analysis 130; calculates that the object monitoring area in the plurality of frames includes the objects detected by execution of the first command 401. A command is issued for the ratio of the frame at the position of the operator 5, the object monitoring area corresponds to the object step in the plurality of steps 3; and corresponding to the situation where the ratio exceeds a threshold value, it is determined to be in There is a command that the operator 5 exists in the object monitoring area at the shooting time of the object frame, and the providing units 13, 101, and 104 further provide a third screen 60 for setting the threshold value, so The setting units 14, 101, and 106 further set the threshold value in response to the operation on the third screen 60.

(Structure 6)

The setting device 100 according to any one of structures 1 to 5, wherein the first command 401 includes: for each frame of the dynamic image for analysis 130, detect a rectangle on which the operator 5 is reflected. area; and a command to determine the center point of the rectangular area as the position of the operator 5, the detection parts 12, 101, based on the same command as the first command 401, for the setting dynamic Each frame of the image 120 detects the rectangular area, and the first frame 30 further includes a first frame line 41 representing the rectangular area detected by the detection parts 12 and 101, The first frame line 41 is overlapped and displayed on the setting dynamic image 120 .

(Structure 7)

The setting device 100 as described in any one of structures 1 to 5, wherein the dynamic image 120 for setting is in the first screen 30 according to the steps from play, pause, forward frame by frame, backward frame by frame and The method selected for time specification is displayed.

(Structure 8)

The setting device 100 according to any one of structures 1 to 7, wherein the first screen 30 includes more than one second frame line 32 respectively representing more than one monitoring area, and the setting parts 14, 101, 106. Determine the monitoring area represented by a designated second frame line 32 among the one or more second frame lines 32 as the monitoring area to be edited, corresponding to the specified second frame line 32. The monitoring area as the editing object is edited by operating the frame line 32.

(Structure 9)

A setting method for setting a monitoring area for an analysis program 400 including a first command 401 and a second command 402 for each analysis dynamic image 130 acquired from the camera 200 of the shooting work site 2. frame to detect the position of the operator 5, the job site 2 includes a plurality of steps 3, the second command 402 is based on the monitoring area corresponding to the plurality of steps 3 and the detected The position of the operator 5 is used to determine whether there is the operator 5 in the monitoring area. The setting method includes the following steps: obtaining from the camera 200 the image of the operation site 2 and the image of the operation being performed. The setting dynamic image 120 of the operator 5 performing work at the industrial site 2; according to the same command as the first command 401, the operator is detected for each frame of the setting dynamic image 120 5; provide a first screen 30 that plays the setting dynamic image 120; and set the monitoring area corresponding to the operation on the first screen 30, the first screen 30 includes a The mark 31 of the position detected by the detection units 12 and 101 is superimposed and displayed on the setting moving image 120 .

(Structure 10)

A setting program 110 for setting a monitoring area for an analysis program 400 including a first command 401 and a second command 402 for the analysis dynamic image 130 acquired from the camera 200 of the shooting work site 2 Each frame is used to detect the position of the operator 5. The two pairs of work sites include a plurality of steps 3. The second command 402 is based on the monitoring area corresponding to the plurality of steps 3 and the detected The position of the operator 5 is used to determine whether there is the operator 5 in the monitoring area. The setting program 110 causes the computer to perform the following steps: obtain from the camera 200 the image of the operation site 2 and the current The dynamic image 120 for setting of the operator 5 working at the work site 2; according to the same command as the first command 401, the dynamic image 120 for setting is detected for each frame of the dynamic image 120. The position of the operator 5; providing the first screen 30 that plays the setting dynamic image 120; and setting the monitoring area corresponding to the operation on the first screen 30, The first screen 30 includes a mark 31 indicating the position detected by the detection units 12 and 101 , and the mark 31 is superimposed on the setting moving image 120 .

Although the embodiments of the present invention have been described, it should be understood that the embodiments disclosed this time are illustrative and not restrictive in every respect. The scope of the present invention is shown by the patent application scope, and it is intended to include all changes within the meaning and scope that are equivalent to the patent application range.

1: System

2:Working site

3_1~3_n, S1~S4: steps

5: Operator

30: Setting screen

31: mark

32: Frame line

100: Set device

200:Camera

300:Display device

350:Input device

400:Analysis program

401, 402: Command

Claims (9)

A monitoring area setting device that sets a monitoring area for an analysis program including a first command and a second command for detecting each frame of a dynamic image for analysis acquired from a camera at a shooting work site. The position of the operator, the job site includes multiple steps, and the second command is to determine the position of the operator based on the monitoring area corresponding to each of the multiple steps and the detected position of the operator. The monitoring area is monitored for the presence of the operator, and the setting device for the monitoring area includes an acquisition unit that acquires from the camera the setting dynamics that map the work site and the operator working at the work site. image; a detection unit that detects the position of the operator for each frame of the setting dynamic image according to the same command as the first command; and a providing unit that provides playback of the setting dynamic image a first screen; and a setting unit that sets the monitoring area corresponding to the operation on the first screen, the first screen including a mark indicating the position detected by the detection unit, the mark being The providing unit further provides a second screen showing the monitoring area using the setting moving image and the setting unit. According to the analysis results obtained by the analysis program, the second screen includes at least one of the following: for each of the plurality of steps A diagram representing the time period during which the operator exists in the corresponding monitoring area, and a deviation of the time period during which the operator exists in the corresponding monitoring area for each step of the plurality of steps. Box plot. A monitoring area setting device that sets a monitoring area for an analysis program including a first command and a second command for detecting each frame of a dynamic image for analysis acquired from a camera at a shooting work site. The position of the operator, the job site includes multiple steps, and the second command is to determine the position of the operator based on the monitoring area corresponding to each of the multiple steps and the detected position of the operator. The monitoring area is monitored for the presence of the operator, and the setting device for the monitoring area includes an acquisition unit that acquires from the camera the setting dynamics that map the work site and the operator working at the work site. image; a detection unit that detects the position of the operator for each frame of the setting dynamic image according to the same command as the first command; and a providing unit that provides playback of the setting dynamic image a first screen; and a setting unit that sets the monitoring area corresponding to the operation on the first screen, the first screen including a mark indicating the position detected by the detection unit, the mark being The providing unit further provides a second screen showing the monitoring area using the setting moving image and the setting unit. Said points The analysis result obtained by the analysis program, the second command includes: a command to extract multiple consecutive frames including the object frame from the dynamic image for analysis; calculate the object monitoring value in the multiple frames. a command in which a region includes a proportion of a frame of the operator's position detected by execution of the first command, the object monitoring region corresponding to an object step among the plurality of steps; and corresponding to the proportion When the threshold value is exceeded, it is determined that the operator exists in the object monitoring area at the shooting time of the object frame, and the providing unit further provides a third parameter for setting the threshold value. Three screens, the setting part corresponds to the operation on the third screen, and further sets the threshold value. A monitoring area setting device that sets a monitoring area for an analysis program including a first command and a second command for detecting each frame of a dynamic image for analysis acquired from a camera at a shooting work site. The position of the operator, the job site includes multiple steps, and the second command is to determine the position of the operator based on the monitoring area corresponding to each of the multiple steps and the detected position of the operator. The monitoring area is monitored for the presence of the operator, and the setting device for the monitoring area includes an acquisition unit that acquires from the camera the setting dynamics that map the work site and the operator working at the work site. Image; detection unit, based on the same command as the first command, for the setting Each frame of the dynamic image is used to detect the position of the operator; the providing unit provides a first screen for playing the dynamic image for setting; and the setting unit sets the settings corresponding to the operation on the first screen. The monitoring area, the first screen includes a mark indicating the position detected by the detection unit, the mark is superimposed on the setting dynamic image, and the first command includes: for the Each frame of the dynamic image for analysis is a command to detect a rectangular area in which the operator is mapped; and a command to determine the center point of the rectangular area as the position of the operator, and the detection unit is based on the The same command as the first command is used to detect the rectangular area for each frame of the setting dynamic image, and the first screen further includes a first character representing the rectangular area detected by the detection unit. Frame line, the first frame line is overlapped and displayed on the setting dynamic image. The monitoring area setting device according to any one of claims 1 to 3, wherein the dynamic image for setting is in the first screen according to the sequence of play, pause, frame by frame, frame by frame. The method selected for rewind and time designation is displayed. The monitoring area setting device as described in any one of claims 1 to 3, wherein The first screen includes one or more second frame lines respectively representing one or more monitoring areas, and the setting unit is represented by a designated one of the one or more second frame lines. The monitoring area is determined as the monitoring area as the editing target, and the monitoring area as the editing target is edited corresponding to the operation on the designated second frame line. A method of setting a monitoring area for setting a monitoring area for an analysis program including a first command and a second command for detecting each frame of a dynamic image for analysis acquired from a camera at a shooting work site. The position of the operator, the job site includes multiple steps, and the second command is to determine the position of the operator based on the monitoring area corresponding to each of the multiple steps and the detected position of the operator. Whether the operator is present in the monitoring area, the setting method of the monitoring area includes the following steps: an acquisition step of acquiring, from the camera, images of the operation site and the operator who is working at the operation site. A dynamic image for setting; a detection step for detecting the position of the operator for each frame of the dynamic image for setting according to the same command as the first command; and a step for playing the dynamic image for setting. a first screen of a dynamic image; and a setting step of setting the monitoring area corresponding to an operation on the first screen, the first screen including a mark indicating the detected position of the operator, The mark is displayed superimposed on the setting dynamic image, and the providing step includes providing a second screen showing the use of the setting dynamic image and the settings set by the setting unit. The analysis results obtained by the analysis program of the monitoring area, the second screen includes at least one of the following: for each step of the plurality of steps, it indicates that the corresponding monitoring area has the A graph of the time period in which the operator exists, and a box plot showing, for each of the plurality of steps, a deviation of the time in which the operator exists in the corresponding monitoring area. A method of setting a monitoring area for setting a monitoring area for an analysis program including a first command and a second command for detecting each frame of a dynamic image for analysis acquired from a camera at a shooting work site. The position of the operator, the job site includes multiple steps, and the second command is to determine the position of the operator based on the monitoring area corresponding to each of the multiple steps and the detected position of the operator. Whether the operator is present in the monitoring area, the setting method of the monitoring area includes the following steps: an acquisition step of acquiring, from the camera, images of the operation site and the operator who is working at the operation site. A dynamic image for setting; a detection step for detecting the position of the operator for each frame of the dynamic image for setting according to the same command as the first command; and a step for playing the dynamic image for setting. The first picture of the dynamic image; and the setting step of setting the monitoring area corresponding to the operation on the first picture, The first screen includes a mark indicating the detected position of the operator, and the mark is superimposed on the setting dynamic image, and the providing step includes providing a second screen, and the The second screen shows the analysis results obtained by the analysis program using the setting dynamic image and the monitoring area set by the setting unit, and the second command includes: from the analysis dynamic image A command for extracting a plurality of consecutive frames including an object frame from an image; and calculating a frame among the plurality of frames that includes the position of the operator detected by execution of the first command in the object monitoring area. a proportional command, the object monitoring area corresponds to an object step in the plurality of steps; and corresponding to the situation where the proportion exceeds a threshold value, it is determined that the object is present at the shooting time of the object frame A command for an operator to exist in the object monitoring area, the providing step includes a step of providing a third screen for setting the threshold value, and the setting step includes the following steps: corresponding to the third screen operation, and then set the threshold value. A method of setting a monitoring area for setting a monitoring area for an analysis program including a first command and a second command for detecting each frame of a dynamic image for analysis acquired from a camera at a shooting work site. The location of the operator, the job site includes multiple steps, the second command is based on the Multiple steps respectively correspond to the monitoring area and the detected position of the operator to determine whether there is the operator in the monitoring area. The setting method of the monitoring area includes the following steps: obtaining step , acquiring a dynamic image for setting that reflects the work site and the operator working at the work site from the camera; the detection step is to perform the setting for the setting based on the same command as the first command. Use each frame of the dynamic image to detect the position of the operator; provide a step to provide a first screen for playing the dynamic image for setting; and a setting step to set in response to the operation on the first screen In the monitoring area, the first screen includes a mark indicating the detected position of the operator, the mark is displayed overlaid on the setting dynamic image, and the first command includes: for all Each frame of the dynamic image for analysis is used to detect a rectangular area mapped with the operator; and a command is used to determine the center point of the rectangular area as the position of the operator. The detection step includes the following: The step of: detecting the rectangular area for each frame of the setting dynamic image according to the same command as the first command, and the first screen further includes a representation of all the frames detected by the detection unit. The first frame line of the rectangular area is overlapped and displayed on the setting dynamic image. A monitoring area setting program enables a computer to execute the monitoring area setting method described in any one of claims 6 to 8.

Images