JP7133375B2 - Dangerous work detection system, analysis device, display device, dangerous work detection method, and dangerous work detection program - Google Patents

Description

TECHNICAL FIELD The present invention relates to a technique for detecting work that is presumed to be dangerous from a video of a worker's work.

In the past, as a measure to ensure the safety of workers' work, the state of work was filmed with a video camera, and the video footage taken was taken home and visually checked to determine whether dangerous work was being carried out. There was an activity in which a human judges and uses it to improve future work.

For example, in equipment installation work in a server room, when laying cables on the ceiling of a server rack, a worker climbs a stepladder or ladder to perform the work. At this time, there have been some cases where a person slips and falls from a stepladder or the stepladder topples over, resulting in serious injury. In order to prevent this, work procedures such as wearing safety equipment such as fall prevention belts are stipulated. Japanese Patent Laid-Open Publication No. 2002-300000 discloses a technique for monitoring the work status of a worker.

Japanese Patent Application Laid-Open No. 2006-285639

The actual situation is that the observance of the work procedure is entrusted to the individual worker. Therefore, a third party confirms at a later date whether the safety equipment is properly worn during such work by checking the captured video, and provides guidance for work improvement. .

However, there is a problem that the confirmation of the video depends entirely on manual visual inspection, which requires an enormous amount of time and effort, and how to reduce this time and effort has been an issue.

The present invention has been made in view of the above problems, and an object of the present invention is to detect work that is presumed to be dangerous from a video of a worker's work, and reduce the user's workload. It is to provide technology.

The present invention is a dangerous work detection system having an analysis device and a display device, wherein the analysis device analyzes a plurality of still images extracted from a moving image of the work, and extracts still images showing the dangerous work being performed. The display device reproduces the portion of the still image in which the dangerous work is being performed at a first speed, and the portion other than the still image in which the dangerous work is being performed , play at a second speed higher than the first speed, or skip without playing.

The present invention is an analysis device for detecting dangerous work, which analyzes a plurality of still images extracted from a moving image of the work, detects a person from each still image, and detects a person from each of the still images. The plurality of still images are analyzed using an object detection unit that detects an object that may cause dangerous work from each still image, and the detection result of the person detection unit and the detection result of the object detection unit. and a moving image creation unit that creates a moving image by editing only the still image portion having no risk so that the playback speed is increased by n times.

The present invention is a display device for reproducing a moving image of a work, wherein the screen of the display device comprises an area for displaying the moving image, a seek bar indicating a playback position of the moving image, and a time corresponding to the seek bar. A color bar indicating the presence or absence of danger or the level of danger of work in the time zone is displayed with different colors or different patterns, and a moving image of the first time zone in which the danger is present or the danger level is high. is reproduced at a first speed, and a moving image in a second time period other than the first time period is reproduced at a second speed higher than the first speed, or is skipped without being reproduced. Prepare.

The present invention is a dangerous work detection method performed by a dangerous work detection system having an analysis device and a display device, wherein the analysis device analyzes a plurality of still images extracted from a moving image of the work, and determines whether the dangerous work is detected. performing a detection step of detecting a still image in which the dangerous work is being performed, wherein the display device reproduces a portion of the still image in which the dangerous work is being performed at a first speed, and displays the still image in which the dangerous work is being performed; Other parts are reproduced at a second speed higher than the first speed, or are skipped without being reproduced.

The present invention is a dangerous work detection program for causing a computer to function as the analysis device.

The present invention is a dangerous work detection program for causing a computer to function as the display device.

Advantageous Effects of Invention According to the present invention, it is possible to provide a technology for detecting work that is presumed to be dangerous from a video of a worker's work and reducing the user's work load.

Shows the system configuration. Indicates the still image to be analyzed. It shows how a person and a platform are detected from a still image. 4 shows a flowchart of the operation of the determination result calculation unit. It shows how a frame is drawn on a still image. A list of judgment results for each time zone of video is shown. It shows how to generate a moving image with changed playback speed. 11 shows a list of determination results reflecting changes in playback speed. Indicates what is displayed on the screen. Shows other display contents on the screen. The screen of FIG. 10 shows how a part of the moving image is skipped and reproduced. FIG. 10 shows a state in which the moving image is moved to a predetermined portion on the screen of FIG. 10 and played. A method for calculating the height of a person from the floor using a ratio will be shown.

One embodiment of the present invention is shown below.

FIG. 1 shows the configuration of a system according to this embodiment.

This system has an analysis device 101 and a display device 102 , and these devices are connected via a network 103 .

Analyzing device 101 receives a moving image captured by video camera 105 .

Analysis device 101 has still image extraction unit 111 , dangerous work analysis unit 112 , and moving image creation unit 113 .

The dangerous work analysis unit 112 analyzes a plurality of still images extracted from a moving image of work, and detects a still image in which dangerous work is being performed. The illustrated dangerous work analysis unit 112 includes a person detection unit 121 that analyzes a still image and detects a person, and a table detection unit 122 (object detection) that analyzes the still image and detects a table (object) from the still image. unit) and a risk determination unit 123 that determines the risk of a plurality of still images using the detection results of the person detection unit 121 and the platform detection unit 122 .

Person detection unit 121 has trained model 131 and deep learning (DL) library 132 . The platform detection unit 122 has a trained model 141 and a DL library 142 . Risk determination section 123 has first determination section 151 , second determination section 152 , third determination section 153 , and determination result calculation section 154 .

The moving image creation unit 113 creates a moving image by editing only the portion of the still image with no risk so that the playback speed is increased by N times. The moving image creating section 113 has a frame drawing section 161 and a playback speed changing section 162 .

The display device 102 comprises a screen 171 , user input means 172 and a playback section 173 . User input means 172 may be various input devices including a mouse, touchpad, touchscreen, keyboard, buttons, and the like. The reproducing unit 173 reproduces the portion of the still image in which the dangerous work is being performed at a first speed, and reproduces the portion other than the still image in which the dangerous work is being performed at a second speed, which is faster than the first speed. Play or skip without playing.

These devices 101 and 102 operate to detect and display a video of a work that is presumed to be dangerous, according to the procedures shown below.

First, analysis device 101 receives a moving image captured by video camera 105 as an input. Here, it is assumed that, for example, a moving image with a playback time of 10 minutes is captured and the file is input to the analysis device 101 .

The still image extraction unit 111 divides the moving image into a plurality of still images (frames), and thins out the still images. That is, one frame is extracted from consecutive M frames. Specifically, assuming that the moving image is shot at 30 frames per second, for example, only one frame out of six is selected and thinned out by 5 frames per second. This thinning process can reduce the total amount of calculation required for subsequent analysis processes.

In a video recording of the state of work, the state does not change rapidly in a short period of time much less than 0.2 seconds, so such thinning processing is appropriate in light of the application.

Thereafter, the dangerous work analysis unit 112 receives a plurality of still images (still image group) extracted by the thinning process, and performs image analysis processing on each still image.

Originally, moving images are realized by continuously displaying a collection of still images shot continuously, such as at 30 frames per second. Therefore, when handling moving images, a file format that realizes moving images may be used, or a set of still image files may be used. Can handle videos practically. Here, moving images include not only a file format that realizes moving images, but also a collection of still image files.

FIG. 2 shows an example of a still image to be analyzed.

Here, for example, as a result of the thinning processing performed by the still image extraction unit 111, the image at the point where 4 minutes have passed from the start is shown in FIG. A still image 201 shows a person 202 and a stepladder 203 . In this embodiment, an example of detecting a platform such as a stepladder as a predetermined object (tool, device, etc.) that may cause dangerous work will be described below.

Note that the object is not limited to a stepladder, ladder, stepping stool, scaffold, workbench, or other platform on which a person sits to work at a high position. For example, the object may be a tool (scissors, knife, etc.) for cutting cables, wires, and the like.

Dangerous work analysis unit 112 first detects a person by person detection unit 121 from the input still image. The person detection unit 121 incorporates a trained model 131 obtained as a result of learning an image of a person in advance as a component. The DL library 132 uses the learned model 131 to determine whether or not a person appears in the input image (still image), and outputs the certainty of the determination as a degree of certainty. Furthermore, the DL library 132 acquires the position of the person in the still image (here, the upper left and lower right coordinates).

The implementation of the DL library that processes images is widely available as open source, etc. By inputting a large amount of training data in advance to the learning function of the DL, the connections of the neural network are learned. As a result, when the DL library is actually used like the DL library 132 of the person detection unit 121, it determines whether or not an image of the same type as the pre-learned data is reflected. It is possible to output the probability as a degree of certainty, and obtain the upper left and lower right coordinates of the position of the image of the same type reflected.

By using such a DL library implementation, the person detection unit 121 described above can be realized.

Dangerous work analysis unit 112 next detects an object, that is, a stepladder in the present embodiment, using platform detection unit 122 . The platform detection unit 122 incorporates a trained model 141 obtained as a result of pre-learning the image of the object as a component. The DL library 142 uses the trained model 141 to determine whether or not the target object is reflected in the input image, outputs the certainty at the time of determination as a degree of certainty, and furthermore, determines whether the target is reflected Get the top left and bottom right coordinates of the object's position.

FIG. 3 shows how a person and a platform are detected from a still image.

In this example, both the person 202 and the platform 203 appear in the still image 201, so the person and the platform are successfully detected. Therefore, the dangerous work analysis unit 112 acquires the detection result that the X coordinate of the person 202 is in the range of 301-302 and the Y coordinate of the person is in the range of 311-312. The dangerous work analysis unit 112 also acquires the detection result that the X coordinate of the platform is in the range 303-304 and the Y coordinate of the platform is in the range 313-314. Also, the dangerous work analysis unit 112 acquires certainty factors (percentage values) for each of the person and the platform.

Dangerous work analysis unit 112 next uses risk determination unit 123 to determine the degree of risk according to the following procedure based on the obtained detection result.

First, first determination unit 151 of risk determination unit 123 determines whether person detection unit 121 and platform detection unit 122 have detected both a person and a platform from the input still image. Next, the second determination unit 152 of the risk determination unit 123 determines that the detected person and the platform overlap in the horizontal direction (X-axis direction). Specifically, the second determination unit 152 determines that the X coordinates 301 to 302 of the person and the X coordinates 303 to 304 of the table overlap each other in the horizontal direction when the intersection of the sets is not an empty set. determined to be Further, the second determination unit 152 may determine whether the detected person and the platform overlap not only in the horizontal direction but also in the vicinity.

Next, the third determination unit 153 of the risk determination unit 123 determines whether or not the person is above and the platform is below the detected person and platform. That is, the third determination unit 153 determines whether the person is above the platform. Specifically, when the value of the lower limit 312 of the person's Y coordinates 311 to 312 is a value indicating a value above the value of the lower limit 314 of the table's Y coordinates 313 to 314, It is determined that the person is above and the platform is below. A case where the value of the lower limit 312 of the person is above the value of the lower limit 314 of the platform means that the value of the lower limit 312 is greater than the value of the lower limit 314 .

Next, the determination result calculation unit 154 of the risk determination unit 123 determines results as follows based on the determination results obtained from the first determination unit 151 to the third determination unit 153 described above.

FIG. 4 shows a flowchart of the operation of the determination result calculation unit 154. As shown in FIG.

First, when the determination result calculation unit 154 starts processing (step 401), it checks whether or not both a person and a platform are detected based on the determination result of the first determination unit 151 (step 402). For example, the presence or absence of risk is set to none, and the level of risk is set to Lv0 as detection values (step 412).

Step 402: If YES, the determination result calculation unit 154 next checks whether or not the person and the table overlap in the horizontal direction based on the determination result of the second determination unit 152 (step 403). If there is, the presence or absence of risk is set to none, and the level of risk is set to Lv1 as detection values (step 413).

Step 403: If YES, next, the determination result calculation unit 154 confirms whether or not the person is above and the table is below based on the determination result of the third determination unit 153 (step 404). If so, the presence/absence of risk is set to none, and the level of risk is set to Lv1 as detection values (step 414). In this case, the determination result calculation unit 154 determines whether or not there is a degree of risk because the person and the object overlap in the horizontal direction but the person is not above the table (for example, the person is not on the table). is judged to be null.

Step 404: If YES, the determination result calculation unit 154 sets the detected value as presence of risk and Lv2 as the level of risk (step 415). In this case, the determination result calculation unit 154 determines that there is a degree of risk because the person is above the platform, for example, the person is on the platform.

Next, the determination result calculation unit 154 calculates or acquires the index value of the degree of danger in common regardless of which detection value is the above (step 406). Here, determination result calculation section 154 acquires two index values. As the first index value, the determination result calculation unit 154 acquires the percentage value of the certainty output by the DL library 132 of the person detection unit 121 at the time of determination. As the second index value, the determination result calculation unit 154 acquires the percentage value of the certainty output by the DL library 142 of the platform detection unit 122 at the time of determination. Note that the determination result calculation unit 154 calculates the average value of the certainty (percentage value) output by the person detection unit 121 and the certainty (percentage value) output by the platform detection unit 122 as an index value. good too.

Thus, the judgment result calculation unit 154 finishes the processing (step 407).

Through the above series of procedures, the dangerous work analysis unit 112 of the analysis device 101 can detect dangerous work for each still image. Since a plurality of still images are processed as input, the result of detection is obtained as a set of analysis results for each still image.

Next, the moving image creation unit 113 of the analysis device 101 performs the processing described below.

First, the frame drawing unit 161 of the moving image creating unit 113 creates a new still image by drawing a frame surrounding a still image in which either or both of a person and a table are detected.

FIG. 5 shows how a frame is drawn on the input still image shown in FIG. 2 based on the detection result. As the coordinates used for drawing the frame, the coordinate values indicated by 301 to 304 and 311 to 314 in FIG. 3 may be used as they are. Frame drawing unit 161 draws a frame for each still image extracted by still image extraction unit 111 when at least one of a person and a table is detected.

Moving image creation unit 113 next generates a new moving image based on the group of still images with the frame drawn above. Note that the original image may be used as it is for the still image in which the frame is not drawn.

However, at this time, the moving image creation unit 113 performs further processing described below so that the user can view the generated still image more conveniently.

As an example for explaining the operation of the moving image creation unit 113, a case where a video in which dangerous work is performed or not performed depending on the time period is taken as an example, and the video is analyzed. Based on the determination result, how the reproduction speed changing unit 162 of the moving image creating unit 113 operates will be described.

FIG. 6 shows a list of determination results for each time period of video. The playback speed changing unit 162 generates, for example, a list as shown in FIG. It should be noted that FIG. 6 is a chart presented for convenience in explaining the present embodiment, and it is not always necessary for the analysis apparatus 101 to store a database with such a configuration, and the intermediate results of analysis are stored in array variables and files. etc., so long as it is held appropriately.

Line 601 captures a scene in which a less dangerous operation was performed during the time period of 0 to 4 minutes from the start of the video. , we obtain the result that there is a data set A that represents two indices of the degree of risk in these time periods.

Similarly, row 602 shows that a high-risk work was performed in the time period of 4 to 5 minutes from the start of the video. is Lv2, and the result is obtained that there is a data set B that represents two index values of the degree of danger in these time periods. Lines 603 to 605 also express similar content. In the example shown in FIG. 6, the playback speed changing unit 162 generates a new line (record) when "presence or absence of risk" or "level of risk" changes.

The playback speed changing unit 162 of the moving image creating unit 113 is configured to increase the playback speed by N times (here, 4 times) for the time period determined to be "not dangerous". make edits.

Specifically, the playback speed changing unit 162 discards 3 out of 4 consecutive still images and extracts only 1 out of 4 still images in the time period of lines 601, 603, and 605. do. In addition, the playback speed changing unit 162 adopts all still images as they are in the time periods of lines 602 and 604 . Then, the playback speed changing unit 162 connects these still images and encodes the moving image to generate a new moving image.

FIG. 7 shows how a moving image with changed playback speed is generated. Symbols 601 to 605 in FIG. 7 correspond to lines 601 to 605 in FIG. 6, respectively, and schematically represent the passage of playback time.

The resultant moving image generated in this manner is reproduced at four times the speed in the time zones 601, 603, and 605 shown in FIG. 7, so the total reproduction time is shortened from 10 minutes to 4 minutes.

Further, the reproduction speed changing unit 162 of the moving image creating unit 113 updates the time information by reflecting the reduction in the reproduction time of the reproduction time of the moving image, and also in association with the detection result of the degree of danger. conduct.

FIG. 8 shows a list of determination results reflecting changes in playback speed. Reflecting the shortening of the time shown in FIG. 7, for example, in line 801, the time period is changed from "0 minutes to 4 minutes" to "0 minutes to 1 minute". Furthermore, for the data sets representing the two index values of the degree of risk in these time periods, the original data set A is converted to four times the playback speed to create a new data set A'. Specifically, processing for thinning out three out of four data is performed.

Also, for example, in line 802, the time zone is read from "4 minutes to 5 minutes" to "1 minute to 2 minutes". The original data set B can be used as it is because the reproduction speed does not change for the data sets representing the two index values of the degree of danger in these time periods. The same is true for lines 803 to 805.

Through the above-described series of procedures, analysis device 101 obtains risk level data, which is the result of analysis, and a new moving image in which frames have been changed and the playback speed of the moving image has been changed. Then, these data are transmitted to the display device 102. FIG.

Here, the effects obtained by a series of processes for changing the playback speed of moving images will be mentioned. Since the videos taken are used to confirm the state of dangerous work by manual visual inspection, if the total video time is shortened from 10 minutes to 4 minutes, the work efficiency can be improved 2.5 times by simple calculation.

In this case, the playback speed is fast-forwarded only when the person or platform is not shown in the video, or even if it is shown, it is not possible to assume that the person is on the platform and is dangerous due to the positional relationship. be. Since the playback is performed at the normal speed during the period when the person is on the platform and there may be danger, the work efficiency is improved without impairing the original purpose of checking the video by visual inspection. can improve.

Generally, in work, it takes a long time to prepare for the work, and the time to perform the core work content such as work on a table is small compared to the whole time. In other words, if it is possible to improve the efficiency of visual confirmation by fast-forwarding playback during times when there is little danger of high-place work such as preparation, it can be said that confirmation efficiency can be greatly improved without impairing the purpose of confirmation.

Conversely, when extracting a time period that you want to pay attention to from a video using AI or deep learning technology, which is a field of AI, as in this embodiment, the principle is to increase the recognition accuracy of AI to 100%. Since it is impossible to do so, misrecognition will inevitably occur. If the AI simply cuts out all the videos during the time period that the AI judges as "no need for attention" and does not display them, the time period in which the above-mentioned misrecognition occurred should be confirmed by paying attention. Although it should be, it is cut from the video, and there is a problem that it is not noticed by manual visual inspection.

On the other hand, according to the present embodiment, even if the AI misrecognizes, the reproduction itself is performed only by speeding up the reproduction speed. The advantage is that it is also possible to check for In this way, the present embodiment, which enhances the efficiency of manual viewing work by fast-forwarding playback as described above, produces a particularly remarkable effect when combined with automatic video detection by AI.

In this embodiment, an example is shown in which the playback speed is quadrupled for a time zone that does not require attention. The scope of the present invention is not limited to this, and other playback speeds may be used, or the time period may be skipped and not played back. Also, these behaviors may be changed by allowing the user to specify these behaviors when performing analysis or displaying.

In the following, the manner in which the display device 102 displays the moving image of the analysis result will be shown.

FIG. 9 shows the contents displayed on the screen of the display device 102. As shown in FIG. In the screen 171, a section (area) 901 for displaying moving images, a seek bar 902, a color bar 903, a graph A 904, and a graph B 905 are arranged. The combination of the section 901 displaying the moving image and the seek bar 902 is common in moving image players and can be realized using a library.

A moving image whose playback speed has been changed by the analysis device 101 is displayed in the moving image display area 901 . The reproduction unit 173 of the display device 102 reproduces the entire moving image acquired from the analysis device 101 at 1× speed during reproduction, and as a result, the portion of the still image in which the dangerous work is being performed is reproduced at the first speed. On the other hand, portions other than the still image in which dangerous work is being performed are reproduced at the second speed, which is faster than the first speed (see FIG. 7).

A seek bar 902 indicates the playback position of the moving image. The seek bar 902 has a slider 906. The slider moves according to the time of video playback, and the position of video playback can be jumped by moving the slider by mouse operation or the like.

The color bar 903 indicates the presence or absence of risk or the level of risk in the time zone corresponding to the seek bar 902 with different colors or different patterns. The illustrated color bar 903 indicates the level of risk in the data of risk obtained by the analysis device 101 as an analysis result. That is, the positions 801 and 805 corresponding to the time zone of Lv0 are colored gray, the position 803 corresponding to the time zone Lv1 is colored yellow, and the positions 802 and 804 corresponding to the time zone Lv2 are colored Draw red, respectively. Note that the color bar 903 may indicate the presence or absence of the degree of risk in the data of the degree of risk obtained as the analysis result by the analysis device 101 with different colors or different patterns.

Symbols 801 to 805 in FIG. 9 correspond to rows 801 to 805 in FIG. 8, respectively, and express how the risk level at each reproduction time is drawn on the color bar 903. In FIG.

A graph A 904 displays, in a line graph, the first risk index value, that is, the value of the certainty regarding human detection, among the risk data obtained by the analysis device 101 as an analysis result. A graph B 905 displays, in a line graph, the index value of the second risk level, that is, the value of the degree of certainty relating to platform detection, among the data of the risk level obtained by the analysis device 101 as the analysis result.

In this embodiment, the graph A 904 and the graph B 905 are graphs of the confidence value output by the DL libraries 132 and 142. However, if the index value is used to determine the presence or absence of risk or the level of risk, , an index value other than the confidence value may be used. In addition, the presence or absence of risk or the level of risk may be displayed as a graph.

The color bar 903, the graph A 904, and the graph B 905 are arranged side by side with the seek bar 902 vertically (up and down). For example, when the slider 906 of the seek bar 902 is at the position of 2 minutes from the start of the video, the color or value at the same X coordinate position in the color bar 903, graph A 904, and graph B 905 is the analysis result at the position of 2 minutes from the start of the video. The obtained risk information is shown.

FIG. 10 shows another screen example of the display device 102 . 10, a section 901 for displaying video, a seek bar 902, a color bar 903, various operation buttons 911 to 913, and a check box 914 are arranged. The operation button 911 is a button for receiving an instruction to stop the video being reproduced. The operation button 912 is a button for accepting an instruction to change the video playback speed. The operation button 913 is a button for receiving an instruction to move the video being reproduced. A check box 914 is provided corresponding to each color indicating the risk level of the color bar 903 . Graph A 904 and graph B 905 shown in FIG. 9 may be displayed on the screen example of FIG.

FIG. 11 shows that when the check box of a certain color is unchecked on the screen of FIG. 10, the reproduction of the moving image of the portion 915 of that color on the color bar 903 is skipped, and the slider 906 also moves accordingly.

FIG. 12 shows that when the operation button 913 for moving to the next is clicked with the check box of at least one color unchecked on the screen of FIG. It shows that the moving image at the beginning of the portion is played and the slider 906 moves accordingly.

Advantages of the display device 102 according to the present embodiment shown above will be described.

The degree of danger that changes according to the time period is displayed with a color bar, and the red time period indicates that dangerous work is being carried out, so it is necessary to watch the video. can be used as a useful indicator. At this time, since the position of the X coordinate is the same as the slider of the seek bar, by looking at the color bar while focusing on the position of the slider that moves automatically as playback progresses, you will not miss the point you should pay attention to. Easier to see. Conversely, when repeatedly checking the video, moving the slider while focusing on the red color bar makes it easy to cue the video.

Graphs A 904 and B 905 in FIG. 9 graphically display the degrees of certainty of detection by the DL libraries 132 and 142, making it easier to perform manual visual inspection while paying attention to misrecognition of images. For example, even if the color bar 903 is yellow or gray, if the value of graph A or graph B shows a relatively low index value, it can be read from the graph that the DL libraries 132 and 142 are confused about the judgment. Therefore, it can be said that misrecognition may occur due to, for example, a large number of objects appearing in the image. Therefore, when the graph indicates a relatively low index value, it is possible to obtain the advantage of being able to recognize that it is a point that should be carefully checked during manual visual inspection work.

In the present embodiment described above, person detection unit 121 and platform detection unit 122 detect a person and a platform using trained models 131 and 141 in which a person and platform are learned in advance. For example, in another embodiment, the person may wear a first color or pattern and the platform may have a second color or pattern. In this case, the person detection unit 121 detects the person by detecting the first color or the first pattern, and the table detection unit 122 detects the table by detecting the second color or the second pattern. To detect.

Specifically, for example, a person may be dressed in red clothes or wearing a red patch or the like, and the person detection unit 121 may detect the person by detecting red in the image. Alternatively, the table may be painted blue or attached with a blue plate, and the table detection unit 122 may detect the blue color in the image to detect the table. Note that only one of the person detection unit 121 and the platform detection unit 121 may perform such detection.

In addition, in the present embodiment described above, when the third determination unit 153 determines that the person is above and the platform is below, the Y coordinate of the pixel is used for the determination. In general, when a person is in a position higher than the floor by a certain level, it can be said that the danger is always high. Recognizing what is above is of high practical value.

For example, in another embodiment, the height of the person from the floor is calculated using the distance between the person and the video camera 105 that captures the moving image and the position of the person in the still image. It can be said that it is an estimation. That is, assuming that the distance between the video camera 105 and the server rack to be worked on is known, the height of the person from the floor is calculated based on the Y coordinates 311 to 312 of the person in the still image, and the height exceeds, for example, 1.2 m, it can be recognized that "a person is above".

FIG. 13 shows an example of calculating the height of a person from the floor using a ratio. This figure shows how a video camera 1002 captures a person 1003 at a position higher than a floor surface 1001 as seen from the lateral direction. It is assumed that the video camera 1002 is installed at a height 1010 from the floor, which is 1 m here. The video camera 1002 used here is assumed to shoot between an upward shootable range 1012 and a downward shootable range 1013, and the angle of view, that is, the angle formed by the vertically shootable range is The range of angles is 1014, here 30 degrees.

In view of the purpose of judging the danger of a person working above the server rack, the person 1003 is photographed at a distance 1011 from the video camera 1002 to the work place to be photographed, that is, the server rack. , where the distance 1011 is 3.46 m.

Assuming that all the objects to be photographed are present on the photographing plane 1005 at a distance of 1011, the upper half of the image photographed by the video camera 1002 extends from the center 1020 of the photographing range to the upper end of the photographing range on the photographing plane 1005. An object existing in the range up to 1021 is reflected. Assuming that the vertical resolution of the video camera 1002 is 1000 dots, the range from the center 1020 of the shooting range to the upper end 1021 of the shooting range is reflected in the upper half of the shot image with 500 dots.

Assume that under the above conditions, the lower end 1023 of the person 1003 appears in the captured image as the 400th dot from the top of the captured image, for example. At this time, the distance 1024 from the center 1020 of the shooting range to the bottom edge 1023 of the person can be calculated as follows.

distance 1022 = distance 1011 × tan (angle 1014)
= 2m
Distance 1022 : 500 dots = Distance 1024 : (500-400) dots ∴ Distance 1024 = Distance 1022 ÷ 500 × 100
= 0.4m
Therefore, it can be calculated from the image that the person 1003 is at a height of 1.4 m above the floor. Since this satisfies the condition of a height of 1.2 m or more, which is the aforementioned standard, it is possible to determine that the person is above.

Even if the distance to the server rack is not known, it is possible to detect an object whose width is known to be constant, such as a server rack, and measure the distance to the object from that width, or use a laser rangefinder. (first laser rangefinder) can be used to measure the distance to a work target, or parallax can be used with a stereo camera to measure the distance to a work target. can be used to calculate This laser rangefinder illuminates laterally from the position of the video camera toward the work object. A laser rangefinder, also called LiDAR, is a device that measures the distance to a target object by laser irradiation. Such devices are commercially available and readily available.

Further, for example, in another embodiment, the operator holds an altimeter, the data obtained from the altimeter is used as an additional input to the analysis device 101, and the third determination unit 153 determines the obtained data. Based on this, the height of the person from the floor is obtained, and when the height exceeds, for example, 1 m, it can be recognized that "the person is above". An altimeter may be a device that measures altitude differences using changes in air pressure. Altitude is measured by the difference between the air pressure when it is on the floor and the air pressure when it reaches a high altitude. Such devices are commercially available and readily available.

Further, for example, in another embodiment, the operator holds the laser rangefinder (second laser rangefinder) vertically downward. Since the laser range finder measures the distance from the floor vertically or the distance to the work object, etc., the data obtained from the laser range finder is used as an additional input to the analysis device 101, and the third determination unit 153 can obtain the height of the person from the floor based on the obtained data, and can recognize that "the person is above" when the height exceeds, for example, 1 m.

Further, the judgment by the third judgment unit 153 may be made by combining the judgment using the Y coordinate of the pixel, the judgment shown in FIG. 13, the judgment using the altimeter, and the judgment using the laser rangefinder. may be

In the embodiments of the present invention described above, various configurations are possible without departing from the gist of the present invention.

For example, the analysis device 101 and the display device 102 may be the same device, or may be further divided into a plurality of devices. Data transfer from the analysis device 101 to the display device 102 may be performed via a medium such as a CD or memory card without using the network 103 . The display device 102 may be divided into a web server and a web browser, and may be a viewer created using HTML5. In that case, the web server may be arranged in the same device as the web browser, or may be arranged in the same device as the analysis device 101, or may be arranged in another device.

For example, the person detection unit 121 and the table detection unit 122 are made into one functional unit, and a trained model that memorizes both the person and the table at once is prepared in advance, and a single DL library is used for judgment collectively. may be configured.

For example, moving image reproduction may be performed by reproducing a moving image file, or by continuously reproducing a series of still images. Also, in moving image reproduction, the flow of time may be expressed by horizontally arranging frames like a movie film.

For example, the playback speed changing unit 162 may not only fast-forward the non-dangerous time zone, but also slow-play the dangerous time zone. Alternatively, the reproduction speed changing unit 162 may change the reproduction speed in multiple steps according to the level of danger. Alternatively, the playback speed changing unit 162 may change the playback speed continuously at the joint so that the joint where the playback speed changes can be visually recognized comfortably.

In order to change the playback speed, the playback speed changing unit 162 of the analysis device 101 does not generate a moving image whose playback speed has been changed. The display device 102 may change the playback speed at the time of playback of the moving image based on the instruction information using the playback unit 173 capable of varying the playback speed. As the instruction information, the presence or absence or level of risk or an index value may be used. In this case, the playback unit 173 acquires from the analysis device 101 a video whose playback speed has not been changed, and during playback, plays a video in the first time zone with a risk level or a relatively high risk level. , reproduces at a first speed, and reproduces a moving image in a second time period other than the first time period at a second speed higher than the first speed, or skips without reproducing it. Also, as shown in FIGS. 10 to 12, the playback speed may be changed according to the instruction of the user viewing the moving image.

For example, the color bars and graphs may not include any of them, or may include a plurality of them, or a plurality of pieces of information may be superimposed and displayed on one display section. In particular, two or more line graphs may be collectively displayed as one chart. The graph may be not only a line graph but also a bar graph or other display forms. The color arrangement of the color bar may be arbitrary, or may be hatching, pattern, blinking, or the like.

The seek bar, color bar, and graph need not have the video playback start time on the left end, nor the video playback end time on the right end, and any time may be used. For example, when the length of a moving image is 240 minutes, the time represented by one dot on the screen becomes 1/24 of the time when the moving image is 10 minutes long. As a result, the density of color bars, graphs, and the like becomes too high in the horizontal direction, resulting in a problem of difficulty in visual recognition.

Therefore, in the case of a long moving image, the seek bar, color bar and graph may be automatically zoomed. For example, when playing back 93 minutes from the start of the video, the display device 102 automatically adjusts the left end to the 90 minute position and the right end to the 100 minute position, so that the color bar and graph are always displayed. Since it is possible to secure a sufficient width for the image, it is possible to obtain the effect that it is easy to visually recognize regardless of the length of the moving image.

In the present embodiment described above, it is possible to detect work that is presumed to be dangerous from a moving image of the worker's work, thereby reducing the user's work load. In addition, in the present embodiment, it is possible to easily pay attention to and view a moving image during a period when danger is estimated.

The apparatus of this embodiment can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network. Specifically, the analysis device 101 and the display device 102 described above include, for example, a CPU (Central Processing Unit, processor), memory, storage (HDD: Hard Disk Drive, SSD: Solid State Drive), communication A general-purpose computer system with devices, input devices, and output devices can be used. In this computer system, each function of each device is realized by the CPU executing a predetermined program loaded on the memory. For example, each function of the analysis device 101 and the display device 102 is executed by the CPU of the analysis device 101 in the case of the program for the analysis device 101, and by the CPU of the display device 102 in the case of the program for the display device 102. It is realized by Also, the program for the analysis device 101 and the program for the display device 102 can be stored in a computer-readable recording medium such as HDD, SSD, USB memory, CD-ROM, DVD-ROM, MO, etc., or can be stored via a network. can also be delivered.

101: Analyzer
102: Display device
103: Network
105: Video camera
111: Still image extractor
112: Dangerous Work Analysis Department
113: Video creation department
121 : Human detector
122 : Platform detector
123: Risk judgment unit
131, 141 : Trained model
132, 142: LD library
151: 1st judgment part
152: Second judgment unit
153: 3rd judgment unit
154: Judgment result calculator
161: Frame drawing part
162 : Playback speed change part
171: screen
172: User input means
173: Playback

Claims (10)

A dangerous work detection system having an analysis device and a display device,
The analysis device is
Equipped with a dangerous work analysis unit that analyzes multiple still images extracted from videos of work and detects still images of dangerous work being performed,
The display device
reproducing a portion of the still image in which the dangerous work is being performed at a first speed, and reproducing portions other than the still image in which the dangerous work is being performed at a second speed, which is faster than the first speed; or skip without playing,
The dangerous work analysis unit
both a person and an object are detected in the plurality of still images;
the person and the object overlap in the horizontal direction, or the person and the object are in the vicinity;
When the person is above the object,
Determining whether or not there is a degree of risk
A dangerous work detection system characterized by: An analysis device for detecting dangerous work,
a person detection unit that analyzes a plurality of still images extracted from a video of a work and detects a person from each still image;
an object detection unit that analyzes the plurality of still images and detects from each still image an object that may cause dangerous work;
a risk determination unit that determines a risk level of the plurality of still images using the detection result of the person detection unit and the detection result of the object detection unit;
a video creation unit that creates a video by editing only the portion of the still image with no risk so that the playback speed is N times (N>1) ,
The risk determination unit
both the person and the object are detected in the plurality of still images;
the person and the object overlap in the horizontal direction, or the person and the object are in the vicinity;
When the person is above the object,
Determining whether or not there is a degree of risk
An analysis device characterized by The analysis device according to claim 2,
At least one of the person detection unit and the object detection unit detects at least one of the person and the object using a trained model generated by deep learning technology. . The analysis device according to claim 2 or 3,
said person wearing a first color or first pattern,
The analysis device, wherein the person detection unit detects the person by detecting the first color or the first pattern. The analysis device according to any one of claims 2 to 4,
A second color or a second pattern is set on the object,
The analysis apparatus, wherein the object detection unit detects the object by detecting the second color or the second pattern. The analysis device according to any one of claims 2 to 5 ,
The risk determination unit estimates the height of the person from the floor using the distance between the person and the camera that captures the moving image and the position where the person appears in the still image. An analysis device characterized by: The analysis device according to any one of claims 2 to 6 ,
The analysis device , wherein the risk determination unit estimates the height of the person from the floor using measurement data of an altimeter worn by the person. The analysis device according to any one of claims 2 to 7 ,
The analysis apparatus , wherein the risk determination unit estimates the height of the person from the floor using measurement data of a laser rangefinder worn by the person facing downward. A dangerous work detection method performed by a dangerous work detection system having an analysis device and a display device,
The analysis device is
performing a detection step of analyzing a plurality of still images extracted from a moving image of work and detecting a still image of dangerous work being performed;
The detection step includes
both a person and an object are detected in the plurality of still images;
the person and the object overlap in the horizontal direction, or the person and the object are in the vicinity;
When the person is above the object,
Judging whether or not there is a degree of danger,
The display device
reproducing a portion of the still image in which the dangerous work is being performed at a first speed, and reproducing portions other than the still image in which the dangerous work is being performed at a second speed, which is faster than the first speed; Alternatively, a dangerous work detection method characterized by performing a regeneration step of skipping without regeneration. A dangerous work detection program for causing a computer to function as the analysis device according to any one of claims 2 to 8.

Images